diff --git a/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/CONCEPT.md b/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/CONCEPT.md index ccce517..36faec0 100644 --- a/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/CONCEPT.md +++ b/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/CONCEPT.md @@ -1,6 +1,6 @@ # AI 费用闭环与价值证明 概念文档 -更新时间：2026-07-16 +更新时间：2026-07-17 文档路径：document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/CONCEPT.md @@ -619,3 +619,10 @@ docker exec -w /app -e SERVER_VENV_DIR=/tmp/x-financial-server-venv \ - 2026-07-16（持久任务与并发）：新增 migration-owned `attachment_association_jobs` 和 `20260716_0006`。任务状态、结构化结果、owner 上下文、attempt、租约与 generation 写入数据库；GET 可恢复 queued 或租约过期任务，`attempt_count + running` 作为栅栏阻止旧 worker 或迟到回调覆盖新终态。同票据和同 Claim 分别使用进程锁与 PostgreSQL advisory lock 串行化，Claim 锁内清理旧事务并重新匹配，避免不同票据并发选择同一空明细。待确认或失败任务保留原代历史，再次发起创建新 generation 并重新评估；已自动关联成功的代际继续幂等复用。评分改为纯只读查询，每份票据必须独立达到最小证据，避免无关票据被同批强证据带入。 - 2026-07-16（小财管家交互与验证）：任务结果新增 Case、申请、置信度、原因、异常、缺失项、风险项、候选和草稿载荷；前端可跨会话恢复，自动完成直接查看草稿，仅申请候选查看申请，待确认不伪装成功；幂等重放显示“已关联”，成功结果仍展示风险和复核要求。容器内后端归集专项 20 项、归集与相邻服务/迁移所有权组合回归 71 项、前端关联链路组合回归 29 项、Ruff F/I/UP 和 Vite 生产构建通过；一次性 tmpfs PostgreSQL 17 的 0006 完整迁移循环 4 项通过并已清理，持久开发库未修改。既有大型报销服务与接口套件仍存在旧审批、删除和风控断言失败，未把这些基线问题误报为已解决。 - 2026-07-16（保留边界）：预算、项目、成本中心和个人记忆偏好尚未接入票据候选评分；任务已持久化并支持租约恢复，但尚未建设独立消息队列、运维重试面板和死信治理；完整 G2 与平台级异步任务治理仍未完成。 +- 2026-07-17（费用价值闭环）：新增 Savings Ledger、CFO 价值分析和财务确认/冲回链路；现金、工时、风险暴露和预计机会严格分账。住宿标准调整从服务端锁定原金额创建机会，付款/生产连接器回执产生 actual realization，独立财务确认后才进入现金 KPI，退款以负向追加事实冲回。 +- 2026-07-17（财务连接器）：建立配置生命周期、HMAC、防重放、事件幂等、对账、ERP 入账、退款/冲回、operational event 和 mock/test/staging 隔离。端到端以本地自签的 production-mode 事件验证申请、票据、报销、预审、审批、回执、ERP、归档、Savings 与商业价值契约；它不代表真实银行/ERP 回执，模拟回执也不会改变核心财务事实。 +- 2026-07-17（真实发布遥测）：规则发布从真实 observation、可信 disposition/reviewer label、独立盲审负样本和保守 recall 进入 Monitor/Guard；collecting、积压、聚合失败或低 precision 均不晋级并保持 stable。0023 提供 audit sample、双人票、append-only 和 PostgreSQL 并发保护。 +- 2026-07-17（商业闭环）：新增套餐、订阅、账期、权益、配额、用量、成本、ROI、毛利和定价走廊；中央 Orchestrator、Runtime Chat、OCR、金融连接器和附件源文件写入按权威数量预占、结算或释放，客户价值、平台收入和内部成本不混账。 +- 2026-07-17（全链租户安全）：新增 0025-0028，统一 Tenant、Employee、Claim、Agent Asset、Knowledge、Ontology、Hermes、Finance Report、Qdrant、文件和缓存作用域；ONLYOFFICE 改为数据库一次性会话与 SSRF/重放保护。审批员工解析按认证企业或结构化 Claim 企业首层过滤，跨租户身份和相同名称不再命中。 +- 2026-07-17（工程收口验证）：176 个后端测试文件在主应用容器内分片或专项通过，费用主服务 121 项通过；fresh PostgreSQL 迁移/并发总探针 `87 passed / 0 skipped / 0 failed`，head 为 0028；Web 全量 `815 passed / 0 failed` 与 Vite build 通过；Mobile lint/typecheck、197 个新增 Python 文件 Ruff、目标 compileall、受门禁核心类/组件 800 行检查和 `git diff --check` 通过。 +- 2026-07-17（完成边界）：工程代码和容器验收已收口；生产 ONLYOFFICE DNS/TLS、备份副本迁移演练、逐租户 SMTP、真实 provider/会计规则、移动/浏览器实机联调、30/90 天企业试点、客户财务签字和合同定价仍必须使用目标环境与真实业务证据完成，不能由 mock 或本地测试替代。总验收见 `document/development/2026-07-17/feature/engineering-closure-and-production-readiness/`。 diff --git a/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/TODO.md b/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/TODO.md index fe74daa..3cee3d5 100644 --- a/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/TODO.md +++ b/document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/TODO.md @@ -1,6 +1,6 @@ # AI 费用闭环与价值证明 开发 TODO -更新时间：2026-07-16 +更新时间：2026-07-17 文档路径：document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/TODO.md @@ -34,21 +34,28 @@ ## 3. 契约与设计 -- [ ] [CONCEPT: 费用领域与编排] 定义 `Expense Case`、申请、票据、报销、审批、付款、凭证和归档的领域边界与迁移关系。 -- [ ] [CONCEPT: 数据与契约] 定义 `expense_cases`、`expense_case_links` 和最小状态机，明确非法状态跃迁。 -- [ ] [CONCEPT: 数据与契约] 定义 `business_events` 事件信封、事件词典、correlation ID、幂等键和版本策略。 +- [x] [CONCEPT: 费用领域与编排] 定义 `Expense Case`、申请、票据、报销、审批、付款、凭证和归档的领域边界与迁移关系。 + 证据：`expense_cases.py`、Expense Case/Link/Event 模型、连接器付款/ERP/归档与 Savings 冲回 E2E。 +- [x] [CONCEPT: 数据与契约] 定义 `expense_cases`、`expense_case_links` 和最小状态机，明确非法状态跃迁。 + 证据：`20260713_0001_expense_case_business_events.py`、`ExpenseCaseService` 与状态/事件回归。 +- [x] [CONCEPT: 数据与契约] 定义 `business_events` 事件信封、事件词典、correlation ID、幂等键和版本策略。 + 证据：`BusinessEvent` schema/service、业务事件词典及申请/报销/审批/支付/ERP/Savings 回归。 - [x] [CONCEPT: 业务事件与 AI 决策] 定义 `ai_decisions`、`ai_decision_feedback` 和 `workflow_outcomes` 契约。 证据：`ai_learning.py`、`expense_application_learning.py`、`20260714_0003_ai_learning_loop.py`；三类事实分别表达 AI 建议、用户采纳/显式字段纠正和业务结果，技术执行成功、用户反馈与工作流结果不复用同一状态。 - [x] [CONCEPT: 业务事件与 AI 决策] 定义落单前服务端预览决策的签发、授权、过期、一次消费、重放和续签契约。 证据：`ai_application_preview.py`、`expense_application_preview_decisions.py`、`expense_application_snapshot.py`；预览不创建空 Case，绑定租户、actor、登录会话与 conversation，30 分钟过期，保存/提交成功后一次消费；保存草稿在业务提交后尽力返回基于服务端事实的新 `decision_id`，续签失败不反转已成功动作。 - [x] [CONCEPT: 记忆与学习] 定义并实现 `memory_entries`、`memory_evidence_links`、优先级、有效期、敏感白名单、撤销和遗忘契约；首个切片仅覆盖 `travel_application.transport_mode`。 证据：`ai_memory.py`、`expense_application_memory.py`、`expense_application_memory_evidence.py`、`20260714_0005_ai_memory.py`；记忆使用租户、主体、场景、字段和值指纹精确隔离，值只允许“飞机/火车/轮船”，服务异常降级为不应用记忆。 -- [ ] [CONCEPT: 自动化决策] 定义动作风险、金额阈值、置信度、证据完整度、可逆性、抽检率和企业授权策略。 -- [ ] [CONCEPT: 节省与价值] 定义 Savings Ledger 的机会、执行、实现、确认、去重和归因状态。 +- [x] [CONCEPT: 自动化决策] 定义动作风险、金额阈值、置信度、证据完整度、可逆性、抽检率和企业授权策略。 + 证据：`automation_eligibility.py` 与 `test_automation_eligibility.py`；资金、制度和高风险动作始终人控。 +- [x] [CONCEPT: 节省与价值] 定义 Savings Ledger 的机会、执行、实现、确认、去重和归因状态。 + 证据：Savings 模型、0015 迁移、状态服务与并发/E2E 测试。 - [ ] [CONCEPT: 连接器] 定义票据邮箱、税务、企业卡、商旅、支付、银行、ERP、消息和 SSO 连接器协议。 -- [ ] [CONCEPT: 权限与安全] 定义服务端会话、租户数据范围、角色和动作级授权契约。 +- [x] [CONCEPT: 权限与安全] 定义服务端会话、租户数据范围、角色和动作级授权契约。 + 证据：不透明 Bearer Session、`CurrentUserContext`、租户访问策略和 endpoint role dependencies；0025-0028 多租户安全迁移。 - [ ] [CONCEPT: 指标与验收] 完成首个试点指标字典、基线采集方案、分子分母、数据源和负责人。 -- [ ] [CONCEPT: 方案设计] 完成分阶段架构评审，确认新增 service 不继续堆入 `ExpenseClaimService`、`UserAgentService` 或大型前端 composable。 +- [x] [CONCEPT: 方案设计] 完成分阶段架构评审，确认新增 service 不继续堆入 `ExpenseClaimService`、`UserAgentService` 或大型前端 composable。 + 证据：访问策略、员工解析、附件、预审、Savings、商业、连接器、发布遥测及前端 composable 均按职责拆分；受门禁核心类/组件 800 行检查通过。 ## 4. P0 后端实现：费用闭环与数据基础 @@ -56,12 +63,18 @@ 证据：`auth_sessions.py`、`auth_session.py`、`deps.py`、`auth.py`、`authSessionStorage.js`；登录签发不透明 Bearer token，数据库仅保存 SHA-256 摘要，生产代码不再信任 `X-Auth-*`，过期/撤销/伪造会话回归测试通过。 - [x] [CONCEPT: 权限与安全] 为管理面、Bootstrap、Settings、模型连通性、缓存、审计和系统日志补齐平台管理员保护。 证据：`bootstrap.py`、`settings.py`、`audit_logs.py`、`agent_traces.py`、`system_logs.py`、`vite.config.js`；已初始化 Bootstrap 脱敏且拒绝匿名重配，平台管理员/业务经理权限边界和 Vite Setup 锁测试通过。 -- [ ] [CONCEPT: 权限与安全] 继续盘点并收口风险规则发布、制度发布及其他尚未纳入本轮的敏感动作，按动作定义平台管理员或双人复核权限。 -- [ ] [CONCEPT: 权限与安全] 为所有新增表和共享核心数据补齐最小 `tenant_id`、数据库约束、查询守卫及默认租户迁移。 +- [x] [CONCEPT: 权限与安全] 继续盘点并收口风险规则发布、制度发布及其他尚未纳入本轮的敏感动作，按动作定义平台管理员或双人复核权限。 + 证据：Agent Asset/Rule Editor/Reviewer、发布盲审双人票、风险豁免独立决定、平台资产只读和 ONLYOFFICE 写入权限均有服务端守卫与安全测试。 +- [x] [CONCEPT: 权限与安全] 为本轮新增表及 Claim、Employee、Agent Asset、Knowledge、Ontology、Hermes 和 Report 等纳入范围的共享核心数据补齐 `tenant_id`、约束、查询守卫和默认租户迁移。 + 证据：0025-0028 迁移、复合租户外键、首层 SQL 过滤和 tenant security 测试；fresh PostgreSQL 迁移总探针通过。 +- [ ] [CONCEPT: 权限与安全] 将 `agent_conversations` 等仍依赖 JSON tenant 的 legacy 状态迁移到结构化租户列和数据库约束。 + 证据要求：后继迁移、历史回填、复合约束和跨租户回归；当前只能由服务层校验 `state_json.tenant_id`，不得表述为数据库边界已完成。 - [x] [CONCEPT: 权限与安全] 为 Expense Case 查询提供面向用户的精简事件 DTO，移除幂等键、correlation、causation 和 Outbox 投递字段，并明确审批意见、退回原因和操作人可见范围。 证据：`expense_case.py`、`test_expense_case_endpoints.py`；用户态响应只保留安全流程摘要，关联资源 ID 和未知嵌套 payload 被递归过滤，本人、当前审批人、财务、管理员、无权限及跨租户边界测试通过。 -- [ ] [CONCEPT: 权限与安全] 为 Qdrant collection/namespace、对象存储前缀和缓存键补齐租户隔离回归测试。 -- [ ] [CONCEPT: 数据与契约] 建立 Alembic baseline 和正式迁移链，停止请求路径运行 DDL。 +- [x] [CONCEPT: 权限与安全] 为 Qdrant collection/namespace、对象存储前缀和缓存键补齐租户隔离回归测试。 + 证据：Knowledge tenant scope、RAG workspace、Qdrant namespace、票据/附件路径、运行缓存和财务快照 tenant fingerprint 回归。 +- [x] [CONCEPT: 数据与契约] 建立 Alembic baseline 和正式迁移链，停止请求路径运行 DDL。 + 证据：migration ownership/preflight 与 0001-0028 正式链；fresh PostgreSQL 完整 upgrade/downgrade/re-upgrade 通过，请求路径不再创建 migration-owned 表。 - [x] [CONCEPT: 兼容策略] 集中 migration-owned 表所有权并在标准启动迁移前执行只读漂移预检，禁止 legacy bootstrap 越权建表。 证据：`schema_ownership.py`、`migration_preflight.py`、`server_start.sh`、`test_migration_preflight.py`、`test_schema_ownership.py`；无版本自有表、缺表、多表、未知/多 revision 均 fail-fast，不自动 stamp 或修改数据库。 - [x] [CONCEPT: 费用领域与编排] 新增 `ExpenseCaseService` 和费用事件查询接口，保持编排与具体职责分离。 @@ -99,7 +112,8 @@ - [ ] [CONCEPT: 兼容策略] 制定旧 `ReimbursementRequest` 只读兼容、迁移和停止新增编排的计划。 - [ ] [CONCEPT: 兼容策略] 把新增审批、付款、归档和关系事件移出 `risk_flags_json`，保留旧数据读取兼容。 - [ ] [CONCEPT: 数据与契约] 补齐撤回、取消、驳回、作废、补件、支付失败、对账异常和归档状态。 -- [ ] [CONCEPT: 连接器] 实现统一连接器基类、幂等、重试、错误状态和回执事件。 +- [x] [CONCEPT: 连接器] 实现财务连接器统一事件信封、认证、幂等、重试、错误状态和回执事件。 + 证据：`financial_connector_auth.py`、`financial_connector_ingestion.py`、`payment_reconciliation.py`、配置生命周期及 operational events；服务/API/并发测试通过。 - [ ] [CONCEPT: 连接器] 实现支付批次、回执、重复付款防护、ERP 凭证和对账的内部契约，首期允许 mock connector 但不得再只写单一“已付款”状态。 - [x] [CONCEPT: 降级策略] 将附件关联和关联报销草稿后台任务迁为可持久化、可恢复、可幂等的任务状态。 证据：新增 migration-owned `attachment_association_jobs` 与 `20260716_0006`；任务使用租户、owner、票据集合和 generation 去重，运行态带租约与 `attempt_count + running` 栅栏，GET 可恢复 queued/租约过期任务。同票据和同 Claim 均由进程锁与 PostgreSQL advisory lock 串行化，Claim 锁内重新匹配；待确认或失败历史保留原代并以新 generation 重新评估，自动关联成功代继续幂等复用。进程状态清空、租约过期、旧 worker 回写、不同票据并发同 Claim 和代际重评估回归通过。 @@ -137,9 +151,12 @@ ## 6. P1 算法与规则实现：安全自动化与持续学习 -- [ ] [CONCEPT: 自动化决策] 实现 L0-L5 动作级自动化等级和资格计算器。 -- [ ] [CONCEPT: 自动化决策] 为每个动作实现硬白名单、金额上限、证据要求、抽检率和企业上限。 -- [ ] [CONCEPT: 风险与预审] 统一风险输出为事实、规则、证据、判断、建议动作和降级原因。 +- [x] [CONCEPT: 自动化决策] 实现 L0-L5 动作级自动化等级和资格计算器。 + 证据：`AutomationEligibilityCalculator` 将 L0-L4 资格与 L5 人控分开计算，不直接执行动作；单元测试覆盖保守降级。 +- [x] [CONCEPT: 自动化决策] 为每个可自动动作实现硬白名单、金额上限、证据要求、抽检率和企业上限。 + 证据：系统硬白名单与企业白名单取交集；资金、制度和高风险动作永远人控，金额、置信度、证据、历史精度、样本和抽检任一不足即降级。 +- [x] [CONCEPT: 风险与预审] 统一风险输出为事实、规则、证据、判断、建议动作和降级原因。 + 证据：结构化 pre-review findings、平台风险投影、整改动作、historical evidence 和 fail-closed 原因已用于申请/报销提交与审批风险卡。 - [x] [CONCEPT: 记忆激活] 为首个个人出行方式切片实现 candidate/active/suppressed/expired/revoked 记忆状态机。 证据：Candidate/Active 默认有效期分别为 90/180 天；反向或非白名单纠正抑制已激活记忆，过期后新证据创建新 generation，忘记后清值并阻止旧请求复活。 - [x] [CONCEPT: 记忆激活] 实现用户、部门、企业记忆优先级、冲突解释、时间衰减和最小样本要求。 @@ -160,9 +177,14 @@ - [ ] [CONCEPT: 记忆与学习] 从字段接受/修改/拒绝、退回、审批覆盖、付款和审计结果生成记忆证据。 - [x] [CONCEPT: 记忆与学习] 将已确认 few-shot 扩展到报销预审和审批辅助，并按租户、场景、制度版本过滤。 证据：`few_shot_ingestion.py`、`few_shot_retrieval.py`、`expense_claim_historical_evidence.py`、`expense_claim_pre_review.py`；样本关系库和 Qdrant 同时绑定租户、场景、制度与规则版本，检索命中后再由关系库校验，旧版本标记 stale，依赖异常返回空证据。公开 `historical_case_evidence` 与 AI 助手只显示“历史已确认/历史误报，仅供复核”的固定脱敏摘要，不暴露 sample ID、单号、人工评论或结论原文，也不改变确定性结论、阻断数量、预算复核和审批路由。 -- [ ] [CONCEPT: 风险与预审] 完成 golden case、Prompt/规则版本、Canary、回归门禁和自动回滚。 -- [ ] [CONCEPT: 自动化决策] 先上线 shadow，再按动作逐项开放 L3；L4 必须单独评审。 -- [ ] [CONCEPT: 降级策略] 对模型、OCR、Qdrant、连接器和记忆服务实现稳定降级和可观测状态。 +- [x] [CONCEPT: 风险与预审] 完成 golden case、Prompt/规则版本、Canary、回归门禁和自动回滚的工程闭环。 + 证据：Golden evaluator、版本化资产、真实 observation/label、盲审负样本、Release Monitor/Guard、Canary 与 stable 回滚组合测试通过；生产效果仍由试点章节单独验收。 +- [x] [CONCEPT: 自动化决策] 实现默认 shadow、按动作开放 L3、L4 单独受控的代码策略。 + 证据：Automation Policy 的 `release_stage`、企业上限、硬白名单和 L4 资格检查；资金/制度/高风险动作不进入自动执行。 +- [ ] [CONCEPT: 自动化决策] 在真实企业按周运行 shadow 并依据签字阈值逐项开放 L3/L4。 + 证据要求：生产样本、抽检、回滚演练和企业授权；本地代码测试不得替代。 +- [x] [CONCEPT: 降级策略] 对模型、OCR、Qdrant、连接器和记忆服务实现稳定降级和可观测状态。 + 证据：Runtime Chat attempt、OCR worker、Knowledge RAG、连接器 operational events/health、记忆 fail-closed 与发布告警均有明确失败状态和回归。 ## 7. P1 前端实现：审批例外与 AI 记忆 @@ -192,23 +214,38 @@ ## 8. P2 实现：费用经营与价值证明 -- [ ] [CONCEPT: 节省与价值] 新增 `savings_opportunities`、`savings_realizations` 和去重/确认表结构。 -- [ ] [CONCEPT: 节省与价值] 实现风险暴露、预计节省、执行中、实际节省和财务确认的严格状态转换。 -- [ ] [CONCEPT: 费用分析与节省] 持久化员工、部门、费用类型、供应商、城市、项目和流程基线，记录窗口和样本量。 -- [ ] [CONCEPT: 费用分析与节省] 实现预算预测、异常归因、供应商价格漂移、重复小额浪费和政策模拟。 -- [ ] [CONCEPT: CFO 价值看板] 展示现金节省、工时价值、直通率、风险护栏、节省来源和责任人。 -- [ ] [CONCEPT: CFO 价值看板] 实现部门、项目、费用类型、供应商、城市、时间和单据下钻。 -- [ ] [CONCEPT: CFO 价值看板] 每项节省支持查看基线、建议、执行、实际结果、确认人和证据。 +- [x] [CONCEPT: 节省与价值] 新增 Savings baseline、opportunity、realization、evidence、event 和去重/确认表结构。 + 证据：`models/savings.py`、`20260716_0015_savings_value_ledger.py` 与模型/迁移测试。 +- [x] [CONCEPT: 节省与价值] 实现预计机会、接受、执行中、实际结果、财务确认、拒绝、过期和冲回的严格状态转换。 + 证据：Savings actions/realization 服务、版本/指纹、canonical benefit 去重与 PostgreSQL 并发测试；风险暴露保持独立护栏，不混入节省金额。 +- [x] [CONCEPT: 费用分析与节省] 持久化员工、部门、费用类型、城市、项目和流程基线，记录窗口和样本量。 + 证据：`savings_baseline_generation.py` 与 baseline/insight 测试；维度、窗口、样本、算法版本、查询指纹和质量等级均冻结。 +- [ ] [CONCEPT: 费用分析与节省] 接入租户化供应商、合同价、数量和单位价格事实后生成供应商基线。 + 证据要求：真实供应商主数据和合同/采购事实；当前明确返回 `supplier_dimension_unavailable`，不从模拟应付数据伪造。 +- [x] [CONCEPT: 费用分析与节省] 实现预算预测、异常归因、重复小额浪费和只读政策模拟准备项。 + 证据：Savings insight budget/analysis/attribution；缺正式反事实时 `estimated_savings=None` 且不创建机会。 +- [ ] [CONCEPT: 费用分析与节省] 接入真实供应商价格漂移分析。 + 证据要求：租户化合同价、数量、单位价和供应商证据；当前保持 coverage gap。 +- [x] [CONCEPT: CFO 价值看板] 展示现金节省、工时价值、直通率、风险护栏、节省来源和责任人。 + 证据：CFO API/analytics/dashboard；现金、工时、风险和机会分卡，缺数据使用 collecting/unavailable。 +- [x] [CONCEPT: CFO 价值看板] 实现部门、项目、费用类型、供应商、城市、时间和单据下钻。 + 证据：CFO filters、`cfoValueSourceLinks.js`、URL 状态恢复与前端回归；供应商无事实时显示缺口而非零。 +- [x] [CONCEPT: CFO 价值看板] 每项节省支持查看基线、建议、执行、实际结果、确认人和证据。 + 证据：`CfoValueOpportunityDrawer.vue` 与 Savings detail projection；无证据时不开放实际结果登记。 - [ ] [CONCEPT: 指标与验收] 生成客户月度 ROI 报告，现金节省与工时价值分开披露。 - [ ] [CONCEPT: 风险与开放问题] 建立节省归因复核、重复收益去重和客户财务签字流程。 ## 9. P3 实现：商业化与规模复制 -- [ ] [CONCEPT: 商业计量] 在 P0 最小租户隔离基础上新增套餐、配额、用量和增值模块授权模型。 -- [ ] [CONCEPT: 商业计量] 按客户、模型、OCR、文档、分析任务和模块记录成本。 -- [ ] [CONCEPT: 商业计量] 建立客户贡献毛利、实施成本摊销和私有部署成本看板。 +- [x] [CONCEPT: 商业计量] 在 P0 最小租户隔离基础上新增套餐、订阅、账期、配额、用量和增值模块授权模型。 + 证据：商业模型、0016/0019/0021/0024 迁移、管理/查询 API 与商业工作台。 +- [x] [CONCEPT: 商业计量] 按客户和权威 meter 记录模型、OCR、附件存储、连接器、分析任务与模块用量/成本。 + 证据：Orchestrator、Runtime Chat、OCR、附件、连接器 permit/预占/结算；已识别运行入口资源组合 63 项通过。 +- [x] [CONCEPT: 商业计量] 建立客户 ROI、平台贡献毛利、成本明细和私有部署成本输入模型。 + 证据：`commercial_analytics.py`、商业价值/成本前端；客户价值、平台收入和内部成本分账，多币种不合并。 - [ ] [CONCEPT: 目标与非目标] 定义年度基础订阅、用量超额、智能风控、预算经营、价值洞察、企业集成和私有部署包。 -- [ ] [CONCEPT: 目标与非目标] 仅对财务确认的已实现节省提供可选节省分成合同。 +- [x] [CONCEPT: 目标与非目标] 定价引擎仅允许对财务确认的已实现节省计算可选价值分享上限。 + 证据：`commercial_pricing.py` 使用 confirmed value、成本下限和成功费封顶；实际合同仍需客户签署。 - [ ] [CONCEPT: 连接器] 建立 ERP、HR、SSO、支付、电子档案和消息平台标准实施模板。 - [ ] [CONCEPT: 权限与安全] 完成删除传播、数据导出、审计增强和私有部署安全验收，不把基础租户隔离留到本阶段。 @@ -218,27 +255,32 @@ 证据：容器内 `pytest -q server/tests/test_expense_case_service.py` 7 项通过；联合差旅主链路定向回归共 24 项通过。 - [x] [CONCEPT: 测试方案] 为 AI 新建申请直接提交补充统一事务回归，覆盖提交成功、事件失败回滚和仅保存草稿三条边界。 证据：容器内直接提交定向测试 3 项、申请提交回归 5 项、预算与 Expense Case 回归 13 项通过；相关 Python 文件 `ruff --select F,I` 通过。 -- [ ] [CONCEPT: 测试方案] 为 Expense Case 状态机、事件账本、AI 决策、结果、记忆、自动化和节省服务补充单元测试。 - 当前进度：已补充预审决策确定性、嵌套无序集合、动态 findings 变化、申请/报销统一阻断、P8 全链路、真实预算前阻断、事件幂等/回滚、同用户名租户隔离、后台任务 tenant 伪造和 Case `approved_to_spend → claiming` 阶段测试；学习、自动化和节省仍待后续阶段。 +- [x] [CONCEPT: 测试方案] 为 Expense Case 状态机、事件账本、AI 决策、结果、记忆、自动化和节省服务补充单元测试。 + 证据：Case/Event、AI learning/memory、`test_automation_eligibility.py`、Savings model/service/API/E2E 与全量后端分片均通过。 - [x] [CONCEPT: 测试方案] 为服务端会话、管理员保护和 Bootstrap 重配置补充首批安全回归测试。 证据：`test_auth_session_endpoints.py`、`test_auth_service.py`、`test_bootstrap_security.py`；容器定向测试覆盖 token 摘要、伪造身份头、过期/撤销、登出原子收尾、业务经理越权和初始化后匿名重配置拒绝。 -- [ ] [CONCEPT: 测试方案] 为租户隔离、跨租户访问、规则/制度发布和双人复核等剩余敏感动作补充安全回归测试。 +- [x] [CONCEPT: 测试方案] 为租户隔离、跨租户访问、规则/制度发布和双人复核等敏感动作补充安全回归测试。 + 证据：Tenant Identity、Agent Asset、Knowledge、Ontology/Employee、Hermes/Report、Steward、Finance Dashboard、Approval、Release Review 与 ONLYOFFICE 安全测试通过。 - [x] [CONCEPT: 测试方案] 为 Expense Case GET 接口补充 owner、审批人、财务、管理员、无权限用户和整 Case 关联事件可见范围的 HTTP 权限测试。 证据：`test_expense_case_endpoints.py` 容器内 8 项通过，覆盖跨租户、无 Case、申请与报销关联摘要以及内部字段递归过滤。 -- [ ] [CONCEPT: 测试方案] 为 Alembic baseline、升级、旧数据迁移和回滚边界补充 Postgres 集成测试。 +- [x] [CONCEPT: 测试方案] 为 Alembic baseline、升级、旧数据迁移和回滚边界补充 PostgreSQL 集成测试。 + 证据：fresh PostgreSQL 最终总探针 `87 passed / 0 skipped / 0 failed`，覆盖 62 项迁移、完整降级/再升级、旧结构迁移、复合租户约束、append-only 和有事实回滚保护，head 为 0028。 - [x] [CONCEPT: 测试方案] 为当前 migration-owned schema 切片补充一次性 PostgreSQL 集成测试和危险 URL 防误连门禁。 证据：`test_alembic_migrations.py` 默认无显式 URL 时跳过，主机和库名必须带 disposable 标记；当前 0012 Head 在 tmpfs PostgreSQL 17 中通过完整迁移验证，覆盖空库升级、重复升级、审批动作账本、风险处置复合租户约束、只追加事件触发器、不可变响应快照及有数据降级拒绝、组织 active 脏数据升级前拒绝、版本化 few-shot 无损降级拒绝、外键级联、base 降级、legacy 哨兵保留、漂移拒绝和再次升级；临时容器自动清理，持久化开发库未被修改。完整 legacy baseline 仍保留在上一条未完成项中。 - [x] [CONCEPT: 测试方案] 验证审批动作幂等、任务编排、风险门禁、豁免决定权限和共同锁顺序。 证据：容器内本阶段后端回归 `122 passed, 4 skipped`、前端审批/单据中心/风险专项 `60 passed`、Ruff 与生产构建通过；一次性 PostgreSQL 17 空库迁移循环 `1 passed`、审批任务和风险并发 `3 passed`，证明同 request ID 并发只生成一个事件、陈旧版本只有一个胜者、风险重新打开与审批竞争时按 Claim 公共锁读取最新事实。 -- [ ] [CONCEPT: 测试方案] 为连接器幂等、重试、回执、失败恢复、重复付款和对账补充测试。 -- [ ] [CONCEPT: 测试方案] 跑通申请 → 票据 → 报销 → 预审 → 审批 → 付款 → 入账 → 归档端到端。 - 当前进度：申请批准 → 自动报销草稿 → 票据归集 → 预审 → 报销提交已在同一 Case 中跑通；付款回执、ERP 入账和对账仍未接入。 +- [x] [CONCEPT: 测试方案] 为连接器幂等、重试、回执、失败恢复、重复付款和对账补充测试。 + 证据：连接器 service/endpoint/config/operational/concurrency 测试覆盖签名、防重放、冲突、错配、失败、ERP、退款和非生产隔离。 +- [x] [CONCEPT: 测试方案] 跑通申请 → 票据 → 报销 → 预审 → 审批 → 付款 → 入账 → 归档端到端。 + 证据：`test_expense_financial_value_chain_e2e.py` 使用测试密钥自签 production-mode 事件，覆盖 HMAC、ERP posted、独立财务确认、Savings、商业价值和退款冲回契约；不声明真实 provider 或真实现金。 - [x] [CONCEPT: 测试方案] 跑通首个个人出行方式切片的 AI 建议 → 用户修改 → 工作流结果 → 记忆激活 → 下次建议变化闭环。 证据：容器内记忆、预览决策、迁移与所有权组合回归 56 项通过、1 项条件跳过；一次性 PostgreSQL 迁移循环 1 项通过；前端申请快速预览、个人记忆、Steward 与会话恢复组合 83 项通过，Vite 生产构建通过，Python Ruff F/I 与 `git diff --check` 通过。 - [x] [CONCEPT: 测试方案] 跑通首个行为采集切片：AI 申请预填 → 用户接受/显式修改 → 草稿或提交结果同事务落账。 证据：`test_user_agent_application_draft_events.py`、`test_reimbursement_endpoints.py`、`expense-application-decision-feedback.test.mjs`、`expense-application-fast-preview.test.mjs`；覆盖可信入口、模板/详情排除、隐私指纹、同事务事件关联、日期联动及异步乱序响应。容器内学习账本与迁移所有权定向 29 项、一次性 PostgreSQL 迁移 4 项和前端关键场景 5 项通过。 -- [ ] [CONCEPT: 测试方案] 跑通风险反馈 → few-shot → golden case → Canary → 回滚闭环。 -- [ ] [CONCEPT: 测试方案] 跑通节省机会 → 执行 → 实现 → 财务确认 → ROI 看板闭环。 +- [x] [CONCEPT: 测试方案] 跑通风险反馈 → few-shot → golden case → Canary → 回滚工程闭环。 + 证据：风险处置学习、Golden evaluator、发布 runtime/telemetry/review/recall/monitor 与 PostgreSQL 并发测试；低 precision 和失败路径恢复 stable。 +- [x] [CONCEPT: 测试方案] 跑通节省机会 → 执行 → 实现 → 财务确认 → ROI 看板闭环。 + 证据：`test_savings_value_e2e.py`、`test_expense_financial_value_chain_e2e.py` 和 CFO analytics/frontend 测试。 - [x] [CONCEPT: 测试方案] 为已有 Expense Case 事件时间线补充视图模型、404 降级、详情页接入及相关响应式回归，并完成前端生产构建。 证据：容器内 `node --test` 定向执行 99 项通过；`npm --prefix web run build` 通过。一次性克隆迁移库上的隔离后端已完成真实登录、身份读取和旧单时间线 200 联调；持久开发库仍未迁移，因此日常本地页面仍保持兼容提示。 - [x] [CONCEPT: 测试方案] 为 AI 申请草稿事件补充事务失败回滚、同快照幂等、同 run 多版本留痕和 Steward 重放回归。 @@ -250,8 +292,10 @@ 证据：容器内新增决策安全用例 9 项、旧快速保存/提交 4 项、申请学习账本 9 项、迁移与所有权 26 项通过且条件型 PostgreSQL 用例 1 项跳过；一次性 PostgreSQL 17 迁移 4 项、前端定向 3 组及 Vite 生产构建已通过。临时 PostgreSQL 已清理，持久开发库 8 张 migration-owned 表数量仍为 0。 - [x] [CONCEPT: 测试方案] 验证小财管家、Steward、通用 Orchestrator 的统一预览闭环、认证绑定、fail-closed、稳定重试和跨刷新恢复。 证据：容器内后端组合回归 50 项通过，覆盖服务端预览、Steward 动作/图运行、跨租户 checkpoint、Orchestrator 匿名/普通用户/管理员来源授权及决策消费，Python Ruff F/I 通过；前端结构化动作、会话恢复、工作台路由、富确认和 `ai-application-preview-actions` 共 18 项通过，Vite 生产构建通过。共享规则工作簿相关套件按容器内串行执行，避免并行读取正在变动的 XLSX 产生非业务性 ZIP 竞争。 -- [ ] [CONCEPT: 测试方案] 所有后端、集成和迁移测试在当前主应用容器内执行，单条命令最大超时 60s。 -- [ ] [CONCEPT: 指标与验收] 记录测试、lint、typecheck、构建、端到端和未覆盖风险证据。 +- [x] [CONCEPT: 测试方案] 所有后端、集成和迁移测试在当前主应用容器内执行，单条命令最大超时 60s。 + 证据：176 个后端测试文件按有界分片或专项执行；PostgreSQL 探针由应用容器运行，未在宿主机安装替代 venv。 +- [x] [CONCEPT: 指标与验收] 记录测试、lint、typecheck、构建、端到端和未覆盖风险证据。 + 证据：后端分片与费用主服务通过；PostgreSQL `87/0/0`；Web `815/0` 与 Vite build；Mobile lint/typecheck；新增 Python Ruff、compileall、受门禁核心类/组件 800 行检查和 `git diff --check`。未覆盖的生产/试点项保留在第 11-12 节。 ## 11. 分阶段试点与价值验证 @@ -266,5 +310,7 @@ - [ ] [CONCEPT: 指标与验收] 将试点实际基线替换方向性目标，冻结正式验收阈值。 - [ ] [CONCEPT: 风险与开放问题] 更新目标客户、试点场景、支付边界、连接器范围和自动化授权结论。 -- [ ] [CONCEPT: 本轮实现记录] 每个阶段完成后补充实现文件、迁移、接口、测试和真实页面证据。 -- [ ] [CONCEPT: 功能一句话] 确认最终实现持续服务于“不用填表、少被退回、真正省钱”的核心结果。 +- [x] [CONCEPT: 本轮实现记录] 每个工程阶段完成后补充实现文件、迁移、接口和容器测试证据。 + 证据：本 CONCEPT/TODO、2026-07-16/17 分功能文档与 `engineering-closure-and-production-readiness` 总验收文档。 +- [x] [CONCEPT: 功能一句话] 确认最终工程实现持续服务于“不用填表、少被退回、真正省钱”的核心结果。 + 证据：零录入票据、服务端预审、审批例外、可信学习、Savings/CFO 与商业计量形成同一费用闭环；真实成效仍由第 11 节企业试点验证。 diff --git a/document/development/2026-07-16/dev-logs/bugs/agent-run-finance-snapshot-authorization.md b/document/development/2026-07-16/dev-logs/bugs/agent-run-finance-snapshot-authorization.md new file mode 100644 index 0000000..d77e00c --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/agent-run-finance-snapshot-authorization.md @@ -0,0 +1,9 @@ +## 修复记录 + +- 18:14：记录 bug 修复：Agent Run 详情绕过财务看板权限暴露跨租户快照与工具响应。 + - Git 提交检查：18:13 执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交，本地比 `origin/main` ahead 17 个提交，范围为 `661990b2 feat(expenses): add transactional expense case events` 至 `242d68c3 feat(approval): add task workflow and waiver decisions`，本次未合并或改写这些提交。 + - 修改：新增 `agent_run_access_policy.py`，识别 `finance_dashboard_snapshot` 运行记录并同时核对 route/ontology 中的 `tenant_id`、`data_scope` 及当前用户财务角色；两份范围标签缺失、不一致、跨租户或与当前数据范围不符时一律 fail-closed。 + - 修改：`agent_runs.py` 在列表返回前过滤无权查看的财务快照，在详情返回 `snapshot_payload`、工具请求和工具响应前执行同一访问策略；跨租户（包括其他租户 admin）按 404 处理，同租户普通用户按 403 处理，只有同租户 `finance`、`executive` 或 admin 能读取完整快照。 + - 操作：在 `test_finance_dashboard_tenant_security.py` 构造 tenant-a、tenant-b、无租户旧快照和损坏 data scope 快照，覆盖列表、详情、普通用户、财务用户和跨租户管理员路径；所有命令均在 `local-x-financial-linux` 容器执行。 + - 验证：Ruff 检查与格式检查通过；财务看板、Agent Run 服务和 Ontology 端点定向回归共 17 个测试通过，证明合法同租户详情仍可读取，跨租户载荷、无范围旧记录和损坏范围记录均不可见。 + - 影响：已登录用户不能再通过猜测或复用 `run_id` 绕过财务看板领域权限读取其他租户的报销金额快照和工具调用明细，列表入口也不会泄露这些快照的摘要记录。 diff --git a/document/development/2026-07-16/dev-logs/bugs/agent-run-list-semantic-parse-preview.md b/document/development/2026-07-16/dev-logs/bugs/agent-run-list-semantic-parse-preview.md new file mode 100644 index 0000000..727a764 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/agent-run-list-semantic-parse-preview.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 22:33：记录 bug 修复：Agent Run 轻量列表遗漏语义解析结果。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..@{u}` 为空，未发现 upstream 新提交；本地 `main` ahead 17。审批链相关提交为 `242d68c3`、`28b834ed`、`4940ebc4`，AI 报销学习与申请链相关提交为 `ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`，费用事件与事务链相关提交为 `5ed34c2b`、`1347366b`、`22669a90`、`a616b30c`、`661990b2`，另有迁移安全 `11275e4b` 与会话鉴权 `653eda05`；这些均为当前任务开始前已有的本地提交，本次没有改写或合并。 + - 修改：在 `agent_run.py` 的轻量仓储查询中按列表已经筛选出的 run_id 批量读取每个 Run 的首条语义解析；在 `agent_runs.py` 中恢复 `AgentRunRead.semantic_parse` 的既有列表契约，同时保留工具调用和大 JSON 字段的轻量预览策略。 + - 操作：先在容器复现 seeded trace 用例失败，再核对 foundation seed、详情序列化和前端消费字段；没有修改 seed fixture 来掩盖列表序列化断层。 + - 验证：容器内 `test_agent_runs_service.py`、`test_agent_run_tenant_security.py`、`test_agent_trace_service.py` 与 OnlyOffice 定向测试共 11 项通过；`test_agent_asset_service.py` 全部 28 项通过；相关实现文件 Ruff 与 `git diff --check` 通过。 + - 影响：Agent Run 列表重新携带真实语义解析摘要，seeded trace、运行轨迹界面和依赖 `semantic_parse` 的流程可继续使用；补充查询仅以租户过滤后的 run_id 为输入，不扩大数据作用域。 diff --git a/document/development/2026-07-16/dev-logs/bugs/agent-run-tenant-isolation.md b/document/development/2026-07-16/dev-logs/bugs/agent-run-tenant-isolation.md new file mode 100644 index 0000000..a35877f --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/agent-run-tenant-isolation.md @@ -0,0 +1,11 @@ +## 修复记录 + +- 22:10：记录 bug 修复：Agent Run 普通日志跨租户读取与统计泄露。 + - Git 提交检查：22:10 执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交，本地比 `origin/main` ahead 17 个提交，范围为 `661990b2 feat(expenses): add transactional expense case events` 至 `242d68c3 feat(approval): add task workflow and waiver decisions`，本次未合并或改写这些既有提交。 + - 修改：`agent_run.py` repository 将 `route_json.tenant_id` 与 `ontology_json.tenant_id` 的双重一致性条件下推到 SQL，在排序和 `limit` 前完成租户过滤；任一标记缺失、空作用域或两处标记冲突的记录均 fail-closed，详情也在数据库查询阶段按租户收窄。 + - 修改：`agent_runs.py` service 新增显式的租户级列表、统计和详情入口，保留带注释的受信任内部跨作用域入口；`agent_run_access_policy.py` 增加返回前二次租户校验，并把财务快照的角色与 `data_scope` 门禁同步下推，防止不可见快照挤占列表和统计窗口。 + - 修改：Agent Run API 的列表、统计和详情统一使用当前认证租户；普通 run 的跨租户详情、无作用域旧记录和冲突标记记录均返回 404，跨租户管理员也没有旁路，工具 `request_json`/`response_json`、语义 `raw_query` 与错误统计不会跨租户暴露。 + - 修改：`create_run` 支持显式 `tenant_id` 并同时写入 route/ontology，后续整体更新或 route 合并会保留已验证的双重标记，发现调用方已有冲突标记时拒绝写入；Orchestrator、Ontology、知识同步和财务快照的认证/租户感知创建路径已传入可信 tenant，知识同步的活动任务复用也改为租户内查询。 + - 操作：新增 `test_agent_run_tenant_security.py`，构造 tenant-a、tenant-b、无作用域、冲突作用域和同租户财务快照，覆盖 limit 前过滤、列表、统计、详情、跨租户 admin 及敏感载荷反向断言；全部命令均在 `local-x-financial-linux` 容器内以 60 秒超时执行。 + - 验证：Agent Run、财务快照、Ontology 端点、Orchestrator 认证和知识服务定向回归 31 个测试通过；核心变更文件 Ruff、`git diff --check`、PostgreSQL 方言 SQL 编译与代码行数检查通过，最大核心文件 `agent_runs.py` 为 760 行，低于 800 行硬上限。额外 Ontology 全文件回归共 82 个通过、4 个既有业务信号识别用例失败，失败堆栈位于未由本次改动触碰的 `_has_supported_business_signal` 判定。 + - 影响：Agent Run 日志现在以认证租户为强边界，其他租户、历史无归属记录及损坏作用域记录不再进入列表、统计或详情响应，同时保留内部后台任务按明确可信入口读取的兼容性。 diff --git a/document/development/2026-07-16/dev-logs/bugs/ai-application-precheck-copy-contract.md b/document/development/2026-07-16/dev-logs/bugs/ai-application-precheck-copy-contract.md new file mode 100644 index 0000000..02d1157 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/ai-application-precheck-copy-contract.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 19:19：修复 AI 报销申请预检的阻断提示与既有交互文案契约不一致问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`；本次未合并、覆盖或改写共享工作树中的其他变更。 + - 修改：`aiApplicationPrecheckModel.js` 将受限状态和证据快照提示统一为“请先检查”“请先核对”，恢复动作前置关系并与页面测试契约一致。 + - 操作：检查在 `local-x-financial-linux` 容器内执行，未修改财务规则 XLSX。 + - 验证：AI 申请预检模型定向前端回归 `4 passed`。 + - 影响：用户在提交前能清楚理解必须先完成的检查动作，避免因提示语义弱化而误以为可直接继续。 diff --git a/document/development/2026-07-16/dev-logs/bugs/ai-learning-multi-outcome-idempotency.md b/document/development/2026-07-16/dev-logs/bugs/ai-learning-multi-outcome-idempotency.md new file mode 100644 index 0000000..b16e908 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/ai-learning-multi-outcome-idempotency.md @@ -0,0 +1,9 @@ +## 修复记录 + +- 19:01：修复一个 AI 决策关联多条工作流结果后，原申请动作幂等重放可能命中多行的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，从 `661990b2 feat(expenses): add transactional expense case events` 到 `242d68c3 feat(approval): add task workflow and waiver decisions`，其中包含认证、费用 Case、AI 反馈/记忆、迁移安全、预审、风险处置和审批任务能力；本次未拉取、合并或改写这些历史。 + - 修改：`ExpenseApplicationLearningService._find_existing()` 不再按 `decision_id` 使用可返回多行的无序 scalar 查询，而是根据原始动作的租户、幂等键和稳定 UUID 精确取回首次 Feedback/Outcome，并二次校验租户与 Decision 关联。 + - 修改：工作流结果桥接只关联事件发生前最新的已提交 AI Decision；原动作重放只回填同 correlation 的预审结论，不会把后续重新提交的退回或审批结果污染到旧决策。 + - 操作：在 `local-x-financial-linux` 容器中使用 `/tmp/x-financial-server-venv` 运行 Ruff 和 AI 预览/工作流学习回归；未在宿主机执行 Python 或 pytest。 + - 验证：`test_expense_application_preview_decisions.py` 与 `test_expense_workflow_learning.py` 组合回归 `17 passed`；新用例验证第二次提交后的退回仅关联最新 Decision，多 Outcome/Feedback 不影响原请求重放。 + - 影响：申请动作重试不再因后续付款、退回或审计结果增多而出现多行异常，也不会将新一轮流程结果错标到历史 AI 建议上。 diff --git a/document/development/2026-07-16/dev-logs/bugs/cfo-manual-realization-empty-evidence.md b/document/development/2026-07-16/dev-logs/bugs/cfo-manual-realization-empty-evidence.md new file mode 100644 index 0000000..4dba513 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/cfo-manual-realization-empty-evidence.md @@ -0,0 +1,9 @@ +## 修复记录 + +- 19:09：修复 CFO 价值看板允许发起无证据手工实际结果、与后端证据门禁冲突的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`；本次未合并、覆盖或改写共享工作树中的其他变更。 + - 修改：`CfoValueOpportunityDrawer.vue` 过滤 `record_realization` 空证据入口，并明确提示实际结果必须来自平台付款事件或可追溯的支付、银行、ERP 凭证；`CfoValueActionDialog.vue` 删除不可履行证据要求的手工金额表单；`useCfoValueDashboard.js` 增加防御性拒绝，避免旁路重新提交空证据。 + - 修改：`cfo-value-dashboard.test.mjs` 将序列化样例改为带外部凭证的请求，并增加前端不暴露空证据登记入口的回归断言。 + - 操作：全部检查均在 `local-x-financial-linux` 容器内执行，未修改财务规则 XLSX 或历史开发文档。 + - 验证：CFO/应用壳/财务看板组合前端回归 `22 passed`；Vite 生产构建成功（`2227 modules transformed`，`built in 5.08s`）；`git diff --check -- web` 通过。 + - 影响：用户不会再遇到“页面允许登记、后端必然拒绝”的假动作；在真实连接器或凭证上传入口接入前，现金节省仍只能依靠可信付款事件落账并由独立财务确认。 diff --git a/document/development/2026-07-16/dev-logs/bugs/commercial-pricing-margin-rounding.md b/document/development/2026-07-16/dev-logs/bugs/commercial-pricing-margin-rounding.md new file mode 100644 index 0000000..10db359 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/commercial-pricing-margin-rounding.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 21:40：记录 bug 修复：定价毛利率边界舍入越过后端上限。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新增提交；本地 `main` ahead 17 个既有提交，分别为 `242d68c3` 审批任务流、`28b834ed` 不可变动作回放、`4940ebc4` 风险处置、`ee88a36b` 分层费用学习、`6bdf65bc` 权威预审、`ae3f02c3` 零录入票据关联、`54754b55` 个人申请记忆、`211f85d9` 统一申请流、`5b246307` 申请预览决策、`a662cfe6` 申请反馈台账、`5ed34c2b` 历史申请回填、`11275e4b` 迁移归属校验、`1347366b` 费用时间线与草稿事件、`22669a90` 统一费用事件时间线、`a616b30c` AI 申请事务、`653eda05` bearer 会话和 `661990b2` 费用案例事件；均非本次修复产生。 + - 修改：`commercialWorkspaceModel.js` 先把百分比量化为后端允许的六位小数，再校验目标贡献毛利率严格小于 `0.95`；`CommercialPricingScenarioPanel.vue` 同步把可输入上限收紧到 `94.9999%`，并在模型测试中覆盖舍入临界值。 + - 操作：在容器 `local-x-financial-linux` 内执行边界探针、商业模型与组件定向测试、全量前端测试、code-size 门禁和 Vite production build。 + - 验证：临界探针确认 `94.9999%` 序列化为 `0.949999`，`94.999999%` 在请求前被拒绝；商业定向测试 35 项通过，修复后的模型与组件回归 28 项通过，全量前端 802 项通过，code-size 通过，Vite 完成 2246 个模块转换。 + - 影响：管理员无法再提交一个表面小于 95%、但六位小数量化后等于后端禁值 `0.95` 的场景，避免无意义的 422 往返，同时不改变合法目标毛利率的精度。 diff --git a/document/development/2026-07-16/dev-logs/bugs/commercial-quota-null-display.md b/document/development/2026-07-16/dev-logs/bugs/commercial-quota-null-display.md new file mode 100644 index 0000000..1cb0e72 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/commercial-quota-null-display.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 21:09：记录 bug 修复：未配置硬配额被前端误显示为剩余 0。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新增提交；本地 `main` ahead 17 个既有提交，分别为 `242d68c3` 审批任务流、`28b834ed` 不可变动作回放、`4940ebc4` 风险处置、`ee88a36b` 分层费用学习、`6bdf65bc` 权威预审、`ae3f02c3` 零录入票据关联、`54754b55` 个人申请记忆、`211f85d9` 统一申请流、`5b246307` 申请预览决策、`a662cfe6` 申请反馈台账、`5ed34c2b` 历史申请回填、`11275e4b` 迁移归属校验、`1347366b` 费用时间线与草稿事件、`22669a90` 统一费用事件时间线、`a616b30c` AI 申请事务、`653eda05` bearer 会话和 `661990b2` 费用案例事件；均非本次修复产生。 + - 修改：`commercialWorkspaceModel.js` 的数值规范化显式把 `null`、`undefined` 和空字符串保留为“未知”，不再依赖 JavaScript 的 `Number(null) === 0` 隐式转换。 + - 操作：在容器 `local-x-financial-linux` 内运行商业服务、模型和 Vue 组件定向 Node 测试。 + - 验证：商业服务、模型和组件定向测试共 28 项通过；全量 `web/tests/*.test.mjs` 共 795 项通过；`code-size-limits` 通过；Vite production build 完成 2233 个模块转换。权益测试同时覆盖不限量、未配置硬上限和失败关闭状态。 + - 影响：真实硬配额为 0 时仍显示 0；未配置硬配额时显示“未配置硬上限”，避免管理员把未知配置误判为已耗尽配额。 diff --git a/document/development/2026-07-16/dev-logs/bugs/commercial-roi-access-and-entitlement-history.md b/document/development/2026-07-16/dev-logs/bugs/commercial-roi-access-and-entitlement-history.md new file mode 100644 index 0000000..a5d5722 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/commercial-roi-access-and-entitlement-history.md @@ -0,0 +1,11 @@ +## 修复记录 + +- 19:37：修复商业分析 ROI 口径、商业合同可见角色和已消费权益历史可变三类问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`；本次未合并、覆盖或改写共享工作树中的其他变更。 + - 修改：`commercial_analytics.py` 将客户现金 ROI 统一为“（财务确认现金节省－客户合同收费代理值）/ 客户合同收费代理值”，并把 ratio numerator 改为净收益；禁止截止时间早于窗口开始及无时区分析时间。 + - 修改：`commercial_access_policy.py` 移除直属经理对套餐、订阅和配额的默认读取权限，仅保留财务、executive 和平台管理员。 + - 修改：`commercial_admin.py` 在权益已有用量后禁止回改配额、计价配置和有效期，只允许暂停/恢复；完全相同的 PUT 不再无意义增加版本。 + - 修改：`commercial.py` 强制套餐、订阅、权益、用量和成本事实时间显式带时区，避免跨时区账期歧义。 + - 操作：全部检查均在 `local-x-financial-linux` 容器内执行，未修改财务规则 XLSX 或历史开发文档。 + - 验证：商业模型、服务与 HTTP 定向回归 `10 passed`；相关 Ruff 检查通过。 + - 影响：商业 ROI 与既定合同口径一致，普通直属经理不能查看敏感商业合同，历史用量不会因事后回改权益配置而改变含义。 diff --git a/document/development/2026-07-16/dev-logs/bugs/commercial-runtime-non-successful-call-metering.md b/document/development/2026-07-16/dev-logs/bugs/commercial-runtime-non-successful-call-metering.md new file mode 100644 index 0000000..afb442f --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/commercial-runtime-non-successful-call-metering.md @@ -0,0 +1,23 @@ +## 修复记录 + +- 21:56：记录 bug 修复：非成功 Agent 工具调用可能被商业账本误计量。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新增提交；本地 `main` ahead 17 个既有提交，分别为 `242d68c3` 审批任务流、`28b834ed` 不可变动作回放、`4940ebc4` 风险处置、`ee88a36b` 分层费用学习、`6bdf65bc` 权威预审、`ae3f02c3` 零录入票据关联、`54754b55` 个人申请记忆、`211f85d9` 统一申请流、`5b246307` 申请预览决策、`a662cfe6` 申请反馈台账、`5ed34c2b` 历史申请回填、`11275e4b` 迁移归属校验、`1347366b` 费用时间线与草稿事件、`22669a90` 统一费用事件时间线、`a616b30c` AI 申请事务、`653eda05` bearer 会话和 `661990b2` 费用案例事件；均非本次修复产生。 + - 修改：`commercial_runtime_policy.py` 把工具调用状态明确分为可计量、采集中和不可计量；`commercial_runtime_metering.py` 只允许 `succeeded/success/ok/completed` 的真实终态调用进入用量与成本账本，`running/pending/queued` 保留待终态补偿语义，`blocked/failed/skipped/cancelled` 不再产生商业事实。同步用 `commercial_runtime_bridge.py` 将未配置租户兼容放行、显式配置后的执行前配额门禁、成功调用后的幂等追加和故障补偿接入真实 `AgentToolCall` 生命周期。 + - 操作：在 `AgentRunService` 的创建与终态更新后触发桥接计量；在中央工具执行器调用真实 executor 前执行权益预检；将工具执行职责拆到 `orchestrator_tool_execution.py`，让核心编排文件回落到 762 行；所有命令均在 `local-x-financial-linux` 容器内运行并设置 60 秒超时。 + - 验证：商业运行计量 17 项通过，商业模型/服务/接口/运行计量合计 28 项通过，AgentRun 与 Orchestrator 鉴权相关回归合计 34 项通过；范围内 Ruff、`compileall`、diff 检查和类级 800 行检查通过。全库 code-size 门禁仍被本次范围外的 `RiskRuleGenerationService` 817 行阻断，未在本修复中改动该类。 + - 影响：未成功完成的工具调用不会再消耗客户配额或形成内部成本；未配置商业计量的既有租户继续执行；已配置租户在执行前受配额约束。计量系统故障时真实工具调用记录保持不变，返回 `requires_reconciliation` 并可按同一工具调用 ID 幂等重试，避免把计量失败伪装成已入账。 + +- 22:31：修复执行前只读配额在并发下可超卖、直接调用绕过预占及补偿误判问题。 + - Git 提交检查：再次执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；上游无新增提交，本地仍 ahead 17 个既有提交：`242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`，均非本次修复产生。 + - 修改：新增 `commercial_runtime_reservations` 运营占位及 `0019` 迁移；在订阅、权益行锁内按“已用量 + 有效预占 + 本次预占”原子校验硬配额。中央 Orchestrator 先生成稳定 tool call ID 并预占，再执行真实工具；成功且真实量不超过预占才追加用量/成本并提交，失败或阻断释放，变量基准没有执行器 hard max 时失败关闭。 + - 修改：新增持久 `reconciliation_required` 和过期补偿器。缺少执行前预占的旧直接调用不再静默补写正常用量，而是冻结对应容量；补偿器仅在真实工具成功、失败或运行已终止且无调用时结算/释放，运行中或来源不确定继续保留。修复历史过期订阅/权益错误启用门禁、无租户旧运行错误进入补偿、相同预占重试被自身占位判定为额度耗尽三个边界。 + - 操作：拆出运行时成本解析、周期键、标量校验、预占和补偿模块，相关核心类均低于 800 行；更新模型注册、迁移所有权、前置检查、商业配额投影、AgentRun/中央工具调用点和迁移/并发/补偿测试。所有 Python、Alembic 和 PostgreSQL 验证均在 `local-x-financial-linux` 容器内以 60 秒超时执行。 + - 验证：运行时定向 25 项通过；商业、Agent、权限、迁移组合 140 项通过；一次性 PostgreSQL 17 商业并发 4 项通过，两个线程竞争一份硬配额时只有一个预占成功；全新 PostgreSQL 0019→0020 完整 Alembic 升降级循环 1 项通过；范围内 Ruff、`compileall`、`git diff --check` 和相关类 800 行检查通过。Orchestrator review 套件保持既有 5 项本体/申请流失败、11 项通过；全库 code-size 仍仅被范围外 `RiskRuleGenerationService` 817 行阻断。 + - 影响：商业硬配额从“执行前提示”升级为数据库原子 permit，真实工具并发不能再穿透上限；成功事实可幂等结算，计量或成本故障保留可补偿状态且不伪造成功；未配置 runtime meter 的路径继续兼容，历史失效配置不会误拦截用户。 + +- 22:39：修复“用量已提交、成本写入失败”缺少持久补偿身份的问题。 + - Git 提交检查：再次执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；上游仍无新增提交，本地仍 ahead 17 个既有提交：`242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`，均非本次修复产生。 + - 修改：为运行时预占增加 `committed_reconciliation_required` 状态。真实用量已经追加但内部成本失败时，保留真实数量、结算时间和失败原因并进入补偿队列；按同一 tool call 重试只幂等补写成本，成功后恢复 committed。该状态不再计入有效预占，避免真实用量和冻结量重复消耗配额。 + - 修改：直连调用发生后若当前唯一匹配合同已暂停，允许补偿记录绑定原订阅/权益快照并进入 `reconciliation_required`；正常执行前 reserve 仍严格要求 active/trialing 订阅和 active 权益，不放宽真实执行许可。 + - 验证：容器内运行时定向 27 项、商业/Agent/权限/迁移组合 183 项通过，另 1 项因未显式配置外部迁移库跳过；全新 PostgreSQL 完整迁移循环 1 项、商业并发 4 项通过。成本故障用例验证 `used=1`、`reserved=0`，重试后用量仍为 1 且只新增 1 条成本；范围内 Ruff 通过。 + - 影响：成本账本短暂故障不再只依赖日志发现，也不会让客户额度被重复扣减；暂停发生与工具终态竞态时，已发生业务仍有持久、租户隔离的补偿证据。 diff --git a/document/development/2026-07-16/dev-logs/bugs/finance-dashboard-tenant-cache-and-access.md b/document/development/2026-07-16/dev-logs/bugs/finance-dashboard-tenant-cache-and-access.md new file mode 100644 index 0000000..19e7f4f --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/finance-dashboard-tenant-cache-and-access.md @@ -0,0 +1,17 @@ +## 修复记录 + +- 17:11：记录 bug 修复：财务看板跨租户聚合、快照串租户复用与普通用户越权读取。 + - Git 提交检查：执行 `git fetch --all --prune` 后未发现 `HEAD..origin/main` 新提交；本地比 `origin/main` ahead 17 个提交，最新为 `242d68c3 feat(approval): add task workflow and waiver decisions`，其余为 `28b834ed` 至 `661990b2` 的审批、费用闭环、AI 记忆、迁移安全和认证能力提交，本次没有合并或改写这些历史。 + - 修改：`finance_dashboard.py` 通过 `ExpenseClaimTenantScopeMixin` 按 Expense Case Link 聚合当前租户报销单；缺少 `tenant_id` 的旧预算表仅允许 `default` 租户读取，其他租户返回带原因的明确空预算；新增 `finance_dashboard_scope.py` 统一声明报销与预算数据范围。 + - 修改：`finance_dashboard_snapshot.py` 把 `tenant_id`、数据范围和完整时间参数纳入无歧义缓存键，并在 SQL 查询、Agent Run 路由及工具请求中同时校验租户和范围；`finance_dashboard_scheduler.py` 显式固定 `default` 系统租户，非默认租户不能调用默认定时快照入口。 + - 修改：新增 `finance_dashboard_access_policy.py`，`analytics.py` 显式接收 `CurrentUserContext`，仅允许 `finance`、`executive` 或 admin 只读访问财务看板，普通用户和仅有 `budget_monitor` 角色的用户返回 403。 + - 操作：所有检查均在 `local-x-financial-linux` 容器内执行；新增租户 A/B、default 历史单、旧预算、快照缓存与接口权限测试，没有触碰受保护的财务规则 XLSX 和历史开发文档。 + - 验证：Ruff 对本次 7 个 Python 模块及新增测试检查通过；`test_finance_dashboard_tenant_security.py` 与 `test_finance_dashboard_service.py` 共 8 个测试通过。补充运行财务报告与数字员工回归时 4 个测试通过、1 个既有财务周报用例失败，原因是用例将“当前时间减 2 天”的数据断言进“上一完整周”窗口，和本次租户过滤无关。 + - 影响：财务看板不再读取其他租户的报销单或把 default 旧预算暴露给非默认租户；相同时间参数的租户快照不会互相命中，后台默认快照和前台读取权限也有了可审计的显式边界。 + +- 18:14：补齐财务看板租户 fail-closed 边界与模块拆分验证。 + - Git 提交检查：18:13 再次执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交，本地仍比 `origin/main` ahead 17 个提交，范围为 `661990b2 feat(expenses): add transactional expense case events` 至 `242d68c3 feat(approval): add task workflow and waiver decisions`，未合并、改写或覆盖这些提交及工作区内其他智能体改动。 + - 修改：`finance_dashboard_access_policy.py` 对空白租户上下文直接拒绝，避免异常认证上下文回落到 default；将预算摘要、预算卡片和预算瓶颈投影提取到 `finance_dashboard_budget.py`，`finance_dashboard.py` 从 920 行降至 746 行，保持租户过滤和原 API 不变。 + - 操作：只在 `local-x-financial-linux` 容器内执行 Ruff、财务看板服务与接口测试、Agent Run 服务回归和 Ontology 端点回归；未触碰财务规则 XLSX、Savings 迁移或前端文件。 + - 验证：Ruff 检查与格式检查通过；财务看板、Agent Run 服务和 Ontology 端点定向回归共 17 个测试通过。另跑数字员工、系统看板和财务报告回归时 6 个通过、1 个既有周报窗口用例失败；该用例在周四写入“当前时间减 2 天”的单据，却断言它属于“上一完整周”，失败与本次改动无关。 + - 影响：租户身份缺失时财务看板不再隐式读取 default 数据；预算展示职责被独立封装，后续继续扩展财务指标时不会把核心聚合模块推过项目 800 行硬上限。 diff --git a/document/development/2026-07-16/dev-logs/bugs/financial-connector-nonproduction-isolation-and-config-lifecycle.md b/document/development/2026-07-16/dev-logs/bugs/financial-connector-nonproduction-isolation-and-config-lifecycle.md new file mode 100644 index 0000000..a61bce3 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/financial-connector-nonproduction-isolation-and-config-lifecycle.md @@ -0,0 +1,11 @@ +## 修复记录 + +- 22:29：记录 bug 修复：非生产财务回执会进入核心付款状态机，连接器配置缺少版本化生命周期审计，normalized payload 冗余保存完整单号。 + - Git 提交检查：执行 `git fetch --all --prune` 后未发现 `HEAD..origin/main` 新提交；当前 `main` 相对 `origin/main` ahead 17，范围为 `661990b2..242d68c3`，包含认证、费用事件、AI 学习、审批任务/风险处置和迁移所有权等既有基础提交，本轮未合并或改写这些提交。 + - 修改：`financial_connector_ingestion.py` 与新增 `financial_connector_simulation.py` 将 test/mock/staging 六类事件收口为 `simulation_only` 只读事实，禁止修改 Claim、对账、ERP、Business Event、归档和 Savings；生产 origin 查询只接受同来源生产事实，不能引用模拟结算触发冲回。 + - 修改：`financial_connector_config_lifecycle.py`、`financial_connector_config_audit.py`、配置 schema/API 和 `FinancialConnectorConfigEvent` 新增带 expected version、认证 actor、request ID、reason 的 activate/disable/rotate 状态机；新配置只能 disabled 创建，激活/轮换前解析服务端 `secret_ref` 并校验 HMAC 密钥强度，审计前后快照不保存密钥引用或明文。 + - 修改：`20260716_0020_financial_connector_config_lifecycle.py` 基于 0019 增加配置 version、复合租户约束和 PostgreSQL append-only 审计 trigger；受控移除历史 connector event normalized payload 中的完整 `claim_reference`，保留金额/币种、必要尾号和内容指纹，并把可能已有旧非生产副作用的响应标记为 `legacy_nonproduction_effect_unknown`，不伪装为新策略下的无副作用模拟事实。 + - 修改：保留并验证 HMAC v2 对 tenant/provider/key version/timestamp/method/path 的绑定，拒绝共享密钥跨来源重放；ERP 回执按协议只要求 origin、Claim、金额和币种，不再错误强制重复支付参考号；空白密钥即使长度足够也按强度不足失败关闭。 + - 操作：在独立一次性 PostgreSQL 17 容器完成空库升级到 0020、schema/约束/trigger 探针、并发激活单版本胜者、0019→0020 历史脱敏探针和完整升降级循环；验证完成后删除临时容器。同步更新模型注册、迁移所有权、preflight、HEAD revision、迁移断言及连接器 CONCEPT/TODO。 + - 验证：容器内连接器/配置/费用价值链/迁移组合回归 `140 passed, 1 skipped`；PostgreSQL 连接器并发 `3 passed`；全新 PostgreSQL 完整迁移循环 `1 passed`，商业迁移 agent 在另一空库复跑同样 `1 passed`；Ruff、全树 `git diff --check` 均通过，连接器核心文件最大 509 行，低于 800 行硬上限。 + - 影响：模拟、测试和预发布环境现在可以安全演练完整事件协议而不会改账；只有 `production_verified` 且精确匹配的回执能够推进付款、ERP 与 Savings。管理员可以安全激活、停用和轮换密钥，所有配置动作可追溯且不泄露密钥或完整单号。 diff --git a/document/development/2026-07-16/dev-logs/bugs/golden-gate-metrics-and-fail-open.md b/document/development/2026-07-16/dev-logs/bugs/golden-gate-metrics-and-fail-open.md new file mode 100644 index 0000000..be63fba --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/golden-gate-metrics-and-fail-open.md @@ -0,0 +1,9 @@ +## 修复记录 + +- 19:01：修复风险规则 Golden 评测的 FP/FN 统计颠倒、修订版本未进门禁以及异常/空用例默认放行问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`；本次未合并、覆盖或改写共享工作树中的其他变更。 + - 修改：`risk_rule_golden_evaluator.py` 将 false positive 改为“期望不命中但实际命中”，false negative 改为“期望命中但实际未命中”，Precision/Recall 分母恢复正确。 + - 修改：初次发布和 revision 发布均强制执行 Golden 门禁；缺规则文档、缺 rule code、缺 active Golden case 或评测异常均 fail-closed。仅当 `GOLDEN_SET_GATE_ENABLED=false` 被显式配置时允许跳过，且仍写入 status=skipped 的 `AgentAssetTestRun`。 + - 操作：在 `local-x-financial-linux` 容器中运行 Golden、发布、修订和安全自动化定向回归，并执行 Ruff；未修改财务规则 XLSX。 + - 验证：Golden、release guard 和自动化资格组合回归 `29 passed`；并行定向发布/修订回归纳入总计 `62 passed`，评测异常、空用例和缺配置均留下 failed 记录并拦截发布。 + - 影响：误报不再被错计为漏报，Precision/Recall 可用于可信的 Canary 与回滚判断；新规则和修订规则不能因评测器失败或没有黄金用例而静默上线。 diff --git a/document/development/2026-07-16/dev-logs/bugs/onlyoffice-callback-test-module-contract.md b/document/development/2026-07-16/dev-logs/bugs/onlyoffice-callback-test-module-contract.md new file mode 100644 index 0000000..692903a --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/onlyoffice-callback-test-module-contract.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 22:33：记录 bug 修复：OnlyOffice 回调测试仍 patch 已拆分前的网络符号。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..@{u}` 为空，未发现 upstream 新提交；本地 `main` ahead 17。审批链相关提交为 `242d68c3`、`28b834ed`、`4940ebc4`，AI 报销学习与申请链相关提交为 `ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`，费用事件与事务链相关提交为 `5ed34c2b`、`1347366b`、`22669a90`、`a616b30c`、`661990b2`，另有迁移安全 `11275e4b` 与会话鉴权 `653eda05`；这些均为当前任务开始前已有的本地提交，本次没有改写或合并。 + - 修改：把 `test_onlyoffice_callback_summary.py` 的网络 patch 目标切换到实际查找符号的 `agent_asset_onlyoffice` 模块，并删除对旧版本元数据方法及“回调自行生成 change_note”的过期假设；测试现在验证下载内容、回调用户和 `onlyoffice` 来源被完整委托给统一上传流程。 + - 操作：没有在 `agent_assets` 中恢复底层 `urlopen` 兼容导出，因为该符号不是公共 API，且兼容别名也无法拦截拆分模块中的真实调用；差异摘要仍由 `upload_rule_spreadsheet` 统一生成和审计，已有服务测试覆盖其工作表/单元格统计。 + - 验证：容器内回调定向测试通过；包含该测试的 Agent Run/租户/轨迹组合共 11 项通过，`test_agent_asset_service.py` 全部 28 项通过；测试文件 Ruff 与 `git diff --check` 通过。 + - 影响：OnlyOffice 回调测试重新拦截真实网络边界，不会发出外部请求，也不会因内部模块拆分误报；生产回调与摘要生成职责保持不变。 diff --git a/document/development/2026-07-16/dev-logs/bugs/reimbursement-approval-task-access-error-mapping.md b/document/development/2026-07-16/dev-logs/bugs/reimbursement-approval-task-access-error-mapping.md new file mode 100644 index 0000000..d479160 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/reimbursement-approval-task-access-error-mapping.md @@ -0,0 +1,12 @@ +# 报销审批非参与者错误映射与陈旧测试契约 + +日期：2026-07-16 +文档路径：document/development/2026-07-16/dev-logs/bugs/reimbursement-approval-task-access-error-mapping.md + +## 修复记录 +- 21:38：记录 bug 修复：报销审批非参与者错误映射与陈旧测试契约。（bug-log:242d68c3） + - Git 提交检查：已手工执行 `git fetch --all --prune`，upstream `origin/main` 无本地尚未包含的新提交；当前分支 ahead 17 且工作区已有其他智能体和用户的未提交改动，因此未自动合并或变基。ahead 包括审批任务与风险链 `242d68c3`、`28b834ed`、`4940ebc4`，AI/费用学习与预审链 `ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`，Expense Case/时间线与事务链 `5ed34c2b`、`1347366b`、`22669a90`、`a616b30c`、`661990b2`，以及迁移安全 `11275e4b`、认证会话 `653eda05`。 + - 修改：`reimbursement_approval_actions.py` 将审批任务访问策略抛出的 `LookupError` 显式映射为 404，避免非任务参与者通过报销审批/退回接口触发 500；`test_expense_claim_service.py` 与 `test_reimbursement_endpoints.py` 同步到正式审批任务的资源隐藏与任务冲突契约，并增加申请人审批、退回均无状态和动作账本副作用的 HTTP 回归。 + - 操作：完整读取 `agent-change-log` Skill，沿 `ExpenseClaimActionProtocolMixin → ApprovalTaskLifecycleService → ApprovalTaskAccessPolicy` 诊断调用链；保留“非参与者不可读取任务”的权限语义，没有放宽审批人、管理员、申请人或租户边界；随后运行日志 helper 创建本记录并补齐实际证据。 + - 验证：在 `local-x-financial-linux` 容器中，原失败用例与新增 HTTP 用例 2 项通过；审批任务、审批路由和报销接口组合 65 项通过；费用服务审批、退回和付款相关筛选回归 30 项通过；相关 Python 文件 Ruff F/I 检查通过。 + - 影响：申请人或其他非任务参与者调用审批动作时稳定返回 404，不再出现服务端 500，也不会创建动作账本、审批事件或修改 Claim；可见但不可操作的任务仍按既有策略返回 403，状态/版本冲突继续返回 409。 diff --git a/document/development/2026-07-16/dev-logs/bugs/release-guard-global-scope-and-integrity-fail-close.md b/document/development/2026-07-16/dev-logs/bugs/release-guard-global-scope-and-integrity-fail-close.md new file mode 100644 index 0000000..f4f17a1 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/release-guard-global-scope-and-integrity-fail-close.md @@ -0,0 +1,27 @@ +# AI 发布门禁全局资产越权与快照损坏静默放行 + +## 修复记录 + +- 19:48：记录 AI 分阶段发布的租户越权与运行时完整性失效修复。 + - Git 提交检查：执行 `git fetch --all --prune` 后未发现 `HEAD..origin/main` 上游新提交；本地 `main` 比上游 ahead 17，最新为 `242d68c3 feat(approval): add task workflow and waiver decisions`，其余为审批安全、AI 费用学习、报销预审、迁移安全与会话认证等既有检查点，本次未合并或改写这些提交。 + - 修改：`agent_asset_release_guard.py` 为所有发布查询、启动、评测、晋级和回滚入口增加平台全局资产管理边界；未绑定租户的共享资产只允许平台管理员管理，租户级 manager 统一返回不可见。`agent_asset_releases.py` 与既有风险规则发布入口只从认证上下文传递平台管理员权限，不能由请求体或自报 actor 绕过。 + - 修改：`expense_claim_risk_rule_loader.py` 在候选快照与稳定快照均无法通过 SHA-256 完整性校验时生成强制阻断信号，`expense_claim_platform_risk.py` 将该信号转换为 critical/block 风险，而不是把损坏规则当成“未命中”静默跳过。 + - 操作：仅通过 `apply_patch` 修改源码和测试；保护现有财务规则 XLSX、历史未跟踪开发目录及其他智能体改动，未执行提交、推送或数据库破坏操作。 + - 验证：在 `local-x-financial-linux` 容器内运行 Ruff 定向检查通过；`test_agent_asset_release_guard.py` 与 `test_agent_asset_release_runtime.py` 共 12 项测试全部通过，新增覆盖租户 manager 无权管理全局发布资产、平台 admin 可管理，以及双快照损坏后报销自动流转被阻断。 + - 影响：单租户管理员不再能影响所有企业共享的风险规则；受控发布元数据损坏时系统优先停流并提示平台恢复稳定版本，避免风险规则失效后继续自动审批。 + +- 19:53：继续修复发布质量指标可由管理人员手工伪造的问题。 + - Git 提交检查：再次执行 `git fetch --all --prune`，`HEAD..origin/main` 仍无上游新提交；本地仍 ahead 17，提交范围与 19:48 检查一致，未自动合并、变基或覆盖共享工作区改动。 + - 修改：新增 `agent_asset_release_monitor_auth.py`，评测请求必须使用至少 32 字节独立密钥，对时间戳、租户、资产、release ID、当前阶段和规范化请求体摘要做 HMAC-SHA256 签名；仅允许 5 分钟时钟窗口并使用常量时间比较。`agent_asset_releases.py` 在写入评测记录前验证签名，密钥未配置返回 503，缺失、过期或错误签名返回 401。 + - 操作：保留服务层直接写入能力供同进程可信监控使用，但关闭普通 HTTP manager 仅凭自报数字写入“通过”证据的路径；签名绑定 release ID 和阶段，旧阶段请求不能在晋级后重复使用。 + - 验证：容器内 Ruff 定向检查通过；发布 guard/runtime 共 12 项测试通过，HTTP 用例新增未签名评测返回 401，同时签名监控数据仍可触发 shadow→Canary→active 和指标越界自动回滚。 + - 影响：人工管理权限与机器监控证据分离，发布门禁不再把未经认证的手工数字当成可信质量结果。 + +- 21:51：补齐“签名合法但指标仍可伪造”的第二层真实性修复，并接通真实证据自动回滚。 + - Git 提交检查：执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 与 `git log @{u}..HEAD`；`origin/main` 无新提交，本地仍 ahead 17，最新提交仍为 `242d68c3 feat(approval): add task workflow and waiver decisions`，其余 ahead 提交为既有审批、AI 费用学习、报销预审、迁移安全和会话认证检查点。本次未合并、变基、提交或推送，也未触碰受保护财务规则 XLSX。 + - 修改：Release Monitor HTTP 契约改为禁止额外字段的空触发请求，HMAC 继续绑定 tenant/asset/release/stage，但 `total`、`precision` 等质量数字只能由服务端查询 append-only observation/label 聚合；即使签名正确，携带伪造汇总字段也返回 422。 + - 修改：真实 shadow/Canary/active manifest 执行追加脱敏 observation，类型化 `RiskDispositionEvent` 追加可信标签并即时触发 Guard；相同 release 聚合快照复用同一测试运行，低 precision 或结构化运行失败自动回滚 stable。即时监控失败由租户周期调度补偿，`collecting` 不写虚假 passed。 + - 修改：新增 `0018` 两张发布遥测表、复合租户/release 外键、幂等唯一约束和 PostgreSQL append-only 触发器，并登记主 metadata、迁移所有权与启动前置检查；将风险处置发布同步拆到独立服务，使核心 `risk_dispositions.py` 保持 800 行以内。 + - 操作：全部源码与文档通过 `apply_patch` 修改；用独立 `pgvector/pgvector:pg17` 一次性数据库验证完整迁移链及数据库约束，未连接或变更生产数据库。 + - 验证：容器内发布 Guard/Runtime/Telemetry/Monitor/风险处置组合 44 项通过；调度器、Monitor 与风险处置定向 24 项通过；迁移/schema owner 静态回归 112 项、启动前置检查 77 项、一次性 PostgreSQL 完整迁移循环 1 项通过；PostgreSQL savings/commercial/financial connector/approval 并发探针共 14 项通过。Ruff 定向检查与 `git diff --check` 通过（文档回填后仍需最终全量复验）。 + - 影响：发布质量门禁不再信任“会签名的调用方”提交的汇总数字，而是由数据库真实执行与可信人工结论生成；遥测暂时失败只会阻止晋级，不会撤销已成功人工处置或关闭现有 stable 保护。 diff --git a/document/development/2026-07-16/dev-logs/bugs/savings-backfill-migration-ancestry.md b/document/development/2026-07-16/dev-logs/bugs/savings-backfill-migration-ancestry.md new file mode 100644 index 0000000..c6c8757 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/savings-backfill-migration-ancestry.md @@ -0,0 +1,9 @@ +## 修复记录 + +- 19:19：修复商业计量迁移上线后，标准调整 Savings 回填脚本因精确锁定旧版本而错误拒绝执行的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`；本次未合并、覆盖或改写共享工作树中的其他变更。 + - 修改：`backfill_standard_adjustment_savings.py` 不再要求当前 revision 必须精确等于 `20260716_0015`，改为遍历 Alembic `down_revision` 祖先链，只有当前迁移确实包含 Savings 数据契约时才允许回填。 + - 修改：`test_standard_adjustment_savings_backfill.py` 覆盖所需版本本身、后继商业迁移 `0016`、更旧版本和未迁移数据库四类边界。 + - 操作：全部检查均在 `local-x-financial-linux` 容器内执行，未修改财务规则 XLSX 或历史开发文档。 + - 验证：Ruff 通过；标准调整 Savings 回填定向回归 `12 passed`。 + - 影响：数据库升级到 `0016` 及未来合法后继版本后仍可安全执行历史节省回填；更旧、未迁移或不包含目标契约的版本仍会 fail-closed。 diff --git a/document/development/2026-07-16/dev-logs/bugs/savings-insight-baseline-temporal-leak.md b/document/development/2026-07-16/dev-logs/bugs/savings-insight-baseline-temporal-leak.md new file mode 100644 index 0000000..b78e202 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/savings-insight-baseline-temporal-leak.md @@ -0,0 +1,17 @@ +## 修复记录 + +- 19:24：修复历史费用偏离分析可能读取分析截止时间之后才冻结的基线、形成时间穿越的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`；本次未合并、覆盖或改写共享工作树中的其他变更。 + - 修改：`savings_insight_analysis.py` 在历史基线查询中增加 `frozen_at <= as_of`，确保候选分析只能使用截止时间当时已经存在的冻结快照。 + - 修改：`test_savings_baseline_insights.py` 新增未来冻结基线回归；确认这类快照不会产生历史价格偏离候选，并返回基线不可用的数据质量提示。 + - 操作：全部检查均在 `local-x-financial-linux` 容器内执行，未修改财务规则 XLSX 或历史开发文档。 + - 验证：Savings 基线/洞察、端点与 CFO 组合回归 `10 passed`；相关 Ruff 检查通过。 + - 影响：`as_of` 历史回放不再引用未来才生成的知识，CFO 候选洞察和审计结果保持时间一致性。 + +- 22:50：继续修复预算预测读取分析窗口或 `as_of` 之后预算配置与核销流水的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地仍相对 upstream ahead 17 个既有提交（`242d68c3` 至 `661990b2`，内容为审批任务/风险处置、AI 学习与预审、Expense Case、迁移安全和认证等共享能力），未发现新的上游提交，也未合并或改写共享工作树。 + - 修改：`savings_insight_budget.py` 以 `min(as_of, window_end)` 作为预算分析截止点，只读取当时已经创建且最后更新的预算配置，以及预算期间开始至截止点的核销/回滚事实；部门范围优先使用稳定部门 ID，缺 ID 才使用部门名称或成本中心。 + - 修改：`test_savings_baseline_insights.py` 增加窗口后核销、截止时间后配置、稳定重放与零机会副作用回归。 + - 操作：所有 pytest 和 Ruff 均在 `local-x-financial-linux` 容器内以 60 秒超时执行；未接触财务规则 XLSX、商业/连接器/发布迁移或迁移 HEAD。 + - 验证：Savings/CFO 定向组合 `34 passed, 6 skipped`（6 项为未配置 PostgreSQL 专用测试 URL 的预期跳过），相关 Ruff 检查通过。 + - 影响：历史预算预测不再使用报告窗口之后才出现的配置或交易，异常归因、政策模拟候选与 CFO 审计回放保持同一时间边界。 diff --git a/document/development/2026-07-16/dev-logs/bugs/standard-adjustment-server-authoritative-amount.md b/document/development/2026-07-16/dev-logs/bugs/standard-adjustment-server-authoritative-amount.md new file mode 100644 index 0000000..6ddcb3c --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/standard-adjustment-server-authoritative-amount.md @@ -0,0 +1,9 @@ +## 修复记录 + +- 18:11：记录 bug 修复：接受住宿职级标准调整时不再信任客户端金额、天数或金额快照。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 未发现 upstream 新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3 feat(approval): add task workflow and waiver decisions`、`28b834ed fix(approval): replay immutable action responses`、`4940ebc4 feat(approval): add safe risk disposition workflow`、`ee88a36b feat(ai): add tenant-safe hierarchical expense learning`、`6bdf65bc feat(expenses): add authoritative pre-review workflow`、`ae3f02c3 feat(expense): add persistent zero-entry receipt association`、`54754b55 feat(ai): add personal expense application memory`、`211f85d9 feat(ai): unify verified expense application workflow`、`5b246307 feat(ai): issue verified application preview decisions`、`a662cfe6 feat(ai): add expense application feedback ledger`、`5ed34c2b feat(expenses): backfill historical claims into expense cases`、`11275e4b fix(migrations): enforce schema ownership safety`、`1347366b feat(expenses): secure timeline and draft events`、`22669a90 feat(expenses): show unified expense event timeline`、`a616b30c fix(expenses): unify AI application submission transaction`、`653eda05 feat(auth): add opaque bearer sessions`、`661990b2 feat(expenses): add transactional expense case events`；这些提交均早于本次未提交修复，本次没有拉取、合并或覆盖共享工作树。 + - 修改：`expense_claim_standard_adjustment.py` 只从已锁定的 `ExpenseClaimItem.item_amount` 读取原始金额，只接受服务端差旅规则计算出的最终可报金额；客户端携带的 `application_days`、`original_amount`、`reimbursable_amount` 仅作为旧界面兼容展示字段，不参与计算。服务端快照新增规则名、规则版本（无发布版本时使用内容指纹）、地点、匹配城市、职级、职级档、天数、每日住宿标准、住宿标准总额及计算指纹。 + - 修改：为标准调整增加 PostgreSQL advisory lock + Claim/Item 行锁和非 PostgreSQL 进程内串行锁；请求支持 `request_id` 幂等键与 `expected_updated_at` 乐观前置条件。相同请求直接重放且不改 `created_at`/计算快照，同请求号改选其他明细会被拒绝；单次调整只替换被选明细的快照，不再误删其他明细已接受的标准调整。 + - 操作：在 `local-x-financial-linux` 容器及 `/tmp/x-financial-server-venv` 中运行定向、接口全量和服务全量测试；运行 scoped Ruff 与 `git diff --check`，未在宿主机运行 Python/pytest，也未修改规则表或其他用户文件。 + - 验证：标准调整定向回归（含非 PostgreSQL 同租户同单据锁竞争）`9 passed`；`test_reimbursement_endpoints.py` 全量 `22 passed`；Scoped Ruff 与 `git diff --check` 通过。`test_expense_claim_service.py` 全量为 `112 passed, 8 failed`，8 个失败均位于既有审批任务配置/旧错误文案断言（直属领导任务、费用申请提交、本人审批、重复退回），不经过标准调整实现；本次新增及关联标准调整用例全部通过。 + - 影响：伪造低原金额、任意可报金额或超长住宿天数不能降低或抬高实际报销额；规则缺失时整次操作失败关闭且不改金额。审批人看到的原额、可报额和差额均可追溯到数据库明细与服务端规则证据，重复点击和并发请求不会重写金额证据。 diff --git a/document/development/2026-07-16/dev-logs/bugs/workbench-ai-runtime-test-contract.md b/document/development/2026-07-16/dev-logs/bugs/workbench-ai-runtime-test-contract.md new file mode 100644 index 0000000..d218e67 --- /dev/null +++ b/document/development/2026-07-16/dev-logs/bugs/workbench-ai-runtime-test-contract.md @@ -0,0 +1,9 @@ +## 修复记录 + +- 19:37：修复 Workbench AI 超大运行时导致职责耦合，以及前端回归测试仍绑定旧单体文件和宿主机缺失 Pillow 的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`；`HEAD..origin/main` 无新提交；本地相对 upstream ahead 17 个既有提交，依次为 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`、`5ed34c2b`、`11275e4b`、`1347366b`、`22669a90`、`a616b30c`、`653eda05`、`661990b2`；本次未合并、覆盖或改写共享工作树中的其他变更。 + - 修改：将会话滚动、流式响应、持久化/重置提取到 `useWorkbenchAiConversationRuntime.js`，把模型意图规划、低置信确认和多任务衔接提取到 `useWorkbenchAiIntentExecution.js`；`usePersonalWorkbenchAiMode.js` 降至 768 行。 + - 修改：Workbench、会话删除、报销关联与快速申请预览测试改为联合审计入口与职责模块；视觉验证复用容器已有 ImageMagick，保留像素和动画断言，不再依赖未安装的 Pillow。 + - 操作：全部 node 测试和构建均在 `local-x-financial-linux` 容器内执行，未修改后端或财务规则 XLSX。 + - 验证：Workbench AI/会话组合 `85 passed`，快速申请预览 `67 passed`；Vite 生产构建成功（2229 modules）；`git diff --check` 通过。 + - 影响：会话清理、详情智能录入、关联门禁和申请预览的回归测试不再因内部职责迁移误报，核心运行时恢复到项目 800 行硬上限内。 diff --git a/document/development/2026-07-16/feature/ai-release-real-telemetry/CONCEPT.md b/document/development/2026-07-16/feature/ai-release-real-telemetry/CONCEPT.md new file mode 100644 index 0000000..5f7b31c --- /dev/null +++ b/document/development/2026-07-16/feature/ai-release-real-telemetry/CONCEPT.md @@ -0,0 +1,333 @@ +# AI 分阶段发布真实遥测 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +把真实 shadow/Canary 规则执行、正负样本盲审真值和服务端保守聚合串成可审计证据链，只有精确率、召回率下界与运行质量同时满足门禁时才允许晋级，越界时自动回滚到稳定版本。 + +## 背景与问题 + +风险规则已经具备 Golden Case、`shadow → canary → active → rolled_back` 状态机、稳定流量路由和自动回滚能力，但线上评测仍有一个关键证据缺口：`ReleaseEvaluationInput` 由外部调用方直接提交 `total`、`failure_count`、`precision` 和 `baseline_precision` 汇总数字，Release Guard 无法证明这些数字来自哪一次真实规则执行、哪一个租户、哪一个 release，也无法证明精度分子和分母来自可信人工结论。 + +现有运行与学习链路的事实边界如下： + +- `ExpenseClaimRiskRuleLoader` 能按租户和稳定路由键选择 stable、shadow 或 Canary 候选版本，并在快照损坏时保守阻断。 +- `evaluate_platform_risk_rules()` 会返回 shadow 候选的 `asset_id / rule_code / rule_version / release_stage / hit / severity`；Canary 命中会进入带版本和阶段的风险 flag，但当前返回值不会持久化成发布样本。 +- `RiskDispositionEvent` 是类型化、租户化、只追加的人工处置事实，`confirm` 和 `false_positive` 可作为正例命中的可信真值来源。 +- `RiskObservationFeedback` 的自由评论、`AIDecisionFeedback` 和 `WorkflowOutcome` 可以支持业务学习，但它们没有同时绑定 `asset_id + release_id + stage + version`，不能直接作为某次发布的精度标签。 +- Release Monitor 的 HMAC 能认证请求来自持有共享密钥的调用方，并限制传输重放；它不能证明请求体中的汇总数字由真实数据库 observation 和人工 label 计算得出。 + +因此，线上门禁必须从“相信外部汇总数字”改为“服务端从只追加事实计算指标”。本能力是 `ai-data-flywheel` 在线质量闭环和 `ai-expense-closed-loop-and-value-proof` 分阶段发布目标的证据层，不替代离线 Golden Case。 + +## 目标与非目标 + +### 目标 + +- [G1] 为每次真实候选规则执行记录租户、资产、release、阶段、版本、规则、命中、基线命中和结构化运行状态。 +- [G2] 将专用发布复核或数据库中可信 `RiskDispositionEvent` 转换成只追加的正例判断；将独立盲审转换成与模型预测语义分离的 `risk_present / risk_absent` 真值。 +- [G3] 使用稳定幂等键处理同一执行或人工动作的安全重放，不同内容复用同一来源时拒绝冲突。 +- [G4] 从真实 observation 和最新可信 label 聚合运行总量、运行失败、候选 precision 和基线 precision。 +- [G5] 未标注候选命中保持 `collecting`，不得把“没有人工结论”当成成功或正确。 +- [G6] 聚合结果可转换成现有 `ReleaseEvaluationInput`，供 shadow/Canary 门禁、自动晋级和自动回滚使用。 +- [G7] 全链路租户隔离、数据脱敏、append-only，并拒绝跨租户和陈旧 release/stage/version 标签。 +- [G8] 对候选未命中人群实施分层盲审：基线命中分歧样本全量复核，其余负样本按不可预测稳定分数随机抽检；证据不足时 recall/FN 仍显式不可用。 +- [G9] 使用抽样漏检率估计总体 FN，并以 Wilson 上界反推保守召回率下界；发布门禁只使用下界，不把点估计冒充确定事实。 + +### 非目标 + +- [NG1] 不用线上遥测替代离线 Golden Case；前者验证真实分布，后者验证覆盖明确预期的回归集合。 +- [NG2] 不把候选未命中直接判为 false negative，也不从“后续未退回”反推规则正确。 +- [NG3] 不保存报销事由、票据内容、人工评论、单号、操作者账号原值或原始风险 payload。 +- [NG4] 不接受客户端、浏览器或普通管理员提交的 precision/recall 作为发布真值。 +- [NG5] 不因为 HMAC 校验通过就信任汇总数字；HMAC 只解决传输来源和重放，不解决数据生成真实性。 +- [NG6] 不让通过质量门禁的规则绕过审批、风险处置或资金动作的人类控制。 +- [NG7] 不把遥测或自动回滚做成绕开共享风险循环、迁移所有权、审批控制或稳定版本保护的旁路。 + +## 用户与场景 + +### 用户 + +1. 风险运营/规则管理员：查看某个 release 的真实样本数、待标注数、误报率和基线比较，决定是否继续采集或人工回滚。 +2. 财务审计/审批人：在既有风险处置或专用发布复核队列中给出类型化确认/误报结论，不接触发布汇总公式。 +3. Release Monitor：只从数据库聚合当前 release，满足证据条件后把服务端计算结果交给 Release Guard。 +4. 平台审计员：按租户、release、版本和来源指纹回放 observation、label、评测与状态转换，不读取业务正文。 + +### 核心场景 + +1. shadow 阶段同时执行 stable 和候选规则；记录候选 hit/miss，并记录同一单据上 stable 是否命中。 +2. 候选或 stable 命中进入人工复核；`confirm` 表示风险事实成立，`false_positive` 表示该次正向命中是误报。 +3. 每条成功 observation 同步决定盲审层：候选命中全量入队、候选未命中但基线命中全量入队、双方均未命中按发布时冻结比例随机入队。 +4. 复核队列混排正负样本，只提供业务单据、规则和业务阶段，不返回 candidate/baseline hit；复核人只回答“存在真实风险/确认无该风险”。 +5. 所有候选未命中样本必须由两个不同复核人给出一致结论；同一人的重复动作不增加法定票数，冲突时继续 collecting。 +6. 候选命中仍有任何未标注项时，聚合状态保持 `collecting`，Release Monitor 不提交通过评测。 +7. 候选正例全部标注后，服务端计算 candidate precision；基线正例也全部标注时才计算 baseline precision。 +8. 负样本达到最小独立复核量后，服务端分别披露实际观察 FN、总体估计 FN、FN 置信上界、recall 点估计和 recall 置信下界。 +9. Canary 路由中的候选 hit、miss 和结构化运行失败继续追加；错误率、precision 或 recall 下界越界时 Release Guard 自动回滚稳定快照。 +10. release 已晋级、回滚或被新 release 替代后，旧 observation/sample/label 仍保留，但不能再接收新标签或作为当前晋级输入。 + +## 功能能力 + +- [C1] 运行样本生产：消费现有 `shadow_evaluations`、Canary/active flag，并提供 manifest 执行循环级 hook 记录 Canary 未命中和结构化失败。 +- [C2] 可信标签：支持认证发布复核动作和数据库中真实 `RiskDispositionEvent`；不接受评论文本作为标签。 +- [C3] 保守聚合：分别计算 observation、completed、runtime failure、candidate hit/labeled/pending、baseline hit/labeled/pending。 +- [C4] 门禁转换：仅 `ready` 聚合可生成 `ReleaseEvaluationInput`；`collecting` 调用转换时明确拒绝。 +- [C5] 证据隔离：运行来源、标签来源、操作者均只保存租户内、带密钥版本的 + HMAC-SHA-256 指纹；原始 claim、事件和账号值不进入遥测表。 +- [C6] 幂等与冲突：同一租户、release、阶段、版本、规则和来源形成稳定键；相同重放复用首次记录，不同载荷冲突。 +- [C7] 历史不可变：observation 与 label 均只追加；标签纠正追加新 label，聚合取最新可信标签，不原地覆盖旧事实。 +- [C8] 盲审抽样：候选正例和候选/基线分歧样本全量入队，其余负样本按发布策略的 `negative_sample_percent` 与 HMAC 来源伪名生成稳定随机分数。 +- [C9] 真值盲化：样本表不保存明文单据 ID，业务来源加密保存；API 不返回 candidate/baseline hit，前端也拒绝接收预测字段。 +- [C10] 保守召回：证据未满足时 `false_negative_count / estimated_false_negative_count / recall / recall_lower_bound` 保持 `null`；满足后分别披露，不用点估计替代下界。 + +## 方案设计 + +### 证据链与自动判定 + +```text +[真实 stable / candidate 执行] + │ + ▼ +[append-only Observation] + tenant + asset + release + stage + version + hit + runtime status + │ + ▼ +[append-only Audit Sample] + 正例全量 + 分歧全量 + 其余负例稳定随机抽样 + │ + ▼ +[盲化可信 Label] + typed disposition / release review / blind release review + │ + ▼ +[服务端 Aggregate] + runtime + precision + baseline + sampled FN + recall lower bound + │ + ┌───────┴────────┐ + │ collecting │ ready + ▼ ▼ +[继续采集/告警] [ReleaseEvaluationInput] + │ + ▼ + [Release Guard 判定] + shadow → canary → active + 或自动 rolled_back +``` + +该链路中的每一层只消费上一层可验证的结构化事实。离线 Golden Case 仍是进入 shadow 前的回归门禁；线上 telemetry 是进入 Canary、active 及运行中回滚的分布证据，两者不能互相替代。 + +### 前端 + +- 发布控制台分别展示运行样本、候选待标注、候选/基线 precision、运行失败、负样本池/抽样/积压、实际 FN、估计 FN、FN 上界、recall 点估计和置信下界。 +- 盲审队列只接收服务端安全字段；`candidate_hit / baseline_hit` 即使误入响应也不会进入页面状态。 +- 复核按钮使用中性业务语义“存在真实风险 / 确认无该风险”，不使用“模型命中/误报”暗示预测；来源单据在新标签页打开并隔离 opener。 +- `null` 指标统一显示“证据不足/暂不可用”，不得渲染成 0；`collecting`、`ready`、`failed/rolled_back` 使用不同状态。 +- `collecting`、`ready`、`failed/rolled_back` 使用不同状态，不把空样本或缺标签显示为 100%。 + +### 后端 + +- `AgentAssetReleaseTelemetryService.record_expense_risk_result()` 可从当前风险评测返回值生产 shadow 样本和 Canary/active 命中样本。 +- `record_manifest_evaluation()` 设计为风险 manifest 执行循环 hook，可记录 Candidate 未命中及 `evaluator_error / artifact_integrity_error / unsupported_evaluator / timeout` 等结构化失败。 +- `record_review_label()` 只接收类型化 label、认证 actor ID 和 request ID；actor 与 request 进入表前被指纹化。 +- `AgentAssetReleaseSamplingService` 在 observation 同一事务内完成分层选择与来源加密；加密失败会连同 observation 一起回滚，避免留下无法复核的孤立事实。 +- `AgentAssetReleaseReviewService` 只查询当前租户、当前 release 的样本，混排后输出去预测队列；发布发起人不可自审，负样本强制两个不同 actor。 +- `agent_asset_release_aggregation` 将精度和召回证据拆开聚合；`agent_asset_release_recall` 使用随机层漏检率与 Wilson 上界生成总体 FN 估计和 recall 下界。 +- `record_risk_disposition_label()` 会重新查询数据库中的同租户 `RiskDispositionEvent` 和 `RiskObservation`，校验 action、claim/rule 来源指纹和版本，不信任调用方提供的人工结论副本。 +- `aggregate()` 只读取同租户、同资产、同 release、同阶段、同版本事实，并验证它仍是资产当前 release。 +- `to_release_evaluation_input()` 只允许 `ready` 聚合转换；未标注、无候选正例或空样本会抛出 collecting 错误。 +- `AgentAssetReleaseMonitor` 与周期调度器只在服务端查询事实并调用上述方法；HTTP 入口只接受签名触发，不再接收外部汇总数字。 +- Agent 资产基础 CRUD/表格/版本接口与风险规则生成、测试、启停和发布子路由分离； + 资产版本只读投影由独立序列化 mixin 承担，Release Guard 只编排状态与持久化， + 阈值归一化和质量门禁计算下沉为无数据库副作用的纯策略模块。 + +### 算法/规则 + +- shadow 同时保留候选与 stable 的命中信息，用同一人工真值分别估计 candidate precision 和 baseline precision。 +- Canary 使用稳定路由键分流；必须在 manifest 执行循环记录候选 miss，否则只从最终 flag 采集会产生“只有命中样本”的选择偏差。 +- candidate 正向命中使用 `confirmed / false_positive` 计算 precision；盲审统一使用 `risk_present / risk_absent` 描述业务真值，再在聚合层规范化，不向复核人暴露预测结论。 +- candidate miss 只有进入服务端抽样表并完成独立双人盲审后才可形成 FN/TN 证据；未抽中的个体不能直接被标签，也不能由“后续无退回”反推正确。 +- 运行失败与业务误报分开：`failure_count` 表示 evaluator/快照/超时等结构化运行失败，`false_positive_count` 只进入 precision。 +- 阶段最小样本数、最大错误率、最低 precision 和最大 precision drop 仍由 `ReleaseGuardPolicy` 统一判定。 +- `ReleaseGuardPolicy.reviewer_quorum` 以 `1..2` 的受控整数随 release 策略固化， + telemetry 只统计不同 actor 指纹的最新票；票数不足或不同复核人结论冲突时保持 + `collecting`，不会把部分意见交给 Release Guard。 +- 新发布默认开启 recall 门禁，并冻结 `negative_sample_percent / negative_min_reviewed / min_recall / recall_confidence_level`；旧 release 未携带开关时保持兼容，不追溯伪造历史抽样事实。 +- recall 门禁只比较 `recall_lower_bound` 与 `min_recall`；点估计再高，只要保守下界不足也不能晋级。明显低 precision 可直接失败，不必等待召回样本凑齐。 + +### 数据 + +#### `agent_asset_release_observations` + +- 身份：`tenant_id / asset_id / release_id / stage / version / rule_code`。 +- 运行事实：`candidate_hit / baseline_hit / runtime_status / failure_code / business_stage`。 +- 脱敏来源：`source_kind / source_fingerprint`；不保存 claim ID、单号或业务正文。 +- 一致性：租户幂等键唯一，保存 payload fingerprint；同一来源不同内容冲突。 + +#### `agent_asset_release_labels` + +- 身份复制：tenant、observation、asset、release、stage、version，并通过复合外键绑定原 observation。 +- 标签：正例判断使用 `confirmed / false_positive`，盲审真值使用 `risk_present / risk_absent`。 +- 来源：`typed_risk_disposition / release_review / blind_release_review`；数据库组合约束禁止标签语义与来源交叉使用。 +- 历史：只追加；纠正写新行，不修改或删除旧标签。 + +#### `agent_asset_release_audit_samples` + +- 身份复制：tenant、observation、asset、release、stage、version，通过复合外键绑定原 observation。 +- 分层：`candidate_positive_census / candidate_disagreement_census / candidate_negative_random`。 +- 抽样事实：保存入样概率和稳定选择分数；同租户 observation 最多一条样本，重放必须匹配 payload fingerprint。 +- 来源保护：原始单据引用使用 SecretBox 加密，表中不保存明文；只有通过租户与复核角色检查的队列读取才解密。 + +三类模型同时具备 ORM 层 UPDATE/DELETE 拒绝。`0018` 建立 observation/label，后继 `0023` 建立 audit sample、扩展标签约束并复用 PostgreSQL append-only 触发器；存在盲审事实或新标签语义时拒绝有损降级。 + +### 权限 + +- 所有写入、读取和聚合以 `tenant_id` 为第一条件；租户绑定资产不允许其他租户观察或标签。 +- 平台共享资产可为不同租户分别保存 observation/label，但各租户样本和 precision 不混算。 +- 标签前重新校验资产当前 `release_id + stage + candidate_version`；旧 release、已晋级阶段或已回滚阶段拒绝新增标签。 +- 类型化处置标签必须来自数据库中真实存在且同租户的 `RiskDispositionEvent`，action 只允许 `confirm / false_positive`。 +- 专用复核队列只允许 `manager` 或 `admin`；租户和 actor 全部来自认证上下文， + 跨租户资产返回 404，非复核角色返回 403，发布发起人自审返回 400。 +- 标签写入必须携带 `X-Request-Id`；新客户端只发送 `risk_present / risk_absent`，旧客户端的 `confirmed / false_positive` 仅在复核 API 边界映射为盲审真值。标签、 + actor 指纹和 request 来源只追加保存，客户端不能覆盖 tenant、release 或 actor。 +- `reviewer_quorum=2` 时必须由两个不同复核人给出相同结论；同一人的重复提交不增加票数， + 冲突结论进入待仲裁状态并继续阻止晋级。 + +### HMAC 与数据真实性边界 + +- HMAC 可以证明传输请求由持有密钥的一方生成、请求在允许时间窗口内且签名未被修改。 +- HMAC 不能证明调用方提交的 `total=100`、`precision=0.99` 真的来自 100 条数据库 observation，也不能证明人工标签存在。 +- 因此 HMAC 只保留为自动 Monitor 的传输认证和防重放手段；指标必须由接收端使用当前数据库 observation/label 重新计算。 +- 最终 Monitor 请求应只携带受控 release 触发信息或聚合作业游标，而非可被签名后照单采用的质量汇总数字。 +- 即使 HMAC 认证失败，也不得影响 stable 规则继续保护业务；应停止晋级、记录安全告警并保持 `collecting`。 + +### 降级策略 + +- 遥测表或写入暂不可用：不阻断已生效 stable 风险规则和报销主流程，但当前 release 不能晋级，状态保持 collecting 并告警。 +- 人工标签迟到：保留 observation，待标签追加后重新聚合；不使用默认正确值填补。 +- 运营端同时展示待标注数量和最早积压时长；超过 24 小时生成结构化逾期告警。 +- 处置事件与 observation 无法安全关联：拒绝标签，不按相似文本、姓名或评论做模糊匹配。 +- baseline 标签不完整：`baseline_precision = null`；候选指标可继续采集，但不能声称已完成可靠基线比较。 +- 负样本未抽中、未完成双人复核或未达到最小复核量：recall、估计 FN 和置信下界保持不可用，继续 collecting 并告警；不会用零填充。 +- 聚合或 Guard 判定越界：按现有冻结快照恢复 stable；回滚不删除候选 observation 和 label。 +- 周期聚合异常形成 `release_aggregation_failed` 告警并隔离到单资产；运行失败率、 + precision 下降、baseline 不可用和自动回滚分别使用独立告警码，稳定版本继续服务。 + +## 算法与公式 + +### 候选精确率 + +```text +candidate_precision = candidate_confirmed / ( + candidate_confirmed + candidate_false_positive +) +``` + +- 分母只包含候选 `candidate_hit=true` 且已有可信最新标签的 observation。 +- 任一候选正向命中仍未标注时，聚合保持 `collecting`，不得将部分 precision 交给 Release Guard 作为通过证据。 + +### 基线精确率 + +```text +baseline_precision = baseline_confirmed / ( + baseline_confirmed + baseline_false_positive +) +``` + +- 只使用同一 shadow 样本上 `baseline_hit=true` 的可信标签。 +- 任一基线正向命中待标注时，baseline precision 显式不可用，不用部分样本制造有利比较。 + +### 运行错误率 + +```text +runtime_error_rate = runtime_failure_count / observed_count +``` + +- `observed_count` 是该 release/stage/version 的真实运行 observation 数。 +- `runtime_failure_count` 只统计结构化执行失败，不把业务误报混成技术错误。 +- 误报通过 precision 体现；运行错误通过 `ReleaseGuardPolicy.max_error_rate` 体现。 + +### Recall 与 false negative + +```text +recall = TP / (TP + FN) +``` + +线上负样本分为两层，不能把抽检样本数直接当总体 FN： + +```text +disagreement_FN = 全量复核(candidate_hit=false, baseline_hit=true)中的真实风险数 +random_FN_rate = 随机盲审层真实风险数 / 已完成双人复核的随机样本数 +estimated_FN = disagreement_FN + random_FN_rate * random_negative_population +recall_point = TP / (TP + estimated_FN) + +random_FN_rate_upper = WilsonUpper(random_FN, reviewed, confidence) +FN_upper = disagreement_FN + random_FN_rate_upper * random_negative_population +recall_lower_bound = TP / (TP + FN_upper) +``` + +- `false_negative_count` 仅表示已完成法定复核样本中实际观察到的 FN，不等于总体 FN。 +- `estimated_false_negative_count` 是总体点估计，`false_negative_upper_bound` 是保守上界，二者必须分字段展示。 +- 随机层存在但尚无已复核样本、仍有抽中样本待审或未达到最小复核量时，上述估计统一保持 `null`。 +- 没有随机负例人群时可使用全量复核的精确 recall；否则发布门禁只消费 `recall_lower_bound`。 +- 离线 Golden Case recall 只证明测试集表现，不能冒充线上 recall；线上抽样也不能替代 Golden Case 的边界覆盖。 + +## 测试方案 + +- 模型：租户/release 复合身份、sample/label 到 observation 复合外键、标签来源组合约束和 append-only。 +- 样本生产：shadow hit/miss、stable baseline hit、Canary hit、manifest 循环 Canary miss 和结构化运行失败。 +- 标签：专用复核、真实 `RiskDispositionEvent`、盲审语义、负样本双人法定票、来源/actor 脱敏。 +- 幂等：相同 observation/label 稳定重放；同一来源不同载荷返回冲突。 +- 租户与时效：跨租户隐藏、陈旧 release/stage/version 拒绝、错误规则/单据来源拒绝。 +- 聚合：无样本、无候选正例、无标签、部分标签、完整标签、baseline 部分标签、运行失败和 precision drop。 +- 运营告警：待标注数量/最早时长、24 小时积压、运行失败率、聚合失败、baseline + 不可用、precision 下降和自动回滚使用去敏结构化告警。 +- 证据边界：未抽中负样本拒绝标签；预测字段不进入队列/API；抽样不足时 recall/FN 为 null，充分时点估计与下界分离。 +- 组合回归：Telemetry 生成的 `ReleaseEvaluationInput` 可被现有 Release Guard 消费，且不改变 shadow/Canary/回滚状态机。 +- PostgreSQL：0018/0023 upgrade/downgrade、复合外键、标签组合约束、数据库 append-only、并发同幂等键单赢家、同 actor 去重和标签/阶段竞争。 +- 所有验证在 `local-x-financial-linux` 容器内执行，单条命令最大 60 秒。 + +## 指标与验收 + +- [A1] 每条线上发布样本可追溯到 tenant、asset、release、stage、version、rule 和来源指纹，且不含业务正文。 +- [A2] 相同运行/标签重放只保留一条事实；不同载荷复用同一来源 100% 拒绝。 +- [A3] 任一候选正向命中未标注时状态为 collecting，不能生成 Release Guard 通过输入。 +- [A4] precision 和 baseline precision 只由真实 observation 与可信类型化标签计算，外部汇总值不作为权威事实。 +- [A5] recall/FN 证据不足时明确 unavailable；充分时同时披露实际 FN、估计 FN、FN 上界、recall 点估计和置信下界，门禁只使用下界。 +- [A6] 跨租户、陈旧 release/stage/version、错误处置来源和纯负样本伪标签均被拒绝。 +- [A7] 质量越界时自动回滚 stable，遥测故障或 HMAC 故障时停止晋级但不关闭既有稳定保护。 +- [A8] PostgreSQL 迁移、append-only、并发、后端组合回归、Ruff 和 `git diff --check` 全部在容器内通过。 + +## 风险与开放问题 + +- 模型注册、真实 manifest hook、类型化处置标签、盲审抽样、服务端聚合、即时 Guard、 + 租户周期调度、发布复核队列和运营控制台已经接通；`0023` 后继迁移与完整 PostgreSQL + 循环仍须完成最终验证后才能关闭本能力。 +- 租户调度器只自动处理租户绑定资产。平台共享资产可以按租户保存隔离样本，但在建设跨租户、加权且可审计的聚合口径前，不能由单租户样本自动回滚全局版本。 +- 线上标签可能集中在高风险或有争议样本，precision 仍可能受人工复核选择偏差影响；控制台必须同时披露 hit、labeled 和 pending 数量。 +- 规则稀有时可能长期没有候选正例；不能为了晋级降低为“零命中等于 100% precision”，需要延长 shadow 或补充经审核 Golden Case。 +- 标签纠正采用追加新事实和“每个 actor 最新票”聚合；数据库索引、复合外键和只追加 + 约束已在 PostgreSQL 验证，同 observation/label 的并发单赢家、阶段晋级竞争和调度器 + advisory leader lease 均有 PostgreSQL 并发验证。 +- HMAC 密钥泄露会让攻击者通过传输认证，但仍不应允许其伪造数据库 observation/label；服务端重算是不可省略的第二道边界。 +- 线上 recall 已有分层抽样、预测盲化、双人复核、冲突保持 collecting、最小样本量和置信下界；仍需用试点数据校准抽样比例、人工一致率与业务风险容忍度，不能把默认阈值当行业通用真理。 + +## 本轮实现记录 + +- 2026-07-16：完成现有 Loader、平台风险评测、shadow_evaluations、风险处置和 AI workflow feedback/outcome 的只读审计，确认 `RiskDispositionEvent` 是当前最可信线上人工标签源，通用学习结果缺少 release 身份不能直接用于门禁。 +- 2026-07-16：新增独立 observation/label 模型与服务，完成脱敏、append-only、稳定幂等、租户/陈旧 release 拒绝、shadow/Canary 样本生产、可信标签和保守聚合。 +- 2026-07-16：独立切片阶段新增遥测测试 7 项，与当时 Release Guard/Runtime 组合共 19 项通过；该阶段留下的共享注册、迁移和运行 hook 已在后续记录中完成。 +- 2026-07-16：完成主模型注册、迁移所有权与 `0018`；一次性 PostgreSQL 17 完整迁移循环、复合外键和数据库 append-only 探针通过。 +- 2026-07-16：真实 shadow/Canary/active manifest 执行已写 observation；类型化风险处置自动追加标签并即时触发 Guard，相同聚合快照幂等复用测试运行，低 precision 或运行失败自动恢复 stable。 +- 2026-07-16：Release Monitor HTTP 改为空触发 + HMAC，禁止调用方提交 precision/total；新增租户级周期调度作为即时监控失败的补偿链。发布组合回归 44 项、调度/风险定向回归 24 项通过。 +- 2026-07-16：完成后端大文件职责拆分：`agent_assets.py` endpoint 降至 714 行、 + `AgentAssetService` 降至 675 行、`AgentAssetReleaseGuardService` 降至 636 行； + 风险规则子路由、资产序列化和发布纯策略分别独立，旧路由路径、公开类型导入和状态机行为保持兼容。 +- 2026-07-16：发布纯策略新增 `reviewer_quorum`（默认 1、范围 1..2）并随 release state 保存， + 专用去敏复核队列按独立 actor 计票，禁止发布人自审，双人同意前或结论冲突时保持 collecting。 +- 2026-07-16：发布控制台展示真实样本、命中、待审、最早积压时长、运行失败率、 + candidate/baseline precision 和 recall 不可用；新增积压、超时、运行故障、聚合失败、 + precision 下降、baseline 不可用和自动回滚结构化告警。 +- 2026-07-16：新增 append-only 盲审样本、正例/分歧全量与其余负例稳定随机抽样；来源引用加密保存，observation 与 sample 同事务写入，保护失败整体回滚。 +- 2026-07-16：发布复核队列改为预测盲化混排，负样本强制两个不同复核人；新增实际/估计 FN、Wilson FN 上界、recall 点估计与保守下界，Release Guard 只消费下界。 +- 2026-07-16：前端拒绝 prediction hit 字段，使用中性业务真值动作，并显示负样本池、抽样进度、积压及置信方法；证据缺失保持不可用，不渲染为零。 +- 2026-07-16：新增 `0023` 后继迁移和标签来源组合约束；最终 PostgreSQL 全链升级/降级、并发和全量回归结果待验证后回填。 diff --git a/document/development/2026-07-16/feature/ai-release-real-telemetry/TODO.md b/document/development/2026-07-16/feature/ai-release-real-telemetry/TODO.md new file mode 100644 index 0000000..539f145 --- /dev/null +++ b/document/development/2026-07-16/feature/ai-release-real-telemetry/TODO.md @@ -0,0 +1,120 @@ +# AI 分阶段发布真实遥测 开发 TODO + +更新时间：2026-07-17 + +## 使用规则 + +- 每项必须回链 `CONCEPT.md` 对应章节；没有代码、迁移、接口或容器证据不得勾选。 +- observation、label、聚合和 Release Guard 输入必须按证据层分开，不能用 HMAC 请求或客户端汇总值替代数据库事实。 +- `collecting` 不得包装成通过；recall/false-negative 没有达到独立盲审证据阈值时必须保持 unavailable。 +- 所有后端、迁移和并发验证只在 `local-x-financial-linux` 容器内执行，单条命令最长 60 秒。 + +## 1. 调研与边界 + +- [x] [CONCEPT: 背景与问题] 审计 Loader、平台风险评测、shadow_evaluations、风险处置和 AI workflow feedback/outcome 链路，确认线上发布评测缺少持久真实样本。 + 证据：`expense_claim_risk_rule_loader.py`、`expense_claim_platform_risk.py`、`risk_dispositions.py`、`expense_workflow_learning.py`、`ai_learning.py` 只读审计。 +- [x] [CONCEPT: 背景与问题] 确认 `RiskDispositionEvent confirm/false_positive` 是当前可绑定风险命中的可信人工结论，通用 feedback/outcome 缺少完整 release 身份。 + 证据：`risk_disposition.py`、`risk_dispositions.py` 与 `agent_asset_release_telemetry.py` 的可信事件查询和关联校验。 +- [x] [CONCEPT: HMAC 与数据真实性边界] 冻结 HMAC 只负责传输认证、防篡改和防重放，不证明汇总指标的数据真实性。 + 证据：`CONCEPT.md`“HMAC 与数据真实性边界”；现有 `agent_asset_release_monitor_auth.py` 与 `ReleaseEvaluationInput` 契约对照审计。 +- [x] [CONCEPT: 目标与非目标] 明确未标注 collecting、敏感数据不落遥测表、线上 recall/FN 证据不足不可用。 + 证据：`CONCEPT.md`“目标与非目标”“Recall 与 false negative”。 + +## 2. 契约与设计 + +- [x] [CONCEPT: 数据] 定义 observation 与 label 的 tenant/asset/release/stage/version 复合身份、稳定幂等键和只追加边界。 + 证据：`server/src/app/models/agent_asset_release_telemetry.py`。 +- [x] [CONCEPT: 证据链与自动判定] 定义 observation → trusted label → aggregate → ReleaseEvaluationInput → Release Guard 的证据链。 + 证据：`CONCEPT.md`“证据链与自动判定”、`ReleaseTelemetryAggregate.to_release_evaluation_input()`。 +- [x] [CONCEPT: 算法/规则] 分开定义运行失败、业务误报、candidate precision、baseline precision 和缺失负样本真值。 + 证据：`agent_asset_release_telemetry.py` 的 `aggregate()`；`test_agent_asset_release_telemetry.py` 的 collecting、baseline 和 FN 不可用断言。 +- [x] [CONCEPT: 权限] 冻结专用发布复核接口的角色矩阵、双人复核阈值和跨租户 HTTP 错误契约。 + 证据：`require_rule_reviewer_user` 只允许 manager/admin；跨租户 404、非角色 403、 + 发布人自审 400；`reviewer_quorum` 限制 1..2 且按不同 actor 指纹计票。 + +## 3. 独立模型与服务 + +- [x] [CONCEPT: 数据] 新增 append-only observation/label ORM 模型、复合租户/release 外键和更新/删除拒绝。 + 证据：`server/src/app/models/agent_asset_release_telemetry.py`。 +- [x] [CONCEPT: 后端] 实现真实 shadow 结果、Canary/active 命中和 manifest 执行循环样本生产器。 + 证据：`AgentAssetReleaseTelemetryService.record_expense_risk_result()`、`record_manifest_evaluation()`。 +- [x] [CONCEPT: 后端] 实现专用发布复核标签和可信 `RiskDispositionEvent` 标签转换，不接收自由评论。 + 证据：`record_review_label()`、`record_risk_disposition_label()`。 +- [x] [CONCEPT: 后端] 实现租户隔离、陈旧 release/stage/version 拒绝、来源关联校验和稳定幂等冲突。 + 证据：`_require_current_release()`、`_observation_replay()`、`_label_replay()` 及对应测试。 +- [x] [CONCEPT: 算法与公式] 实现保守聚合；未标注候选命中保持 collecting，基线标签不完整时 baseline precision 不可用。 + 证据：`ReleaseTelemetryAggregate`、`aggregate()`、`to_release_evaluation_input()`。 +- [x] [CONCEPT: 数据] 实现 claim、事件、actor 来源指纹化，不保存业务正文、评论和账号原值。 + 证据：模型无自由文本业务字段；`_fingerprint()`；脱敏测试断言。 + +## 4. 共享注册、迁移与运行接入 + +- [x] [CONCEPT: 数据] 在 `db/base.py` 与 `models/__init__.py` 注册 `AgentAssetReleaseObservation` 和 `AgentAssetReleaseLabel`，保证主应用 metadata 与迁移所有权检查可见。 + 证据：`db/base.py`、`models/__init__.py`、`schema_ownership.py`、`migration_preflight.py` 已登记两张迁移自有表；前置检查 77 项通过。 +- [x] [CONCEPT: 数据] 新增后继 `20260716_0018` Alembic 迁移，创建两张表、复合租户/release 约束、检查约束、索引和数据库级 append-only UPDATE/DELETE 触发器。 + 证据：`20260716_0018_agent_asset_release_telemetry.py`；一次性 PostgreSQL 17 完整升级/降级循环 1 项通过，静态迁移/schema owner 回归 112 项通过。 +- [x] [CONCEPT: 算法/规则] 在真实候选 manifest 执行循环接入 `record_manifest_evaluation()`，完整记录 shadow/Canary hit、miss 和结构化执行失败。 + 证据：`expense_claim_platform_risk.py`、`expense_claim_release_telemetry.py`；`test_agent_asset_release_runtime.py` 覆盖 shadow、Canary、active 和损坏快照，`test_agent_asset_release_telemetry.py` 覆盖结构化运行失败。 +- [x] [CONCEPT: 后端] 在类型化风险处置事务接入 release label，按同租户、同 claim/rule、当前 release 精确关联 observation；关联失败保守拒绝。 + 证据：`agent_asset_release_disposition_labels.py`、`risk_disposition_release_sync.py`；风险确认/误报会追加标签，误报越界自动回滚 stable。 +- [x] [CONCEPT: 后端] 将现有 Release Monitor 从“提交外部汇总数字”改为“触发服务端聚合”，只在 aggregate ready 时构造 `ReleaseEvaluationInput`。 + 证据：`AgentAssetReleaseMonitor.evaluate_current()` 只接受 release 身份；HTTP body 为禁止额外字段的空触发契约，真实聚合未 ready 时不调用 Guard。 +- [x] [CONCEPT: HMAC 与数据真实性边界] 保留 HMAC 作为 Monitor 传输认证，但禁止签名请求覆盖数据库聚合结果，并补充签名通过但汇总伪造的反向测试。 + 证据：`agent_asset_releases.py`、`AgentAssetReleaseMonitorTriggerWrite`；带有效签名的伪造 `precision/total` 请求仍返回 422。 +- [x] [CONCEPT: 降级策略] 遥测持久化或聚合失败时保持 stable 规则、停止晋级并记录结构化告警，不让观测故障阻断正常报销。 + 证据：`ExpenseClaimReleaseTelemetryRecorder`、`risk_disposition_release_sync.py` 将遥测/标签/监控故障隔离并记录日志；未获得 ready 真实聚合不会写 passed，也不会改变 stable 路由。 +- [x] [CONCEPT: 后端] 按职责拆分 Agent 资产接口、版本只读投影和 Release Guard 纯策略计算，保持公开 API 与状态机行为稳定。 + 证据：`agent_assets.py` endpoint 714 行、`agent_asset_risk_rules.py` 534 行、`agent_assets.py` service 675 行、`agent_asset_serialization.py` 217 行、`agent_asset_release_guard.py` 636 行、`agent_asset_release_policy.py` 201 行，相关核心文件均低于 800 行；纯策略模块保存范围为 1..2 的 `reviewer_quorum`，第 5 节已完成复核执行与运营闭环。 + +## 5. 自动聚合、告警与运营闭环 + +- [x] [CONCEPT: 证据链与自动判定] 实现按 tenant/asset/release/stage/version 的周期聚合作业与幂等快照。 + 证据：`agent_asset_release_scheduler.py` 按租户有界扫描；`AgentAssetReleaseMonitor._existing_evaluation()` 复用相同 release 聚合快照，不重复生成测试运行。 +- [x] [CONCEPT: 证据链与自动判定] aggregate ready 后自动调用 Release Guard；collecting 只更新采集状态，不写虚假 passed test run。 + 证据：人工标签提交后即时触发 monitor，后台 scheduler 提供失败补偿；`test_agent_asset_release_monitor.py` 覆盖 collecting 不调用 Guard、低 precision/运行失败自动回滚与相同快照幂等。 +- [x] [CONCEPT: 降级策略] 增加待标注数量/时长、运行失败率、precision 下降、baseline 不可用、聚合失败和自动回滚告警。 + 证据：`agent_asset_release_alerts.py`、`AgentAssetReleaseMonitor._metrics()`；24 小时 + 待审逾期和单资产聚合失败均返回结构化告警，定向 Monitor/Telemetry/Runtime 29 项通过。 +- [x] [CONCEPT: 前端] 在发布控制台展示 observed/hit/labeled/pending、候选/基线 precision、运行失败和召回证据。 + 证据：`AuditReleaseMonitorPanel.vue` 展示负样本池、抽样进度、积压、实际/估计 FN、FN 上界、recall 点估计/下界/置信方法；空值不渲染为零。 +- [x] [CONCEPT: 权限] 实现认证发布复核队列、操作审计和需要时的双人复核，不允许规则发布人独自伪造所有标签。 + 证据：`agent_asset_release_review.py`、`agent_asset_release_label_votes.py`、专用 GET/POST + API；`X-Request-Id`、append-only label、actor HMAC 指纹、发布人隔离和独立双人同意均有测试。 +- [x] [CONCEPT: Recall 与 false negative] 实现独立分层负样本抽样、预测盲化、双人标注、冲突保持 collecting 和保守召回估计。 + 证据：`agent_asset_release_sampling.py`、`agent_asset_release_review.py`、 + `agent_asset_release_aggregation.py`、`agent_asset_release_recall.py`；前端只发送 + `risk_present / risk_absent`，负样本要求两个不同 actor，门禁只读取 recall 下界。 +- [x] [CONCEPT: 数据] 完成 `0023` audit sample 迁移注册、数据库标签组合约束、append-only 触发器和无损降级验证。 + 证据：`20260716_0023_agent_asset_release_blind_audit.py`、`release_telemetry_migration_assertions.py`；fresh PostgreSQL 完整迁移链、约束、触发器及降级/再升级均通过。 + +## 6. 测试与验证 + +- [x] [CONCEPT: 测试方案] 独立模型/服务测试覆盖 shadow、Canary、baseline、真实 typed disposition、脱敏、幂等、租户、陈旧 release、append-only、collecting 和 FN 不可用。 + 证据：容器内 `test_agent_asset_release_telemetry.py` 7 项通过。 +- [x] [CONCEPT: 测试方案] 与现有 Release Guard 和 Runtime 组合回归通过。 + 证据：容器内 `test_agent_asset_release_guard.py`、`test_agent_asset_release_runtime.py`、`test_agent_asset_release_telemetry.py` 共 19 项通过；Ruff 通过，`git diff --check` 通过。 +- [x] [CONCEPT: 测试方案] 新增 0018 upgrade/downgrade、schema owner、复合外键和数据库 append-only PostgreSQL 验证。 + 证据：一次性 `pgvector/pgvector:pg17` 数据库中完整迁移循环 1 项通过；迁移运行探针验证复合租户外键、幂等唯一约束和两张表的数据库级 UPDATE/DELETE 拒绝。 +- [x] [CONCEPT: 测试方案] 新增 PostgreSQL 并发同 observation、同 label、标签与阶段晋级竞争和幂等冲突测试。 + 证据：一次性 PostgreSQL 17 中 `test_agent_asset_release_telemetry_concurrency_postgres.py` 4 项通过；相同重放只保留一条，不同载荷单赢家，阶段转换持锁后旧标签保守拒绝。 +- [x] [CONCEPT: 测试方案] 验证 audit sample 并发单赢家、同 actor 重复不增加负样本法定票、第二独立 actor 完成双人复核，以及预测字段不进入前端队列状态。 + 证据：`test_agent_asset_release_telemetry_concurrency_postgres.py` 覆盖单赢家和独立双人票；`test_agent_asset_release_telemetry.py` 与 `agent-release-monitor-panel.test.mjs` 验证队列不暴露 candidate/baseline 命中预测。 +- [x] [CONCEPT: 测试方案] 跑通真实风险循环 → observation → disposition label → aggregate → Release Guard 回滚端到端。 + 证据：`test_agent_asset_release_runtime.py` 与 `test_risk_dispositions.py` 覆盖真实执行样本、类型化处置标签、服务端聚合、低 precision 自动恢复 stable；发布组合回归 44 项通过。 +- [x] [CONCEPT: 测试方案] 验证职责拆分后的资产服务、风险规则子路由、Release Guard/Runtime、Monitor/Scheduler/Telemetry 和 API schema 兼容性。 + 证据：容器内资产服务 27 项、Guard/Runtime 13 项、Monitor/Scheduler/Telemetry 22 项、风险规则生成/解释 30 项、修订/反馈 14 项通过；迁移后的既有 publish HTTP 防绕过用例通过，Ruff 与 `git diff --check` 通过。 +- [x] [CONCEPT: 指标与验收] 验证 HMAC 失败、遥测库失败、标签积压和聚合失败均停止晋级但不破坏 stable 业务保护。 + 证据：`test_agent_asset_release_runtime.py` 覆盖签名失败；`test_agent_asset_release_monitor.py` 覆盖 collecting、积压和聚合失败；`ExpenseClaimReleaseTelemetryRecorder` 隔离遥测持久化异常，stable 报销路径继续服务。 +- [x] [CONCEPT: 指标与验收] 完成相关后端全量回归、Ruff、迁移完整循环和 `git diff --check`，逐项回填 A1-A8 证据。 + 证据：遥测组合 45 项通过；fresh PostgreSQL 总探针 `87 passed / 0 skipped / 0 failed`，其中发布遥测并发 7 项；Web 全量 815 项及 Vite build 通过；新增 Python 文件 Ruff 与 `git diff --check` 通过。全仓既有 Ruff 基线债不伪装为本轮新增错误。 + +## 7. 文档收尾 + +- [x] [CONCEPT: 本轮实现记录] 新建独立 CONCEPT/TODO，记录证据边界、已实现切片和共享集成缺口。 + 证据：`document/development/2026-07-16/feature/ai-release-real-telemetry/CONCEPT.md`、`TODO.md`。 +- [x] [CONCEPT: 风险与开放问题] 共享集成完成后回填 0018、运行 hook、自动作业、PostgreSQL 和端到端证据。 + 证据：本 TODO 第 4-6 节与 `CONCEPT.md`“本轮实现记录”已回填；发布面板已展示积压、失败、precision 和 recall，生产运营效果由下一条真实流量验收单独保留。 +- [x] [CONCEPT: 指标与验收] 与上位 AI 闭环 TODO 对齐代码与容器验证状态。 + 证据：`document/development/2026-07-13/feature/ai-expense-closed-loop-and-value-proof/TODO.md` 已回填 Golden、Canary、盲审和回滚的工程证据。 +- [ ] [CONCEPT: 指标与验收] 使用生产真实流量、独立复核样本和试点阈值验证 Golden/Canary/自动回滚运营效果。 + 证据要求：目标企业生产 observation/label、盲审样本、阈值签字和真实回滚演练；本地测试不得替代。 diff --git a/document/development/2026-07-16/feature/commercial-metering-and-roi/CONCEPT.md b/document/development/2026-07-16/feature/commercial-metering-and-roi/CONCEPT.md new file mode 100644 index 0000000..5252f92 --- /dev/null +++ b/document/development/2026-07-16/feature/commercial-metering-and-roi/CONCEPT.md @@ -0,0 +1,237 @@ +# 商业计量、客户 ROI 与可持续定价 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +把套餐、订阅、权益、真实用量、内部成本、客户确认价值和平台毛利拆成可审计事实，并只在成本与价值证据同时成立时给出可持续定价走廊。 + +## 背景与问题 + +- 平台要成为可长期经营的产品，既要证明客户省了钱，也要知道每个租户消耗了多少 OCR、AI、存储、连接器和支持成本。 +- “风险金额”“预计节省”“流程耗时”不能直接作为客户 ROI；平台收入、平台内部成本和客户价值也不能混在一个指标里。 +- 只有套餐创建而没有暂停、取消、历史查询和配额硬门禁，商业后台无法真正运营。 +- 仅在执行前读取 `SUM(usage)` 再放行无法抵抗并发；两个工具可同时看到剩余额度并一起执行，事后计量再准确也已形成不可逆超卖。 +- 固定拍一个价格无法适配客户规模、真实成本和价值覆盖。需要先计算平台可持续下限，再计算客户价值可接受上限；没有交集时不能强行报价。 +- 试点期通常缺少 30/90 天真实成本和财务确认收益，因此系统必须显示“采集中”，而不是用 mock 或估计值伪造单位经济性。 + +本能力是 `2026-07-13/feature/ai-expense-closed-loop-and-value-proof` 中商业模式与价值证明部分的实施拆分。 + +## 目标与非目标 + +### 目标 + +- [G1] 建立租户隔离、版本化的套餐、订阅和权益配置。 +- [G2] 建立追加式用量与内部成本事实，支持幂等、冲回、配额和并发门禁。 +- [G3] 把商业权益门禁与安全门禁做收紧式合并，付费不能绕过高风险人工审核。 +- [G4] 严格分开客户收费、内部成本、平台贡献毛利、财务确认现金节省、客户 ROI 和工时价值。 +- [G5] 支持套餐/订阅/权益生命周期、历史查询、用量成本查询和数据质量状态。 +- [G6] 根据真实成本和财务确认节省给出基础费下限、价值上限和可封顶成功费，不自动改合同。 +- [G7] 形成“试点采集 → 基础订阅 → 基础费 + 封顶成功费”的可验证商业演进路径。 +- [G8] 用不可变账期承载收费、用量、成本与配额历史，并对商业配置和自动续期保留脱敏追加式审计。 + +### 非目标 + +- [NG1] 不在代码中硬编码某个客户的最终价格、税率、折扣或合同条款。 +- [NG2] 不把风险暴露、预计机会、未确认结果、未锁定汇率或未经客户认可的工时估值计入价值定价。 +- [NG3] 不让套餐或配额放宽审批、风险、租户和人工确认门禁。 +- [NG4] 不自建开票、税务、收款或第三方订阅扣费网络；只保存受控外部订阅引用。 +- [NG5] 不跨币种直接求和，也不在缺少同币种成本/价值时生成综合 ROI。 +- [NG6] 不把定价场景结果自动写成生效套餐，最终合同仍需平台商业负责人审批。 + +## 用户与场景 + +### 用户 + +1. 平台商业管理员：维护套餐版本、订阅、权益、状态和定价场景。 +2. 平台运营/财务：查看用量、成本、毛利和异常数据质量。 +3. 客户 CFO/财务负责人：查看自己租户的当前套餐、用量、配额和客户价值口径。 +4. 产品运行时：在执行 OCR、AI 或连接器能力前检查配额，并写入真实用量。 + +### 核心场景 + +1. 试点客户先配置 `pilot` 套餐和明确的合同周期，开始采集真实用量、成本与已确认价值。 +2. 运行时检查某项权益；只有商业配额允许且安全决策为 allow 时才最终允许。 +3. 同一用量事件重试返回首次结果；不同内容复用幂等键返回冲突。 +4. 客户暂停、逾期、取消或过期时，商业权益即时失败关闭，历史用量和成本不被删除。 +5. 商业负责人选择 90 天窗口，输入目标贡献毛利率和客户最大价值分享比例，系统按币种输出定价走廊。 +6. 成本下限高于价值上限时，系统建议先优化单位经济性或扩大可信价值，不生成强行报价。 + +## 功能能力 + +- [C1] 套餐版本：subscription、usage、hybrid、pilot、custom，支持生效区间和旧版本退役。 +- [C2] 订阅快照：合同周期、计费周期、席位、基础费、外部订阅引用和状态历史。 +- [C3] 权益与配额：feature、metered、unlimited，包含量、硬上限、重置周期和超额策略。 +- [C4] 用量事实：usage、credit、adjustment、reversal，保存主体、来源、correlation 和首次请求指纹。 +- [C5] 成本事实：AI、OCR、存储、连接器、支持、实施、基础设施、支付等分类及汇率快照。 +- [C6] 商业分析：收费、成本、贡献毛利、确认节省、客户 ROI 和工时价值分账展示。 +- [C7] 定价走廊：最低可持续收费、最高价值对齐收费、最大成功费和证据状态。 +- [C8] 生命周期与查询：暂停、恢复、逾期、取消、过期，以及套餐/订阅/权益/用量/成本历史。 +- [C9] 账期与审计：月/季/年自动续期、合同边界失败关闭、账期历史和商业管理追加审计。 + +## 方案设计 + +### 前端 + +- 商业工作台分为“当前账户”“套餐与订阅”“权益与配额”“用量与成本”“价值与定价”五块。 +- 客户 ROI 与平台贡献毛利必须使用不同卡片、不同说明，不允许用一个“综合收益”混合展示。 +- 多币种按币种分行；缺成本、缺确认价值、仅有试点数据和证据冲突使用不同状态。 +- 暂停、取消和定价场景均需要确认；终态操作明确提示不能原地恢复。 +- 普通客户财务只能查看本租户账户；平台级配置、成本、毛利和定价仅平台管理员可见。 + +### 后端 + +- `CommercialAdminService` 管理套餐、订阅、权益和状态转换。 +- `CommercialBillingPeriodService` 签发和定位不可变账期；用量、成本和运行时预占必须绑定真实账期编号。 +- `CommercialSubscriptionRolloverService` 在订阅行锁内按月/季/年边界幂等续期；合同制或跨越 `ends_at` 时失败关闭。 +- `CommercialRolloverScheduler` 以 PostgreSQL advisory lock 选举单一执行者，再按订阅行锁串行签发到期账期。 +- `CommercialAdminAuditService` 只记录字段白名单快照，排除合同正文、配置、外部订阅编号、元数据和凭证类字段。 +- `CommercialQueryService` 负责租户范围内历史与追加事实查询。 +- `CommercialEntitlementService` 计算配额和商业/安全合并门禁。 +- `CommercialMeteringService` 写追加式用量和成本、执行幂等与冲回。 +- `CommercialRuntimeReservationService` 在订阅/权益锁内完成执行前额度预占，管理 reserved、committed、released、expired、reconciliation_required 和 committed_reconciliation_required 状态。 +- `CommercialRuntimeBridge` 把可信 AgentRun、中央工具执行、真实 AgentToolCall 和商业事实接成预占—执行—结算链；未配置商业计量时保持兼容。 +- `CommercialRuntimeReconciler` 只根据可验证的工具/运行终态补偿过期预占，不按超时猜测业务是否发生。 +- `CommercialAnalyticsService` 按窗口和币种分账聚合，不伪造缺失数据。 +- `CommercialPricingService` 只读分析真实成本和确认节省，输出价格区间，不写套餐。 +- HTTP 管理入口仅平台管理员可用；租户账户读取仅 finance、executive 或平台管理员可用。 + +### 算法/规则 + +- 用量硬配额在数据库锁内计算，最终用量超过限制时整个写入失败。 +- 真实工具执行前按 `已用量 + 有效预占 + 本次预占 <= 硬上限` 原子判断；成功时真实量不得超过预占，失败/阻断只释放预占，不写用量或成本。 +- call 基准可直接预占一次；token、duration 等变量基准必须由执行器声明并强制最大量。缺少可信 hard max 时拒绝执行，不用正文长度或估算值代替。 +- 已有相同 tool call 的预占请求按指纹稳定重放；真实 AgentToolCall 的用量/成本继续使用追加式幂等键。用量已写入但成本失败时进入 `committed_reconciliation_required`，重试只补缺失成本，不重复占用额度或追加用量。 +- 权益已有用量后，配额、定价和有效期不可回改，只允许暂停/恢复；结构变化必须新建订阅版本。 +- 客户 ROI 只使用财务确认 canonical 现金节省与客户收费。 +- 贡献毛利只使用平台收费与内部成本，不混入客户节省。 +- 定价只在相同币种内计算，并保留成本/价值证据状态。 + +### 数据 + +- `tenant_commercial_plans`：租户、套餐编码、版本、价格模型、基础费、币种、生效期和合同条款摘要。 +- `tenant_subscriptions`：租户、套餐、周期、基础费快照、状态、席位、外部引用和版本。 +- `commercial_entitlements`：租户、订阅、权益键、计量键、配额、状态和有效期。 +- `usage_meter_events`：追加式用量、冲回引用、幂等键、请求指纹和关联链。 +- `commercial_cost_events`：追加式内部成本、原币/报告币、汇率、分摊键和冲回引用。 +- `commercial_runtime_reservations`：工具执行前的可变运营占位，保存 tenant/subscription/entitlement/run/tool call、基准、预占量、真实量、周期、配置快照、状态和补偿原因；它参与配额但不是客户用量事实。 +- `commercial_billing_periods`：不可变账期签发事实，保存订阅/套餐引用、窗口、顺序、币种、基础费、席位、计价模式和来源快照；PostgreSQL 禁止更新和删除。 +- `commercial_admin_events`：套餐、订阅、权益、账期和续期动作的脱敏追加审计，保存 tenant、actor、`X-Request-Id`、原因、动作、资源版本和白名单 before/after。 +- `usage_meter_events.period_key` 表示真实账期键，`quota_period_key` 独立表示权益重置周期；成本与运行时预占同样通过 `billing_period_id` 绑定账期，避免订阅当前周期滚动后污染历史。 +- 事实表在 PostgreSQL 使用触发器禁止 UPDATE/DELETE，租户复合外键防止跨租户引用。 + +运行时占位状态如下： + +```text +reserved -> committed # 成功且 actual <= reserved +reserved -> released # 工具失败或执行前阻断 +reserved -> expired # 运行已终止且没有真实工具调用 +reserved -> reconciliation_required # 已发生调用但缺少执行前预占等人工补偿场景 +committed -> committed_reconciliation_required -> committed + # 用量已提交、成本等后续事实失败，幂等补齐后恢复 +``` + +过期但运行仍在进行、运行记录缺失或工具终态不确定时继续保留额度，不允许补偿器仅因 TTL 到期释放后造成超卖。 +`committed_reconciliation_required` 已有真实用量事实，因此不再计入有效预占；配额只消费一次，同时保留明确的待补偿队列。 + +### 权限 + +- 平台管理员可配置所有租户商业账户，但商业配置不能授予财务确认或风险审批能力。 +- finance/executive 只读本租户当前账户，不读取平台内部成本或其他租户数据。 +- manager、employee 默认无商业账户与商业分析权限。 +- 所有管理接口从认证上下文判断平台管理员，目标租户来自受控路径参数。 + +### 降级策略 + +- 无订阅：返回 unavailable 和明确说明，不自动赠送无限权益。 +- 订阅非 active/trialing：配额保留展示但最终消费失败关闭。 +- 缺成本：贡献毛利和可持续价格下限不可用。 +- 缺财务确认价值：客户 ROI、价值上限和成功费不可用，建议继续试点采集。 +- 多币种缺少共同币种：分别展示，禁止跨币种净额。 +- 并发或幂等冲突：返回 409，不覆盖首次事实。 +- 生产中央工具路径未配置 runtime meter：兼容执行且不生成商业事实；配置只在当前订阅周期和权益有效期内参与门禁，历史过期配置不会误触发 enforcement。 +- 已发生的旧直接工具路径缺少执行前预占：不补写为正常用量，持久化 `reconciliation_required` 并冻结对应容量，等待受控补偿；即使当前合同已暂停，也保留其唯一可验证的租户、订阅和权益归属。 +- 用量已追加但内部成本写入失败：持久化 `committed_reconciliation_required`，配额以真实用量为准且预占归零；按相同 tool call 重试只补成本，完成后回到 committed。 +- 自动续期只处理 `trialing/active + auto_renew`；月、季、年按自然月边界滚动。合同制、缺少可推导边界或下一完整账期越过 `ends_at` 时失败关闭，不创建部分账期或猜测续约。 +- 调度器即使发生重复扫描或多进程竞争，也先获取 leader lease，再锁定订阅；账期窗口和幂等键的唯一约束保证同一周期最多签发一次。数据库触发器还会按租户与订阅获取事务级 advisory lock，并拒绝任何半开区间重叠账期，防止绕过服务层直接写入破坏时间线。 + +## 算法与公式 + +### 客户 ROI + +```text +customer_roi = (verified_cash_savings - customer_charges) / customer_charges +``` + +- `verified_cash_savings` 只包含独立财务确认、canonical、已计冲回的现金节省。 +- `customer_charges` 来自订阅基础费快照和有可信同币种单价的用量计费。 +- 分母必须大于 0；否则状态为 unavailable。 + +### 平台贡献毛利 + +```text +contribution_margin = customer_charges - internal_costs +contribution_margin_rate = contribution_margin / customer_charges +``` + +- 内部成本来自追加式成本账本，不使用估算页面数字。 + +### 可持续定价走廊 + +```text +minimum_sustainable_charge = internal_costs / (1 - target_margin_rate) +maximum_value_aligned_charge = verified_cash_savings * max_value_share +maximum_success_fee = max(0, maximum_value_aligned_charge - minimum_sustainable_charge) +``` + +- 当 `minimum_sustainable_charge <= maximum_value_aligned_charge` 时，建议 hybrid：基础费不低于成本下限，成功费封顶为剩余价值空间。 +- 只有成本证据时建议 subscription；成本与价值都不足时建议 pilot_collecting。 +- 成本下限高于价值上限时建议 optimize_unit_economics，不自动提高客户报价。 + +## 测试方案 + +- 模型/迁移：租户复合外键、状态检查、金额符号、冲回引用和 append-only。 +- 服务：套餐版本、订阅终态、权益不可回改、配额硬门禁、幂等、冲回和多币种。 +- 权限：平台管理员、租户财务、manager、employee 与跨租户访问。 +- 分析：收费/成本/节省/毛利/ROI 分账，缺证据状态和 `as_of` 回放。 +- 定价：可行区间、成本高于价值、仅成本、完全无证据和多币种。 +- 前端：状态、权限、操作确认、空态、错误态、币种分组与生产构建。 +- PostgreSQL：并发配额、同幂等键、成本冲回单赢家和迁移完整性。 +- PostgreSQL 账期：0020→0021→0020 升降级、账期/审计 UPDATE/DELETE 拒绝、重叠账期 INSERT 拒绝、历史事实账期绑定和双线程续期单赢家。 +- 运行时预占：无配置兼容、成功结算、失败释放、变量 hard max、真实量超预占、幂等重放、历史配置隔离、直接路径补偿和过期补偿。 +- 所有命令在 `local-x-financial-linux` 容器内执行，单次最长 60 秒。 + +## 指标与验收 + +- [A1] 每个商业消费可追溯到租户、订阅、权益、用量事件、来源和 correlation。 +- [A2] 并发不能突破硬配额；重复事件稳定重放，冲突内容被拒绝。 +- [A3] 付费状态不能绕过安全或人工审核门禁。 +- [A4] 客户 ROI、平台毛利、客户节省和工时价值在 API/UI 中不混算。 +- [A5] 暂停、恢复、取消、过期和历史查询可操作，终态不可原地复活。 +- [A6] 定价场景只使用真实同币种成本与确认节省；证据不足时不输出虚假价格。 +- [A7] 相关后端、PostgreSQL、前端、构建、Ruff 与迁移验证在容器内通过。 + +## 风险与开放问题 + +- 真实计费仍需把 OCR、LLM、存储、连接器和支持运行事件自动接入用量/成本账本；手工管理员写入只能用于校验，不是最终生产采集。 +- 中央 Orchestrator 工具已经执行前预占；绕过中央执行器的其他生产入口仍须逐一迁移到 permit 契约，当前只会形成可见补偿积压，不会伪装成正常计量。 +- 合同制自动续期仍不推断新合同窗口；管理员或外部订阅连接器必须先提供显式续约事实，再创建后继合同/订阅版本。 +- 首个客户的实际套餐金额、席位、包含量、毛利目标、价值分享比例和折扣需要商业负责人确认。 +- 发票、税率、回款、坏账、渠道分成与收入确认尚未接入，当前 `customer_charges` 是合同/用量计费基准，不等同已收现金。 +- 价值分享合同必须定义基线、排除项、冲回、确认人、封顶和争议期。 +- 工时价值默认不进入现金 ROI，只有客户确认活跃工时基线、角色成本和可释放比例后才单独披露。 + +## 本轮实现记录 + +- 2026-07-16：完成五张商业事实表与 0016 迁移、套餐/订阅/权益、用量/成本、配额门禁、幂等和冲回服务。 +- 2026-07-16：完成客户收费、内部成本、贡献毛利、财务确认现金节省、客户 ROI 和工时价值分账分析；修正 ROI 为净收益口径并收紧角色、时区和权益历史不可变边界。 +- 2026-07-16：补齐订阅暂停/逾期/取消/过期、恢复和套餐/订阅/权益/用量/成本历史查询。 +- 2026-07-16：完成基于真实成本下限与确认价值上限的定价场景，禁止风险暴露、预计节省或跨币种混入报价。 +- 2026-07-16：完成商业工作台五块能力、订阅生命周期确认、用量/成本与价值分账、按币种定价场景两步确认；全量前端 802 项、code-size 和生产构建通过。 +- 2026-07-16：一次性 PostgreSQL 17 并发配额、幂等和成本冲回 3 项通过；真实开票、回款与客户合同参数继续保留为外部试点边界，不以 mock 标记完成。 +- 2026-07-16：新增 `0019` 运行时预占迁移和中央 Agent 工具 permit；在订阅/权益锁内按已用量加有效预占原子控额，成功按真实 AgentToolCall 结算，失败/阻断释放，变量用量超过预占拒绝写入。 +- 2026-07-16：新增持久补偿状态和过期补偿器；旧直接调用缺少预占时冻结容量并进入 reconciliation_required，运行终态不确定时不按 TTL 误释放；用量已提交但成本失败进入 committed_reconciliation_required，相同预占与用量/成本均可幂等重试。 +- 2026-07-16：新增 `0021` 不可变账期和商业管理审计；订阅创建即签发首期，月/季/年到期由 leader 调度器和订阅行锁幂等滚动，合同制与越界周期失败关闭。 +- 2026-07-16：用量、成本、运行时预占和配额查询改为绑定 `billing_period_id`，并以独立 `quota_period_key` 保留权益重置语义；客户收费基础费改从账期快照聚合，不再读取可变订阅当前周期。 +- 2026-07-16：容器内商业/迁移前置定向 131 项、前端商业 35 项和 Vite 构建通过；一次性 PostgreSQL 17 商业并发 5 项通过。0021 的升级、模型/约束/触发器（含账期重叠阻断）/运行不变量及 0021→0020 降级通过；全链 44/45 唯一失败来自后继 0023 尚未接管的盲审临时表，不属于 0021。 +- 2026-07-16：容器内商业/Agent/迁移组合 183 项通过、1 项因未显式配置外部迁移库跳过；运行时定向 27 项、PostgreSQL 商业并发 4 项和 0019→0020 完整迁移循环通过；全局 code-size 仅剩共享 `RiskRuleGenerationService` 817 行既有门禁失败，本轮所有相关核心类低于 800 行。 diff --git a/document/development/2026-07-16/feature/commercial-metering-and-roi/TODO.md b/document/development/2026-07-16/feature/commercial-metering-and-roi/TODO.md new file mode 100644 index 0000000..44e4bbe --- /dev/null +++ b/document/development/2026-07-16/feature/commercial-metering-and-roi/TODO.md @@ -0,0 +1,87 @@ +# 商业计量、客户 ROI 与可持续定价 开发 TODO + +更新时间：2026-07-17 + +## 使用规则 + +- 每项必须回链 `CONCEPT.md` 对应章节。 +- 只有代码、接口或容器验证提供证据后才能勾选。 +- 客户价值、平台收入、内部成本和工时估值必须分账；mock 与手工事件不得标记为生产事实。 + +## 1. 调研与边界 + +- [x] [CONCEPT: 背景与问题] 明确商业权益、用量、成本、客户 ROI、平台毛利和定价不是同一事实。 + 证据：`CONCEPT.md`“背景与问题”“目标与非目标”。 +- [x] [CONCEPT: 目标与非目标] 确认不硬编码客户价格、不混算风险暴露、不跨币种求和、不让付费绕过安全门禁。 + 证据：`CONCEPT.md`“目标与非目标”。 + +## 2. 契约与设计 + +- [x] [CONCEPT: 数据] 定义套餐、订阅、权益、用量和内部成本五类事实及状态。 + 证据：`commercial.py` 模型与 schema、`20260716_0016_commercial_metering.py`。 +- [x] [CONCEPT: 算法与公式] 定义客户 ROI、贡献毛利和可持续定价走廊公式。 + 证据：`CONCEPT.md`“算法与公式”、`commercial_analytics.py`、`commercial_pricing.py`。 +- [x] [CONCEPT: 权限] 定义平台配置、租户只读和商业/安全门禁分离。 + 证据：`commercial_access_policy.py`、`commercial_entitlements.py`。 + +## 3. 后端实现 + +- [x] [CONCEPT: 数据] 新增五张商业表、复合租户约束、幂等、冲回和 append-only 迁移。 + 证据：`models/commercial.py`、`20260716_0016_commercial_metering.py`、迁移/模型测试。 +- [x] [CONCEPT: 数据] 新增运行时预占运营表、状态约束、复合租户外键、全局 tool call 幂等和迁移所有权。 + 证据：`models/commercial_runtime.py`、`20260716_0019_commercial_runtime_reservations.py`、`commercial_migration_assertions.py`；0019→0020 一次性 PostgreSQL 完整升降级循环通过。 +- [x] [CONCEPT: 后端] 实现套餐版本、订阅、权益、配额、用量、成本与商业分析服务。 + 证据：`commercial_admin.py`、`commercial_entitlements.py`、`commercial_metering.py`、`commercial_analytics.py`。 +- [x] [CONCEPT: 生命周期与查询] 实现暂停、逾期、取消、过期、恢复与五类历史查询。 + 证据：`CommercialAdminService.transition_subscription()`、`commercial_queries.py`、`/commercial/admin/tenants/{tenant_id}/...` 分资源接口。 +- [x] [CONCEPT: 定价走廊] 实现成本下限、确认价值上限、成功费封顶和商业模式建议。 + 证据：`commercial_pricing.py`、`POST /commercial/admin/tenants/{tenant_id}/pricing-scenarios`。 +- [x] [CONCEPT: 后端] 把中央 Orchestrator 工具接入执行前预占、真实 AgentToolCall 结算和失败释放。 + 证据：`orchestrator_tool_execution.py`、`agent_runs.py`、`commercial_runtime_bridge.py`；无配置兼容，配置后先 reserve 再执行，成功只追加真实用量/成本，失败和阻断不计量。 +- [x] [CONCEPT: 降级策略] 持久化缺预占和计量故障补偿状态，并安全处理过期预占。 + 证据：`commercial_runtime_reservations.py`、`commercial_runtime_reconciler.py`；直接调用形成 reconciliation_required，运行中/未知终态继续持有额度，终态无调用才过期释放；用量成功但成本失败形成 committed_reconciliation_required，重试只补成本且不重复冻结额度。 +- [x] [CONCEPT: 风险与开放问题] 把已识别的权威运行入口迁移到 permit 契约。 + 证据：中央 Orchestrator、`ocr_commercial.py`、`runtime_chat_commercial.py`、`financial_connector_commercial.py` 和 `expense_claim_attachment_commercial.py` 已接通预占/结算/释放；资源组合 63 项通过。 +- [ ] [CONCEPT: 风险与开放问题] 对后续新增的知识库/ONLYOFFICE 存储、实施和支持等资源入口持续执行 meter 盘点，不允许绕过 permit。 + 证据要求：新增真实资源入口时提供权威数量口径、事务边界、成本来源和回归测试；当前不把未发生的未来入口伪装成已计量。 +- [x] [CONCEPT: 风险与开放问题] 通过不可变 billing period 和幂等 rollover 实现 `auto_renew` 周期滚动。 + 证据：`20260716_0021_commercial_billing_periods.py`、`commercial_billing_periods.py`、`commercial_subscription_rollover.py`、`commercial_rollover_scheduler.py`；月/季/年自动滚动，合同制和 `ends_at` 越界失败关闭，PostgreSQL 双线程仅生成一个账期，数据库触发器拒绝重叠账期。 +- [x] [CONCEPT: 数据] 让用量、成本、运行时预占和配额历史绑定不可变账期，并分离配额重置键。 + 证据：`UsageMeterEvent`、`CommercialCostEvent`、`CommercialRuntimeReservation` 的 `billing_period_id`，以及 usage/reservation 的 `quota_period_key`；收费分析从账期快照读取基础费和币种。 +- [x] [CONCEPT: 权限] 为套餐、订阅、权益和续期建立脱敏追加式审计与租户安全历史 API。 + 证据：`commercial_admin_events`、`commercial_admin_audit.py`、`commercial_billing.py`；管理写接口强制 `X-Request-Id` 和原因，普通 finance/executive 只能读本租户账期，审计仅平台管理员可读。 +- [ ] [CONCEPT: 非目标] 对接真实开票/收款/订阅提供商并区分合同计费、已开票与已收现金。 + 证据：等待目标客户和 provider 选择，不使用 mock 冒充完成。 + +## 4. 前端实现 + +- [x] [CONCEPT: 前端] 完成商业工作台的账户、套餐/订阅、权益/配额、用量/成本、价值/定价五块。 + 证据：`CommercialWorkspace.vue` 组合账户、生命周期、权益、用量成本、价值分析与定价场景面板。 +- [x] [CONCEPT: 前端] 接入订阅暂停/取消/恢复、历史查询和操作确认。 + 证据：`useCommercialWorkspace.js`、`CommercialSubscriptionLifecyclePanel.vue`；终态和定价均有确认步骤，操作后按租户重新加载。 +- [x] [CONCEPT: 前端] 严格分开展示客户 ROI 与平台毛利，并支持多币种和证据缺口状态。 + 证据：`CommercialValueAnalysisPanel.vue`、`CommercialPricingScenarioPanel.vue`、`commercialWorkspaceModel.js`；按币种分组，null/unavailable 显示“不可用”，不跨币种合计。 +- [x] [CONCEPT: 前端] 接入现有应用入口、权限态、移动端和生产构建。 + 证据：`OverviewView.vue`/顶部导航已接入“商业化管理”；商业定向 35 项、全量 web 802 项、code-size 与 Vite 2246 modules 构建通过。 + +## 5. 测试与验证 + +- [x] [CONCEPT: 测试方案] 后端模型、服务、HTTP、权限、生命周期、查询和定价回归通过。 + 证据：容器内 Ruff 通过；`test_commercial_models.py`、`test_commercial_services.py`、`test_commercial_endpoints.py` 当前 12 项通过。 +- [x] [CONCEPT: 测试方案] 一次性 PostgreSQL 并发用量、原子预占、硬配额和成本冲回验证通过并记录当前命令结果。 + 证据：一次性 PostgreSQL 17 中 `test_commercial_concurrency_postgres.py` 4 项通过；两个并发工具竞争 1 份额度时仅一个 reservation 成功。 +- [x] [CONCEPT: 测试方案] 运行时预占、结算、释放、幂等、变量上限、历史配置和补偿回归通过。 + 证据：`test_commercial_runtime_metering.py`、`test_commercial_runtime_reservations.py` 27 项通过；商业/Agent/权限/迁移相关组合 183 项通过、1 项因未显式配置外部迁移库跳过，Ruff、compileall 和相关类 800 行检查通过。 +- [x] [CONCEPT: 测试方案] 前端行为测试、全量 web 测试、code-size 门禁和 Vite 构建通过。 + 证据：商业定向 35 项、全量 web 802 项通过；code-size 通过；Vite production build 转换 2246 个模块。 +- [x] [CONCEPT: 测试方案] 不可变账期、脱敏审计、自动续期和调度器验证通过。 + 证据：容器内商业/迁移前置定向 131 项、前端商业 35 项及 Vite build 通过；PostgreSQL 17 商业并发 5 项通过，含双线程续期单赢家；0021 升级、重叠账期阻断、运行不变量和 0021→0020 降级通过。 +- [x] [CONCEPT: 指标与验收] 逐项核对 A1-A7，并回填最终文件、接口和容器证据。 + 证据：商业模型/服务/API/前端、硬配额、账期、生命周期、ROI/毛利分账和定价走廊均有回归；资源边界组合 63 项、PostgreSQL 商业并发 5 项、Web 全量 815 项及 Vite build 通过。 + +## 6. 商业与试点收尾 + +- [ ] [CONCEPT: 风险与开放问题] 用真实试点 30/90 天数据冻结目标毛利率、最大价值分享、包含量、超额策略和封顶。 +- [ ] [CONCEPT: 风险与开放问题] 确认发票、税率、回款、坏账、渠道和收入确认边界。 +- [x] [CONCEPT: 本轮实现记录] 同步更新上位闭环文档与工程验收手册，不删除证据不足项。 + 证据：上位 AI 闭环 TODO 与 `engineering-closure-and-production-readiness` CONCEPT/TODO 已区分工程完成、生产上线和真实试点。 diff --git a/document/development/2026-07-16/feature/financial-connector-reconciliation/CONCEPT.md b/document/development/2026-07-16/feature/financial-connector-reconciliation/CONCEPT.md new file mode 100644 index 0000000..b2c2d0b --- /dev/null +++ b/document/development/2026-07-16/feature/financial-connector-reconciliation/CONCEPT.md @@ -0,0 +1,226 @@ +# 财务连接器与支付对账闭环 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +把经过租户、来源、密钥版本和请求路径共同认证的 production-mode 财务事件契约接入付款、ERP、对账与 Savings 闭环，同时让所有非生产回执严格停留在只读模拟事实层；真实外部现金仍以目标 provider 联调为准。 + +## 背景与问题 + +当前报销单可以由财务人员在平台内确认“已付款”，并能联动申请归档和 Savings 实现记录，但这只是内部业务状态，不是银行、支付平台或 ERP 的外部现金事实。若继续把单一状态当成真实回执，会留下重复付款、金额/币种错配、回执伪造、凭证缺失、对账异常未处置和虚假现金节省等风险。 + +本功能把外部财务系统接入收敛成统一、可审计的连接器契约。首期允许使用 mock adapter 验证协议和流程，但 mock 必须完整模拟签名、幂等、失败、重试、乱序、退款、凭证和对账差异，不以直接写“已付款”代替连接器事实。 + +## 目标与非目标 + +### 目标 + +- 建立租户隔离、来源可验证、追加式的支付/银行/ERP 事件账本。 +- 支持支付批次、结算成功、支付失败、退款/冲回、ERP 入账凭证和对账结果。 +- 只有单据、金额、币种、外部引用和签名均通过校验时，才推进报销付款状态。 +- 把外部事件与 Expense Case、Business Event、Savings Evidence、归档事件用同一 correlation 链关联。 +- 对重复、冲突、乱序和不完整事件 fail-closed，并提供可人工处理的对账异常。 +- 以版本化 activate/disable/rotate 状态机管理连接器密钥，所有配置动作形成不含密钥材料的追加式审计。 + +### 非目标 + +- 不自建银行清算、企业支付网络、税务开票网络或 ERP 总账。 +- 不保存银行卡号、银行流水原文、完整付款人账号或连接器密钥明文。 +- 不允许连接器绕过报销审批、风险门禁、租户权限或财务确认。 +- 不把 mock 回执标记为生产级外部现金证据；运行环境和证据等级必须显式区分。 + +## 用户与场景 + +- 财务付款员：提交或查看付款批次，处理失败与待匹配回执。 +- 财务复核员：复核金额/币种/收款主体和对账异常，确认或拒绝处置。 +- 财务负责人/CFO：查看已匹配、待对账、失败、退款和未入账金额。 +- 平台管理员：配置连接器公钥/密钥版本、来源白名单和健康状态，但不能代替财务确认业务结果。 +- 外部连接器：按租户和来源签名推送支付、银行或 ERP 事件，安全重试并读取幂等结果。 + +## 功能能力 + +- 连接器注册与密钥版本：来源、环境、允许事件、时钟偏差、启停和轮换状态。 +- 统一事件信封：tenant、provider、event ID、event type、occurred at、payload hash、correlation、signature version。 +- 支付批次与回执：批次创建、提交、受理、成功、失败和部分成功。 +- ERP 入账：凭证号、会计期间、入账时间、受控摘要和原始内容哈希。 +- 对账：按单据、金额、币种和外部引用自动匹配；差异进入人工处置。 +- 冲回：退款、撤销和补付使用新事件，不更新或删除原事件。 +- 可观测性：最近成功时间、失败率、重试次数、积压、签名失败和对账差异。 + +## 方案设计 + +### 模块职责 + +- `financial_connector_auth`：验证来源、签名、时间戳、密钥版本和重放窗口。 +- `financial_connector_ingestion`：规范化事件、计算指纹、幂等写入、生产/模拟分流和冲突检测。 +- `financial_connector_simulation`：只读校验 test/mock/staging 回执，不创建或修改 Claim、对账、ERP、Business Event 或 Savings。 +- `financial_connector_mock_adapter`：平台管理员显式触发的非生产场景适配器；用已激活配置的真实签名链确定性生成成功、失败、乱序、重复、冲突、退款和 ERP 回执，但不提供任意 payload 注入能力。 +- `financial_connector_observability`：按租户和配置聚合最近成功、失败率、重试、积压、签名失败、冲突和对账异常；只读取最小化事实与追加式运行事件。 +- `financial_connector_payment_evidence`：从 Claim 的付款事实中提取统一证据 DTO，明确区分 production-mode 外部回执分类、非生产模拟回执和人工付款内部状态；分类本身不证明真实 provider 已接通。 +- `financial_connector_config_lifecycle`：执行带 expected version 的激活、停用和原子密钥轮换。 +- `payment_reconciliation`：匹配 Claim、金额、币种和状态，生成 matched / exception 结果。 +- `financial_connector_actions`：在可信匹配后调用现有付款动作；支付失败、退款和 ERP 入账分别旁写事件。 +- `financial_connector_projection`：为财务工作台提供脱敏列表、详情、差异和健康度。 +- 供应商 adapter 只负责供应商字段映射，不直接修改 Claim、Savings 或预算。 + +### 数据与契约 + +首期新增以下 migration-owned 表，均带 `tenant_id`： + +- `financial_connector_configs`：provider、environment、allowed event types、secret/key version、status、last success/error；只保存密钥引用或不可逆验证材料。 +- `financial_connector_config_events`：created、activated、disabled、rotation started/replacement created 的追加式配置审计；保存 actor、request、reason、expected version 和脱敏前后状态，不保存 `secret_ref`。 +- `financial_connector_events`：方向、事件类型、外部事件 ID、请求指纹、原始内容哈希、发生/接收时间、验证等级、处理状态、关联单据/Case 和错误码；UPDATE/DELETE 禁止。 +- `payment_reconciliation_cases`：Claim、期望/实际金额与币种、匹配状态、差异、处置版本、负责人和最后事件；作为可变投影，历史动作另存事件。 +- `payment_reconciliation_events`：创建、自动匹配、人工确认、拒绝、重开、退款和关闭的追加式审计事件，保存请求指纹和首次响应。 +- `financial_connector_operational_events`（`0022`）：保存 `replay / auth_failure / payload_conflict` 三类运行事实的 tenant/config/provider/environment、受控原因码、两类 HMAC 指纹、幂等键和发生时间；不保存原始 payload、签名、外部事件 ID、Claim 引用、correlation 或密钥材料。复合租户外键防止跨租户归属，PostgreSQL 触发器禁止 UPDATE/DELETE；存在运行事实时拒绝有损降级。 + +关键约束： + +- `(tenant_id, provider, external_event_id)` 唯一。 +- 配置从 `disabled/version=1` 创建；激活和停用必须命中 expected version，轮换原子地产生新 active key version 并把旧版本置为 rotating。 +- 同幂等键不同 payload hash 返回冲突，不能覆盖首次事件。 +- 同一个运行事实 candidate 的补偿写入按 `(tenant_id, idempotency_key)` 幂等；不同 HTTP 尝试即使请求内容相同，也因可信发生时间不同而分别计数，避免把真实重放次数永久压成一次。 +- Claim、Case、配置和对账记录使用复合租户外键。 +- 退款/冲回必须引用同租户原结算事件。 +- 生产事件必须通过已激活密钥验证；test/mock/staging 即使签名和业务字段全部匹配，也只能生成 `projection_scope=simulation_only` 的连接器事实，绝不进入核心财务状态机。 + +### 接口 + +- `POST /api/v1/integrations/financial-events`：连接器签名事件入口；返回稳定接收/重放结果。 +- `POST /api/v1/financial-connectors/admin/tenants/{tenant}/configs/{id}/activate|disable|rotate`：带版本、actor、request ID 和 reason 的配置状态机。 +- `POST /api/v1/financial-connectors/admin/tenants/{tenant}/configs/{id}/simulate`：平台管理员运行确定性非生产场景；production 配置和未激活配置 fail-closed。 +- `GET /api/v1/financial-connectors/admin/tenants/{tenant}/config-events`：读取不含密钥材料的配置审计时间线。 +- `GET /api/v1/financial-connectors/admin/tenants/{tenant}/observability`：平台管理员读取目标租户脱敏运行指标；从 config/event/reconciliation/operational event 聚合指定窗口真实值。 +- `GET /api/v1/financial-connectors/observability`：财务角色读取当前租户脱敏运行指标；返回 `window_started_at / as_of / generated_at / source_revision`，以及 replay、认证失败、签名失败、payload conflict 的真实计数与最近发生时间。`0022` 起四类指标均标记 `available`，没有事实时真实返回零而不是“待采集”占位。 +- `GET /api/v1/financial-connectors/payment-evidence/{claim}`：有单据读取权限的当前租户用户读取付款证据等级，不返回原始连接器内容。 +- `GET /api/v1/financial-reconciliation/cases`：财务角色分页查看匹配与异常。 +- `GET /api/v1/financial-reconciliation/cases/{id}`：查看脱敏证据和追加式时间线。 +- `POST /api/v1/financial-reconciliation/cases/{id}/confirm`：独立财务确认差异或人工匹配。 +- `POST /api/v1/financial-reconciliation/cases/{id}/reject`：拒绝错误回执并记录原因。 +- 连接器配置管理接口仅平台管理员可用，业务确认接口仅财务角色可用，两类权限不互相继承。 + +### 匹配算法 + +1. 验证 tenant/provider/key version/timestamp/signature 和事件类型；HMAC canonical request 同时绑定固定 HTTP method/path，禁止共享密钥跨 provider 或 key version 重放。 +2. 以 canonical JSON 生成 payload hash；按外部事件 ID 与幂等键检查首次请求。 +3. 通过受控引用解析 Claim，不允许仅凭模糊姓名或备注自动匹配。 +4. 校验 Claim 已完成审批且处于待付款；比较金额、币种、外部业务引用和事件方向。 +5. 只有 `production_verified` 且完全一致时才自动 matched；非生产来源只生成模拟投影，任何生产差异进入 exception 且不改变 Claim。 +6. matched 事件在同事务调用付款动作、记录 Business Event，并把外部事件哈希作为 Savings 证据。 +7. ERP posted 只表示入账，不重复触发付款；refund/reversal 追加负向业务与 Savings 冲回候选。 + +### 权限与安全 + +- 连接器入口不使用普通用户会话，使用租户绑定的签名认证；普通 Bearer token 不能伪装连接器。 +- HMAC/签名比较使用常量时间函数，限制时间窗口并记录 nonce/外部事件 ID 防重放。 +- 激活和轮换前由服务端解析 `secret_ref` 并校验至少 128-bit HMAC 密钥；数据库和审计 DTO 均不返回引用或明文。 +- 日志、响应和 DTO 最多暴露外部引用后八位或不可逆摘要，不返回原始 payload、签名或密钥。 +- replay 随成功重放事务提交；认证失败与 payload 冲突先回滚失败业务事务，再独立提交脱敏运行事实，审计写入异常不得覆盖原始 401/409 响应。 +- 只有服务端按 tenant/provider/key version 解析出 active 配置，并成功解析该配置的服务端密钥后，才构造可归属的 operational context。缺认证头、请求 tenant 不一致、未知/未激活配置或密钥不可用均不接受客户端自报租户，只写不带租户归属的结构化警告；签名、时间窗和事件白名单失败才可安全归入已解析配置。 +- 运行表只保存 `hmac-sha256:` 请求/外部事件指纹和 `sha256:` 幂等键。HMAC 输入可以包含外部 ID 和业务引用，但这些原值不会进入表、DTO 或错误日志。 +- 不可变连接器事实的 normalized payload 不再保存完整 `claim_reference`；`0020` 受控迁移会删除历史冗余值，保留内容哈希、受控 Claim ID 与必要尾号。 +- 跨租户资源统一按 404 隐藏；配置管理员不能确认对账，付款申请人不能确认自己的异常。 +- 连接器故障、未知密钥、签名异常、金额/币种不一致和数据库异常均 fail-closed。 + +### 状态转换 + +- 连接器事件：`received → verified → processed`，失败进入 `rejected`；事实本身追加只读。 +- 连接器配置：`disabled → active → rotating → disabled`；正常启用走 `disabled → active`，轮换时旧 active 原子进入 rotating、新 key version 以 active 创建。 +- 对账记录：`pending → matched | exception → confirmed | rejected`；退款可从 confirmed 进入 `reopened`，重新处置后关闭。 +- Claim 仅在可信 `payment_settled + matched` 后从 `pending_payment` 进入 `paid`。 +- ERP 凭证从 `pending_posting` 进入 `posted | posting_failed`，不反向伪造支付成功。 + +### 降级策略 + +- 连接器离线：保留待付款，不自动标记已付；显示积压和最后成功时间。 +- 回执乱序：先保存事实，等待前置事件或进入 pending，不猜测状态。 +- 回执冲突：保留首次事实并返回 409，生成对账异常。 +- ERP 未接入：付款事实可进入已付，但“已入账/凭证号”保持待采集。 +- test/mock/staging：返回 `simulation_only`，只保留追加式 connector fact/response projection，不创建对账 Case、不修改 Claim/ERP/归档/Savings，也不进入生产现金证明。 +- 显式 mock adapter 只能选择预定义场景和当前租户 Claim；事件 ID、correlation 和受控外部引用由 tenant/config/scenario/request ID 确定性派生。重复执行同一请求只产生稳定重放，不能借模拟接口注入生产配置或任意字段。 +- 升级前若已有非生产事件且响应曾关联对账 Case，`0020` 将其标记为 `legacy_nonproduction_effect_unknown` 供审计，不伪装成新策略下的无副作用模拟事实，也不在迁移中猜测性冲回历史财务状态。 + +### 兼容策略 + +- 保留现有人工“确认已付款”作为低等级内部证据；付款证据 DTO 使用 `internal_manual_payment`，生产连接器使用 `external_cash`，非生产连接器使用 `simulated_connector`/`staging_connector`。UI 必须同时展示来源标签和可信等级，不能只显示“已付款”。 +- 新连接器路径复用现有幂等付款动作、Case 时间线和 Savings 实现服务，不复制第二套状态机。 +- 现有 `risk_flags_json` 付款摘要继续只读兼容，新连接器事实进入正式事件表。 + +## 测试方案 + +- 单元：签名、时间窗口、canonical hash、幂等重放、冲突、乱序和字段白名单。 +- 权限：跨租户、普通用户伪造、管理员越权、申请人自证和密钥停用。 +- PostgreSQL：复合外键、唯一键、append-only、并发同事件、不同 payload 冲突和安全降级。 +- 集成：申请 → 票据 → 报销 → 预审 → 审批 → 外部付款 → 对账 → ERP 入账 → 归档 → Savings 待确认。 +- 反向：支付失败不推进、金额/币种错配不推进、重复回执只写一次、退款追加冲回。 +- adapter：逐场景验证确定性结果、重复请求、跨租户 Claim 隐藏、production 拒绝，以及 Claim/对账/ERP/Business Event/Savings 零副作用。 +- 可观测性：验证租户隔离、财务/管理员权限、窗口边界、签名失败与重放计数，并断言 DTO/日志无原始 payload、签名和密钥。 +- 运行事件：验证同 candidate 并发/补偿重试单赢家、不同请求尝试分别计数、冲突回滚后独立持久化、复合租户外键、HMAC 格式检查和数据库 append-only。 +- 所有验证只在 `local-x-financial-linux` 容器内执行，每条命令最长 60 秒。 + +## 算法与公式 + +本能力不做概率预测，核心是确定性门禁： + +```text +canonical_effect_allowed = ( + verification_level == production_verified + AND signature_valid + AND tenant_provider_key_path_bound + AND claim_amount_currency_reference_match +) +``` + +任一条件为假都不得产生核心财务副作用；非生产环境无论其他条件是否为真，`canonical_effect_allowed` 固定为 false。 + +运行指标使用确定性窗口聚合，不从应用日志估算： + +```text +operational_count(type, window) = COUNT( + tenant_id = current_tenant + AND event_type = type + AND window_started_at <= occurred_at <= as_of +) + +operational_idempotency_key = SHA256( + source_revision, tenant, config, type, reason, + HMAC(request), HMAC(external_event), occurred_at +) +``` + +`source_revision=20260716_0022` 表示当前运行指标的数据源与聚合契约版本；它不是 provider 协议版本。发生时间属于本次接收尝试，因此同一 candidate 重试仍稳定，而新尝试会形成新的真实计数。 + +## 指标与验收 + +- 100% 外部结算事件具有租户、来源、签名版本、payload hash、外部 ID 和接收时间。 +- 重复相同事件稳定重放，冲突 payload 100% 拒绝。 +- 任何金额/币种/Claim/审批状态不一致均不会推进已付款。 +- 外部支付、ERP 凭证、对账处置、归档和 Savings 证据可由 correlation 链回放。 +- 连接器离线或异常时不出现虚假“已付款”“已入账”或现金节省。 +- observability 响应 100% 标明窗口起止和 source revision；三类运行事实按租户真实计数，并提供各类最近发生时间。 + +## 风险与开放问题 + +- 首期真实 provider、签名算法、事件字段和 SLA 需要目标客户确认。 +- 部分 ERP 只有批次级凭证，需要明确批次到单据的拆分和舍入规则。 +- 多币种付款需要锁定汇率来源和会计期间；本功能不自行猜测汇率。 +- 退款、补付、员工自担调整是否进入现金节省，仍需客户财务签字口径。 +- 对账大额阈值及双人复核需按企业策略配置。 + +## 本轮实现记录 + +- 2026-07-16：完成现有内部付款、申请归档、Business Event、Savings 实现链路盘点，并冻结统一连接器、安全认证、对账状态和完整 E2E 方案；实现与容器证据保留在同目录 TODO 中继续执行。 +- 2026-07-16：完成 `0017` 四表迁移、HMAC/密钥版本/时间窗认证、租户隔离、追加式事件、幂等冲突和对账投影；一次性 PostgreSQL 17 迁移循环与 2 项并发探针通过。 +- 2026-07-16：支付结算只在金额、币种、Claim、审批状态与来源全部匹配时复用正式付款动作；ERP、失败、退款/冲回、Savings 失效和财务处置均进入同一 correlation 审计链，连接器服务/HTTP 7 项通过。 +- 2026-07-16：真实 provider、批次拆分、汇率、会计期间和大额双人复核仍由目标客户确认；当前 mock/test 证据始终标记 simulated，不冒充生产现金事实。 +- 2026-07-16：补齐 `0020` 配置版本与追加式审计迁移；新配置只能停用创建，激活前校验服务端密钥，轮换原子切换 key version，HTTP 时间线不暴露 `secret_ref`。 +- 2026-07-16：把 test/mock/staging 六类事件收口到 `simulation_only` 只读投影；生产事件继续完成付款、ERP、归档、Savings 和冲回，生产冲回也不能引用模拟原事件。 +- 2026-07-16：normalized payload 删除完整 `claim_reference`，历史冗余值由 `0020` 受控脱敏，HMAC v2 继续以内容哈希和 tenant/provider/key version/path 证明请求边界。 +- 2026-07-16：容器内连接器/配置/费用价值链组合 16 项、PostgreSQL 并发 3 项、迁移静态 124 项及全新 PostgreSQL 17 完整升降级循环通过；独立 0019→0020 探针确认版本回填、历史脱敏和 legacy 不确定性标记符合契约。 +- 2026-07-16：新增显式非生产 adapter，以 tenant/config/claim/scenario/request ID 派生稳定签名回执，覆盖成功、失败、乱序、重复、冲突、退款和 ERP 场景；HTTP 与服务测试确认 simulation-only 且 Claim、对账、Business Event、ERP 与 Savings 零副作用。 +- 2026-07-16：新增租户级脱敏可观测性 API 与财务看板面板，现有 config/event/reconciliation 表提供最后成功、失败率、积压和对账异常真实值；retry/auth_failure 在 `0022` 运行事件落地前明确显示 unavailable，不伪造零值。 +- 2026-07-16：新增付款证据等级 DTO 和 UI 口径，通过 production-mode 签名契约的回执分类为 `external_cash`，人工确认标为低等级 `internal_manual_payment`，模拟/预发布回执明确不进入核心账；当前本地自签测试不作为真实现金或 provider 接通证据。 +- 2026-07-17：完成 `0022` 追加式运行事实迁移与服务接入，耐久记录 replay、可信归属后的 auth failure 和 payload conflict；请求与外部事件只保存 HMAC 指纹，失败事务回滚后独立补偿提交。 +- 2026-07-17：可观测性改为返回 `20260716_0022` source revision、明确窗口、真实计数和最近时间；同一 candidate 补偿重试幂等，不同 HTTP 尝试分别计数。迁移总头继续串到既有 `0023`，`0022` 不越界创建后继 AI 表。 +- 2026-07-17：全新一次性 PostgreSQL 17 完整迁移循环 51 项、连接器并发 4 项、后继盲审并发 7 项通过；验证租户复合外键、HMAC 格式、运行事实 append-only 和并发单赢家。 diff --git a/document/development/2026-07-16/feature/financial-connector-reconciliation/TODO.md b/document/development/2026-07-16/feature/financial-connector-reconciliation/TODO.md new file mode 100644 index 0000000..bdcc1a6 --- /dev/null +++ b/document/development/2026-07-16/feature/financial-connector-reconciliation/TODO.md @@ -0,0 +1,76 @@ +# 财务连接器与支付对账闭环 TODO + +更新时间：2026-07-17 + +## 使用规则 + +- 每项必须回链 `CONCEPT.md`；只有代码、迁移、接口或容器验证提供证据后才能勾选。 +- 外部回执、内部付款状态、ERP 入账和财务确认必须分开，mock 不得伪装成生产现金事实。 + +## 1. 契约与安全 + +- [x] [CONCEPT: 背景与问题] 盘点内部付款、申请归档、Business Event、Savings 实现和证据边界。 + 证据：`expense_claim_approval_flow.py`、`expense_claim_application_handoff.py`、`expense_cases.py`、`savings_realization.py` 只读审计。 +- [x] [CONCEPT: 目标与非目标] 冻结签名事件、幂等、对账、ERP 凭证、冲回和 mock 环境边界。 + 证据：`CONCEPT.md`“目标与非目标”“数据与契约”“匹配算法”“降级策略”。 +- [x] [CONCEPT: 权限与安全] 实现连接器签名认证、密钥版本、时间窗口、来源白名单和防重放。 + 证据：`financial_connector_auth.py`、`financial_connector_ingestion.py`；HMAC 使用常量时间比较，tenant/provider/key version/timestamp/method/path 均进入签名边界，共享密钥跨 provider/key version 重放被拒绝，相同事件幂等重放、冲突 payload 返回 409。 +- [x] [CONCEPT: 权限与安全] 实现平台配置权限与财务处置权限分离、跨租户 404 和申请人自证拒绝。 + 证据：`financial_connectors.py`、`financial_connector_projection.py`；HTTP/服务回归覆盖普通用户、平台管理员、财务角色、跨租户隐藏与申请人自证拒绝。 + +## 2. 数据与迁移 + +- [x] [CONCEPT: 数据与契约] 新增配置、外部事件、对账投影和对账事件模型。 + 证据：`models/financial_connector.py`、`schemas/financial_connector.py`。 +- [x] [CONCEPT: 数据与契约] 新增后继 Alembic 迁移、迁移所有权、复合租户外键、唯一/检查约束和 append-only 触发器。 + 证据：`20260716_0017_financial_connector_reconciliation.py`、`schema_ownership.py`、`migration_preflight.py`；一次性 PostgreSQL 17 完整迁移循环通过。 +- [x] [CONCEPT: 数据与契约] 实现外部事件与退款/冲回引用、首次响应和 payload 指纹冲突。 + 证据：`financial_connector_ingestion.py`、`payment_reconciliation.py`；同外部事件不同 payload 拒绝，退款/冲回必须绑定同租户已处理结算原事件。 +- [x] [CONCEPT: 数据与契约] 新增配置 version、追加式配置审计和历史 normalized payload 脱敏迁移。 + 证据：`20260716_0020_financial_connector_config_lifecycle.py`、`FinancialConnectorConfigEvent`；配置审计使用 PostgreSQL append-only trigger，历史 `claim_reference` 从不可变事件的 normalized payload 中受控移除，内容哈希继续保留；全新 PostgreSQL 17 完整升降级循环 1 项通过，另一个独立库验证 0019→0020 历史数据脱敏与 legacy 标识。 + +## 3. 服务与接口 + +- [x] [CONCEPT: 模块职责] 拆分认证、ingestion、reconciliation、action 和 projection 服务，核心文件不超过 800 行。 + 证据：`financial_connector_auth.py`、`financial_connector_ingestion.py`、`payment_reconciliation.py`、`financial_connector_actions.py`、`financial_connector_projection.py` 职责独立，最大核心文件低于 800 行。 +- [x] [CONCEPT: 接口] 实现统一事件入口、连接器配置、对账列表/详情、确认和拒绝接口。 + 证据：`api/v1/endpoints/financial_connectors.py`。 +- [x] [CONCEPT: 接口] 实现带 expected version、actor、request ID、reason 的 activate/disable/rotate 状态机与脱敏审计查询。 + 证据：`financial_connector_config_lifecycle.py`、`financial_connector_config_audit.py`、`FinancialConnectorConfigLifecycleAction`、`FinancialConnectorConfigRotateAction`；激活/轮换前解析服务端密钥并校验强度，轮换原子切换新旧 key version。 +- [x] [CONCEPT: 匹配算法] 完全匹配时复用现有幂等付款动作；任何金额、币种、单据或审批状态差异不产生付款副作用。 + 证据：`FinancialConnectorActionService` 复用 `ExpenseClaimService.mark_claim_paid_from_connector()`；反向测试验证 mismatch/failure/conflict 无付款副作用。 +- [x] [CONCEPT: 证据与审计] 把外部事件哈希写入 Expense Case/Business Event/Savings 证据 correlation 链。 + 证据：连接器结算动作写入脱敏内容哈希、verification/evidence classification 与 correlation；服务 E2E 可回放付款、Case、Business Event 和 Savings evidence。 +- [x] [CONCEPT: 状态转换] 实现支付失败、ERP posted/posting_failed、退款/冲回和对账重开。 + 证据：`PaymentReconciliationService` 对六类生产事件分流；ERP 不重复付款，生产退款/冲回恢复 Claim 并追加 Savings 冲回事实。 +- [x] [CONCEPT: 降级策略] test/mock/staging 六类事件只写 simulation-only connector fact/response projection，不修改核心财务状态。 + 证据：`financial_connector_simulation.py`、`financial_connector_ingestion.py`；`test_financial_connector_services.py` 参数化覆盖三类非生产环境和六类事件，Claim、申请归档、对账、Business Event、ERP 与 Savings 均保持不变。 + +## 4. Mock 与可观测性 + +- [x] [CONCEPT: 降级策略] 实现明确标识 test/mock 的 adapter，覆盖成功、失败、乱序、重复、冲突、退款和 ERP 回执。 + 证据：`financial_connector_mock_adapter.py`、`FinancialConnectorSimulationCreate/Read` 与平台管理员 simulate API；仅 active test/mock/staging 可运行，tenant/config/claim/scenario/request ID 确定性派生事件，production、disabled 和跨租户请求 fail-closed；7 场景服务/HTTP 回归通过。 +- [x] [CONCEPT: 可观测性] 从现有事实输出最后成功时间、失败率、积压和对账异常，并在安全 DTO/UI 声明未采集指标。 + 证据：`financial_connector_observability.py`、当前租户/管理员 observability API、`FinancialConnectorHealthPanel.vue`；所有查询先绑定 tenant，仅聚合 config/event/reconciliation 最小化事实，不返回原始 payload、签名和密钥。 +- [x] [CONCEPT: 可观测性] 用后继 `0022` 追加式运行事件补齐 replay、auth_failure 和 payload conflict 耐久计数。 + 证据：`20260716_0022_financial_connector_operational_events.py`、`financial_connector_operational_events.py`、`financial_connector_auth.py`、`financial_connector_ingestion.py`、`financial_connector_observability.py`；只有可信配置与服务端密钥解析后才归属认证失败，表内只保存 HMAC 指纹；同 candidate 重试幂等、不同接收尝试分别计数，API 返回窗口、source revision、真实数量和最近时间。 +- [x] [CONCEPT: 兼容策略] 保留人工付款为低等级内部证据，并在 DTO/UI 区分外部回执和内部确认。 + 证据：`financial_connector_payment_evidence.py`、payment-evidence API、`FinancialPaymentEvidenceRead` 与面板证据口径；人工付款=`internal_manual_payment`，通过 production-mode 契约验证的外部回执分类=`external_cash`，非生产回执=`simulated_connector|staging_connector`。真实 provider 仍待联调。 + +## 5. 测试与验收 + +- [x] [CONCEPT: 测试方案] 签名、防重放、幂等、冲突、字段白名单和错误恢复单元测试通过。 + 证据：`test_financial_connector_services.py`、`test_financial_connector_endpoints.py`、`test_financial_connector_config_lifecycle.py` 与费用价值链 E2E 共 16 项通过；乱序原事件缺失保守进入 exception，不推进付款。 +- [x] [CONCEPT: 测试方案] PostgreSQL 迁移、复合租户约束、append-only、并发和安全降级验证通过。 + 证据：fresh PostgreSQL 17 最终迁移总探针 62 项通过；`financial_connector_migration_assertions.py` 验证配置/事件/运行事实 trigger、租户约束、HMAC 格式和 version check；`test_financial_connector_concurrency_postgres.py` 4 项通过，覆盖同事件、冲突补偿事实、配置单版本胜者和 operational candidate 单赢家；最终 head 为 `20260717_0028`。 +- [x] [CONCEPT: 测试方案] 申请 → 票据 → 报销 → 预审 → 审批 → 外部付款 → 对账 → ERP 入账 → 归档端到端通过。 + 证据：`test_expense_financial_value_chain_e2e.py` 使用测试密钥自签 production-mode HMAC 事件，覆盖申请审批、报销审批、ERP 入账、独立财务确认、Savings、商业价值与退款冲回契约；与连接器定向组合共 16 项通过，不代表真实 provider 回执或现金。 +- [x] [CONCEPT: 测试方案] 支付失败、金额/币种错配、重复回执和退款反向链路通过。 + 证据：`test_financial_connector_services.py` 覆盖 production-mode 失败/错配无副作用、稳定重放、ERP、reversal/refund 追加 Savings 冲回，以及非生产回执完全不创建 Savings。 +- [x] [CONCEPT: 容器验证] 相关 pytest、Ruff、前端测试和构建均在 `local-x-financial-linux` 内通过。 + 证据：历史连接器/配置/费用价值链/迁移组合 `140 passed, 1 skipped`；adapter/观测/证据 DTO 与既有连接器回归 `21 passed, 3 skipped`。2026-07-17 的 `0022` 收尾在容器内新增/定向回归 `115 passed`，全新 PostgreSQL 17 完整迁移循环 `51 passed`，连接器并发 `4 passed`，后继 `0023` 并发 `7 passed`；相关 Ruff、文件行数和全树 `git diff --check` 通过。历史连接器前端 3 项与 Vite 生产构建已通过；共享前端曾有 5 项旧路径测试失败，已单独记录且不属于本切片。 + +## 6. 客户配置待确认 + +- [ ] [CONCEPT: 风险与开放问题] 确认首个 provider、签名算法、字段映射、事件 SLA 和重试窗口。 +- [ ] [CONCEPT: 风险与开放问题] 确认批次到单据映射、多币种汇率、会计期间、大额双人复核和退款口径。 diff --git a/document/development/2026-07-16/feature/savings-ledger-and-cfo-value/CONCEPT.md b/document/development/2026-07-16/feature/savings-ledger-and-cfo-value/CONCEPT.md new file mode 100644 index 0000000..d92c964 --- /dev/null +++ b/document/development/2026-07-16/feature/savings-ledger-and-cfo-value/CONCEPT.md @@ -0,0 +1,373 @@ +# 节省事实账本与 CFO 经营价值看板 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +把费用优化从“发现风险和预计能省”推进为“执行、实际结果、独立财务确认、可回放冲回”的事实账本，并让 CFO 只看到有来源、有基线、有证据、可去重的企业价值。 + +## 背景与问题 + +- 现有财务看板能够回答支出、单量、待付款、预算使用和风险分布，但不能可信回答企业已经节省了多少钱。 +- 风险观察金额、暂缓付款、未使用预算和未执行建议都不是现金节省；如果直接汇总，会形成虚假 ROI。 +- 当前最接近真实节省的链路是“住宿超标准 → 用户接受职级标准重算 → 审批 → 付款”。它已经真实降低报销金额，但尚未形成独立机会、付款后结果、财务签字、去重和冲回记录。 +- `ExpenseClaim`、预算和应付旧表没有结构化租户键，Claim 只能通过 `ExpenseCaseLink` 判断租户；旧 `FinanceDashboardService` 仍有全表读取和跨租户快照复用风险，不能作为客户 ROI 的事实源。 +- 现有 `accept_standard_adjustment` 优先使用客户端传入的原金额，客户端理论上可以放大差额；任何节省计算必须改为只使用服务端锁定的明细金额和政策计算快照。 +- 当前“已付款”是财务人员在系统内确认的业务状态，尚没有银行流水、支付回执或 ERP 凭证。因此付款只能把机会推进到“实际结果待确认”，不能直接进入财务确认 KPI。 +- 报销提交时间到同租户首个 `payment_completed` 业务事件可以形成可审计的端到端流程周期，但它包含等待与系统处理，不是人工活跃工时；在没有人工计时基线、角色成本和客户认可释放比例前，工时价值必须显示“待采集”，不能伪造为 0 或现金节省。 + +本方案是 `2026-07-13/feature/ai-expense-closed-loop-and-value-proof` 中 P2“费用经营与价值证明”的实施拆分。 + +## 目标与非目标 + +### 目标 + +- [G1] 建立租户安全的 Savings Ledger，完整区分风险暴露、预计机会、执行中、实际结果、财务确认和冲回。 +- [G2] 建立不可变基线和证据链，所有金额都能追溯到费用事件、单据明细、政策版本、执行动作、付款事件和确认人。 +- [G3] 建立经济收益去重键和归因约束，避免同一单据、付款义务或政策差额被多个风险重复计入。 +- [G4] 建立严格状态机、乐观版本、幂等响应和数据库并发约束，防止重复确认、陈旧操作和跨租户访问。 +- [G5] 建立 CFO 价值看板，分开展示财务确认现金节省、可释放工时价值、安全智能直通率、经营漏斗和风险护栏。 +- [G6] 支持部门、项目、费用类型、供应商、城市、时间、负责人、来源和单据下钻，并显示数据覆盖、口径和新鲜度。 +- [G7] 持久化费用基线快照，记录窗口、样本量、算法版本、政策版本、来源指纹和数据质量。 +- [G8] 修复旧财务聚合与快照的租户边界，禁止真实接口失败时回退成看似真实的演示数字。 + +### 非目标 + +- [NG1] 不把风险关联金额、暂缓付款金额、未采纳建议、未使用预算或预计金额计入已确认节省。 +- [NG2] 首个切片不宣称已具备外部银行或 ERP 付款凭证；后续通过连接器补齐。 +- [NG3] 首个切片不使用缺少租户、合同价、采购数量和付款凭证的 `AccountsPayableRecord` 计算供应商节省。 +- [NG4] 不把申请金额与最终报销差额默认归因给 AI；缺少具体 AI 决策、采纳动作和结果链时，AI 归因金额为 0。 +- [NG5] 不把流程经过时长换算为人工工时，不直接暴露个人薪酬或个人成本。 +- [NG6] 不跨币种直接求和；没有锁定汇率的金额只按原币展示并进入数据质量提醒。 +- [NG7] 不删除、覆盖已确认收益；补付、退款、申诉或归因修正使用追加负向冲回事件。 +- [NG8] 不在本阶段重写整个 Overview，也不把不可信的预算中心模拟数据接入价值看板。 + +## 用户与场景 + +### 目标用户 + +1. CFO/管理层：查看企业已经确认的现金价值、价值兑现速度和风险护栏。 +2. 财务运营：复核机会、实际结果、重复归因、凭证和冲回事项。 +3. 费用治理负责人：领取机会、执行动作、补充结果和跟进逾期。 +4. 预算负责人：只在授权部门或成本中心范围内查看机会与驱动。 +5. 审计/风控：回放基线、政策、执行、付款、确认、冲回和操作事件。 +6. 普通员工：仅在自己的费用事件中看到与本人相关的调整说明，不访问企业 CFO 汇总。 + +### 核心场景 + +1. 员工接受住宿职级标准重算。服务端锁定明细原金额、城市、天数、职级、政策版本和可报销上限，同事务生成唯一节省机会。 +2. 机会进入执行后仍只展示预计金额；审批未通过、单据取消或超过期限时保留失败/到期事实，不从兑现率分母中消失。 +3. 单据完成付款业务事件后，系统根据冻结差额记录实际结果，但不进入财务确认 KPI。 +4. 与机会负责人和结果填报人不同的财务人员检查证据、去重、币种和成本后确认；确认后才计入 CFO 现金节省。 +5. 后续发生例外补付或申诉时，追加负向冲回并保留原确认，历史月报按报告 `as_of` 可回放。 +6. CFO 从价值总览下钻到部门、项目、费用类型、城市、负责人和具体单据，查看基线、建议、执行、实际、确认人和证据。 +7. 费用治理负责人查看异常集中维度和只读政策模拟准备项；历史中位数只能作为异常信号，缺少正式政策反事实时不显示预计节省，也不自动创建机会。 + +### 异常场景 + +- 服务端政策无法计算、明细金额缺失或差额不为正：原报销流程可继续，但不创建可货币化节省机会。 +- Claim 没有合法 `ExpenseCaseLink` 或租户不一致：fail-closed，不自动归入默认租户。 +- 相同请求重复发送：返回首次不可变响应；相同请求 ID 内容不同：409 拒绝。 +- 陈旧版本、重复付款事件或重复经济收益：通过版本锁、事件唯一键和收益去重键拒绝。 +- 缺少付款/凭证、汇率、独立确认或证据不完整：停留在实际待确认，不进入主 KPI。 +- 看板接口失败、无权限、无数据、基线不足或快照过期：分别展示明确状态，绝不使用模拟数字伪装真实指标。 + +## 功能能力 + +- [C1] 机会发现：从服务端核验的政策调整、后续分析洞察或风险复核创建机会。 +- [C2] 状态管理：支持 identified、accepted、in_progress、realized、verified、rejected、expired 和 reversed 事实。 +- [C3] 实现记录：保存实际毛收益、新增执行成本、净收益、发生时间、币种和结果证据。 +- [C4] 财务确认：独立确认人复核去重、证据、汇率、成本和归因后签字。 +- [C5] 证据与审计：只追加事件、内容指纹、before/after、首次响应和 correlation 全链路回放。 +- [C6] 基线快照：按员工、部门、费用类型、供应商、城市、项目和流程持久化窗口、样本量和版本。 +- [C7] 价值分析：经营漏斗、兑现率、周期、逾期、来源、责任人和数据质量。 +- [C8] CFO 看板：真实指标、全局筛选、URL 恢复、下钻、移动端和口径抽屉。 +- [C9] 安全边界：租户、角色、数据范围、自证禁止、管理员业务权限分离和字段白名单。 +- [C10] 冲回能力：补付、退款、申诉或归因修正只能追加负向记录，不改历史。 + +## 方案设计 + +### 前端 + +- 在现有“分析看板”增加 `value` / “经营价值看板”，复用统一时间筛选，不新增一级导航。 +- `OverviewView.vue` 只负责挂载独立 `CfoValueDashboard.vue`；价值加载、筛选和展示模型拆到 `useCfoValueDashboard.js`、`cfoValueDashboardModel.js` 与 `analyticsValue.js`，避免继续扩大接近 800 行的 `useOverviewView.js`。 +- 默认视图从上到下为：主 KPI 与护栏、价值漏斗、现金节省趋势、来源/组织驱动、机会执行表、数据质量和口径说明。 +- 全局筛选只保留时间、部门、费用类型和价值类型；项目、供应商、城市、负责人、状态和置信度进入高级筛选。 +- 看板状态同步到 URL query；当前机会使用 `value_opportunity` 保存，刷新、浏览器前进/后退和分享链接能够恢复同一抽屉。非法 ID、403/404、跨租户不可见或不再符合当前筛选/时间窗口的机会会 fail-closed 清理，避免残留上一租户详情。 +- 机会详情展示基线、建议、执行、实际结果、财务确认、去重与证据时间线；证据只使用服务端可见性 DTO。来源动作由独立 helper 根据 Claim、Expense Case、AI Decision、维度和 Evidence Resource 构造，不在抽屉组件内拼接路由规则。 +- 单据来源进入 `app-document-detail`；风险来源优先进入关联单据，并只携带风险 focus、观察/决策 ID 与现有锚点。详情返回动作恢复 `dashboard=value`、时间窗口和 `value_*` 查询。 +- 预算来源进入 `app-budget` 的“预算配置视图”，按授权范围应用部门和费用类型焦点；页面明确说明配置、阈值及当前演示金额不是该机会的真实预算事实。未配置的费用科目显示“未找到配置”，不得解释为预算为零。 +- 部门、项目、费用类型、供应商、城市、负责人和来源维度可返回 CFO 看板相应筛选；切换维度时移除旧机会 ID，避免筛选与抽屉详情不一致。 +- 实际结果登记必须具备真实付款事件或可追溯外部凭证；外部凭证上传/连接器尚未接入时，前端隐藏无证据手工登记并解释下一步，不发送必然失败或可能污染价值账本的空证据请求。 +- 真实为 0、无数据、基线不足、无权限、接口失败、部分数据和快照过期使用不同状态。 +- 禁止复用 `data/metrics.js`、`BudgetCenterView` 静态种子或遗留 `demoTotals` 作为 CFO 真实回退。 + +### 后端 + +- `SavingsDiscoveryService` 只负责从可信业务事实发现/创建机会，不提交事务。 +- `SavingsActionService` 负责机会状态动作、版本、权限、幂等和事件。 +- `SavingsRealizationService` 负责付款后实际结果、财务确认、拒绝和冲回。 +- `SavingsQueryService` 负责租户安全分页、详情和可见动作投影。 +- `SavingsFactScopeReader` 只读取当前租户与授权部门中的已归档报销事实，并以报告窗口和 `as_of` 排除未来单据、修改和完成事件。 +- `SavingsBaselineGenerationService` 分开冻结金额中位数与流程历时中位数；流程只使用提交时间和首个付款完成业务事件，指标固定为 elapsed minutes。 +- `SavingsAnomalyAttributionAnalyzer` 只生成描述性异常集中归因和政策模拟准备项，不声称因果，不写 `SavingsOpportunity`。 +- `CfoValueAnalyticsService` 只从 Savings Ledger、风险事实和明确资格快照聚合，不从 UI mock 或风险金额推导节省。 +- 标准重算只使用数据库行锁中的 `ExpenseClaimItem.item_amount` 作为原金额；客户端原金额和可报销金额只可作为展示输入，不能成为节省事实。 +- 标准重算在同一事务内写 Claim 调整、机会、证据、Savings 事件和 `saving_opportunity_created` 业务事件；API 边界统一提交。 +- 付款动作在 Claim → Opportunity 的固定锁顺序中创建 actual realization 和 `saving_action_completed`，与 `payment_completed` 同事务。 +- 财务确认写 `saving_confirmed`，拒绝和冲回写对应只追加事件；相同请求安全重放。 +- 旧 `/analytics/finance-dashboard` 必须接收可信 `CurrentUserContext`，Claim 通过 `ExpenseCaseLink` 限定租户；快照键至少包含租户与数据权限范围，后台任务必须显式指定租户。 + +### 算法与规则 + +#### 第一条可信机会 + +```text +server_original_amount = locked ExpenseClaimItem.item_amount +policy_target_amount = server policy calculator result +estimated_net_saving = max(0, server_original_amount - policy_target_amount) +``` + +- 仅当政策计算成功、输入快照完整、币种一致、差额大于 0 时创建可货币化机会。 +- 机会唯一键首期为 `tenant + claim + item + policy_version + policy_input_fingerprint`。 +- 接受重算表示建议已采纳，机会进入 `in_progress`；付款完成后进入 `realized`，独立财务确认后进入 `verified`。 +- 员工自行承担差额同时是员工体验护栏，必须跟踪申诉和例外补付率，防止通过不合理转嫁美化节省。 + +#### 流程基线、异常归因与政策模拟 + +```text +workflow_elapsed_minutes += first_tenant_payment_completed_event.occurred_at - claim.submitted_at +``` + +- 流程窗口按首个付款完成事件归属；提交时间缺失、完成早于提交、跨租户事件、`as_of` 之后完成或截止后被修改的单据全部排除。 +- 流程快照使用 `median_submission_to_payment_elapsed_minutes`、`minutes` 单位、独立算法版本和来源指纹；证据元数据固定声明 `elapsed_cycle_not_active_labor`。 +- 异常归因按部门、费用类型、城市和项目聚合质量合格的历史偏离候选，只表示异常集中度，不表示该维度导致支出。 +- 政策模拟候选只输出版本化政策、生效期、适用范围、限额和例外规则等必需输入；历史中位数不是政策反事实，缺少正式反事实时 `estimated_savings=None`。 +- 预算预测复用现有预算分配和核销事实，以 `min(as_of, window_end)` 为截止点；旧预算表没有租户字段时仅允许 default 租户，部门权限优先按稳定部门 ID 收紧。 +- 供应商缺少核验 ID、数量和单位价格时继续返回 unavailable，不读取 `AccountsPayableRecord` 演示或应付种子。 + +#### 收益去重 + +- `benefit_key` 表达同一个经济结果，不表达同一个风险观察。 +- 多条风险可指向一个机会；同一发票、付款义务、报销明细或价格变化只能有一个 canonical 确认收益。 +- 同一收益多个动作的归因比例之和不得超过 1。 +- 确认后大额、超预计、手工基线、缺外部凭证和归因异常进入二次复核或数据质量队列。 + +#### 状态转换 + +```text +identified -> accepted -> in_progress -> realized -> verified -> reversed + | | | | + +-------- rejected -------+------------+ + +-------- expired --------+ +``` + +- `identified`：冻结基线、方法、价值类型、币种、预计净值、负责人、截止时间、去重键和来源证据。 +- `accepted`：负责人明确采纳。 +- `in_progress`：保存执行动作、执行人、开始时间和动作证据；预计值不得静默上调。 +- `realized`：保存实际结果、净值、发生时间、归因和付款/结果证据，但不计主 KPI。 +- `verified`：完成去重、币种、成本、证据和独立财务确认。 +- `reversed`：追加负向冲回，原确认不可删除。 +- `rejected/expired`：保留失败事实，防止只保留成功机会美化兑现率。 + +### 数据与契约 + +#### `profile_baseline_snapshots` + +- 租户、基线类型、稳定维度 ID、指标、单位和原币。 +- 基线值、窗口开始/结束、样本量、方法、查询指纹和数据质量。 +- 算法版本、政策版本、冻结时间/人和有效期。 +- 历史群组基线强制窗口与样本量；政策反事实基线强制政策版本、生效区间和目标明细。 +- 金额基线按员工、部门、费用类型、城市和项目分组；流程基线按稳定流程键分组，使用独立 metric/unit，不能与币种金额比较或求和。 + +#### `savings_opportunities` + +- 租户、费用事件、Claim 软引用、来源类型/ID、类别和价值类型。 +- 风险暴露只作护栏；基线、目标、预计毛收益、预计成本、预计净收益和区间分开保存。 +- 原币、报告币、负责人、截止时间、状态、版本、`benefit_key` 和去重组。 +- 部门、项目、费用类型、供应商、城市和流程维度使用明确快照字段或受控 JSON。 +- 唯一约束至少覆盖 `(tenant_id, opportunity_key)`。 + +#### `savings_realizations` + +- 租户、机会、费用事件、Claim、BusinessEvent 和实际发生时间。 +- 实际毛收益、新增执行成本、实际净收益、原币、报告金额和汇率快照。 +- 归因方法/比例、`benefit_key`、去重状态和 canonical realization。 +- 财务确认/拒绝/冲回人、时间、说明和证据。 +- 只追加金额事实；确认投影可更新，但每次变更必须有不可变事件。 + +#### `savings_evidence_links` 与 `savings_events` + +- 证据保存实体、证据角色、资源类型/ID、来源系统、外部事件 ID、内容哈希、发生/采集时间和验证状态。 +- 事件保存动作、请求 ID、操作人、版本、指纹、before/after、首次响应、correlation 和时间。 +- PostgreSQL 触发器禁止修改或删除 `savings_events`。 + +### 权限 + +- `finance`、`executive` 可读取租户 CFO 汇总；预算负责人仅看被授权部门/成本中心。 +- 普通员工、普通经理不得读取 CFO 汇总；只能看到本人费用事件中的最小调整说明。 +- 机会接受、拒绝和指派需要 finance/executive 或明确负责人权限。 +- 财务确认必须是 finance/executive，且不能是机会负责人或实际结果填报人。 +- 只有 `admin` 而没有财务角色时允许运维只读，不允许业务确认。 +- 所有 API、聚合、快照、导出和后台任务强制 `tenant_id` 与数据范围；不允许默认全表扫描。 +- Claim 通过 `ExpenseCaseLink` 校验租户；缺 Link 的非默认历史数据不自动猜测归属。 + +### 降级策略 + +- 政策或基线服务失败：不创建货币化机会，原报销主流程保留人工处理。 +- 外部付款/ERP 连接器未接入：付款业务事件只能推进到 realized，必须人工财务确认。 +- 汇率缺失：保留原币明细，不进入跨币种总计。 +- 工时基线缺失：显示“待采集”，不显示 0，不计扩展 ROI。 +- 流程历时可用但活跃工时缺失：只展示 elapsed cycle 驱动指标，CFO 工时价值仍保持 collecting。 +- CFO 聚合失败：显示错误和重试，不加载演示值；旧财务支出看板独立可用。 +- 快照过期：展示过期提示并触发受控刷新，不能跨租户复用旧快照。 + +## 算法与公式 + +### 主 KPI 1：财务确认净现金节省 + +```text +verified_net_cash_savings += sum(actual_gross_saving - incremental_execution_cost + reversal_amount) +where value_kind = cash + and confirmation_status = finance_confirmed + and dedupe_status = canonical + and confirmed_at <= report_as_of +``` + +- 按 `realized_at` 归属业务期间，按 `confirmed_at` 和报告 `as_of` 保证历史可回放。 +- 风险暴露、预计金额、执行中金额和未确认实际金额不得进入。 +- 多币种只有存在锁定汇率时才折算；否则按原币分组。 + +### 主 KPI 2：财务确认可释放工时价值 + +```text +verified_releasable_labor_value += max(0, baseline_active_minutes_per_unit - actual_active_minutes_per_unit) + * eligible_units + * approved_role_cost_per_minute + * approved_releasable_ratio +``` + +- 现金与工时分账、分卡、分报告，默认不相加。 +- 缺少上线前后活跃分钟、角色完全成本、生效期或客户认可释放比例时不可计算。 + +### 主 KPI 3：安全智能直通率 + +```text +safe_straight_through_rate += qualified_completed_cases_without_manual_correction_or_return + and no_major_post_audit_issue + / eligible_completed_cases_frozen_at_creation +``` + +- 必须保存 eligibility 快照、策略版本、必要审批完成和事后抽检结果。 + +### 驱动指标 + +- 现金兑现率：同一成熟机会队列的财务确认净现金 / 冻结预计净现金。 +- 机会到财务确认 P50 天数、逾期负责人占比。 +- 提交至首个付款完成的端到端 P50 elapsed minutes；它与每单人工活跃分钟分开，后者在采集前保持不可用。 +- 人工触点、首次提交完整率和 AI 字段采纳率。 + +### 风险护栏 + +- 开放且已确认的高危/重大风险暴露，按单据或经济义务去重；它不是节省。 +- 重大风险漏检率、事后审计重大问题率、误报率和人工覆盖率。 +- 财务确认后冲回率、实际超过预计异常率、去重待复核金额和证据不完整金额。 +- 标准重算员工申诉/例外补付率。 + +## 测试方案 + +### 后端 + +- 状态机合法/非法转换、确认人独立性、管理员只读和角色权限。 +- 机会/实现/确认/冲回幂等、请求内容冲突、陈旧版本和租户隔离。 +- 服务端明细金额锁定、客户端放大原金额无效、政策快照完整性。 +- 付款事件重复、经济收益去重、跨币种、成本扣除、冲回和归因上限。 +- 看板按租户、时间、部门、项目、费用类型、城市、来源和负责人聚合对账。 +- 六维基线验证员工、部门、费用类型、城市、项目和流程的窗口、样本量、算法版本、来源指纹、租户/部门范围与稳定重放。 +- 洞察验证预算截止点、描述性归因、政策模拟准备项、供应商 unavailable、所有无反事实候选 `estimated_savings=None` 且不会创建机会。 +- 旧财务看板 Claim、预算和缓存键租户隔离回归。 +- Alembic 空库升级、重复升级、约束、append-only 触发器、无损降级和 PostgreSQL 并发。 + +### 前端 + +- API snake/camel 归一化、部分数据和过期快照。 +- verified、realized、estimated 和 risk exposure 严格分区，不得混算。 +- loading/error/empty/partial/stale/permission-denied/baseline-missing 状态。 +- 时间、部门、费用类型、价值类型筛选与 URL 恢复。 +- 机会详情证据链、可用动作、版本冲突、幂等重放和确认反馈。 +- 单据、风险、预算和维度下钻参数。 +- 机会 `value_opportunity` 的恢复、关闭清理、非法格式、403/404、跨筛选和时间窗口清理。 +- 风险来源最小定位参数、单据返回 CFO、预算配置焦点和“非真实预算金额”口径。 +- 响应式、键盘操作、44px 触控目标和生产构建。 + +### 集成 + +- 住宿超标准 → 服务端重算 → 机会 → 审批 → 付款 → actual realization → 独立财务确认 → CFO 看板。 +- 相同重算/付款/确认并发只产生一个经济收益和一条对应版本事件。 +- 确认后补付/申诉 → 负向冲回 → 历史报告 `as_of` 可回放。 +- 多租户同单号、同员工名、相同 request ID 和快照缓存隔离。 + +### 容器验证 + +所有 pytest、Alembic、PostgreSQL 并发、前端测试和构建必须在 `local-x-financial-linux` 容器内完成，单条命令超时不超过 60 秒。 + +## 指标与验收 + +- [A1] 任一 verified cash saving 可追溯到费用事件、明细原金额、政策快照、执行动作、付款事件、确认人、去重键和证据。 +- [A2] 风险暴露、预计、执行中、实际待确认、财务确认和冲回在 API、数据库和 UI 中均不混算。 +- [A3] 客户端伪造原金额、跨租户访问、陈旧版本、自证确认和重复经济收益被服务端拒绝。 +- [A4] CFO 三个主 KPI 有口径、时间窗口、来源、新鲜度、数据覆盖和护栏；缺数据时明确“待采集”。 +- [A4.1] 流程 elapsed cycle 有独立 metric/unit/算法版本/来源指纹，且不会进入工时价值或现金节省。 +- [A5] 价值看板支持部门、项目、费用类型、供应商、城市、时间、负责人、来源和单据下钻。 +- [A6] 真实接口失败不出现演示数字；零值、无数据、无权限、错误和过期可区分。 +- [A7] 迁移在一次性 PostgreSQL 空库完成升级、重复升级、约束验证和安全降级边界测试。 +- [A8] 相关后端、前端、Ruff、构建、端到端和并发测试全部在容器内通过。 + +## 风险与开放问题 + +### 风险 + +- 员工承担差额不一定等同企业创造价值；需要跟踪申诉、补付和政策公平性，避免激励扭曲。 +- 当前付款是人工状态，不是外部现金事实；确认页必须清晰披露证据等级。 +- 旧 Claim/预算缺租户列，读取必须经过 Case Link 或正式迁移，不能依赖默认租户猜测。 +- 数据稀疏且包含模拟种子，试点目标值必须在真实基线采集后冻结。 +- 当前预算中心仍是配置演示视图；CFO 来源入口只用于定位部门/费用类型配置，页面和价值计算均不得把其中金额当成预算事实或节省事实。 +- 多币种、分摊收益和跨期冲回会显著增加财务口径复杂度，必须保留原始事实和版本。 +- 工时价值若没有活跃时间采集与客户认可成本率，容易被夸大，因此默认不计现金 ROI。 +- 历史异常集中不是因果，历史中位数也不是政策反事实；模拟候选必须在正式政策版本和客户确认适用口径补齐后才可货币化。 + +### 已处理决策 + +- 首条闭环选择“住宿职级标准重算”，不选择缺少付款事实的重复支付阻止。 +- 价值看板进入现有分析看板，独立拆组件和 composable。 +- 只设 3 个主 KPI，其余作为驱动和护栏；现金与工时分开披露。 +- Savings Ledger 直接带结构化租户键，不复用旧财务快照作为价值事实源。 +- 流程基线采用提交至首个付款完成的端到端 elapsed cycle；人工活跃工时继续作为独立待采集事实。 +- 异常归因仅做描述性集中度，政策模拟候选保持只读，不自动写入节省机会。 + +### 待后续真实客户确认 + +- 独立财务确认是否要求双人复核及大额阈值。 +- 报告币、汇率来源和月末汇率锁定规则。 +- 工时价值是否进入扩展 ROI、角色成本口径和可释放比例。 +- 标准重算差额在客户会计政策中属于现金节省、成本避免还是员工自担调整。 +- 试点 30/90 天目标、CFO 月报签字人和节省分成合同边界。 + +## 本轮实现记录 + +- 2026-07-16：完成现有财务、预算、风险、付款、标准重算、前端入口与 KPI 口径盘点；确认旧财务聚合租户边界和客户端原金额信任问题。 +- 2026-07-16：冻结 Savings Ledger、CFO KPI、状态机、权限、去重、证据和首条纵向闭环方案；尚未完成的代码与验证全部保留在同目录 TODO 中继续执行。 +- 2026-07-16：完成 Savings Ledger 五表与 0015 迁移、租户/权限/幂等/并发/证据/冲回契约，并把住宿标准重算和付款动作接入同事务价值链。 +- 2026-07-16：完成独立财务确认、证据复核留痕、canonical 收益去重、负向冲回、`as_of` 回放和历史标准调整安全回填。 +- 2026-07-16：完成 CFO 价值分析 API；只将已确认 canonical 现金结果计入主 KPI，多币种分组，工时、安全直通率和审计事实缺口显式披露，不使用演示数据回退。 +- 2026-07-16：完成 CFO 经营价值前端入口、URL 筛选恢复、主 KPI、漏斗、趋势、驱动维度、护栏、数据质量、机会证据链与响应式状态；详情明确展示财务确认人、时间和说明。 +- 2026-07-16：收紧无证据手工登记边界。真实付款事件仍可自动形成待确认实际结果；在支付/银行/ERP 凭证连接器接入前，页面不再提交空证据，并向用户解释必须先完成付款或等待外部回执。 +- 2026-07-16：完成租户隔离的员工、部门、费用类型、城市和项目五维历史中位数冻结基线，以及预算预测偏差、重复小额模式和历史中位数偏离候选；非政策反事实信号只披露暴露，不生成节省金额，并补齐 `as_of` 基线时间一致性门禁。 +- 2026-07-16：补齐第六维流程周期基线，以同租户首个付款完成业务事件冻结提交到完成的 elapsed minutes；窗口、样本量、算法版本、来源指纹、质量状态和证据完整保存，明确禁止推算人工活跃工时。 +- 2026-07-16：新增部门/费用类型/城市/项目异常集中归因和版本化政策模拟准备项；复用预算预测并收紧配置/交易截止点，所有缺少反事实的候选保持 `estimated_savings=None`、零机会副作用，供应商分析继续明确 unavailable。 +- 2026-07-16：完成机会抽屉 `value_opportunity` 深链、刷新/前进/后退恢复和非法/越权/跨筛选安全清理；来源动作可跳关联单据、风险证据位置、预算配置视图及 CFO 维度筛选，单据返回时恢复经营价值查询状态。 +- 2026-07-16：预算来源只应用授权部门与费用类型配置焦点；未覆盖科目显示“未找到配置”，所有入口和页面均明确演示预算金额不是当前机会的真实预算事实。 diff --git a/document/development/2026-07-16/feature/savings-ledger-and-cfo-value/TODO.md b/document/development/2026-07-16/feature/savings-ledger-and-cfo-value/TODO.md new file mode 100644 index 0000000..a2649fe --- /dev/null +++ b/document/development/2026-07-16/feature/savings-ledger-and-cfo-value/TODO.md @@ -0,0 +1,141 @@ +# 节省事实账本与 CFO 经营价值看板 开发 TODO + +更新时间：2026-07-17 + +## 使用规则 + +- 每条 TODO 必须回链 `CONCEPT.md` 的章节或语义段落。 +- 只有代码、接口或容器验证提供真实证据后才能勾选 `[x]`。 +- 现金节省、工时价值、风险暴露和预计机会始终分开，不以演示数据代替缺失事实。 +- 实施顺序为：安全前置 → 账本 → 首条闭环 → 基线/分析 → CFO 前端 → PostgreSQL/E2E → 文档收口。 + +## 1. 调研与边界 + +- [x] [CONCEPT: 背景与问题] 盘点财务看板、预算、Claim、Expense Case、Business Event、风险、审批、付款和标准重算事实源。 + 证据：`finance_dashboard.py`、`finance_dashboard_snapshot.py`、`financial_record.py`、`expense_cases.py`、`expense_claim_standard_adjustment.py`、`expense_claim_approval_flow.py` 的只读审计。 +- [x] [CONCEPT: 目标与非目标] 确认风险金额、未用预算、预计机会和未确认结果不得进入已确认节省。 + 证据：CONCEPT「目标与非目标」「算法与公式」。 +- [x] [CONCEPT: 用户与场景] 选择住宿职级标准重算作为首条纵向闭环，不选择证据不足的重复支付或供应商议价。 + 证据：现有标准重算和付款业务事件可形成最短可信证据链；`AccountsPayableRecord` 已有租户字段，但仍缺合同价、数量、单位价格和外部付款事实。 +- [x] [CONCEPT: 风险与开放问题] 识别旧财务聚合全表读取、快照键缺租户、接口缺角色限制和标准重算信任客户端原金额问题。 + 证据：`FinanceDashboardMetricMixin._fetch_claims()`、`_fetch_budget_allocations()`、`FinanceDashboardSnapshotService._cache_key()`、`accept_standard_adjustment()`。 + +## 2. 契约与设计 + +- [x] [CONCEPT: 数据与契约] 定义 baseline、opportunity、realization、evidence 和 append-only event 的职责与关键字段。 + 证据：CONCEPT「数据与契约」。 +- [x] [CONCEPT: 算法与规则] 定义 identified → accepted → in_progress → realized → verified → reversed 状态和 rejected/expired 终态。 + 证据：CONCEPT「状态转换」。 +- [x] [CONCEPT: 算法与规则] 定义 benefit key、canonical realization、归因比例和追加冲回规则。 + 证据：CONCEPT「收益去重」。 +- [x] [CONCEPT: 权限] 定义租户、finance/executive、预算范围、管理员只读、自证禁止和数据范围。 + 证据：CONCEPT「权限」。 +- [x] [CONCEPT: 算法与公式] 定义 3 个主 KPI、驱动指标、风险护栏和缺数据边界。 + 证据：CONCEPT「算法与公式」。 + +## 3. 安全前置 + +- [x] [CONCEPT: 后端] 让旧财务看板显式接收可信租户与数据范围，Claim/预算查询不得全表混算。 + 证据：`finance_dashboard_access_policy.py`、`finance_dashboard_scope.py`、`finance_dashboard_budget.py`、`finance_dashboard.py`；Claim 直接按结构化 `ExpenseClaim.tenant_id` 首层过滤，预算历史兼容仅限 default 租户。 +- [x] [CONCEPT: 后端] 给财务快照缓存键和后台任务加入租户/数据范围，禁止跨租户复用。 + 证据：`finance_dashboard_snapshot.py`、`finance_dashboard_scheduler.py`；快照键包含 tenant 与 scope fingerprint，调度入口必须显式携带租户。 +- [x] [CONCEPT: 权限] 给财务与 CFO 分析接口增加后端角色和范围校验，管理员身份不自动获得业务确认权。 + 证据：`finance_dashboard_access_policy.py`、`savings_access_policy.py`、`agent_run_access_policy.py`、`GET /api/v1/analytics/cfo-value`；容器租户/缓存/角色回归 17 项通过。 +- [x] [CONCEPT: 第一条可信机会] 标准重算只使用锁定数据库明细原金额，客户端原金额不能影响结果和节省。 + 证据：`expense_claim_standard_adjustment.py`；原金额只读锁定 `ExpenseClaimItem.item_amount`，政策结果只由服务端重算，陈旧版本使用内容指纹并降低基线质量等级。 +- [x] [CONCEPT: 后端] 将标准重算、审计和账本写入收口到同一事务边界。 + 证据：`expense_claim_standard_adjustment.py`以 `commit=False` 写审计，同一 API 事务写 Claim、Baseline、Opportunity、Evidence、SavingsEvent 和 BusinessEvent；事务失败整体回滚。 + +## 4. Savings Ledger 后端 + +- [x] [CONCEPT: 数据与契约] 新增 `profile_baseline_snapshots`、`savings_opportunities`、`savings_realizations`、`savings_evidence_links` 和 `savings_events` 模型。 + 证据：`savings.py`（模型）及 `test_savings_models.py`；五类事实分离保存基线、机会、结果、证据和不可变操作。 +- [x] [CONCEPT: 数据与契约] 新增 Alembic 0015、迁移所有权、复合租户外键、唯一键、检查约束、索引和 append-only 触发器。 + 证据：`20260716_0015_savings_value_ledger.py`、`migration_preflight.py`、`schema_ownership.py`；一次性 PostgreSQL 17 空库完整迁移循环通过。 +- [x] [CONCEPT: 后端] 实现 discovery、action、realization、query、access policy 和 response builder 独立服务。 + 证据：`savings_discovery.py`、`savings_actions.py`、`savings_realization.py`、`savings_query.py`、`savings_access_policy.py`、`savings_read_projection.py`、`savings_protocol.py`。 +- [x] [CONCEPT: 后端] 实现分页、筛选、排序、详情、可用动作、证据和事件 DTO。 + 证据：`GET /api/v1/savings/opportunities`、`GET /api/v1/savings/opportunities/{id}`、`savings.py`（schema/API）、`test_savings_endpoints.py`。 +- [x] [CONCEPT: 算法与规则] 实现版本锁、请求指纹、不可变响应重放、收益去重和归因比例约束。 + 证据：`SavingsRequestProtocol`、数据库唯一/检查约束与 `test_savings_concurrency_postgres.py`；同 request 并发仅一次写入，同 benefit 双确认仅一个 canonical winner。 +- [x] [CONCEPT: 状态转换] 实现 accept/start/record/verify/reject/expire/reverse 合法与非法转换。 + 证据：`savings_actions.py`、`savings_realization.py`、`test_savings_ledger_services.py`；负向 reversal 只追加，不改写原已确认金额。 +- [x] [CONCEPT: 证据与审计] 把 Savings 关键动作写入同 Case 的 `BusinessEvent` 并保留 correlation/causation。 + 证据：`ExpenseCaseService` 资源链接与 `savings_*` 服务；端到端测试核验 opportunity/payment/action/confirm/reverse 事件同 Case 可回放。 +- [x] [CONCEPT: 后端] 新增历史标准重算机会 dry-run/apply 回填脚本；缺 Case/租户/政策证据只输出数据质量报告。 + 证据：`savings_standard_adjustment_backfill.py`、`backfill_standard_adjustment_savings.py`；显式租户/目标库、指纹重算、批次锁、稳定键重放，容器直接测试 8 项通过。 + +## 5. 首条真实纵向闭环 + +- [x] [CONCEPT: 第一条可信机会] 接受住宿标准重算时冻结服务端原金额、政策输入/版本、目标金额、差额、维度和证据。 + 证据：`SavingsDiscoveryService`冻结 `ProfileBaselineSnapshot` 与内容指纹，政式发布版本为 complete，内容指纹版本显式标记 partial。 +- [x] [CONCEPT: 第一条可信机会] 同事务以稳定 opportunity key 创建或重放机会，并进入 in_progress。 + 证据：稳定键由 tenant + claim + item + policy version + calculation fingerprint 生成；重试返回原机会。 +- [x] [CONCEPT: 后端] 付款业务事件后同事务创建 actual realization，重复付款事件不重复计入。 + 证据：`expense_claim_approval_flow.py`调用 `realize_paid_claim()`；PostgreSQL 同付款事件并发结果为 `[0, 1]`，仅一条 realization 和一条完成事件。 +- [x] [CONCEPT: 财务确认] 实现独立财务 verify/reject，未确认实际结果不得进入主 KPI。 + 证据：所有人工实际结果必须至少一条可追溯证据；独立确认人会固化证据复核人/时间，pending 结果在 CFO 口径中为 0。 +- [x] [CONCEPT: 冲回能力] 实现补付/申诉/归因修正的负向 reversal，原确认不可删除。 + 证据：原 actual 保留 `finance_confirmed`，新增负向 canonical reversal；`as_of` 可回放冲回前结果。 +- [x] [CONCEPT: 权限] 验证申请人、机会负责人、结果填报人、纯管理员和跨租户用户不能自证或越权。 + 证据：`SavingsAccessPolicy`与 PostgreSQL 并发安全测试；owner/recorder/admin-only 自证拒绝，独立 finance 可确认，跨租户无副作用。 + +## 6. 费用基线与经营分析 + +- [x] [CONCEPT: 基线快照] 按员工、部门、费用类型、城市、项目和流程从租户安全真实数据生成基线窗口、样本量和版本。 + 证据：`savings_fact_scope.py`、`savings_baseline_generation.py`、`savings_insights.py`；金额五维使用已归档明细中位数，流程维度只使用同租户报销提交时间与首个 `payment_completed` 业务事件，冻结 elapsed minutes、窗口、样本量、算法版本、查询指纹、质量、证据和审计事件；`source_workflow_cycle_count` 明确披露覆盖，活跃工时保持 unavailable。 +- [x] [CONCEPT: 基线快照] 供应商数据缺少租户/合同事实时显示 coverage gap，不从应付模拟种子生成可信基线。 + 证据：基线和洞察 API 均返回 `supplier_dimension_unavailable` / `supplier_price_drift_unavailable`，要求核验供应商、数量和单位价格；不会读取 `AccountsPayableRecord` 模拟事实或创建货币化机会。 +- [x] [CONCEPT: 算法与规则] 实现预算预测、描述性异常归因、重复小额浪费、历史偏离和只读政策模拟准备项；证据不足时不自动货币化。 + 证据：`savings_insight_budget.py` 复用预算配置/核销事实并以 `min(as_of, window_end)` 截止；`savings_insight_analysis.py`、`savings_insight_attribution.py` 输出历史偏离、部门/费用类型/城市/项目异常集中和版本化政策模拟必需输入。所有缺少正式反事实的候选 `estimated_savings=None`、`created_opportunity_ids=[]`，稳定重放不写 `SavingsOpportunity`。 +- [x] [CONCEPT: 后端] 实现 `GET /analytics/cfo-value`，统一时间、维度、币种、状态和 `as_of` 口径。 + 证据：`cfo_value.py`（API/schema）、`cfo_value_analytics.py`；租户、角色、预算范围、时间和维度过滤均在服务端执行。 +- [x] [CONCEPT: 算法与公式] 实现确认现金、工时待采集、安全直通率资格、漏斗、兑现周期、逾期、来源和护栏聚合。 + 证据：确认现金只求和 finance_confirmed + canonical + cash；工时和安全直通率显式返回 collecting；漏斗、趋势、驱动、逾期、冲回和数据质量不与主 KPI 混算。 +- [x] [CONCEPT: 降级策略] 多币种、工时、外部付款和审计结果缺失时返回明确数据质量状态，不伪造 0。 + 证据：多币种按原币分组；无人工活跃时间/审计事实时返回 collecting/unavailable 和 coverage gap，不使用 mock 回退。 + +## 7. CFO 前端 + +- [x] [CONCEPT: 前端] 在分析看板新增 `value` 入口，并把 dashboard 状态同步 URL。 + 证据：`useTopBarOverviewRange.js`、`AppShellRouteView.vue`、`OverviewView.vue`；`dashboard=value` 与价值筛选/页码写入 query，刷新和返回可恢复。 +- [x] [CONCEPT: 前端] 新增独立 CFO 组件、composable、API service 和展示模型，不继续扩大 `useOverviewView.js`。 + 证据：`CfoValueDashboard.vue`、`CfoValueTrendChart.vue`、`CfoValueOpportunityDrawer.vue`、`CfoValueActionDialog.vue`、`useCfoValueDashboard.js`、`analyticsValue.js`、`cfoValueDashboardModel.js`；业务组件和状态职责已拆分，核心文件均低于 800 行。 +- [x] [CONCEPT: CFO 看板] 实现主 KPI、护栏、价值漏斗、趋势、来源/组织驱动、机会表和数据质量。 + 证据：`CfoValueDashboard.vue` 与 `cfo-value-dashboard.css`；现金、工时和直通率分卡，趋势按币种切换，预计/实际/确认/冲回不混算，缺数据显式展示。 +- [x] [CONCEPT: 前端] 实现时间、部门、费用类型、价值类型筛选和项目/供应商/城市/负责人高级筛选。 + 证据：顶部时间窗口与价值 query 联动，`createEmptyValueFilters`、`readValueFiltersFromQuery`、`writeValueFiltersToQuery` 和 API query 白名单覆盖全部筛选字段。 +- [x] [CONCEPT: 前端] 实现基线、建议、执行、实际、确认、去重和证据详情。 + 证据：`CfoValueOpportunityDrawer.vue` 展示冻结基线、机会状态、实际净值、记录人、canonical 去重、财务确认人/时间/说明、证据索引和不可变事件；无可追溯凭证时不开放手工实际结果登记。 +- [x] [CONCEPT: 前端] 实现单据、风险、预算、维度下钻与返回状态恢复。 + 证据：`cfoValueSourceLinks.js` 统一构造来源路由；Claim 进入 `app-document-detail`，风险携带最小 focus/观察/决策参数和现有锚点，预算进入 `app-budget` 配置视图并应用部门/费用类型焦点，维度返回 `app-overview?dashboard=value` 相应筛选。`useCfoValueDashboard.js` 以 `value_opportunity` 恢复抽屉，并在非法 ID、403/404、跨租户不可见或不符合当前筛选/时间窗口时安全清除；`useAppShell.js` 从单据详情恢复 CFO 查询。限制：预算中心仍是演示配置视图，页面明确金额不是当前机会的真实预算事实；未覆盖费用科目显示未配置而不是零预算。 +- [x] [CONCEPT: 降级策略] 区分零、无数据、基线不足、无权限、失败、部分数据和快照过期；删除 CFO 演示回退。 + 证据：`classifyCfoDashboardState`、`buildValueKpis` 与页面状态区；接口失败不读取 `data/metrics.js` 或 demo/fallback 数字。 +- [x] [CONCEPT: 前端] 完成移动端、键盘、焦点、44px 触控和无障碍状态提示。 + 证据：CFO 样式移动断点、44px 按钮、语义化 `label`/`role=alert`/`aria-live`、抽屉关闭标签及趋势表格降级；生产构建通过。 + +## 8. 测试与验证 + +- [x] [CONCEPT: 测试方案] 后端状态、金额、权限、租户、幂等、证据、去重、冲回和看板聚合单测通过。 + 证据：容器组合回归 84 项通过；本轮 Savings/CFO 基线、洞察、端点、账本、回填与 E2E 组合 `34 passed, 6 skipped`，6 项为未配置 PostgreSQL 专用 URL 的预期跳过。 +- [x] [CONCEPT: 测试方案] 标准重算客户端金额伪造、政策失败、Case 缺失和事务回滚测试通过。 + 证据：`test_expense_claim_service.py -k standard_adjustment` 8 项加付款集成 1 项通过，HTTP 标准重算 1 项通过。 +- [x] [CONCEPT: 测试方案] 前端数据归一化、状态、筛选、URL、证据动作和响应式测试通过。 + 证据：容器内 `cfo-value-dashboard.test.mjs` 14 项通过，新增机会 URL 恢复/清理、筛选上下文、风险/单据/预算/维度链接和预算非事实口径断言；与 App Shell 返回链、路由加载和筛选样式组合回归 37 项通过；带凭证序列化和无证据入口 fail-closed 均有断言。 +- [x] [CONCEPT: 测试方案] 一次性 PostgreSQL 空库迁移、重复升级、约束、append-only、无损降级和并发测试通过。 + 证据：一次性 PostgreSQL 17 迁移循环通过；`test_savings_concurrency_postgres.py` 6 项通过，覆盖重放、canonical 竞态、跨租户、独立确认、付款单事实和 append-only DB 触发器。 +- [x] [CONCEPT: 集成] 住宿标准重算 → 机会 → 审批 → 付款 → 实际 → 财务确认 → CFO 看板 E2E 通过。 + 证据：`test_savings_value_e2e.py` 从服务端政策差额、付款动作、待确认排除、独立财务确认到 CFO 金额对账单项通过。 +- [x] [CONCEPT: 集成] 确认后负向冲回和报告 `as_of` 回放 E2E 通过。 + 证据：`test_savings_value_e2e.py`与 `test_cfo_value_analytics.py`同时验证当前净值归零和冲回前历史金额回放。 +- [x] [CONCEPT: 容器验证] 相关 pytest、Ruff、前端测试和生产构建均在 `local-x-financial-linux` 内通过。 + 证据：Savings/CFO 相关切片与端到端均通过；fresh PostgreSQL 总探针 `87 passed / 0 skipped / 0 failed`，其中 Savings 并发 6 项；Web 全量 `815 passed / 0 failed` 与 Vite build 通过；新增 Python 文件 Ruff 和 `git diff --check` 通过。 + +## 9. 文档收尾 + +- [x] [CONCEPT: 指标与验收] 逐项核对 A1-A8，并把文件、接口、迁移、测试和运行结果写回证据。 + 证据：A1 纵向闭环由 `test_savings_value_e2e.py`；A2-A4/A4.1 由 Savings schema、`cfo_value_analytics.py`、基线/分析测试；A5-A6 由 CFO 组件、来源下钻和前端状态测试；A7 由 0015 迁移与 PostgreSQL 并发；A8 由本节最终容器汇总证明。 +- [ ] [CONCEPT: 风险与开放问题] 记录付款证据等级、汇率、双人复核、工时口径和试点目标的最终边界。 + 证据： +- [x] [CONCEPT: 本轮实现记录] 同步更新上位 `ai-expense-closed-loop-and-value-proof` 文档，不删除历史证据。 + 证据：上位 TODO 已回填 Savings Ledger、状态机、CFO 看板、下钻、证据详情和 E2E；历史证据原样保留。 diff --git a/document/development/2026-07-17/dev-logs/bugs/agent-asset-tenant-isolation-and-onlyoffice-security.md b/document/development/2026-07-17/dev-logs/bugs/agent-asset-tenant-isolation-and-onlyoffice-security.md new file mode 100644 index 0000000..029b55a --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/agent-asset-tenant-isolation-and-onlyoffice-security.md @@ -0,0 +1,13 @@ +## 修复记录 + +- 13:08：修复 AgentAsset 全局读写、跨租户子记录注入、风险样本串租户、审核身份伪造和 ONLYOFFICE 匿名/可重放回调问题。 + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；`origin/main` 无新提交，本地 `main` ahead 17，工作区包含多智能体并行未提交变更且未被清理或覆盖。 + - 本地 ahead 摘要：`242d68c3` 审批任务与豁免、`28b834ed` 审批动作幂等回放、`4940ebc4` 风险处置流程、`ee88a36b` 租户安全分层学习、`6bdf65bc` 权威预审、`ae3f02c3` 零录入票据关联、`54754b55` 个人报销记忆、`211f85d9` 已验证申请工作流、`5b246307` 申请预览决策、`a662cfe6` AI 反馈账本、`5ed34c2b` 历史申请回填、`11275e4b` 迁移 ownership 安全、`1347366b` 时间线与草稿事件安全、`22669a90` 统一费用时间线、`a616b30c` AI 申请事务、`653eda05` 不透明会话、`661990b2` 费用案例事务事件。 + - 修改：为 AgentAsset、Version、Review、TestRun 和 RuleFeedback 增加结构化 `tenant_id + scope`；所有资产、发布、监控、召回、遥测、调度、foundation 和风险运行时查询接入企业/平台作用域；同编码按企业优先、平台回退解析，跨企业资源统一不可见，平台资产仅平台管理员可写。 + - 修改：真实风险场景强制显式目标企业并先按 `ExpenseClaim.tenant_id` 过滤；测试证据归目标企业；版本、审核和规则表变更主体改为登录会话中的稳定 employee/username 标识，客户端 actor/reviewer 不能覆盖审计事实。 + - 修改：新增 DB-backed ONLYOFFICE content/callback 会话和 `active → processing → consumed|failed` 原子状态机；token 绑定租户、资源、资产、document key/version/fingerprint、权限、actor、audience、时间和 JTI；平台只读、跨资产、旧版本、错 key、过期和重放回调均拒绝。 + - 修改：回调复用安全下载器，限制配置 origin，校验 DNS 全部地址并固定已验证公网 IP，拒绝重定向、超限、错误 MIME、危险 ZIP 和异常 OOXML；新增 `20260717_0026` 迁移并对无法归属的旧数据和有企业事实的 downgrade fail-closed。 + - 操作：拆出 AgentAsset access/serialization/ONLYOFFICE security 与风险规则字段推断模块；将核心文件控制在 800 行以内；补齐功能 CONCEPT/TODO 和迁移、权限、安全回归测试。 + - 验证：容器内发布/监控/召回/运行时/调度/遥测/租户安全/ONLYOFFICE 汇总 58 项通过；AgentAsset service/foundation 28 项通过；风险生成/修订/golden 49 项通过；相关文件 Ruff、py_compile、ORM mapper（84 张表）和 `git diff --check` 均通过。 + - 验证：一次性 PostgreSQL 探针完成旧平台数据升级、同编码多企业、scope/check、跨租户复合外键和有事实 downgrade 保护；新库 `base → head(0028)` 与 `head → 0025 → head` 均成功，AgentAsset 与 Knowledge ONLYOFFICE 会话表正常创建。 + - 影响：企业只能读取本企业和平台只读资产，无法观察或修改其他企业的资产、版本、审核和测试证据；规则测试不会抽取其他企业费用；审计身份不可由请求伪造；文档回写失败时保持原文件不变，也不会向任意或内部地址发起下载。 diff --git a/document/development/2026-07-17/dev-logs/bugs/agent-asset-tenant-migration-downgrade.md b/document/development/2026-07-17/dev-logs/bugs/agent-asset-tenant-migration-downgrade.md new file mode 100644 index 0000000..d6736be --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/agent-asset-tenant-migration-downgrade.md @@ -0,0 +1,10 @@ +## 修复记录 + +- 13:40：修复 0026 Agent 资产租户安全迁移在真实历史建表路径下无法完整回退的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、`git rev-parse --abbrev-ref --symbolic-full-name @{u}`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；`origin/main` 无新提交，本地 `main` ahead 17，工作区包含多智能体并行变更，未合并、覆盖或提交。 + - 本地 ahead 摘要：17 个提交覆盖审批任务与幂等回放（`242d68c3`、`28b834ed`、`4940ebc4`）、租户安全费用学习/预审/票据关联/申请记忆与反馈（`ee88a36b`、`6bdf65bc`、`ae3f02c3`、`54754b55`、`211f85d9`、`5b246307`、`a662cfe6`）、历史回填与迁移安全（`5ed34c2b`、`11275e4b`）、费用时间线与事务（`1347366b`、`22669a90`、`a616b30c`、`661990b2`）及不透明认证会话（`653eda05`）。 + - 根因：空库先经过 0026 时 Agent 资产旧表尚不存在，迁移会按设计跳过这些表；之后旧表由当前模型补建，PostgreSQL 为列级租户外键生成 `*_tenant_id_fkey`，而 0026 回退硬编码删除 `fk_*_tenant`，首个 `agent_asset_rule_feedback` 约束不存在即中断。 + - 修改：`20260717_0026_agent_asset_tenant_security.py` 新增带 PostgreSQL 标识符引用的约束安全删除器，0026 回退对它负责的复合外键、租户外键、范围检查和唯一约束统一使用 `DROP CONSTRAINT IF EXISTS`；升级路径与最终升级约束保持不变，模型自动命名的列级外键仍随租户列删除安全清理。 + - 操作：在 `financial-internal` 网络启动独立 `postgres:16-alpine` 一次性数据库，使用应用容器和项目 venv 执行真实 Alembic 循环；验证结束后停止探针，并确认 `--rm` 已删除容器。 + - 验证：相关 Ruff 检查通过；PostgreSQL-only 迁移防误用测试 42 项通过；一次性 PostgreSQL 上的完整迁移循环 1 项通过（7.00 秒），覆盖空库升级至 0028、回退至 0008、再次升级、完整回退至 base、最终再次升级至 0028，同时验证约束存在与缺失两种删除路径。 + - 影响：真实历史路径与当前模型补建路径都能安全回退 0026；缺少迁移命名约束时不再失败，已有约束仍被正常移除，且不会改变 head 升级结构。 diff --git a/document/development/2026-07-17/dev-logs/bugs/commercial-lookup-session-sqlite-rollback.md b/document/development/2026-07-17/dev-logs/bugs/commercial-lookup-session-sqlite-rollback.md new file mode 100644 index 0000000..ab358c9 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/commercial-lookup-session-sqlite-rollback.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 13:25：记录 bug 修复：未配置商业计量器时独立 SQLite Session 回滚调用方事务。 + - Git 提交检查：执行 `git fetch --all --prune` 后，`HEAD..origin/main` 无新提交；本地 `main` ahead 17 个既有提交，依次为 `242d68c3` 审批任务与豁免、`28b834ed` 审批幂等响应、`4940ebc4` 风险处置、`ee88a36b` 分层费用学习、`6bdf65bc` 权威预审、`ae3f02c3` 票据零入口归集、`54754b55` 个人申请记忆、`211f85d9` 申请流程统一、`5b246307` 预览决策、`a662cfe6` 反馈账本、`5ed34c2b` 历史费用 Case 回填、`11275e4b` migration ownership、`1347366b` 时间线与草稿事件、`22669a90` 费用时间线、`a616b30c` AI 申请提交事务、`653eda05` bearer session、`661990b2` 费用 Case 事务事件；这些提交均早于本轮且未改写。 + - 修改：`commercial_direct_operation.py` 增加调用方 `lookup_session` 的只读未配置短路，OCR、RuntimeChat、金融连接器和附件 observer 均显式传入当前 Session。这样没有计量器时不再创建第二个 Session，也不会在 SQLite `StaticPool` 共用连接上意外 rollback 已 flush 的费用明细。 + - 操作：先用附件归集回归复现 `expense_claim_items expected to update 1 row; 0 were matched`，再将未配置判断前移到调用方 Session；保留真正配置计量器时的独立事务预占与结算。 + - 验证：容器内附件归集与票据夹 `31 passed`；商业资源边界 `9 passed`；报销端点与连接器组合 `38 passed`；相关 Ruff 通过。 + - 影响：未启用商业计量的开发、测试和兼容租户不再因为商业探测破坏调用方事务；生产 PostgreSQL 的独立商业事务语义保持不变。 diff --git a/document/development/2026-07-17/dev-logs/bugs/commercial-released-reservation-retry.md b/document/development/2026-07-17/dev-logs/bugs/commercial-released-reservation-retry.md new file mode 100644 index 0000000..35d124a --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/commercial-released-reservation-retry.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 13:25：记录 bug 修复：业务回滚释放预占后相同请求无法安全重试。 + - Git 提交检查：执行 `git fetch --all --prune` 后，`HEAD..origin/main` 无新提交；本地 `main` ahead 17 个既有提交，依次为 `242d68c3` 审批任务与豁免、`28b834ed` 审批幂等响应、`4940ebc4` 风险处置、`ee88a36b` 分层费用学习、`6bdf65bc` 权威预审、`ae3f02c3` 票据零入口归集、`54754b55` 个人申请记忆、`211f85d9` 申请流程统一、`5b246307` 预览决策、`a662cfe6` 反馈账本、`5ed34c2b` 历史费用 Case 回填、`11275e4b` migration ownership、`1347366b` 时间线与草稿事件、`22669a90` 费用时间线、`a616b30c` AI 申请提交事务、`653eda05` bearer session、`661990b2` 费用 Case 事务事件；这些提交均早于本轮且未改写。 + - 修改：`commercial_runtime_reservations.py` 在相同指纹、相同订阅/权益/账期下允许 `released → reserved`，重开时重新锁定合同、校验 meter 快照并执行硬配额判断；跨账期重试继续失败关闭。 + - 操作：补充 Direct operation 的 not-sent 后重试测试，并把金融连接器“业务 rollback 后同一外部事件重试并提交”加入资源边界回归。 + - 验证：容器内 Direct + 商业资源组合 `22 passed`，商业资源边界 `9 passed`；回滚阶段无 usage，重试提交后仅一个 usage 且预占终态为 committed。 + - 影响：数据库瞬时失败或显式回滚不再把同一幂等业务请求永久卡在 released；重试仍受当前硬配额和账期约束，不会绕过额度。 diff --git a/document/development/2026-07-17/dev-logs/bugs/employee-import-membership-transaction.md b/document/development/2026-07-17/dev-logs/bugs/employee-import-membership-transaction.md new file mode 100644 index 0000000..74d39c3 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/employee-import-membership-transaction.md @@ -0,0 +1,13 @@ +# 员工导入与成员资格部分提交 + +日期：2026-07-17 +文档路径：document/development/2026-07-17/dev-logs/bugs/employee-import-membership-transaction.md + +## 修复记录 +- 14:07：记录 bug 修复：员工导入与成员资格部分提交。（bug-log:242d68c3） + - Git 提交检查：已执行 `git fetch --all --prune`、状态、upstream 和双向日志检查；`origin/main` 无新提交，本地 `main` ahead 17。ahead 摘要：17 个提交覆盖审批/风险处置、租户安全费用学习、权威预审、票据关联、个人记忆、历史回填、迁移安全、费用事件事务和不透明认证会话；本轮未改写这些提交。 + - 根因：`EmployeeImportCoordinator._apply_import_rows()` 在写员工和上级关系后先 commit，`EmployeeService.import_employees()` 才补 `TenantMembership` 并第二次 commit；成员资格失败时接口报错，但员工已永久落库。 + - 修改：协调器只 flush 员工、角色、组织、上级和变更日志，不再拥有 commit；外层服务仅在成功结果后补齐租户成员资格，并对两部分执行一次统一 commit，任一异常统一 rollback。 + - 操作：新增成员资格同步注入失败测试，导入新员工后故意抛错并验证员工记录不存在；校验失败结果不触发成员资格或无意义提交。 + - 验证：容器内员工服务、导入、认证和行为画像 35 项通过；差旅计算器 5 项通过；目标 Ruff、format check、compileall、代码体积门禁和 `git diff --check` 通过。 + - 影响：员工批量导入现在满足“全部员工数据与认证成员资格一起成功或一起失败”，不会出现接口失败但部分账号已经创建/修改的状态。 diff --git a/document/development/2026-07-17/dev-logs/bugs/employee-session-directory-regressions.md b/document/development/2026-07-17/dev-logs/bugs/employee-session-directory-regressions.md new file mode 100644 index 0000000..cd76b59 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/employee-session-directory-regressions.md @@ -0,0 +1,11 @@ +## 修复记录 + +- 13:19：修复用户会话结算测试身份错配、缓存命中后旧部门不再归一化，以及 Excel 导入清空上级时误清空员工租户的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；`origin/main` 无新提交，本地 `main` ahead 17，工作区仍包含多智能体并行变更，未做清理、覆盖或提交。 + - 本地 ahead 摘要：17 个提交覆盖审批任务与幂等回放（`242d68c3`、`28b834ed`、`4940ebc4`）、租户安全费用学习/预审/票据关联/申请记忆与反馈（`ee88a36b` 至 `a662cfe6`）、历史回填与迁移安全（`5ed34c2b`、`11275e4b`）、费用时间线/事务（`1347366b`、`22669a90`、`a616b30c`、`661990b2`）及不透明认证会话（`653eda05`）。 + - 修改：会话结算正向用例改用会话真实所有者认证，保留服务端 username ownership 校验；新增其他用户不能关闭该会话的反向测试，避免用放宽授权掩盖 `durationMs=0`。 + - 修改：目录建表/种子初始化继续按 bind 与租户缓存，但缓存命中时仍以租户过滤查询旧部门编码并持久化映射到规范部门；不再因初始化缓存永久跳过外部同步产生的旧编码。 + - 修改：Employee 与 OrganizationUnit 的复合租户关系只把 `organization_unit_id`、`manager_id` 标记为 SQLAlchemy 可同步外键，`tenant_id` 只参与关联过滤；清空部门或上级不会再把员工租户写成 `NULL`，数据库复合外键仍阻止跨租户关联。 + - 测试：补充会话所有权反例、导入后 `tenant_id/manager_id` 持久化断言，以及旧部门归一化后租户与数据库组织归属断言。 + - 验证：容器内三个原失败点与新增反例 4 项通过；员工服务、Excel 导入、行为画像/会话和认证会话相关回归 31 项通过；ORM mapper 确认关系同步列仅为 `organization_unit_id/manager_id`；相关文件 Ruff 与 `git diff --check` 均通过。 + - 影响：会话时长能在正确登录主体下正常结算，其他用户仍不能关闭该会话；旧组织编码会持续收敛到标准部门；员工导入或资料更新清空上级/部门时不会破坏不可为空的租户归属。 diff --git a/document/development/2026-07-17/dev-logs/bugs/expense-claim-approver-tenant-isolation.md b/document/development/2026-07-17/dev-logs/bugs/expense-claim-approver-tenant-isolation.md new file mode 100644 index 0000000..a6e873e --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/expense-claim-approver-tenant-isolation.md @@ -0,0 +1,13 @@ +# 报销审批身份解析跨租户串读 + +日期：2026-07-17 +文档路径：document/development/2026-07-17/dev-logs/bugs/expense-claim-approver-tenant-isolation.md + +## 修复记录 +- 13:48：记录 bug 修复：报销审批身份解析跨租户串读。（bug-log:242d68c3） + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、upstream 解析及双向日志检查；`origin/main` 无新提交，本地 `main` ahead 17。ahead 摘要：`242d68c3/28b834ed/4940ebc4` 为审批任务、不可变回放和风险处置，`ee88a36b/6bdf65bc/ae3f02c3/54754b55/211f85d9/5b246307/a662cfe6` 为租户安全费用学习、预审、票据关联、个人记忆和申请反馈，`5ed34c2b/11275e4b` 为历史回填与迁移安全，`1347366b/22669a90/a616b30c/661990b2` 为费用时间线及事务，`653eda05` 为不透明会话；本轮未改写这些提交。 + - 修改：`expense_claim_access_policy.py` 的当前员工、申请人、直属领导、部门预算负责人和财务负责人解析全部先绑定认证用户或报销单的结构化 `tenant_id`；员工、组织和下属子查询增加租户首层谓词，避免相同姓名、邮箱前缀、部门或角色在另一企业命中。 + - 修改：把身份候选、申请人回填和唯一姓名判断拆入 `expense_claim_employee_resolver.py`，保持公开策略 API 不变，并将访问策略主文件降到 701 行；无可信租户、跨租户关联或结构化归属冲突统一失败关闭。 + - 操作：同步补齐审批任务、风险并发、层级记忆和报销测试夹具的显式企业归属；没有放宽生产授权，也没有用默认企业兼容掩盖跨租户错误。 + - 验证：容器内 `test_expense_claim_service.py` 121 项通过，访问策略文件大小与租户作用域定向 10 项通过；审批任务、PostgreSQL 并发与全量后端分片均通过，fresh PostgreSQL 专项最终 `87 passed / 0 skipped / 0 failed`；新 Python 文件 Ruff 和 `git diff --check` 通过。 + - 影响：审批队列、审批人快照、退回/通过权限和历史回填不会再因另一企业存在同名员工或同名部门而串租户；多租户环境下报销审批保持可解释且失败关闭。 diff --git a/document/development/2026-07-17/dev-logs/bugs/finance-dashboard-structured-tenant-test-fixture.md b/document/development/2026-07-17/dev-logs/bugs/finance-dashboard-structured-tenant-test-fixture.md new file mode 100644 index 0000000..a65b271 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/finance-dashboard-structured-tenant-test-fixture.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 13:33：记录 bug 修复：财务驾驶舱租户安全测试仍按旧 Case Link 契约构造报销单。 + - Git 提交检查：`git fetch --all --prune` 后未发现 upstream 新提交；本地相对 `origin/main` ahead 17 个既有提交，分别为 `242d68c3` 审批任务工作流、`28b834ed` 不可变响应重放、`4940ebc4` 风险处置工作流、`ee88a36b` 租户安全分层费用学习、`6bdf65bc` 权威预审、`ae3f02c3` 零录入票据关联、`54754b55` 个人申请记忆、`211f85d9` 申请工作流统一、`5b246307` 申请预览决策、`a662cfe6` 反馈账本、`5ed34c2b` 历史 Expense Case 回填、`11275e4b` 迁移所有权安全、`1347366b` 时间线与草稿事件安全、`22669a90` 统一事件时间线、`a616b30c` AI 申请事务统一、`653eda05` 不透明 bearer 会话、`661990b2` 事务化 Expense Case 事件；本次未改写这些提交。 + - 修改：`test_finance_dashboard_tenant_security.py` 的 Claim 构造器显式接收并写入 `tenant_id`，租户夹具不再只依赖历史 `ExpenseCaseLink`；新增“结构化 Claim 租户与旧 Link 不一致时以 Claim 为准”的隔离回归。 + - 操作：先在容器内单跑复现 3 个失败，确认不是测试顺序或全局 monkeypatch 污染，再执行最小夹具修复并串行复跑财务、连接器与 Hermes 相邻测试组。 + - 验证：单文件 `8 passed`；排序相邻组 `67 passed, 4 skipped`；目标文件 Ruff 检查与 `git diff --check` 均通过。 + - 影响：财务驾驶舱测试与新的结构化租户模型保持一致，同时固定了旧关联索引不能移动报销单租户归属的安全边界；生产 fail-close 与首条 SQL 租户过滤没有放宽。 diff --git a/document/development/2026-07-17/dev-logs/bugs/financial-connector-operational-counting-clock.md b/document/development/2026-07-17/dev-logs/bugs/financial-connector-operational-counting-clock.md new file mode 100644 index 0000000..4ed5197 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/financial-connector-operational-counting-clock.md @@ -0,0 +1,13 @@ +# 财务连接器运行事件计数与固定时钟偏差 + +日期：2026-07-17 +文档路径：document/development/2026-07-17/dev-logs/bugs/financial-connector-operational-counting-clock.md + +## 修复记录 + +- 11:39：记录 bug 修复：财务连接器相同载荷的不同运行尝试被永久合并，固定时钟场景的接收时间偏离观测窗口。 + - Git 提交检查：已执行 `git fetch --all --prune`；upstream `origin/main` 无新提交；本地 ahead 17 条，包含 `242d68c3 feat(approval): add task workflow and waiver decisions`、`28b834ed fix(approval): replay immutable action responses`、`4940ebc4 feat(approval): add safe risk disposition workflow`、`ee88a36b feat(ai): add tenant-safe hierarchical expense learning` 等共享工作区既有提交，本次未改写这些提交。 + - 修改：`financial_connector_operational_events.py` 把规范化 UTC 发生时间纳入运行事实幂等命名空间，使同一 candidate 的补偿重试保持单条、不同 HTTP 尝试分别计数；`financial_connector_ingestion.py` 让注入的可信 `now_epoch` 同时驱动接收时间和配置健康时间；`financial_connector_observability.py` 统一把数据库时间规范为 UTC，避免 SQLite/驱动返回 naive datetime 时窗口结果不稳定。 + - 操作：补充运行事实服务、HTTP 冲突、可信认证归属、敏感原值不落库和 PostgreSQL 并发探针；所有 Python、pytest、Ruff 与迁移操作均在 `local-x-financial-linux` 容器中执行，PostgreSQL 验证使用新建的一次性 `disposable-probe` 容器，没有连接项目配置中的外部数据库。 + - 验证：容器内连接器与迁移前置定向 `115 passed`，全新 PostgreSQL 17 完整迁移循环 `51 passed`，连接器并发 `4 passed`，后继 0023 并发 `7 passed`；相关 Ruff 和 `git diff --check` 通过。 + - 影响：可观测性不会再把多次真实重放压成一次，也不会因测试/模拟可信时钟与实际系统日期不同而漏掉事件；同一补偿 candidate 仍由数据库唯一约束保证幂等。 diff --git a/document/development/2026-07-17/dev-logs/bugs/hermes-ontology-tenant-isolation.md b/document/development/2026-07-17/dev-logs/bugs/hermes-ontology-tenant-isolation.md new file mode 100644 index 0000000..4dc8254 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/hermes-ontology-tenant-isolation.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 13:15：修复本体参考目录、员工画像、Hermes 扫描/看板/提醒、财务报告和内部 Orchestrator 的跨租户读取与错误归属风险。 + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；`origin/main` 无新增提交，本地 `main` ahead 17。ahead 摘要：`242d68c3/28b834ed/4940ebc4` 审批与风险处置，`ee88a36b/54754b55/211f85d9/5b246307/a662cfe6` AI 费用学习与申请流程，`6bdf65bc/ae3f02c3/5ed34c2b/1347366b/22669a90/a616b30c/661990b2` 费用预审、费用事件与时间线，`11275e4b` migration ownership，`653eda05` opaque bearer session；本轮未改写这些提交。 + - 修改：本体解析在模型前创建租户化 Agent Run，并对员工、组织、费用、应收、应付和项目目录首 SQL 过滤；员工画像新增租户字段、复合员工外键和本人/直属领导/财务/高管/管理员访问矩阵；Hermes 风险、画像、线索、提醒和看板逐租户运行；财务报告按企业配置收件人、内容、路径和幂等账本；内部 Orchestrator 缺失、停用、不存在或冲突租户时在建 Run 前拒绝。 + - 操作：新增 `20260717_0028` 迁移和租户财务报告配置/运行表；upgrade/downgrade 在任何 DDL 前拒绝非 PostgreSQL；完成 fresh、旧结构回填、回滚重升及复合外键 PostgreSQL 探针，验证后停止并自动清理一次性探针容器。 + - 验证：容器内新增安全/提醒测试 9 项、旧本体与 Orchestrator 88 项、Hermes/看板/财务报告 8 项、画像/鉴权/关联草稿 25 项通过；Alembic 静态回归 61 项通过、1 项因未提供外部 PostgreSQL DSN 跳过；相关文件 Ruff check/format check 和 `git diff --check` 通过。 + - 影响：企业 A 的模型提示词、员工画像、风险扫描、提醒、看板、报告附件和邮件收件人不再包含企业 B 数据；无可信租户的内部任务不能生成不可归属记录；历史无双租户快照的 Run 不会被企业看板错误统计。 diff --git a/document/development/2026-07-17/dev-logs/bugs/knowledge-global-storage-tenant-isolation.md b/document/development/2026-07-17/dev-logs/bugs/knowledge-global-storage-tenant-isolation.md new file mode 100644 index 0000000..aa5633b --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/knowledge-global-storage-tenant-isolation.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 12:45：修复知识文件、元数据与 LightRAG/Qdrant 全局共享导致的跨租户串读和覆盖风险。 + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；upstream 无新提交，本地 `main` ahead 17，包含 `242d68c3`、`28b834ed`、`4940ebc4`、`ee88a36b`、`11275e4b`、`653eda05` 等既有审批、AI、迁移与鉴权提交，本次未改写或合并这些提交。 + - 修改：新增 tenant/platform 存储作用域，把文件、`.index.json`、`.lightrag`、运行时缓存和 Qdrant workspace 分区；旧全局制度无损复制到平台只读层；知识 API、同步、Orchestrator 查询和后台索引统一从认证用户或数据库 Agent Run 获取可信 tenant。 + - 操作：按职责拆出 scope、index state 和 RAG scoring 小模块；删除知识文件工具中已废弃的弱 ONLYOFFICE token 代码；所有命令在 Docker 容器 `local-x-financial-linux` 的 `/app` 下运行，未提交、未推送、未删除旧知识资料。 + - 验证：容器内租户隔离 6 项、既有知识回归 31 项和相关 Agent Run/鉴权 20 项通过；目标 Ruff 与 compileall 通过。 + - 影响：租户只能看到和操作自己的知识数据，并可读取平台只读制度；RAG 本地状态、缓存和向量 workspace 不再跨企业复用，缺 tenant 时统一 fail-closed。 diff --git a/document/development/2026-07-17/dev-logs/bugs/knowledge-onlyoffice-callback-security.md b/document/development/2026-07-17/dev-logs/bugs/knowledge-onlyoffice-callback-security.md new file mode 100644 index 0000000..ebdb173 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/knowledge-onlyoffice-callback-security.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 12:45：修复匿名 ONLYOFFICE 回调可重放、可跨资源覆盖并可借下载 URL 发起 SSRF 的高风险问题。 + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；upstream 无新提交，本地 `main` ahead 17，既有提交未被改写。 + - 修改：新增 DB-backed 一次性会话与 `20260717_0027` 迁移；token 绑定租户、资源、文档、key、版本、权限、audience、过期时间和 JTI；预览禁止写入，编辑回调原子 claim 后只能消费一次；下载器拒绝错误 origin、私网/回环/链路本地解析、DNS 重绑定、重定向、超限、错误 MIME 和异常 OOXML。 + - 操作：把 content/callback 解析与网络下载从主 KnowledgeService 拆出；content token 保持短时，callback session 默认 4 小时；失败会话记录受控原因且原文件保持不变。 + - 验证：容器内 ONLYOFFICE 租户安全 8 项通过，覆盖平台只读、错 key 不 claim、过期、成功单次回写、重放、SSRF、IP pinning、大小/MIME/OOXML；迁移/preflight/ownership 163 项通过、1 项静态套件因未配置测试 DSN 跳过；共享 PostgreSQL 探针另完成 `base → 0028` 与 `0028 → 0025 → 0028`，Knowledge/AgentAsset 会话表均正常落库。 + - 影响：未签名、过期、跨租户、跨文档、旧版本、只读和重放回调均不能覆盖文件；文档服务下载失败时安全拒绝，不向内部网络或任意 URL 发请求。 diff --git a/document/development/2026-07-17/dev-logs/bugs/knowledge-scheduler-default-tenant.md b/document/development/2026-07-17/dev-logs/bugs/knowledge-scheduler-default-tenant.md new file mode 100644 index 0000000..358d954 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/knowledge-scheduler-default-tenant.md @@ -0,0 +1,8 @@ +## 修复记录 + +- 12:45：修复知识索引调度器使用默认租户、后台线程信任传入上下文导致任务错误归属的问题。 + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、`git log HEAD..@{u}` 和 `git log @{u}..HEAD`；upstream 无新提交，本地 `main` ahead 17，既有提交未被改写。 + - 修改：调度器查询租户注册表并逐个处理 `status=active` 的租户；无活跃租户时跳过；内部 Hermes 身份显式携带 tenant；索引 worker 从数据库 Agent Run 的 route/ontology 重新取得并比对 tenant，拒绝缺失或冲突上下文。 + - 操作：同步任务、活跃任务复用、stale 状态回收和 ingest 状态写入均绑定当前 tenant，不再构造无作用域 KnowledgeService。 + - 验证：容器内 active/suspended 调度隔离、Agent Run tenant 冲突、知识同步和既有知识组合测试通过；扫描全部 `KnowledgeService`/`KnowledgeRagService` 调用点，生产代码仅保留显式 tenant 或显式 platform 构造。 + - 影响：定时知识任务不会再落入默认企业或把一个租户的文档写进另一个租户的索引；后台参数被篡改或丢失时任务 fail-closed。 diff --git a/document/development/2026-07-17/dev-logs/bugs/steward-linked-reimbursement-tenant-context.md b/document/development/2026-07-17/dev-logs/bugs/steward-linked-reimbursement-tenant-context.md new file mode 100644 index 0000000..acbcbfc --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/steward-linked-reimbursement-tenant-context.md @@ -0,0 +1,12 @@ +# 后台报销与 Steward 可信租户上下文缺失 + +日期：2026-07-17 +文档路径：document/development/2026-07-17/dev-logs/bugs/steward-linked-reimbursement-tenant-context.md + +## 修复记录 +- 02:09：记录 bug 修复：后台报销与 Steward 可信租户上下文缺失。（bug-log:242d68c3） + - Git 提交检查：已执行 `git fetch --all --prune`；upstream `origin/main` 无新提交；本地 ahead 17 条，包含 `242d68c3 feat(approval): add task workflow and waiver decisions`、`28b834ed fix(approval): replay immutable action responses`、`4940ebc4 feat(approval): add safe risk disposition workflow`、`ee88a36b feat(ai): add tenant-safe hierarchical expense learning` 等共享工作区既有提交，本次未改写这些提交。 + - 修改：`linked_reimbursement_draft_jobs.py` 将已认证 `current_user` 显式传给 Orchestrator，避免后台 AgentRun 丢失 tenant；`steward.py` 为 plans、slot-decisions、runtime-decisions、plans/stream 注入认证用户，用登录态覆盖 payload 身份字段，并补充会话归属校验和申请候选单租户过滤；对应测试覆盖未认证、空 tenant、伪造 tenant、跨租户会话及候选单隔离。 + - 操作：所有静态检查和测试均在 Docker 容器 `local-x-financial-linux` 的 `/app` 下执行；没有修改 commercial、迁移、连接器、AI 发布或 Savings 代码，也没有提交或推送。 + - 验证：容器内定向 pytest 两组共 54 项通过；Ruff 排除目标旧文件既有 E501 长行后全部规则通过。完整 Ruff 仍报告目标旧文件历史 E501，不涉及本次新增安全逻辑。 + - 影响：后台报销任务不再因漏传登录态落入缺失/default tenant；四类 Steward AI 入口统一要求登录，客户端伪造 tenant、用户或权限字段不能覆盖服务端身份，跨租户会话和申请候选单读取会 fail-closed。 diff --git a/document/development/2026-07-17/dev-logs/bugs/travel-calculator-employee-tenant-isolation.md b/document/development/2026-07-17/dev-logs/bugs/travel-calculator-employee-tenant-isolation.md new file mode 100644 index 0000000..202a1f1 --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/travel-calculator-employee-tenant-isolation.md @@ -0,0 +1,13 @@ +# 差旅计算器员工解析跨租户命中 + +日期：2026-07-17 +文档路径：document/development/2026-07-17/dev-logs/bugs/travel-calculator-employee-tenant-isolation.md + +## 修复记录 +- 14:07：记录 bug 修复：差旅计算器员工解析跨租户命中。（bug-log:242d68c3） + - Git 提交检查：已执行 `git fetch --all --prune`、状态、upstream 和双向日志检查；`origin/main` 无新提交，本地 `main` ahead 17。ahead 摘要：`242d68c3/28b834ed/4940ebc4` 为审批与风险处置，`ee88a36b` 至 `a662cfe6` 为费用学习/预审/票据/记忆闭环，`5ed34c2b/11275e4b` 为回填和迁移安全，`1347366b/22669a90/a616b30c/661990b2` 为费用事件与事务，`653eda05` 为认证会话；本轮未改写这些提交。 + - 根因：差旅计算器按邮箱、工号或姓名查员工时没有租户谓词，也忽略认证上下文的 `employee_id/employee_no`；员工身份现为租户内唯一，不同企业可以合法存在相同值，首条命中会把另一企业职级和办公城市带入计算。 + - 修改：`travel_reimbursement_calculator.py` 先校验可信 `tenant_id`，将 `employee_id` 加入最高优先候选，所有 ID/邮箱/工号/姓名查询都以 `Employee.tenant_id` 为首层条件。 + - 操作：新增两企业相同邮箱、工号和姓名的反向测试，故意先插入另一企业员工，确认当前登录企业的 employee ID、职级和地点始终胜出。 + - 验证：容器内差旅计算器 5 项通过；员工服务、导入、认证和行为画像 35 项通过；目标 Ruff、format check、compileall、代码体积门禁和 `git diff --check` 通过。 + - 影响：差旅住宿、补助和交通估算不会再读取另一企业员工的职级或办公地点；无合法租户时直接失败关闭。 diff --git a/document/development/2026-07-17/dev-logs/bugs/travel-calculator-rule-sync-transaction.md b/document/development/2026-07-17/dev-logs/bugs/travel-calculator-rule-sync-transaction.md new file mode 100644 index 0000000..35adabc --- /dev/null +++ b/document/development/2026-07-17/dev-logs/bugs/travel-calculator-rule-sync-transaction.md @@ -0,0 +1,13 @@ +# 差旅计算器只读流程触发规则资产提交 + +日期：2026-07-17 +文档路径：document/development/2026-07-17/dev-logs/bugs/travel-calculator-rule-sync-transaction.md + +## 修复记录 +- 13:48：记录 bug 修复：差旅计算器只读流程触发规则资产提交。（bug-log:242d68c3） + - Git 提交检查：已执行 `git fetch --all --prune`、`git status -sb`、upstream 解析及双向日志检查；`origin/main` 无新提交，本地 `main` ahead 17。ahead 摘要：`242d68c3/28b834ed/4940ebc4` 为审批与风险处置，`ee88a36b` 至 `a662cfe6` 为 AI 费用学习和申请闭环，`5ed34c2b/11275e4b` 为历史回填与迁移安全，`1347366b/22669a90/a616b30c/661990b2` 为费用事件与事务，`653eda05` 为认证会话；本轮未改写这些提交。 + - 根因：差旅计算器为读取规则先调用 `AgentAssetService.list_assets()`，该入口可能执行资产初始化并提交共享 Session，导致外层报销事务被提前提交；同时规则目录未带认证企业，企业覆盖规则可能退回到错误的全局目录。 + - 修改：`travel_reimbursement_calculator.py` 删除计算路径中的资产同步调用，改为使用当前 Session 和认证用户 `tenant_id` 直接只读加载 `ExpenseRuleRuntimeService` 目录；规则初始化只保留在启动/管理边界，不再混入金额计算事务。 + - 操作：保留地点、职级、住宿、补助和交通估算算法，仅收紧规则读取的事务与租户边界；测试夹具同步补齐结构化企业身份。 + - 验证：容器内差旅计算器 4 项金额、地区拒绝和地点归一化回归通过；AgentAsset service/foundation 28 项、报销服务 121 项及后端全量分片通过；相关新文件 Ruff、compileall 和 `git diff --check` 通过。 + - 影响：用户计算差旅标准时不会意外提交正在编辑的报销单；企业规则按当前登录企业解析，读取失败会明确报错而不会静默使用另一企业或全局可写状态。 diff --git a/document/development/2026-07-17/feature/agent-asset-tenant-security/CONCEPT.md b/document/development/2026-07-17/feature/agent-asset-tenant-security/CONCEPT.md new file mode 100644 index 0000000..e33d7d7 --- /dev/null +++ b/document/development/2026-07-17/feature/agent-asset-tenant-security/CONCEPT.md @@ -0,0 +1,188 @@ +# Agent 资产多租户隔离与安全规则编辑 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +让规则、技能、MCP、任务及其版本、审核、测试和编辑会话都拥有可验证的租户归属，并以“企业资产可写、平台资产只读”的双层模型安全贯通规则生成、真实场景验证、审核、发布和 ONLYOFFICE 编辑。 + +## 背景与问题 + +原 AgentAsset 数据模型没有结构化 `tenant_id` 与 `scope`，部分读取和内部查询可以在没有用户上下文时返回全局资产。版本、审核、测试、反馈和发布链路主要依赖 `asset_id` 或业务约定关联，无法由数据库阻止跨租户子记录注入。同编码资产也不能由不同企业独立维护。 + +规则场景测试可以在未声明目标企业时抽取费用数据，审核主体还可能使用客户端传入的 actor/reviewer 字段，导致测试证据和盲审身份缺乏稳定、可追溯的企业边界。 + +规则表的 ONLYOFFICE 内容与回调接口原先缺少持久化的一次性会话。回调下载地址、文档 key、版本、租户和编辑权限之间没有不可变绑定，存在匿名读取、跨资产回写、身份伪造、重放和服务端请求伪造风险。 + +## 目标与非目标 + +### 目标 + +- AgentAsset、Version、Review、TestRun、RuleFeedback 和 ONLYOFFICE Session 均保存结构化租户作用域。 +- 租户用户只能看到本企业资产和平台资产；同编码时企业资产优先覆盖平台默认资产。 +- 跨租户详情、版本、审核、发布、测试和反馈统一表现为不存在，避免泄露资源存在性。 +- 平台资产对所有租户只读，仅平台管理员可新增、修改、发布或编辑。 +- 所有 HTTP 入口使用认证会话中的 `CurrentUserContext.tenant_id`，不接受客户端覆盖租户。 +- 写入、版本和审核操作使用 RuleEditor、RuleReviewer 或平台管理员权限，并以稳定身份写入审计证据。 +- 真实风险场景必须显式声明当前企业 `target_tenant_id`，费用样本的第一层 SQL 条件就是该租户。 +- ONLYOFFICE 内容和回调使用不同 audience/scope 的签名 token，并绑定数据库一次性会话。 +- 回调下载拒绝错误 origin、非公网解析、DNS 重绑定、重定向、超限和非安全 OOXML 文件。 + +### 非目标 + +- 不允许普通企业用户创建或修改平台资产。 +- 不把所有企业资产放在全局结果集中后仅靠前端过滤。 +- 不允许平台管理员借普通企业会话修改其他企业的私有资产；跨企业运维需要独立受控流程。 +- 不提供允许私网、回环或任意下载地址的 ONLYOFFICE 安全降级开关。 +- 不在本切片中重做 Agent 资产管理前端视觉或商业定价页面。 + +## 用户与场景 + +- 企业规则编辑者：维护本企业规则资产、上传规则表并创建新版本。 +- 企业规则审核者：以稳定登录身份进行盲审、驳回或批准规则版本。 +- 企业风控人员：用本企业真实费用申请生成测试样本和质量证据。 +- 普通企业用户：读取本企业资产和平台只读资产，但不能写入。 +- 平台管理员：维护跨企业可见的平台基础规则和模板。 +- 运行时与调度器：在显式租户范围内加载、测试、监控和召回资产，不进行全局扫描。 +- ONLYOFFICE 文档服务：使用资源专用 token 读取一次文档，并通过单次 callback session 回写允许编辑的当前版本。 + +## 功能能力 + +- 双层可见性：`tenant:{tenant_id}` 与 `platform:platform`。 +- 企业内唯一编码：数据库唯一键为 `(tenant_id, scope, code)`，不同企业可拥有同编码资产。 +- 确定性覆盖：按编码加载时先找当前企业资产，再回退平台资产。 +- 服务端可信租户：HTTP 服务由登录会话构造 `AgentAssetAccessScope`；无用户上下文的内部读取只允许平台作用域。 +- 稳定审计主体：优先记录 `employee:{employee_id}`，没有员工 ID 时记录大小写归一的 `username:{username}`。 +- 真实样本隔离：场景请求必须传 `target_tenant_id`，且必须等于登录企业；TestRun 记录样本所属企业。 +- 平台资产可在企业场景中验证，但测试证据仍归目标企业，不能变成平台或其他企业事实。 +- ONLYOFFICE content token 有效期 15 分钟，callback session 有效期 4 小时。 +- 回调状态机保证同一个 JTI 最多一次进入写入阶段。 + +## 方案设计 + +### 前端契约 + +- AgentAsset DTO 提供 `tenantId` 与 `scope`，前端可明确标识企业资产和平台资产。 +- 平台资产在非平台管理员会话中必须隐藏或禁用修改、发布、上传和 ONLYOFFICE 编辑动作。 +- 场景测试请求必须携带 `targetTenantId`；它是目标企业的显式确认，不是可切换企业的授权参数。 +- 旧 `X-Actor`/reviewer 头仅为兼容保留，服务端忽略其身份值并使用登录会话主体。 +- ONLYOFFICE 配置根据权限返回 `view` 或 `edit`；内容和回调 token 只供文档服务使用。 + +### 后端职责 + +- `agent_asset_scope`：定义 platform/tenant 常量和合法作用域基础规则。 +- `agent_asset_access`：从可信用户构造访问范围、生成稳定主体并提供可见/可写谓词。 +- `agent_asset` repository：所有列表、详情、版本、审核、测试和反馈查询都注入结构化租户条件。 +- `agent_assets`：编排资产 CRUD、版本、审核与序列化，不通过未过滤 ORM 关系返回子记录。 +- `agent_asset_risk_rule_testing`：校验目标企业、先按租户过滤 ExpenseClaim，再生成测试证据。 +- 发布、监控、召回、调度、遥测和风险运行时服务：沿资产租户作用域读取和写入，禁止全局 asset id 查询。 +- `agent_asset_onlyoffice_security`：签发/验证持久化会话并原子消费 callback JTI。 +- `agent_asset_onlyoffice`：按会话中的租户、资产、key、版本和指纹定位文档，安全下载后创建新版本。 +- `knowledge_onlyoffice_security`：复用统一安全下载器，执行 origin、DNS/IP、响应和 OOXML 校验。 + +### 数据 + +`20260717_0026_agent_asset_tenant_security.py` 负责以下结构： + +- `agent_assets`：新增 `tenant_id`、`scope`，建立作用域约束、租户外键和企业内 code 唯一键。 +- `agent_asset_versions`、`agent_asset_reviews`：新增租户作用域，并以 `(tenant_id, scope, asset_id)` 复合外键绑定父资产。 +- `agent_asset_test_runs`、`agent_asset_rule_feedback`：保存证据所属企业；平台资产的企业测试/反馈也归企业事实域。 +- `agent_asset_onlyoffice_sessions`：保存 JTI、租户、资源 scope、asset、document key/version/fingerprint、audience、权限、actor、过期时间和状态。 +- 旧资产默认回填为平台资产；能从父资产确定的历史版本、审核和证据同步回填。 +- 无法确认租户的历史数据或不完整旧表结构会 fail-closed，不猜测企业归属。 + +### 权限与信任边界 + +- 当前企业只来自 `CurrentUserContext.tenant_id`；空值、`platform` 伪企业或请求参数不能构造企业访问范围。 +- 租户读取谓词是“当前企业或平台”，写入谓词只允许当前企业；平台写入还要求 `is_admin=true`。 +- 跨租户资源统一返回 404；权限不足的本作用域操作返回受控错误。 +- RuleEditor 可维护规则和版本，RuleReviewer 执行审核；平台资产的任意写操作额外要求平台管理员。 +- 版本 created_by、审核 reviewer、规则表变更 actor 都由稳定登录主体生成。 +- 后台 bootstrap/foundation 按平台作用域精确查找种子资产，不能误改同编码企业资产。 +- 风险运行时按“企业优先、平台回退”加载发布规则，不扫描其他企业版本。 +- ORM 关系不是授权边界；对外响应必须经过带 scope 的 repository/service 查询。 + +### 资产解析顺序 + +```text +find_by_code(code, current_tenant): + 1. tenant_id = current_tenant AND scope = tenant + 2. tenant_id = platform AND scope = platform + 3. not found +``` + +该顺序让企业能够在不修改平台模板的情况下覆盖默认规则，同时保持其他企业和平台资产不受影响。 + +### 风险场景测试 + +```text +认证用户 tenant + → 校验 target_tenant_id == tenant + → 校验目标资产为 tenant 自有或 platform 只读资产 + → SQL 第一层条件 ExpenseClaim.tenant_id == target_tenant_id + → 应用时间、费用类型、城市等业务筛选 + → 创建 tenant-scoped TestRun +``` + +没有目标企业、目标企业不一致或资产属于其他企业时均拒绝，不使用 mock 的默认企业或全局样本补齐。 + +### ONLYOFFICE 状态机 + +```text +issue(view) → active ── callback status 2/6 ──拒绝写入 +issue(edit) → active ── atomic claim ──→ processing + ├─校验/下载/写入成功→ consumed + └─任一步失败────────→ failed + +active -- exp 超时 --> 验证拒绝 +processing/consumed/failed/revoked -- replay --> 409/拒绝 +``` + +token 同时绑定 issuer、audience、scope、JTI、tenant、resource scope、asset、document key、version、fingerprint、writable、actor、iat/nbf/exp。回调 payload 只能提供状态和下载位置，不能覆盖这些授权事实。 + +### 降级与回滚策略 + +- 无可信租户：HTTP 请求拒绝；无用户上下文的内部服务只看平台资产，绝不回退全局查询。 +- 跨租户资产、版本或测试证据：按不存在处理，不尝试平台管理员越权兼容。 +- 场景样本为空：返回空样本测试事实，不改查其他企业数据。 +- ONLYOFFICE token、key、版本、指纹、DNS、MIME 或 OOXML 校验失败：拒绝回写并保持原文件不变。 +- 生产文档服务必须使用配置白名单中的公网 TLS origin，或经满足相同约束的安全代理访问。 +- migration downgrade 在存在企业资产/证据或 ONLYOFFICE 会话时拒绝有损回滚；必须先通过受控数据迁移清理事实。 + +## 测试方案 + +- 可见性：两企业同编码资产、企业覆盖平台、平台只读、跨企业详情 404、无上下文仅平台。 +- 数据完整性：scope/tenant check、企业内 code 唯一、Version/Review 复合父子外键、TestRun 证据企业。 +- 权限与身份：RuleEditor/RuleReviewer、平台管理员、伪造 actor/reviewer 无效、稳定 employee/username 主体。 +- 风险场景：必须目标企业、目标不一致拒绝、SQL 租户首过滤、平台资产的企业 TestRun。 +- 发布链路：发布门禁、监控、召回、运行时、调度和遥测均使用显式租户。 +- ONLYOFFICE：content/callback scope、tenant/asset/key/version/fingerprint 绑定、平台只读、原子 claim、失败终态和重放拒绝。 +- SSRF/文件：白名单 origin、全量公网 DNS、固定已校验 IP、拒绝重定向、大小/MIME/ZIP/OOXML 限制。 +- 迁移：旧平台数据回填、同编码多企业、非法 scope/跨租户外键拒绝、事实存在时 downgrade 拒绝、清理后可回滚。 +- 所有后端验证只在 `local-x-financial-linux` 容器中执行，单条命令限制 60 秒。 + +## 指标与验收 + +- 所有 AgentAsset 业务读取都能由 `tenant_id + scope` 确定结果范围。 +- 跨企业资产、版本、审核、测试、反馈和回调用例 100% 不可见或拒绝。 +- 平台资产对非平台管理员的写入和回调 100% 无副作用。 +- 真实费用样本查询 100% 具有服务端校验的企业首过滤条件。 +- 审计主体 100% 来自稳定登录身份，客户端 actor/reviewer 不能改变事实。 +- 同一 ONLYOFFICE callback JTI 最多一次进入 `processing`。 +- 相关 Python 文件通过 Ruff、py_compile 和 ORM mapper 配置；定向发布、安全与风险规则回归通过。 + +## 风险与开放问题 + +- 旧测试或内部调用若直接构造没有 `tenant_id` 的 `CurrentUserContext`，会按新信任边界 fail-closed；应由对应调用方补齐真实租户，而不是放宽服务契约。 +- 两个旧费用风险测试使用尚未持久化、没有 claim tenant 的对象；共享租户作用域已要求先保存并确认归属，需由费用申请切片统一调整测试夹具。 +- 两个旧风险发布测试手工注入 aggregate 后直接 promote，与当前必须有真实质量 TestRun 的发布门禁不一致；需由发布门禁切片统一口径。 +- ONLYOFFICE 实际回写依赖部署环境提供公网可解析、可信 TLS 的文档服务或安全代理；开发网络解析到保留/私网地址时会按设计拒绝。 +- 企业间受控复制、平台资产签名发布和跨企业运维审计属于后续独立能力，不应通过放宽本轮隔离实现。 + +## 本轮实现记录 + +- 2026-07-17：完成 AgentAsset、Version、Review、TestRun、Feedback 的结构化租户作用域、企业覆盖平台读取和跨企业 fail-closed。 +- 2026-07-17：完成可信目标企业风险场景、真实费用样本 SQL 首过滤、企业 TestRun 证据和稳定盲审身份。 +- 2026-07-17：完成 DB-backed ONLYOFFICE 一次性会话、平台只读、文档基线绑定与安全下载。 +- 2026-07-17：完成 `20260717_0026` 迁移及一次性 PostgreSQL `base → head`、`head → 0025 → head` 验证。 +- 2026-07-17：完成发布、监控、召回、运行时、调度、遥测、foundation 和风险规则生成链路的租户接线与容器回归。 diff --git a/document/development/2026-07-17/feature/agent-asset-tenant-security/TODO.md b/document/development/2026-07-17/feature/agent-asset-tenant-security/TODO.md new file mode 100644 index 0000000..49fc9f8 --- /dev/null +++ b/document/development/2026-07-17/feature/agent-asset-tenant-security/TODO.md @@ -0,0 +1,67 @@ +# Agent 资产多租户隔离与安全规则编辑 开发 TODO + +更新时间：2026-07-17 + +关联方案：[CONCEPT.md](./CONCEPT.md) + +## 使用规则 + +- 任务边界、信任模型、数据归属和上线约束以 CONCEPT 对应章节为准。 +- `[x]` 只表示已有代码或容器验证证据；生产环境尚未验证的项目保持 `[ ]`。 +- 所有后端测试必须在 `local-x-financial-linux` 容器内执行，单命令最长 60 秒。 + +## 1. 调研与边界 + +- [x] [CONCEPT: 背景与问题] 盘点 AgentAsset、版本、审核、测试、反馈、发布和规则表编辑中的无租户/弱租户查询。证据：repository、service、endpoint 和 release 全链路调用扫描。 +- [x] [CONCEPT: 目标与非目标] 冻结“企业资产可写 + 平台资产只读 + 企业覆盖平台”的双层模型。证据：`AgentAssetAccessScope` 与两企业同编码测试。 +- [x] [CONCEPT: 权限与信任边界] 冻结可信租户只来自登录会话、客户端 tenant/actor/reviewer 不能覆盖事实。证据：认证依赖、稳定主体函数和伪造头测试。 + +## 2. 契约与设计 + +- [x] [CONCEPT: 数据] 为 Asset、Version、Review、TestRun、Feedback 定义 `tenant_id + scope`、约束和索引。证据：模型与 `20260717_0026`。 +- [x] [CONCEPT: 资产解析顺序] 定义企业同编码资产优先、平台资产回退的确定性加载顺序。证据：scoped repository 和 runtime loader 测试。 +- [x] [CONCEPT: ONLYOFFICE 状态机] 定义 `active → processing → consumed|failed`，以及过期、撤销和重放拒绝。证据：Session 模型、迁移与 callback 测试。 +- [x] [CONCEPT: 风险场景测试] 定义显式 `target_tenant_id` 与 TestRun 证据企业，不允许默认或全局样本。证据：scenario schema/service 测试。 + +## 3. 后端实现 + +- [x] [CONCEPT: 后端职责] 所有 AgentAsset 列表、详情和 code 查询接入结构化可见谓词。证据：`agent_asset.py` repository 与 `agent_assets.py`。 +- [x] [CONCEPT: 权限与信任边界] 资产写入、版本和审核接入 RuleEditor/RuleReviewer/平台管理员依赖。证据：AgentAsset 和风险规则 endpoints。 +- [x] [CONCEPT: 权限与信任边界] 用 `employee:{id}` 或归一化 `username:{name}` 替代请求 actor/reviewer。证据：`stable_user_principal()` 与审计断言。 +- [x] [CONCEPT: 后端职责] 发布门禁、评审、监控、召回、遥测、调度和处置标签均接入租户作用域。证据：8 个 release 定向测试文件。 +- [x] [CONCEPT: 后端职责] foundation/bootstrap 只查平台 seed，避免修改同编码企业资产。证据：foundation helper 拆分与回归测试。 +- [x] [CONCEPT: 后端职责] 风险运行时按企业优先、平台回退加载规则，禁止跨企业版本。证据：expense runtime/loader 改造。 +- [x] [CONCEPT: 风险场景测试] 场景请求校验目标企业，并将 `ExpenseClaim.tenant_id` 作为 SQL 第一层谓词。证据：真实样本场景与 TestRun 断言。 +- [x] [CONCEPT: ONLYOFFICE 状态机] 新增持久化 content/callback token、原子 claim 和终态处理。证据：`agent_asset_onlyoffice_security.py`。 +- [x] [CONCEPT: 权限与信任边界] content/callback 按会话重建 machine scope，拒绝跨企业、跨资产、跨版本和平台只读回写。证据：ONLYOFFICE 安全测试。 +- [x] [CONCEPT: 降级与回滚策略] 复用安全下载器，拒绝错误 origin、非公网 DNS、重定向、超限及异常 OOXML。证据：下载器单测与 callback 回归。 +- [x] [CONCEPT: 数据] 新增 `20260717_0026`，完成旧平台数据回填、约束、索引和有事实 downgrade 保护。证据：迁移文件与 PostgreSQL 探针。 + +## 4. 代码结构 + +- [x] [CONCEPT: 后端职责] 将访问策略、ONLYOFFICE 安全和序列化从大型 AgentAsset service 中拆出。证据：新增小职责模块，核心文件均低于 800 行。 +- [x] [CONCEPT: 后端职责] 将风险规则字段推断/草稿对齐从生成主服务抽到独立模块。证据：`risk_rule_generation.py` 619 行、`risk_rule_generation_fields.py` 272 行。 +- [x] [CONCEPT: 后端职责] foundation 按资产 helper、seed、topup、财务规则和电子员工任务拆分。证据：拆分模块与 28 项定向回归。 + +## 5. 前端契约 + +- [x] [CONCEPT: 前端契约] 后端 DTO 已返回 `tenantId/scope`，客户端可识别平台只读资产。证据：schema 与 AgentAsset API 回归。 +- [x] [CONCEPT: 前端契约] ONLYOFFICE 配置按权限返回 view/edit，平台资产仅平台管理员可编辑。证据：config 单测与平台回写拒绝测试。 +- [x] [CONCEPT: 目标与非目标] 本切片不做 Agent 资产管理页面视觉重构。证据：CONCEPT 非目标。 + +## 6. 测试与验证 + +- [x] [CONCEPT: 测试方案] 完成租户可见性、企业覆盖平台、无上下文平台只读、跨租户 404、目标企业场景和稳定身份测试。证据：`test_agent_asset_tenant_security.py`。 +- [x] [CONCEPT: 测试方案] 完成发布门禁、监控、召回、运行时、调度、遥测及 ONLYOFFICE callback 汇总回归。证据：容器内 58 项通过。 +- [x] [CONCEPT: 测试方案] 完成 AgentAsset service 与 foundation 兼容回归。证据：容器内 28 项通过、4 项无关差旅计算器用例主动排除。 +- [x] [CONCEPT: 测试方案] 完成风险生成、修订和 golden evaluator 回归。证据：容器内 49 项通过，2 项旧发布门禁口径由对应切片处理。 +- [x] [CONCEPT: 测试方案] 完成费用风险租户接线回归。证据：13 项通过；2 项旧测试使用未保存 claim，已记录为共享测试夹具问题。 +- [x] [CONCEPT: 测试方案] 完成一次性 PostgreSQL 迁移验证。证据：旧数据升级、同编码多企业、非法约束、跨租户 FK、有事实 downgrade 拒绝，以及 `base → head`、`head → 0025 → head` 均通过。 +- [x] [CONCEPT: 指标与验收] 相关文件 Ruff 通过、py_compile 通过、ORM mapper 84 张表完成配置、`git diff --check` 通过。 + +## 7. 文档与上线 + +- [x] [CONCEPT: 本轮实现记录] 完成 CONCEPT、分阶段 TODO 和安全 bug 修复日志。证据：本目录及 `dev-logs/bugs/agent-asset-tenant-isolation-and-onlyoffice-security.md`。 +- [x] [CONCEPT: 风险与开放问题] 记录无 tenant 旧调用、未保存 claim 测试、旧发布 aggregate 口径和企业间受控复制边界。证据：CONCEPT 风险章节。 +- [ ] [CONCEPT: 降级与回滚策略] 上线前确认 ONLYOFFICE 下载 origin 在应用容器内解析为公网地址并使用可信 TLS。证据要求：生产白名单配置、容器 DNS/TLS 与实际编辑回写验证。 +- [ ] [CONCEPT: 降级与回滚策略] 上线前在备份副本执行 `0025 → head`，确认不存在无法归属的历史记录，并演练有事实情况下的受控回滚流程。 diff --git a/document/development/2026-07-17/feature/commercial-resource-boundaries/CONCEPT.md b/document/development/2026-07-17/feature/commercial-resource-boundaries/CONCEPT.md new file mode 100644 index 0000000..ddb9f0e --- /dev/null +++ b/document/development/2026-07-17/feature/commercial-resource-boundaries/CONCEPT.md @@ -0,0 +1,161 @@ +# 商业资源权威边界闭环 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +只对已经通过可信边界且随业务事务成功持久化的连接器事件和附件源文件写入计量，并在任何拒绝、失败或回滚时先释放额度、恢复文件，不生成虚假用量。 + +## 背景与问题 + +商业底座已经具备套餐、权益、硬配额、执行前预占和追加式用量/成本事实，但真实资源入口仍存在两处断点： + +- 金融连接器能验证 HMAC、阻断重放与 payload 冲突并持久化事件，却没有把“首次接受且提交成功”接到 `events` 权威计量。 +- 报销附件会在上传开始时直接删除旧目录再写新文件。若配额在写入后才判断，旧文件已经遭到破坏；若数据库随后回滚，文件系统和商业事实还可能与业务记录分叉。 +- 独立商业 Session 不能提交调用方业务 Session。反过来，商业结算也不能早于业务提交，否则数据库回滚后仍会留下客户用量。 +- SQLite 单连接测试环境中，另开 Session 检查“未配置计量器”会回滚同一底层连接上的业务事务；未配置路径必须先在调用方 Session 只读短路。 + +因此，本能力把“预占—业务变更—事务终态—结算/释放”定义为统一资源协议，而不是在端点成功返回前后零散补记用量。 + +## 目标与非目标 + +### 目标 + +- [G1] 金融连接器只对新建、已认证、已提交的 `FinancialConnectorEvent` 计量一个 `events`。 +- [G2] 鉴权失败、稳定重放、payload 冲突、配额拒绝和业务事务回滚均不产生连接器用量。 +- [G3] 附件上传按源文件 `bytes` 在覆盖旧文件前执行硬配额预占。 +- [G4] 附件业务事务回滚时恢复旧文件树、删除未提交的新文件并释放预占；提交后才追加用量。 +- [G5] 删除单附件、费用明细和整张报销单时，把文件删除绑定到数据库事务；回滚恢复、提交后最终清理。 +- [G6] 商业事实只保存哈希化操作身份、固定工具维度、数量和安全来源，不保存外部事件原文、关联号、单号、文件名或文件正文。 +- [G7] 同一已释放操作允许在相同账期重新预占，支持业务回滚后的安全重试。 + +### 非目标 + +- [NG1] 本切片不把附件 `bytes` 定义为实时磁盘容量；它表示成功持久化的源文件写入流量。 +- [NG2] 删除附件不追加正向用量，也不自动冲销历史写入事实；容量型 GB-month 计费需要独立快照账本。 +- [NG3] 不对连接器重放、鉴权失败和冲突运营事件收费。 +- [NG4] 不在商业事实中复制金融 payload、OCR 文本、文件名、报销事由或客户外部引用。 +- [NG5] 不在本切片新增商业配置页面或改变套餐价格。 + +## 用户与场景 + +- 企业员工上传或替换报销附件：平台先检查源文件字节配额，再覆盖文件；数据库提交后才形成用量。 +- 企业员工删除附件、费用明细或草稿报销单：删除操作不被配额阻止，数据库失败时文件恢复。 +- 金融连接器发送回执：只有首次通过签名验证、去重并持久化的事件消费一个连接器事件额度。 +- 连接器因网络重试再次发送相同事件：返回既有业务响应，不重复预占或计量。 +- 平台运营排查账单：能看到 `connector/financial.ingest/events` 或 `storage/attachment.upload/bytes`，但看不到业务原文和敏感标识。 + +## 功能能力 + +- 连接器权威观察器：认证和首次事件判定之后预占，业务事务 `after_commit` 后结算。 +- 附件权威观察器：固定使用 `storage + attachment.upload + bytes`，以 `len(content)` 作为执行前与执行后的同一权威数量。 +- 事务回调批次：同一 Session 可登记多个资源操作；提交按登记顺序结算，回滚按相反顺序补偿。 +- 文件暂存事务：旧文件或目录先原子重命名为同目录隐藏备份；提交删除备份，回滚删除新目录并恢复原路径。 +- 删除事务：单附件、费用明细附件和整张报销单附件树均先暂存，数据库提交后才最终删除。 +- 已释放预占重开：相同指纹、相同订阅/权益/账期的 `released` 预占可重新进入 `reserved`，并重新执行硬配额判断。 +- 未配置兼容：调用方 Session 先只读确认没有匹配计量器，直接兼容执行且不创建商业事实。 + +## 方案设计 + +### 前端 + +- 当前不新增页面。 +- 附件上传端点接受可选 `X-Request-ID`，商业硬配额拒绝返回 HTTP 429；现有 400/404 业务错误保持不变。 +- 客户端应为同一次上传重试复用 `X-Request-ID`。未提供时，服务端用认证会话、租户、Claim/Item 哈希和内容摘要形成保守幂等身份。 + +### 后端 + +- `FinancialConnectorCommercialObserver` 使用认证后配置租户、配置摘要和请求指纹构造哈希身份，固定工具维度为 `connector/financial.ingest`。 +- `FinancialConnectorIngestionService` 先完成认证、外部事件串行化和既有事实判断；只有新事件才申请预占。事件 flush 成功后把完成/释放绑定到调用方 Session。 +- `CommercialTransactionCallbacks` 把多个独立商业操作绑定到一个业务事务。`after_commit` 执行完成回调，`after_rollback` 逆序执行补偿回调；回调异常只记录日志并保留可补偿预占，不伪装业务提交失败。 +- `ExpenseClaimAttachmentCommercialObserver` 固定使用 `bytes`，配额拒绝发生在任何旧文件移动、`rmtree`、`unlink` 或新文件写入之前。 +- `ExpenseClaimAttachmentFileTransaction` 负责旧路径暂存、提交清理和回滚恢复；上传、删除附件、删除费用明细和删除报销单复用同一协议。 +- `CommercialRuntimeReservationService` 允许同一指纹的 `released` 预占在相同账期重新预占；重新检查当前合同、计量器快照和硬配额。 +- OCR 与 RuntimeChat 的直接商业桥同样增加调用方 Session 的只读未配置短路，避免 SQLite 单连接环境的独立 Session 回滚调用方事务。 + +### 算法/规则 + +- 连接器的权威数量恒为一个已提交事件：`actual_events = 1`。 +- 附件的权威数量为请求中源文件真实字节数：`actual_bytes = len(content)`。 +- 连接器只允许 `quantity_basis=events`；附件只允许 `quantity_basis=bytes`。错误基准在预占前失败关闭。 +- 同一业务事务包含多个附件操作时，回滚补偿使用 LIFO，确保同一 Item 连续替换也能恢复到事务开始前状态。 + +### 数据 + +- 不新增数据库表和迁移。 +- 复用 `commercial_runtime_reservations`、`usage_meter_events` 和可选 `commercial_cost_events`。 +- 连接器用量 metadata 只包含固定 meter 版本、哈希化 operation call、固定工具名、数量基准、权益、预占、provider 代码、结果和安全来源。 +- 附件用量不包含文件名、storage key、Claim/Item 原值、MIME、OCR 文本或文件正文。 + +### 权限 + +- 连接器租户只来自 HMAC 认证后配置，不能由未验证 header 或 payload 单独决定。 +- 附件租户只来自 `CurrentUserContext.tenant_id`，Claim 和 Item 已先通过费用单租户访问策略校验。 +- 付费或配额状态不能放宽现有报销状态、附件可变性、连接器签名、租户或财务对账规则。 + +### 降级策略 + +- 没有匹配计量器：保持原业务兼容，不写预占、用量或成本。 +- 商业配置错误、重复计量器、错误数量基准或硬配额不足：在业务/文件副作用前拒绝。 +- 业务事务回滚：释放预占；连接器不留事件用量，附件恢复原文件。 +- 商业完成回调异常：业务提交不反转；预占保持可审计状态，由既有补偿流程处理。 +- 文件恢复失败：记录错误并保留隐藏备份，不把失败删除误报为成功；需要运维根据事务日志和隐藏路径人工恢复。 + +## 算法与公式 + +### 硬配额判断 + +```text +allowed = used_quantity + held_quantity + requested_quantity <= hard_limit +``` + +- 连接器 `requested_quantity = 1 event`。 +- 附件 `requested_quantity = len(content) bytes`。 +- 判断在订阅与权益锁内完成；拒绝时真实连接器处理和附件破坏性操作尚未发生。 + +### 业务事务终态 + +```text +permit -> business mutation -> commit -> usage(actual_quantity) + -> rollback -> release reservation + restore files +``` + +### 附件写入计量口径 + +```text +billable_attachment_bytes = Σ len(source_content) +``` + +- 只汇总成功随业务事务提交的源文件写入。 +- OCR 临时文件、预览图、metadata 和隐藏事务备份不进入本 meter。 +- 删除不产生负数；实时容量需另建周期快照和保留量口径。 + +## 测试方案 + +- 连接器：首次提交、稳定重放、payload 冲突、签名失败、业务回滚、回滚后重试、硬配额拒绝和敏感 metadata 反向断言。 +- 附件商业：bytes 提交、回滚释放、错误 basis、硬配额拒绝发生在旧文件移动前、相同事务连续替换逆序恢复。 +- 附件文件：单附件删除、费用明细删除、整单删除的 commit/rollback 文件终态。 +- 既有回归：连接器服务/端点/配置生命周期、附件归集任务、票据夹、报销端点、附件专项、OCR 与 RuntimeChat。 +- 质量：相关 Python 文件 Ruff；核心服务、Mixin 和端点继续低于 800 行。 +- 所有后端测试只在 `local-x-financial-linux` 容器内执行，单命令超时不超过 60 秒。 + +## 指标与验收 + +- [A1] 每个首次提交的连接器事件最多一个 `events` 用量；重放、鉴权失败和冲突为零。 +- [A2] 连接器或附件业务回滚后 `usage_meter_events` 为零，预占为 `released`，相同操作可重新预占。 +- [A3] 附件配额拒绝时原文件内容和路径完全不变。 +- [A4] 附件数据库回滚后原文件恢复；提交后新文件/删除结果与数据库一致。 +- [A5] 用量 metadata 中不存在 external event ID、correlation ID、报销单号、文件名或业务正文。 +- [A6] 连接器、附件归集、报销端点、OCR、RuntimeChat 和商业直接运行相关回归在容器内通过。 + +## 风险与开放问题 + +- 本地文件系统与数据库不是分布式事务。进程在文件暂存后、事务终态回调前被强杀时，隐藏备份可能需要启动恢复器扫描；当前正常异常和显式 commit/rollback 已闭环。 +- `X-Request-ID` 当前为可选。缺失时采用内容绑定的保守幂等身份，优先避免客户端网络重试重复收费；若未来按每次写请求收费，应把请求 ID 升级为必填并为业务上传建立持久幂等记录。 +- `bytes` 是源文件成功写入流量，不等于实时保留容量。若商业模式采用存储包或 GB-month，需要新增周期容量快照、删除后容量释放和归档层级计价。 + +## 本轮实现记录 + +- 2026-07-17：完成连接器首次接受事件的 `events` 预占—提交后结算，并确保重放、鉴权失败、payload 冲突、配额拒绝和回滚不计量。 +- 2026-07-17：完成附件源文件 `bytes` 执行前硬配额、事务文件暂存、上传/删除/费用明细删除/整单删除的提交与回滚闭环。 +- 2026-07-17：完成事务回调批次逆序补偿、未配置计量器调用方 Session 短路和已释放预占同账期安全重开。 diff --git a/document/development/2026-07-17/feature/commercial-resource-boundaries/TODO.md b/document/development/2026-07-17/feature/commercial-resource-boundaries/TODO.md new file mode 100644 index 0000000..0c42fa7 --- /dev/null +++ b/document/development/2026-07-17/feature/commercial-resource-boundaries/TODO.md @@ -0,0 +1,59 @@ +# 商业资源权威边界闭环 开发 TODO + +更新时间：2026-07-17 + +关联方案：[CONCEPT.md](./CONCEPT.md) + +## 使用规则 + +- 任务边界、计量口径和安全约束以 CONCEPT 对应章节为准。 +- `[x]` 只表示已有代码或容器验证证据；没有证据不得勾选。 +- 所有后端测试必须在 `local-x-financial-linux` 容器内运行，单命令最长 60 秒。 + +## 1. 调研与边界 + +- [x] [CONCEPT: 背景与问题] 盘点连接器认证、重放、冲突、业务 commit 和 operational event 边界。证据：`financial_connector_ingestion.py`、`financial_connectors.py` 与连接器服务/端点测试。 +- [x] [CONCEPT: 背景与问题] 盘点附件上传覆盖、单附件删除、费用明细删除、整单删除和批量归集事务。证据：`expense_claim_attachment_operations.py`、`expense_claims.py`、`expense_receipt_association.py`。 +- [x] [CONCEPT: 目标与非目标] 冻结连接器 `events` 与附件源文件写入 `bytes` 两个唯一 meter，不把删除伪装成正向用量。证据：两个商业 observer 的固定工具名和 required basis。 + +## 2. 契约与设计 + +- [x] [CONCEPT: 后端] 定义业务事务提交后结算、回滚释放的回调协议。证据：`commercial_transaction_callbacks.py`。 +- [x] [CONCEPT: 算法/规则] 定义 `events=1` 与 `bytes=len(content)` 的权威数量，错误 basis 预占前拒绝。证据：`required_quantity_basis` 与资源边界测试。 +- [x] [CONCEPT: 数据] 定义商业 metadata 脱敏白名单，不复制业务原文与敏感标识。证据：`test_connector_only_meters_new_authenticated_committed_event` 反向断言。 + +## 3. 后端实现 + +- [x] [CONCEPT: 后端] 连接器在认证和既有事件判断之后预占，事件提交后结算。证据：`financial_connector_commercial.py`、`financial_connector_ingestion.py`。 +- [x] [CONCEPT: 降级策略] 连接器重放、鉴权失败、冲突和事务回滚不写用量。证据：`test_commercial_resource_boundaries.py`。 +- [x] [CONCEPT: 后端] 附件在任何旧文件移动或新文件写入前按源文件 bytes 预占。证据：`stage_attachment_replacement()` 调用顺序与配额拒绝测试。 +- [x] [CONCEPT: 后端] 上传回滚恢复旧目录，提交后清理备份并追加用量。证据：附件 commit/rollback 资源测试。 +- [x] [CONCEPT: 后端] 单附件、费用明细和整张报销单删除均绑定数据库事务。证据：`stage_attachment_deletion()`、`stage_claim_attachment_deletion()` 与既有删除回归。 +- [x] [CONCEPT: 后端] 同一 Session 多操作提交顺序结算、回滚逆序恢复。证据：事务批次实现与连续替换 LIFO 测试。 +- [x] [CONCEPT: 降级策略] 未配置 meter 时在调用方 Session 只读短路，不让独立 SQLite Session 回滚业务。证据：Direct bridge `lookup_session`、附件归集 31 项通过。 +- [x] [CONCEPT: 后端] 允许相同 released 预占在相同账期重开并重新检查配额。证据：Direct operation released retry 测试与连接器回滚后重试测试。 + +## 4. 算法/规则实现 + +- [x] [CONCEPT: 硬配额判断] 使用 `used + held + requested <= hard_limit` 的既有锁内配额算法。证据：`CommercialRuntimeReservationService.reserve()` 与资源配额拒绝测试。 +- [x] [CONCEPT: 附件写入计量口径] 只结算成功持久化的源文件 bytes，不包含 OCR/预览/metadata。证据：observer authoritative quantity 与 metadata。 +- [x] [CONCEPT: 业务事务终态] 回滚不记 usage/cost，已释放操作可再次 permit。证据：商业直接运行与资源边界回归。 + +## 5. 前端实现 + +- [x] [CONCEPT: 前端] 上传端点接收 `X-Request-ID` 并把商业拒绝映射为 HTTP 429。证据：`reimbursements.py`。 +- [x] [CONCEPT: 目标与非目标] 本切片不新增商业或附件页面。证据：CONCEPT 非目标；本轮无前端文件变更。 + +## 6. 测试与验证 + +- [x] [CONCEPT: 测试方案] 连接器首次提交、重放、冲突、鉴权失败、回滚、重试、配额和脱敏通过。证据：容器内 `test_commercial_resource_boundaries.py` 9 项通过；商业资源、Direct、reservation、OCR、RuntimeChat 与连接器组合最终 `63 passed`。 +- [x] [CONCEPT: 测试方案] 商业 Direct、OCR、RuntimeChat 相关回归通过。证据：容器内组合 33 项通过；新增 released retry 后 Direct + 资源组合 22 项通过；最终商业/连接器组合 63 项通过。 +- [x] [CONCEPT: 测试方案] 附件归集和票据夹回归通过。证据：容器内与报销端点最终组合 `54 passed`。 +- [x] [CONCEPT: 测试方案] 报销端点与附件专项通过。证据：容器内报销/归集组合 54 项、Expense Claim attachment `20 passed, 101 deselected`。 +- [x] [CONCEPT: 测试方案] 连接器既有服务、端点和配置生命周期回归通过。证据：容器内 15 项通过；最终纳入商业/连接器组合 63 项通过。 +- [x] [CONCEPT: 指标与验收] 相关 Python 文件 Ruff、compileall 通过，核心文件均低于 800 行。证据：容器检查退出码 0；最大相关文件 `expense_claim_attachment_operations.py` 为 787 行。 + +## 7. 文档收尾 + +- [x] [CONCEPT: 本轮实现记录] 创建 CONCEPT 与分阶段 TODO，记录 meter、事务、降级和验证口径。证据：本目录两份文档。 +- [x] [CONCEPT: 风险与开放问题] 记录进程强杀恢复、可选请求 ID 和 GB-month 容量计费边界。证据：CONCEPT 风险章节。 diff --git a/document/development/2026-07-17/feature/engineering-closure-and-production-readiness/CONCEPT.md b/document/development/2026-07-17/feature/engineering-closure-and-production-readiness/CONCEPT.md new file mode 100644 index 0000000..1fc6fa3 --- /dev/null +++ b/document/development/2026-07-17/feature/engineering-closure-and-production-readiness/CONCEPT.md @@ -0,0 +1,157 @@ +# AI 费用闭环工程收口与生产就绪边界 + +日期：2026-07-17 + +## 功能一句话 + +把 X-Financial 收口为一条租户安全、可学习、可解释、可计量的费用闭环：用户从申请、票据、报销、预审、审批到付款归档尽量少填少等，企业能看见风险、节省和真实成本，同时不把模拟数据包装成生产价值。 + +## 背景与问题 + +此前系统已经有申请、报销、审批、AI 助手和分析页面，但存在四类系统性断点： + +- 业务链路能跑，但申请、票据、审批、支付、ERP、归档和价值事实没有统一闭环。 +- AI 能给建议，但用户反馈、工作流结果、记忆、few-shot 和发布质量没有形成受控学习链。 +- 单租户演示可用，但员工、知识、规则资产、Hermes、报告、缓存、向量库和文档编辑仍有跨租户风险。 +- 能展示费用，却不能严格区分确认现金节省、工时价值、风险暴露、预计机会、平台收入和内部成本。 + +本轮工程改造围绕上述断点逐步完成，不以页面数量或 mock 日志作为完成标准。 + +## 目标与非目标 + +### 目标 + +- 完成申请到付款、ERP、归档和冲回的可验证费用链路。 +- 让 AI 从可信的字段修改、提交、审批、付款、风险处置和人工标签中学习，并保留解释、撤销和发布门禁。 +- 对本轮纳入的 Claim、Employee、Agent Asset、Knowledge、Ontology、Hermes、Report 等共享核心数据建立可信会话、显式租户、复合约束、首层查询过滤和跨租户失败关闭；仍以 JSON 保存 tenant 的 legacy 状态继续列为后续迁移。 +- 建立 Savings Ledger、CFO 价值看板、商业权益、资源计量、客户 ROI、平台成本和定价走廊。 +- 将大型 Service 按访问策略、身份解析、持久化、规则、附件、计量和投影职责拆分，受代码体积门禁的核心类/组件保持低于 800 行。 + +### 非目标 + +- 不替客户决定首个支付/ERP provider、签名映射、会计期间、汇率来源、退款口径或大额双签阈值。 +- 不用 mock 回执冒充真实现金、真实开票、真实回款或真实客户 ROI。 +- 不在没有生产域名、可信 TLS、备份副本和真实 SMTP 的情况下声称完成生产上线。 +- 不以一次工程验证替代 30/90 天真实企业试点和商业定价验证。 + +## 用户与场景 + +- 员工：通过 AI 预填、票据归集、结构化预审和断点续办，减少填表和退回。 +- 直属领导、预算负责人和财务：在租户安全的任务队列中处理例外、风险、豁免、支付和财务确认。 +- CFO/管理层：按币种和证据等级查看节省、机会、周期、预算、风险护栏和数据质量。 +- 租户管理员：管理企业知识、规则资产、记忆、报告配置和商业权益，但不能越权替业务人员自证。 +- 平台运营：管理套餐、订阅、计量、成本、发布门禁和连接器配置，不能跨租户读取业务正文。 + +## 功能能力 + +### 费用闭环 + +- Expense Case、Link、Business Event 将申请、票据、报销、预审、审批、付款、ERP、归档和冲回串成可回放链路。 +- 服务端预览决策、预审握手、审批动作和风险处置使用版本、指纹、请求 ID、事务和乐观前置条件防止陈旧重放。 +- 财务连接器区分 production-mode 外部事件契约、内部人工确认和 test/mock/staging 模拟事实；错误金额、币种、单据或状态不会推进付款，真实外部现金仍须 provider 联调证明。 + +### 越用越智能 + +- AI Decision、Feedback、Workflow Outcome、Memory Evidence 和 few-shot 按租户、主体、场景、规则版本与证据等级隔离。 +- 个人及企业/部门低敏偏好可解释、可过期、可撤销；企业规则和当前输入始终高于个人记忆。 +- 发布遥测从真实 observation、可信人工 label、盲审负样本和保守 recall 进入 Canary/Release Guard；证据不足保持 collecting。 + +### 风险与安全 + +- 不透明 Bearer 会话是身份事实；请求中的 tenant、actor、reviewer、role 不能覆盖服务端上下文。 +- Employee、Claim、Agent Asset、Knowledge、Ontology、Hermes、Report、Qdrant、文件路径和缓存键均按租户隔离。 +- ONLYOFFICE 使用数据库一次性会话、资源绑定 token、DNS/IP 校验、可信 origin、大小/MIME/OOXML 校验和重放拒绝。 + +### 节省与商业闭环 + +- Savings Ledger 分离 baseline、opportunity、realization、evidence 和 append-only event；未确认结果不进入确认现金 KPI。 +- CFO 看板将现金、工时、风险暴露和预计机会分开，并显式展示 collecting、unavailable 和 coverage gap。 +- 商业层分离套餐、订阅、权益、用量、内部成本、账期、客户 ROI、贡献毛利和定价建议。 +- Orchestrator、OCR、Runtime Chat、连接器和附件源文件写入均接入权威 permit/reserve/commit/release 计量边界。 + +## 方案设计 + +```mermaid +flowchart LR + A["申请与票据"] --> B["服务端预览与预审"] + B --> C["审批任务与风险处置"] + C --> D["支付/ERP 连接器"] + D --> E["归档与冲回"] + B --> F["反馈、记忆与 few-shot"] + C --> F + D --> G["Savings Ledger"] + E --> G + G --> H["CFO 价值与 ROI"] + A --> I["商业预占与计量"] + B --> I + D --> I + F --> J["盲审遥测与 Release Guard"] +``` + +核心边界如下： + +1. 认证层生成可信 `CurrentUserContext`，业务入口不得从请求体补造身份。 +2. 业务服务以 `tenant_id` 作为第一层 SQL 条件，ORM 复合外键和数据库约束作为第二层保护。 +3. 状态与业务事件在同一事务提交；外部副作用和资源计量使用稳定请求 ID、追加事实和补偿状态。 +4. 学习只消费可信服务端事实；自由文本评论、mock 数据和未确认推断不得进入训练或价值 KPI。 +5. 分析投影只读事实账本，并保留币种、时间窗口、证据等级和数据质量状态。 + +## 数据与契约 + +- Alembic 正式链从 Expense Case、认证、AI 学习一直升级到 `20260717_0028`，migration-owned 表由启动前置检查统一管理。 +- `20260716_0015` 至 `0024` 建立 Savings、商业计量、连接器、发布遥测、账期、运行事件、盲审和资源数量口径。 +- `20260717_0025` 至 `0028` 建立租户身份、Agent Asset、Knowledge、Hermes/Ontology/Report 安全基础。 +- 关键 append-only 表由数据库 trigger 阻止 UPDATE/DELETE；幂等键和请求指纹区分安全重放与冲突载荷。 +- production-mode 签名事件、内部确认、模拟回执、确认节省、预计机会、收入和成本使用不同类型，不互相降级替代；本地自签事件不作为真实现金证据。 + +## 算法与规则 + +- 硬配额：`used + held + requested <= hard_limit`，预占在业务提交后结算，回滚后释放并允许同账期安全重试。 +- 确认现金节省：仅汇总 `finance_confirmed + canonical + cash` 的 realization，冲回通过负向追加事实抵消。 +- 客户 ROI：按币种分别计算确认价值与客户费用，不跨币种强行相加；证据不足返回 unavailable。 +- 贡献毛利：平台收入减去可归属模型、OCR、存储、连接器和其他运行成本；内部成本与客户价值分账。 +- 定价走廊：成本下限、确认价值上限、成功费封顶和合同约束共同决定建议，系统不自动替客户签订价格。 +- 发布门禁：只有 observation、独立可信 label、盲审负样本和保守置信下界达到阈值才允许晋级；失败或 collecting 保持 stable。 + +## 测试方案 + +所有后端、集成、迁移和依赖验证以 Docker 容器 `local-x-financial-linux` 的 `/app` 为唯一事实来源，单命令限制 60 秒。 + +- 后端：176 个测试文件按有界分片或专项运行，费用主服务 121 项单独回归；所有检查通过，条件跳过的 PostgreSQL 项随后在真实 PostgreSQL 探针补跑。 +- PostgreSQL：fresh schema、完整 upgrade/downgrade/re-upgrade、复合租户约束、append-only、迁移保护和并发专项最终 `87 passed / 0 skipped / 0 failed`，最终 head 为 `20260717_0028`。 +- 前端：Node 全量 `815 passed / 0 failed`，Vite production build 通过；仅保留 chunk size 提示。 +- 移动端：`npm run lint` 与 `npx tsc --noEmit` 通过；真实移动 API 与设备浏览器链路仍属于上线验收。 +- 静态质量：所有 197 个新增 Python 文件通过 Ruff；目标模块 compileall、受门禁核心类/组件 800 行检查和 `git diff --check` 通过。全仓 Ruff 仍包含既有基线格式债，不在本轮批量改写用户已有代码。 + +## 指标与验收 + +### 工程验收 + +- A1：申请到支付/ERP/归档/冲回的纵向事实链可通过 E2E 重放。 +- A2：现金、工时、风险和预计机会分账，缺数据不伪造为 0。 +- A3：身份、租户、角色、业务范围和双人复核在服务端及数据库层失败关闭。 +- A4：费用基线、预算、异常归因、节省漏斗和 CFO 下钻使用租户安全事实。 +- A5：商业权益、配额、账期、用量、成本、ROI、毛利和定价建议可审计。 +- A6：AI 反馈、记忆、few-shot、盲审、Canary 和回滚均有证据等级和降级路径。 +- A7：迁移、并发、后端、前端、移动静态检查和差异检查形成可重复验证记录。 +- A8：文档明确区分工程完成、生产上线和真实商业验证，不用 mock 冒充后两者。 + +### 真实试点指标 + +以下指标必须由首个企业在 30/90 天试点中建立基线后评估：报销创建时间、自动填充率、首次提交完整率、退回率、人工触点、完成周期、风险反馈、确认现金节省、工时价值、客户 ROI 和平台贡献毛利。 + +## 本轮实现记录 + +本轮约定的六个工程步骤已完成：商业资源计量与 EmployeeService 拆分、Expense Claim 访问策略与身份解析拆分、全链租户安全、后端/迁移/并发验证、前端/移动静态验证，以及文档与 bug 记录收口。 + +这六步范围内不再存在需要继续编码才能证明的阻断项。生产上线和商业验证仍需要目标环境与客户决策；上位长期路线图中统一 Outbox/legacy 清理、移动实机闭环、供应商事实和高级 AI 管理等扩展能力也没有被本轮文档悄悄标成完成，详见同目录 `TODO.md` 第 6-8 节。 + +## 风险与开放问题 + +- ONLYOFFICE 生产下载域名必须在应用容器解析为允许的公网地址并使用可信 TLS；开发网络的保留地址会按设计拒绝。 +- `0025 → 0028` 必须在生产备份副本演练历史归属和受控回滚，不能用 disposable 空库代替真实数据演练。 +- 首个支付/ERP provider、字段映射、SLA、会计期间、汇率、批次拆分、退款和大额双签需要客户确认。 +- SMTP、企业报告收件人和实际投递审计需要逐租户配置。 +- 消息平台、移动设备真实流程、浏览器关键链路和私有部署安全验收需要目标环境联调。 +- 30/90 天基线、客户财务签字、目标毛利、价值分享比例、合同、税率、开票和回款边界不能由代码自行完成。 +- 上位长期路线图仍保留统一 correlation/Outbox、旧模型收敛、完整移动端、供应商事实、消息/SSO 模板、数据导出与高级 AI 管理等产品扩展；它们不阻断本轮六步收口，但属于“完整产品愿景”后续工作。 diff --git a/document/development/2026-07-17/feature/engineering-closure-and-production-readiness/TODO.md b/document/development/2026-07-17/feature/engineering-closure-and-production-readiness/TODO.md new file mode 100644 index 0000000..0266118 --- /dev/null +++ b/document/development/2026-07-17/feature/engineering-closure-and-production-readiness/TODO.md @@ -0,0 +1,81 @@ +# AI 费用闭环工程收口与生产就绪 TODO + +更新时间：2026-07-17 + +关联方案：[CONCEPT.md](./CONCEPT.md) + +## 使用规则 + +- 每项必须回链 `CONCEPT.md`；没有代码、迁移、接口、容器或真实环境证据不得勾选。 +- `[x]` 代表本轮工程范围已完成，不代表生产环境或真实商业试点自动完成。 +- mock/test/staging 只能验证契约和降级，不能证明真实现金、开票、回款、客户 ROI 或生产可用性。 + +## 1. 功能闭环 + +- [x] [CONCEPT: 费用闭环] 完成申请、票据、报销、预审、审批、付款、ERP、归档和冲回的可回放链路。 + 证据：Expense Case/Business Event、财务连接器、Savings Ledger；`test_expense_financial_value_chain_e2e.py` 与相关服务测试通过。 +- [x] [CONCEPT: 越用越智能] 完成可信反馈、工作流结果、个人/企业记忆、few-shot、盲审遥测、Canary 和 Release Guard。 + 证据：AI learning/memory、release telemetry/review/recall 模块；相关后端与 PostgreSQL 并发测试通过。 +- [x] [CONCEPT: 节省与商业闭环] 完成 Savings Ledger、CFO 价值看板、商业权益/计量/成本/ROI/定价建议。 + 证据：0015/0016/0019/0021/0024 迁移，Savings/CFO/Commercial 服务、端点和前端组件。 + +## 2. 租户安全与代码结构 + +- [x] [CONCEPT: 风险与安全] 收口 Bearer 会话、Claim、Employee、Agent Asset、Knowledge、Ontology、Hermes、Report、Qdrant、文件和缓存租户边界。 + 证据：0025-0028 迁移与 tenant security 测试；生产 `CurrentUserContext` 无缺失 tenant 构造。 +- [x] [CONCEPT: 风险与安全] 完成 ONLYOFFICE 一次性会话、资源绑定、SSRF/DNS/IP、格式和重放保护。 + 证据：Agent Asset/Knowledge ONLYOFFICE 安全服务与回归测试。 +- [x] [CONCEPT: 目标与非目标] 完成大型核心模块职责拆分，保持受门禁核心类/组件低于 800 行。 + 证据：Employee 776 行、ExpenseClaimAccessPolicy 701 行；访问策略、身份解析、目录维护、附件计量等均为独立模块。 + +## 3. 商业资源权威边界 + +- [x] [CONCEPT: 节省与商业闭环] Orchestrator、OCR、Runtime Chat、连接器和附件写入接入 permit/reserve/commit/release。 + 证据：commercial direct/runtime bridge、connector observer、attachment commercial；资源组合 63 项、附件/端点 54 项通过。 +- [x] [CONCEPT: 算法与规则] 连接器仅按已认证且成功提交的事件计 `events=1`，附件仅按成功持久化源文件计 bytes。 + 证据：`test_commercial_resource_boundaries.py` 覆盖重放、冲突、鉴权失败、回滚、配额、重试和脱敏。 +- [x] [CONCEPT: 算法与规则] 回滚释放、同账期安全重试、硬配额和未配置兼容均保持事务正确。 + 证据：Direct、reservation、OCR、Runtime Chat、连接器和附件组合回归通过。 + +## 4. 容器验证 + +- [x] [CONCEPT: 测试方案] 后端 176 个测试文件完成有界分片或专项检查，费用主服务 121 项单独通过。 + 证据：所有分片退出码 0；条件 PostgreSQL 跳过已在真实探针补跑。 +- [x] [CONCEPT: 测试方案] fresh PostgreSQL 完成迁移、降级、再升级、并发和数据库约束专项。 + 证据：`87 passed / 0 skipped / 0 failed`，最终 head `20260717_0028`，一次性数据库已清理。 +- [x] [CONCEPT: 测试方案] Web 全量测试和生产构建通过。 + 证据：`815 passed / 0 failed`；Vite production build 通过。 +- [x] [CONCEPT: 测试方案] Mobile lint 与 TypeScript 静态检查通过。 + 证据：`npm run lint`、`npx tsc --noEmit` 均退出码 0。 +- [x] [CONCEPT: 测试方案] 新增 Python、编译、文件大小和差异质量门禁通过。 + 证据：197 个新增 Python 文件 Ruff 通过；目标 compileall、受门禁核心类/组件 800 行检查和 `git diff --check` 通过；全仓历史 Ruff 债单独披露。 + +## 5. 文档与可追溯性 + +- [x] [CONCEPT: 本轮实现记录] 回填 Savings、商业、连接器、发布遥测和上位 AI 闭环 TODO 的真实完成状态。 + 证据：2026-07-13、2026-07-16 对应功能文档及本目录。 +- [x] [CONCEPT: 本轮实现记录] 为本轮生产 bug 创建独立修复日志，并先完成 upstream/local-ahead 检查。 + 证据：`document/development/2026-07-17/dev-logs/bugs/`；`origin/main` 无新提交，本地 ahead 17 已记录。 +- [x] [CONCEPT: 指标与验收] 明确工程完成、生产上线、真实试点三种完成口径。 + 证据：CONCEPT“目标与非目标”“指标与验收”“风险与开放问题”。 + +## 6. 生产上线（需要目标环境） + +- [ ] [CONCEPT: 风险与开放问题] 配置生产 ONLYOFFICE 允许 origin，并在应用容器验证公网 DNS、可信 TLS 和真实编辑回写。 +- [ ] [CONCEPT: 风险与开放问题] 在生产备份副本演练 `0025 → 0028`、历史默认企业归属、回滚保护和恢复。 +- [ ] [CONCEPT: 风险与开放问题] 为每个启用企业配置 SMTP、报告收件人并验证实际投递审计。 +- [ ] [CONCEPT: 风险与开放问题] 完成真实浏览器、移动设备、消息平台和私有部署环境的关键流程验收。 + +## 7. 客户与商业验证（需要业务决策） + +- [ ] [CONCEPT: 风险与开放问题] 确认首个支付/ERP provider、签名、字段映射、SLA、重试、批次、汇率、会计期间、退款和大额双签口径。 +- [ ] [CONCEPT: 真实试点指标] 采集首个企业 30/90 天基线并验证效率、风险、现金节省、工时价值和客户 ROI。 +- [ ] [CONCEPT: 风险与开放问题] 由客户财务签字确认节省归因、去重、汇率、工时价值和报告口径。 +- [ ] [CONCEPT: 风险与开放问题] 冻结目标毛利、包含量、超额策略、价值分享、合同、税率、开票、回款、坏账和收入确认。 + +## 8. 长期产品路线图（不属于本轮六步阻断） + +- [ ] [CONCEPT: 风险与开放问题] 统一所有阶段 correlation/事务 Outbox，并完成 `agent_conversations` 结构化租户、旧 `ReimbursementRequest`、`risk_flags_json` 和少见状态迁移。 +- [ ] [CONCEPT: 风险与开放问题] 完成移动端真实 API、拍照/OCR/票据/草稿实机闭环，以及全浏览器键盘、焦点和响应式验收。 +- [ ] [CONCEPT: 风险与开放问题] 接入租户化供应商/合同/单位价格事实、真实消息/SSO/电子档案模板、删除传播和数据导出。 +- [ ] [CONCEPT: 风险与开放问题] 扩展“我的 AI 记忆”、保留/敏感级别/动作上限配置，以及自动化依据、撤销、抽检和版本可视化。 diff --git a/document/development/2026-07-17/feature/hermes-ontology-tenant-security/CONCEPT.md b/document/development/2026-07-17/feature/hermes-ontology-tenant-security/CONCEPT.md new file mode 100644 index 0000000..0e0c033 --- /dev/null +++ b/document/development/2026-07-17/feature/hermes-ontology-tenant-security/CONCEPT.md @@ -0,0 +1,106 @@ +# Hermes、本体解析与财务报告多租户安全 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +让本体解析、员工行为画像、Hermes 扫描、数字员工看板和定时财务报告从请求到存储全程绑定可信企业，并在任何租户上下文缺失或冲突时停止运行。 + +## 背景与问题 + +本体解析曾在建立 `AgentRun` 前调用模型，并从全局员工、组织、客户、供应商、项目和单据字典构造提示词；员工画像详情可按任意员工 ID 查询；Hermes 扫描、看板和财务报告存在全表读取、全局收件人以及跨企业复用存储路径的风险。内部 Orchestrator 调用还可在没有认证用户时继续运行，无法证明 `AgentRun` 的企业归属。 + +## 目标与非目标 + +### 目标 + +- HTTP 入口只信任 `CurrentUserContext.tenant_id`，内部任务只接受显式、已注册且启用的 `trusted_tenant_id`。 +- 本体解析在任何模型调用前创建租户化 `AgentRun`，商业运行上下文和失败证据可追溯。 +- 员工、组织、费用、应收、应付、画像、风险、提醒和看板查询在首个 SQL 中限定租户。 +- 员工画像仅允许本人、直属领导、财务/高管或管理员读取；越权和跨租户统一返回 404。 +- 财务报告按租户配置收件人、生成内容、存储文件和幂等运行账本，不使用全局邮件回退。 +- 所有 Hermes 定时任务逐个枚举 `status=active` 的企业，运行记录同时在 route/ontology 保存租户快照。 + +### 非目标 + +- 不提供客户端选择或覆盖租户的兼容参数。 +- 不实现跨企业合并分析、集团穿透报表或平台运营后台。 +- 不在本切片中重构预算分配等仍属旧模型的全部业务域。 + +## 信任边界 + +```text +HTTP 请求 ── CurrentUserContext.tenant_id ─┐ + ├─→ tenant-scoped AgentRun +内部任务 ── active trusted_tenant_id ──────┘ ├─ route_json.tenant_id + └─ ontology_json.tenant_id +``` + +- 客户端 `context_json.tenant_id` 会被可信租户覆盖，不能参与授权。 +- 同时传入登录用户和内部租户时，两者必须一致。 +- 内部任务租户必须存在于租户注册表且状态为 active；缺失、停用或不存在均在创建 Run 前拒绝。 +- 后台消费者从数据库 Run 恢复租户，并校验 route/ontology 两份快照一致。 + +## 功能设计 + +### 本体解析 + +- 接口覆盖请求中的 user id，并把认证企业传给 `SemanticOntologyService`。 +- `parse()` 先建立 `running/pending_model_analysis` Run，再加载租户字典、商业运行上下文并调用模型。 +- 模型、规则降级和失败路径都更新同一 Run；失败不会留下无企业、无状态的调用。 +- 员工、部门、费用申请、应收、应付和项目参考目录均以租户作为第一层 SQL 条件。 + +### 员工画像 + +- 快照保存 `tenant_id`，并用 `(tenant_id, subject_id)` 复合外键绑定员工。 +- Profile Service 的快照、费用单和 Agent Run 查询均先限定租户。 +- 详情接口先在当前企业解析目标员工，再执行本人/直属领导/财务/高管/管理员访问矩阵。 +- 附带 `claim_id` 时必须同时属于目标员工和当前企业。 + +### Hermes 与数字员工 + +- 风险扫描、画像扫描、风险线索、提醒扫描和看板均接受显式租户。 +- 调度器只枚举 active 企业，并为每个企业独立创建任务与结果。 +- 看板只统计 route/ontology 租户快照均匹配的 Run；缺少或冲突快照的历史记录不会被猜测归属。 +- 提醒任务的费用单、员工和关联报销查询在首个 SQL 限制当前企业。 + +### 财务分析报告 + +- `TenantFinanceReportConfig` 保存企业启停状态和经校验的收件人。 +- `TenantFinanceReportRun` 以 `(tenant_id, idempotency_key)` 防止同企业同周期重复发送。 +- 报告上下文只读取当前企业的费用、风险、画像和 Agent Run。 +- 邮件发送只使用当前企业配置；未配置、停用或无有效收件人时 fail-closed。 +- 文件存储目录使用企业标识哈希，避免路径注入和跨企业文件覆盖。 + +## 数据与迁移 + +`20260717_0028_hermes_ontology_tenant_security.py` 完成： + +- 员工行为画像、Hermes 任务配置/日志和风险报告新增租户列、索引、唯一约束和复合外键。 +- 新增租户财务报告配置和周期运行账本。 +- 旧记录回填为 `default`，再移除 server default，后续写入必须显式确定企业。 +- 仅支持 PostgreSQL；upgrade/downgrade 在任何 DDL 前检查方言，其他数据库直接拒绝。 +- downgrade 在存在非默认企业事实或报告账本时拒绝有损回滚。 + +## 降级与回滚 + +- 租户缺失、停用、不存在或上下文冲突：不创建 Run、不查询业务数据。 +- 跨企业员工、画像、费用单：统一按不存在处理。 +- 企业未配置报告收件人：保留失败/跳过证据，不回退环境变量中的全局地址。 +- 历史 Run 没有双租户快照：看板不纳入企业统计。 +- 生产升级前必须在数据库备份副本演练 `0027 → 0028 → 0027 → 0028`。 + +## 测试与验收 + +- 两企业本体目录、画像 IDOR、Hermes 扫描、风险线索、看板和报告内容/收件人/路径/幂等隔离。 +- Orchestrator 缺租户、停用租户、不存在租户和认证/内部租户冲突均在 Run 前拒绝。 +- 本体模型调用前存在 running Run，失败后仍保存同企业失败证据。 +- PostgreSQL 完成 fresh `base → 0028`、`0028 → 0027 → 0028` 及旧表回填/外键验证。 +- 所有后端测试和 Ruff 均在 `local-x-financial-linux` 容器内执行，单命令不超过 60 秒。 + +## 本轮实现记录 + +- 2026-07-17：完成本体解析前置 Run、可信商业上下文和租户参考目录。 +- 2026-07-17：完成员工画像模型、服务和 API 的租户隔离与访问矩阵。 +- 2026-07-17：完成 Hermes 扫描、提醒、看板、调度和财务报告的逐租户运行。 +- 2026-07-17：完成 `20260717_0028` PostgreSQL 迁移、反向回滚和安全回归。 diff --git a/document/development/2026-07-17/feature/hermes-ontology-tenant-security/TODO.md b/document/development/2026-07-17/feature/hermes-ontology-tenant-security/TODO.md new file mode 100644 index 0000000..bb7e1c0 --- /dev/null +++ b/document/development/2026-07-17/feature/hermes-ontology-tenant-security/TODO.md @@ -0,0 +1,40 @@ +# Hermes、本体解析与财务报告多租户安全 开发 TODO + +更新时间：2026-07-17 + +关联方案：[CONCEPT.md](./CONCEPT.md) + +## 1. 信任边界 + +- [x] [CONCEPT: 信任边界] HTTP 入口只使用认证用户企业，覆盖请求中的 user/tenant 上下文。 +- [x] [CONCEPT: 信任边界] Orchestrator 内部调用必须显式传入已注册、启用的 `trusted_tenant_id`。 +- [x] [CONCEPT: 信任边界] Agent Run 的 route/ontology 同时保存租户快照，冲突时 fail-closed。 + +## 2. 本体与画像 + +- [x] [CONCEPT: 本体解析] 在模型调用前创建 running Run，并在成功、规则降级和失败路径闭环状态。 +- [x] [CONCEPT: 本体解析] 员工、组织、费用、应收、应付和项目目录全部首 SQL 租户过滤。 +- [x] [CONCEPT: 员工画像] 画像模型新增租户字段、复合员工外键和租户索引。 +- [x] [CONCEPT: 员工画像] 实现本人、直属领导、财务/高管/管理员访问矩阵及 claim 归属校验。 + +## 3. Hermes 与报告 + +- [x] [CONCEPT: Hermes 与数字员工] 风险扫描、画像扫描、风险线索、提醒和看板接入显式租户。 +- [x] [CONCEPT: Hermes 与数字员工] 调度器逐 active 企业运行，不创建默认或全局扫描。 +- [x] [CONCEPT: 财务分析报告] 新增企业报告配置、收件人校验和周期幂等账本。 +- [x] [CONCEPT: 财务分析报告] 报告上下文、邮件收件人和文件路径按企业隔离。 + +## 4. 数据与验证 + +- [x] [CONCEPT: 数据与迁移] 完成 `20260717_0028` 数据回填、约束、索引、报告表和 downgrade 保护。 +- [x] [CONCEPT: 数据与迁移] upgrade/downgrade 在 DDL 前拒绝非 PostgreSQL 方言。 +- [x] [CONCEPT: 测试与验收] 完成本体、画像、Hermes、报告和 Orchestrator 两企业安全测试。 +- [x] [CONCEPT: 测试与验收] 完成旧本体 72 项、Orchestrator 16 项及认证/关联草稿 18 项回归。 +- [x] [CONCEPT: 测试与验收] 完成一次性 PostgreSQL fresh、回滚重升和旧结构迁移探针。 +- [x] [CONCEPT: 测试与验收] 相关 Python 文件通过容器 Ruff，Git diff 无空白错误。 + +## 5. 上线 + +- [x] [CONCEPT: 本轮实现记录] 完成概念文档、TODO 和安全修复日志。 +- [ ] [CONCEPT: 降级与回滚] 上线前在生产备份副本执行 `0027 → 0028 → 0027 → 0028`，确认历史默认企业归属。 +- [ ] [CONCEPT: 财务分析报告] 为每个启用企业确认 SMTP、报告收件人和实际投递审计，不使用全局兜底地址。 diff --git a/document/development/2026-07-17/feature/knowledge-tenant-security/CONCEPT.md b/document/development/2026-07-17/feature/knowledge-tenant-security/CONCEPT.md new file mode 100644 index 0000000..1672cb8 --- /dev/null +++ b/document/development/2026-07-17/feature/knowledge-tenant-security/CONCEPT.md @@ -0,0 +1,162 @@ +# 知识库多租户隔离与安全编辑 概念文档 + +更新时间：2026-07-17 + +## 功能一句话 + +让每个企业只读写自己的知识文件、元数据和 LightRAG/Qdrant 命名空间，同时以平台制度只读层和一次性 ONLYOFFICE 会话安全地贯通知识查询、预览与编辑。 + +## 背景与问题 + +原知识库把所有企业的文件、`.index.json`、`.lightrag`、运行时缓存和 Qdrant workspace 放在同一全局空间。API、后台索引和定时任务还能在没有可信 `tenant_id` 时继续工作，导致同名文件覆盖、跨租户列表/详情/原文读取、索引串读和错误的默认租户归属风险。 + +ONLYOFFICE 原回调仅依赖可伪造或可重放的短 payload，并直接下载回调 URL。匿名请求可以借此访问内部地址、跟随重定向、下载超大或错误格式内容，最终覆盖知识文件。预览会话与编辑会话也没有不可变的租户、资源、文档 key、版本和一次性状态绑定。 + +## 目标与非目标 + +### 目标 + +- 文件、索引 JSON、LightRAG 本地状态、运行时实例和 Qdrant workspace 全部按可信租户隔离。 +- 旧全局制度资料无损复制到显式 `platform` 空间，并始终以只读方式提供给租户。 +- 列表、详情、原文、上传、删除、同步和查询均从认证用户或数据库 Agent Run 获取租户，不接受客户端上下文覆盖。 +- ONLYOFFICE token 同时绑定 tenant、resource scope、document、key、version、editable、audience、expiry 和 JTI。 +- 编辑回调以数据库状态机实现一次性消费；预览会话永不回写。 +- 回调下载仅访问配置的文档服务，拒绝重定向、私网/回环/链路本地解析、DNS 重绑定、超限和非 OOXML 内容。 +- 定时索引只枚举租户注册表中的 `active` 租户，不再使用默认租户。 + +### 非目标 + +- 不允许租户经普通 API 新建或修改平台制度；平台资料由受控部署流程维护。 +- 不把租户知识数据合并成一个共享向量集合后再依赖过滤器补救。 +- 不为私网 ONLYOFFICE 地址提供安全降级开关；不满足公网解析要求时回写必须 fail-closed。 +- 不在本切片中实现知识内容质量评估、模型微调或新的知识运营前端。 + +## 用户与场景 + +- 租户管理员：上传、覆盖、删除和触发当前企业的知识同步。 +- 普通员工：联合查询本企业知识与平台只读制度，预览有权限的原文。 +- 知识运营人员：在受控编辑模式中修改租户 Office 文档，保存后形成新版本。 +- Hermes 调度器：逐个处理活跃租户的增量知识，不创建或猜测默认租户。 +- 平台管理员：通过部署资产提供跨租户可见但不可写的平台制度。 + +## 功能能力 + +- 租户目录：`storage/knowledge/tenants/{tenant_id}/`。 +- 平台目录：`storage/knowledge/platform/`，所有业务入口只读。 +- 每个作用域独立 `.index.json`、`.lightrag`、workspace 和运行时缓存键。 +- 租户查询采用“租户空间 + 平台只读空间”隔离检索后融合，不在存储层混库。 +- 旧 `storage/knowledge/{固定目录}` 与旧索引首次启动时复制到平台空间，源文件不删除。 +- ONLYOFFICE content token 默认 5 分钟；callback session 默认 4 小时、最长 12 小时，并在首次写回前原子进入 `processing`。 +- 回写成功进入 `consumed`，失败进入 `failed`；已消费、失败、撤销或过期会话不能再次写入。 +- 后台索引线程从数据库 Agent Run 的 route/ontology 双份租户上下文重新校验，线程参数只能作为一致性断言。 + +## 方案设计 + +### 前端契约 + +- 知识文档新增 `scope` 与 `readOnly` 字段。 +- 平台文档可以查看、下载和预览，但编辑入口必须隐藏或禁用。 +- ONLYOFFICE 配置默认 `mode=view`；只有租户管理员显式请求 `editable=true` 时才返回编辑配置。 +- 回调和 content URL 中的 token 是文档服务专用凭证，不暴露为普通用户授权能力。 + +### 后端职责 + +- `knowledge_tenant_scope`：校验租户标识，生成文件路径、workspace 和缓存键，执行旧资料到平台空间的无损迁移。 +- `knowledge`：编排租户/平台文档、权限、文件操作、查询融合和 ONLYOFFICE 配置。 +- `knowledge_index_state`：管理当前作用域的 JSON 元数据与 ingest 状态。 +- `knowledge_rag`：只使用当前作用域的 working dir、workspace、缓存实例与 Qdrant 命名空间。 +- `knowledge_run_scope`：从持久化 Agent Run 的 route/ontology 双份上下文还原并校验后台任务租户。 +- `knowledge_onlyoffice_security`：签发/验证会话、原子消费 JTI、执行安全网络下载与 OOXML 校验。 +- `knowledge_onlyoffice_callback`：按 token 中已验证的资源作用域定位文件，校验基线后回写新版本。 +- `knowledge_scheduler`：从 `tenants.status=active` 枚举租户并创建显式内部身份。 + +### 数据 + +新增 migration-owned 表 `knowledge_onlyoffice_sessions`： + +- 身份：`jti`、`tenant_id`、`resource_scope`、`document_id`。 +- 不可变绑定：`document_key`、`document_version`、`audience`、`editable`、`created_by`、`expires_at`。 +- 生命周期：`active → processing → consumed | failed`，另支持 `revoked`。 +- 平台会话仍绑定发起租户，但数据库约束要求 `editable=false`。 +- `tenant_id` 外键指向租户注册表；租户删除时清理其会话。 +- 存在会话证据时 migration downgrade 拒绝有损删除。 + +### 权限与信任边界 + +- HTTP 文档 API 只使用 `CurrentUserContext.tenant_id`。 +- 同步任务只使用认证用户租户；后台线程再与数据库 Agent Run 租户交叉校验。 +- 平台 scope 必须由服务端显式构造，客户端不能通过参数选择。 +- 租户文档 ID 在其他租户下按不存在处理；平台文档仅作为只读 fallback。 +- ONLYOFFICE token 必须同时通过签名、issuer、audience、scope、时间、数据库行和全部资源字段比较。 +- callback token 只有写入状态 `2/6` 才尝试 claim；key 不匹配时不会消耗会话。 +- 下载 URL 的 origin 必须等于配置白名单；DNS 解析的全部地址必须为公网地址，实际连接固定到已校验 IP，HTTPS 仍校验原主机证书和 SNI。 +- 不跟随 3xx；限制响应大小、MIME、ZIP 条目数、解压体积、路径穿越、加密条目和 OOXML 目录结构。 + +### 查询算法 + +租户查询先在两个物理隔离空间各自检索，再对候选做确定性融合： + +```text +tenant_workspace = base + "__tenant_" + SHA256(tenant_id)[0:20] +platform_workspace = base + "__platform" + +merged_hits = top_k( + deduplicate(tenant_hits ∪ platform_hits, by=code), + order_by=score DESC +) +``` + +运行时缓存同样使用租户哈希键，不把原始 tenant ID 放进 Qdrant workspace 名称。 + +### ONLYOFFICE 状态机 + +```text +issue(view) → active ── callback status 2/6 ──拒绝写入 +issue(edit) → active ── atomic claim ──→ processing + ├─验证/下载/写入成功→ consumed + └─任一步失败────────→ failed + +active -- exp 超时 --> 验证拒绝 +consumed/failed/revoked -- replay --> 409/拒绝 +``` + +### 降级策略 + +- 租户缺失、格式非法或与 Agent Run 冲突：拒绝任务，不回退默认租户。 +- 平台迁移源不存在：建立空平台作用域，不影响租户空间。 +- 平台或租户 RAG 不可用：保留已有本地检索降级；不会改查其他租户 workspace。 +- ONLYOFFICE 地址、JWT、DNS、MIME 或 OOXML 校验失败：返回错误并保持原文件不变。 +- 当前容器若通过网络代理把文档域名解析为非公网保留地址，真实回写会按安全策略拒绝；部署需提供满足公网解析与 TLS 的文档服务或安全反向代理。 + +## 测试方案 + +- 服务：两租户同名文件、列表/详情/原文隔离、平台只读 fallback、无 scope fail-closed。 +- RAG：workspace、缓存键、本地 chunks 和运行签名隔离。 +- 后台：只枚举 active tenant；worker 从 Agent Run 重取租户并拒绝 route/ontology 冲突。 +- ONLYOFFICE：tenant/resource/key/version/aud/exp/JTI 绑定，平台只读，会话过期，错 key 不 claim，成功只写一次，重放拒绝。 +- SSRF：错误 origin、私网 DNS、重定向、IP pinning、超限、错误 MIME、损坏 OOXML。 +- 迁移：revision/down_revision、表/外键/check、非 PostgreSQL 拒绝、ownership/preflight 和有证据 downgrade 拒绝。 +- 所有后端验证只在 `local-x-financial-linux` 容器内执行，单命令超时不超过 60 秒。 + +## 指标与验收 + +- 100% 租户知识文件、索引、LightRAG 与 Qdrant workspace 可由可信 tenant 唯一确定。 +- 跨租户文档读取、删除、同步和查询用例 100% 拒绝或不可见。 +- 平台文档 `readOnly=true`，任何编辑回调均不产生文件副作用。 +- 同一 callback JTI 最多一次进入 `processing`，重复回调 100% 拒绝。 +- 非白名单 origin、非公网解析、重定向、超限或非 OOXML 回写 100% fail-closed。 +- 定时任务只为注册表 `active` 租户建任务；无活跃租户时不生成默认数据。 +- 新增/修改 Python 文件通过 Ruff 与 compileall，定向知识安全、既有知识回归和迁移 ownership 测试全部通过。 + +## 风险与开放问题 + +- 当前 JSON 索引采用原子文件级写入之外的旧读改写模型；同一租户多进程高并发上传仍应在后续演进为数据库元数据或跨进程锁。 +- 平台制度的发布、签名和回滚需要独立的受控平台资产流程，本切片只保证业务 API 只读。 +- 部署必须确认 ONLYOFFICE 下载域名从应用容器解析为真实公网地址；不得为开发便利开放私网通配。 +- 长时编辑使用 4 小时 callback session；超过时限需重新打开文档生成新会话。 + +## 本轮实现记录 + +- 2026-07-17：完成租户/平台文件与 RAG 命名空间拆分、旧资料无损平台迁移、API 与后台任务可信租户接线。 +- 2026-07-17：完成 DB-backed ONLYOFFICE 一次性会话、平台只读预览、文档 key/version 基线校验和 SSRF/大小/MIME/OOXML 防护。 +- 2026-07-17：完成 `20260717_0027` 迁移、模型 ownership/preflight 注册、定时器 active tenant 枚举及容器定向回归；共享 PostgreSQL 探针完成 `base → 0028` 与 `0028 → 0025 → 0028`，Knowledge/AgentAsset 两类会话表均正常落库。 diff --git a/document/development/2026-07-17/feature/knowledge-tenant-security/TODO.md b/document/development/2026-07-17/feature/knowledge-tenant-security/TODO.md new file mode 100644 index 0000000..3d6428a --- /dev/null +++ b/document/development/2026-07-17/feature/knowledge-tenant-security/TODO.md @@ -0,0 +1,63 @@ +# 知识库多租户隔离与安全编辑 开发 TODO + +更新时间：2026-07-17 + +关联方案：[CONCEPT.md](./CONCEPT.md) + +## 使用规则 + +- 任务边界、信任模型和上线约束以 CONCEPT 对应章节为准。 +- `[x]` 只表示已有代码或容器验证证据；运维环境尚未验证的项目保持 `[ ]`。 +- 所有后端测试必须在 `local-x-financial-linux` 容器内运行，单命令最长 60 秒。 + +## 1. 调研与边界 + +- [x] [CONCEPT: 背景与问题] 盘点旧文件、index、LightRAG、运行时缓存和 Qdrant 的全局共享路径。证据：`knowledge.py`、`knowledge_rag.py` 调用点扫描与两租户隔离测试。 +- [x] [CONCEPT: 目标与非目标] 冻结“租户可写 + 平台只读”的双层资源模型，不提供客户端 scope 选择或私网下载降级。证据：`KnowledgeStorageScope` 与 platform 只读反向测试。 +- [x] [CONCEPT: 用户与场景] 明确 API 用户、平台资料、Hermes 调度器和 ONLYOFFICE 文档服务四类主体。证据：认证端点、scheduler 和 session service 的显式调用契约。 + +## 2. 契约与设计 + +- [x] [CONCEPT: 功能能力] 定义 `tenants/{tenant}/`、`platform/`、workspace 和 runtime cache key。证据：`knowledge_tenant_scope.py`。 +- [x] [CONCEPT: 方案设计] 为文档 DTO 增加 `scope/readOnly`，平台文档只能只读 fallback。证据：`schemas/knowledge.py` 与平台文档测试。 +- [x] [CONCEPT: ONLYOFFICE 状态机] 定义 `active → processing → consumed|failed` 和过期/重放拒绝。证据：`KnowledgeOnlyOfficeSession` 模型、`20260717_0027` 迁移与 replay 测试。 + +## 3. 后端实现 + +- [x] [CONCEPT: 后端职责] 建立 tenant/platform 文件、index 与 LightRAG 存储作用域。证据：`knowledge_tenant_scope.py`、`knowledge_index_state.py`、`knowledge_rag.py`。 +- [x] [CONCEPT: 后端职责] 将列表、详情、原文、上传、删除、同步和查询接入可信租户。证据：`knowledge.py`、`endpoints/knowledge.py`、`knowledge_sync.py`。 +- [x] [CONCEPT: 权限与信任边界] 让 Orchestrator 从数据库 Agent Run 取 tenant，并让索引 worker 比对 route/ontology tenant。证据：`orchestrator_execution.py`、`knowledge_run_scope.py`、`knowledge_index_tasks.py`、Agent Run 冲突测试。 +- [x] [CONCEPT: 后端职责] 定时器只枚举 `Tenant.status=active`，没有活跃租户时跳过。证据：`knowledge_scheduler.py` 与 active/suspended 调度测试。 +- [x] [CONCEPT: 后端职责] 迁移旧全局制度到 platform 空间且不删除源文件。证据：`migrate_legacy_library_to_platform()` 与 legacy migration 测试。 +- [x] [CONCEPT: 权限与信任边界] 新增带数据库状态的 ONLYOFFICE token、平台只读预览和一次性 callback。证据：`knowledge_onlyoffice_security.py`、`knowledge_onlyoffice_callback.py`。 +- [x] [CONCEPT: 权限与信任边界] 加固下载 origin、DNS/IP pinning、重定向、大小、MIME、ZIP 和 OOXML 校验。证据：`download_onlyoffice_document()` 与 SSRF/格式测试。 +- [x] [CONCEPT: 数据] 新增 `20260717_0027`，注册模型、schema ownership 和 migration preflight。证据：迁移文件、`db/base.py`、`models/__init__.py`、`migration_preflight.py`。 +- [x] [CONCEPT: 后端职责] 删除知识文件工具中已废弃的无状态弱 token 实现。证据：`knowledge_file_utils.py` 差异与全仓引用扫描。 + +## 4. 算法/规则实现 + +- [x] [CONCEPT: 查询算法] 以租户哈希生成不可冲突的 workspace/cache key，再隔离检索 tenant 与 platform 候选。证据：workspace/cache/local chunks 两租户测试。 +- [x] [CONCEPT: 查询算法] 对两个隔离结果按 code 去重、score 排序并截取 top-k。证据：`_merge_scoped_search_results()`。 +- [x] [CONCEPT: ONLYOFFICE 状态机] content token 默认 5 分钟，callback session 默认 4 小时且最长 12 小时。证据：JWT exp 差值断言与会话测试。 + +## 5. 前端实现 + +- [x] [CONCEPT: 前端契约] 后端 DTO 已提供 `scope/readOnly`，现有列表可安全识别平台只读资源。证据：OpenAPI 回归与 `KnowledgeDocumentRead`。 +- [x] [CONCEPT: 前端契约] ONLYOFFICE 默认返回 view 模式，只有管理员显式 `editable=true` 才获得 edit 权限。证据：config 单元测试与平台编辑拒绝测试。 +- [x] [CONCEPT: 目标与非目标] 本切片不新增知识运营页面或视觉改版。证据：CONCEPT 非目标；本轮无知识前端文件变更。 + +## 6. 测试与验证 + +- [x] [CONCEPT: 测试方案] 完成租户文件、平台只读、RAG namespace、scheduler 与 worker trust 测试。证据：容器内 `test_knowledge_tenant_security.py` 6 项通过。 +- [x] [CONCEPT: 测试方案] 完成 token 绑定、平台只读、错 key、过期、单次回写、重放和 SSRF/OOXML 测试。证据：容器内 `test_knowledge_onlyoffice_tenant_security.py` 8 项通过。 +- [x] [CONCEPT: 测试方案] 完成既有知识服务、RAG、同步、配置、解析、runtime 与 OpenAPI 回归。证据：容器内 8 个测试文件 31 项通过。 +- [x] [CONCEPT: 测试方案] 完成 migration、preflight 与 ownership 静态回归。证据：容器内 163 项通过、1 项因无 PostgreSQL 测试 DSN 跳过。 +- [x] [CONCEPT: 测试方案] 完成相关 Agent Run 与鉴权回归。证据：容器内 20 项通过。 +- [x] [CONCEPT: 指标与验收] 完成目标 Python 文件 Ruff 和 compileall。证据：容器命令均退出码 0。 +- [x] [CONCEPT: 测试方案] 在一次性 PostgreSQL 测试数据库执行迁移链。证据：共享迁移探针完成 `base → 0028` 与 `0028 → 0025 → 0028`，`knowledge_onlyoffice_sessions` 和 `agent_asset_onlyoffice_sessions` 均正常落库；有事实 downgrade 仍由各迁移显式拒绝。 + +## 7. 文档收尾 + +- [x] [CONCEPT: 本轮实现记录] 更新 CONCEPT、分阶段 TODO 和三份 bug 修复日志。证据：本目录两份文档及 `dev-logs/bugs/knowledge-*.md`。 +- [x] [CONCEPT: 风险与开放问题] 记录 JSON index 多进程竞争、平台发布流程和长时编辑边界。证据：CONCEPT 风险章节。 +- [ ] [CONCEPT: 降级策略] 上线前确认 ONLYOFFICE 下载域名从应用容器解析为公网地址并使用可信 TLS。证据要求：生产 `ONLYOFFICE_DOWNLOAD_ALLOWED_ORIGINS` 配置与容器 DNS/TLS 验证；当前开发网络解析到 `198.18.0.0/15` 时会按设计拒绝真实回写。 diff --git a/server/alembic/versions/20260716_0015_savings_value_ledger.py b/server/alembic/versions/20260716_0015_savings_value_ledger.py new file mode 100644 index 0000000..ed5f0a2 --- /dev/null +++ b/server/alembic/versions/20260716_0015_savings_value_ledger.py @@ -0,0 +1,800 @@ +"""add tenant-safe savings value ledger and append-only audit events + +Revision ID: 20260716_0015 +Revises: 20260716_0014 +Create Date: 2026-07-16 20:10:00 +""" + +from __future__ import annotations + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0015" +down_revision: str | None = "20260716_0014" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0015 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_ledger_for_downgrade() -> None: + bind = op.get_bind() + counts = { + table_name: int( + bind.scalar(sa.text(f"SELECT COUNT(*) FROM {table_name}")) or 0 + ) + for table_name in ( + "profile_baseline_snapshots", + "savings_opportunities", + "savings_realizations", + "savings_evidence_links", + "savings_events", + ) + } + if any(counts.values()): + summary = ", ".join(f"{name}={count}" for name, count in counts.items()) + raise RuntimeError( + "cannot downgrade savings value ledger: immutable value facts exist " + f"({summary})" + ) + + +def _json_object_default() -> sa.TextClause: + return sa.text("'{}'::json") + + +def _json_array_default() -> sa.TextClause: + return sa.text("'[]'::json") + + +def upgrade() -> None: + _require_postgresql() + op.create_table( + "profile_baseline_snapshots", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("baseline_key", sa.String(length=160), nullable=False), + sa.Column("baseline_type", sa.String(length=32), nullable=False), + sa.Column("dimension_type", sa.String(length=50), nullable=False), + sa.Column("dimension_id", sa.String(length=160), nullable=False), + sa.Column("metric_key", sa.String(length=100), nullable=False), + sa.Column("unit", sa.String(length=30), nullable=False), + sa.Column("original_currency", sa.String(length=3), nullable=True), + sa.Column("baseline_value", sa.Numeric(20, 4), nullable=False), + sa.Column("window_start", sa.DateTime(timezone=True), nullable=True), + sa.Column("window_end", sa.DateTime(timezone=True), nullable=True), + sa.Column("sample_count", sa.Integer(), nullable=False, server_default="0"), + sa.Column("method", sa.String(length=80), nullable=False), + sa.Column("query_fingerprint", sa.String(length=80), nullable=False), + sa.Column("data_quality_status", sa.String(length=20), nullable=False), + sa.Column("data_quality_score", sa.Numeric(5, 4), nullable=False), + sa.Column( + "quality_issues_json", + sa.JSON(), + nullable=False, + server_default=_json_array_default(), + ), + sa.Column("algorithm_version", sa.String(length=80), nullable=False), + sa.Column("policy_version", sa.String(length=120), nullable=True), + sa.Column("policy_effective_from", sa.Date(), nullable=True), + sa.Column("policy_effective_to", sa.Date(), nullable=True), + sa.Column("target_resource_type", sa.String(length=50), nullable=True), + sa.Column("target_resource_id", sa.String(length=160), nullable=True), + sa.Column("frozen_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("frozen_by", sa.String(length=120), nullable=False), + sa.Column("valid_until", sa.DateTime(timezone=True), nullable=True), + sa.Column("version", sa.Integer(), nullable=False, server_default="1"), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "baseline_type IN ('historical_cohort', 'policy_counterfactual', 'manual')", + name="ck_profile_baseline_snapshots_type", + ), + sa.CheckConstraint( + "data_quality_status IN ('complete', 'partial', 'insufficient', 'invalid')", + name="ck_profile_baseline_snapshots_quality_status", + ), + sa.CheckConstraint( + "baseline_value >= 0 AND sample_count >= 0", + name="ck_profile_baseline_snapshots_values", + ), + sa.CheckConstraint( + "data_quality_score >= 0 AND data_quality_score <= 1", + name="ck_profile_baseline_snapshots_quality_score", + ), + sa.CheckConstraint( + "window_end IS NULL OR window_start IS NOT NULL", + name="ck_profile_baseline_snapshots_window_pair", + ), + sa.CheckConstraint( + "window_start IS NULL OR window_end IS NULL OR window_end >= window_start", + name="ck_profile_baseline_snapshots_window_order", + ), + sa.CheckConstraint( + "baseline_type != 'historical_cohort' OR " + "(window_start IS NOT NULL AND window_end IS NOT NULL AND sample_count > 0)", + name="ck_profile_baseline_snapshots_historical_shape", + ), + sa.CheckConstraint( + "baseline_type != 'policy_counterfactual' OR " + "(policy_version IS NOT NULL AND length(trim(policy_version)) > 0 " + "AND policy_effective_from IS NOT NULL AND target_resource_type IS NOT NULL " + "AND target_resource_id IS NOT NULL)", + name="ck_profile_baseline_snapshots_policy_shape", + ), + sa.CheckConstraint( + "policy_effective_to IS NULL OR policy_effective_from IS NOT NULL", + name="ck_profile_baseline_snapshots_policy_pair", + ), + sa.CheckConstraint( + "policy_effective_from IS NULL OR policy_effective_to IS NULL " + "OR policy_effective_to >= policy_effective_from", + name="ck_profile_baseline_snapshots_policy_order", + ), + sa.CheckConstraint( + "valid_until IS NULL OR valid_until >= frozen_at", + name="ck_profile_baseline_snapshots_validity", + ), + sa.CheckConstraint( + "length(trim(baseline_key)) > 0 AND length(trim(query_fingerprint)) > 0", + name="ck_profile_baseline_snapshots_keys", + ), + sa.CheckConstraint("version >= 1", name="ck_profile_baseline_snapshots_version"), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", "id", name="uq_profile_baseline_snapshots_tenant_id" + ), + sa.UniqueConstraint( + "tenant_id", + "baseline_key", + name="uq_profile_baseline_snapshots_tenant_key", + ), + ) + op.create_index( + "ix_profile_baseline_snapshots_lookup", + "profile_baseline_snapshots", + [ + "tenant_id", + "baseline_type", + "dimension_type", + "dimension_id", + "metric_key", + "frozen_at", + ], + ) + op.create_index( + "ix_profile_baseline_snapshots_quality", + "profile_baseline_snapshots", + ["tenant_id", "data_quality_status", "frozen_at"], + ) + op.create_table( + "savings_opportunities", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("opportunity_key", sa.String(length=180), nullable=False), + sa.Column("benefit_key", sa.String(length=180), nullable=False), + sa.Column("expense_case_id", sa.String(length=36), nullable=False), + # expense_claims/items 仍由 legacy bootstrap 管理，仅保存已校验软引用。 + sa.Column("claim_id", sa.String(length=36), nullable=False), + sa.Column("claim_no_snapshot", sa.String(length=80), nullable=False), + sa.Column("claim_item_id", sa.String(length=36), nullable=True), + sa.Column("discovery_business_event_id", sa.String(length=36), nullable=True), + sa.Column("source_type", sa.String(length=50), nullable=False), + sa.Column("source_id", sa.String(length=160), nullable=False), + sa.Column("category", sa.String(length=60), nullable=False), + sa.Column("value_kind", sa.String(length=20), nullable=False), + sa.Column("title", sa.String(length=200), nullable=False), + sa.Column("description", sa.Text(), nullable=False), + sa.Column("exposure_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("baseline_snapshot_id", sa.String(length=36), nullable=False), + sa.Column("baseline_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("target_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("estimated_gross", sa.Numeric(20, 4), nullable=False), + sa.Column("estimated_cost", sa.Numeric(20, 4), nullable=False), + sa.Column("estimated_net", sa.Numeric(20, 4), nullable=False), + sa.Column("estimated_low", sa.Numeric(20, 4), nullable=False), + sa.Column("estimated_high", sa.Numeric(20, 4), nullable=False), + sa.Column("confidence", sa.Numeric(5, 4), nullable=False), + sa.Column("currency", sa.String(length=3), nullable=False), + sa.Column("reporting_currency", sa.String(length=3), nullable=False), + sa.Column("attribution_method", sa.String(length=60), nullable=False), + sa.Column("ai_decision_id", sa.String(length=36), nullable=True), + sa.Column("suggested_action", sa.Text(), nullable=False), + sa.Column("owner_id", sa.String(length=120), nullable=False), + sa.Column("owner_name", sa.String(length=120), nullable=False), + sa.Column("owner_role", sa.String(length=60), nullable=False), + sa.Column("due_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("status", sa.String(length=24), nullable=False, server_default="identified"), + sa.Column("version", sa.Integer(), nullable=False, server_default="1"), + sa.Column( + "dimension_json", sa.JSON(), nullable=False, server_default=_json_object_default() + ), + sa.Column( + "baseline_snapshot_json", + sa.JSON(), + nullable=False, + server_default=_json_object_default(), + ), + sa.Column( + "evidence_json", sa.JSON(), nullable=False, server_default=_json_array_default() + ), + sa.Column("accepted_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("started_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("realized_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("closed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column( + "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.Column( + "updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "value_kind IN ('cash', 'labor')", + name="ck_savings_opportunities_value_kind", + ), + sa.CheckConstraint( + "status IN ('identified', 'accepted', 'in_progress', 'realized', " + "'verified', 'reversed', 'rejected', 'expired')", + name="ck_savings_opportunities_status", + ), + sa.CheckConstraint( + "exposure_amount >= 0 AND baseline_amount >= 0 AND target_amount >= 0 " + "AND estimated_gross >= 0 AND estimated_cost >= 0 AND estimated_net >= 0 " + "AND estimated_low >= 0 AND estimated_high >= 0", + name="ck_savings_opportunities_amounts", + ), + sa.CheckConstraint( + "estimated_net = estimated_gross - estimated_cost", + name="ck_savings_opportunities_net_math", + ), + sa.CheckConstraint( + "estimated_low <= estimated_net AND estimated_net <= estimated_high", + name="ck_savings_opportunities_interval", + ), + sa.CheckConstraint( + "confidence >= 0 AND confidence <= 1", + name="ck_savings_opportunities_confidence", + ), + sa.CheckConstraint("version >= 1", name="ck_savings_opportunities_version"), + sa.CheckConstraint( + "length(trim(benefit_key)) > 0 AND length(trim(opportunity_key)) > 0", + name="ck_savings_opportunities_keys", + ), + sa.CheckConstraint( + "length(trim(currency)) = 3 AND length(trim(reporting_currency)) = 3", + name="ck_savings_opportunities_currencies", + ), + sa.CheckConstraint( + "status NOT IN ('accepted', 'in_progress', 'realized', 'verified', 'reversed') " + "OR accepted_at IS NOT NULL", + name="ck_savings_opportunities_acceptance", + ), + sa.CheckConstraint( + "status NOT IN ('in_progress', 'realized', 'verified', 'reversed') " + "OR started_at IS NOT NULL", + name="ck_savings_opportunities_started", + ), + sa.CheckConstraint( + "status NOT IN ('realized', 'verified', 'reversed') OR realized_at IS NOT NULL", + name="ck_savings_opportunities_realized", + ), + sa.CheckConstraint( + "status NOT IN ('verified', 'reversed') OR verified_at IS NOT NULL", + name="ck_savings_opportunities_verified", + ), + sa.CheckConstraint( + "status NOT IN ('verified', 'reversed', 'rejected', 'expired') " + "OR closed_at IS NOT NULL", + name="ck_savings_opportunities_closed", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + name="fk_savings_opportunities_tenant_case", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "expense_case_id", "discovery_business_event_id"], + [ + "business_events.tenant_id", + "business_events.expense_case_id", + "business_events.id", + ], + name="fk_savings_opportunities_tenant_event", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "baseline_snapshot_id"], + ["profile_baseline_snapshots.tenant_id", "profile_baseline_snapshots.id"], + name="fk_savings_opportunities_tenant_baseline", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "expense_case_id", "ai_decision_id"], + ["ai_decisions.tenant_id", "ai_decisions.expense_case_id", "ai_decisions.id"], + name="fk_savings_opportunities_tenant_ai_decision", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_savings_opportunities_tenant_id"), + sa.UniqueConstraint( + "tenant_id", "opportunity_key", name="uq_savings_opportunities_tenant_key" + ), + ) + op.create_index( + "ix_savings_opportunities_tenant_status_due", + "savings_opportunities", + ["tenant_id", "status", "due_at"], + ) + op.create_index( + "ix_savings_opportunities_tenant_case", + "savings_opportunities", + ["tenant_id", "expense_case_id", "created_at"], + ) + op.create_index( + "ix_savings_opportunities_tenant_benefit", + "savings_opportunities", + ["tenant_id", "benefit_key"], + ) + op.create_index( + "ix_savings_opportunities_tenant_owner", + "savings_opportunities", + ["tenant_id", "owner_id", "status"], + ) + op.create_table( + "savings_realizations", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("realization_key", sa.String(length=180), nullable=False), + sa.Column("opportunity_id", sa.String(length=36), nullable=False), + sa.Column("expense_case_id", sa.String(length=36), nullable=False), + sa.Column("claim_id", sa.String(length=36), nullable=False), + sa.Column("claim_item_id", sa.String(length=36), nullable=True), + sa.Column("business_event_id", sa.String(length=36), nullable=True), + sa.Column("realization_type", sa.String(length=20), nullable=False), + sa.Column("reversal_of_realization_id", sa.String(length=36), nullable=True), + sa.Column("realized_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("recorded_by_id", sa.String(length=120), nullable=False), + sa.Column("recorded_by_name", sa.String(length=120), nullable=False), + sa.Column("actual_gross", sa.Numeric(20, 4), nullable=False), + sa.Column("incremental_cost", sa.Numeric(20, 4), nullable=False), + sa.Column("actual_net", sa.Numeric(20, 4), nullable=False), + sa.Column("original_currency", sa.String(length=3), nullable=False), + sa.Column("reporting_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("reporting_currency", sa.String(length=3), nullable=False), + sa.Column("fx_rate", sa.Numeric(20, 8), nullable=False), + sa.Column("fx_source", sa.String(length=80), nullable=False), + sa.Column("fx_date", sa.Date(), nullable=False), + sa.Column("fx_version", sa.String(length=80), nullable=False), + sa.Column("attribution_method", sa.String(length=60), nullable=False), + sa.Column("attribution_ratio", sa.Numeric(7, 6), nullable=False), + sa.Column("benefit_key", sa.String(length=180), nullable=False), + sa.Column( + "dedupe_status", + sa.String(length=24), + nullable=False, + server_default="pending_review", + ), + sa.Column("canonical_realization_id", sa.String(length=36), nullable=True), + sa.Column( + "status", + sa.String(length=24), + nullable=False, + server_default="pending_confirmation", + ), + sa.Column("finance_confirmer_id", sa.String(length=120), nullable=True), + sa.Column("finance_confirmer_name", sa.String(length=120), nullable=True), + sa.Column("confirmed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("confirmation_note", sa.Text(), nullable=True), + sa.Column("rejected_by_id", sa.String(length=120), nullable=True), + sa.Column("rejected_by_name", sa.String(length=120), nullable=True), + sa.Column("rejected_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("rejection_reason", sa.Text(), nullable=True), + sa.Column("reversed_by_id", sa.String(length=120), nullable=True), + sa.Column("reversed_by_name", sa.String(length=120), nullable=True), + sa.Column("reversed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("reversal_reason", sa.Text(), nullable=True), + sa.Column( + "baseline_snapshot_json", + sa.JSON(), + nullable=False, + server_default=_json_object_default(), + ), + sa.Column( + "final_snapshot_json", + sa.JSON(), + nullable=False, + server_default=_json_object_default(), + ), + sa.Column( + "evidence_json", sa.JSON(), nullable=False, server_default=_json_array_default() + ), + sa.Column("version", sa.Integer(), nullable=False, server_default="1"), + sa.Column( + "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.Column( + "updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "realization_type IN ('actual', 'reversal')", + name="ck_savings_realizations_type", + ), + sa.CheckConstraint( + "dedupe_status IN ('pending_review', 'canonical', 'duplicate', 'excluded')", + name="ck_savings_realizations_dedupe_status", + ), + sa.CheckConstraint( + "status IN ('pending_confirmation', 'finance_confirmed', 'rejected', 'reversed')", + name="ck_savings_realizations_status", + ), + sa.CheckConstraint( + "attribution_ratio > 0 AND attribution_ratio <= 1", + name="ck_savings_realizations_attribution", + ), + sa.CheckConstraint( + "incremental_cost >= 0 AND fx_rate > 0", + name="ck_savings_realizations_cost_fx", + ), + sa.CheckConstraint( + "actual_net = actual_gross - incremental_cost", + name="ck_savings_realizations_net_math", + ), + sa.CheckConstraint( + "(realization_type = 'actual' AND reversal_of_realization_id IS NULL " + "AND actual_gross >= 0 AND actual_net >= 0 AND reporting_amount >= 0) OR " + "(realization_type = 'reversal' AND reversal_of_realization_id IS NOT NULL " + "AND actual_gross <= 0 AND actual_net <= 0 AND reporting_amount <= 0)", + name="ck_savings_realizations_amount_direction", + ), + sa.CheckConstraint( + "(dedupe_status = 'duplicate' AND canonical_realization_id IS NOT NULL " + "AND canonical_realization_id <> id) OR " + "(dedupe_status != 'duplicate' AND canonical_realization_id IS NULL)", + name="ck_savings_realizations_duplicate_target", + ), + sa.CheckConstraint( + "status != 'finance_confirmed' OR " + "(finance_confirmer_id IS NOT NULL AND finance_confirmer_name IS NOT NULL " + "AND confirmed_at IS NOT NULL AND confirmation_note IS NOT NULL " + "AND (realization_type = 'reversal' OR finance_confirmer_id <> recorded_by_id) " + "AND dedupe_status = 'canonical')", + name="ck_savings_realizations_confirmation", + ), + sa.CheckConstraint( + "status != 'rejected' OR (rejected_by_id IS NOT NULL " + "AND rejected_by_name IS NOT NULL AND rejected_at IS NOT NULL " + "AND rejection_reason IS NOT NULL)", + name="ck_savings_realizations_rejection", + ), + sa.CheckConstraint( + "status != 'reversed' OR (reversed_by_id IS NOT NULL " + "AND reversed_by_name IS NOT NULL AND reversed_at IS NOT NULL " + "AND reversal_reason IS NOT NULL)", + name="ck_savings_realizations_reversal", + ), + sa.CheckConstraint("version >= 1", name="ck_savings_realizations_version"), + sa.CheckConstraint( + "length(trim(realization_key)) > 0 AND length(trim(benefit_key)) > 0", + name="ck_savings_realizations_keys", + ), + sa.CheckConstraint( + "length(trim(original_currency)) = 3 " + "AND length(trim(reporting_currency)) = 3", + name="ck_savings_realizations_currencies", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "opportunity_id"], + ["savings_opportunities.tenant_id", "savings_opportunities.id"], + name="fk_savings_realizations_tenant_opportunity", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + name="fk_savings_realizations_tenant_case", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "expense_case_id", "business_event_id"], + [ + "business_events.tenant_id", + "business_events.expense_case_id", + "business_events.id", + ], + name="fk_savings_realizations_tenant_event", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + [ + "tenant_id", "opportunity_id", "benefit_key", "reversal_of_realization_id" + ], + [ + "savings_realizations.tenant_id", "savings_realizations.opportunity_id", + "savings_realizations.benefit_key", "savings_realizations.id", + ], + name="fk_savings_realizations_tenant_reversal", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "benefit_key", "canonical_realization_id"], + [ + "savings_realizations.tenant_id", "savings_realizations.benefit_key", + "savings_realizations.id", + ], + name="fk_savings_realizations_tenant_canonical", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_savings_realizations_tenant_id"), + sa.UniqueConstraint( + "tenant_id", "realization_key", name="uq_savings_realizations_tenant_key" + ), + sa.UniqueConstraint( + "tenant_id", "opportunity_id", "benefit_key", "id", + name="uq_savings_realizations_tenant_opportunity_benefit_id", + ), + sa.UniqueConstraint( + "tenant_id", "benefit_key", "id", + name="uq_savings_realizations_tenant_benefit_id", + ), + ) + op.create_index( + "uq_savings_realizations_actual_canonical_benefit", + "savings_realizations", + ["tenant_id", "benefit_key"], + unique=True, + postgresql_where=sa.text( + "realization_type = 'actual' AND dedupe_status = 'canonical'" + ), + ) + op.create_index( + "ix_savings_realizations_tenant_status_time", + "savings_realizations", + ["tenant_id", "status", "realized_at"], + ) + op.create_index( + "ix_savings_realizations_tenant_opportunity", + "savings_realizations", + ["tenant_id", "opportunity_id", "realized_at"], + ) + op.create_index( + "ix_savings_realizations_tenant_benefit", + "savings_realizations", + ["tenant_id", "benefit_key", "dedupe_status"], + ) + op.create_table( + "savings_evidence_links", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("evidence_key", sa.String(length=180), nullable=False), + sa.Column("entity_type", sa.String(length=20), nullable=False), + sa.Column("entity_id", sa.String(length=36), nullable=False), + sa.Column("baseline_snapshot_id", sa.String(length=36), nullable=True), + sa.Column("opportunity_id", sa.String(length=36), nullable=True), + sa.Column("realization_id", sa.String(length=36), nullable=True), + sa.Column("evidence_role", sa.String(length=50), nullable=False), + sa.Column("resource_type", sa.String(length=50), nullable=False), + sa.Column("resource_id", sa.String(length=160), nullable=False), + sa.Column("source_system", sa.String(length=80), nullable=False), + sa.Column("external_event_id", sa.String(length=160), nullable=True), + sa.Column("content_hash", sa.String(length=80), nullable=False), + sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("collected_at", sa.DateTime(timezone=True), nullable=False), + sa.Column( + "verification_status", + sa.String(length=20), + nullable=False, + server_default="unverified", + ), + sa.Column("verified_by", sa.String(length=120), nullable=True), + sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True), + sa.Column( + "metadata_json", sa.JSON(), nullable=False, server_default=_json_object_default() + ), + sa.Column( + "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "entity_type IN ('baseline', 'opportunity', 'realization')", + name="ck_savings_evidence_links_entity_type", + ), + sa.CheckConstraint( + "(entity_type = 'baseline' AND baseline_snapshot_id = entity_id " + "AND opportunity_id IS NULL AND realization_id IS NULL) OR " + "(entity_type = 'opportunity' AND opportunity_id = entity_id " + "AND baseline_snapshot_id IS NULL AND realization_id IS NULL) OR " + "(entity_type = 'realization' AND realization_id = entity_id " + "AND baseline_snapshot_id IS NULL AND opportunity_id IS NULL)", + name="ck_savings_evidence_links_entity_shape", + ), + sa.CheckConstraint( + "verification_status IN ('unverified', 'verified', 'rejected', 'unavailable')", + name="ck_savings_evidence_links_verification", + ), + sa.CheckConstraint( + "verification_status != 'verified' OR " + "(verified_by IS NOT NULL AND verified_at IS NOT NULL)", + name="ck_savings_evidence_links_verifier", + ), + sa.CheckConstraint( + "length(trim(evidence_key)) > 0 AND length(trim(content_hash)) > 0", + name="ck_savings_evidence_links_keys", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "baseline_snapshot_id"], + ["profile_baseline_snapshots.tenant_id", "profile_baseline_snapshots.id"], + name="fk_savings_evidence_links_tenant_baseline", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "opportunity_id"], + ["savings_opportunities.tenant_id", "savings_opportunities.id"], + name="fk_savings_evidence_links_tenant_opportunity", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "realization_id"], + ["savings_realizations.tenant_id", "savings_realizations.id"], + name="fk_savings_evidence_links_tenant_realization", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_savings_evidence_links_tenant_id"), + sa.UniqueConstraint( + "tenant_id", "evidence_key", name="uq_savings_evidence_links_tenant_key" + ), + ) + op.create_index( + "ix_savings_evidence_links_entity", + "savings_evidence_links", + ["tenant_id", "entity_type", "entity_id", "collected_at"], + ) + op.create_index( + "ix_savings_evidence_links_resource", + "savings_evidence_links", + ["tenant_id", "resource_type", "resource_id"], + ) + op.create_table( + "savings_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("aggregate_type", sa.String(length=20), nullable=False), + sa.Column("aggregate_id", sa.String(length=36), nullable=False), + sa.Column("baseline_snapshot_id", sa.String(length=36), nullable=True), + sa.Column("opportunity_id", sa.String(length=36), nullable=True), + sa.Column("realization_id", sa.String(length=36), nullable=True), + sa.Column("action", sa.String(length=60), nullable=False), + sa.Column("actor_id", sa.String(length=120), nullable=False), + sa.Column("actor_name", sa.String(length=120), nullable=False), + sa.Column("actor_type", sa.String(length=20), nullable=False), + sa.Column("request_id", sa.String(length=120), nullable=False), + sa.Column("expected_version", sa.Integer(), nullable=False), + sa.Column("result_version", sa.Integer(), nullable=False), + sa.Column("payload_fingerprint", sa.String(length=80), nullable=False), + sa.Column( + "payload_json", sa.JSON(), nullable=False, server_default=_json_object_default() + ), + sa.Column("before_json", sa.JSON(), nullable=False, server_default=_json_object_default()), + sa.Column("after_json", sa.JSON(), nullable=False, server_default=_json_object_default()), + sa.Column( + "response_json", sa.JSON(), nullable=False, server_default=_json_object_default() + ), + sa.Column("correlation_id", sa.String(length=64), nullable=True), + sa.Column("causation_id", sa.String(length=64), nullable=True), + sa.Column( + "occurred_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "aggregate_type IN ('baseline', 'opportunity', 'realization')", + name="ck_savings_events_aggregate_type", + ), + sa.CheckConstraint( + "(aggregate_type = 'baseline' AND baseline_snapshot_id = aggregate_id " + "AND opportunity_id IS NULL AND realization_id IS NULL) OR " + "(aggregate_type = 'opportunity' AND opportunity_id = aggregate_id " + "AND baseline_snapshot_id IS NULL AND realization_id IS NULL) OR " + "(aggregate_type = 'realization' AND realization_id = aggregate_id " + "AND baseline_snapshot_id IS NULL AND opportunity_id IS NULL)", + name="ck_savings_events_aggregate_shape", + ), + sa.CheckConstraint("length(trim(action)) > 0", name="ck_savings_events_action"), + sa.CheckConstraint( + "actor_type IN ('user', 'system', 'agent', 'service')", + name="ck_savings_events_actor_type", + ), + sa.CheckConstraint( + "expected_version >= 0 AND result_version >= 1 " + "AND result_version >= expected_version", + name="ck_savings_events_version", + ), + sa.CheckConstraint( + "length(trim(request_id)) > 0 AND length(trim(payload_fingerprint)) > 0", + name="ck_savings_events_request", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "baseline_snapshot_id"], + ["profile_baseline_snapshots.tenant_id", "profile_baseline_snapshots.id"], + name="fk_savings_events_tenant_baseline", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "opportunity_id"], + ["savings_opportunities.tenant_id", "savings_opportunities.id"], + name="fk_savings_events_tenant_opportunity", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "realization_id"], + ["savings_realizations.tenant_id", "savings_realizations.id"], + name="fk_savings_events_tenant_realization", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_savings_events_tenant_id"), + sa.UniqueConstraint( + "tenant_id", "actor_id", "request_id", name="uq_savings_events_actor_request" + ), + sa.UniqueConstraint( + "tenant_id", + "aggregate_type", + "aggregate_id", + "result_version", + name="uq_savings_events_aggregate_version", + ), + ) + op.create_index( + "ix_savings_events_tenant_aggregate_time", + "savings_events", + ["tenant_id", "aggregate_type", "aggregate_id", "occurred_at"], + ) + op.create_index( + "ix_savings_events_tenant_correlation", + "savings_events", + ["tenant_id", "correlation_id", "occurred_at"], + ) + op.execute( + """ + CREATE OR REPLACE FUNCTION prevent_savings_events_mutation() + RETURNS trigger AS $$ + BEGIN + RAISE EXCEPTION 'savings_events are append-only'; + END; + $$ LANGUAGE plpgsql + """ + ) + op.execute( + """ + CREATE TRIGGER trg_savings_events_append_only + BEFORE UPDATE OR DELETE ON savings_events + FOR EACH ROW EXECUTE FUNCTION prevent_savings_events_mutation() + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_ledger_for_downgrade() + op.execute("DROP TRIGGER IF EXISTS trg_savings_events_append_only ON savings_events") + op.execute("DROP FUNCTION IF EXISTS prevent_savings_events_mutation()") + op.drop_table("savings_events") + op.drop_table("savings_evidence_links") + op.drop_table("savings_realizations") + op.drop_table("savings_opportunities") + op.drop_table("profile_baseline_snapshots") diff --git a/server/alembic/versions/20260716_0016_commercial_metering.py b/server/alembic/versions/20260716_0016_commercial_metering.py new file mode 100644 index 0000000..e8d3725 --- /dev/null +++ b/server/alembic/versions/20260716_0016_commercial_metering.py @@ -0,0 +1,590 @@ +"""add tenant commercial contracts, entitlements, usage and cost ledgers + +Revision ID: 20260716_0016 +Revises: 20260716_0015 +Create Date: 2026-07-16 23:10:00 +""" + +from __future__ import annotations + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0016" +down_revision: str | None = "20260716_0015" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0016 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_commercial_domain_for_downgrade() -> None: + bind = op.get_bind() + counts = { + table_name: int(bind.scalar(sa.text(f"SELECT COUNT(*) FROM {table_name}")) or 0) + for table_name in ( + "tenant_commercial_plans", + "tenant_subscriptions", + "commercial_entitlements", + "usage_meter_events", + "commercial_cost_events", + ) + } + if any(counts.values()): + summary = ", ".join(f"{name}={count}" for name, count in counts.items()) + raise RuntimeError( + "cannot downgrade commercial metering: contracts or immutable facts exist " + f"({summary})" + ) + + +def _json_object_default() -> sa.TextClause: + return sa.text("'{}'::json") + + +def upgrade() -> None: + _require_postgresql() + op.create_table( + "tenant_commercial_plans", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("plan_code", sa.String(length=80), nullable=False), + sa.Column("name", sa.String(length=160), nullable=False), + sa.Column("pricing_model", sa.String(length=24), nullable=False), + sa.Column("billing_interval", sa.String(length=20), nullable=False), + sa.Column("currency", sa.String(length=3), nullable=False), + sa.Column("base_fee", sa.Numeric(20, 4), nullable=False), + sa.Column("included_seats", sa.Integer(), nullable=False, server_default="0"), + sa.Column( + "overage_enabled", sa.Boolean(), nullable=False, server_default=sa.text("false") + ), + sa.Column("status", sa.String(length=16), nullable=False, server_default="draft"), + sa.Column("effective_from", sa.DateTime(timezone=True), nullable=False), + sa.Column("effective_to", sa.DateTime(timezone=True), nullable=True), + sa.Column("version", sa.Integer(), nullable=False, server_default="1"), + sa.Column( + "contract_terms_json", + sa.JSON(), + nullable=False, + server_default=_json_object_default(), + ), + sa.Column("created_by", sa.String(length=120), nullable=False), + sa.Column( + "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.Column( + "updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "pricing_model IN ('subscription', 'usage', 'hybrid', 'pilot', 'custom')", + name="ck_tenant_commercial_plans_pricing_model", + ), + sa.CheckConstraint( + "billing_interval IN ('monthly', 'quarterly', 'annual', 'contract')", + name="ck_tenant_commercial_plans_billing_interval", + ), + sa.CheckConstraint( + "status IN ('draft', 'active', 'retired')", + name="ck_tenant_commercial_plans_status", + ), + sa.CheckConstraint( + "base_fee >= 0 AND included_seats >= 0 AND version >= 1", + name="ck_tenant_commercial_plans_values", + ), + sa.CheckConstraint( + "length(trim(plan_code)) > 0 AND length(trim(name)) > 0", + name="ck_tenant_commercial_plans_keys", + ), + sa.CheckConstraint( + "length(trim(currency)) = 3", name="ck_tenant_commercial_plans_currency" + ), + sa.CheckConstraint( + "effective_to IS NULL OR effective_to > effective_from", + name="ck_tenant_commercial_plans_effective_window", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", "id", name="uq_tenant_commercial_plans_tenant_id" + ), + sa.UniqueConstraint( + "tenant_id", + "plan_code", + "version", + name="uq_tenant_commercial_plans_tenant_code_version", + ), + ) + op.create_index( + "uq_tenant_commercial_plans_active_code", + "tenant_commercial_plans", + ["tenant_id", "plan_code"], + unique=True, + postgresql_where=sa.text("status = 'active'"), + ) + op.create_index( + "ix_tenant_commercial_plans_tenant_status", + "tenant_commercial_plans", + ["tenant_id", "status", "effective_from"], + ) + op.create_table( + "tenant_subscriptions", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("subscription_key", sa.String(length=120), nullable=False), + sa.Column("plan_id", sa.String(length=36), nullable=False), + sa.Column("status", sa.String(length=20), nullable=False), + sa.Column("starts_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("ends_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("current_period_start", sa.DateTime(timezone=True), nullable=False), + sa.Column("current_period_end", sa.DateTime(timezone=True), nullable=False), + sa.Column("seats", sa.Integer(), nullable=False), + sa.Column("base_fee_snapshot", sa.Numeric(20, 4), nullable=False), + sa.Column("currency", sa.String(length=3), nullable=False), + sa.Column("billing_interval", sa.String(length=20), nullable=False), + sa.Column("auto_renew", sa.Boolean(), nullable=False, server_default=sa.text("false")), + sa.Column("external_provider", sa.String(length=60), nullable=True), + sa.Column("external_subscription_id", sa.String(length=160), nullable=True), + sa.Column("canceled_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("version", sa.Integer(), nullable=False, server_default="1"), + sa.Column( + "metadata_json", sa.JSON(), nullable=False, server_default=_json_object_default() + ), + sa.Column("created_by", sa.String(length=120), nullable=False), + sa.Column( + "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.Column( + "updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "status IN ('trialing', 'active', 'past_due', 'suspended', 'canceled', 'expired')", + name="ck_tenant_subscriptions_status", + ), + sa.CheckConstraint( + "billing_interval IN ('monthly', 'quarterly', 'annual', 'contract')", + name="ck_tenant_subscriptions_billing_interval", + ), + sa.CheckConstraint( + "seats > 0 AND base_fee_snapshot >= 0 AND version >= 1", + name="ck_tenant_subscriptions_values", + ), + sa.CheckConstraint( + "length(trim(subscription_key)) > 0 AND length(trim(currency)) = 3", + name="ck_tenant_subscriptions_keys", + ), + sa.CheckConstraint( + "current_period_end > current_period_start", + name="ck_tenant_subscriptions_period", + ), + sa.CheckConstraint( + "ends_at IS NULL OR ends_at > starts_at", + name="ck_tenant_subscriptions_contract_window", + ), + sa.CheckConstraint( + "(external_provider IS NULL AND external_subscription_id IS NULL) OR " + "(external_provider IS NOT NULL AND external_subscription_id IS NOT NULL)", + name="ck_tenant_subscriptions_external_pair", + ), + sa.CheckConstraint( + "status != 'canceled' OR canceled_at IS NOT NULL", + name="ck_tenant_subscriptions_cancellation", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "plan_id"], + ["tenant_commercial_plans.tenant_id", "tenant_commercial_plans.id"], + name="fk_tenant_subscriptions_tenant_plan", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_tenant_subscriptions_tenant_id"), + sa.UniqueConstraint( + "tenant_id", "subscription_key", name="uq_tenant_subscriptions_tenant_key" + ), + sa.UniqueConstraint( + "tenant_id", + "external_provider", + "external_subscription_id", + name="uq_tenant_subscriptions_external_ref", + ), + ) + op.create_index( + "uq_tenant_subscriptions_current", + "tenant_subscriptions", + ["tenant_id"], + unique=True, + postgresql_where=sa.text( + "status IN ('trialing', 'active', 'past_due', 'suspended')" + ), + ) + op.create_index( + "ix_tenant_subscriptions_tenant_status_period", + "tenant_subscriptions", + ["tenant_id", "status", "current_period_end"], + ) + op.create_table( + "commercial_entitlements", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("subscription_id", sa.String(length=36), nullable=False), + sa.Column("entitlement_key", sa.String(length=120), nullable=False), + sa.Column("metric_key", sa.String(length=120), nullable=False), + sa.Column("entitlement_type", sa.String(length=20), nullable=False), + sa.Column("unit", sa.String(length=40), nullable=False), + sa.Column("included_quantity", sa.Numeric(20, 6), nullable=True), + sa.Column("hard_limit_quantity", sa.Numeric(20, 6), nullable=True), + sa.Column("reset_interval", sa.String(length=20), nullable=False), + sa.Column("overage_policy", sa.String(length=16), nullable=False), + sa.Column("status", sa.String(length=16), nullable=False), + sa.Column("effective_from", sa.DateTime(timezone=True), nullable=False), + sa.Column("effective_to", sa.DateTime(timezone=True), nullable=True), + sa.Column("version", sa.Integer(), nullable=False, server_default="1"), + sa.Column("config_json", sa.JSON(), nullable=False, server_default=_json_object_default()), + sa.Column( + "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.Column( + "updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "entitlement_type IN ('feature', 'metered', 'unlimited')", + name="ck_commercial_entitlements_type", + ), + sa.CheckConstraint( + "reset_interval IN ('none', 'monthly', 'quarterly', 'annual', 'contract')", + name="ck_commercial_entitlements_reset_interval", + ), + sa.CheckConstraint( + "overage_policy IN ('block', 'allow', 'alert')", + name="ck_commercial_entitlements_overage_policy", + ), + sa.CheckConstraint( + "status IN ('active', 'suspended', 'expired')", + name="ck_commercial_entitlements_status", + ), + sa.CheckConstraint( + "version >= 1 AND (included_quantity IS NULL OR included_quantity >= 0) " + "AND (hard_limit_quantity IS NULL OR hard_limit_quantity >= 0)", + name="ck_commercial_entitlements_values", + ), + sa.CheckConstraint( + "(entitlement_type = 'unlimited' AND included_quantity IS NULL " + "AND hard_limit_quantity IS NULL) OR " + "(entitlement_type = 'feature' AND included_quantity IN (0, 1) " + "AND (hard_limit_quantity IS NULL OR hard_limit_quantity IN (0, 1))) OR " + "(entitlement_type = 'metered' AND included_quantity IS NOT NULL " + "AND (hard_limit_quantity IS NULL OR hard_limit_quantity >= included_quantity))", + name="ck_commercial_entitlements_quota_shape", + ), + sa.CheckConstraint( + "length(trim(entitlement_key)) > 0 AND length(trim(metric_key)) > 0 " + "AND length(trim(unit)) > 0", + name="ck_commercial_entitlements_keys", + ), + sa.CheckConstraint( + "effective_to IS NULL OR effective_to > effective_from", + name="ck_commercial_entitlements_effective_window", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_entitlements_tenant_subscription", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", "id", name="uq_commercial_entitlements_tenant_id" + ), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "id", + name="uq_commercial_entitlements_tenant_subscription_id", + ), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "entitlement_key", + name="uq_commercial_entitlements_subscription_key", + ), + ) + op.create_index( + "ix_commercial_entitlements_subscription_status", + "commercial_entitlements", + ["tenant_id", "subscription_id", "status"], + ) + op.create_table( + "usage_meter_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("subscription_id", sa.String(length=36), nullable=False), + sa.Column("entitlement_id", sa.String(length=36), nullable=False), + sa.Column("event_type", sa.String(length=16), nullable=False), + sa.Column("metric_key", sa.String(length=120), nullable=False), + sa.Column("quantity", sa.Numeric(20, 6), nullable=False), + sa.Column("unit", sa.String(length=40), nullable=False), + sa.Column("period_key", sa.String(length=32), nullable=False), + sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("source_system", sa.String(length=80), nullable=False), + sa.Column("idempotency_key", sa.String(length=160), nullable=False), + sa.Column("request_fingerprint", sa.String(length=80), nullable=False), + sa.Column("reversal_of_event_id", sa.String(length=36), nullable=True), + sa.Column("subject_type", sa.String(length=60), nullable=True), + sa.Column("subject_id", sa.String(length=160), nullable=True), + sa.Column("actor_type", sa.String(length=20), nullable=False), + sa.Column("actor_id", sa.String(length=120), nullable=False), + sa.Column("correlation_id", sa.String(length=120), nullable=True), + sa.Column("trace_id", sa.String(length=120), nullable=True), + sa.Column( + "metadata_json", sa.JSON(), nullable=False, server_default=_json_object_default() + ), + sa.Column( + "recorded_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "event_type IN ('usage', 'credit', 'adjustment', 'reversal')", + name="ck_usage_meter_events_type", + ), + sa.CheckConstraint( + "(event_type = 'usage' AND quantity > 0) OR " + "(event_type = 'credit' AND quantity < 0) OR " + "(event_type IN ('adjustment', 'reversal') AND quantity <> 0)", + name="ck_usage_meter_events_quantity", + ), + sa.CheckConstraint( + "(event_type = 'reversal' AND reversal_of_event_id IS NOT NULL) OR " + "(event_type != 'reversal' AND reversal_of_event_id IS NULL)", + name="ck_usage_meter_events_reversal", + ), + sa.CheckConstraint( + "(subject_type IS NULL AND subject_id IS NULL) OR " + "(subject_type IS NOT NULL AND subject_id IS NOT NULL)", + name="ck_usage_meter_events_subject_pair", + ), + sa.CheckConstraint( + "actor_type IN ('system', 'user', 'integration', 'admin')", + name="ck_usage_meter_events_actor_type", + ), + sa.CheckConstraint( + "length(trim(metric_key)) > 0 AND length(trim(unit)) > 0 " + "AND length(trim(period_key)) > 0 AND length(trim(source_system)) > 0 " + "AND length(trim(idempotency_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + name="ck_usage_meter_events_keys", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_usage_meter_events_tenant_subscription", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id", "entitlement_id"], + [ + "commercial_entitlements.tenant_id", + "commercial_entitlements.subscription_id", + "commercial_entitlements.id", + ], + name="fk_usage_meter_events_tenant_entitlement", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id", "entitlement_id", "reversal_of_event_id"], + [ + "usage_meter_events.tenant_id", + "usage_meter_events.subscription_id", + "usage_meter_events.entitlement_id", + "usage_meter_events.id", + ], + name="fk_usage_meter_events_tenant_reversal", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_usage_meter_events_tenant_id"), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "id", + name="uq_usage_meter_events_tenant_subscription_id", + ), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "entitlement_id", + "id", + name="uq_usage_meter_events_entitlement_id", + ), + sa.UniqueConstraint( + "tenant_id", + "source_system", + "idempotency_key", + name="uq_usage_meter_events_source_request", + ), + ) + op.create_index( + "ix_usage_meter_events_quota_window", + "usage_meter_events", + ["tenant_id", "subscription_id", "metric_key", "period_key", "occurred_at"], + ) + op.create_index( + "ix_usage_meter_events_correlation", + "usage_meter_events", + ["tenant_id", "correlation_id", "occurred_at"], + ) + op.create_table( + "commercial_cost_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("subscription_id", sa.String(length=36), nullable=True), + sa.Column("usage_event_id", sa.String(length=36), nullable=True), + sa.Column("event_type", sa.String(length=16), nullable=False), + sa.Column("cost_category", sa.String(length=32), nullable=False), + sa.Column("quantity", sa.Numeric(20, 6), nullable=False), + sa.Column("unit", sa.String(length=40), nullable=False), + sa.Column("unit_cost", sa.Numeric(20, 8), nullable=False), + sa.Column("cost_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("original_currency", sa.String(length=3), nullable=False), + sa.Column("reporting_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("reporting_currency", sa.String(length=3), nullable=False), + sa.Column("fx_rate", sa.Numeric(20, 8), nullable=False), + sa.Column("provider", sa.String(length=120), nullable=True), + sa.Column("sku", sa.String(length=120), nullable=True), + sa.Column("model_name", sa.String(length=120), nullable=True), + sa.Column("allocation_key", sa.String(length=160), nullable=False), + sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("source_system", sa.String(length=80), nullable=False), + sa.Column("idempotency_key", sa.String(length=160), nullable=False), + sa.Column("request_fingerprint", sa.String(length=80), nullable=False), + sa.Column("reversal_of_cost_event_id", sa.String(length=36), nullable=True), + sa.Column("correlation_id", sa.String(length=120), nullable=True), + sa.Column("trace_id", sa.String(length=120), nullable=True), + sa.Column( + "metadata_json", sa.JSON(), nullable=False, server_default=_json_object_default() + ), + sa.Column( + "recorded_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now() + ), + sa.CheckConstraint( + "event_type IN ('incurred', 'credit', 'adjustment', 'reversal')", + name="ck_commercial_cost_events_type", + ), + sa.CheckConstraint( + "cost_category IN ('ai_inference', 'ocr', 'storage', 'connector', " + "'support', 'implementation', 'infrastructure', 'payment', 'other')", + name="ck_commercial_cost_events_category", + ), + sa.CheckConstraint( + "quantity > 0 AND unit_cost >= 0 AND fx_rate > 0", + name="ck_commercial_cost_events_values", + ), + sa.CheckConstraint( + "(event_type = 'incurred' AND cost_amount >= 0 AND reporting_amount >= 0) OR " + "(event_type = 'credit' AND cost_amount <= 0 AND reporting_amount <= 0) OR " + "(event_type IN ('adjustment', 'reversal') AND cost_amount <> 0 " + "AND reporting_amount <> 0)", + name="ck_commercial_cost_events_amount_direction", + ), + sa.CheckConstraint( + "(event_type = 'reversal' AND reversal_of_cost_event_id IS NOT NULL) OR " + "(event_type != 'reversal' AND reversal_of_cost_event_id IS NULL)", + name="ck_commercial_cost_events_reversal", + ), + sa.CheckConstraint( + "usage_event_id IS NULL OR subscription_id IS NOT NULL", + name="ck_commercial_cost_events_usage_pair", + ), + sa.CheckConstraint( + "length(trim(unit)) > 0 AND length(trim(source_system)) > 0 " + "AND length(trim(idempotency_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0 " + "AND length(trim(original_currency)) = 3 " + "AND length(trim(reporting_currency)) = 3", + name="ck_commercial_cost_events_keys", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_cost_events_tenant_subscription", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id", "usage_event_id"], + [ + "usage_meter_events.tenant_id", + "usage_meter_events.subscription_id", + "usage_meter_events.id", + ], + name="fk_commercial_cost_events_tenant_usage", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "reversal_of_cost_event_id"], + ["commercial_cost_events.tenant_id", "commercial_cost_events.id"], + name="fk_commercial_cost_events_tenant_reversal", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", "id", name="uq_commercial_cost_events_tenant_id" + ), + sa.UniqueConstraint( + "tenant_id", + "source_system", + "idempotency_key", + name="uq_commercial_cost_events_source_request", + ), + ) + op.create_index( + "ix_commercial_cost_events_tenant_period", + "commercial_cost_events", + ["tenant_id", "occurred_at", "cost_category"], + ) + op.create_index( + "ix_commercial_cost_events_subscription_period", + "commercial_cost_events", + ["tenant_id", "subscription_id", "occurred_at"], + ) + op.create_index( + "ix_commercial_cost_events_allocation", + "commercial_cost_events", + ["tenant_id", "allocation_key", "occurred_at"], + ) + op.execute( + """ + CREATE OR REPLACE FUNCTION prevent_commercial_events_mutation() + RETURNS trigger AS $$ + BEGIN + RAISE EXCEPTION '% is append-only', TG_TABLE_NAME; + END; + $$ LANGUAGE plpgsql + """ + ) + for table_name in ("usage_meter_events", "commercial_cost_events"): + op.execute( + f""" + CREATE TRIGGER trg_{table_name}_append_only + BEFORE UPDATE OR DELETE ON {table_name} + FOR EACH ROW EXECUTE FUNCTION prevent_commercial_events_mutation() + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_commercial_domain_for_downgrade() + for table_name in ("commercial_cost_events", "usage_meter_events"): + op.execute(f"DROP TRIGGER IF EXISTS trg_{table_name}_append_only ON {table_name}") + op.execute("DROP FUNCTION IF EXISTS prevent_commercial_events_mutation()") + op.drop_table("commercial_cost_events") + op.drop_table("usage_meter_events") + op.drop_table("commercial_entitlements") + op.drop_table("tenant_subscriptions") + op.drop_table("tenant_commercial_plans") diff --git a/server/alembic/versions/20260716_0017_financial_connector_reconciliation.py b/server/alembic/versions/20260716_0017_financial_connector_reconciliation.py new file mode 100644 index 0000000..f0509e1 --- /dev/null +++ b/server/alembic/versions/20260716_0017_financial_connector_reconciliation.py @@ -0,0 +1,375 @@ +"""add tenant-safe financial connector and reconciliation ledger + +Revision ID: 20260716_0017 +Revises: 20260716_0016 +Create Date: 2026-07-16 20:00:00 +""" + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0017" +down_revision: str | None = "20260716_0016" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0017 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_connector_domain_for_downgrade() -> None: + bind = op.get_bind() + counts = { + table_name: int(bind.scalar(sa.text(f"SELECT COUNT(*) FROM {table_name}")) or 0) + for table_name in ( + "financial_connector_configs", + "financial_connector_events", + "payment_reconciliation_cases", + "payment_reconciliation_events", + ) + } + if any(counts.values()): + summary = ", ".join(f"{name}={count}" for name, count in counts.items()) + raise RuntimeError( + "cannot downgrade financial connector: configurations or immutable facts exist " + f"({summary})" + ) + + +def upgrade() -> None: + _require_postgresql() + op.create_table( + "financial_connector_configs", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("provider", sa.String(length=80), nullable=False), + sa.Column("environment", sa.String(length=16), nullable=False), + sa.Column("key_version", sa.String(length=40), nullable=False), + sa.Column("secret_ref", sa.String(length=180), nullable=False), + sa.Column("allowed_event_types_json", sa.JSON(), nullable=False), + sa.Column("clock_skew_seconds", sa.Integer(), nullable=False), + sa.Column("status", sa.String(length=16), nullable=False), + sa.Column("last_success_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("last_error_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("last_error_code", sa.String(length=80), nullable=True), + sa.Column("created_by", sa.String(length=120), nullable=False), + sa.Column( + "created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False + ), + sa.Column( + "updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False + ), + sa.CheckConstraint( + "environment IN ('test', 'mock', 'staging', 'production')", + name="ck_financial_connector_configs_environment", + ), + sa.CheckConstraint( + "status IN ('active', 'disabled', 'rotating')", + name="ck_financial_connector_configs_status", + ), + sa.CheckConstraint( + "clock_skew_seconds BETWEEN 30 AND 900", + name="ck_financial_connector_configs_clock_skew", + ), + sa.CheckConstraint( + "length(trim(provider)) > 0 AND length(trim(key_version)) > 0 " + "AND length(trim(secret_ref)) > 0", + name="ck_financial_connector_configs_keys", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_financial_connector_configs_tenant_id"), + sa.UniqueConstraint( + "tenant_id", + "provider", + "key_version", + name="uq_financial_connector_configs_tenant_provider_key", + ), + ) + op.create_index( + "ix_financial_connector_configs_tenant_status", + "financial_connector_configs", + ["tenant_id", "status", "provider"], + ) + + op.create_table( + "financial_connector_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("config_id", sa.String(length=36), nullable=False), + sa.Column("provider", sa.String(length=80), nullable=False), + sa.Column("environment", sa.String(length=16), nullable=False), + sa.Column("direction", sa.String(length=12), nullable=False), + sa.Column("external_event_id", sa.String(length=160), nullable=False), + sa.Column("event_type", sa.String(length=40), nullable=False), + sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False), + sa.Column( + "received_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False + ), + sa.Column("key_version", sa.String(length=40), nullable=False), + sa.Column("verification_level", sa.String(length=32), nullable=False), + sa.Column("request_fingerprint", sa.String(length=80), nullable=False), + sa.Column("content_hash", sa.String(length=80), nullable=False), + sa.Column("processing_status", sa.String(length=20), nullable=False), + sa.Column("error_code", sa.String(length=80), nullable=True), + sa.Column("claim_id", sa.String(length=36), nullable=True), + sa.Column("expense_case_id", sa.String(length=36), nullable=True), + sa.Column("origin_event_id", sa.String(length=36), nullable=True), + sa.Column("correlation_id", sa.String(length=64), nullable=False), + sa.Column("external_reference_tail", sa.String(length=8), nullable=True), + sa.Column("normalized_payload_json", sa.JSON(), nullable=False), + sa.Column("response_json", sa.JSON(), nullable=False), + sa.CheckConstraint("direction = 'inbound'", name="ck_financial_connector_events_direction"), + sa.CheckConstraint( + "event_type IN ('payment_settled', 'payment_failed', 'erp_posted', " + "'erp_posting_failed', 'payment_refunded', 'payment_reversed')", + name="ck_financial_connector_events_type", + ), + sa.CheckConstraint( + "environment IN ('test', 'mock', 'staging', 'production')", + name="ck_financial_connector_events_environment", + ), + sa.CheckConstraint( + "verification_level IN ('simulated', 'staging_verified', 'production_verified')", + name="ck_financial_connector_events_verification", + ), + sa.CheckConstraint( + "processing_status IN ('processed', 'exception', 'pending')", + name="ck_financial_connector_events_processing_status", + ), + sa.CheckConstraint( + "length(trim(external_event_id)) > 0 " + "AND length(trim(request_fingerprint)) >= 16 " + "AND length(trim(content_hash)) >= 16", + name="ck_financial_connector_events_fingerprints", + ), + sa.CheckConstraint( + "(event_type IN ('payment_refunded', 'payment_reversed', " + "'erp_posted', 'erp_posting_failed') " + "AND (origin_event_id IS NOT NULL OR processing_status = 'exception')) " + "OR (event_type IN ('payment_settled', 'payment_failed') " + "AND origin_event_id IS NULL)", + name="ck_financial_connector_events_origin", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "config_id"], + ["financial_connector_configs.tenant_id", "financial_connector_configs.id"], + name="fk_financial_connector_events_tenant_config", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + name="fk_financial_connector_events_tenant_expense_case", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "origin_event_id"], + ["financial_connector_events.tenant_id", "financial_connector_events.id"], + name="fk_financial_connector_events_tenant_origin", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_financial_connector_events_tenant_id"), + sa.UniqueConstraint( + "tenant_id", + "provider", + "external_event_id", + name="uq_financial_connector_events_external_id", + ), + ) + op.create_index( + "ix_financial_connector_events_tenant_received", + "financial_connector_events", + ["tenant_id", "received_at"], + ) + op.create_index( + "ix_financial_connector_events_tenant_claim", + "financial_connector_events", + ["tenant_id", "claim_id", "occurred_at"], + ) + + op.create_table( + "payment_reconciliation_cases", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("provider", sa.String(length=80), nullable=False), + sa.Column("claim_id", sa.String(length=36), nullable=False), + sa.Column("expense_case_id", sa.String(length=36), nullable=True), + sa.Column("expected_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("actual_amount", sa.Numeric(20, 4), nullable=False), + sa.Column("amount_difference", sa.Numeric(20, 4), nullable=False), + sa.Column("expected_currency", sa.String(length=3), nullable=False), + sa.Column("actual_currency", sa.String(length=3), nullable=False), + sa.Column("expected_reference", sa.String(length=160), nullable=False), + sa.Column("external_reference_tail", sa.String(length=8), nullable=True), + sa.Column("status", sa.String(length=20), nullable=False), + sa.Column("exception_code", sa.String(length=80), nullable=True), + sa.Column("erp_status", sa.String(length=20), nullable=False), + sa.Column("erp_document_tail", sa.String(length=8), nullable=True), + sa.Column("erp_document_hash", sa.String(length=80), nullable=True), + sa.Column("assigned_to", sa.String(length=120), nullable=True), + sa.Column("last_connector_event_id", sa.String(length=36), nullable=False), + sa.Column("version", sa.Integer(), nullable=False), + sa.Column( + "created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False + ), + sa.Column( + "updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False + ), + sa.CheckConstraint( + "status IN ('pending', 'matched', 'exception', 'confirmed', " + "'rejected', 'reopened', 'closed')", + name="ck_payment_reconciliation_cases_status", + ), + sa.CheckConstraint( + "erp_status IN ('pending_posting', 'posted', 'posting_failed')", + name="ck_payment_reconciliation_cases_erp_status", + ), + sa.CheckConstraint( + "expected_amount >= 0 AND actual_amount >= 0", + name="ck_payment_reconciliation_cases_amounts", + ), + sa.CheckConstraint( + "length(trim(expected_currency)) = 3 AND length(trim(actual_currency)) = 3", + name="ck_payment_reconciliation_cases_currencies", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + name="fk_payment_reconciliation_cases_tenant_expense_case", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "last_connector_event_id"], + ["financial_connector_events.tenant_id", "financial_connector_events.id"], + name="fk_payment_reconciliation_cases_tenant_last_event", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_payment_reconciliation_cases_tenant_id"), + sa.UniqueConstraint( + "tenant_id", + "provider", + "claim_id", + name="uq_payment_reconciliation_cases_tenant_provider_claim", + ), + ) + op.create_index( + "ix_payment_reconciliation_cases_tenant_status", + "payment_reconciliation_cases", + ["tenant_id", "status", "updated_at"], + ) + + op.create_table( + "payment_reconciliation_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("reconciliation_case_id", sa.String(length=36), nullable=False), + sa.Column("connector_event_id", sa.String(length=36), nullable=False), + sa.Column("action", sa.String(length=32), nullable=False), + sa.Column("actor_type", sa.String(length=20), nullable=False), + sa.Column("actor_id", sa.String(length=120), nullable=False), + sa.Column("request_fingerprint", sa.String(length=80), nullable=False), + sa.Column("before_json", sa.JSON(), nullable=False), + sa.Column("after_json", sa.JSON(), nullable=False), + sa.Column("response_json", sa.JSON(), nullable=False), + sa.Column("reason", sa.Text(), nullable=True), + sa.Column("correlation_id", sa.String(length=64), nullable=False), + sa.Column( + "occurred_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False + ), + sa.CheckConstraint( + "action IN ('auto_matched', 'exception_created', 'erp_posted', " + "'erp_posting_failed', 'reopened', 'confirmed', 'rejected', 'closed')", + name="ck_payment_reconciliation_events_action", + ), + sa.CheckConstraint( + "length(trim(request_fingerprint)) >= 16", + name="ck_payment_reconciliation_events_fingerprint", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "connector_event_id"], + ["financial_connector_events.tenant_id", "financial_connector_events.id"], + name="fk_payment_reconciliation_events_tenant_connector_event", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "reconciliation_case_id"], + ["payment_reconciliation_cases.tenant_id", "payment_reconciliation_cases.id"], + name="fk_payment_reconciliation_events_tenant_case", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_payment_reconciliation_events_tenant_id"), + sa.UniqueConstraint( + "tenant_id", + "connector_event_id", + "action", + name="uq_payment_reconciliation_events_connector_action", + ), + ) + op.create_index( + "ix_payment_reconciliation_events_tenant_case_time", + "payment_reconciliation_events", + ["tenant_id", "reconciliation_case_id", "occurred_at"], + ) + + op.execute( + """ + CREATE FUNCTION reject_financial_connector_append_only_mutation() + RETURNS trigger AS $$ + BEGIN + RAISE EXCEPTION 'financial connector facts are append-only'; + END; + $$ LANGUAGE plpgsql; + """ + ) + for table_name in ("financial_connector_events", "payment_reconciliation_events"): + op.execute( + f""" + CREATE TRIGGER trg_{table_name}_append_only + BEFORE UPDATE OR DELETE ON {table_name} + FOR EACH ROW EXECUTE FUNCTION reject_financial_connector_append_only_mutation(); + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_connector_domain_for_downgrade() + for table_name in ("payment_reconciliation_events", "financial_connector_events"): + op.execute(f"DROP TRIGGER IF EXISTS trg_{table_name}_append_only ON {table_name}") + op.execute("DROP FUNCTION IF EXISTS reject_financial_connector_append_only_mutation()") + op.drop_index( + "ix_payment_reconciliation_events_tenant_case_time", + table_name="payment_reconciliation_events", + ) + op.drop_table("payment_reconciliation_events") + op.drop_index( + "ix_payment_reconciliation_cases_tenant_status", + table_name="payment_reconciliation_cases", + ) + op.drop_table("payment_reconciliation_cases") + op.drop_index( + "ix_financial_connector_events_tenant_claim", + table_name="financial_connector_events", + ) + op.drop_index( + "ix_financial_connector_events_tenant_received", + table_name="financial_connector_events", + ) + op.drop_table("financial_connector_events") + op.drop_index( + "ix_financial_connector_configs_tenant_status", + table_name="financial_connector_configs", + ) + op.drop_table("financial_connector_configs") diff --git a/server/alembic/versions/20260716_0018_agent_asset_release_telemetry.py b/server/alembic/versions/20260716_0018_agent_asset_release_telemetry.py new file mode 100644 index 0000000..6c85284 --- /dev/null +++ b/server/alembic/versions/20260716_0018_agent_asset_release_telemetry.py @@ -0,0 +1,237 @@ +"""add append-only real release telemetry and review labels + +Revision ID: 20260716_0018 +Revises: 20260716_0017 +Create Date: 2026-07-16 22:00:00 +""" + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0018" +down_revision: str | None = "20260716_0017" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +_TELEMETRY_TABLES = ( + "agent_asset_release_observations", + "agent_asset_release_labels", +) + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0018 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_telemetry_domain_for_downgrade() -> None: + bind = op.get_bind() + counts = { + table_name: int(bind.scalar(sa.text(f"SELECT COUNT(*) FROM {table_name}")) or 0) + for table_name in _TELEMETRY_TABLES + } + if any(counts.values()): + summary = ", ".join(f"{name}={count}" for name, count in counts.items()) + raise RuntimeError( + "cannot downgrade release telemetry: immutable observations or labels exist " + f"({summary})" + ) + + +def upgrade() -> None: + _require_postgresql() + op.create_table( + "agent_asset_release_observations", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("asset_id", sa.String(length=36), nullable=False), + sa.Column("release_id", sa.String(length=64), nullable=False), + sa.Column("stage", sa.String(length=16), nullable=False), + sa.Column("version", sa.String(length=30), nullable=False), + sa.Column("rule_code", sa.String(length=100), nullable=False), + sa.Column("business_stage", sa.String(length=40), nullable=False), + sa.Column( + "source_kind", + sa.String(length=32), + server_default="expense_claim_risk", + nullable=False, + ), + sa.Column("source_fingerprint", sa.String(length=64), nullable=False), + sa.Column("candidate_hit", sa.Boolean(), nullable=False), + sa.Column("baseline_hit", sa.Boolean(), nullable=True), + sa.Column("runtime_status", sa.String(length=16), nullable=False), + sa.Column( + "failure_code", + sa.String(length=40), + server_default="none", + nullable=False, + ), + sa.Column("idempotency_key", sa.String(length=80), nullable=False), + sa.Column("payload_fingerprint", sa.String(length=64), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.CheckConstraint( + "stage IN ('shadow', 'canary', 'active')", + name="ck_agent_asset_release_observations_stage", + ), + sa.CheckConstraint( + "runtime_status IN ('completed', 'failed')", + name="ck_agent_asset_release_observations_runtime_status", + ), + sa.CheckConstraint( + "source_kind IN ('expense_claim_risk')", + name="ck_agent_asset_release_observations_source_kind", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "id", + name="uq_agent_asset_release_observations_tenant_id", + ), + sa.UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_agent_asset_release_observations_tenant_idempotency", + ), + sa.UniqueConstraint( + "tenant_id", + "id", + "asset_id", + "release_id", + "stage", + "version", + name="uq_agent_asset_release_observations_release_identity", + ), + ) + op.create_index( + "ix_agent_asset_release_observations_release", + "agent_asset_release_observations", + ["tenant_id", "asset_id", "release_id", "stage", "version", "created_at"], + ) + op.create_index( + "ix_agent_asset_release_observations_source", + "agent_asset_release_observations", + ["tenant_id", "source_fingerprint"], + ) + + op.create_table( + "agent_asset_release_labels", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("observation_id", sa.String(length=36), nullable=False), + sa.Column("asset_id", sa.String(length=36), nullable=False), + sa.Column("release_id", sa.String(length=64), nullable=False), + sa.Column("stage", sa.String(length=16), nullable=False), + sa.Column("version", sa.String(length=30), nullable=False), + sa.Column("label", sa.String(length=24), nullable=False), + sa.Column("verification_source", sa.String(length=32), nullable=False), + sa.Column("source_event_fingerprint", sa.String(length=64), nullable=False), + sa.Column("actor_fingerprint", sa.String(length=64), nullable=False), + sa.Column("idempotency_key", sa.String(length=80), nullable=False), + sa.Column("payload_fingerprint", sa.String(length=64), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.CheckConstraint( + "label IN ('confirmed', 'false_positive')", + name="ck_agent_asset_release_labels_label", + ), + sa.CheckConstraint( + "verification_source IN ('typed_risk_disposition', 'release_review')", + name="ck_agent_asset_release_labels_source", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "observation_id", "asset_id", "release_id", "stage", "version"], + [ + "agent_asset_release_observations.tenant_id", + "agent_asset_release_observations.id", + "agent_asset_release_observations.asset_id", + "agent_asset_release_observations.release_id", + "agent_asset_release_observations.stage", + "agent_asset_release_observations.version", + ], + name="fk_agent_asset_release_labels_release_observation", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "id", + name="uq_agent_asset_release_labels_tenant_id", + ), + sa.UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_agent_asset_release_labels_tenant_idempotency", + ), + ) + op.create_index( + "ix_agent_asset_release_labels_observation_time", + "agent_asset_release_labels", + ["tenant_id", "observation_id", "created_at"], + ) + op.create_index( + "ix_agent_asset_release_labels_release", + "agent_asset_release_labels", + ["tenant_id", "asset_id", "release_id", "stage", "version"], + ) + + op.execute( + """ + CREATE FUNCTION reject_agent_asset_release_telemetry_mutation() + RETURNS trigger AS $$ + BEGIN + RAISE EXCEPTION 'agent asset release telemetry is append-only'; + END; + $$ LANGUAGE plpgsql; + """ + ) + for table_name in _TELEMETRY_TABLES: + op.execute( + f""" + CREATE TRIGGER trg_{table_name}_append_only + BEFORE UPDATE OR DELETE ON {table_name} + FOR EACH ROW EXECUTE FUNCTION reject_agent_asset_release_telemetry_mutation(); + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_telemetry_domain_for_downgrade() + for table_name in reversed(_TELEMETRY_TABLES): + op.execute(f"DROP TRIGGER IF EXISTS trg_{table_name}_append_only ON {table_name}") + op.execute("DROP FUNCTION IF EXISTS reject_agent_asset_release_telemetry_mutation()") + op.drop_index( + "ix_agent_asset_release_labels_release", + table_name="agent_asset_release_labels", + ) + op.drop_index( + "ix_agent_asset_release_labels_observation_time", + table_name="agent_asset_release_labels", + ) + op.drop_table("agent_asset_release_labels") + op.drop_index( + "ix_agent_asset_release_observations_source", + table_name="agent_asset_release_observations", + ) + op.drop_index( + "ix_agent_asset_release_observations_release", + table_name="agent_asset_release_observations", + ) + op.drop_table("agent_asset_release_observations") diff --git a/server/alembic/versions/20260716_0019_commercial_runtime_reservations.py b/server/alembic/versions/20260716_0019_commercial_runtime_reservations.py new file mode 100644 index 0000000..2ff1f93 --- /dev/null +++ b/server/alembic/versions/20260716_0019_commercial_runtime_reservations.py @@ -0,0 +1,178 @@ +"""add commercial runtime quota reservations + +Revision ID: 20260716_0019 +Revises: 20260716_0018 +Create Date: 2026-07-16 22:10:00 +""" + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0019" +down_revision: str | None = "20260716_0018" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0019 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_reservations_for_downgrade() -> None: + count = int( + op.get_bind().scalar( + sa.text("SELECT COUNT(*) FROM commercial_runtime_reservations") + ) + or 0 + ) + if count: + raise RuntimeError( + "cannot downgrade commercial runtime reservations: " + f"operational quota holds exist ({count})" + ) + + +def upgrade() -> None: + _require_postgresql() + op.create_table( + "commercial_runtime_reservations", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("subscription_id", sa.String(length=36), nullable=False), + sa.Column("entitlement_id", sa.String(length=36), nullable=False), + sa.Column("run_id", sa.String(length=50), nullable=False), + sa.Column("tool_call_id", sa.String(length=36), nullable=False), + sa.Column("tool_type", sa.String(length=30), nullable=False), + sa.Column("tool_name", sa.String(length=100), nullable=False), + sa.Column("quantity_basis", sa.String(length=20), nullable=False), + sa.Column("reserved_quantity", sa.Numeric(20, 6), nullable=False), + sa.Column("actual_quantity", sa.Numeric(20, 6), nullable=True), + sa.Column("period_key", sa.String(length=32), nullable=False), + sa.Column("status", sa.String(length=32), nullable=False), + sa.Column("request_fingerprint", sa.String(length=64), nullable=False), + sa.Column("meter_config_json", sa.JSON(), nullable=False), + sa.Column("resolution_code", sa.String(length=64), nullable=True), + sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("settled_at", sa.DateTime(timezone=True), nullable=True), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.CheckConstraint( + "status IN ('reserved', 'committed', 'released', 'expired', " + "'reconciliation_required', 'committed_reconciliation_required')", + name="ck_commercial_runtime_reservations_status", + ), + sa.CheckConstraint( + "quantity_basis IN ('call', 'input_tokens', 'output_tokens', " + "'total_tokens', 'duration_ms')", + name="ck_commercial_runtime_reservations_basis", + ), + sa.CheckConstraint( + "reserved_quantity > 0 AND " + "(actual_quantity IS NULL OR actual_quantity > 0)", + name="ck_commercial_runtime_reservations_quantity", + ), + sa.CheckConstraint( + "expires_at > created_at", + name="ck_commercial_runtime_reservations_expiry", + ), + sa.CheckConstraint( + "(status = 'reserved' AND actual_quantity IS NULL AND settled_at IS NULL " + "AND resolution_code IS NULL) OR " + "(status = 'committed' AND actual_quantity IS NOT NULL " + "AND actual_quantity <= reserved_quantity AND settled_at IS NOT NULL " + "AND resolution_code IS NULL) OR " + "(status IN ('released', 'expired') AND actual_quantity IS NULL " + "AND settled_at IS NOT NULL AND resolution_code IS NOT NULL) OR " + "(status = 'reconciliation_required' AND settled_at IS NULL " + "AND resolution_code IS NOT NULL) OR " + "(status = 'committed_reconciliation_required' " + "AND actual_quantity IS NOT NULL " + "AND actual_quantity <= reserved_quantity " + "AND settled_at IS NOT NULL AND resolution_code IS NOT NULL)", + name="ck_commercial_runtime_reservations_state", + ), + sa.CheckConstraint( + "length(trim(run_id)) > 0 AND length(trim(tool_call_id)) > 0 " + "AND length(trim(tool_type)) > 0 AND length(trim(tool_name)) > 0 " + "AND length(trim(period_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + name="ck_commercial_runtime_reservations_keys", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_runtime_reservations_tenant_subscription", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id", "entitlement_id"], + [ + "commercial_entitlements.tenant_id", + "commercial_entitlements.subscription_id", + "commercial_entitlements.id", + ], + name="fk_commercial_runtime_reservations_tenant_entitlement", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "id", + name="uq_commercial_runtime_reservations_tenant_id", + ), + sa.UniqueConstraint( + "tool_call_id", + name="uq_commercial_runtime_reservations_tool_call", + ), + ) + op.create_index( + "ix_commercial_runtime_reservations_quota", + "commercial_runtime_reservations", + ["tenant_id", "subscription_id", "entitlement_id", "period_key", "status"], + ) + op.create_index( + "ix_commercial_runtime_reservations_expiry", + "commercial_runtime_reservations", + ["status", "expires_at"], + ) + op.create_index( + "ix_commercial_runtime_reservations_run", + "commercial_runtime_reservations", + ["tenant_id", "run_id", "created_at"], + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_reservations_for_downgrade() + op.drop_index( + "ix_commercial_runtime_reservations_run", + table_name="commercial_runtime_reservations", + ) + op.drop_index( + "ix_commercial_runtime_reservations_expiry", + table_name="commercial_runtime_reservations", + ) + op.drop_index( + "ix_commercial_runtime_reservations_quota", + table_name="commercial_runtime_reservations", + ) + op.drop_table("commercial_runtime_reservations") diff --git a/server/alembic/versions/20260716_0020_financial_connector_config_lifecycle.py b/server/alembic/versions/20260716_0020_financial_connector_config_lifecycle.py new file mode 100644 index 0000000..bfa0811 --- /dev/null +++ b/server/alembic/versions/20260716_0020_financial_connector_config_lifecycle.py @@ -0,0 +1,181 @@ +"""add versioned financial connector config lifecycle audit + +Revision ID: 20260716_0020 +Revises: 20260716_0019 +Create Date: 2026-07-16 22:40:00 +""" + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0020" +down_revision: str | None = "20260716_0019" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0020 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_lifecycle_for_downgrade() -> None: + bind = op.get_bind() + counts = { + table_name: int(bind.scalar(sa.text(f"SELECT COUNT(*) FROM {table_name}")) or 0) + for table_name in ( + "financial_connector_configs", + "financial_connector_config_events", + ) + } + if any(counts.values()): + summary = ", ".join(f"{name}={count}" for name, count in counts.items()) + raise RuntimeError( + "cannot downgrade financial connector lifecycle: versioned configuration " + f"state or immutable audit facts exist ({summary})" + ) + + +def upgrade() -> None: + _require_postgresql() + # 0017 曾冗余保存完整 claim_reference。这里以受控迁移完成一次性脱敏， + # 内容哈希和签名指纹仍足以证明原始回执，运行期触发器随后立即恢复。 + op.execute( + "ALTER TABLE financial_connector_events " + "DISABLE TRIGGER trg_financial_connector_events_append_only" + ) + op.execute( + "UPDATE financial_connector_events " + "SET normalized_payload_json = " + "(normalized_payload_json::jsonb - 'claim_reference')::json " + "WHERE normalized_payload_json::jsonb ? 'claim_reference'" + ) + op.execute( + "UPDATE financial_connector_events SET response_json = " + "jsonb_set(response_json::jsonb, '{projection_scope}', " + "to_jsonb((CASE WHEN verification_level = 'production_verified' " + "THEN 'canonical' " + "WHEN COALESCE(response_json::jsonb ->> 'reconciliation_case_id', '') <> '' " + "THEN 'legacy_nonproduction_effect_unknown' " + "ELSE 'simulation_only' END)::text), true)::json " + "WHERE NOT (response_json::jsonb ? 'projection_scope')" + ) + op.execute( + "ALTER TABLE financial_connector_events " + "ENABLE TRIGGER trg_financial_connector_events_append_only" + ) + op.add_column( + "financial_connector_configs", + sa.Column("version", sa.Integer(), server_default="1", nullable=False), + ) + op.create_check_constraint( + "ck_financial_connector_configs_version", + "financial_connector_configs", + "version >= 1", + ) + op.alter_column( + "financial_connector_configs", + "version", + existing_type=sa.Integer(), + server_default=None, + ) + + op.create_table( + "financial_connector_config_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("config_id", sa.String(length=36), nullable=False), + sa.Column("action", sa.String(length=40), nullable=False), + sa.Column("actor_id", sa.String(length=120), nullable=False), + sa.Column("request_id", sa.String(length=120), nullable=False), + sa.Column("reason", sa.Text(), nullable=False), + sa.Column("expected_version", sa.Integer(), nullable=True), + sa.Column("before_json", sa.JSON(), nullable=False), + sa.Column("after_json", sa.JSON(), nullable=False), + sa.Column( + "occurred_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.CheckConstraint( + "action IN ('created', 'activated', 'disabled', " + "'rotation_started', 'rotation_replacement_created')", + name="ck_financial_connector_config_events_action", + ), + sa.CheckConstraint( + "expected_version IS NULL OR expected_version >= 1", + name="ck_financial_connector_config_events_expected_version", + ), + sa.CheckConstraint( + "length(trim(actor_id)) > 0 AND length(trim(request_id)) > 0 " + "AND length(trim(reason)) > 0", + name="ck_financial_connector_config_events_required_text", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "config_id"], + ["financial_connector_configs.tenant_id", "financial_connector_configs.id"], + name="fk_financial_connector_config_events_tenant_config", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "id", + name="uq_financial_connector_config_events_tenant_id", + ), + sa.UniqueConstraint( + "tenant_id", + "request_id", + "action", + name="uq_financial_connector_config_events_tenant_request_action", + ), + ) + op.create_index( + "ix_financial_connector_config_events_tenant_config_time", + "financial_connector_config_events", + ["tenant_id", "config_id", "occurred_at"], + ) + op.create_index( + "ix_financial_connector_config_events_tenant_request", + "financial_connector_config_events", + ["tenant_id", "request_id"], + ) + op.execute( + """ + CREATE TRIGGER trg_financial_connector_config_events_append_only + BEFORE UPDATE OR DELETE ON financial_connector_config_events + FOR EACH ROW EXECUTE FUNCTION reject_financial_connector_append_only_mutation(); + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_lifecycle_for_downgrade() + op.execute( + "DROP TRIGGER IF EXISTS trg_financial_connector_config_events_append_only " + "ON financial_connector_config_events" + ) + op.drop_index( + "ix_financial_connector_config_events_tenant_request", + table_name="financial_connector_config_events", + ) + op.drop_index( + "ix_financial_connector_config_events_tenant_config_time", + table_name="financial_connector_config_events", + ) + op.drop_table("financial_connector_config_events") + op.drop_constraint( + "ck_financial_connector_configs_version", + "financial_connector_configs", + type_="check", + ) + op.drop_column("financial_connector_configs", "version") diff --git a/server/alembic/versions/20260716_0021_commercial_billing_periods.py b/server/alembic/versions/20260716_0021_commercial_billing_periods.py new file mode 100644 index 0000000..31e3eab --- /dev/null +++ b/server/alembic/versions/20260716_0021_commercial_billing_periods.py @@ -0,0 +1,724 @@ +"""add immutable commercial billing periods and administration audit + +Revision ID: 20260716_0021 +Revises: 20260716_0020 +Create Date: 2026-07-16 22:45:00 +""" + +from __future__ import annotations + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0021" +down_revision: str | None = "20260716_0020" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0021 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_billing_lifecycle_for_downgrade() -> None: + bind = op.get_bind() + counts = { + table_name: int(bind.scalar(sa.text(f"SELECT COUNT(*) FROM {table_name}")) or 0) + for table_name in ("commercial_admin_events", "commercial_billing_periods") + } + if any(counts.values()): + summary = ", ".join(f"{name}={count}" for name, count in counts.items()) + raise RuntimeError( + "cannot downgrade commercial billing lifecycle: immutable periods or audit " + f"facts exist ({summary})" + ) + + +def _json_object_default() -> sa.TextClause: + return sa.text("'{}'::json") + + +def upgrade() -> None: + _require_postgresql() + _create_billing_periods() + _create_admin_events() + _backfill_current_periods_and_audit() + _bind_usage_to_periods() + _bind_costs_to_periods() + _bind_reservations_to_periods() + _create_period_overlap_guard() + _create_append_only_triggers() + + +def _create_billing_periods() -> None: + op.create_table( + "commercial_billing_periods", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("subscription_id", sa.String(length=36), nullable=False), + sa.Column("plan_id", sa.String(length=36), nullable=False), + sa.Column("period_sequence", sa.Integer(), nullable=False), + sa.Column("period_key", sa.String(length=64), nullable=False), + sa.Column("status", sa.String(length=16), nullable=False, server_default="issued"), + sa.Column("period_start", sa.DateTime(timezone=True), nullable=False), + sa.Column("period_end", sa.DateTime(timezone=True), nullable=False), + sa.Column("subscription_status_snapshot", sa.String(length=20), nullable=False), + sa.Column("plan_code_snapshot", sa.String(length=80), nullable=False), + sa.Column("plan_version_snapshot", sa.Integer(), nullable=False), + sa.Column("pricing_model_snapshot", sa.String(length=24), nullable=False), + sa.Column("billing_interval", sa.String(length=20), nullable=False), + sa.Column("currency", sa.String(length=3), nullable=False), + sa.Column("base_fee_snapshot", sa.Numeric(20, 4), nullable=False), + sa.Column("seats_snapshot", sa.Integer(), nullable=False), + sa.Column("source", sa.String(length=32), nullable=False), + sa.Column("idempotency_key", sa.String(length=160), nullable=False), + sa.Column("created_by", sa.String(length=120), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "status = 'issued'", + name="ck_commercial_billing_periods_status", + ), + sa.CheckConstraint( + "subscription_status_snapshot IN ('trialing', 'active', 'past_due', " + "'suspended', 'canceled', 'expired')", + name="ck_commercial_billing_periods_subscription_status", + ), + sa.CheckConstraint( + "pricing_model_snapshot IN ('subscription', 'usage', 'hybrid', 'pilot', 'custom')", + name="ck_commercial_billing_periods_pricing_model", + ), + sa.CheckConstraint( + "billing_interval IN ('monthly', 'quarterly', 'annual', 'contract')", + name="ck_commercial_billing_periods_interval", + ), + sa.CheckConstraint( + "source IN ('subscription_created', 'auto_renew', 'migration_backfill')", + name="ck_commercial_billing_periods_source", + ), + sa.CheckConstraint( + "period_sequence >= 1 AND period_end > period_start " + "AND plan_version_snapshot >= 1 AND base_fee_snapshot >= 0 " + "AND seats_snapshot > 0", + name="ck_commercial_billing_periods_values", + ), + sa.CheckConstraint( + "length(trim(period_key)) > 0 AND length(trim(plan_code_snapshot)) > 0 " + "AND length(trim(currency)) = 3 AND length(trim(idempotency_key)) > 0 " + "AND length(trim(created_by)) > 0", + name="ck_commercial_billing_periods_keys", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_billing_periods_tenant_subscription", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "plan_id"], + ["tenant_commercial_plans.tenant_id", "tenant_commercial_plans.id"], + name="fk_commercial_billing_periods_tenant_plan", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_commercial_billing_periods_tenant_id"), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "id", + name="uq_commercial_billing_periods_tenant_subscription_id", + ), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "period_sequence", + name="uq_commercial_billing_periods_subscription_sequence", + ), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "period_key", + name="uq_commercial_billing_periods_subscription_key", + ), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "period_start", + name="uq_commercial_billing_periods_subscription_start", + ), + sa.UniqueConstraint( + "tenant_id", + "subscription_id", + "idempotency_key", + name="uq_commercial_billing_periods_subscription_request", + ), + ) + op.create_index( + "ix_commercial_billing_periods_tenant_window", + "commercial_billing_periods", + ["tenant_id", "period_start", "period_end"], + ) + op.create_index( + "ix_commercial_billing_periods_subscription_window", + "commercial_billing_periods", + ["tenant_id", "subscription_id", "period_start", "period_end"], + ) + + +def _create_admin_events() -> None: + op.create_table( + "commercial_admin_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("actor_type", sa.String(length=16), nullable=False), + sa.Column("actor_id", sa.String(length=120), nullable=False), + sa.Column("request_id", sa.String(length=120), nullable=False), + sa.Column("reason", sa.Text(), nullable=False), + sa.Column("action", sa.String(length=48), nullable=False), + sa.Column("resource_type", sa.String(length=24), nullable=False), + sa.Column("resource_id", sa.String(length=36), nullable=False), + sa.Column("resource_version", sa.Integer(), nullable=False), + sa.Column("before_json", sa.JSON(), nullable=False, server_default=_json_object_default()), + sa.Column("after_json", sa.JSON(), nullable=False, server_default=_json_object_default()), + sa.Column( + "occurred_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "actor_type IN ('user', 'system', 'migration')", + name="ck_commercial_admin_events_actor_type", + ), + sa.CheckConstraint( + "action IN ('plan_created', 'plan_activated', 'plan_retired', " + "'subscription_created', 'subscription_activated', " + "'subscription_transitioned', 'entitlement_created', " + "'entitlement_updated', 'entitlement_activated', " + "'billing_period_created', 'subscription_rolled_over', " + "'legacy_state_imported')", + name="ck_commercial_admin_events_action", + ), + sa.CheckConstraint( + "resource_type IN ('plan', 'subscription', 'entitlement', 'billing_period')", + name="ck_commercial_admin_events_resource_type", + ), + sa.CheckConstraint( + "resource_version >= 1", + name="ck_commercial_admin_events_resource_version", + ), + sa.CheckConstraint( + "length(trim(actor_id)) > 0 AND length(trim(request_id)) > 0 " + "AND length(trim(reason)) > 0 AND length(trim(resource_id)) > 0", + name="ck_commercial_admin_events_required_text", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("tenant_id", "id", name="uq_commercial_admin_events_tenant_id"), + sa.UniqueConstraint( + "tenant_id", + "request_id", + "action", + "resource_type", + "resource_id", + name="uq_commercial_admin_events_request_resource", + ), + ) + op.create_index( + "ix_commercial_admin_events_tenant_time", + "commercial_admin_events", + ["tenant_id", "occurred_at", "id"], + ) + op.create_index( + "ix_commercial_admin_events_tenant_resource", + "commercial_admin_events", + ["tenant_id", "resource_type", "resource_id", "occurred_at"], + ) + op.create_index( + "ix_commercial_admin_events_tenant_request", + "commercial_admin_events", + ["tenant_id", "request_id"], + ) + + +def _backfill_current_periods_and_audit() -> None: + op.execute( + """ + INSERT INTO commercial_billing_periods ( + id, tenant_id, subscription_id, plan_id, period_sequence, period_key, + status, period_start, period_end, subscription_status_snapshot, + plan_code_snapshot, plan_version_snapshot, pricing_model_snapshot, + billing_interval, currency, base_fee_snapshot, seats_snapshot, + source, idempotency_key, created_by + ) + SELECT + substr(digest, 1, 8) || '-' || substr(digest, 9, 4) || '-' || + substr(digest, 13, 4) || '-' || substr(digest, 17, 4) || '-' || + substr(digest, 21, 12), + subscription.tenant_id, + subscription.id, + subscription.plan_id, + 1, + 'bp-' || substr(digest, 1, 24), + 'issued', + subscription.current_period_start, + subscription.current_period_end, + subscription.status, + plan.plan_code, + plan.version, + plan.pricing_model, + subscription.billing_interval, + subscription.currency, + subscription.base_fee_snapshot, + subscription.seats, + 'migration_backfill', + 'migration-0021:' || subscription.id, + 'migration:20260716_0021' + FROM tenant_subscriptions AS subscription + JOIN tenant_commercial_plans AS plan + ON plan.tenant_id = subscription.tenant_id + AND plan.id = subscription.plan_id + CROSS JOIN LATERAL ( + SELECT md5( + subscription.tenant_id || ':' || subscription.id || ':' || + subscription.current_period_start::text || ':' || + subscription.current_period_end::text + ) AS digest + ) AS identity + """ + ) + for resource_type, table_name, version_column in ( + ("plan", "tenant_commercial_plans", "version"), + ("subscription", "tenant_subscriptions", "version"), + ("entitlement", "commercial_entitlements", "version"), + ("billing_period", "commercial_billing_periods", "period_sequence"), + ): + op.execute( + f""" + INSERT INTO commercial_admin_events ( + id, tenant_id, actor_type, actor_id, request_id, reason, action, + resource_type, resource_id, resource_version, before_json, after_json + ) + SELECT + substr(digest, 1, 8) || '-' || substr(digest, 9, 4) || '-' || + substr(digest, 13, 4) || '-' || substr(digest, 17, 4) || '-' || + substr(digest, 21, 12), + resource.tenant_id, + 'migration', + 'migration:20260716_0021', + 'migration:0021:{resource_type}:' || resource.id, + '0021 建立商业管理审计基线，不推断迁移前操作人或原始请求。', + 'legacy_state_imported', + '{resource_type}', + resource.id, + resource.{version_column}, + '{{}}'::json, + json_build_object('id', resource.id, 'version', resource.{version_column}) + FROM {table_name} AS resource + CROSS JOIN LATERAL ( + SELECT md5( + resource.tenant_id || ':audit:{resource_type}:' || resource.id + ) AS digest + ) AS identity + """ + ) + + +def _bind_usage_to_periods() -> None: + op.add_column( + "usage_meter_events", + sa.Column("billing_period_id", sa.String(length=36), nullable=True), + ) + op.add_column( + "usage_meter_events", + sa.Column("quota_period_key", sa.String(length=64), nullable=True), + ) + op.alter_column( + "usage_meter_events", + "period_key", + existing_type=sa.String(length=32), + type_=sa.String(length=64), + existing_nullable=False, + ) + op.execute( + """ + UPDATE usage_meter_events AS usage + SET billing_period_id = period.id, + quota_period_key = usage.period_key, + period_key = period.period_key + FROM commercial_billing_periods AS period + WHERE period.tenant_id = usage.tenant_id + AND period.subscription_id = usage.subscription_id + AND usage.occurred_at >= period.period_start + AND usage.occurred_at < period.period_end + """ + ) + _fail_if_unbound( + "usage_meter_events", + "billing_period_id IS NULL OR quota_period_key IS NULL", + "existing usage facts do not map to one immutable current billing period", + ) + op.alter_column( + "usage_meter_events", "billing_period_id", existing_type=sa.String(36), nullable=False + ) + op.alter_column( + "usage_meter_events", "quota_period_key", existing_type=sa.String(64), nullable=False + ) + op.drop_constraint("ck_usage_meter_events_keys", "usage_meter_events", type_="check") + op.create_check_constraint( + "ck_usage_meter_events_keys", + "usage_meter_events", + "length(trim(metric_key)) > 0 AND length(trim(unit)) > 0 " + "AND length(trim(period_key)) > 0 AND length(trim(source_system)) > 0 " + "AND length(trim(quota_period_key)) > 0 " + "AND length(trim(idempotency_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + ) + op.create_foreign_key( + "fk_usage_meter_events_tenant_billing_period", + "usage_meter_events", + "commercial_billing_periods", + ["tenant_id", "subscription_id", "billing_period_id"], + ["tenant_id", "subscription_id", "id"], + ondelete="RESTRICT", + ) + op.drop_index("ix_usage_meter_events_quota_window", table_name="usage_meter_events") + op.create_index( + "ix_usage_meter_events_quota_window", + "usage_meter_events", + ["tenant_id", "subscription_id", "metric_key", "quota_period_key", "occurred_at"], + ) + op.create_index( + "ix_usage_meter_events_billing_period", + "usage_meter_events", + ["tenant_id", "billing_period_id", "occurred_at"], + ) + + +def _bind_costs_to_periods() -> None: + op.add_column( + "commercial_cost_events", + sa.Column("billing_period_id", sa.String(length=36), nullable=True), + ) + op.execute( + """ + UPDATE commercial_cost_events AS cost + SET billing_period_id = usage.billing_period_id + FROM usage_meter_events AS usage + WHERE usage.tenant_id = cost.tenant_id + AND usage.subscription_id = cost.subscription_id + AND usage.id = cost.usage_event_id + """ + ) + op.execute( + """ + UPDATE commercial_cost_events AS cost + SET billing_period_id = period.id + FROM commercial_billing_periods AS period + WHERE cost.subscription_id IS NOT NULL + AND cost.billing_period_id IS NULL + AND period.tenant_id = cost.tenant_id + AND period.subscription_id = cost.subscription_id + AND cost.occurred_at >= period.period_start + AND cost.occurred_at < period.period_end + """ + ) + _fail_if_unbound( + "commercial_cost_events", + "subscription_id IS NOT NULL AND billing_period_id IS NULL", + "existing subscription cost facts do not map to one immutable billing period", + ) + op.create_check_constraint( + "ck_commercial_cost_events_billing_period_pair", + "commercial_cost_events", + "(subscription_id IS NULL AND billing_period_id IS NULL) OR " + "(subscription_id IS NOT NULL AND billing_period_id IS NOT NULL)", + ) + op.create_foreign_key( + "fk_commercial_cost_events_tenant_billing_period", + "commercial_cost_events", + "commercial_billing_periods", + ["tenant_id", "subscription_id", "billing_period_id"], + ["tenant_id", "subscription_id", "id"], + ondelete="RESTRICT", + ) + op.drop_index( + "ix_commercial_cost_events_subscription_period", + table_name="commercial_cost_events", + ) + op.create_index( + "ix_commercial_cost_events_subscription_period", + "commercial_cost_events", + ["tenant_id", "subscription_id", "billing_period_id", "occurred_at"], + ) + + +def _bind_reservations_to_periods() -> None: + op.add_column( + "commercial_runtime_reservations", + sa.Column("billing_period_id", sa.String(length=36), nullable=True), + ) + op.add_column( + "commercial_runtime_reservations", + sa.Column("quota_period_key", sa.String(length=64), nullable=True), + ) + op.alter_column( + "commercial_runtime_reservations", + "period_key", + existing_type=sa.String(length=32), + type_=sa.String(length=64), + existing_nullable=False, + ) + op.execute( + """ + UPDATE commercial_runtime_reservations AS reservation + SET billing_period_id = period.id, + quota_period_key = reservation.period_key, + period_key = period.period_key + FROM commercial_billing_periods AS period + WHERE period.tenant_id = reservation.tenant_id + AND period.subscription_id = reservation.subscription_id + AND reservation.created_at >= period.period_start + AND reservation.created_at < period.period_end + """ + ) + _fail_if_unbound( + "commercial_runtime_reservations", + "billing_period_id IS NULL OR quota_period_key IS NULL", + "existing runtime reservations do not map to one immutable billing period", + ) + op.alter_column( + "commercial_runtime_reservations", + "billing_period_id", + existing_type=sa.String(36), + nullable=False, + ) + op.alter_column( + "commercial_runtime_reservations", + "quota_period_key", + existing_type=sa.String(64), + nullable=False, + ) + op.drop_constraint( + "ck_commercial_runtime_reservations_keys", + "commercial_runtime_reservations", + type_="check", + ) + op.create_check_constraint( + "ck_commercial_runtime_reservations_keys", + "commercial_runtime_reservations", + "length(trim(run_id)) > 0 AND length(trim(tool_call_id)) > 0 " + "AND length(trim(tool_type)) > 0 AND length(trim(tool_name)) > 0 " + "AND length(trim(period_key)) > 0 AND length(trim(quota_period_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + ) + op.create_foreign_key( + "fk_commercial_runtime_reservations_tenant_billing_period", + "commercial_runtime_reservations", + "commercial_billing_periods", + ["tenant_id", "subscription_id", "billing_period_id"], + ["tenant_id", "subscription_id", "id"], + ondelete="RESTRICT", + ) + op.drop_index( + "ix_commercial_runtime_reservations_quota", + table_name="commercial_runtime_reservations", + ) + op.create_index( + "ix_commercial_runtime_reservations_quota", + "commercial_runtime_reservations", + ["tenant_id", "subscription_id", "entitlement_id", "quota_period_key", "status"], + ) + op.create_index( + "ix_commercial_runtime_reservations_billing_period", + "commercial_runtime_reservations", + ["tenant_id", "billing_period_id", "status"], + ) + + +def _fail_if_unbound(table_name: str, predicate: str, message: str) -> None: + op.execute( + f""" + DO $$ + BEGIN + IF EXISTS (SELECT 1 FROM {table_name} WHERE {predicate}) THEN + RAISE EXCEPTION '{message}'; + END IF; + END; + $$ + """ + ) + + +def _create_append_only_triggers() -> None: + for table_name in ("commercial_billing_periods", "commercial_admin_events"): + op.execute( + f""" + CREATE TRIGGER trg_{table_name}_append_only + BEFORE UPDATE OR DELETE ON {table_name} + FOR EACH ROW EXECUTE FUNCTION prevent_commercial_events_mutation() + """ + ) + + +def _create_period_overlap_guard() -> None: + op.execute( + """ + CREATE FUNCTION prevent_commercial_billing_period_overlap() + RETURNS trigger AS $$ + BEGIN + PERFORM pg_advisory_xact_lock( + hashtextextended(NEW.tenant_id || ':' || NEW.subscription_id, 0) + ); + IF EXISTS ( + SELECT 1 + FROM commercial_billing_periods AS existing + WHERE existing.tenant_id = NEW.tenant_id + AND existing.subscription_id = NEW.subscription_id + AND tstzrange( + existing.period_start, + existing.period_end, + '[)' + ) && tstzrange(NEW.period_start, NEW.period_end, '[)') + ) THEN + RAISE EXCEPTION 'commercial billing period overlaps an issued period'; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql + """ + ) + op.execute( + """ + CREATE TRIGGER trg_commercial_billing_periods_no_overlap + BEFORE INSERT ON commercial_billing_periods + FOR EACH ROW EXECUTE FUNCTION prevent_commercial_billing_period_overlap() + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_billing_lifecycle_for_downgrade() + _unbind_reservations() + _unbind_costs() + _unbind_usage() + op.execute( + "DROP TRIGGER IF EXISTS trg_commercial_billing_periods_no_overlap " + "ON commercial_billing_periods" + ) + for table_name in ("commercial_admin_events", "commercial_billing_periods"): + op.execute(f"DROP TRIGGER IF EXISTS trg_{table_name}_append_only ON {table_name}") + op.drop_table("commercial_admin_events") + op.drop_table("commercial_billing_periods") + op.execute("DROP FUNCTION IF EXISTS prevent_commercial_billing_period_overlap()") + + +def _unbind_reservations() -> None: + op.drop_index( + "ix_commercial_runtime_reservations_billing_period", + table_name="commercial_runtime_reservations", + ) + op.drop_index( + "ix_commercial_runtime_reservations_quota", + table_name="commercial_runtime_reservations", + ) + op.create_index( + "ix_commercial_runtime_reservations_quota", + "commercial_runtime_reservations", + ["tenant_id", "subscription_id", "entitlement_id", "period_key", "status"], + ) + op.drop_constraint( + "fk_commercial_runtime_reservations_tenant_billing_period", + "commercial_runtime_reservations", + type_="foreignkey", + ) + op.drop_constraint( + "ck_commercial_runtime_reservations_keys", + "commercial_runtime_reservations", + type_="check", + ) + op.create_check_constraint( + "ck_commercial_runtime_reservations_keys", + "commercial_runtime_reservations", + "length(trim(run_id)) > 0 AND length(trim(tool_call_id)) > 0 " + "AND length(trim(tool_type)) > 0 AND length(trim(tool_name)) > 0 " + "AND length(trim(period_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + ) + op.drop_column("commercial_runtime_reservations", "quota_period_key") + op.drop_column("commercial_runtime_reservations", "billing_period_id") + op.alter_column( + "commercial_runtime_reservations", + "period_key", + existing_type=sa.String(length=64), + type_=sa.String(length=32), + existing_nullable=False, + ) + + +def _unbind_costs() -> None: + op.drop_index( + "ix_commercial_cost_events_subscription_period", + table_name="commercial_cost_events", + ) + op.create_index( + "ix_commercial_cost_events_subscription_period", + "commercial_cost_events", + ["tenant_id", "subscription_id", "occurred_at"], + ) + op.drop_constraint( + "fk_commercial_cost_events_tenant_billing_period", + "commercial_cost_events", + type_="foreignkey", + ) + op.drop_constraint( + "ck_commercial_cost_events_billing_period_pair", + "commercial_cost_events", + type_="check", + ) + op.drop_column("commercial_cost_events", "billing_period_id") + + +def _unbind_usage() -> None: + op.drop_index("ix_usage_meter_events_billing_period", table_name="usage_meter_events") + op.drop_index("ix_usage_meter_events_quota_window", table_name="usage_meter_events") + op.create_index( + "ix_usage_meter_events_quota_window", + "usage_meter_events", + ["tenant_id", "subscription_id", "metric_key", "period_key", "occurred_at"], + ) + op.drop_constraint( + "fk_usage_meter_events_tenant_billing_period", + "usage_meter_events", + type_="foreignkey", + ) + op.drop_constraint("ck_usage_meter_events_keys", "usage_meter_events", type_="check") + op.create_check_constraint( + "ck_usage_meter_events_keys", + "usage_meter_events", + "length(trim(metric_key)) > 0 AND length(trim(unit)) > 0 " + "AND length(trim(period_key)) > 0 AND length(trim(source_system)) > 0 " + "AND length(trim(idempotency_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + ) + op.drop_column("usage_meter_events", "quota_period_key") + op.drop_column("usage_meter_events", "billing_period_id") + op.alter_column( + "usage_meter_events", + "period_key", + existing_type=sa.String(length=64), + type_=sa.String(length=32), + existing_nullable=False, + ) diff --git a/server/alembic/versions/20260716_0022_financial_connector_operational_events.py b/server/alembic/versions/20260716_0022_financial_connector_operational_events.py new file mode 100644 index 0000000..7d56dc0 --- /dev/null +++ b/server/alembic/versions/20260716_0022_financial_connector_operational_events.py @@ -0,0 +1,148 @@ +"""add durable financial connector operational events + +Revision ID: 20260716_0022 +Revises: 20260716_0021 +Create Date: 2026-07-17 00:30:00 +""" + +from __future__ import annotations + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0022" +down_revision: str | None = "20260716_0021" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0022 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_operational_events_for_downgrade() -> None: + count = int( + op.get_bind().scalar( + sa.text("SELECT COUNT(*) FROM financial_connector_operational_events") + ) + or 0 + ) + if count: + raise RuntimeError( + "cannot downgrade financial connector operational events: " + f"immutable operational facts exist (financial_connector_operational_events={count})" + ) + + +def upgrade() -> None: + _require_postgresql() + op.create_table( + "financial_connector_operational_events", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("config_id", sa.String(length=36), nullable=False), + sa.Column("provider", sa.String(length=80), nullable=False), + sa.Column("environment", sa.String(length=16), nullable=False), + sa.Column("event_type", sa.String(length=32), nullable=False), + sa.Column("reason_code", sa.String(length=80), nullable=False), + sa.Column("request_fingerprint", sa.String(length=76), nullable=False), + sa.Column("external_event_fingerprint", sa.String(length=76), nullable=False), + sa.Column("idempotency_key", sa.String(length=71), nullable=False), + sa.Column( + "occurred_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "event_type IN ('replay', 'auth_failure', 'payload_conflict')", + name="ck_financial_connector_operational_events_type", + ), + sa.CheckConstraint( + "environment IN ('test', 'mock', 'staging', 'production')", + name="ck_financial_connector_operational_events_environment", + ), + sa.CheckConstraint( + "length(trim(provider)) > 0 AND length(trim(reason_code)) > 0", + name="ck_financial_connector_operational_events_required_text", + ), + sa.CheckConstraint( + "length(request_fingerprint) = 76 " + "AND request_fingerprint LIKE 'hmac-sha256:%' " + "AND length(external_event_fingerprint) = 76 " + "AND external_event_fingerprint LIKE 'hmac-sha256:%' " + "AND length(idempotency_key) = 71 " + "AND idempotency_key LIKE 'sha256:%'", + name="ck_financial_connector_operational_events_fingerprints", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "config_id"], + ["financial_connector_configs.tenant_id", "financial_connector_configs.id"], + name="fk_financial_connector_operational_events_tenant_config", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "id", + name="uq_financial_connector_operational_events_tenant_id", + ), + sa.UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_financial_connector_operational_events_tenant_request", + ), + ) + op.create_index( + "ix_financial_connector_operational_events_tenant_config_time", + "financial_connector_operational_events", + ["tenant_id", "config_id", "occurred_at"], + ) + op.create_index( + "ix_financial_connector_operational_events_tenant_type_time", + "financial_connector_operational_events", + ["tenant_id", "event_type", "occurred_at"], + ) + op.create_index( + "ix_financial_connector_operational_events_tenant_provider_time", + "financial_connector_operational_events", + ["tenant_id", "provider", "occurred_at"], + ) + op.execute( + """ + CREATE TRIGGER trg_financial_connector_operational_events_append_only + BEFORE UPDATE OR DELETE ON financial_connector_operational_events + FOR EACH ROW EXECUTE FUNCTION reject_financial_connector_append_only_mutation() + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_operational_events_for_downgrade() + op.execute( + "DROP TRIGGER IF EXISTS " + "trg_financial_connector_operational_events_append_only " + "ON financial_connector_operational_events" + ) + op.drop_index( + "ix_financial_connector_operational_events_tenant_provider_time", + table_name="financial_connector_operational_events", + ) + op.drop_index( + "ix_financial_connector_operational_events_tenant_type_time", + table_name="financial_connector_operational_events", + ) + op.drop_index( + "ix_financial_connector_operational_events_tenant_config_time", + table_name="financial_connector_operational_events", + ) + op.drop_table("financial_connector_operational_events") diff --git a/server/alembic/versions/20260716_0023_agent_asset_release_blind_audit.py b/server/alembic/versions/20260716_0023_agent_asset_release_blind_audit.py new file mode 100644 index 0000000..75206f1 --- /dev/null +++ b/server/alembic/versions/20260716_0023_agent_asset_release_blind_audit.py @@ -0,0 +1,206 @@ +"""add blind negative audit samples and recall ground-truth labels + +Revision ID: 20260716_0023 +Revises: 20260716_0022 +Create Date: 2026-07-16 23:50:00 +""" + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260716_0023" +down_revision: str | None = "20260716_0022" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +_AUDIT_SAMPLE_TABLE = "agent_asset_release_audit_samples" +_LABEL_TABLE = "agent_asset_release_labels" + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260716_0023 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_lossless_downgrade() -> None: + bind = op.get_bind() + audit_sample_count = int( + bind.scalar(sa.text(f"SELECT COUNT(*) FROM {_AUDIT_SAMPLE_TABLE}")) or 0 + ) + extended_label_count = int( + bind.scalar( + sa.text( + f"SELECT COUNT(*) FROM {_LABEL_TABLE} " + "WHERE label IN ('risk_present', 'risk_absent') " + "OR verification_source = 'blind_release_review'" + ) + ) + or 0 + ) + if audit_sample_count or extended_label_count: + raise RuntimeError( + "cannot downgrade release blind audit: immutable audit evidence exists " + f"(audit_samples={audit_sample_count}, extended_labels={extended_label_count})" + ) + + +def _replace_label_constraints(*, expanded: bool) -> None: + if not expanded: + op.drop_constraint( + "ck_agent_asset_release_labels_semantics", + _LABEL_TABLE, + type_="check", + ) + op.drop_constraint( + "ck_agent_asset_release_labels_label", + _LABEL_TABLE, + type_="check", + ) + op.drop_constraint( + "ck_agent_asset_release_labels_source", + _LABEL_TABLE, + type_="check", + ) + if expanded: + label_values = ( + "label IN ('confirmed', 'false_positive', 'risk_present', 'risk_absent')" + ) + source_values = ( + "verification_source IN ('typed_risk_disposition', 'release_review', " + "'blind_release_review')" + ) + else: + label_values = "label IN ('confirmed', 'false_positive')" + source_values = ( + "verification_source IN ('typed_risk_disposition', 'release_review')" + ) + op.create_check_constraint( + "ck_agent_asset_release_labels_label", + _LABEL_TABLE, + label_values, + ) + op.create_check_constraint( + "ck_agent_asset_release_labels_source", + _LABEL_TABLE, + source_values, + ) + if expanded: + op.create_check_constraint( + "ck_agent_asset_release_labels_semantics", + _LABEL_TABLE, + "(verification_source = 'blind_release_review' " + "AND label IN ('risk_present', 'risk_absent')) OR " + "(verification_source IN ('typed_risk_disposition', 'release_review') " + "AND label IN ('confirmed', 'false_positive'))", + ) + + +def upgrade() -> None: + _require_postgresql() + _replace_label_constraints(expanded=True) + op.create_table( + _AUDIT_SAMPLE_TABLE, + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("observation_id", sa.String(length=36), nullable=False), + sa.Column("asset_id", sa.String(length=36), nullable=False), + sa.Column("release_id", sa.String(length=64), nullable=False), + sa.Column("stage", sa.String(length=16), nullable=False), + sa.Column("version", sa.String(length=30), nullable=False), + sa.Column("stratum", sa.String(length=40), nullable=False), + sa.Column("sampling_probability_ppm", sa.Integer(), nullable=False), + sa.Column("selection_score_ppm", sa.Integer(), nullable=False), + sa.Column("source_reference_encrypted", sa.Text(), nullable=False), + sa.Column("idempotency_key", sa.String(length=80), nullable=False), + sa.Column("payload_fingerprint", sa.String(length=64), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.CheckConstraint( + "stratum IN ('candidate_positive_census', " + "'candidate_disagreement_census', 'candidate_negative_random')", + name="ck_agent_asset_release_audit_samples_stratum", + ), + sa.CheckConstraint( + "sampling_probability_ppm BETWEEN 1 AND 1000000", + name="ck_agent_asset_release_audit_samples_probability", + ), + sa.CheckConstraint( + "selection_score_ppm BETWEEN 0 AND 999999", + name="ck_agent_asset_release_audit_samples_score", + ), + sa.ForeignKeyConstraint( + [ + "tenant_id", + "observation_id", + "asset_id", + "release_id", + "stage", + "version", + ], + [ + "agent_asset_release_observations.tenant_id", + "agent_asset_release_observations.id", + "agent_asset_release_observations.asset_id", + "agent_asset_release_observations.release_id", + "agent_asset_release_observations.stage", + "agent_asset_release_observations.version", + ], + name="fk_agent_asset_release_audit_samples_observation", + ondelete="RESTRICT", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "id", + name="uq_agent_asset_release_audit_samples_tenant_id", + ), + sa.UniqueConstraint( + "tenant_id", + "observation_id", + name="uq_agent_asset_release_audit_samples_observation", + ), + sa.UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_agent_asset_release_audit_samples_idempotency", + ), + ) + op.create_index( + "ix_agent_asset_release_audit_samples_release", + _AUDIT_SAMPLE_TABLE, + ["tenant_id", "asset_id", "release_id", "stage", "version", "created_at"], + ) + op.execute( + f""" + CREATE TRIGGER trg_{_AUDIT_SAMPLE_TABLE}_append_only + BEFORE UPDATE OR DELETE ON {_AUDIT_SAMPLE_TABLE} + FOR EACH ROW EXECUTE FUNCTION reject_agent_asset_release_telemetry_mutation(); + """ + ) + + +def downgrade() -> None: + _require_postgresql() + _require_lossless_downgrade() + op.execute( + f"DROP TRIGGER IF EXISTS trg_{_AUDIT_SAMPLE_TABLE}_append_only " + f"ON {_AUDIT_SAMPLE_TABLE}" + ) + op.drop_index( + "ix_agent_asset_release_audit_samples_release", + table_name=_AUDIT_SAMPLE_TABLE, + ) + op.drop_table(_AUDIT_SAMPLE_TABLE) + _replace_label_constraints(expanded=False) diff --git a/server/alembic/versions/20260717_0024_commercial_resource_quantity_bases.py b/server/alembic/versions/20260717_0024_commercial_resource_quantity_bases.py new file mode 100644 index 0000000..8a5d18f --- /dev/null +++ b/server/alembic/versions/20260717_0024_commercial_resource_quantity_bases.py @@ -0,0 +1,82 @@ +"""expand commercial runtime resource quantity bases + +Revision ID: 20260717_0024 +Revises: 20260716_0023 +Create Date: 2026-07-17 09:30:00 +""" + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260717_0024" +down_revision: str | None = "20260716_0023" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + +_CONSTRAINT = "ck_commercial_runtime_reservations_basis" +_LEGACY_BASES = ( + "quantity_basis IN ('call', 'input_tokens', 'output_tokens', " + "'total_tokens', 'duration_ms')" +) +_RESOURCE_BASES = ( + "quantity_basis IN ('call', 'input_tokens', 'output_tokens', " + "'total_tokens', 'duration_ms', 'bytes', 'pages', 'objects', 'events')" +) + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260717_0024 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_no_resource_reservations_for_downgrade() -> None: + count = int( + op.get_bind().scalar( + sa.text( + "SELECT COUNT(*) FROM commercial_runtime_reservations " + "WHERE quantity_basis IN ('bytes', 'pages', 'objects', 'events')" + ) + ) + or 0 + ) + if count: + raise RuntimeError( + "cannot downgrade commercial resource quantity bases: " + f"resource reservations exist ({count})" + ) + + +def upgrade() -> None: + _require_postgresql() + op.drop_constraint( + _CONSTRAINT, + "commercial_runtime_reservations", + type_="check", + ) + op.create_check_constraint( + _CONSTRAINT, + "commercial_runtime_reservations", + _RESOURCE_BASES, + ) + + +def downgrade() -> None: + _require_postgresql() + _require_no_resource_reservations_for_downgrade() + op.drop_constraint( + _CONSTRAINT, + "commercial_runtime_reservations", + type_="check", + ) + op.create_check_constraint( + _CONSTRAINT, + "commercial_runtime_reservations", + _LEGACY_BASES, + ) diff --git a/server/alembic/versions/20260717_0025_tenant_identity_foundation.py b/server/alembic/versions/20260717_0025_tenant_identity_foundation.py new file mode 100644 index 0000000..2e4514b --- /dev/null +++ b/server/alembic/versions/20260717_0025_tenant_identity_foundation.py @@ -0,0 +1,639 @@ +"""establish trusted tenant identity foundation + +Revision ID: 20260717_0025 +Revises: 20260717_0024 +Create Date: 2026-07-17 10:20:00 +""" + +from __future__ import annotations + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260717_0025" +down_revision: str | None = "20260717_0024" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + +_DEFAULT_TENANT = "default" +_TENANT_SOURCE_TABLES = ( + "auth_sessions", + "expense_cases", + "commercial_billing_periods", + "tenant_subscriptions", + "financial_connector_configs", + "agent_asset_release_observations", +) + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260717_0025 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _inspector() -> sa.Inspector: + return sa.inspect(op.get_bind()) + + +def _has_table(table_name: str) -> bool: + return _inspector().has_table(table_name) + + +def _has_column(table_name: str, column_name: str) -> bool: + if not _has_table(table_name): + return False + return column_name in { + str(item["name"]) for item in _inspector().get_columns(table_name) + } + + +def _constraint_exists(table_name: str, constraint_name: str) -> bool: + inspector = _inspector() + names = { + str(item.get("name") or "") + for item in ( + *inspector.get_unique_constraints(table_name), + *inspector.get_foreign_keys(table_name), + *inspector.get_check_constraints(table_name), + ) + } + return constraint_name in names + + +def _index_exists(table_name: str, index_name: str) -> bool: + return index_name in { + str(item.get("name") or "") for item in _inspector().get_indexes(table_name) + } + + +def _drop_unique_for_columns(table_name: str, columns: tuple[str, ...]) -> None: + if not _has_table(table_name): + return + for item in _inspector().get_unique_constraints(table_name): + if tuple(item.get("column_names") or ()) != columns: + continue + name = str(item.get("name") or "") + if name: + op.drop_constraint(name, table_name, type_="unique") + + +def _drop_foreign_keys_for_columns( + table_name: str, + columns: tuple[str, ...], +) -> None: + if not _has_table(table_name): + return + for item in _inspector().get_foreign_keys(table_name): + if tuple(item.get("constrained_columns") or ()) != columns: + continue + name = str(item.get("name") or "") + if name: + op.drop_constraint(name, table_name, type_="foreignkey") + + +def _ensure_unique( + table_name: str, + name: str, + columns: tuple[str, ...], +) -> None: + if not _constraint_exists(table_name, name): + op.create_unique_constraint(name, table_name, list(columns)) + + +def _ensure_index( + table_name: str, + name: str, + columns: tuple[str, ...], +) -> None: + if not _index_exists(table_name, name): + op.create_index(name, table_name, list(columns)) + + +def _ensure_tenant_column(table_name: str) -> None: + if not _has_table(table_name): + return + if not _has_column(table_name, "tenant_id"): + op.add_column( + table_name, + sa.Column("tenant_id", sa.String(length=64), nullable=True), + ) + if table_name == "expense_claims" and _has_table("expense_case_links"): + op.execute( + sa.text( + "UPDATE expense_claims AS claim SET tenant_id = COALESCE((" + "SELECT link.tenant_id FROM expense_case_links AS link " + "WHERE link.resource_type = 'expense_claim' " + "AND link.resource_id = claim.id ORDER BY link.created_at ASC LIMIT 1" + "), :default_tenant) WHERE claim.tenant_id IS NULL" + ).bindparams(default_tenant=_DEFAULT_TENANT) + ) + else: + op.execute( + sa.text( + f"UPDATE {table_name} SET tenant_id = :default_tenant " + "WHERE tenant_id IS NULL" + ).bindparams(default_tenant=_DEFAULT_TENANT) + ) + op.alter_column( + table_name, + "tenant_id", + existing_type=sa.String(length=64), + nullable=False, + server_default=None, + ) + + +def _create_tenant_registry() -> None: + if not _has_table("tenants"): + op.create_table( + "tenants", + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("tenant_code", sa.String(length=64), nullable=False), + sa.Column("name", sa.String(length=160), nullable=False), + sa.Column("status", sa.String(length=20), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.CheckConstraint( + "status IN ('active', 'suspended', 'disabled')", + name="ck_tenants_status", + ), + sa.CheckConstraint( + "length(trim(tenant_id)) > 0 AND length(trim(tenant_code)) > 0 " + "AND length(trim(name)) > 0", + name="ck_tenants_identity", + ), + sa.PrimaryKeyConstraint("tenant_id"), + sa.UniqueConstraint("tenant_code", name="uq_tenants_tenant_code"), + ) + op.create_index("ix_tenants_status", "tenants", ["status"]) + op.execute( + sa.text( + "INSERT INTO tenants (tenant_id, tenant_code, name, status) VALUES " + "('default', 'default', '默认企业', 'active'), " + "('platform', 'platform', '平台管理域', 'active') " + "ON CONFLICT DO NOTHING" + ) + ) + for table_name in _TENANT_SOURCE_TABLES: + if not _has_column(table_name, "tenant_id"): + continue + op.execute( + sa.text( + f"INSERT INTO tenants (tenant_id, tenant_code, name, status) " + f"SELECT DISTINCT trim(tenant_id), trim(tenant_id), trim(tenant_id), " + f"'active' FROM {table_name} " + "WHERE tenant_id IS NOT NULL AND length(trim(tenant_id)) > 0 " + "ON CONFLICT DO NOTHING" + ) + ) + + +def _tenantize_organizations() -> None: + table = "organization_units" + if not _has_table(table): + return + _ensure_tenant_column(table) + _drop_unique_for_columns(table, ("unit_code",)) + _drop_foreign_keys_for_columns(table, ("parent_id",)) + _ensure_unique(table, "uq_organization_units_tenant_id", ("tenant_id", "id")) + _ensure_unique( + table, + "uq_organization_units_tenant_code", + ("tenant_id", "unit_code"), + ) + if not _constraint_exists(table, "fk_organization_units_tenant"): + op.create_foreign_key( + "fk_organization_units_tenant", + table, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + if not _constraint_exists(table, "fk_organization_units_tenant_parent"): + op.create_foreign_key( + "fk_organization_units_tenant_parent", + table, + table, + ["tenant_id", "parent_id"], + ["tenant_id", "id"], + ondelete="RESTRICT", + ) + _ensure_index( + table, + "ix_organization_units_tenant_name", + ("tenant_id", "name"), + ) + + +def _tenantize_employees() -> None: + table = "employees" + if not _has_table(table): + return + _ensure_tenant_column(table) + _drop_unique_for_columns(table, ("employee_no",)) + _drop_unique_for_columns(table, ("email",)) + _drop_foreign_keys_for_columns(table, ("organization_unit_id",)) + _drop_foreign_keys_for_columns(table, ("manager_id",)) + _ensure_unique(table, "uq_employees_tenant_id", ("tenant_id", "id")) + _ensure_unique( + table, + "uq_employees_tenant_employee_no", + ("tenant_id", "employee_no"), + ) + _ensure_unique(table, "uq_employees_tenant_email", ("tenant_id", "email")) + if not _constraint_exists(table, "fk_employees_tenant"): + op.create_foreign_key( + "fk_employees_tenant", + table, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + if _has_table("organization_units") and not _constraint_exists( + table, + "fk_employees_tenant_organization_unit", + ): + op.create_foreign_key( + "fk_employees_tenant_organization_unit", + table, + "organization_units", + ["tenant_id", "organization_unit_id"], + ["tenant_id", "id"], + ondelete="RESTRICT", + ) + if not _constraint_exists(table, "fk_employees_tenant_manager"): + op.create_foreign_key( + "fk_employees_tenant_manager", + table, + table, + ["tenant_id", "manager_id"], + ["tenant_id", "id"], + ondelete="RESTRICT", + ) + _ensure_index(table, "ix_employees_tenant_status", ("tenant_id", "employment_status")) + _ensure_index(table, "ix_employees_tenant_name", ("tenant_id", "name")) + + +def _tenantize_financial_records() -> None: + if _has_table("expense_claims"): + table = "expense_claims" + _ensure_tenant_column(table) + _drop_unique_for_columns(table, ("claim_no",)) + _drop_foreign_keys_for_columns(table, ("employee_id",)) + _drop_foreign_keys_for_columns(table, ("department_id",)) + _ensure_unique(table, "uq_expense_claims_tenant_id", ("tenant_id", "id")) + _ensure_unique( + table, + "uq_expense_claims_tenant_claim_no", + ("tenant_id", "claim_no"), + ) + if not _constraint_exists(table, "fk_expense_claims_tenant"): + op.create_foreign_key( + "fk_expense_claims_tenant", + table, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + if _has_table("employees") and not _constraint_exists( + table, + "fk_expense_claims_tenant_employee", + ): + op.create_foreign_key( + "fk_expense_claims_tenant_employee", + table, + "employees", + ["tenant_id", "employee_id"], + ["tenant_id", "id"], + ondelete="RESTRICT", + ) + if _has_table("organization_units") and not _constraint_exists( + table, + "fk_expense_claims_tenant_department", + ): + op.create_foreign_key( + "fk_expense_claims_tenant_department", + table, + "organization_units", + ["tenant_id", "department_id"], + ["tenant_id", "id"], + ondelete="RESTRICT", + ) + _ensure_index(table, "ix_expense_claims_tenant_status", ("tenant_id", "status")) + _ensure_index( + table, + "ix_expense_claims_tenant_occurred", + ("tenant_id", "occurred_at"), + ) + for table, number_column, constraint_name, dimension_column, index_name in ( + ( + "accounts_receivable", + "receivable_no", + "uq_accounts_receivable_tenant_no", + "customer_id", + "ix_accounts_receivable_tenant_customer", + ), + ( + "accounts_payable", + "payable_no", + "uq_accounts_payable_tenant_no", + "vendor_id", + "ix_accounts_payable_tenant_vendor", + ), + ): + if not _has_table(table): + continue + _ensure_tenant_column(table) + _drop_unique_for_columns(table, (number_column,)) + _ensure_unique(table, constraint_name, ("tenant_id", number_column)) + tenant_fk = f"fk_{table}_tenant" + if not _constraint_exists(table, tenant_fk): + op.create_foreign_key( + tenant_fk, + table, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + _ensure_index(table, index_name, ("tenant_id", dimension_column)) + + +def _ensure_memberships() -> None: + if not _has_table("employees"): + return + if not _has_table("tenant_memberships"): + op.create_table( + "tenant_memberships", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("employee_id", sa.String(length=36), nullable=False), + sa.Column("status", sa.String(length=20), nullable=False), + sa.Column("is_primary", sa.Boolean(), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + server_default=sa.func.now(), + nullable=False, + ), + sa.CheckConstraint( + "status IN ('active', 'inactive')", + name="ck_tenant_memberships_status", + ), + sa.ForeignKeyConstraint( + ["tenant_id"], + ["tenants.tenant_id"], + name="fk_tenant_memberships_tenant", + ondelete="RESTRICT", + ), + sa.ForeignKeyConstraint( + ["tenant_id", "employee_id"], + ["employees.tenant_id", "employees.id"], + name="fk_tenant_memberships_tenant_employee", + ondelete="CASCADE", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "employee_id", + name="uq_tenant_memberships_tenant_employee", + ), + ) + op.create_index( + "ix_tenant_memberships_employee_active", + "tenant_memberships", + ["employee_id", "status"], + ) + op.create_index( + "ix_tenant_memberships_tenant_active", + "tenant_memberships", + ["tenant_id", "status"], + ) + op.execute( + sa.text( + "INSERT INTO tenant_memberships " + "(id, tenant_id, employee_id, status, is_primary) " + "SELECT gen_random_uuid()::text, employee.tenant_id, employee.id, " + "CASE WHEN employee.employment_status = '停用' THEN 'inactive' " + "ELSE 'active' END, true FROM employees AS employee " + "ON CONFLICT (tenant_id, employee_id) DO NOTHING" + ) + ) + + +def _bind_auth_sessions() -> None: + if not _has_table("auth_sessions"): + return + if not _constraint_exists("auth_sessions", "fk_auth_sessions_tenant"): + op.create_foreign_key( + "fk_auth_sessions_tenant", + "auth_sessions", + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + + +def upgrade() -> None: + _require_postgresql() + _create_tenant_registry() + _tenantize_organizations() + _tenantize_employees() + _tenantize_financial_records() + _ensure_memberships() + _bind_auth_sessions() + + +def _require_safe_downgrade() -> None: + for table_name in ( + "organization_units", + "employees", + "expense_claims", + "accounts_receivable", + "accounts_payable", + ): + if not _has_column(table_name, "tenant_id"): + continue + count = int( + op.get_bind().scalar( + sa.text( + f"SELECT COUNT(*) FROM {table_name} " + "WHERE tenant_id <> :default_tenant" + ).bindparams(default_tenant=_DEFAULT_TENANT) + ) + or 0 + ) + if count: + raise RuntimeError( + "cannot downgrade tenant identity foundation: " + f"{table_name} contains non-default tenant data ({count})" + ) + + +def _restore_legacy_financial_constraints() -> None: + if _has_table("expense_claims"): + table = "expense_claims" + for name in ( + "fk_expense_claims_tenant_department", + "fk_expense_claims_tenant_employee", + "fk_expense_claims_tenant", + "uq_expense_claims_tenant_claim_no", + "uq_expense_claims_tenant_id", + ): + if _constraint_exists(table, name): + op.drop_constraint(name, table) + for name in ( + "ix_expense_claims_tenant_occurred", + "ix_expense_claims_tenant_status", + ): + if _index_exists(table, name): + op.drop_index(name, table_name=table) + _ensure_unique(table, "uq_expense_claims_claim_no", ("claim_no",)) + if _has_table("employees"): + op.create_foreign_key( + "fk_expense_claims_employee_id", + table, + "employees", + ["employee_id"], + ["id"], + ) + if _has_table("organization_units"): + op.create_foreign_key( + "fk_expense_claims_department_id", + table, + "organization_units", + ["department_id"], + ["id"], + ) + op.drop_column(table, "tenant_id") + for table, constraint_name, number_column, index_name in ( + ( + "accounts_receivable", + "uq_accounts_receivable_tenant_no", + "receivable_no", + "ix_accounts_receivable_tenant_customer", + ), + ( + "accounts_payable", + "uq_accounts_payable_tenant_no", + "payable_no", + "ix_accounts_payable_tenant_vendor", + ), + ): + if not _has_column(table, "tenant_id"): + continue + tenant_fk = f"fk_{table}_tenant" + if _constraint_exists(table, tenant_fk): + op.drop_constraint(tenant_fk, table, type_="foreignkey") + if _constraint_exists(table, constraint_name): + op.drop_constraint(constraint_name, table, type_="unique") + if _index_exists(table, index_name): + op.drop_index(index_name, table_name=table) + _ensure_unique(table, f"uq_{table}_{number_column}", (number_column,)) + op.drop_column(table, "tenant_id") + + +def _restore_legacy_employee_constraints() -> None: + if _has_table("employees"): + table = "employees" + for name in ( + "fk_employees_tenant_manager", + "fk_employees_tenant_organization_unit", + "fk_employees_tenant", + "uq_employees_tenant_email", + "uq_employees_tenant_employee_no", + "uq_employees_tenant_id", + ): + if _constraint_exists(table, name): + op.drop_constraint(name, table) + for name in ("ix_employees_tenant_name", "ix_employees_tenant_status"): + if _index_exists(table, name): + op.drop_index(name, table_name=table) + _ensure_unique(table, "uq_employees_employee_no", ("employee_no",)) + _ensure_unique(table, "uq_employees_email", ("email",)) + if _has_table("organization_units"): + op.create_foreign_key( + "fk_employees_organization_unit_id", + table, + "organization_units", + ["organization_unit_id"], + ["id"], + ) + op.create_foreign_key( + "fk_employees_manager_id", + table, + table, + ["manager_id"], + ["id"], + ) + op.drop_column(table, "tenant_id") + if _has_table("organization_units"): + table = "organization_units" + for name in ( + "fk_organization_units_tenant_parent", + "fk_organization_units_tenant", + "uq_organization_units_tenant_code", + "uq_organization_units_tenant_id", + ): + if _constraint_exists(table, name): + op.drop_constraint(name, table) + if _index_exists(table, "ix_organization_units_tenant_name"): + op.drop_index("ix_organization_units_tenant_name", table_name=table) + _ensure_unique(table, "uq_organization_units_unit_code", ("unit_code",)) + op.create_foreign_key( + "fk_organization_units_parent_id", + table, + table, + ["parent_id"], + ["id"], + ) + op.drop_column(table, "tenant_id") + + +def downgrade() -> None: + _require_postgresql() + _require_safe_downgrade() + if _has_table("auth_sessions") and _constraint_exists( + "auth_sessions", + "fk_auth_sessions_tenant", + ): + op.drop_constraint( + "fk_auth_sessions_tenant", + "auth_sessions", + type_="foreignkey", + ) + if _has_table("tenant_memberships"): + op.drop_table("tenant_memberships") + _restore_legacy_financial_constraints() + _restore_legacy_employee_constraints() + if _has_table("tenants"): + if _index_exists("tenants", "ix_tenants_status"): + op.drop_index("ix_tenants_status", table_name="tenants") + op.drop_table("tenants") diff --git a/server/alembic/versions/20260717_0026_agent_asset_tenant_security.py b/server/alembic/versions/20260717_0026_agent_asset_tenant_security.py new file mode 100644 index 0000000..52cd809 --- /dev/null +++ b/server/alembic/versions/20260717_0026_agent_asset_tenant_security.py @@ -0,0 +1,444 @@ +"""add structural tenant and platform scope to Agent assets + +Revision ID: 20260717_0026 +Revises: 20260717_0025 +Create Date: 2026-07-17 14:30:00 +""" + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260717_0026" +down_revision: str | None = "20260717_0025" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + +_ASSET_TABLE = "agent_assets" +_ASSET_CHILD_TABLES = ( + "agent_asset_versions", + "agent_asset_reviews", + "agent_asset_test_runs", + "agent_asset_rule_feedback", +) +_OWNERSHIP_CHILD_TABLES = ( + "agent_asset_versions", + "agent_asset_reviews", +) +_ONLYOFFICE_SESSION_TABLE = "agent_asset_onlyoffice_sessions" + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260717_0026 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _has_complete_agent_asset_schema() -> bool: + inspector = sa.inspect(op.get_bind()) + table_names = (_ASSET_TABLE, *_ASSET_CHILD_TABLES) + existing = {table_name for table_name in table_names if inspector.has_table(table_name)} + if not existing: + return False + missing = set(table_names) - existing + if missing: + raise RuntimeError( + "cannot migrate partial Agent asset schema; missing tables: " + + ", ".join(sorted(missing)) + ) + return True + + +def _create_onlyoffice_session_table() -> None: + if sa.inspect(op.get_bind()).has_table(_ONLYOFFICE_SESSION_TABLE): + return + op.create_table( + _ONLYOFFICE_SESSION_TABLE, + sa.Column("jti", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("resource_scope", sa.String(length=16), nullable=False), + sa.Column("asset_id", sa.String(length=100), nullable=False), + sa.Column("document_key", sa.String(length=200), nullable=False), + sa.Column("document_version", sa.String(length=30), nullable=False), + sa.Column("document_fingerprint", sa.String(length=160), nullable=False), + sa.Column("audience", sa.String(length=80), nullable=False), + sa.Column("writable", sa.Boolean(), nullable=False), + sa.Column("status", sa.String(length=16), nullable=False), + sa.Column("actor", sa.String(length=160), nullable=False), + sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("claimed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("consumed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("failure_reason", sa.Text(), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "(resource_scope = 'platform' AND tenant_id = 'platform') OR " + "(resource_scope = 'tenant' AND tenant_id <> 'platform')", + name="ck_agent_asset_onlyoffice_sessions_scope_tenant", + ), + sa.CheckConstraint( + "status IN ('active', 'processing', 'consumed', 'failed', 'revoked')", + name="ck_agent_asset_onlyoffice_sessions_status", + ), + sa.CheckConstraint( + "(status = 'active' AND claimed_at IS NULL AND consumed_at IS NULL) OR " + "(status IN ('processing', 'failed') AND claimed_at IS NOT NULL " + "AND consumed_at IS NULL) OR " + "(status = 'consumed' AND claimed_at IS NOT NULL AND consumed_at IS NOT NULL) OR " + "(status = 'revoked' AND consumed_at IS NULL)", + name="ck_agent_asset_onlyoffice_sessions_lifecycle", + ), + sa.ForeignKeyConstraint( + ["tenant_id"], + ["tenants.tenant_id"], + name="fk_agent_asset_onlyoffice_sessions_tenant", + ondelete="CASCADE", + ), + sa.PrimaryKeyConstraint("jti"), + ) + op.create_index( + "ix_agent_asset_onlyoffice_sessions_tenant_asset", + _ONLYOFFICE_SESSION_TABLE, + ["tenant_id", "resource_scope", "asset_id", "created_at"], + ) + op.create_index( + "ix_agent_asset_onlyoffice_sessions_status_expiry", + _ONLYOFFICE_SESSION_TABLE, + ["status", "expires_at"], + ) + + +def _add_scope_columns(table_name: str) -> None: + op.add_column(table_name, sa.Column("tenant_id", sa.String(length=64), nullable=True)) + op.add_column(table_name, sa.Column("scope", sa.String(length=16), nullable=True)) + + +def _assert_known_asset_tenants() -> None: + unknown = int( + op.get_bind().scalar( + sa.text( + """ + SELECT COUNT(*) + FROM agent_assets AS asset + LEFT JOIN tenants AS tenant + ON tenant.tenant_id = NULLIF(BTRIM(asset.config_json ->> 'tenant_id'), '') + WHERE NULLIF(BTRIM(asset.config_json ->> 'tenant_id'), '') IS NOT NULL + AND tenant.tenant_id IS NULL + """ + ) + ) + or 0 + ) + if unknown: + raise RuntimeError( + "cannot migrate Agent assets: config_json contains tenant ids absent from tenants " + f"(unknown_assets={unknown})" + ) + + +def _backfill_scope() -> None: + _assert_known_asset_tenants() + op.execute( + """ + UPDATE agent_assets + SET tenant_id = COALESCE( + NULLIF(BTRIM(config_json ->> 'tenant_id'), ''), + 'platform' + ), + scope = CASE + WHEN NULLIF(BTRIM(config_json ->> 'tenant_id'), '') IS NULL + THEN 'platform' + ELSE 'tenant' + END + """ + ) + for table_name in _OWNERSHIP_CHILD_TABLES: + op.execute( + f""" + UPDATE {table_name} AS child + SET tenant_id = asset.tenant_id, + scope = asset.scope + FROM agent_assets AS asset + WHERE child.asset_id = asset.id + """ + ) + op.execute( + """ + UPDATE agent_asset_test_runs AS child + SET tenant_id = COALESCE( + NULLIF(BTRIM(child.input_json ->> 'target_tenant_id'), ''), + NULLIF(BTRIM(asset.config_json ->> 'tenant_id'), ''), + asset.tenant_id + ), + scope = CASE + WHEN COALESCE( + NULLIF(BTRIM(child.input_json ->> 'target_tenant_id'), ''), + NULLIF(BTRIM(asset.config_json ->> 'tenant_id'), ''), + asset.tenant_id + ) = 'platform' THEN 'platform' + ELSE 'tenant' + END + FROM agent_assets AS asset + WHERE child.asset_id = asset.id + """ + ) + op.execute( + """ + UPDATE agent_asset_rule_feedback AS child + SET tenant_id = COALESCE( + NULLIF(BTRIM(asset.config_json ->> 'tenant_id'), ''), + asset.tenant_id + ), + scope = CASE + WHEN COALESCE( + NULLIF(BTRIM(asset.config_json ->> 'tenant_id'), ''), + asset.tenant_id + ) = 'platform' THEN 'platform' + ELSE 'tenant' + END + FROM agent_assets AS asset + WHERE child.asset_id = asset.id + """ + ) + + +def _set_scope_not_null_and_defaults(table_name: str) -> None: + op.alter_column( + table_name, + "tenant_id", + existing_type=sa.String(length=64), + nullable=False, + server_default="platform", + ) + op.alter_column( + table_name, + "scope", + existing_type=sa.String(length=16), + nullable=False, + server_default="platform", + ) + + +def _assert_known_scoped_tenants() -> None: + bind = op.get_bind() + for table_name in (_ASSET_TABLE, *_ASSET_CHILD_TABLES): + unknown = int( + bind.scalar( + sa.text( + f""" + SELECT COUNT(*) + FROM {table_name} AS scoped + LEFT JOIN tenants AS tenant ON tenant.tenant_id = scoped.tenant_id + WHERE tenant.tenant_id IS NULL + """ + ) + ) + or 0 + ) + if unknown: + raise RuntimeError( + f"cannot migrate {table_name}: scoped tenant is absent from tenants " + f"(unknown_rows={unknown})" + ) + + +def _create_scope_check(table_name: str) -> None: + op.create_check_constraint( + f"ck_{table_name}_scope_tenant", + table_name, + "(scope = 'platform' AND tenant_id = 'platform') OR " + "(scope = 'tenant' AND tenant_id <> 'platform')", + ) + + +def _drop_legacy_code_uniqueness() -> None: + op.execute("ALTER TABLE agent_assets DROP CONSTRAINT IF EXISTS agent_assets_code_key") + op.execute("DROP INDEX IF EXISTS ix_agent_assets_code") + op.create_index("ix_agent_assets_code", _ASSET_TABLE, ["code"], unique=False) + + +def upgrade() -> None: + _require_postgresql() + has_agent_asset_schema = _has_complete_agent_asset_schema() + _create_onlyoffice_session_table() + if not has_agent_asset_schema: + return + _add_scope_columns(_ASSET_TABLE) + for table_name in _ASSET_CHILD_TABLES: + _add_scope_columns(table_name) + _backfill_scope() + _set_scope_not_null_and_defaults(_ASSET_TABLE) + for table_name in _ASSET_CHILD_TABLES: + _set_scope_not_null_and_defaults(table_name) + _assert_known_scoped_tenants() + + _drop_legacy_code_uniqueness() + op.create_unique_constraint( + "uq_agent_assets_tenant_scope_id", + _ASSET_TABLE, + ["tenant_id", "scope", "id"], + ) + op.create_unique_constraint( + "uq_agent_assets_tenant_scope_code", + _ASSET_TABLE, + ["tenant_id", "scope", "code"], + ) + op.create_foreign_key( + "fk_agent_assets_tenant", + _ASSET_TABLE, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + _create_scope_check(_ASSET_TABLE) + op.create_index( + "ix_agent_assets_scope_tenant", + _ASSET_TABLE, + ["scope", "tenant_id"], + ) + + for table_name in _ASSET_CHILD_TABLES: + op.create_foreign_key( + f"fk_{table_name}_tenant", + table_name, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + _create_scope_check(table_name) + op.create_index( + f"ix_{table_name}_tenant_asset", + table_name, + ["tenant_id", "scope", "asset_id"], + ) + for table_name in _OWNERSHIP_CHILD_TABLES: + op.create_foreign_key( + f"fk_{table_name}_tenant_asset", + table_name, + _ASSET_TABLE, + ["tenant_id", "scope", "asset_id"], + ["tenant_id", "scope", "id"], + ondelete="CASCADE", + ) + + +def _require_lossless_downgrade() -> None: + bind = op.get_bind() + tenant_assets = int( + bind.scalar( + sa.text("SELECT COUNT(*) FROM agent_assets WHERE scope = 'tenant'") + ) + or 0 + ) + tenant_evidence = sum( + int( + bind.scalar( + sa.text(f"SELECT COUNT(*) FROM {table_name} WHERE scope = 'tenant'") + ) + or 0 + ) + for table_name in ("agent_asset_test_runs", "agent_asset_rule_feedback") + ) + if tenant_assets or tenant_evidence: + raise RuntimeError( + "cannot downgrade Agent asset tenant security: tenant-owned facts exist " + f"(tenant_assets={tenant_assets}, tenant_evidence={tenant_evidence})" + ) + + +def _require_no_onlyoffice_sessions() -> None: + if not sa.inspect(op.get_bind()).has_table(_ONLYOFFICE_SESSION_TABLE): + return + session_count = int( + op.get_bind().scalar( + sa.text(f"SELECT COUNT(*) FROM {_ONLYOFFICE_SESSION_TABLE}") + ) + or 0 + ) + if session_count: + raise RuntimeError( + "cannot downgrade Agent asset tenant security: ONLYOFFICE session evidence " + f"exists (sessions={session_count})" + ) + + +def _drop_onlyoffice_session_table() -> None: + if not sa.inspect(op.get_bind()).has_table(_ONLYOFFICE_SESSION_TABLE): + return + op.drop_index( + "ix_agent_asset_onlyoffice_sessions_status_expiry", + table_name=_ONLYOFFICE_SESSION_TABLE, + ) + op.drop_index( + "ix_agent_asset_onlyoffice_sessions_tenant_asset", + table_name=_ONLYOFFICE_SESSION_TABLE, + ) + op.drop_table(_ONLYOFFICE_SESSION_TABLE) + + +def _drop_constraint_if_exists( + table_name: str, + constraint_name: str, +) -> None: + """删除本迁移负责的约束，兼容模型建表产生的不同外键名称。""" + preparer = op.get_bind().dialect.identifier_preparer + op.execute( + sa.text( + f"ALTER TABLE {preparer.quote(table_name)} " + f"DROP CONSTRAINT IF EXISTS {preparer.quote(constraint_name)}" + ) + ) + + +def downgrade() -> None: + _require_postgresql() + has_agent_asset_schema = _has_complete_agent_asset_schema() + _require_no_onlyoffice_sessions() + _drop_onlyoffice_session_table() + if not has_agent_asset_schema: + return + _require_lossless_downgrade() + for table_name in reversed(_OWNERSHIP_CHILD_TABLES): + _drop_constraint_if_exists( + table_name, + f"fk_{table_name}_tenant_asset", + ) + for table_name in reversed(_ASSET_CHILD_TABLES): + _drop_constraint_if_exists( + table_name, + f"fk_{table_name}_tenant", + ) + op.drop_index(f"ix_{table_name}_tenant_asset", table_name=table_name) + _drop_constraint_if_exists( + table_name, + f"ck_{table_name}_scope_tenant", + ) + op.drop_index("ix_agent_assets_scope_tenant", table_name=_ASSET_TABLE) + _drop_constraint_if_exists(_ASSET_TABLE, "ck_agent_assets_scope_tenant") + _drop_constraint_if_exists(_ASSET_TABLE, "fk_agent_assets_tenant") + _drop_constraint_if_exists( + _ASSET_TABLE, + "uq_agent_assets_tenant_scope_code", + ) + _drop_constraint_if_exists( + _ASSET_TABLE, + "uq_agent_assets_tenant_scope_id", + ) + op.drop_index("ix_agent_assets_code", table_name=_ASSET_TABLE) + op.create_index("ix_agent_assets_code", _ASSET_TABLE, ["code"], unique=True) + for table_name in reversed(_ASSET_CHILD_TABLES): + op.drop_column(table_name, "scope") + op.drop_column(table_name, "tenant_id") + op.drop_column(_ASSET_TABLE, "scope") + op.drop_column(_ASSET_TABLE, "tenant_id") diff --git a/server/alembic/versions/20260717_0027_knowledge_tenant_security.py b/server/alembic/versions/20260717_0027_knowledge_tenant_security.py new file mode 100644 index 0000000..07346c5 --- /dev/null +++ b/server/alembic/versions/20260717_0027_knowledge_tenant_security.py @@ -0,0 +1,122 @@ +"""add tenant-bound one-time OnlyOffice sessions + +Revision ID: 20260717_0027 +Revises: 20260717_0026 +Create Date: 2026-07-17 10:00:00 +""" + +from __future__ import annotations + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260717_0027" +down_revision: str | None = "20260717_0026" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260717_0027 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _require_empty_for_downgrade() -> None: + count = int( + op.get_bind().scalar( + sa.text("SELECT COUNT(*) FROM knowledge_onlyoffice_sessions") + ) + or 0 + ) + if count: + raise RuntimeError( + "cannot downgrade knowledge tenant security: OnlyOffice session evidence exists " + f"(knowledge_onlyoffice_sessions={count})" + ) + + +def upgrade() -> None: + _require_postgresql() + op.create_table( + "knowledge_onlyoffice_sessions", + sa.Column("jti", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("resource_scope", sa.String(length=16), nullable=False), + sa.Column("document_id", sa.String(length=64), nullable=False), + sa.Column("document_key", sa.String(length=160), nullable=False), + sa.Column("document_version", sa.Integer(), nullable=False), + sa.Column("audience", sa.String(length=80), nullable=False), + sa.Column("editable", sa.Boolean(), nullable=False), + sa.Column("status", sa.String(length=16), nullable=False), + sa.Column("created_by", sa.String(length=100), nullable=False), + sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("claimed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("consumed_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("failure_reason", sa.Text(), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "resource_scope IN ('tenant', 'platform')", + name="ck_knowledge_onlyoffice_sessions_scope", + ), + sa.CheckConstraint( + "status IN ('active', 'processing', 'consumed', 'failed', 'revoked')", + name="ck_knowledge_onlyoffice_sessions_status", + ), + sa.CheckConstraint( + "tenant_id IS NOT NULL AND " + "(resource_scope = 'tenant' OR " + "(resource_scope = 'platform' AND editable = false))", + name="ck_knowledge_onlyoffice_sessions_scope_tenant", + ), + sa.CheckConstraint( + "(status = 'active' AND claimed_at IS NULL AND consumed_at IS NULL) OR " + "(status IN ('processing', 'failed') AND claimed_at IS NOT NULL " + "AND consumed_at IS NULL) OR " + "(status = 'consumed' AND claimed_at IS NOT NULL AND consumed_at IS NOT NULL) OR " + "(status = 'revoked' AND consumed_at IS NULL)", + name="ck_knowledge_onlyoffice_sessions_lifecycle", + ), + sa.ForeignKeyConstraint( + ["tenant_id"], + ["tenants.tenant_id"], + name="fk_knowledge_onlyoffice_sessions_tenant", + ondelete="CASCADE", + ), + sa.PrimaryKeyConstraint("jti"), + ) + op.create_index( + "ix_knowledge_onlyoffice_sessions_tenant_document", + "knowledge_onlyoffice_sessions", + ["tenant_id", "document_id", "created_at"], + ) + op.create_index( + "ix_knowledge_onlyoffice_sessions_status_expiry", + "knowledge_onlyoffice_sessions", + ["status", "expires_at"], + ) + + +def downgrade() -> None: + _require_postgresql() + _require_empty_for_downgrade() + op.drop_index( + "ix_knowledge_onlyoffice_sessions_status_expiry", + table_name="knowledge_onlyoffice_sessions", + ) + op.drop_index( + "ix_knowledge_onlyoffice_sessions_tenant_document", + table_name="knowledge_onlyoffice_sessions", + ) + op.drop_table("knowledge_onlyoffice_sessions") diff --git a/server/alembic/versions/20260717_0028_hermes_ontology_tenant_security.py b/server/alembic/versions/20260717_0028_hermes_ontology_tenant_security.py new file mode 100644 index 0000000..2ff2ba1 --- /dev/null +++ b/server/alembic/versions/20260717_0028_hermes_ontology_tenant_security.py @@ -0,0 +1,484 @@ +"""tenant-scope Hermes profiles, reports, and scheduled work + +Revision ID: 20260717_0028 +Revises: 20260717_0027 +Create Date: 2026-07-17 +""" + +from __future__ import annotations + +from collections.abc import Sequence + +import sqlalchemy as sa + +from alembic import op + +revision: str = "20260717_0028" +down_revision: str | None = "20260717_0027" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def _require_postgresql() -> None: + dialect_name = op.get_bind().dialect.name + if dialect_name != "postgresql": + raise RuntimeError( + "20260717_0028 only supports PostgreSQL; " + f"refusing to mutate {dialect_name} without transactional constraint DDL" + ) + + +def _inspector() -> sa.Inspector: + return sa.inspect(op.get_bind()) + + +def _has_table(table_name: str) -> bool: + return _inspector().has_table(table_name) + + +def _has_column(table_name: str, column_name: str) -> bool: + if not _has_table(table_name): + return False + return column_name in {str(item["name"]) for item in _inspector().get_columns(table_name)} + + +def _constraint_exists(table_name: str, constraint_name: str) -> bool: + if not _has_table(table_name): + return False + inspector = _inspector() + names = { + str(item.get("name") or "") + for item in ( + *inspector.get_unique_constraints(table_name), + *inspector.get_foreign_keys(table_name), + *inspector.get_check_constraints(table_name), + ) + } + return constraint_name in names + + +def _index_exists(table_name: str, index_name: str) -> bool: + if not _has_table(table_name): + return False + return index_name in { + str(item.get("name") or "") for item in _inspector().get_indexes(table_name) + } + + +def _drop_foreign_keys_for_columns( + table_name: str, + columns: tuple[str, ...], +) -> None: + if not _has_table(table_name): + return + for item in _inspector().get_foreign_keys(table_name): + if tuple(item.get("constrained_columns") or ()) != columns: + continue + name = str(item.get("name") or "") + if name: + op.drop_constraint(name, table_name, type_="foreignkey") + + +def upgrade() -> None: + _require_postgresql() + _scope_profile_snapshots() + _scope_hermes_task_tables() + _scope_hermes_risk_reports() + _create_finance_report_tables() + + +def downgrade() -> None: + _require_postgresql() + _assert_safe_downgrade() + if _has_table("tenant_finance_report_runs"): + op.drop_table("tenant_finance_report_runs") + if _has_table("tenant_finance_report_configs"): + op.drop_table("tenant_finance_report_configs") + + _downgrade_risk_reports() + _downgrade_task_tables() + _downgrade_profile_snapshots() + + +def _scope_profile_snapshots() -> None: + table = "employee_behavior_profile_snapshots" + if not _has_table(table): + return + op.add_column( + table, + sa.Column("tenant_id", sa.String(length=64), nullable=False, server_default="default"), + ) + op.create_foreign_key( + "fk_employee_behavior_profiles_tenant", + table, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + op.create_unique_constraint( + "uq_employee_behavior_profiles_tenant_id", + table, + ["tenant_id", "id"], + ) + if _has_table("employees"): + op.create_foreign_key( + "fk_employee_behavior_profiles_tenant_employee", + table, + "employees", + ["tenant_id", "subject_id"], + ["tenant_id", "id"], + ondelete="CASCADE", + ) + if _index_exists(table, "ix_employee_behavior_profile_latest"): + op.drop_index("ix_employee_behavior_profile_latest", table_name=table) + op.create_index( + "ix_employee_behavior_profile_latest", + table, + [ + "tenant_id", + "subject_id", + "profile_type", + "window_days", + "expense_type_scope", + "calculated_at", + ], + ) + op.alter_column(table, "tenant_id", server_default=None) + + +def _scope_hermes_task_tables() -> None: + config_table = "hermes_task_configs" + log_table = "hermes_task_execution_logs" + for table, fk_name in ( + (config_table, "fk_hermes_task_configs_tenant"), + (log_table, "fk_hermes_task_execution_logs_tenant"), + ): + if not _has_table(table): + continue + op.add_column( + table, + sa.Column( + "tenant_id", + sa.String(length=64), + nullable=False, + server_default="default", + ), + ) + op.create_foreign_key( + fk_name, + table, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + op.alter_column(table, "tenant_id", server_default=None) + + if _has_table(config_table): + op.create_unique_constraint( + "uq_hermes_task_configs_tenant_id", + config_table, + ["tenant_id", "id"], + ) + op.create_index( + "ix_hermes_task_configs_tenant_enabled", + config_table, + ["tenant_id", "is_enabled"], + ) + if _has_table(log_table): + op.create_unique_constraint( + "uq_hermes_task_execution_logs_tenant_id", + log_table, + ["tenant_id", "id"], + ) + if _has_table(config_table): + _drop_foreign_keys_for_columns(log_table, ("config_id",)) + op.create_foreign_key( + "fk_hermes_task_logs_tenant_config", + log_table, + config_table, + ["tenant_id", "config_id"], + ["tenant_id", "id"], + ondelete="CASCADE", + ) + op.create_index( + "ix_hermes_task_logs_tenant_started", + log_table, + ["tenant_id", "started_at"], + ) + + +def _scope_hermes_risk_reports() -> None: + table = "hermes_risk_reports" + if not _has_table(table): + return + op.add_column( + table, + sa.Column("tenant_id", sa.String(length=64), nullable=False, server_default="default"), + ) + op.create_foreign_key( + "fk_hermes_risk_reports_tenant", + table, + "tenants", + ["tenant_id"], + ["tenant_id"], + ondelete="RESTRICT", + ) + op.create_unique_constraint( + "uq_hermes_risk_reports_tenant_id", + table, + ["tenant_id", "id"], + ) + if _has_table("expense_claims"): + _drop_foreign_keys_for_columns(table, ("claim_id",)) + op.create_foreign_key( + "fk_hermes_risk_reports_tenant_claim", + table, + "expense_claims", + ["tenant_id", "claim_id"], + ["tenant_id", "id"], + ondelete="CASCADE", + ) + if _has_table("hermes_task_execution_logs"): + _drop_foreign_keys_for_columns(table, ("execution_log_id",)) + op.create_foreign_key( + "fk_hermes_risk_reports_tenant_log", + table, + "hermes_task_execution_logs", + ["tenant_id", "execution_log_id"], + ["tenant_id", "id"], + ondelete="CASCADE", + ) + op.create_index( + "ix_hermes_risk_reports_tenant_status", + table, + ["tenant_id", "status"], + ) + op.alter_column(table, "tenant_id", server_default=None) + + +def _create_finance_report_tables() -> None: + if not _has_table("tenant_finance_report_configs"): + op.create_table( + "tenant_finance_report_configs", + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("status", sa.String(length=20), nullable=False, server_default="disabled"), + sa.Column("delivery_enabled", sa.Boolean(), nullable=False, server_default=sa.false()), + sa.Column("recipients_json", sa.JSON(), nullable=False, server_default="[]"), + sa.Column("updated_by", sa.String(length=100), nullable=False, server_default=""), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.CheckConstraint( + "status IN ('active', 'disabled')", + name="ck_tenant_finance_report_configs_status", + ), + sa.ForeignKeyConstraint( + ["tenant_id"], + ["tenants.tenant_id"], + ondelete="CASCADE", + ), + sa.PrimaryKeyConstraint("tenant_id"), + ) + if not _has_table("tenant_finance_report_runs"): + op.create_table( + "tenant_finance_report_runs", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("tenant_id", sa.String(length=64), nullable=False), + sa.Column("report_type", sa.String(length=20), nullable=False), + sa.Column("period_start", sa.Date(), nullable=False), + sa.Column("period_end", sa.Date(), nullable=False), + sa.Column("idempotency_key", sa.String(length=180), nullable=False), + sa.Column("status", sa.String(length=20), nullable=False, server_default="running"), + sa.Column("agent_run_id", sa.String(length=50), nullable=True), + sa.Column("storage_key", sa.String(length=512), nullable=False, server_default=""), + sa.Column("result_json", sa.JSON(), nullable=False, server_default="{}"), + sa.Column("error_message", sa.Text(), nullable=True), + sa.Column( + "started_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True), + sa.CheckConstraint( + "report_type IN ('weekly', 'quarterly', 'annual')", + name="ck_tenant_finance_report_runs_type", + ), + sa.CheckConstraint( + "status IN ('running', 'succeeded', 'failed')", + name="ck_tenant_finance_report_runs_status", + ), + sa.ForeignKeyConstraint( + ["tenant_id"], + ["tenants.tenant_id"], + ondelete="CASCADE", + ), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_tenant_finance_report_runs_idempotency", + ), + ) + op.create_index( + "ix_tenant_finance_report_runs_period", + "tenant_finance_report_runs", + ["tenant_id", "report_type", "period_start", "period_end"], + ) + op.create_index( + "ix_tenant_finance_report_runs_agent_run_id", + "tenant_finance_report_runs", + ["agent_run_id"], + ) + + +def _assert_safe_downgrade() -> None: + connection = op.get_bind() + for table in ( + "employee_behavior_profile_snapshots", + "hermes_task_configs", + "hermes_task_execution_logs", + "hermes_risk_reports", + ): + if not _has_column(table, "tenant_id"): + continue + count = connection.execute( + sa.text(f"SELECT COUNT(*) FROM {table} WHERE tenant_id <> 'default'") + ).scalar_one() + if int(count or 0) > 0: + raise RuntimeError(f"Refusing downgrade: {table} contains non-default tenant rows.") + for table in ("tenant_finance_report_configs", "tenant_finance_report_runs"): + if not _has_table(table): + continue + count = connection.execute(sa.text(f"SELECT COUNT(*) FROM {table}")).scalar_one() + if int(count or 0) > 0: + raise RuntimeError(f"Refusing downgrade: {table} contains data.") + + +def _drop_constraint_if_exists(table: str, name: str, type_: str) -> None: + if _constraint_exists(table, name): + op.drop_constraint(name, table, type_=type_) + + +def _drop_index_if_exists(table: str, name: str) -> None: + if _index_exists(table, name): + op.drop_index(name, table_name=table) + + +def _downgrade_risk_reports() -> None: + table = "hermes_risk_reports" + if not _has_column(table, "tenant_id"): + return + _drop_index_if_exists(table, "ix_hermes_risk_reports_tenant_status") + _drop_constraint_if_exists(table, "fk_hermes_risk_reports_tenant_log", "foreignkey") + _drop_constraint_if_exists(table, "fk_hermes_risk_reports_tenant_claim", "foreignkey") + _drop_constraint_if_exists(table, "uq_hermes_risk_reports_tenant_id", "unique") + _drop_constraint_if_exists(table, "fk_hermes_risk_reports_tenant", "foreignkey") + if _has_table("expense_claims"): + op.create_foreign_key( + "hermes_risk_reports_claim_id_fkey", + table, + "expense_claims", + ["claim_id"], + ["id"], + ondelete="CASCADE", + ) + if _has_table("hermes_task_execution_logs"): + op.create_foreign_key( + "hermes_risk_reports_execution_log_id_fkey", + table, + "hermes_task_execution_logs", + ["execution_log_id"], + ["id"], + ondelete="CASCADE", + ) + op.drop_column(table, "tenant_id") + + +def _downgrade_task_tables() -> None: + config_table = "hermes_task_configs" + log_table = "hermes_task_execution_logs" + if _has_column(log_table, "tenant_id"): + _drop_index_if_exists(log_table, "ix_hermes_task_logs_tenant_started") + _drop_constraint_if_exists( + log_table, + "fk_hermes_task_logs_tenant_config", + "foreignkey", + ) + _drop_constraint_if_exists( + log_table, + "uq_hermes_task_execution_logs_tenant_id", + "unique", + ) + _drop_constraint_if_exists( + log_table, + "fk_hermes_task_execution_logs_tenant", + "foreignkey", + ) + if _has_table(config_table): + op.create_foreign_key( + "hermes_task_execution_logs_config_id_fkey", + log_table, + config_table, + ["config_id"], + ["id"], + ) + op.drop_column(log_table, "tenant_id") + if _has_column(config_table, "tenant_id"): + _drop_index_if_exists(config_table, "ix_hermes_task_configs_tenant_enabled") + _drop_constraint_if_exists( + config_table, + "uq_hermes_task_configs_tenant_id", + "unique", + ) + _drop_constraint_if_exists( + config_table, + "fk_hermes_task_configs_tenant", + "foreignkey", + ) + op.drop_column(config_table, "tenant_id") + + +def _downgrade_profile_snapshots() -> None: + table = "employee_behavior_profile_snapshots" + if not _has_column(table, "tenant_id"): + return + _drop_index_if_exists(table, "ix_employee_behavior_profile_latest") + _drop_constraint_if_exists( + table, + "fk_employee_behavior_profiles_tenant_employee", + "foreignkey", + ) + _drop_constraint_if_exists( + table, + "uq_employee_behavior_profiles_tenant_id", + "unique", + ) + _drop_constraint_if_exists( + table, + "fk_employee_behavior_profiles_tenant", + "foreignkey", + ) + op.drop_column(table, "tenant_id") + op.create_index( + "ix_employee_behavior_profile_latest", + table, + [ + "subject_id", + "profile_type", + "window_days", + "expense_type_scope", + "calculated_at", + ], + ) diff --git a/server/scripts/backfill_standard_adjustment_savings.py b/server/scripts/backfill_standard_adjustment_savings.py new file mode 100644 index 0000000..0d2f038 --- /dev/null +++ b/server/scripts/backfill_standard_adjustment_savings.py @@ -0,0 +1,398 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import os +import sys +import uuid +from collections import Counter +from dataclasses import asdict +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +from alembic.config import Config +from alembic.script import ScriptDirectory +from sqlalchemy import create_engine, text +from sqlalchemy.engine import Connection +from sqlalchemy.exc import SQLAlchemyError +from sqlalchemy.orm import Session +from sqlalchemy.pool import NullPool + +SERVER_DIR = Path(__file__).resolve().parents[1] +SRC_DIR = SERVER_DIR / "src" +if str(SRC_DIR) not in sys.path: + sys.path.insert(0, str(SRC_DIR)) + +from app.cli.savings_standard_adjustment_backfill import ( # noqa: E402 + DEFAULT_BATCH_SIZE, + MAX_BATCH_SIZE, + StandardAdjustmentBackfillCursor, + StandardAdjustmentSavingsBackfillService, +) +from app.db.maintenance_database_target import ( # noqa: E402 + MaintenanceDatabaseTargetError, + validate_maintenance_database_target, +) +from app.db.migration_preflight import ( # noqa: E402 + MigrationPreflightError, + validate_migration_state, +) + +REQUIRED_ALEMBIC_REVISION = "20260716_0015" +EXIT_SAFETY = 3 +EXIT_LOCKED = 4 +EXIT_RUNTIME = 6 + + +class BackfillCommandError(RuntimeError): + def __init__(self, message: str, *, code: str, exit_code: int) -> None: + super().__init__(message) + self.code = code + self.exit_code = exit_code + + +def revision_contains_required( + current_revision: str | None, + required_revision: str = REQUIRED_ALEMBIC_REVISION, +) -> bool: + """确认当前迁移沿 down_revision 链包含回填所需的数据契约。""" + + current = str(current_revision or "").strip() + required = str(required_revision or "").strip() + if not current or not required: + return False + + config = Config(str(SERVER_DIR / "alembic.ini")) + script_directory = ScriptDirectory.from_config(config) + pending = [current] + visited: set[str] = set() + while pending: + revision_id = pending.pop() + if revision_id in visited: + continue + if revision_id == required: + return True + visited.add(revision_id) + revision = script_directory.get_revision(revision_id) + if revision is None: + return False + down_revision = revision.down_revision + if isinstance(down_revision, str): + pending.append(down_revision) + elif down_revision: + pending.extend(str(item) for item in down_revision) + return False + + +def parse_timestamp(value: str) -> datetime: + normalized = str(value or "").strip() + if normalized.endswith("Z"): + normalized = f"{normalized[:-1]}+00:00" + try: + parsed = datetime.fromisoformat(normalized) + except ValueError as exc: + raise argparse.ArgumentTypeError("必须是带时区的 ISO 8601 时间") from exc + if parsed.tzinfo is None or parsed.utcoffset() is None: + raise argparse.ArgumentTypeError("时间必须显式包含时区") + return parsed.astimezone(UTC) + + +def positive_int(value: str) -> int: + try: + parsed = int(value) + except ValueError as exc: + raise argparse.ArgumentTypeError("必须是正整数") from exc + if parsed < 1: + raise argparse.ArgumentTypeError("必须是正整数") + return parsed + + +def batch_size(value: str) -> int: + parsed = positive_int(value) + if parsed > MAX_BATCH_SIZE: + raise argparse.ArgumentTypeError(f"不能超过 {MAX_BATCH_SIZE}") + return parsed + + +def non_empty_text(value: str) -> str: + normalized = str(value or "").strip() + if not normalized: + raise argparse.ArgumentTypeError("不能为空") + return normalized + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + description="按服务端历史证据回填标准调整节省机会；默认只读预览。", + ) + mode = parser.add_mutually_exclusive_group() + mode.add_argument("--dry-run", action="store_true", help="只读预览（默认）。") + mode.add_argument("--apply", action="store_true", help="按批写入节省机会及证据链。") + parser.add_argument("--tenant-id", required=True, type=non_empty_text) + parser.add_argument("--created-before", required=True, type=parse_timestamp) + parser.add_argument("--batch-size", type=batch_size, default=DEFAULT_BATCH_SIZE) + parser.add_argument("--max-claims", type=positive_int) + parser.add_argument("--sample-limit", type=positive_int, default=20) + parser.add_argument("--expected-host", required=True) + parser.add_argument("--expected-database", required=True) + parser.add_argument( + "--confirm-target", + help="apply 时必须精确等于解析后的 host:port/database。", + ) + parser.add_argument("--allow-non-disposable-target", action="store_true") + return parser + + +def _cursor_payload(cursor: StandardAdjustmentBackfillCursor | None) -> dict[str, str] | None: + if cursor is None: + return None + return {"created_at": _isoformat(cursor.created_at), "claim_id": cursor.claim_id} + + +def _item_payload(item: Any) -> dict[str, Any]: + payload = asdict(item) + payload["disposition"] = item.disposition.value + for field in ("original_amount", "target_amount", "saving_amount"): + if payload[field] is not None: + payload[field] = str(payload[field]) + return payload + + +def _base_summary(args: argparse.Namespace, *, target: Any, revision: str) -> dict[str, Any]: + return { + "mode": "apply" if args.apply else "dry-run", + "database": { + "target": target.exact_target, + "url": target.sanitized_url, + "revision": revision, + }, + "tenant_id": args.tenant_id, + "created_before": _isoformat(args.created_before), + "claims_inspected": 0, + "flags_inspected": 0, + "eligible": 0, + "created": 0, + "replayed": 0, + "skipped": 0, + "reasons": {}, + "batches": 0, + "limited": False, + "last_cursor": None, + "samples": [], + } + + +def _page_size(configured: int, remaining: int | None) -> int: + return configured if remaining is None else min(configured, remaining) + + +def _merge_page(summary: dict[str, Any], page: Any, *, sample_limit: int) -> None: + summary["batches"] += 1 + summary["claims_inspected"] += page.claims_inspected + summary["flags_inspected"] += page.flags_inspected + for field in ("eligible", "created", "replayed", "skipped"): + if hasattr(page, field): + summary[field] += int(getattr(page, field)) + reasons = Counter(summary["reasons"]) + reasons.update(page.reasons) + summary["reasons"] = dict(sorted(reasons.items())) + available = max(0, sample_limit - len(summary["samples"])) + summary["samples"].extend(_item_payload(item) for item in page.items[:available]) + summary["last_cursor"] = _cursor_payload(page.next_cursor) + + +def _preview(session: Session, args: argparse.Namespace, summary: dict[str, Any]) -> None: + service = StandardAdjustmentSavingsBackfillService( + session, + tenant_id=args.tenant_id, + created_before=args.created_before, + ) + cursor = None + remaining = args.max_claims + last_has_more = False + while remaining is None or remaining > 0: + page = service.preview( + batch_size=_page_size(args.batch_size, remaining), + after=cursor, + ) + if page.claims_inspected == 0: + break + _merge_page(summary, page, sample_limit=args.sample_limit) + cursor = page.next_cursor + last_has_more = page.has_more + if remaining is not None: + remaining -= page.claims_inspected + if not page.has_more: + break + summary["limited"] = bool(remaining == 0 and last_has_more) + + +def _acquire_lock(connection: Connection, tenant_id: str) -> str: + lock_name = f"savings-standard-adjustment-backfill:{tenant_id}" + acquired = connection.scalar( + text("SELECT pg_try_advisory_lock(hashtextextended(:name, 0))"), + {"name": lock_name}, + ) + connection.commit() + if not acquired: + raise BackfillCommandError( + "同一租户已有标准调整 Savings 回填正在运行。", + code="advisory_lock_unavailable", + exit_code=EXIT_LOCKED, + ) + return lock_name + + +def _release_lock(connection: Connection, lock_name: str) -> None: + if connection.in_transaction(): + connection.rollback() + connection.execute( + text("SELECT pg_advisory_unlock(hashtextextended(:name, 0))"), + {"name": lock_name}, + ) + connection.commit() + + +def _apply(connection: Connection, args: argparse.Namespace, summary: dict[str, Any]) -> None: + lock_name = _acquire_lock(connection, args.tenant_id) + run_id = f"savings-adjustment-{uuid.uuid4().hex}" + summary["run_id"] = run_id + cursor = None + remaining = args.max_claims + last_has_more = False + try: + with Session(bind=connection, autoflush=False, expire_on_commit=False) as session: + service = StandardAdjustmentSavingsBackfillService( + session, + tenant_id=args.tenant_id, + created_before=args.created_before, + ) + while remaining is None or remaining > 0: + session.execute(text("SET LOCAL lock_timeout = '5s'")) + result = service.apply_batch( + run_id=run_id, + batch_size=_page_size(args.batch_size, remaining), + after=cursor, + ) + if result.claims_inspected == 0: + session.rollback() + break + session.commit() + _merge_page(summary, result, sample_limit=args.sample_limit) + cursor = result.next_cursor + last_has_more = result.has_more + if remaining is not None: + remaining -= result.claims_inspected + if not result.has_more: + break + summary["limited"] = bool(remaining == 0 and last_has_more) + finally: + _release_lock(connection, lock_name) + + +def run(args: argparse.Namespace) -> dict[str, Any]: + if args.apply and not str(args.confirm_target or "").strip(): + raise BackfillCommandError( + "--apply 必须提供 --confirm-target 精确确认数据库目标。", + code="confirm_target_required", + exit_code=EXIT_SAFETY, + ) + database_url = os.environ.get("DATABASE_URL", "") + target = validate_maintenance_database_target( + database_url, + expected_host=args.expected_host, + expected_database=args.expected_database, + apply=args.apply, + allow_non_disposable=args.allow_non_disposable_target, + confirm_target=args.confirm_target, + ) + engine = create_engine(database_url, pool_pre_ping=True, poolclass=NullPool) + try: + with engine.connect() as connection: + database = str(connection.scalar(text("SELECT current_database()")) or "") + if database != target.database: + raise BackfillCommandError( + "连接后的数据库名与 DATABASE_URL 不一致。", + code="connected_database_mismatch", + exit_code=EXIT_SAFETY, + ) + state = validate_migration_state(connection) + if not revision_contains_required(state.revision): + raise BackfillCommandError( + f"数据库迁移链必须包含 {REQUIRED_ALEMBIC_REVISION}，" + f"实际为 {state.revision or 'unversioned/base'}。", + code="migration_revision_mismatch", + exit_code=EXIT_SAFETY, + ) + connection.rollback() + summary = _base_summary(args, target=target, revision=state.revision) + with Session(bind=connection, autoflush=False) as session: + _preview(session, args, summary) + session.rollback() + if not args.apply: + summary["would_create"] = summary["eligible"] + return summary + + preview = dict(summary) + summary.update( + claims_inspected=0, + flags_inspected=0, + eligible=0, + created=0, + replayed=0, + skipped=0, + reasons={}, + batches=0, + limited=False, + last_cursor=None, + samples=[], + preview=preview, + ) + _apply(connection, args, summary) + return summary + finally: + engine.dispose() + + +def _isoformat(value: datetime) -> str: + normalized = value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + return normalized.isoformat().replace("+00:00", "Z") + + +def _error_payload(exc: Exception, *, code: str) -> dict[str, str]: + return {"status": "error", "code": code, "message": str(exc)} + + +def main(argv: list[str] | None = None) -> int: + args = build_parser().parse_args(argv) + try: + payload = run(args) + except BackfillCommandError as exc: + print(json.dumps(_error_payload(exc, code=exc.code), ensure_ascii=False), file=sys.stderr) + return exc.exit_code + except MaintenanceDatabaseTargetError as exc: + print(json.dumps(_error_payload(exc, code=exc.code), ensure_ascii=False), file=sys.stderr) + return EXIT_SAFETY + except MigrationPreflightError as exc: + print( + json.dumps( + _error_payload(exc, code="migration_preflight_failed"), + ensure_ascii=False, + ), + file=sys.stderr, + ) + return EXIT_SAFETY + except (OSError, SQLAlchemyError, ValueError, RuntimeError) as exc: + print( + json.dumps(_error_payload(exc, code="backfill_runtime_error"), ensure_ascii=False), + file=sys.stderr, + ) + return EXIT_RUNTIME + print(json.dumps(payload, ensure_ascii=False, indent=2, default=str)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/server/src/app/api/deps.py b/server/src/app/api/deps.py index 7a59886..43344a8 100644 --- a/server/src/app/api/deps.py +++ b/server/src/app/api/deps.py @@ -24,7 +24,7 @@ class CurrentUserContext: name: str role_codes: list[str] is_admin: bool - tenant_id: str = "default" + tenant_id: str department_name: str = "" department_id: str = "" cost_center: str = "" diff --git a/server/src/app/api/v1/endpoints/agent_asset_releases.py b/server/src/app/api/v1/endpoints/agent_asset_releases.py new file mode 100644 index 0000000..99da266 --- /dev/null +++ b/server/src/app/api/v1/endpoints/agent_asset_releases.py @@ -0,0 +1,308 @@ +from __future__ import annotations + +from typing import Annotated, NoReturn + +from fastapi import APIRouter, Depends, Header, HTTPException, status +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext, get_db, require_rule_reviewer_user +from app.schemas.agent_asset_release import ( + AgentAssetReleaseMonitorRead, + AgentAssetReleaseMonitorTriggerWrite, + AgentAssetReleaseReviewLabelRead, + AgentAssetReleaseReviewLabelWrite, + AgentAssetReleaseReviewQueueRead, + AgentAssetReleaseRollbackWrite, + AgentAssetReleaseServingPlanRead, + AgentAssetReleaseStartWrite, + AgentAssetReleaseStateRead, +) +from app.services.agent_asset_access import stable_user_principal +from app.services.agent_asset_release_guard import ( + AgentAssetReleaseGuardService, + ReleaseGuardPolicy, +) +from app.services.agent_asset_release_monitor import AgentAssetReleaseMonitor +from app.services.agent_asset_release_monitor_auth import ( + ReleaseMonitorAuthenticationError, + ReleaseMonitorConfigurationError, + require_release_monitor_signature, +) +from app.services.agent_asset_release_review import AgentAssetReleaseReviewService + +router = APIRouter(prefix="/agent-assets") +DbSession = Annotated[Session, Depends(get_db)] +RuleReviewerUser = Annotated[CurrentUserContext, Depends(require_rule_reviewer_user)] +RequestIdHeader = Annotated[ + str | None, + Header(description="外部请求 ID，用于串联审计日志和上游调用链。"), +] +ReleaseMonitorTimestampHeader = Annotated[ + str | None, + Header(alias="X-Release-Monitor-Timestamp"), +] +ReleaseMonitorSignatureHeader = Annotated[ + str | None, + Header(alias="X-Release-Monitor-Signature"), +] + + +def _handle_error(exc: Exception) -> NoReturn: + if isinstance(exc, LookupError): + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc + if isinstance(exc, (PermissionError, ValueError)): + raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc + raise exc + + +def _actor(user: CurrentUserContext) -> str: + return stable_user_principal(user) + + +def _state_payload(value: dict) -> AgentAssetReleaseStateRead: + if not value: + return AgentAssetReleaseStateRead() + allowed = set(AgentAssetReleaseStateRead.model_fields) + return AgentAssetReleaseStateRead.model_validate( + {key: item for key, item in value.items() if key in allowed} + ) + + +@router.get( + "/{asset_id}/release", + response_model=AgentAssetReleaseStateRead, + summary="查询 Agent 资产分阶段发布状态", +) +def get_agent_asset_release( + asset_id: str, + current_user: RuleReviewerUser, + db: DbSession, +) -> AgentAssetReleaseStateRead: + try: + return _state_payload( + AgentAssetReleaseGuardService(db).get_state( + asset_id, + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + ) + ) + except Exception as exc: + _handle_error(exc) + + +@router.get( + "/{asset_id}/release/serving-plan", + response_model=AgentAssetReleaseServingPlanRead, + summary="查询运行时版本路由计划", +) +def get_agent_asset_release_serving_plan( + asset_id: str, + current_user: RuleReviewerUser, + db: DbSession, +) -> AgentAssetReleaseServingPlanRead: + try: + return AgentAssetReleaseServingPlanRead.model_validate( + AgentAssetReleaseGuardService(db).get_serving_plan( + asset_id, + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + ) + ) + except Exception as exc: + _handle_error(exc) + + +@router.post( + "/{asset_id}/release/shadow", + response_model=AgentAssetReleaseStateRead, + summary="启动受控影子发布", +) +def start_agent_asset_shadow_release( + asset_id: str, + payload: AgentAssetReleaseStartWrite, + current_user: RuleReviewerUser, + db: DbSession, + x_request_id: RequestIdHeader = None, +) -> AgentAssetReleaseStateRead: + try: + policy = ReleaseGuardPolicy(**payload.policy.model_dump()) + value = AgentAssetReleaseGuardService(db).start_shadow( + asset_id, + payload.candidate_version, + actor=_actor(current_user), + policy=policy, + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + request_id=x_request_id, + ) + return _state_payload(value) + except Exception as exc: + _handle_error(exc) + + +@router.post( + "/{asset_id}/release/evaluations", + response_model=AgentAssetReleaseMonitorRead, + summary="聚合真实发布遥测并在越界时自动回滚", +) +def evaluate_agent_asset_release( + asset_id: str, + payload: AgentAssetReleaseMonitorTriggerWrite, + current_user: RuleReviewerUser, + db: DbSession, + x_release_monitor_timestamp: ReleaseMonitorTimestampHeader = None, + x_release_monitor_signature: ReleaseMonitorSignatureHeader = None, +) -> AgentAssetReleaseMonitorRead: + try: + service = AgentAssetReleaseGuardService(db) + state = service.get_state( + asset_id, + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + ) + require_release_monitor_signature( + timestamp=x_release_monitor_timestamp, + signature=x_release_monitor_signature, + tenant_id=current_user.tenant_id, + asset_id=asset_id, + release_id=str(state.get("release_id") or ""), + stage=str(state.get("stage") or ""), + payload=payload.model_dump(mode="json", exclude_unset=True), + ) + value = AgentAssetReleaseMonitor(db).evaluate_current( + asset_id=asset_id, + actor=_actor(current_user), + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + ) + return AgentAssetReleaseMonitorRead.model_validate(value) + except ReleaseMonitorConfigurationError as exc: + raise HTTPException( + status_code=status.HTTP_503_SERVICE_UNAVAILABLE, + detail=str(exc), + ) from exc + except ReleaseMonitorAuthenticationError as exc: + raise HTTPException( + status_code=status.HTTP_401_UNAUTHORIZED, + detail=str(exc), + ) from exc + except Exception as exc: + _handle_error(exc) + + +@router.get( + "/{asset_id}/release/review-queue", + response_model=AgentAssetReleaseReviewQueueRead, + summary="查询当前发布的去敏人工复核队列", +) +def list_agent_asset_release_review_queue( + asset_id: str, + current_user: RuleReviewerUser, + db: DbSession, + limit: int = 50, +) -> AgentAssetReleaseReviewQueueRead: + try: + value = AgentAssetReleaseReviewService(db).list_pending( + tenant_id=current_user.tenant_id, + asset_id=asset_id, + limit=limit, + ) + return AgentAssetReleaseReviewQueueRead.model_validate(value) + except Exception as exc: + _handle_error(exc) + + +@router.post( + "/{asset_id}/release/review-queue/{observation_id}/labels", + response_model=AgentAssetReleaseReviewLabelRead, + summary="提交类型化发布复核结论并重新聚合", +) +def label_agent_asset_release_observation( + asset_id: str, + observation_id: str, + payload: AgentAssetReleaseReviewLabelWrite, + current_user: RuleReviewerUser, + db: DbSession, + x_request_id: RequestIdHeader = None, +) -> AgentAssetReleaseReviewLabelRead: + try: + request_id = str(x_request_id or "").strip() + if not request_id: + raise ValueError("X-Request-Id is required for release review labels.") + label = AgentAssetReleaseReviewService(db).record_label( + tenant_id=current_user.tenant_id, + asset_id=asset_id, + observation_id=observation_id, + label=payload.label, + actor_id=_actor(current_user), + request_id=request_id, + ) + db.commit() + monitor = AgentAssetReleaseMonitor(db).evaluate_current( + asset_id=asset_id, + actor=_actor(current_user), + tenant_id=current_user.tenant_id, + ) + return AgentAssetReleaseReviewLabelRead.model_validate( + { + "label_id": label.id, + "observation_id": label.observation_id, + "label": label.label, + "monitor": monitor, + } + ) + except Exception as exc: + db.rollback() + _handle_error(exc) + + +@router.post( + "/{asset_id}/release/promote", + response_model=AgentAssetReleaseStateRead, + summary="将通过评测的版本晋级到下一阶段", +) +def promote_agent_asset_release( + asset_id: str, + current_user: RuleReviewerUser, + db: DbSession, + x_request_id: RequestIdHeader = None, +) -> AgentAssetReleaseStateRead: + try: + return _state_payload( + AgentAssetReleaseGuardService(db).promote( + asset_id, + actor=_actor(current_user), + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + request_id=x_request_id, + ) + ) + except Exception as exc: + _handle_error(exc) + + +@router.post( + "/{asset_id}/release/rollback", + response_model=AgentAssetReleaseStateRead, + summary="显式回滚到发布前版本", +) +def rollback_agent_asset_release( + asset_id: str, + payload: AgentAssetReleaseRollbackWrite, + current_user: RuleReviewerUser, + db: DbSession, + x_request_id: RequestIdHeader = None, +) -> AgentAssetReleaseStateRead: + try: + return _state_payload( + AgentAssetReleaseGuardService(db).rollback( + asset_id, + actor=_actor(current_user), + reason=payload.reason, + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + request_id=x_request_id, + ) + ) + except Exception as exc: + _handle_error(exc) diff --git a/server/src/app/api/v1/endpoints/agent_asset_risk_rules.py b/server/src/app/api/v1/endpoints/agent_asset_risk_rules.py index b15c2f5..53116c1 100644 --- a/server/src/app/api/v1/endpoints/agent_asset_risk_rules.py +++ b/server/src/app/api/v1/endpoints/agent_asset_risk_rules.py @@ -2,35 +2,52 @@ from __future__ import annotations from typing import Annotated, NoReturn -from fastapi import APIRouter, Depends, Header, HTTPException, Query, status +from fastapi import APIRouter, BackgroundTasks, Depends, Header, HTTPException, Query, status from sqlalchemy.orm import Session from app.api.deps import ( CurrentUserContext, get_current_user, get_db, + require_platform_admin_user, require_rule_editor_user, require_rule_reviewer_user, ) +from app.db.session import get_session_factory from app.schemas.agent_asset import ( AgentAssetRead, AgentAssetRiskRuleDraftUpdate, + AgentAssetRiskRuleEnabledUpdate, AgentAssetRiskRuleFeedbackCreate, AgentAssetRiskRuleFeedbackRead, + AgentAssetRiskRuleGenerateRequest, + AgentAssetRiskRuleLatestTestSummary, + AgentAssetRiskRuleLevelUpdate, AgentAssetRiskRuleRegenerateRequest, + AgentAssetRiskRuleReportRequest, + AgentAssetRiskRuleReturnRequest, AgentAssetRiskRuleRevisionCreate, + AgentAssetRiskRuleSampleTestRequest, + AgentAssetRiskRuleScenarioTestRequest, + AgentAssetRiskRuleSimulationRead, + AgentAssetRiskRuleSimulationRequest, AgentAssetRiskRuleTemplateGroupRead, + AgentAssetRiskRuleTestRunRead, + AgentAssetRuleJsonRead, + AgentAssetRuleJsonWrite, ) +from app.services.agent_asset_access import stable_user_principal from app.services.agent_asset_risk_rule_regeneration import AgentAssetRiskRuleRegenerationService from app.services.agent_asset_risk_rule_revision import AgentAssetRiskRuleRevisionService from app.services.agent_assets import AgentAssetService +from app.services.risk_rule_generation_jobs import RiskRuleGenerationJobService from app.services.risk_rule_template_catalog import list_risk_rule_template_groups router = APIRouter(prefix="/agent-assets") DbSession = Annotated[Session, Depends(get_db)] ActorHeader = Annotated[ str | None, - Header(description="审计操作人。未传时使用当前登录用户名称。"), + Header(description="兼容旧客户端；审计主体始终以登录会话中的稳定身份为准。"), ] RequestIdHeader = Annotated[ str | None, @@ -38,6 +55,7 @@ RequestIdHeader = Annotated[ ] RuleEditorUser = Annotated[CurrentUserContext, Depends(require_rule_editor_user)] RuleReviewerUser = Annotated[CurrentUserContext, Depends(require_rule_reviewer_user)] +PlatformAdminUser = Annotated[CurrentUserContext, Depends(require_platform_admin_user)] CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] @@ -50,21 +68,49 @@ def _handle_asset_error(exc: Exception) -> NoReturn: def _actor_name(current_user: CurrentUserContext, x_actor: str | None) -> str: - return (x_actor or current_user.name or current_user.username or "system").strip() or "system" + del x_actor + return stable_user_principal(current_user) -def _read_asset(db: Session, asset_id: str) -> AgentAssetRead: - asset = AgentAssetService(db).get_asset(asset_id) +def _read_asset( + db: Session, + asset_id: str, + current_user: CurrentUserContext, +) -> AgentAssetRead: + asset = AgentAssetService(db, current_user=current_user).get_asset(asset_id) if asset is None: raise LookupError("Asset not found") return asset +def _complete_risk_rule_generation_task( + asset_id: str, + payload: dict, + actor: str, + request_id: str | None, + tenant_id: str, +) -> None: + db = get_session_factory()() + try: + body = AgentAssetRiskRuleGenerateRequest.model_validate(payload) + RiskRuleGenerationJobService(db).complete_rule_asset_generation( + asset_id, + body, + tenant_id=tenant_id, + actor=actor, + request_id=request_id, + ) + finally: + db.close() + + @router.get( "/risk-rules/templates", response_model=list[AgentAssetRiskRuleTemplateGroupRead], summary="查询常见费控风险规则模板", - description="返回模板分组、默认自然语言、字段清单和 DSL 样例；模板只用于预填，不绕过通用生成链路。", + description=( + "返回模板分组、默认自然语言、字段清单和 DSL 样例；模板只用于预填，不绕过通用生成链路。" + ), ) def list_risk_rule_templates(_: CurrentUser) -> list[AgentAssetRiskRuleTemplateGroupRead]: return list_risk_rule_template_groups() @@ -85,13 +131,13 @@ def update_risk_rule_draft( x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - AgentAssetRiskRuleRevisionService(db).update_unpublished_draft( + AgentAssetRiskRuleRevisionService(db, current_user=current_user).update_unpublished_draft( asset_id, payload, actor=_actor_name(current_user, x_actor), request_id=x_request_id, ) - return _read_asset(db, asset_id) + return _read_asset(db, asset_id, current_user) except Exception as exc: _handle_asset_error(exc) @@ -112,13 +158,13 @@ def create_risk_rule_revision( x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - AgentAssetRiskRuleRevisionService(db).create_revision_draft( + AgentAssetRiskRuleRevisionService(db, current_user=current_user).create_revision_draft( asset_id, payload, actor=_actor_name(current_user, x_actor), request_id=x_request_id, ) - return _read_asset(db, asset_id) + return _read_asset(db, asset_id, current_user) except Exception as exc: _handle_asset_error(exc) @@ -138,14 +184,16 @@ def regenerate_risk_rule( x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - AgentAssetRiskRuleRegenerationService(db).regenerate( + AgentAssetRiskRuleRegenerationService( + db, current_user=current_user + ).regenerate( asset_id, payload, tenant_id=current_user.tenant_id, actor=_actor_name(current_user, x_actor), request_id=x_request_id, ) - return _read_asset(db, asset_id) + return _read_asset(db, asset_id, current_user) except Exception as exc: _handle_asset_error(exc) @@ -166,7 +214,7 @@ def create_risk_rule_feedback( x_request_id: RequestIdHeader = None, ) -> AgentAssetRiskRuleFeedbackRead: try: - return AgentAssetService(db).create_risk_rule_feedback( + return AgentAssetService(db, current_user=current_user).create_risk_rule_feedback( asset_id, payload, actor=_actor_name(current_user, x_actor), @@ -184,14 +232,14 @@ def create_risk_rule_feedback( ) def list_risk_rule_feedback( asset_id: str, - _: RuleReviewerUser, + current_user: RuleReviewerUser, db: DbSession, version: Annotated[str | None, Query(max_length=30)] = None, status_value: Annotated[str | None, Query(alias="status", max_length=30)] = None, limit: Annotated[int, Query(ge=1, le=200)] = 50, ) -> list[AgentAssetRiskRuleFeedbackRead]: try: - return AgentAssetService(db).list_risk_rule_feedback( + return AgentAssetService(db, current_user=current_user).list_risk_rule_feedback( asset_id, version=version, status=status_value, @@ -199,3 +247,304 @@ def list_risk_rule_feedback( ) except Exception as exc: _handle_asset_error(exc) + + +@router.get( + "/{asset_id}/rule-json", + response_model=AgentAssetRuleJsonRead, + summary="读取风险规则 JSON", + description="读取 JSON 风险规则资产绑定的规则文件内容。", +) +def get_agent_asset_rule_json( + asset_id: str, + current_user: CurrentUser, + db: DbSession, +) -> AgentAssetRuleJsonRead: + try: + return AgentAssetService(db, current_user=current_user).read_rule_json(asset_id) + except Exception as exc: + _handle_asset_error(exc) + + +@router.put( + "/{asset_id}/rule-json", + response_model=AgentAssetRuleJsonRead, + summary="保存风险规则 JSON", + description="保存 JSON 风险规则资产绑定的规则文件内容，并写入审计日志。", +) +def save_agent_asset_rule_json( + asset_id: str, + payload: AgentAssetRuleJsonWrite, + current_user: RuleEditorUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRuleJsonRead: + try: + return AgentAssetService(db, current_user=current_user).write_rule_json( + asset_id, + body=payload, + actor=_actor_name(current_user, x_actor), + request_id=x_request_id, + ) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/risk-rules/generate", + response_model=AgentAssetRead, + status_code=status.HTTP_201_CREATED, + summary="根据自然语言新建风险规则草稿", + description=( + "根据业务域、自然语言描述和风险评分模型生成 JSON 风险规则，并保存为待上线草稿资产。" + ), +) +def generate_agent_asset_risk_rule( + payload: AgentAssetRiskRuleGenerateRequest, + background_tasks: BackgroundTasks, + current_user: RuleReviewerUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRead: + try: + actor = _actor_name(current_user, x_actor) + asset_id = RiskRuleGenerationJobService(db).enqueue_rule_asset_generation( + payload, + tenant_id=current_user.tenant_id, + actor=actor, + request_id=x_request_id, + ) + background_tasks.add_task( + _complete_risk_rule_generation_task, + asset_id, + payload.model_dump(mode="json"), + actor, + x_request_id, + current_user.tenant_id, + ) + return _read_asset(db, asset_id, current_user) + except Exception as exc: + _handle_asset_error(exc) + + +@router.get( + "/{asset_id}/risk-rule-tests/latest", + response_model=AgentAssetRiskRuleLatestTestSummary, + summary="读取风险规则最近测试摘要", + description="返回当前风险规则工作版本最近一次样例测试、场景试运行和测试报告。", +) +def get_agent_asset_risk_rule_latest_test( + asset_id: str, + current_user: CurrentUser, + db: DbSession, +) -> AgentAssetRiskRuleLatestTestSummary: + try: + return AgentAssetService( + db, current_user=current_user + ).get_latest_risk_rule_test_summary(asset_id) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/risk-rule-tests/simulate", + response_model=AgentAssetRiskRuleSimulationRead, + summary="执行风险规则对话仿真", + description="基于临时对话输入和附件元信息执行风险识别，不创建业务单据，不写入测试记录。", +) +def simulate_agent_asset_risk_rule_test( + asset_id: str, + payload: AgentAssetRiskRuleSimulationRequest, + current_user: PlatformAdminUser, + db: DbSession, +) -> AgentAssetRiskRuleSimulationRead: + try: + return AgentAssetService( + db, current_user=current_user + ).simulate_risk_rule_message(asset_id, payload) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/risk-rule-tests/sample", + response_model=AgentAssetRiskRuleTestRunRead, + summary="执行风险规则快速样例测试", + description="使用人工样例或系统默认样例执行当前 JSON 风险规则，不依赖大模型判断结果。", +) +def run_agent_asset_risk_rule_sample_test( + asset_id: str, + payload: AgentAssetRiskRuleSampleTestRequest, + current_user: PlatformAdminUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRiskRuleTestRunRead: + try: + return AgentAssetService(db, current_user=current_user).run_risk_rule_sample_test( + asset_id, + payload, + actor=_actor_name(current_user, x_actor), + request_id=x_request_id, + ) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/risk-rule-tests/scenario", + response_model=AgentAssetRiskRuleTestRunRead, + summary="执行风险规则真实场景试运行", + description="按测试意图读取真实业务样本并沙盒执行风险规则，不写回业务单据。", +) +def run_agent_asset_risk_rule_scenario_test( + asset_id: str, + payload: AgentAssetRiskRuleScenarioTestRequest, + current_user: PlatformAdminUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRiskRuleTestRunRead: + try: + return AgentAssetService( + db, current_user=current_user + ).run_risk_rule_scenario_test( + asset_id, + payload, + actor=_actor_name(current_user, x_actor), + request_id=x_request_id, + ) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/risk-rule-tests/report", + response_model=AgentAssetRiskRuleTestRunRead, + summary="确认风险规则测试报告", + description="在样例测试和真实场景试运行通过后，保存当前版本测试通过记录。", +) +def confirm_agent_asset_risk_rule_test_report( + asset_id: str, + payload: AgentAssetRiskRuleReportRequest, + current_user: PlatformAdminUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRiskRuleTestRunRead: + try: + return AgentAssetService( + db, current_user=current_user + ).confirm_risk_rule_test_report( + asset_id, + payload, + actor=_actor_name(current_user, x_actor), + request_id=x_request_id, + ) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/risk-rule-enabled", + response_model=AgentAssetRead, + summary="设置风险规则启用状态", + description=( + "高级财务人员可独立启用或停用 JSON 风险规则；停用后即使已上线也不会进入真实业务扫描。" + ), +) +def set_agent_asset_risk_rule_enabled( + asset_id: str, + payload: AgentAssetRiskRuleEnabledUpdate, + current_user: RuleReviewerUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRead: + try: + asset = AgentAssetService(db, current_user=current_user).set_risk_rule_enabled( + asset_id, + enabled=payload.enabled, + actor=_actor_name(current_user, x_actor), + request_id=x_request_id, + ) + return _read_asset(db, asset.id, current_user) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/risk-rule-level", + response_model=AgentAssetRead, + summary="风险规则风险等级已由评分模型接管", + description="风险规则等级和分数由自然语言规则评分模型生成，不再允许人工调整。", +) +def set_agent_asset_risk_rule_level( + asset_id: str, + payload: AgentAssetRiskRuleLevelUpdate, + current_user: RuleEditorUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRead: + try: + del asset_id, payload, current_user, db, x_actor, x_request_id + raise ValueError("风险等级和分数由评分模型自动计算，不能手动修改。") + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/return", + response_model=AgentAssetRiskRuleLatestTestSummary, + summary="回退待审核风险规则", + description="高级财务人员将待审核风险规则回退到草稿，并记录回退原因。", +) +def return_agent_asset_risk_rule( + asset_id: str, + payload: AgentAssetRiskRuleReturnRequest, + current_user: RuleReviewerUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRiskRuleLatestTestSummary: + try: + return AgentAssetService(db, current_user=current_user).return_risk_rule( + asset_id, + note=payload.note, + actor=_actor_name(current_user, x_actor), + request_id=x_request_id, + ) + except Exception as exc: + _handle_asset_error(exc) + + +@router.post( + "/{asset_id}/publish", + response_model=AgentAssetRead, + summary="审核并启动风险规则影子发布", + description=( + "高级财务人员确认测试与 Golden 门禁通过后，将候选规则送入 shadow；" + "该入口不会直接 active，后续必须通过 Canary 质量门禁。" + ), +) +def publish_agent_asset_risk_rule( + asset_id: str, + current_user: RuleReviewerUser, + db: DbSession, + x_actor: ActorHeader = None, + x_request_id: RequestIdHeader = None, +) -> AgentAssetRead: + try: + asset = AgentAssetService(db, current_user=current_user).publish_risk_rule( + asset_id, + actor=_actor_name(current_user, x_actor), + tenant_id=current_user.tenant_id, + allow_global_management=current_user.is_admin, + request_id=x_request_id, + ) + return _read_asset(db, asset.id, current_user) + except Exception as exc: + _handle_asset_error(exc) diff --git a/server/src/app/api/v1/endpoints/agent_assets.py b/server/src/app/api/v1/endpoints/agent_assets.py index aa845c6..dbbb0cd 100644 --- a/server/src/app/api/v1/endpoints/agent_assets.py +++ b/server/src/app/api/v1/endpoints/agent_assets.py @@ -2,7 +2,7 @@ from __future__ import annotations from typing import Annotated -from fastapi import APIRouter, BackgroundTasks, Body, Depends, Header, HTTPException, Query, status +from fastapi import APIRouter, Body, Depends, Header, HTTPException, Query, status from fastapi.responses import FileResponse from sqlalchemy.orm import Session @@ -15,7 +15,6 @@ from app.api.deps import ( require_rule_reviewer_user, ) from app.api.pagination import PageNumber, PageSize, page_payload, wants_page -from app.db.session import get_session_factory from app.schemas.agent_asset import ( AgentAssetCreate, AgentAssetListItem, @@ -25,19 +24,6 @@ from app.schemas.agent_asset import ( AgentAssetRead, AgentAssetReviewCreate, AgentAssetReviewRead, - AgentAssetRiskRuleEnabledUpdate, - AgentAssetRiskRuleGenerateRequest, - AgentAssetRiskRuleLatestTestSummary, - AgentAssetRiskRuleLevelUpdate, - AgentAssetRiskRuleReportRequest, - AgentAssetRiskRuleReturnRequest, - AgentAssetRiskRuleSampleTestRequest, - AgentAssetRiskRuleScenarioTestRequest, - AgentAssetRiskRuleSimulationRead, - AgentAssetRiskRuleSimulationRequest, - AgentAssetRiskRuleTestRunRead, - AgentAssetRuleJsonRead, - AgentAssetRuleJsonWrite, AgentAssetSpreadsheetChangeRecordRead, AgentAssetUpdate, AgentAssetVersionCreate, @@ -49,14 +35,20 @@ from app.schemas.agent_asset import ( GoldenEvalRequest, ) from app.schemas.common import ErrorResponse, PaginatedResponse +from app.services.agent_asset_access import stable_user_principal +from app.services.agent_asset_onlyoffice_security import ( + AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + AgentAssetOnlyOfficeReplayError, + AgentAssetOnlyOfficeSecurityError, + AgentAssetOnlyOfficeValidatedSession, +) from app.services.agent_assets import AgentAssetService -from app.services.risk_rule_generation_jobs import RiskRuleGenerationJobService router = APIRouter(prefix="/agent-assets") DbSession = Annotated[Session, Depends(get_db)] ActorHeader = Annotated[ str | None, - Header(description="审计操作者。未传时回退到请求体中的 owner / reviewer 或 `system`。"), + Header(description="兼容旧客户端；审计主体始终以登录会话中的稳定身份为准。"), ] RequestIdHeader = Annotated[ str | None, @@ -68,6 +60,24 @@ RuleEditorUser = Annotated[CurrentUserContext, Depends(require_rule_editor_user) RuleReviewerUser = Annotated[CurrentUserContext, Depends(require_rule_reviewer_user)] +def _actor(current_user: CurrentUserContext) -> str: + return stable_user_principal(current_user) + + +def _onlyoffice_service_for_session( + db: Session, + session: AgentAssetOnlyOfficeValidatedSession, +) -> AgentAssetService: + machine_user = CurrentUserContext( + username=session.actor, + name="ONLYOFFICE", + role_codes=["manager"], + is_admin=session.resource_scope == "platform", + tenant_id=session.tenant_id, + ) + return AgentAssetService(db, current_user=machine_user) + + def _handle_asset_error(exc: Exception) -> None: if isinstance(exc, (LookupError, FileNotFoundError)): raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc @@ -78,27 +88,6 @@ def _handle_asset_error(exc: Exception) -> None: raise exc -def _complete_risk_rule_generation_task( - asset_id: str, - payload: dict, - actor: str, - request_id: str | None, - tenant_id: str, -) -> None: - db = get_session_factory()() - try: - body = AgentAssetRiskRuleGenerateRequest.model_validate(payload) - RiskRuleGenerationJobService(db).complete_rule_asset_generation( - asset_id, - body, - tenant_id=tenant_id, - actor=actor, - request_id=request_id, - ) - finally: - db.close() - - @router.get( "", response_model=list[AgentAssetListItem] | PaginatedResponse[AgentAssetListItem], @@ -106,6 +95,7 @@ def _complete_risk_rule_generation_task( description="按资产类型、状态、领域和关键字筛选规则、技能、MCP 与任务资产。", ) def list_agent_assets( + current_user: CurrentUser, db: DbSession, asset_type: Annotated[ str | None, @@ -126,7 +116,7 @@ def list_agent_assets( page: PageNumber = None, page_size: PageSize = None, ) -> list[AgentAssetListItem] | PaginatedResponse[AgentAssetListItem]: - service = AgentAssetService(db) + service = AgentAssetService(db, current_user=current_user) if wants_page(page, page_size): return page_payload( service.list_assets_page( @@ -158,204 +148,17 @@ def list_agent_assets( } }, ) -def get_agent_asset(asset_id: str, db: DbSession) -> AgentAssetRead: - asset = AgentAssetService(db).get_asset(asset_id) +def get_agent_asset( + asset_id: str, + current_user: CurrentUser, + db: DbSession, +) -> AgentAssetRead: + asset = AgentAssetService(db, current_user=current_user).get_asset(asset_id) if asset is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Asset not found") return asset -@router.get( - "/{asset_id}/rule-json", - response_model=AgentAssetRuleJsonRead, - summary="读取风险规则 JSON", - description="读取 JSON 风险规则资产绑定的规则文件内容。", -) -def get_agent_asset_rule_json( - asset_id: str, - _: CurrentUser, - db: DbSession, -) -> AgentAssetRuleJsonRead: - try: - return AgentAssetService(db).read_rule_json(asset_id) - except Exception as exc: - _handle_asset_error(exc) - - -@router.get( - "/{asset_id}/risk-rule-tests/latest", - response_model=AgentAssetRiskRuleLatestTestSummary, - summary="读取风险规则最近测试摘要", - description="返回当前风险规则工作版本最近一次样例测试、场景试运行和测试报告。", -) -def get_agent_asset_risk_rule_latest_test( - asset_id: str, - _: CurrentUser, - db: DbSession, -) -> AgentAssetRiskRuleLatestTestSummary: - try: - return AgentAssetService(db).get_latest_risk_rule_test_summary(asset_id) - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/risk-rule-tests/simulate", - response_model=AgentAssetRiskRuleSimulationRead, - summary="执行风险规则对话仿真", - description="基于临时对话输入和附件元信息执行风险识别，不创建业务单据，不写入测试记录。", -) -def simulate_agent_asset_risk_rule_test( - asset_id: str, - payload: AgentAssetRiskRuleSimulationRequest, - _: PlatformAdminUser, - db: DbSession, -) -> AgentAssetRiskRuleSimulationRead: - try: - return AgentAssetService(db).simulate_risk_rule_message(asset_id, payload) - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/risk-rule-tests/sample", - response_model=AgentAssetRiskRuleTestRunRead, - summary="执行风险规则快速样例测试", - description="使用人工样例或系统默认样例执行当前 JSON 风险规则，不依赖大模型判断结果。", -) -def run_agent_asset_risk_rule_sample_test( - asset_id: str, - payload: AgentAssetRiskRuleSampleTestRequest, - current_user: PlatformAdminUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRiskRuleTestRunRead: - try: - return AgentAssetService(db).run_risk_rule_sample_test( - asset_id, - payload, - actor=(x_actor or current_user.name or "system").strip() or "system", - request_id=x_request_id, - ) - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/risk-rule-tests/scenario", - response_model=AgentAssetRiskRuleTestRunRead, - summary="执行风险规则真实场景试运行", - description="按测试意图读取真实业务样本并沙盒执行风险规则，不写回业务单据。", -) -def run_agent_asset_risk_rule_scenario_test( - asset_id: str, - payload: AgentAssetRiskRuleScenarioTestRequest, - current_user: PlatformAdminUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRiskRuleTestRunRead: - try: - return AgentAssetService(db).run_risk_rule_scenario_test( - asset_id, - payload, - actor=(x_actor or current_user.name or "system").strip() or "system", - request_id=x_request_id, - ) - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/risk-rule-tests/report", - response_model=AgentAssetRiskRuleTestRunRead, - summary="确认风险规则测试报告", - description="在样例测试和真实场景试运行通过后，保存当前版本测试通过记录。", -) -def confirm_agent_asset_risk_rule_test_report( - asset_id: str, - payload: AgentAssetRiskRuleReportRequest, - current_user: PlatformAdminUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRiskRuleTestRunRead: - try: - return AgentAssetService(db).confirm_risk_rule_test_report( - asset_id, - payload, - actor=(x_actor or current_user.name or "system").strip() or "system", - request_id=x_request_id, - ) - except Exception as exc: - _handle_asset_error(exc) - - -@router.put( - "/{asset_id}/rule-json", - response_model=AgentAssetRuleJsonRead, - summary="保存风险规则 JSON", - description="保存 JSON 风险规则资产绑定的规则文件内容，并写入审计日志。", -) -def save_agent_asset_rule_json( - asset_id: str, - payload: AgentAssetRuleJsonWrite, - current_user: RuleEditorUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRuleJsonRead: - try: - return AgentAssetService(db).write_rule_json( - asset_id, - body=payload, - actor=(x_actor or current_user.name or "system").strip() or "system", - request_id=x_request_id, - ) - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/risk-rules/generate", - response_model=AgentAssetRead, - status_code=status.HTTP_201_CREATED, - summary="根据自然语言新建风险规则草稿", - description="根据业务域、自然语言描述和风险评分模型生成 JSON 风险规则，并保存为待上线草稿资产。", -) -def generate_agent_asset_risk_rule( - payload: AgentAssetRiskRuleGenerateRequest, - background_tasks: BackgroundTasks, - current_user: RuleReviewerUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRead: - try: - actor = (x_actor or current_user.name or "system").strip() or "system" - asset_id = RiskRuleGenerationJobService(db).enqueue_rule_asset_generation( - payload, - tenant_id=current_user.tenant_id, - actor=actor, - request_id=x_request_id, - ) - background_tasks.add_task( - _complete_risk_rule_generation_task, - asset_id, - payload.model_dump(mode="json"), - actor, - x_request_id, - current_user.tenant_id, - ) - asset = AgentAssetService(db).get_asset(asset_id) - if asset is None: - raise LookupError("Asset not found") - return asset - except Exception as exc: - _handle_asset_error(exc) - - @router.get( "/{asset_id}/spreadsheet/onlyoffice-config", response_model=AgentAssetOnlyOfficeConfigRead, @@ -372,7 +175,9 @@ def get_agent_asset_spreadsheet_onlyoffice_config( ] = None, ) -> AgentAssetOnlyOfficeConfigRead: try: - return AgentAssetService(db).build_rule_spreadsheet_onlyoffice_config( + return AgentAssetService( + db, current_user=current_user + ).build_rule_spreadsheet_onlyoffice_config( asset_id, current_user, version=version, @@ -389,7 +194,7 @@ def get_agent_asset_spreadsheet_onlyoffice_config( ) def get_agent_asset_spreadsheet_content( asset_id: str, - _: CurrentUser, + current_user: CurrentUser, db: DbSession, version: Annotated[ str | None, @@ -397,10 +202,9 @@ def get_agent_asset_spreadsheet_content( ] = None, ) -> FileResponse: try: - file_path, media_type, filename = AgentAssetService(db).get_rule_spreadsheet_content( - asset_id, - version=version, - ) + file_path, media_type, filename = AgentAssetService( + db, current_user=current_user + ).get_rule_spreadsheet_content(asset_id, version=version) except Exception as exc: _handle_asset_error(exc) @@ -426,15 +230,19 @@ def get_agent_asset_spreadsheet_onlyoffice_content( ] = None, ) -> FileResponse: try: - service = AgentAssetService(db) - service.validate_rule_spreadsheet_access_token(asset_id, access_token) + bootstrap_service = AgentAssetService(db) + validated_session = bootstrap_service.validate_rule_spreadsheet_access_token( + asset_id, access_token + ) + service = _onlyoffice_service_for_session(db, validated_session) file_path, media_type, filename = service.get_rule_spreadsheet_content( asset_id, version=version, + validated_session=validated_session, ) except FileNotFoundError as exc: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc - except ValueError as exc: + except AgentAssetOnlyOfficeSecurityError as exc: raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc)) from exc except Exception as exc: _handle_asset_error(exc) @@ -464,11 +272,11 @@ def upload_agent_asset_spreadsheet( x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - return AgentAssetService(db).upload_rule_spreadsheet( + return AgentAssetService(db, current_user=current_user).upload_rule_spreadsheet( asset_id, filename=filename, content=content, - actor=current_user.name, + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -497,11 +305,13 @@ def import_agent_asset_spreadsheet_content( x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - return AgentAssetService(db).import_rule_spreadsheet_content( + return AgentAssetService( + db, current_user=current_user + ).import_rule_spreadsheet_content( asset_id, filename=filename, content=content, - actor=current_user.name, + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -518,22 +328,33 @@ def handle_agent_asset_spreadsheet_onlyoffice_callback( asset_id: str, payload: AgentAssetOnlyOfficeCallbackWrite, db: DbSession, + access_token: Annotated[ + str, + Query(min_length=1, description="ONLYOFFICE 回调专用短时令牌。"), + ], version: Annotated[ str | None, Query(description="兼容旧 ONLYOFFICE 回调；当前表格模式不再使用。"), ] = None, - actor_name: Annotated[ - str | None, - Query(description="发起编辑的用户显示名。"), - ] = None, ) -> AgentAssetOnlyOfficeCallbackRead: try: - AgentAssetService(db).handle_rule_spreadsheet_onlyoffice_callback( + bootstrap_service = AgentAssetService(db) + validated_session = bootstrap_service.validate_rule_spreadsheet_access_token( + asset_id, + access_token, + expected_scope=AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + ) + service = _onlyoffice_service_for_session(db, validated_session) + service.handle_rule_spreadsheet_onlyoffice_callback( asset_id, version=version, payload=payload.model_dump(), - actor_name=actor_name, + callback_token=access_token, ) + except AgentAssetOnlyOfficeReplayError as exc: + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from exc + except AgentAssetOnlyOfficeSecurityError as exc: + raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc)) from exc except Exception as exc: _handle_asset_error(exc) @@ -548,12 +369,14 @@ def handle_agent_asset_spreadsheet_onlyoffice_callback( ) def list_agent_asset_spreadsheet_change_records( asset_id: str, - _: CurrentUser, + current_user: CurrentUser, db: DbSession, limit: Annotated[int, Query(ge=1, le=30, description="返回条数，最多 30 条。")] = 30, ) -> list[AgentAssetSpreadsheetChangeRecordRead]: try: - return AgentAssetService(db).list_spreadsheet_change_records(asset_id, limit=limit) + return AgentAssetService( + db, current_user=current_user + ).list_spreadsheet_change_records(asset_id, limit=limit) except Exception as exc: _handle_asset_error(exc) @@ -579,9 +402,9 @@ def create_agent_asset( x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - return AgentAssetService(db).create_asset( + return AgentAssetService(db, current_user=current_user).create_asset( payload, - actor=(x_actor or current_user.name or payload.owner).strip() or "system", + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -607,7 +430,7 @@ def create_agent_asset( def update_agent_asset( asset_id: str, payload: AgentAssetUpdate, - current_user: CurrentUser, + current_user: RuleEditorUser, db: DbSession, x_actor: ActorHeader = None, x_request_id: RequestIdHeader = None, @@ -618,10 +441,10 @@ def update_agent_asset( current_user.is_admin or "manager" in role_codes ): raise PermissionError("只有高级管理员或 admin 管理员可以更改规则上线状态。") - return AgentAssetService(db).update_asset( + return AgentAssetService(db, current_user=current_user).update_asset( asset_id, payload, - actor=(x_actor or current_user.name or "system").strip() or "system", + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -642,6 +465,7 @@ def update_agent_asset( ) def list_agent_asset_versions( asset_id: str, + current_user: CurrentUser, db: DbSession, limit: Annotated[ int, @@ -649,7 +473,9 @@ def list_agent_asset_versions( ] = 20, ) -> list[AgentAssetVersionRead]: try: - return AgentAssetService(db).list_versions(asset_id, limit=limit) + return AgentAssetService(db, current_user=current_user).list_versions( + asset_id, limit=limit + ) except Exception as exc: _handle_asset_error(exc) @@ -659,7 +485,9 @@ def list_agent_asset_versions( response_model=AgentAssetVersionRead, status_code=status.HTTP_201_CREATED, summary="创建资产版本", - description="为指定资产创建新版本；规则和任务源文件可使用 Markdown，技能与 MCP 使用 JSON 快照。", + description=( + "为指定资产创建新版本；规则和任务源文件可使用 Markdown，技能与 MCP 使用 JSON 快照。" + ), responses={ status.HTTP_400_BAD_REQUEST: { "model": ErrorResponse, @@ -674,15 +502,17 @@ def list_agent_asset_versions( def create_agent_asset_version( asset_id: str, payload: AgentAssetVersionCreate, + current_user: RuleEditorUser, db: DbSession, x_actor: ActorHeader = None, x_request_id: RequestIdHeader = None, ) -> AgentAssetVersionRead: try: - return AgentAssetService(db).create_version( + payload = payload.model_copy(update={"created_by": _actor(current_user)}) + return AgentAssetService(db, current_user=current_user).create_version( asset_id, payload, - actor=(x_actor or payload.created_by).strip() or "system", + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -721,10 +551,11 @@ def create_agent_asset_review( raise PermissionError("只有财务人员或高级财务人员可以提交审核。") elif not (current_user.is_admin or "manager" in role_codes): raise PermissionError("只有高级财务人员可以审核规则。") - return AgentAssetService(db).create_review( + payload = payload.model_copy(update={"reviewer": _actor(current_user)}) + return AgentAssetService(db, current_user=current_user).create_review( asset_id, payload, - actor=(x_actor or payload.reviewer).strip() or "system", + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -749,121 +580,17 @@ def create_agent_asset_review( ) def activate_agent_asset( asset_id: str, - _: RuleReviewerUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRead: - try: - return AgentAssetService(db).activate_asset( - asset_id, - actor=(x_actor or "system").strip() or "system", - request_id=x_request_id, - ) - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/risk-rule-enabled", - response_model=AgentAssetRead, - summary="设置风险规则启用状态", - description=( - "高级财务人员可独立启用或停用 JSON 风险规则；停用后即使已上线也不会进入真实业务扫描。" - ), -) -def set_agent_asset_risk_rule_enabled( - asset_id: str, - payload: AgentAssetRiskRuleEnabledUpdate, current_user: RuleReviewerUser, db: DbSession, x_actor: ActorHeader = None, x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - asset = AgentAssetService(db).set_risk_rule_enabled( + return AgentAssetService(db, current_user=current_user).activate_asset( asset_id, - enabled=payload.enabled, - actor=(x_actor or current_user.name or "system").strip() or "system", + actor=_actor(current_user), request_id=x_request_id, ) - detail = AgentAssetService(db).get_asset(asset.id) - if detail is None: - raise LookupError("Asset not found") - return detail - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/risk-rule-level", - response_model=AgentAssetRead, - summary="风险规则风险等级已由评分模型接管", - description="风险规则等级和分数由自然语言规则评分模型生成，不再允许人工调整。", -) -def set_agent_asset_risk_rule_level( - asset_id: str, - payload: AgentAssetRiskRuleLevelUpdate, - current_user: RuleEditorUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRead: - try: - del asset_id, payload, current_user, db, x_actor, x_request_id - raise ValueError("风险等级和分数由评分模型自动计算，不能手动修改。") - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/return", - response_model=AgentAssetRiskRuleLatestTestSummary, - summary="回退待审核风险规则", - description="高级财务人员将待审核风险规则回退到草稿，并记录回退原因。", -) -def return_agent_asset_risk_rule( - asset_id: str, - payload: AgentAssetRiskRuleReturnRequest, - current_user: RuleReviewerUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRiskRuleLatestTestSummary: - try: - return AgentAssetService(db).return_risk_rule( - asset_id, - note=payload.note, - actor=(x_actor or current_user.name or "system").strip() or "system", - request_id=x_request_id, - ) - except Exception as exc: - _handle_asset_error(exc) - - -@router.post( - "/{asset_id}/publish", - response_model=AgentAssetRead, - summary="审核并发布风险规则", - description="高级财务人员确认测试通过后，将待审核风险规则一次性审核通过并发布上线。", -) -def publish_agent_asset_risk_rule( - asset_id: str, - current_user: RuleReviewerUser, - db: DbSession, - x_actor: ActorHeader = None, - x_request_id: RequestIdHeader = None, -) -> AgentAssetRead: - try: - asset = AgentAssetService(db).publish_risk_rule( - asset_id, - actor=(x_actor or current_user.name or "system").strip() or "system", - request_id=x_request_id, - ) - detail = AgentAssetService(db).get_asset(asset.id) - if detail is None: - raise LookupError("Asset not found") - return detail except Exception as exc: _handle_asset_error(exc) @@ -882,9 +609,9 @@ def delete_agent_asset( x_request_id: RequestIdHeader = None, ) -> None: try: - AgentAssetService(db).delete_unpublished_asset( + AgentAssetService(db, current_user=current_user).delete_unpublished_asset( asset_id, - actor=(x_actor or current_user.name or "system").strip() or "system", + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -906,10 +633,12 @@ def restore_agent_asset_version( x_request_id: RequestIdHeader = None, ) -> AgentAssetRead: try: - return AgentAssetService(db).restore_version_as_working_copy( + return AgentAssetService( + db, current_user=current_user + ).restore_version_as_working_copy( asset_id, version, - actor=(x_actor or current_user.name or "system").strip() or "system", + actor=_actor(current_user), request_id=x_request_id, ) except Exception as exc: @@ -924,11 +653,11 @@ def restore_agent_asset_version( ) def get_agent_asset_version_timeline( asset_id: str, - _: CurrentUser, + current_user: CurrentUser, db: DbSession, ) -> list[AgentAssetVersionTimelineItemRead]: try: - return AgentAssetService(db).list_version_timeline(asset_id) + return AgentAssetService(db, current_user=current_user).list_version_timeline(asset_id) except Exception as exc: _handle_asset_error(exc) @@ -999,22 +728,25 @@ def list_golden_cases( def run_golden_eval( asset_id: str, body: GoldenEvalRequest, - _: RuleReviewerUser, + current_user: RuleReviewerUser, db: DbSession, ) -> GoldenEvalRead: from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.risk_rule_golden_evaluator import RiskRuleGoldenEvaluator try: - asset = AgentAssetService(db).get_asset(asset_id) + service = AgentAssetService(db, current_user=current_user) + asset = service.get_asset(asset_id) if asset is None: raise LookupError("Asset not found") config = asset.config_json if isinstance(asset.config_json, dict) else {} - rule_document = config.get("rule_document") if isinstance(config.get("rule_document"), dict) else {} + rule_document = ( + config.get("rule_document") if isinstance(config.get("rule_document"), dict) else {} + ) file_name = str(rule_document.get("file_name") or "").strip() if not file_name: raise ValueError("该规则没有可执行的 manifest 文件。") - manager = AgentAssetService(db).rule_library_manager + manager = service.rule_library_manager manifest = manager.read_rule_library_json(library=RISK_RULES_LIBRARY, file_name=file_name) rule_code = str(manifest.get("rule_code") or "").strip() if not rule_code: diff --git a/server/src/app/api/v1/endpoints/agent_runs.py b/server/src/app/api/v1/endpoints/agent_runs.py index c11c190..16b7634 100644 --- a/server/src/app/api/v1/endpoints/agent_runs.py +++ b/server/src/app/api/v1/endpoints/agent_runs.py @@ -5,13 +5,15 @@ from typing import Annotated from fastapi import APIRouter, Depends, HTTPException, Query, status from sqlalchemy.orm import Session -from app.api.deps import get_current_user, get_db +from app.api.deps import CurrentUserContext, get_current_user, get_db from app.schemas.agent_run import AgentRunRead, AgentRunStatsRead from app.schemas.common import ErrorResponse +from app.services.agent_run_access_policy import AgentRunAccessPolicy from app.services.agent_runs import AgentRunService router = APIRouter(prefix="/agent-runs", dependencies=[Depends(get_current_user)]) DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] @router.get( @@ -22,6 +24,7 @@ DbSession = Annotated[Session, Depends(get_db)] ) def list_agent_runs( db: DbSession, + current_user: CurrentUser, agent: Annotated[ str | None, Query(description="Agent 名称筛选。"), @@ -39,9 +42,17 @@ def list_agent_runs( Query(ge=1, le=100, description="返回记录上限。"), ] = 20, ) -> list[AgentRunRead]: - return AgentRunService(db).list_runs( - agent=agent, status=status_value, source=source, limit=limit + tenant_id = AgentRunAccessPolicy.require_current_tenant_id(current_user) + scope_clause = AgentRunAccessPolicy.build_query_scope(current_user) + runs = AgentRunService(db).list_runs_for_tenant( + tenant_id=tenant_id, + agent=agent, + status=status_value, + source=source, + limit=limit, + scope_clause=scope_clause, ) + return AgentRunAccessPolicy.filter_list_items(runs, current_user, db) @router.get( @@ -52,6 +63,7 @@ def list_agent_runs( ) def summarize_agent_runs( db: DbSession, + current_user: CurrentUser, agent: Annotated[ str | None, Query(description="Agent 名称筛选。"), @@ -69,11 +81,15 @@ def summarize_agent_runs( Query(ge=1, le=500, description="统计最近记录数。"), ] = 200, ) -> AgentRunStatsRead: - return AgentRunService(db).summarize_runs( + tenant_id = AgentRunAccessPolicy.require_current_tenant_id(current_user) + scope_clause = AgentRunAccessPolicy.build_query_scope(current_user) + return AgentRunService(db).summarize_runs_for_tenant( + tenant_id=tenant_id, agent=agent, status=status_value, source=source, limit=limit, + scope_clause=scope_clause, ) @@ -89,8 +105,17 @@ def summarize_agent_runs( } }, ) -def get_agent_run(run_id: str, db: DbSession) -> AgentRunRead: - run = AgentRunService(db).get_run(run_id) +def get_agent_run( + run_id: str, + db: DbSession, + current_user: CurrentUser, +) -> AgentRunRead: + tenant_id = AgentRunAccessPolicy.require_current_tenant_id(current_user) + run = AgentRunService(db).get_run_for_tenant( + run_id, + tenant_id=tenant_id, + ) if run is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Run not found") + AgentRunAccessPolicy.require_detail_read(run, current_user) return run diff --git a/server/src/app/api/v1/endpoints/analytics.py b/server/src/app/api/v1/endpoints/analytics.py index 5f67031..caf4c65 100644 --- a/server/src/app/api/v1/endpoints/analytics.py +++ b/server/src/app/api/v1/endpoints/analytics.py @@ -6,16 +6,18 @@ from typing import Annotated from fastapi import APIRouter, Depends, Query from sqlalchemy.orm import Session -from app.api.deps import get_current_user, get_db +from app.api.deps import CurrentUserContext, get_current_user, get_db from app.schemas.digital_employee_dashboard import DigitalEmployeeDashboardRead from app.schemas.finance_dashboard import FinanceDashboardRead from app.schemas.system_dashboard import SystemDashboardRead from app.services.digital_employee_dashboard import DigitalEmployeeDashboardService +from app.services.finance_dashboard_access_policy import FinanceDashboardAccessPolicy from app.services.finance_dashboard_snapshot import FinanceDashboardSnapshotService from app.services.system_dashboard import SystemDashboardService router = APIRouter(prefix="/analytics", dependencies=[Depends(get_current_user)]) DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] @router.get( @@ -42,6 +44,7 @@ def get_system_dashboard( ) def get_digital_employee_dashboard( db: DbSession, + current_user: CurrentUser, days: Annotated[ int, Query(ge=1, le=30, description="统计窗口天数。"), @@ -51,7 +54,10 @@ def get_digital_employee_dashboard( Query(ge=1, le=1000, description="窗口内最多读取的运行记录数。"), ] = 300, ) -> DigitalEmployeeDashboardRead: - return DigitalEmployeeDashboardService(db).build_dashboard(days=days, limit=limit) + return DigitalEmployeeDashboardService( + db, + tenant_id=current_user.tenant_id, + ).build_dashboard(days=days, limit=limit) @router.get( @@ -62,17 +68,20 @@ def get_digital_employee_dashboard( ) def get_finance_dashboard( db: DbSession, + current_user: CurrentUser, range_key: Annotated[str, Query(max_length=30, description="顶部时间范围。")] = "近10日", start_date: Annotated[date | None, Query(description="自定义开始日期。")] = None, end_date: Annotated[date | None, Query(description="自定义结束日期。")] = None, - trend_range: Annotated[str, Query(max_length=30, description="趋势图时间范围。")] = ( - "近12天" - ), + trend_range: Annotated[str, Query(max_length=30, description="趋势图时间范围。")] = ("近12天"), department_range: Annotated[str, Query(max_length=30, description="排行分析时间范围。")] = ( "本月" ), ) -> FinanceDashboardRead: - return FinanceDashboardSnapshotService(db).build_dashboard( + FinanceDashboardAccessPolicy.require_read(current_user) + return FinanceDashboardSnapshotService( + db, + tenant_id=current_user.tenant_id, + ).build_dashboard( range_key=range_key, start_date=start_date, end_date=end_date, diff --git a/server/src/app/api/v1/endpoints/auth.py b/server/src/app/api/v1/endpoints/auth.py index 9ced8d8..570080a 100644 --- a/server/src/app/api/v1/endpoints/auth.py +++ b/server/src/app/api/v1/endpoints/auth.py @@ -53,7 +53,10 @@ def get_current_auth_user( current_user: Annotated[CurrentUserContext, Depends(get_current_user)], db: DbSession, ) -> AuthUserRead: - user = AuthService(db).get_user_snapshot(current_user.username) + user = AuthService(db).get_user_snapshot( + current_user.username, + tenant_id=current_user.tenant_id, + ) if user is not None: return user @@ -78,6 +81,7 @@ def get_current_auth_user( ), avatar=name[:1].upper(), isAdmin=True, + tenantId=current_user.tenant_id, ) raise HTTPException( diff --git a/server/src/app/api/v1/endpoints/cfo_value.py b/server/src/app/api/v1/endpoints/cfo_value.py new file mode 100644 index 0000000..dc13298 --- /dev/null +++ b/server/src/app/api/v1/endpoints/cfo_value.py @@ -0,0 +1,73 @@ +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from typing import Annotated, Literal + +from fastapi import APIRouter, Depends, HTTPException, Query, status +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.schemas.cfo_value import CfoValueDashboardRead, CfoValueFiltersRead +from app.services.cfo_value_analytics import CfoValueAnalyticsService +from app.services.savings_access_policy import SavingsPermissionError + +router = APIRouter(prefix="/analytics") +DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] + + +@router.get( + "/cfo-value", + response_model=CfoValueDashboardRead, + summary="查询 CFO 经营价值看板", + description=( + "仅汇总 Savings Ledger 中有基线、证据、去重和独立财务确认的价值事实；" + "风险暴露、预计机会和待确认结果不会混入已确认现金节省。" + ), +) +def get_cfo_value_dashboard( + db: DbSession, + current_user: CurrentUser, + start: datetime | None = None, + end: datetime | None = None, + as_of: datetime | None = None, + department_id: Annotated[str | None, Query(max_length=160)] = None, + project_code: Annotated[str | None, Query(max_length=160)] = None, + expense_type: Annotated[str | None, Query(max_length=80)] = None, + supplier_id: Annotated[str | None, Query(max_length=160)] = None, + city: Annotated[str | None, Query(max_length=160)] = None, + owner_id: Annotated[str | None, Query(max_length=120)] = None, + source_type: Annotated[str | None, Query(max_length=50)] = None, + value_kind: Annotated[Literal["cash", "labor"] | None, Query()] = None, +) -> CfoValueDashboardRead: + now = datetime.now(UTC) + normalized_end = end or now + normalized_start = start or (normalized_end - timedelta(days=90)) + normalized_as_of = as_of or now + try: + return CfoValueAnalyticsService(db).build_dashboard( + current_user, + start=normalized_start, + end=normalized_end, + as_of=normalized_as_of, + filters=CfoValueFiltersRead( + department_id=department_id, + project_code=project_code, + expense_type=expense_type, + supplier_id=supplier_id, + city=city, + owner_id=owner_id, + source_type=source_type, + value_kind=value_kind, + ), + ) + except SavingsPermissionError as error: + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail=str(error), + ) from error + except (ValueError, PermissionError) as error: + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail=str(error), + ) from error diff --git a/server/src/app/api/v1/endpoints/commercial.py b/server/src/app/api/v1/endpoints/commercial.py new file mode 100644 index 0000000..120dae9 --- /dev/null +++ b/server/src/app/api/v1/endpoints/commercial.py @@ -0,0 +1,587 @@ +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from typing import Annotated + +from fastapi import APIRouter, Depends, Header, HTTPException, Query, status +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.api.deps import ( + CurrentUserContext, + get_current_user, + get_db, + require_platform_admin_user, +) +from app.schemas.commercial import ( + CommercialAccountRead, + CommercialAnalyticsRead, + CommercialCostEventCreate, + CommercialCostEventRead, + CommercialEntitlementRead, + CommercialEntitlementUpsert, + CommercialMutationRead, + CommercialPlanActivationRead, + CommercialPlanCreate, + CommercialPlanRead, + CommercialPricingScenarioRead, + CommercialPricingScenarioWrite, + CommercialSubscriptionCreate, + CommercialSubscriptionRead, + CommercialSubscriptionTransition, + CommercialVersionAction, + UsageMeterEventCreate, + UsageMeterEventRead, +) +from app.services.commercial_access_policy import ( + CommercialAccessPolicy, + CommercialConfigurationError, + CommercialConflictError, + CommercialPermissionError, +) +from app.services.commercial_admin import CommercialAdminService +from app.services.commercial_analytics import CommercialAnalyticsService +from app.services.commercial_entitlements import CommercialEntitlementService +from app.services.commercial_metering import CommercialMeteringService +from app.services.commercial_pricing import CommercialPricingService +from app.services.commercial_queries import CommercialQueryService + +router = APIRouter(prefix="/commercial") +DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] +PlatformAdmin = Annotated[CurrentUserContext, Depends(require_platform_admin_user)] +RequestId = Annotated[ + str, + Header(alias="X-Request-Id", min_length=1, max_length=120), +] + + +@router.get("/account", response_model=CommercialAccountRead, summary="读取当前租户商业账户") +def get_current_commercial_account( + db: DbSession, + current_user: CurrentUser, + as_of: datetime | None = None, +) -> CommercialAccountRead: + try: + return CommercialEntitlementService(db).get_account(current_user, as_of=as_of) + except Exception as error: + raise _http_error(error) from error + + +@router.get( + "/admin/tenants/{tenant_id}/account", + response_model=CommercialAccountRead, + summary="平台管理员读取租户商业账户", +) +def get_tenant_commercial_account( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + as_of: datetime | None = None, +) -> CommercialAccountRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + return CommercialEntitlementService(db).get_account_for_tenant(target, as_of=as_of) + + +@router.post( + "/admin/tenants/{tenant_id}/plans", + response_model=CommercialPlanRead, + status_code=status.HTTP_201_CREATED, + summary="创建租户套餐新版本", +) +def create_commercial_plan( + tenant_id: str, + payload: CommercialPlanCreate, + db: DbSession, + current_user: PlatformAdmin, + request_id: RequestId, +) -> CommercialPlanRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row = CommercialAdminService(db).create_plan( + target, + payload, + actor_id=current_user.username, + request_id=request_id, + reason=payload.reason, + ) + db.commit() + db.refresh(row) + return row + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/plans/{plan_id}/activate", + response_model=CommercialPlanActivationRead, + summary="激活套餐版本并退役同编码旧版本", +) +def activate_commercial_plan( + tenant_id: str, + plan_id: str, + payload: CommercialVersionAction, + db: DbSession, + current_user: PlatformAdmin, + request_id: RequestId, +) -> CommercialPlanActivationRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + result = CommercialAdminService(db).activate_plan( + target, + plan_id, + expected_version=payload.expected_version, + actor_id=current_user.username, + request_id=request_id, + reason=payload.reason, + ) + db.commit() + return result + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/subscriptions", + response_model=CommercialSubscriptionRead, + status_code=status.HTTP_201_CREATED, + summary="创建并激活租户订阅快照", +) +def create_commercial_subscription( + tenant_id: str, + payload: CommercialSubscriptionCreate, + db: DbSession, + current_user: PlatformAdmin, + request_id: RequestId, +) -> CommercialSubscriptionRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row = CommercialAdminService(db).create_subscription( + target, + payload, + actor_id=current_user.username, + request_id=request_id, + reason=payload.reason, + ) + db.commit() + db.refresh(row) + return row + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/subscriptions/{subscription_id}/activate", + response_model=CommercialSubscriptionRead, + summary="重新激活可恢复的租户订阅", +) +def activate_commercial_subscription( + tenant_id: str, + subscription_id: str, + payload: CommercialVersionAction, + db: DbSession, + current_user: PlatformAdmin, + request_id: RequestId, +) -> CommercialSubscriptionRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row = CommercialAdminService(db).activate_subscription( + target, + subscription_id, + expected_version=payload.expected_version, + actor_id=current_user.username, + request_id=request_id, + reason=payload.reason, + ) + db.commit() + db.refresh(row) + return row + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/subscriptions/{subscription_id}/transition", + response_model=CommercialSubscriptionRead, + summary="暂停、标记逾期或终止租户订阅", +) +def transition_commercial_subscription( + tenant_id: str, + subscription_id: str, + payload: CommercialSubscriptionTransition, + db: DbSession, + current_user: PlatformAdmin, + request_id: RequestId, +) -> CommercialSubscriptionRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row = CommercialAdminService(db).transition_subscription( + target, + subscription_id, + payload, + actor_id=current_user.username, + request_id=request_id, + ) + db.commit() + db.refresh(row) + return row + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.get( + "/admin/tenants/{tenant_id}/plans", + response_model=list[CommercialPlanRead], + summary="查询租户套餐版本历史", +) +def list_commercial_plans( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + plan_status: str | None = None, + limit: int = Query(default=100, ge=1, le=200), + offset: int = Query(default=0, ge=0), +) -> list[CommercialPlanRead]: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + return CommercialQueryService(db).list_plans( + target, + status=plan_status, + limit=limit, + offset=offset, + ) + + +@router.get( + "/admin/tenants/{tenant_id}/subscriptions", + response_model=list[CommercialSubscriptionRead], + summary="查询租户订阅历史", +) +def list_commercial_subscriptions( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + subscription_status: str | None = None, + limit: int = Query(default=100, ge=1, le=200), + offset: int = Query(default=0, ge=0), +) -> list[CommercialSubscriptionRead]: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + return CommercialQueryService(db).list_subscriptions( + target, + status=subscription_status, + limit=limit, + offset=offset, + ) + + +@router.get( + "/admin/tenants/{tenant_id}/entitlements", + response_model=list[CommercialEntitlementRead], + summary="查询租户商业权益历史", +) +def list_commercial_entitlements( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + subscription_id: str | None = None, + billing_period_id: str | None = None, + entitlement_status: str | None = None, + limit: int = Query(default=200, ge=1, le=500), + offset: int = Query(default=0, ge=0), +) -> list[CommercialEntitlementRead]: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + return CommercialQueryService(db).list_entitlements( + target, + subscription_id=subscription_id, + billing_period_id=billing_period_id, + status=entitlement_status, + limit=limit, + offset=offset, + ) + + +@router.get( + "/admin/tenants/{tenant_id}/usage-events", + response_model=list[UsageMeterEventRead], + summary="查询租户追加式用量事实", +) +def list_commercial_usage_events( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + subscription_id: str | None = None, + billing_period_id: str | None = None, + start: datetime | None = None, + end: datetime | None = None, + limit: int = Query(default=200, ge=1, le=500), + offset: int = Query(default=0, ge=0), +) -> list[UsageMeterEventRead]: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + return CommercialQueryService(db).list_usage_events( + target, + subscription_id=subscription_id, + billing_period_id=billing_period_id, + start=start, + end=end, + limit=limit, + offset=offset, + ) + except Exception as error: + raise _http_error(error) from error + + +@router.get( + "/admin/tenants/{tenant_id}/cost-events", + response_model=list[CommercialCostEventRead], + summary="查询租户追加式内部成本事实", +) +def list_commercial_cost_events( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + subscription_id: str | None = None, + billing_period_id: str | None = None, + start: datetime | None = None, + end: datetime | None = None, + limit: int = Query(default=200, ge=1, le=500), + offset: int = Query(default=0, ge=0), +) -> list[CommercialCostEventRead]: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + return CommercialQueryService(db).list_cost_events( + target, + subscription_id=subscription_id, + billing_period_id=billing_period_id, + start=start, + end=end, + limit=limit, + offset=offset, + ) + except Exception as error: + raise _http_error(error) from error + + +@router.put( + "/admin/tenants/{tenant_id}/entitlements", + response_model=CommercialEntitlementRead, + summary="创建或版本化更新租户商业权益", +) +def upsert_commercial_entitlement( + tenant_id: str, + payload: CommercialEntitlementUpsert, + db: DbSession, + current_user: PlatformAdmin, + request_id: RequestId, +) -> CommercialEntitlementRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row = CommercialAdminService(db).upsert_entitlement( + target, + payload, + actor_id=current_user.username, + request_id=request_id, + reason=payload.reason, + ) + db.commit() + db.refresh(row) + return row + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/entitlements/{entitlement_id}/activate", + response_model=CommercialEntitlementRead, + summary="激活租户商业权益", +) +def activate_commercial_entitlement( + tenant_id: str, + entitlement_id: str, + payload: CommercialVersionAction, + db: DbSession, + current_user: PlatformAdmin, + request_id: RequestId, +) -> CommercialEntitlementRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row = CommercialAdminService(db).activate_entitlement( + target, + entitlement_id, + expected_version=payload.expected_version, + actor_id=current_user.username, + request_id=request_id, + reason=payload.reason, + ) + db.commit() + db.refresh(row) + return row + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/usage-events", + response_model=CommercialMutationRead, + summary="幂等写入租户用量事件", +) +def record_commercial_usage( + tenant_id: str, + payload: UsageMeterEventCreate, + db: DbSession, + current_user: PlatformAdmin, +) -> CommercialMutationRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row, created = CommercialMeteringService(db).record_usage( + target, + payload, + actor_type="admin", + actor_id=current_user.username, + ) + db.commit() + return CommercialMutationRead(created=created, usage_event=row) + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/cost-events", + response_model=CommercialMutationRead, + summary="幂等写入平台内部成本事件", +) +def record_commercial_cost( + tenant_id: str, + payload: CommercialCostEventCreate, + db: DbSession, + current_user: PlatformAdmin, +) -> CommercialMutationRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + row, created = CommercialMeteringService(db).record_cost(target, payload) + db.commit() + return CommercialMutationRead(created=created, cost_event=row) + except Exception as error: + db.rollback() + raise _http_error(error) from error + + +@router.get( + "/admin/tenants/{tenant_id}/analytics", + response_model=CommercialAnalyticsRead, + summary="平台商业与客户价值分账分析", +) +def get_commercial_analytics( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + start: datetime | None = None, + end: datetime | None = None, + as_of: datetime | None = None, +) -> CommercialAnalyticsRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + now = datetime.now(UTC) + normalized_end = end or now + normalized_start = start or (normalized_end - timedelta(days=90)) + try: + return CommercialAnalyticsService(db).build( + target, + start=normalized_start, + end=normalized_end, + as_of=as_of or now, + ) + except Exception as error: + raise _http_error(error) from error + + +@router.post( + "/admin/tenants/{tenant_id}/pricing-scenarios", + response_model=CommercialPricingScenarioRead, + summary="按真实成本与确认价值计算可持续定价走廊", +) +def build_commercial_pricing_scenario( + tenant_id: str, + payload: CommercialPricingScenarioWrite, + db: DbSession, + current_user: PlatformAdmin, +) -> CommercialPricingScenarioRead: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + return CommercialPricingService(db).build(target, payload) + except Exception as error: + raise _http_error(error) from error + + +def _http_error(error: Exception) -> HTTPException: + if isinstance(error, LookupError): + return HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) + if isinstance(error, CommercialPermissionError): + return HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(error)) + if isinstance(error, CommercialConflictError): + return HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(error)) + if isinstance(error, IntegrityError): + return HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="商业配置被并发修改或违反数据库唯一性约束，请刷新后重试。", + ) + if isinstance(error, (CommercialConfigurationError, ValueError)): + return HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) + raise error diff --git a/server/src/app/api/v1/endpoints/commercial_billing.py b/server/src/app/api/v1/endpoints/commercial_billing.py new file mode 100644 index 0000000..a0a436f --- /dev/null +++ b/server/src/app/api/v1/endpoints/commercial_billing.py @@ -0,0 +1,142 @@ +"""商业不可变账期和管理审计历史。""" + +from __future__ import annotations + +from datetime import datetime +from typing import Annotated + +from fastapi import APIRouter, Depends, HTTPException, Query, status +from sqlalchemy.orm import Session + +from app.api.deps import ( + CurrentUserContext, + get_current_user, + get_db, + require_platform_admin_user, +) +from app.schemas.commercial_billing import ( + CommercialAdminEventRead, + CommercialBillingPeriodRead, + CommercialBillingPeriodTenantRead, +) +from app.services.commercial_access_policy import ( + CommercialAccessPolicy, + CommercialPermissionError, +) +from app.services.commercial_billing_periods import CommercialBillingPeriodService +from app.services.commercial_queries import CommercialQueryService + +router = APIRouter(prefix="/commercial") +DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] +PlatformAdmin = Annotated[CurrentUserContext, Depends(require_platform_admin_user)] + + +@router.get( + "/billing-periods", + response_model=list[CommercialBillingPeriodTenantRead], + summary="读取当前租户不可变账期历史", +) +def list_current_tenant_billing_periods( + db: DbSession, + current_user: CurrentUser, + subscription_id: str | None = None, + as_of: datetime | None = None, + limit: int = Query(default=200, ge=1, le=500), + offset: int = Query(default=0, ge=0), +) -> list[CommercialBillingPeriodTenantRead]: + try: + tenant_id = CommercialAccessPolicy.require_account_read(current_user) + rows = _period_reads( + db, + tenant_id, + subscription_id=subscription_id, + as_of=as_of, + limit=limit, + offset=offset, + ) + return [CommercialBillingPeriodTenantRead.model_validate(row.model_dump()) for row in rows] + except CommercialPermissionError as error: + raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(error)) from error + + +@router.get( + "/admin/tenants/{tenant_id}/billing-periods", + response_model=list[CommercialBillingPeriodRead], + summary="平台管理员读取租户不可变账期历史", +) +def list_tenant_billing_periods( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + subscription_id: str | None = None, + as_of: datetime | None = None, + limit: int = Query(default=200, ge=1, le=500), + offset: int = Query(default=0, ge=0), +) -> list[CommercialBillingPeriodRead]: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + return _period_reads( + db, + target, + subscription_id=subscription_id, + as_of=as_of, + limit=limit, + offset=offset, + ) + + +@router.get( + "/admin/tenants/{tenant_id}/admin-events", + response_model=list[CommercialAdminEventRead], + summary="平台管理员读取商业配置与续期审计", +) +def list_tenant_commercial_admin_events( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + action: str | None = None, + resource_type: str | None = None, + resource_id: str | None = None, + start: datetime | None = None, + end: datetime | None = None, + limit: int = Query(default=200, ge=1, le=500), + offset: int = Query(default=0, ge=0), +) -> list[CommercialAdminEventRead]: + target = CommercialAccessPolicy.require_platform_admin( + current_user, + target_tenant_id=tenant_id, + ) + try: + return CommercialQueryService(db).list_admin_events( + target, + action=action, + resource_type=resource_type, + resource_id=resource_id, + start=start, + end=end, + limit=limit, + offset=offset, + ) + except ValueError as error: + raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) from error + + +def _period_reads( + db: Session, + tenant_id: str, + *, + subscription_id: str | None, + as_of: datetime | None, + limit: int, + offset: int, +) -> list[CommercialBillingPeriodRead]: + rows = CommercialQueryService(db).list_billing_periods( + tenant_id, + subscription_id=subscription_id, + limit=limit, + offset=offset, + ) + return [CommercialBillingPeriodService.to_read(row, as_of=as_of) for row in rows] diff --git a/server/src/app/api/v1/endpoints/employee_profiles.py b/server/src/app/api/v1/endpoints/employee_profiles.py index 5709420..ab4fd92 100644 --- a/server/src/app/api/v1/endpoints/employee_profiles.py +++ b/server/src/app/api/v1/endpoints/employee_profiles.py @@ -2,12 +2,13 @@ from __future__ import annotations from typing import Annotated -from fastapi import APIRouter, Depends, Query +from fastapi import APIRouter, Depends, HTTPException, Query, status from sqlalchemy import func, or_, select from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext, get_current_user, get_db from app.models.employee import Employee +from app.models.financial_record import ExpenseClaim from app.schemas.employee_profile import EmployeeProfileLatestRead from app.services.account_behavior_profile import AccountBehaviorProfileService from app.services.employee_behavior_profile_service import EmployeeBehaviorProfileService @@ -32,7 +33,10 @@ def get_current_employee_latest_profile( ) -> EmployeeProfileLatestRead: employee = _resolve_current_employee(db, current_user) if employee is None: - return AccountBehaviorProfileService(db).get_latest_account_profile( + return AccountBehaviorProfileService( + db, + tenant_id=current_user.tenant_id, + ).get_latest_account_profile( account_id=current_user.username, account_name=current_user.name, identifiers=_current_account_identifiers(current_user), @@ -41,7 +45,7 @@ def get_current_employee_latest_profile( expense_type_scope=expense_type_scope, ) - service = EmployeeBehaviorProfileService(db) + service = EmployeeBehaviorProfileService(db, tenant_id=current_user.tenant_id) latest = service.get_latest_profile( employee_id=employee.id, scene=scene, @@ -65,13 +69,13 @@ def get_current_employee_latest_profile( @router.get( - "/{employee_id}/latest", + "/{target_employee_id}/latest", response_model=EmployeeProfileLatestRead, summary="读取员工最新业务行为画像", description="返回员工在指定场景下的最新画像快照，审批场景默认只展示费用支出和流程质量画像。", ) def get_employee_latest_profile( - employee_id: str, + target_employee_id: str, db: DbSession, current_user: CurrentUser, scene: Annotated[str, Query(max_length=50)] = "approval", @@ -79,9 +83,25 @@ def get_employee_latest_profile( window_days: Annotated[int, Query(ge=1, le=365)] = 90, expense_type_scope: Annotated[str, Query(max_length=50)] = "overall", ) -> EmployeeProfileLatestRead: - del current_user - return EmployeeBehaviorProfileService(db).get_latest_profile( - employee_id=employee_id, + target = _resolve_tenant_employee( + db, + current_user.tenant_id, + target_employee_id, + ) + if target is None or not _can_read_target(db, current_user, target): + _raise_not_found() + if claim_id and not _claim_matches_target( + db, + tenant_id=current_user.tenant_id, + claim_id=claim_id, + employee_id=target.id, + ): + _raise_not_found() + return EmployeeBehaviorProfileService( + db, + tenant_id=current_user.tenant_id, + ).get_latest_profile( + employee_id=target.id, scene=scene, claim_id=claim_id, window_days=window_days, @@ -93,7 +113,11 @@ def _resolve_current_employee( db: Session, current_user: CurrentUserContext, ) -> Employee | None: + tenant_id = str(current_user.tenant_id or "").strip() + if not tenant_id: + return None identities = [ + str(current_user.employee_id or "").strip(), str(current_user.username or "").strip(), str(current_user.name or "").strip(), ] @@ -108,16 +132,95 @@ def _resolve_current_employee( if email_values: conditions.append(func.lower(Employee.email).in_(email_values)) if exact_values: + conditions.append(Employee.id.in_(exact_values)) conditions.append(Employee.name.in_(exact_values)) conditions.append(Employee.employee_no.in_(exact_values)) if not conditions: return None - stmt = select(Employee).where(or_(*conditions)).order_by(Employee.created_at.asc()).limit(1) + stmt = ( + select(Employee) + .where( + Employee.tenant_id == tenant_id, + or_(*conditions), + ) + .order_by(Employee.created_at.asc()) + .limit(1) + ) return db.scalars(stmt).first() +def _resolve_tenant_employee( + db: Session, + tenant_id: str, + identifier: str, +) -> Employee | None: + normalized_tenant = str(tenant_id or "").strip() + normalized = str(identifier or "").strip() + if not normalized_tenant or not normalized: + return None + conditions = [ + Employee.id == normalized, + Employee.employee_no == normalized, + Employee.name == normalized, + ] + if "@" in normalized: + conditions.append(func.lower(Employee.email) == normalized.lower()) + return db.scalars( + select(Employee) + .where( + Employee.tenant_id == normalized_tenant, + or_(*conditions), + ) + .order_by(Employee.created_at.asc()) + .limit(1) + ).first() + + +def _can_read_target( + db: Session, + current_user: CurrentUserContext, + target: Employee, +) -> bool: + current = _resolve_current_employee(db, current_user) + if current is not None and current.id == target.id: + return True + role_codes = {str(item or "").strip().lower() for item in current_user.role_codes or []} + if current_user.is_admin or role_codes & {"finance", "executive"}: + return True + return bool( + current is not None + and role_codes & {"manager", "approver"} + and target.manager_id == current.id + ) + + +def _claim_matches_target( + db: Session, + *, + tenant_id: str, + claim_id: str, + employee_id: str, +) -> bool: + return bool( + db.scalar( + select(ExpenseClaim.id).where( + ExpenseClaim.tenant_id == str(tenant_id or "").strip(), + ExpenseClaim.id == str(claim_id or "").strip(), + ExpenseClaim.employee_id == employee_id, + ) + ) + ) + + +def _raise_not_found() -> None: + raise HTTPException( + status_code=status.HTTP_404_NOT_FOUND, + detail="员工画像不存在。", + ) + + def _missing_usage_duration_metric(latest: EmployeeProfileLatestRead) -> bool: if latest.scene != "operations": return False diff --git a/server/src/app/api/v1/endpoints/employees.py b/server/src/app/api/v1/endpoints/employees.py index 0dfe940..a3a9472 100644 --- a/server/src/app/api/v1/endpoints/employees.py +++ b/server/src/app/api/v1/endpoints/employees.py @@ -6,7 +6,7 @@ from fastapi import APIRouter, Depends, File, HTTPException, Query, UploadFile, from fastapi.responses import Response from sqlalchemy.orm import Session -from app.api.deps import get_db, require_admin_user +from app.api.deps import CurrentUserContext, get_db, require_admin_user from app.api.pagination import PageNumber, PageSize, page_payload, wants_page from app.schemas.common import ErrorResponse, PaginatedResponse from app.schemas.employee import ( @@ -21,6 +21,11 @@ from app.services.employee_pagination import EmployeePaginationService router = APIRouter(dependencies=[Depends(require_admin_user)]) DbSession = Annotated[Session, Depends(get_db)] +AdminUser = Annotated[CurrentUserContext, Depends(require_admin_user)] + + +def _employee_service(db: Session, current_user: CurrentUserContext) -> EmployeeService: + return EmployeeService(db, tenant_id=current_user.tenant_id) @router.get( @@ -29,8 +34,8 @@ DbSession = Annotated[Session, Depends(get_db)] summary="读取员工目录元数据", description="返回员工总数、状态汇总和可选角色列表，供员工管理页面初始化使用。", ) -def get_employee_meta(db: DbSession) -> EmployeeMetaRead: - return EmployeeService(db).get_employee_meta() +def get_employee_meta(db: DbSession, current_user: AdminUser) -> EmployeeMetaRead: + return _employee_service(db, current_user).get_employee_meta() @router.get( @@ -41,6 +46,7 @@ def get_employee_meta(db: DbSession) -> EmployeeMetaRead: ) def list_employees( db: DbSession, + current_user: AdminUser, status_filter: Annotated[ str | None, Query(alias="status", description="员工状态筛选值。"), @@ -54,14 +60,20 @@ def list_employees( ) -> list[EmployeeRead] | PaginatedResponse[EmployeeRead]: if wants_page(page, page_size): return page_payload( - EmployeePaginationService(db).list_employees_page( + EmployeePaginationService( + db, + tenant_id=current_user.tenant_id, + ).list_employees_page( status=status_filter, keyword=keyword, page=page, page_size=page_size, ) ) - return EmployeeService(db).list_employees(status=status_filter, keyword=keyword) + return _employee_service(db, current_user).list_employees( + status=status_filter, + keyword=keyword, + ) @router.get( @@ -69,8 +81,8 @@ def list_employees( summary="下载员工导入模板", description="下载固定格式的员工 Excel 导入模板。", ) -def download_employee_import_template(db: DbSession) -> Response: - content = EmployeeService(db).build_import_template() +def download_employee_import_template(db: DbSession, current_user: AdminUser) -> Response: + content = _employee_service(db, current_user).build_import_template() return Response( content=content, media_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", @@ -87,6 +99,7 @@ def download_employee_import_template(db: DbSession) -> Response: ) def export_employees( db: DbSession, + current_user: AdminUser, status_filter: Annotated[ str | None, Query(alias="status", description="员工状态筛选值。"), @@ -96,7 +109,10 @@ def export_employees( Query(description="姓名、工号、邮箱等关键字模糊查询。"), ] = None, ) -> Response: - content = EmployeeService(db).export_employees(status=status_filter, keyword=keyword) + content = _employee_service(db, current_user).export_employees( + status=status_filter, + keyword=keyword, + ) return Response( content=content, media_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", @@ -112,6 +128,7 @@ def export_employees( ) async def import_employees( db: DbSession, + current_user: AdminUser, file: Annotated[UploadFile, File(description="待导入的员工 Excel 文件。")], ) -> EmployeeImportResultRead: filename = (file.filename or "").lower() @@ -122,7 +139,10 @@ async def import_employees( ) content = await file.read() - return EmployeeService(db).import_employees(content) + return _employee_service(db, current_user).import_employees( + content, + actor=current_user.username, + ) @router.post( @@ -138,9 +158,13 @@ async def import_employees( } }, ) -def create_employee(payload: EmployeeCreate, db: DbSession) -> EmployeeRead: +def create_employee( + payload: EmployeeCreate, + db: DbSession, + current_user: AdminUser, +) -> EmployeeRead: try: - return EmployeeService(db).create_employee(payload) + return _employee_service(db, current_user).create_employee(payload) except ValueError as exc: raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc @@ -157,8 +181,12 @@ def create_employee(payload: EmployeeCreate, db: DbSession) -> EmployeeRead: } }, ) -def get_employee(employee_id: str, db: DbSession) -> EmployeeRead: - employee = EmployeeService(db).get_employee(employee_id) +def get_employee( + employee_id: str, + db: DbSession, + current_user: AdminUser, +) -> EmployeeRead: + employee = _employee_service(db, current_user).get_employee(employee_id) if employee is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Employee not found") return employee @@ -180,9 +208,14 @@ def get_employee(employee_id: str, db: DbSession) -> EmployeeRead: }, }, ) -def update_employee(employee_id: str, payload: EmployeeUpdate, db: DbSession) -> EmployeeRead: +def update_employee( + employee_id: str, + payload: EmployeeUpdate, + db: DbSession, + current_user: AdminUser, +) -> EmployeeRead: try: - return EmployeeService(db).update_employee(employee_id, payload) + return _employee_service(db, current_user).update_employee(employee_id, payload) except LookupError as exc: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc except ValueError as exc: @@ -201,9 +234,13 @@ def update_employee(employee_id: str, payload: EmployeeUpdate, db: DbSession) -> } }, ) -def disable_employee(employee_id: str, db: DbSession) -> EmployeeRead: +def disable_employee( + employee_id: str, + db: DbSession, + current_user: AdminUser, +) -> EmployeeRead: try: - return EmployeeService(db).disable_employee(employee_id) + return _employee_service(db, current_user).disable_employee(employee_id) except LookupError as exc: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc @@ -220,8 +257,12 @@ def disable_employee(employee_id: str, db: DbSession) -> EmployeeRead: } }, ) -def enable_employee(employee_id: str, db: DbSession) -> EmployeeRead: +def enable_employee( + employee_id: str, + db: DbSession, + current_user: AdminUser, +) -> EmployeeRead: try: - return EmployeeService(db).enable_employee(employee_id) + return _employee_service(db, current_user).enable_employee(employee_id) except LookupError as exc: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc diff --git a/server/src/app/api/v1/endpoints/finance_report_configs.py b/server/src/app/api/v1/endpoints/finance_report_configs.py new file mode 100644 index 0000000..fba2504 --- /dev/null +++ b/server/src/app/api/v1/endpoints/finance_report_configs.py @@ -0,0 +1,76 @@ +from __future__ import annotations + +from typing import Annotated + +from fastapi import APIRouter, Depends, HTTPException, status +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.schemas.finance_report_config import ( + FinanceReportConfigRead, + FinanceReportConfigUpdate, +) +from app.services.expense_claim_access_policy import ExpenseClaimAccessPolicy +from app.services.finance_report_tenant import TenantFinanceReportConfigService + +router = APIRouter(prefix="/finance-report-config") +DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] +MANAGE_ROLES = frozenset({"finance", "executive"}) + + +@router.get("", response_model=FinanceReportConfigRead) +def get_finance_report_config( + db: DbSession, + current_user: CurrentUser, +) -> FinanceReportConfigRead: + _require_manage(current_user) + row = TenantFinanceReportConfigService(db).get(tenant_id=current_user.tenant_id) + if row is None: + return FinanceReportConfigRead( + tenant_id=current_user.tenant_id, + status="disabled", + delivery_enabled=False, + ) + return _serialize(row) + + +@router.put("", response_model=FinanceReportConfigRead) +def update_finance_report_config( + payload: FinanceReportConfigUpdate, + db: DbSession, + current_user: CurrentUser, +) -> FinanceReportConfigRead: + _require_manage(current_user) + try: + row = TenantFinanceReportConfigService(db).upsert( + tenant_id=current_user.tenant_id, + recipients=payload.recipients, + delivery_enabled=payload.delivery_enabled, + updated_by=current_user.username, + ) + except (LookupError, ValueError) as exc: + raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc + return _serialize(row) + + +def _require_manage(current_user: CurrentUserContext) -> None: + tenant_id = str(current_user.tenant_id or "").strip() + roles = ExpenseClaimAccessPolicy.normalize_role_codes(current_user) + if tenant_id and (current_user.is_admin or roles & MANAGE_ROLES): + return + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail="只有本租户财务人员或管理员可以维护报告收件配置。", + ) + + +def _serialize(row) -> FinanceReportConfigRead: + return FinanceReportConfigRead( + tenant_id=row.tenant_id, + status=row.status, + delivery_enabled=row.delivery_enabled, + recipients=[str(item) for item in list(row.recipients_json or [])], + updated_by=row.updated_by, + updated_at=row.updated_at, + ) diff --git a/server/src/app/api/v1/endpoints/financial_connectors.py b/server/src/app/api/v1/endpoints/financial_connectors.py new file mode 100644 index 0000000..1c6e53b --- /dev/null +++ b/server/src/app/api/v1/endpoints/financial_connectors.py @@ -0,0 +1,544 @@ +from __future__ import annotations + +import logging +from typing import Annotated + +from fastapi import APIRouter, Depends, Header, HTTPException, Query, status +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.api.deps import ( + CurrentUserContext, + get_current_user, + get_db, + require_platform_admin_user, +) +from app.schemas.financial_connector import ( + FinancialConnectorConfigCreate, + FinancialConnectorConfigEventRead, + FinancialConnectorConfigLifecycleAction, + FinancialConnectorConfigRead, + FinancialConnectorConfigRotateAction, + FinancialConnectorConfigRotationRead, + FinancialConnectorObservabilityRead, + FinancialConnectorSimulationCreate, + FinancialConnectorSimulationRead, + FinancialEventEnvelope, + FinancialEventIngestionRead, + FinancialPaymentEvidenceRead, + PaymentReconciliationActionCreate, + PaymentReconciliationCaseDetailRead, + PaymentReconciliationListRead, +) +from app.services.expense_claims import ExpenseClaimService +from app.services.financial_connector_auth import FinancialConnectorAuthError +from app.services.financial_connector_commercial import ( + FinancialConnectorCommercialAccessDenied, +) +from app.services.financial_connector_config_lifecycle import ( + FinancialConnectorConfigConflictError, + FinancialConnectorConfigLifecycleService, +) +from app.services.financial_connector_configs import ( + FinancialConnectorConfigError, + FinancialConnectorConfigService, +) +from app.services.financial_connector_ingestion import ( + FinancialConnectorConflictError, + FinancialConnectorIngestionService, +) +from app.services.financial_connector_mock_adapter import ( + FinancialConnectorMockAdapter, + FinancialConnectorMockAdapterError, +) +from app.services.financial_connector_observability import ( + FinancialConnectorObservabilityPermissionError, + FinancialConnectorObservabilityService, +) +from app.services.financial_connector_operational_events import ( + FinancialConnectorOperationalEventCandidate, + FinancialConnectorOperationalEventService, +) +from app.services.financial_connector_payment_evidence import ( + FinancialConnectorPaymentEvidenceService, +) +from app.services.financial_connector_projection import ( + FinancialConnectorProjectionService, + FinancialReconciliationConflictError, + FinancialReconciliationPermissionError, +) + +router = APIRouter() +logger = logging.getLogger(__name__) +DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] +PlatformAdmin = Annotated[CurrentUserContext, Depends(require_platform_admin_user)] + + +@router.post( + "/integrations/financial-events", + response_model=FinancialEventIngestionRead, + summary="接收租户绑定且签名可验证的财务事件", +) +def ingest_financial_event( + payload: FinancialEventEnvelope, + db: DbSession, + tenant_id: Annotated[str, Header(alias="X-Financial-Tenant")], + provider: Annotated[str, Header(alias="X-Financial-Provider")], + key_version: Annotated[str, Header(alias="X-Financial-Key-Version")], + timestamp_value: Annotated[str, Header(alias="X-Financial-Timestamp")], + signature: Annotated[str, Header(alias="X-Financial-Signature")], +) -> FinancialEventIngestionRead: + try: + result = FinancialConnectorIngestionService(db).ingest( + payload, + tenant_header=tenant_id, + provider_header=provider, + key_version_header=key_version, + timestamp_header=timestamp_value, + signature_header=signature, + ) + db.commit() + return result + except FinancialConnectorAuthError as error: + db.rollback() + if error.operational_event is not None: + _persist_operational_event(db, error.operational_event) + else: + logger.warning( + "financial_connector_auth_failure_unattributed code=%s " + "path=/api/v1/integrations/financial-events", + error.code, + ) + raise HTTPException( + status_code=status.HTTP_401_UNAUTHORIZED, + detail={"code": error.code, "message": str(error)}, + ) from error + except FinancialConnectorConflictError as error: + db.rollback() + _persist_operational_event(db, error.operational_event) + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(error)) from error + except IntegrityError as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="事件正由另一请求处理，请使用相同 external_event_id 安全重试。", + ) from error + except FinancialConnectorCommercialAccessDenied as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_429_TOO_MANY_REQUESTS, + detail=str(error), + ) from error + except (ValueError, PermissionError) as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) from error + except Exception: + db.rollback() + raise + + +@router.post( + "/financial-connectors/admin/tenants/{tenant_id}/configs", + response_model=FinancialConnectorConfigRead, + status_code=status.HTTP_201_CREATED, + summary="平台管理员创建不含密钥明文的连接器配置", +) +def create_financial_connector_config( + tenant_id: str, + payload: FinancialConnectorConfigCreate, + db: DbSession, + current_user: PlatformAdmin, +) -> FinancialConnectorConfigRead: + try: + row = FinancialConnectorConfigService(db).create( + tenant_id=tenant_id, + payload=payload, + actor_id=current_user.username, + ) + db.commit() + db.refresh(row) + return FinancialConnectorConfigRead.model_validate(row) + except (FinancialConnectorConfigError, IntegrityError) as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(error)) from error + + +@router.get( + "/financial-connectors/admin/tenants/{tenant_id}/configs", + response_model=list[FinancialConnectorConfigRead], + summary="平台管理员读取脱敏连接器配置", +) +def list_financial_connector_configs( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, +) -> list[FinancialConnectorConfigRead]: + del current_user + return [ + FinancialConnectorConfigRead.model_validate(item) + for item in FinancialConnectorConfigService(db).list_for_tenant(tenant_id) + ] + + +@router.post( + "/financial-connectors/admin/tenants/{tenant_id}/configs/{config_id}/activate", + response_model=FinancialConnectorConfigRead, + summary="校验服务端密钥后激活指定版本连接器", +) +def activate_financial_connector_config( + tenant_id: str, + config_id: str, + payload: FinancialConnectorConfigLifecycleAction, + db: DbSession, + current_user: PlatformAdmin, +) -> FinancialConnectorConfigRead: + return _change_config_status( + tenant_id, + config_id, + payload, + db, + current_user, + action="activate", + ) + + +@router.post( + "/financial-connectors/admin/tenants/{tenant_id}/configs/{config_id}/disable", + response_model=FinancialConnectorConfigRead, + summary="按乐观版本停用连接器", +) +def disable_financial_connector_config( + tenant_id: str, + config_id: str, + payload: FinancialConnectorConfigLifecycleAction, + db: DbSession, + current_user: PlatformAdmin, +) -> FinancialConnectorConfigRead: + return _change_config_status( + tenant_id, + config_id, + payload, + db, + current_user, + action="disable", + ) + + +@router.post( + "/financial-connectors/admin/tenants/{tenant_id}/configs/{config_id}/rotate", + response_model=FinancialConnectorConfigRotationRead, + summary="原子轮换连接器密钥版本", +) +def rotate_financial_connector_config( + tenant_id: str, + config_id: str, + payload: FinancialConnectorConfigRotateAction, + db: DbSession, + current_user: PlatformAdmin, +) -> FinancialConnectorConfigRotationRead: + try: + result = FinancialConnectorConfigLifecycleService(db).rotate( + tenant_id=tenant_id, + config_id=config_id, + payload=payload, + actor_id=current_user.username, + ) + db.commit() + db.refresh(result.previous) + db.refresh(result.replacement) + return FinancialConnectorConfigRotationRead( + previous=FinancialConnectorConfigRead.model_validate(result.previous), + replacement=FinancialConnectorConfigRead.model_validate(result.replacement), + ) + except LookupError as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) from error + except FinancialConnectorAuthError as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail={"code": error.code, "message": str(error)}, + ) from error + except (FinancialConnectorConfigConflictError, IntegrityError) as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(error)) from error + + +@router.get( + "/financial-connectors/admin/tenants/{tenant_id}/config-events", + response_model=list[FinancialConnectorConfigEventRead], + summary="读取不含密钥材料的连接器配置审计时间线", +) +def list_financial_connector_config_events( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + config_id: str | None = Query(default=None), +) -> list[FinancialConnectorConfigEventRead]: + del current_user + return [ + FinancialConnectorConfigEventRead.model_validate(item) + for item in FinancialConnectorConfigLifecycleService(db).list_events( + tenant_id=tenant_id, + config_id=config_id, + ) + ] + + +@router.post( + "/financial-connectors/admin/tenants/{tenant_id}/configs/{config_id}/simulate", + response_model=FinancialConnectorSimulationRead, + summary="平台管理员运行确定性的非生产连接器场景", +) +def simulate_financial_connector_event( + tenant_id: str, + config_id: str, + payload: FinancialConnectorSimulationCreate, + db: DbSession, + current_user: PlatformAdmin, +) -> FinancialConnectorSimulationRead: + del current_user + try: + result = FinancialConnectorMockAdapter(db).run( + tenant_id=tenant_id, + config_id=config_id, + payload=payload, + ) + db.commit() + return result + except LookupError as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) from error + except FinancialConnectorAuthError as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail={"code": error.code, "message": str(error)}, + ) from error + except FinancialConnectorMockAdapterError as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) from error + except FinancialConnectorCommercialAccessDenied as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_429_TOO_MANY_REQUESTS, + detail=str(error), + ) from error + except IntegrityError as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail="模拟场景正由另一请求处理，请使用相同 request_id 安全重试。", + ) from error + except Exception: + db.rollback() + raise + + +@router.get( + "/financial-connectors/observability", + response_model=FinancialConnectorObservabilityRead, + summary="读取当前租户脱敏连接器运行指标", +) +def get_current_tenant_connector_observability( + db: DbSession, + current_user: CurrentUser, + window_hours: int = Query(default=24, ge=1, le=720), +) -> FinancialConnectorObservabilityRead: + try: + return FinancialConnectorObservabilityService(db).read_for_current_user( + current_user, + window_hours=window_hours, + ) + except FinancialConnectorObservabilityPermissionError as error: + raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(error)) from error + + +@router.get( + "/financial-connectors/admin/tenants/{tenant_id}/observability", + response_model=FinancialConnectorObservabilityRead, + summary="平台管理员读取目标租户脱敏连接器运行指标", +) +def get_tenant_connector_observability( + tenant_id: str, + db: DbSession, + current_user: PlatformAdmin, + window_hours: int = Query(default=24, ge=1, le=720), +) -> FinancialConnectorObservabilityRead: + del current_user + return FinancialConnectorObservabilityService(db).read_for_tenant( + tenant_id, + window_hours=window_hours, + ) + + +@router.get( + "/financial-connectors/payment-evidence/{claim_id}", + response_model=FinancialPaymentEvidenceRead, + summary="读取当前租户单据的付款证据等级", +) +def get_financial_payment_evidence( + claim_id: str, + db: DbSession, + current_user: CurrentUser, +) -> FinancialPaymentEvidenceRead: + claim = ExpenseClaimService(db).get_claim(claim_id, current_user) + if claim is None: + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="报销单不存在。") + return FinancialConnectorPaymentEvidenceService.read(claim) + + +@router.get( + "/financial-reconciliation/cases", + response_model=PaymentReconciliationListRead, + summary="分页读取当前租户对账记录", +) +def list_payment_reconciliation_cases( + db: DbSession, + current_user: CurrentUser, + reconciliation_status: str | None = Query(default=None, alias="status"), + page: int = Query(default=1, ge=1), + page_size: int = Query(default=20, ge=1, le=100), +) -> PaymentReconciliationListRead: + try: + return FinancialConnectorProjectionService(db).list_cases( + current_user, + status_filter=reconciliation_status, + page=page, + page_size=page_size, + ) + except FinancialReconciliationPermissionError as error: + raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(error)) from error + + +@router.get( + "/financial-reconciliation/cases/{case_id}", + response_model=PaymentReconciliationCaseDetailRead, + summary="读取当前租户脱敏对账详情和追加式时间线", +) +def get_payment_reconciliation_case( + case_id: str, + db: DbSession, + current_user: CurrentUser, +) -> PaymentReconciliationCaseDetailRead: + try: + result = FinancialConnectorProjectionService(db).detail(case_id, current_user) + except FinancialReconciliationPermissionError as error: + raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(error)) from error + if result is None: + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="对账记录不存在。") + return result + + +@router.post( + "/financial-reconciliation/cases/{case_id}/confirm", + response_model=PaymentReconciliationCaseDetailRead, + summary="财务人员确认异常处置，不伪造付款成功", +) +def confirm_payment_reconciliation_case( + case_id: str, + payload: PaymentReconciliationActionCreate, + db: DbSession, + current_user: CurrentUser, +) -> PaymentReconciliationCaseDetailRead: + return _resolve_case(case_id, payload, db, current_user, action="confirmed") + + +@router.post( + "/financial-reconciliation/cases/{case_id}/reject", + response_model=PaymentReconciliationCaseDetailRead, + summary="财务人员拒绝错误回执", +) +def reject_payment_reconciliation_case( + case_id: str, + payload: PaymentReconciliationActionCreate, + db: DbSession, + current_user: CurrentUser, +) -> PaymentReconciliationCaseDetailRead: + return _resolve_case(case_id, payload, db, current_user, action="rejected") + + +def _resolve_case( + case_id: str, + payload: PaymentReconciliationActionCreate, + db: Session, + current_user: CurrentUserContext, + *, + action: str, +) -> PaymentReconciliationCaseDetailRead: + try: + result = FinancialConnectorProjectionService(db).resolve( + case_id, + payload, + current_user, + action=action, + ) + if result is None: + db.rollback() + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="对账记录不存在。") + db.commit() + return result + except HTTPException: + raise + except FinancialReconciliationPermissionError as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(error)) from error + except FinancialReconciliationConflictError as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(error)) from error + + +def _change_config_status( + tenant_id: str, + config_id: str, + payload: FinancialConnectorConfigLifecycleAction, + db: Session, + current_user: CurrentUserContext, + *, + action: str, +) -> FinancialConnectorConfigRead: + service = FinancialConnectorConfigLifecycleService(db) + try: + operation = service.activate if action == "activate" else service.disable + row = operation( + tenant_id=tenant_id, + config_id=config_id, + payload=payload, + actor_id=current_user.username, + ) + db.commit() + db.refresh(row) + return FinancialConnectorConfigRead.model_validate(row) + except LookupError as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) from error + except FinancialConnectorAuthError as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail={"code": error.code, "message": str(error)}, + ) from error + except (FinancialConnectorConfigConflictError, IntegrityError) as error: + db.rollback() + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(error)) from error + + +def _persist_operational_event( + db: Session, + candidate: FinancialConnectorOperationalEventCandidate, +) -> None: + """失败响应回滚后独立提交运营事实，写入失败不得改变原始业务结果。""" + + try: + FinancialConnectorOperationalEventService(db).record(candidate) + db.commit() + except Exception: # noqa: BLE001 - 审计降级不能覆盖原始 401/409 契约 + db.rollback() + logger.exception( + "financial_connector_operational_event_persist_failed " + "event_type=%s reason_code=%s config_id=%s", + candidate.event_type, + candidate.reason_code, + candidate.context.config_id, + ) diff --git a/server/src/app/api/v1/endpoints/knowledge.py b/server/src/app/api/v1/endpoints/knowledge.py index 37f960e..79e05ea 100644 --- a/server/src/app/api/v1/endpoints/knowledge.py +++ b/server/src/app/api/v1/endpoints/knowledge.py @@ -4,8 +4,7 @@ from typing import Annotated from fastapi import APIRouter, Body, Depends, HTTPException, Query, status from fastapi.responses import FileResponse -from sqlalchemy import select -from sqlalchemy.orm import Session +from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext, get_current_user, get_db, require_admin_user from app.core.agent_enums import AgentName, AgentRunSource @@ -24,9 +23,17 @@ from app.schemas.knowledge import ( LlmWikiSyncWrite, ) from app.services.agent_runs import AgentRunService -from app.services.knowledge import ( - KnowledgeService, -) +from app.services.knowledge import ( + KnowledgeService, +) +from app.services.knowledge_onlyoffice_callback import ( + handle_onlyoffice_callback, + resolve_onlyoffice_content, +) +from app.services.knowledge_onlyoffice_security import ( + OnlyOfficeReplayError, + OnlyOfficeSecurityError, +) from app.services.knowledge_sync import KnowledgeSyncDispatchService router = APIRouter(prefix="/knowledge") @@ -44,11 +51,11 @@ router = APIRouter(prefix="/knowledge") } }, ) -def get_knowledge_library( - _: Annotated[CurrentUserContext, Depends(get_current_user)], - db: Annotated[Session, Depends(get_db)], -) -> KnowledgeLibraryRead: - return KnowledgeService(db=db).list_library() +def get_knowledge_library( + current_user: Annotated[CurrentUserContext, Depends(get_current_user)], + db: Annotated[Session, Depends(get_db)], +) -> KnowledgeLibraryRead: + return KnowledgeService(db=db, tenant_id=current_user.tenant_id).list_library() @router.get( @@ -67,14 +74,18 @@ def get_knowledge_library( }, }, ) -def get_llm_wiki_index( - _: Annotated[CurrentUserContext, Depends(require_admin_user)], +def get_llm_wiki_index( + current_user: Annotated[CurrentUserContext, Depends(require_admin_user)], db: Annotated[Session, Depends(get_db)], ) -> LlmWikiIndexRead: run_service = AgentRunService(db) sync_runs = [ item - for item in run_service.list_runs(agent=AgentName.HERMES.value, limit=200) + for item in run_service.list_runs_for_tenant( + tenant_id=current_user.tenant_id, + agent=AgentName.HERMES.value, + limit=200, + ) if str(item.route_json.get("job_type") or "").strip() == "knowledge_index_sync" ] return LlmWikiIndexRead(documents=[], sync_run_count=len(sync_runs)) @@ -198,7 +209,10 @@ def sync_knowledge_library( raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc if isinstance(exc, FileNotFoundError): raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc - raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=str(exc)) from exc + raise HTTPException( + status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, + detail=str(exc), + ) from exc @router.get( "/documents/{document_id}", @@ -216,13 +230,16 @@ def sync_knowledge_library( }, }, ) -def get_knowledge_document( - document_id: str, - _: Annotated[CurrentUserContext, Depends(get_current_user)], +def get_knowledge_document( + document_id: str, + current_user: Annotated[CurrentUserContext, Depends(get_current_user)], db: Annotated[Session, Depends(get_db)], ) -> KnowledgeDocumentDetailRead: try: - return KnowledgeService(db=db).get_document_detail(document_id) + return KnowledgeService( + db=db, + tenant_id=current_user.tenant_id, + ).get_document_detail(document_id) except FileNotFoundError as exc: raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, @@ -250,12 +267,24 @@ def get_knowledge_document( }, }, ) -def get_knowledge_document_onlyoffice_config( - document_id: str, - current_user: Annotated[CurrentUserContext, Depends(get_current_user)], -) -> KnowledgeOnlyOfficeConfigRead: - try: - return KnowledgeService().build_onlyoffice_config(document_id, current_user) +def get_knowledge_document_onlyoffice_config( + document_id: str, + current_user: Annotated[CurrentUserContext, Depends(get_current_user)], + db: Annotated[Session, Depends(get_db)], + editable: Annotated[ + bool, + Query(description="是否申请租户知识文档编辑会话；默认仅预览。"), + ] = False, +) -> KnowledgeOnlyOfficeConfigRead: + try: + return KnowledgeService( + db=db, + tenant_id=current_user.tenant_id, + ).build_onlyoffice_config( + document_id, + current_user, + editable=editable, + ) except FileNotFoundError as exc: raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, @@ -286,7 +315,7 @@ def get_knowledge_document_onlyoffice_config( }, }, ) -def upload_knowledge_document( +def upload_knowledge_document( content: Annotated[ bytes, Body( @@ -296,10 +325,14 @@ def upload_knowledge_document( ], folder: Annotated[str, Query(min_length=1, description="目标知识库目录名称。")], filename: Annotated[str, Query(min_length=1, description="原始文件名。")], - current_user: Annotated[CurrentUserContext, Depends(require_admin_user)], -) -> KnowledgeDocumentDetailRead: - try: - return KnowledgeService().upload_document(folder, filename, content, current_user) + current_user: Annotated[CurrentUserContext, Depends(require_admin_user)], + db: Annotated[Session, Depends(get_db)], +) -> KnowledgeDocumentDetailRead: + try: + return KnowledgeService( + db=db, + tenant_id=current_user.tenant_id, + ).upload_document(folder, filename, content, current_user) except ValueError as exc: raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc @@ -324,12 +357,16 @@ def upload_knowledge_document( }, }, ) -def delete_knowledge_document( - document_id: str, - _: Annotated[CurrentUserContext, Depends(require_admin_user)], -) -> KnowledgeActionResponse: - try: - KnowledgeService().delete_document(document_id) +def delete_knowledge_document( + document_id: str, + current_user: Annotated[CurrentUserContext, Depends(require_admin_user)], + db: Annotated[Session, Depends(get_db)], +) -> KnowledgeActionResponse: + try: + KnowledgeService( + db=db, + tenant_id=current_user.tenant_id, + ).delete_document(document_id) except FileNotFoundError as exc: raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, @@ -359,19 +396,23 @@ def delete_knowledge_document( }, }, ) -def get_knowledge_document_content( - document_id: str, - disposition: Annotated[ - str, - Query( +def get_knowledge_document_content( + document_id: str, + current_user: Annotated[CurrentUserContext, Depends(get_current_user)], + db: Annotated[Session, Depends(get_db)], + disposition: Annotated[ + str, + Query( pattern="^(inline|attachment)$", - description="内容展示方式，支持 `inline` 或 `attachment`。", - ), - ] = "inline", - _: Annotated[CurrentUserContext, Depends(get_current_user)] = None, -) -> FileResponse: - try: - file_path, media_type, filename = KnowledgeService().get_document_content(document_id) + description="内容展示方式，支持 `inline` 或 `attachment`。", + ), + ] = "inline", +) -> FileResponse: + try: + file_path, media_type, filename = KnowledgeService( + db=db, + tenant_id=current_user.tenant_id, + ).get_document_content(document_id) except FileNotFoundError as exc: raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, @@ -402,24 +443,28 @@ def get_knowledge_document_content( }, }, ) -def get_knowledge_document_onlyoffice_content( +def get_knowledge_document_onlyoffice_content( document_id: str, - access_token: Annotated[ + access_token: Annotated[ str, Query(min_length=1, description="ONLYOFFICE 临时访问令牌。"), - ], -) -> FileResponse: - try: - service = KnowledgeService() - service.validate_onlyoffice_access_token(document_id, access_token) - file_path, media_type, filename = service.get_document_content(document_id) + ], + db: Annotated[Session, Depends(get_db)], +) -> FileResponse: + try: + file_path, media_type, filename = resolve_onlyoffice_content( + db=db, + storage_root=None, + document_id=document_id, + access_token=access_token, + ) except FileNotFoundError as exc: raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, detail="知识库文件不存在。", ) from exc - except ValueError as exc: - raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc)) from exc + except (OnlyOfficeSecurityError, OnlyOfficeReplayError) as exc: + raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc)) from exc return FileResponse(file_path, media_type=media_type, filename=filename) @@ -440,18 +485,33 @@ def get_knowledge_document_onlyoffice_content( }, }, ) -def handle_knowledge_document_onlyoffice_callback( - document_id: str, - payload: KnowledgeOnlyOfficeCallbackWrite, -) -> KnowledgeOnlyOfficeCallbackRead: - try: - KnowledgeService().handle_onlyoffice_callback(document_id, payload.model_dump()) +def handle_knowledge_document_onlyoffice_callback( + document_id: str, + payload: KnowledgeOnlyOfficeCallbackWrite, + callback_token: Annotated[ + str, + Query(min_length=1, description="绑定租户与文档的一次性回调会话令牌。"), + ], + db: Annotated[Session, Depends(get_db)], +) -> KnowledgeOnlyOfficeCallbackRead: + try: + handle_onlyoffice_callback( + db=db, + storage_root=None, + document_id=document_id, + callback_token=callback_token, + payload=payload.model_dump(), + ) except FileNotFoundError as exc: raise HTTPException( status_code=status.HTTP_404_NOT_FOUND, detail="知识库文件不存在。", ) from exc - except ValueError as exc: - raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc + except OnlyOfficeReplayError as exc: + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from exc + except OnlyOfficeSecurityError as exc: + raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail=str(exc)) from exc + except ValueError as exc: + raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc return KnowledgeOnlyOfficeCallbackRead() diff --git a/server/src/app/api/v1/endpoints/ocr.py b/server/src/app/api/v1/endpoints/ocr.py index 6e69b1c..56334da 100644 --- a/server/src/app/api/v1/endpoints/ocr.py +++ b/server/src/app/api/v1/endpoints/ocr.py @@ -2,7 +2,7 @@ from __future__ import annotations from typing import Annotated -from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile, status +from fastapi import APIRouter, Depends, File, Form, Header, HTTPException, UploadFile, status from sqlalchemy.orm import Session from starlette.concurrency import run_in_threadpool @@ -10,6 +10,11 @@ from app.api.deps import CurrentUserContext, get_current_user, get_db from app.schemas.common import ErrorResponse from app.schemas.ocr import OcrRecognizeBatchRead from app.services.ocr import OcrService +from app.services.ocr_commercial import ( + OcrCommercialAccessDenied, + content_digest, + trusted_ocr_operation_context, +) from app.services.receipt_folder import ReceiptFolderService router = APIRouter(prefix="/ocr") @@ -29,6 +34,10 @@ router = APIRouter(prefix="/ocr") "model": ErrorResponse, "description": "未提供当前登录用户。", }, + status.HTTP_429_TOO_MANY_REQUESTS: { + "model": ErrorResponse, + "description": "OCR 商业额度不足或计量配置不允许本次真实执行。", + }, status.HTTP_503_SERVICE_UNAVAILABLE: { "model": ErrorResponse, "description": "OCR 运行时不可用或执行失败。", @@ -39,7 +48,10 @@ async def recognize_ocr_documents( files: Annotated[list[UploadFile], File(description="待识别的票据图片或 PDF。")], current_user: Annotated[CurrentUserContext, Depends(get_current_user)], db: Annotated[Session, Depends(get_db)], - receipt_ids: Annotated[list[str] | None, Form(description="可选，来源于票据夹的持久化票据 ID。")] = None, + receipt_ids: Annotated[ + list[str] | None, Form(description="可选，来源于票据夹的持久化票据 ID。") + ] = None, + x_request_id: Annotated[str | None, Header(alias="X-Request-ID")] = None, ) -> OcrRecognizeBatchRead: try: payload = [] @@ -51,7 +63,15 @@ async def recognize_ocr_documents( upload.content_type, ) ) - result = await run_in_threadpool(lambda: OcrService(db).recognize_files(payload)) + operation_context = trusted_ocr_operation_context( + current_user, + operation_scope="ocr-endpoint", + content_digests=[content_digest(content) for _, content, _ in payload], + request_id=x_request_id or "", + ) + result = await run_in_threadpool( + lambda: OcrService(db, operation_context=operation_context).recognize_files(payload) + ) return ReceiptFolderService().persist_ocr_batch( files=payload, result=result, @@ -60,6 +80,11 @@ async def recognize_ocr_documents( ) except ValueError as exc: raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc + except OcrCommercialAccessDenied as exc: + raise HTTPException( + status_code=status.HTTP_429_TOO_MANY_REQUESTS, + detail=str(exc), + ) from exc except RuntimeError as exc: raise HTTPException( status_code=status.HTTP_503_SERVICE_UNAVAILABLE, diff --git a/server/src/app/api/v1/endpoints/ontology.py b/server/src/app/api/v1/endpoints/ontology.py index 8438bcf..62eea96 100644 --- a/server/src/app/api/v1/endpoints/ontology.py +++ b/server/src/app/api/v1/endpoints/ontology.py @@ -5,13 +5,14 @@ from typing import Annotated from fastapi import APIRouter, Depends, HTTPException, status from sqlalchemy.orm import Session -from app.api.deps import get_db +from app.api.deps import CurrentUserContext, get_current_user, get_db from app.schemas.common import ErrorResponse from app.schemas.ontology import OntologyParseRequest, OntologyParseResult from app.services.ontology import SemanticOntologyService router = APIRouter(prefix="/ontology") DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] @router.post( @@ -19,8 +20,7 @@ DbSession = Annotated[Session, Depends(get_db)] response_model=OntologyParseResult, summary="解析自然语言为语义本体", description=( - "把自然语言问题解析成 Day 3 约定的 8 个核心字段，" - "并写入 AgentRun 与 SemanticParseLog。" + "把自然语言问题解析成 Day 3 约定的 8 个核心字段，并写入 AgentRun 与 SemanticParseLog。" ), responses={ status.HTTP_400_BAD_REQUEST: { @@ -29,8 +29,15 @@ DbSession = Annotated[Session, Depends(get_db)] } }, ) -def parse_ontology(payload: OntologyParseRequest, db: DbSession) -> OntologyParseResult: +def parse_ontology( + payload: OntologyParseRequest, + db: DbSession, + current_user: CurrentUser, +) -> OntologyParseResult: try: - return SemanticOntologyService(db).parse(payload) + return SemanticOntologyService(db).parse( + payload.model_copy(update={"user_id": current_user.username}), + tenant_id=current_user.tenant_id, + ) except ValueError as exc: raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc diff --git a/server/src/app/api/v1/endpoints/reimbursement_approval_actions.py b/server/src/app/api/v1/endpoints/reimbursement_approval_actions.py index 721e3e2..7f61954 100644 --- a/server/src/app/api/v1/endpoints/reimbursement_approval_actions.py +++ b/server/src/app/api/v1/endpoints/reimbursement_approval_actions.py @@ -24,7 +24,12 @@ DbSession = Annotated[Session, Depends(get_db)] CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] -def _raise_action_error(error: ValueError) -> NoReturn: +def _raise_action_error(error: Exception) -> NoReturn: + if isinstance(error, LookupError): + raise HTTPException( + status_code=status.HTTP_404_NOT_FOUND, + detail=str(error), + ) from error if isinstance(error, ExpenseClaimRiskBlockedError): raise HTTPException( status_code=status.HTTP_409_CONFLICT, @@ -87,7 +92,7 @@ def return_expense_claim( task_id=payload.task_id, expected_task_version=payload.expected_task_version, ) - except ValueError as error: + except (LookupError, ValueError) as error: _raise_action_error(error) if claim is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Claim not found") @@ -128,7 +133,7 @@ def approve_expense_claim( task_id=payload.task_id, expected_task_version=payload.expected_task_version, ) - except ValueError as error: + except (LookupError, ValueError) as error: _raise_action_error(error) if claim is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Claim not found") @@ -166,7 +171,7 @@ def pay_expense_claim( expected_status=payload.expected_status, expected_approval_stage=payload.expected_approval_stage, ) - except ValueError as error: + except (LookupError, ValueError) as error: _raise_action_error(error) if claim is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Claim not found") diff --git a/server/src/app/api/v1/endpoints/reimbursements.py b/server/src/app/api/v1/endpoints/reimbursements.py index 3bef5b2..dda78d0 100644 --- a/server/src/app/api/v1/endpoints/reimbursements.py +++ b/server/src/app/api/v1/endpoints/reimbursements.py @@ -27,6 +27,9 @@ from app.schemas.reimbursement import ( TravelReimbursementCalculatorResponse, ) from app.services.budget import BudgetService +from app.services.expense_claim_attachment_commercial import ( + ExpenseClaimAttachmentCommercialAccessDenied, +) from app.services.expense_claims import ExpenseClaimService from app.services.reimbursement import ReimbursementService from app.services.travel_reimbursement_calculator import TravelReimbursementCalculatorService @@ -393,9 +396,14 @@ def delete_expense_claim_item( current_user=current_user, ) except LookupError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) from error except ValueError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) from error + except Exception: + db.rollback() + raise if payload is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Claim not found") @@ -427,6 +435,7 @@ async def upload_expense_claim_item_attachment( receipt_id: Annotated[ str | None, Form(description="可选，来源于票据夹的持久化票据 ID。") ] = None, + request_id: RequestIdHeader = None, ) -> ExpenseClaimAttachmentActionResponse: service = ExpenseClaimService(db) try: @@ -438,11 +447,23 @@ async def upload_expense_claim_item_attachment( media_type=file.content_type, current_user=current_user, source_receipt_id=receipt_id or "", + request_id=request_id or "", ) except LookupError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) from error + except ExpenseClaimAttachmentCommercialAccessDenied as error: + db.rollback() + raise HTTPException( + status_code=status.HTTP_429_TOO_MANY_REQUESTS, + detail=str(error), + ) from error except ValueError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) from error + except Exception: + db.rollback() + raise if payload is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Claim not found") @@ -588,11 +609,17 @@ def delete_expense_claim_item_attachment( current_user=current_user, ) except LookupError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) from error except FileNotFoundError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) from error except ValueError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) from error + except Exception: + db.rollback() + raise if payload is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Claim not found") @@ -682,7 +709,11 @@ def delete_expense_claim( try: claim = service.delete_claim(claim_id, current_user) except ValueError as error: + db.rollback() raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) from error + except Exception: + db.rollback() + raise if claim is None: raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Claim not found") diff --git a/server/src/app/api/v1/endpoints/savings.py b/server/src/app/api/v1/endpoints/savings.py new file mode 100644 index 0000000..12cceed --- /dev/null +++ b/server/src/app/api/v1/endpoints/savings.py @@ -0,0 +1,235 @@ +from __future__ import annotations + +from datetime import datetime +from typing import Annotated, Literal + +from fastapi import APIRouter, Depends, HTTPException, Query, status +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.schemas.savings import ( + SavingsOpportunityActionCreate, + SavingsOpportunityListRead, + SavingsOpportunityMutationRead, + SavingsOpportunityRead, + SavingsRealizationActionCreate, + SavingsRealizationCreate, + SavingsRealizationMutationRead, +) +from app.schemas.savings_insights import ( + SavingsBaselineGenerateRequest, + SavingsBaselineGenerationRead, + SavingsInsightAnalysisRead, + SavingsInsightAnalyzeRequest, +) +from app.services.savings_access_policy import SavingsPermissionError +from app.services.savings_actions import SavingsActionService, SavingsTransitionError +from app.services.savings_baseline_generation import SavingsBaselineGenerationService +from app.services.savings_insight_analysis import SavingsInsightAnalysisService +from app.services.savings_protocol import ( + SavingsIdempotencyConflictError, + SavingsVersionConflictError, +) +from app.services.savings_query import SavingsQueryService +from app.services.savings_realization import ( + SavingsRealizationError, + SavingsRealizationService, +) + +router = APIRouter(prefix="/savings") +DbSession = Annotated[Session, Depends(get_db)] +CurrentUser = Annotated[CurrentUserContext, Depends(get_current_user)] + + +@router.post( + "/baselines/generate", + response_model=SavingsBaselineGenerationRead, + summary="冻结租户费用历史基线", +) +def generate_savings_baselines( + payload: SavingsBaselineGenerateRequest, + db: DbSession, + current_user: CurrentUser, +) -> SavingsBaselineGenerationRead: + try: + return SavingsBaselineGenerationService(db).generate(payload, current_user) + except Exception as error: + raise _mutation_http_error(error) from error + + +@router.post( + "/insights/analyze", + response_model=SavingsInsightAnalysisRead, + summary="分析费用节省候选信号", +) +def analyze_savings_insights( + payload: SavingsInsightAnalyzeRequest, + db: DbSession, + current_user: CurrentUser, +) -> SavingsInsightAnalysisRead: + try: + return SavingsInsightAnalysisService(db).analyze(payload, current_user) + except Exception as error: + raise _mutation_http_error(error) from error + + +@router.get( + "/opportunities", + response_model=SavingsOpportunityListRead, + summary="查询节省机会台账", +) +def list_savings_opportunities( + db: DbSession, + current_user: CurrentUser, + page: Annotated[int, Query(ge=1)] = 1, + page_size: Annotated[int, Query(ge=1, le=100)] = 20, + status_value: Annotated[str | None, Query(alias="status", max_length=24)] = None, + source_type: Annotated[str | None, Query(max_length=50)] = None, + value_kind: Annotated[Literal["cash", "labor"] | None, Query()] = None, + department_id: Annotated[str | None, Query(max_length=160)] = None, + project_code: Annotated[str | None, Query(max_length=160)] = None, + expense_type: Annotated[str | None, Query(max_length=80)] = None, + supplier_id: Annotated[str | None, Query(max_length=160)] = None, + city: Annotated[str | None, Query(max_length=160)] = None, + owner_id: Annotated[str | None, Query(max_length=120)] = None, + claim_id: Annotated[str | None, Query(max_length=36)] = None, + created_from: datetime | None = None, + created_to: datetime | None = None, + sort: Annotated[ + Literal["created_desc", "created_asc", "due_asc", "estimated_desc"], + Query(), + ] = "created_desc", +) -> SavingsOpportunityListRead: + return SavingsQueryService(db).list_opportunities( + current_user, + page=page, + page_size=page_size, + status=status_value, + source_type=source_type, + value_kind=value_kind, + department_id=department_id, + project_code=project_code, + expense_type=expense_type, + supplier_id=supplier_id, + city=city, + owner_id=owner_id, + claim_id=claim_id, + created_from=created_from, + created_to=created_to, + sort=sort, + ) + + +@router.get( + "/opportunities/{opportunity_id}", + response_model=SavingsOpportunityRead, + summary="读取节省机会证据链", +) +def get_savings_opportunity( + opportunity_id: str, + db: DbSession, + current_user: CurrentUser, +) -> SavingsOpportunityRead: + opportunity = SavingsQueryService(db).get_opportunity(opportunity_id, current_user) + if opportunity is None: + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="节省机会不存在。") + return opportunity + + +@router.post( + "/opportunities/{opportunity_id}/actions", + response_model=SavingsOpportunityMutationRead, + summary="执行节省机会状态动作", +) +def execute_savings_opportunity_action( + opportunity_id: str, + payload: SavingsOpportunityActionCreate, + db: DbSession, + current_user: CurrentUser, +) -> SavingsOpportunityMutationRead: + try: + return ( + SavingsActionService(db) + .execute( + opportunity_id, + payload, + current_user, + ) + .response + ) + except Exception as error: + raise _mutation_http_error(error) from error + + +@router.post( + "/opportunities/{opportunity_id}/realizations", + response_model=SavingsRealizationMutationRead, + summary="记录节省机会实际结果", +) +def record_savings_realization( + opportunity_id: str, + payload: SavingsRealizationCreate, + db: DbSession, + current_user: CurrentUser, +) -> SavingsRealizationMutationRead: + try: + return ( + SavingsRealizationService(db) + .record( + opportunity_id, + payload, + current_user, + ) + .response + ) + except Exception as error: + raise _mutation_http_error(error) from error + + +@router.post( + "/realizations/{realization_id}/actions", + response_model=SavingsRealizationMutationRead, + summary="确认、拒绝或冲回实际节省", +) +def execute_savings_realization_action( + realization_id: str, + payload: SavingsRealizationActionCreate, + db: DbSession, + current_user: CurrentUser, +) -> SavingsRealizationMutationRead: + try: + return ( + SavingsRealizationService(db) + .execute_action( + realization_id, + payload, + current_user, + ) + .response + ) + except Exception as error: + raise _mutation_http_error(error) from error + + +def _mutation_http_error(error: Exception) -> HTTPException: + if isinstance(error, LookupError): + return HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(error)) + if isinstance(error, SavingsPermissionError): + return HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=str(error)) + if isinstance(error, SavingsVersionConflictError): + return HTTPException( + status_code=status.HTTP_409_CONFLICT, + detail={"message": str(error), "current_version": error.current_version}, + ) + if isinstance( + error, + ( + SavingsIdempotencyConflictError, + SavingsTransitionError, + SavingsRealizationError, + ), + ): + return HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(error)) + if isinstance(error, (ValueError, PermissionError)): + return HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(error)) + raise error diff --git a/server/src/app/api/v1/endpoints/steward.py b/server/src/app/api/v1/endpoints/steward.py index 417e967..6653491 100644 --- a/server/src/app/api/v1/endpoints/steward.py +++ b/server/src/app/api/v1/endpoints/steward.py @@ -27,6 +27,7 @@ from app.schemas.steward import ( ) from app.services.agent_conversations import AgentConversationService from app.services.expense_claim_draft_flow import APPROVED_APPLICATION_LINK_STATUSES +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin from app.services.expense_claims import ExpenseClaimService from app.services.runtime_chat import RuntimeChatService from app.services.steward_context_resume import ( @@ -61,15 +62,30 @@ StewardPlannerLike = StewardPlannerService | StewardGraphPlannerService } }, ) -def create_steward_plan(payload: StewardPlanRequest, db: DbSession) -> StewardPlanResponse: +def create_steward_plan( + payload: StewardPlanRequest, + db: DbSession, + current_user: CurrentUser, +) -> StewardPlanResponse: try: + payload = _bind_authenticated_plan_request(payload, current_user) planner = _build_steward_planner(db) - hydrated_payload = _hydrate_required_application_gate(db, payload, planner) + hydrated_payload = _hydrate_required_application_gate( + db, + payload, + planner, + tenant_id=_require_steward_tenant_id(current_user), + ) if isinstance(planner, StewardGraphPlannerService): plan = planner.build_plan(hydrated_payload, db=db) else: plan = planner.build_plan(hydrated_payload) - return _attach_conversation_state(db, hydrated_payload, plan) + return _attach_conversation_state( + db, + hydrated_payload, + plan, + current_user=current_user, + ) except ValueError as exc: raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc @@ -83,8 +99,10 @@ def create_steward_plan(payload: StewardPlanRequest, db: DbSession) -> StewardPl def create_steward_slot_decision( payload: StewardSlotDecisionRequest, db: DbSession, + current_user: CurrentUser, ) -> StewardSlotDecisionResponse: - return _decide_steward_slot(payload, RuntimeChatService(db)) + authenticated_payload = _bind_authenticated_slot_request(payload, current_user) + return _decide_steward_slot(authenticated_payload, RuntimeChatService(db)) @router.post( @@ -96,10 +114,21 @@ def create_steward_slot_decision( def create_steward_runtime_decision( payload: StewardRuntimeDecisionRequest, db: DbSession, + current_user: CurrentUser, ) -> StewardRuntimeDecisionResponse: - hydrated_payload = _hydrate_runtime_decision_payload(db, payload) + authenticated_payload = _bind_authenticated_runtime_request(payload, current_user) + hydrated_payload = _hydrate_runtime_decision_payload( + db, + authenticated_payload, + current_user=current_user, + ) decision = _decide_steward_runtime(hydrated_payload, RuntimeChatService(db)) - return _attach_runtime_conversation_state(db, hydrated_payload, decision) + return _attach_runtime_conversation_state( + db, + hydrated_payload, + decision, + current_user=current_user, + ) @router.post( @@ -124,9 +153,19 @@ def execute_steward_action( summary="流式生成小财管家任务计划", description="以 NDJSON 逐条返回小财管家的过程摘要事件，最后返回完整任务计划。", ) -async def stream_steward_plan(payload: StewardPlanRequest, db: DbSession) -> StreamingResponse: +async def stream_steward_plan( + payload: StewardPlanRequest, + db: DbSession, + current_user: CurrentUser, +) -> StreamingResponse: + authenticated_payload = _bind_authenticated_plan_request(payload, current_user) return StreamingResponse( - _iter_steward_plan_events(payload, _build_steward_planner(db), db), + _iter_steward_plan_events( + authenticated_payload, + _build_steward_planner(db), + db, + current_user=current_user, + ), media_type="application/x-ndjson", ) @@ -135,6 +174,8 @@ async def _iter_steward_plan_events( payload: StewardPlanRequest, planner: StewardPlannerLike, db: Session, + *, + current_user: CurrentUserContext, ) -> AsyncIterator[str]: yield _encode_stream_event( "thinking", @@ -149,12 +190,22 @@ async def _iter_steward_plan_events( await asyncio.sleep(0) try: - hydrated_payload = _hydrate_required_application_gate(db, payload, planner) + hydrated_payload = _hydrate_required_application_gate( + db, + payload, + planner, + tenant_id=_require_steward_tenant_id(current_user), + ) if isinstance(planner, StewardGraphPlannerService): plan = planner.build_plan(hydrated_payload, db=db) else: plan = planner.build_plan(hydrated_payload) - plan = _attach_conversation_state(db, hydrated_payload, plan) + plan = _attach_conversation_state( + db, + hydrated_payload, + plan, + current_user=current_user, + ) except ValueError as exc: yield _encode_stream_event("error", {"message": str(exc)}) return @@ -170,6 +221,114 @@ def _encode_stream_event(event: str, data: dict[str, Any]) -> str: return json.dumps({"event": event, "data": data}, ensure_ascii=False) + "\n" +def _bind_authenticated_plan_request( + payload: StewardPlanRequest, + current_user: CurrentUserContext, +) -> StewardPlanRequest: + user_id = _require_steward_user_id(current_user) + return payload.model_copy( + update={ + "user_id": user_id, + "context_json": _build_trusted_steward_context( + payload.context_json, + current_user, + ), + } + ) + + +def _bind_authenticated_slot_request( + payload: StewardSlotDecisionRequest, + current_user: CurrentUserContext, +) -> StewardSlotDecisionRequest: + return payload.model_copy( + update={ + "task_context": _build_trusted_steward_context( + payload.task_context, + current_user, + ) + } + ) + + +def _bind_authenticated_runtime_request( + payload: StewardRuntimeDecisionRequest, + current_user: CurrentUserContext, +) -> StewardRuntimeDecisionRequest: + return payload.model_copy( + update={ + "context_json": _build_trusted_steward_context( + payload.context_json, + current_user, + ), + "runtime_state": _build_trusted_steward_context( + payload.runtime_state, + current_user, + ), + } + ) + + +def _build_trusted_steward_context( + context_json: dict[str, Any] | None, + current_user: CurrentUserContext, +) -> dict[str, Any]: + tenant_id = _require_steward_tenant_id(current_user) + user_id = _require_steward_user_id(current_user) + trusted_context = dict(context_json or {}) + for untrusted_alias in ( + "tenantId", + "userId", + "auth_session_id", + ): + trusted_context.pop(untrusted_alias, None) + trusted_context.update( + { + "tenant_id": tenant_id, + "user_id": user_id, + "username": current_user.username, + "name": current_user.name, + "role_codes": list(current_user.role_codes), + "is_admin": current_user.is_admin, + "department": current_user.department_name, + "department_name": current_user.department_name, + "department_id": current_user.department_id, + "cost_center": current_user.cost_center, + "position": current_user.position, + "grade": current_user.grade, + "employee_grade": current_user.grade, + "employee_no": current_user.employee_no, + "employee_id": current_user.employee_id, + "manager_name": current_user.manager_name, + "requested_by_username": current_user.username, + "requested_by_name": current_user.name, + "actor": user_id, + "actor_id": user_id, + } + ) + return trusted_context + + +def _require_steward_tenant_id(current_user: CurrentUserContext) -> str: + tenant_id = str(current_user.tenant_id or "").strip() + if tenant_id: + return tenant_id + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail="当前登录用户缺少租户归属，无法使用小财管家。", + ) + + +def _require_steward_user_id(current_user: CurrentUserContext) -> str: + user_id = str(current_user.username or current_user.employee_id or "").strip() + if user_id: + return user_id + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail="当前登录用户缺少可信用户标识，无法使用小财管家。", + ) + + def _build_steward_planner(db: Session) -> StewardPlannerLike: runtime_chat = RuntimeChatService(db) if get_settings().steward_agent_runtime.strip().lower() == "langgraph": @@ -217,6 +376,8 @@ def _hydrate_required_application_gate( db: Session, payload: StewardPlanRequest, planner: StewardPlannerLike, + *, + tenant_id: str, ) -> StewardPlanRequest: context_json = dict(payload.context_json or {}) required_gate = context_json.get("required_application_gate") @@ -230,7 +391,12 @@ def _hydrate_required_application_gate( if not planner._looks_like_ambiguous_travel_flow(message, base_date, payload): return payload - candidates = _query_required_application_gate_candidates(db, payload, context_json) + candidates = _query_required_application_gate_candidates( + db, + payload, + context_json, + tenant_id=tenant_id, + ) next_required_gate = dict(required_gate) if isinstance(required_gate, dict) else {} next_required_gate["travel"] = { "checked": True, @@ -251,10 +417,15 @@ def _query_required_application_gate_candidates( db: Session, payload: StewardPlanRequest, context_json: dict[str, Any], + *, + tenant_id: str, ) -> list[dict[str, Any]]: identities = _resolve_required_application_gate_identities(payload, context_json) stmt = ( select(ExpenseClaim) + .where( + ExpenseClaimTenantScopeMixin.build_claim_tenant_condition(tenant_id) + ) .order_by(ExpenseClaim.submitted_at.desc(), ExpenseClaim.updated_at.desc()) .limit(200) ) @@ -383,16 +554,19 @@ def _attach_conversation_state( db: Session, payload: StewardPlanRequest, plan: StewardPlanResponse, + *, + current_user: CurrentUserContext, ) -> StewardPlanResponse: context_json = dict(payload.context_json or {}) context_json["session_type"] = str(context_json.get("session_type") or "steward").strip() or "steward" conversation_service = AgentConversationService(db) conversation = conversation_service.get_or_create_conversation( conversation_id=_resolve_conversation_id(context_json), - user_id=payload.user_id, + user_id=_require_steward_user_id(current_user), source="user_message", context_json=context_json, ) + _require_steward_conversation_access(conversation, current_user) current_state = _resolve_current_steward_state(conversation.state_json, context_json) steward_state = StewardFlowStateService().merge_plan(current_state, plan) conversation = conversation_service.update_state( @@ -433,6 +607,8 @@ def _attach_runtime_conversation_state( db: Session, payload: StewardRuntimeDecisionRequest, decision: StewardRuntimeDecisionResponse, + *, + current_user: CurrentUserContext, ) -> StewardRuntimeDecisionResponse: steward_state = decision.steward_state if not isinstance(steward_state, dict) or not steward_state: @@ -443,6 +619,10 @@ def _attach_runtime_conversation_state( return decision conversation_service = AgentConversationService(db) + conversation = conversation_service.get_conversation(conversation_id) + if conversation is None: + return decision + _require_steward_conversation_access(conversation, current_user) conversation_service.update_state( conversation_id=conversation_id, run_id=None, @@ -459,18 +639,26 @@ def _attach_runtime_conversation_state( def _hydrate_runtime_decision_payload( db: Session, payload: StewardRuntimeDecisionRequest, + *, + current_user: CurrentUserContext, ) -> StewardRuntimeDecisionRequest: context_json = dict(payload.context_json or {}) runtime_state = dict(payload.runtime_state or {}) + conversation_id = _resolve_conversation_id(context_json) + conversation = ( + AgentConversationService(db).get_conversation(conversation_id) + if conversation_id + else None + ) + if conversation is not None: + _require_steward_conversation_access(conversation, current_user) if isinstance(runtime_state.get("steward_state"), dict) and runtime_state["steward_state"]: return payload if isinstance(context_json.get("steward_state"), dict) and context_json["steward_state"]: return payload - conversation_id = _resolve_conversation_id(context_json) if not conversation_id: return payload - conversation = AgentConversationService(db).get_conversation(conversation_id) stored_state = conversation.state_json.get("steward_state") if conversation and isinstance(conversation.state_json, dict) else None if not isinstance(stored_state, dict) or not stored_state: return payload @@ -487,6 +675,30 @@ def _hydrate_runtime_decision_payload( ) +def _require_steward_conversation_access( + conversation: Any, + current_user: CurrentUserContext, +) -> None: + expected_tenant_id = _require_steward_tenant_id(current_user) + expected_user_id = _require_steward_user_id(current_user) + state_json = ( + dict(conversation.state_json) + if isinstance(conversation.state_json, dict) + else {} + ) + conversation_tenant_id = str(state_json.get("tenant_id") or "").strip() + conversation_user_id = str(conversation.user_id or "").strip() + if ( + conversation_tenant_id == expected_tenant_id + and conversation_user_id == expected_user_id + ): + return + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail="当前会话不属于登录用户或租户，已拒绝访问。", + ) + + def _resolve_conversation_id(context_json: dict[str, Any]) -> str | None: return str( context_json.get("conversation_id") diff --git a/server/src/app/api/v1/router.py b/server/src/app/api/v1/router.py index 96c1e06..679984c 100644 --- a/server/src/app/api/v1/router.py +++ b/server/src/app/api/v1/router.py @@ -1,5 +1,6 @@ from fastapi import APIRouter +from app.api.v1.endpoints.agent_asset_releases import router as agent_asset_releases_router from app.api.v1.endpoints.agent_asset_risk_rules import router as agent_asset_risk_rules_router from app.api.v1.endpoints.agent_assets import router as agent_assets_router from app.api.v1.endpoints.agent_feedback import router as agent_feedback_router @@ -15,6 +16,9 @@ from app.api.v1.endpoints.audit_logs import router as audit_logs_router from app.api.v1.endpoints.auth import router as auth_router from app.api.v1.endpoints.bootstrap import router as bootstrap_router from app.api.v1.endpoints.budgets import router as budgets_router +from app.api.v1.endpoints.cfo_value import router as cfo_value_router +from app.api.v1.endpoints.commercial import router as commercial_router +from app.api.v1.endpoints.commercial_billing import router as commercial_billing_router from app.api.v1.endpoints.employee_profiles import router as employee_profiles_router from app.api.v1.endpoints.employees import router as employees_router from app.api.v1.endpoints.expense_application_memories import ( @@ -24,6 +28,8 @@ from app.api.v1.endpoints.expense_application_previews import ( router as expense_application_previews_router, ) from app.api.v1.endpoints.expense_cases import router as expense_cases_router +from app.api.v1.endpoints.financial_connectors import router as financial_connectors_router +from app.api.v1.endpoints.finance_report_configs import router as finance_report_configs_router from app.api.v1.endpoints.health import router as health_router from app.api.v1.endpoints.knowledge import router as knowledge_router from app.api.v1.endpoints.linked_reimbursement_draft_jobs import ( @@ -36,6 +42,7 @@ from app.api.v1.endpoints.orchestrator import router as orchestrator_router from app.api.v1.endpoints.receipt_folder import router as receipt_folder_router from app.api.v1.endpoints.reimbursements import router as reimbursements_router from app.api.v1.endpoints.risk_observations import router as risk_observations_router +from app.api.v1.endpoints.savings import router as savings_router from app.api.v1.endpoints.settings import router as settings_router from app.api.v1.endpoints.steward import router as steward_router from app.api.v1.endpoints.system_logs import router as system_logs_router @@ -45,8 +52,12 @@ router.include_router(health_router, tags=["health"]) router.include_router(bootstrap_router, tags=["bootstrap"]) router.include_router(auth_router, tags=["auth"]) router.include_router(budgets_router, tags=["budgets"]) +router.include_router(cfo_value_router, tags=["analytics"]) +router.include_router(commercial_router, tags=["commercial"]) +router.include_router(commercial_billing_router, tags=["commercial"]) router.include_router(agent_assets_router, tags=["agent-assets"]) router.include_router(agent_asset_risk_rules_router, tags=["agent-assets"]) +router.include_router(agent_asset_releases_router, tags=["agent-assets"]) router.include_router(agent_feedback_router, tags=["agent-feedback"]) router.include_router(agent_runs_router, tags=["agent-runs"]) router.include_router(agent_traces_router, tags=["agent-traces"]) @@ -67,11 +78,14 @@ router.include_router(orchestrator_router, tags=["orchestrator"]) router.include_router(receipt_folder_router, tags=["receipt-folder"]) router.include_router(employees_router, prefix="/employees", tags=["employees"]) router.include_router(expense_cases_router, tags=["expense-cases"]) +router.include_router(financial_connectors_router, tags=["financial-connectors"]) +router.include_router(finance_report_configs_router, tags=["finance-report-config"]) router.include_router(expense_application_memories_router, tags=["expense-application-memories"]) router.include_router(expense_application_previews_router, tags=["reimbursements"]) router.include_router(employee_profiles_router, tags=["employee-profiles"]) router.include_router(reimbursements_router, prefix="/reimbursements", tags=["reimbursements"]) router.include_router(risk_observations_router, tags=["risk-observations"]) +router.include_router(savings_router, tags=["savings"]) router.include_router(settings_router, tags=["settings"]) router.include_router(steward_router, tags=["steward"]) router.include_router(system_logs_router, tags=["system-logs"]) diff --git a/server/src/app/cli/__init__.py b/server/src/app/cli/__init__.py new file mode 100644 index 0000000..5b0dc66 --- /dev/null +++ b/server/src/app/cli/__init__.py @@ -0,0 +1 @@ +"""受控运维命令的可测试业务编排。""" diff --git a/server/src/app/cli/savings_standard_adjustment_backfill.py b/server/src/app/cli/savings_standard_adjustment_backfill.py new file mode 100644 index 0000000..8995c99 --- /dev/null +++ b/server/src/app/cli/savings_standard_adjustment_backfill.py @@ -0,0 +1,700 @@ +from __future__ import annotations + +import hashlib +import hmac +import json +import re +from collections import Counter +from collections.abc import Sequence +from dataclasses import dataclass, replace +from datetime import UTC, datetime +from decimal import Decimal, InvalidOperation +from enum import StrEnum +from typing import Any + +from sqlalchemy import and_, or_, select +from sqlalchemy.orm import Session, selectinload + +from app.api.deps import CurrentUserContext +from app.models.expense_case import ExpenseCase, ExpenseCaseLink +from app.models.financial_record import ExpenseClaim +from app.models.savings import SavingsOpportunity +from app.services.expense_claim_constants import STANDARD_ADJUSTMENT_RISK_SOURCE +from app.services.savings_discovery import SavingsDiscoveryService + +DEFAULT_BATCH_SIZE = 100 +MAX_BATCH_SIZE = 1000 +_FINGERPRINT_PATTERN = re.compile(r"^sha256:[0-9a-f]{64}$") + + +class StandardAdjustmentBackfillDisposition(StrEnum): + ELIGIBLE = "eligible" + CREATED = "created" + REPLAYED = "replayed" + MISSING_CASE_LINK = "missing_case_link" + TENANT_CONFLICT = "tenant_conflict" + INVALID_CASE_LINK = "invalid_case_link" + MISSING_ITEM = "missing_item" + AMBIGUOUS_ITEM_HISTORY = "ambiguous_item_history" + DUPLICATE_FLAG = "duplicate_flag" + MISSING_SERVER_POLICY = "missing_server_policy" + MISSING_POLICY_VERSION = "missing_policy_version" + INVALID_POLICY_SNAPSHOT = "invalid_policy_snapshot" + INVALID_CALCULATION_FINGERPRINT = "invalid_calculation_fingerprint" + CALCULATION_FINGERPRINT_MISMATCH = "calculation_fingerprint_mismatch" + INVALID_ORIGINAL_AMOUNT = "invalid_original_amount" + ORIGINAL_AMOUNT_MISMATCH = "original_amount_mismatch" + INVALID_TARGET_AMOUNT = "invalid_target_amount" + INVALID_SAVING_DIFFERENCE = "invalid_saving_difference" + INVALID_CURRENCY = "invalid_currency" + + +@dataclass(frozen=True, slots=True) +class StandardAdjustmentBackfillCursor: + created_at: datetime + claim_id: str + + +@dataclass(frozen=True, slots=True) +class StandardAdjustmentBackfillItem: + claim_id: str + claim_no: str + item_id: str + disposition: StandardAdjustmentBackfillDisposition + reason: str + calculation_fingerprint: str = "" + policy_version: str = "" + currency: str = "" + original_amount: Decimal | None = None + target_amount: Decimal | None = None + saving_amount: Decimal | None = None + opportunity_id: str | None = None + + +@dataclass(frozen=True, slots=True) +class StandardAdjustmentBackfillPage: + tenant_id: str + claims_inspected: int + flags_inspected: int + eligible: int + replayed: int + skipped: int + reasons: dict[str, int] + has_more: bool + next_cursor: StandardAdjustmentBackfillCursor | None + items: tuple[StandardAdjustmentBackfillItem, ...] + + +@dataclass(frozen=True, slots=True) +class StandardAdjustmentBackfillResult: + tenant_id: str + run_id: str + claims_inspected: int + flags_inspected: int + created: int + replayed: int + skipped: int + reasons: dict[str, int] + has_more: bool + next_cursor: StandardAdjustmentBackfillCursor | None + items: tuple[StandardAdjustmentBackfillItem, ...] + + +@dataclass(slots=True) +class _BatchContext: + links_by_claim_id: dict[str, ExpenseCaseLink] + cases_by_id: dict[str, ExpenseCase] + opportunities_by_key: dict[str, SavingsOpportunity] + + +class StandardAdjustmentSavingsBackfillService: + """严格按历史服务端证据回填节省机会；从不自行提交事务。""" + + def __init__( + self, + db: Session, + *, + tenant_id: str, + created_before: datetime | None = None, + ) -> None: + self.db = db + self.tenant_id = self._required_text(tenant_id, field_name="tenant_id", max_length=64) + self.created_before = ( + self._aware(created_before, field_name="created_before") + if created_before is not None + else None + ) + + def preview( + self, + *, + batch_size: int = DEFAULT_BATCH_SIZE, + after: StandardAdjustmentBackfillCursor | None = None, + ) -> StandardAdjustmentBackfillPage: + """只读预览；不 add、不 flush、不 commit。""" + + with self.db.no_autoflush: + claims, has_more = self._load_claims( + batch_size=self._batch_size(batch_size), + after=after, + lock_rows=False, + ) + context = self._load_context(claims) + items = self._classify(claims, context=context) + return StandardAdjustmentBackfillPage( + tenant_id=self.tenant_id, + claims_inspected=len(claims), + flags_inspected=len(items), + eligible=self._count(items, StandardAdjustmentBackfillDisposition.ELIGIBLE), + replayed=self._count(items, StandardAdjustmentBackfillDisposition.REPLAYED), + skipped=self._skipped(items), + reasons=self._reason_counts(items), + has_more=has_more, + next_cursor=self._next_cursor(claims), + items=items, + ) + + def apply_batch( + self, + *, + run_id: str, + batch_size: int = DEFAULT_BATCH_SIZE, + after: StandardAdjustmentBackfillCursor | None = None, + ) -> StandardAdjustmentBackfillResult: + """锁定并应用一批；调用方负责按批 commit/rollback。""" + + normalized_run_id = self._required_text(run_id, field_name="run_id", max_length=64) + claims, has_more = self._load_claims( + batch_size=self._batch_size(batch_size), + after=after, + lock_rows=True, + ) + context = self._load_context(claims) + classified = self._classify(claims, context=context) + claims_by_id = {str(claim.id): claim for claim in claims} + items_by_claim = { + str(claim.id): {str(item.id): item for item in claim.items} for claim in claims + } + actor = self._system_actor(run_id=normalized_run_id) + result_items: list[StandardAdjustmentBackfillItem] = [] + + for item in classified: + if item.disposition is not StandardAdjustmentBackfillDisposition.ELIGIBLE: + result_items.append(item) + continue + claim = claims_by_id[item.claim_id] + claim_items = items_by_claim[item.claim_id] + source_flag = self._find_flag(claim, item=item) + discovered = SavingsDiscoveryService(self.db).discover_standard_adjustments( + claim=claim, + items_by_id=claim_items, + adjustment_flags=[source_flag], + current_user=actor, + request_id=f"savings-backfill:{normalized_run_id}", + ) + if len(discovered) != 1: + raise RuntimeError("历史标准调整回填未返回唯一节省机会，当前批次必须回滚。") + opportunity = discovered[0] + context.opportunities_by_key[opportunity.opportunity_key] = opportunity + result_items.append( + replace( + item, + disposition=StandardAdjustmentBackfillDisposition.CREATED, + reason="证据完整，已通过 SavingsDiscoveryService 创建机会。", + opportunity_id=opportunity.id, + ) + ) + + result = tuple(result_items) + return StandardAdjustmentBackfillResult( + tenant_id=self.tenant_id, + run_id=normalized_run_id, + claims_inspected=len(claims), + flags_inspected=len(result), + created=self._count(result, StandardAdjustmentBackfillDisposition.CREATED), + replayed=self._count(result, StandardAdjustmentBackfillDisposition.REPLAYED), + skipped=self._skipped(result), + reasons=self._reason_counts(result), + has_more=has_more, + next_cursor=self._next_cursor(claims), + items=result, + ) + + def _load_claims( + self, + *, + batch_size: int, + after: StandardAdjustmentBackfillCursor | None, + lock_rows: bool, + ) -> tuple[list[ExpenseClaim], bool]: + stmt = select(ExpenseClaim).options(selectinload(ExpenseClaim.items)) + if self.created_before is not None: + stmt = stmt.where(ExpenseClaim.created_at < self.created_before) + if after is not None: + cursor_time = self._aware(after.created_at, field_name="after.created_at") + cursor_id = self._required_text( + after.claim_id, + field_name="after.claim_id", + max_length=36, + ) + stmt = stmt.where( + or_( + ExpenseClaim.created_at > cursor_time, + and_(ExpenseClaim.created_at == cursor_time, ExpenseClaim.id > cursor_id), + ) + ) + stmt = stmt.order_by(ExpenseClaim.created_at.asc(), ExpenseClaim.id.asc()).limit( + batch_size + 1 + ) + if lock_rows: + stmt = stmt.with_for_update() + # JSON 文本过滤在不同数据库方言上语义不完全一致，因此在 Python 中确认来源； + # 这里按 Claim 分页，空标志页仍可通过 cursor 继续推进。 + rows = list(self.db.scalars(stmt).unique().all()) + return rows[:batch_size], len(rows) > batch_size + + def _load_context(self, claims: Sequence[ExpenseClaim]) -> _BatchContext: + claim_ids = [str(claim.id) for claim in claims] + if not claim_ids: + return _BatchContext({}, {}, {}) + links = list( + self.db.scalars( + select(ExpenseCaseLink).where( + ExpenseCaseLink.resource_type == "expense_claim", + ExpenseCaseLink.resource_id.in_(claim_ids), + ) + ).all() + ) + links_by_claim_id = {str(link.resource_id): link for link in links} + case_ids = {str(link.expense_case_id) for link in links} + cases_by_id = ( + { + str(expense_case.id): expense_case + for expense_case in self.db.scalars( + select(ExpenseCase).where(ExpenseCase.id.in_(case_ids)) + ).all() + } + if case_ids + else {} + ) + opportunities = list( + self.db.scalars( + select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == self.tenant_id, + SavingsOpportunity.claim_id.in_(claim_ids), + SavingsOpportunity.source_type == "standard_adjustment", + ) + ).all() + ) + return _BatchContext( + links_by_claim_id=links_by_claim_id, + cases_by_id=cases_by_id, + opportunities_by_key={item.opportunity_key: item for item in opportunities}, + ) + + def _classify( + self, + claims: Sequence[ExpenseClaim], + *, + context: _BatchContext, + ) -> tuple[StandardAdjustmentBackfillItem, ...]: + results: list[StandardAdjustmentBackfillItem] = [] + for claim in claims: + flags = self._standard_adjustment_flags(claim) + if not flags: + continue + grouped: dict[str, list[dict[str, Any]]] = {} + for flag in flags: + grouped.setdefault(str(flag.get("item_id") or "").strip(), []).append(flag) + for _item_id, item_flags in grouped.items(): + fingerprints = { + str(flag.get("calculation_fingerprint") or "").strip() + for flag in item_flags + } + if len(fingerprints) > 1: + results.extend( + self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.AMBIGUOUS_ITEM_HISTORY, + "同一明细存在多个不同政策计算快照，无法安全判断应货币化哪一版。", + ) + for flag in item_flags + ) + continue + first, *duplicates = item_flags + results.append(self._classify_flag(claim, first, context=context)) + results.extend( + self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.DUPLICATE_FLAG, + "同一明细存在重复标准调整标志；仅首条进入证据校验。", + ) + for flag in duplicates + ) + return tuple(results) + + def _classify_flag( + self, + claim: ExpenseClaim, + flag: dict[str, Any], + *, + context: _BatchContext, + ) -> StandardAdjustmentBackfillItem: + item_id = str(flag.get("item_id") or "").strip() + link = context.links_by_claim_id.get(str(claim.id)) + if link is None: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.MISSING_CASE_LINK, + "Claim 没有 ExpenseCaseLink，租户归属无法证明。", + ) + if link.tenant_id != self.tenant_id: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.TENANT_CONFLICT, + "Claim 的 ExpenseCaseLink 属于其他租户。", + ) + expense_case = context.cases_by_id.get(str(link.expense_case_id)) + if ( + expense_case is None + or expense_case.tenant_id != self.tenant_id + or not str(link.relation_type or "").strip() + ): + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.INVALID_CASE_LINK, + "ExpenseCaseLink 关联目标缺失、租户不一致或关系类型为空。", + ) + item = next((entry for entry in claim.items if str(entry.id) == item_id), None) + if item is None: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.MISSING_ITEM, + "标准调整引用的费用明细不存在。", + ) + if str(flag.get("calculation_source") or "").strip() != "server_policy": + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.MISSING_SERVER_POLICY, + "缺少 calculation_source=server_policy，不能证明由服务端政策计算。", + ) + policy_version = str(flag.get("policy_rule_version") or "").strip() + if not policy_version: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.MISSING_POLICY_VERSION, + "缺少服务端政策版本。", + ) + calculation_fingerprint = str(flag.get("calculation_fingerprint") or "").strip() + if not _FINGERPRINT_PATTERN.fullmatch(calculation_fingerprint): + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.INVALID_CALCULATION_FINGERPRINT, + "政策计算指纹不是合法 sha256 快照。", + ) + original = self._money(flag.get("original_amount")) + if original is None or original <= Decimal("0"): + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.INVALID_ORIGINAL_AMOUNT, + "原金额缺失、格式错误或不为正数。", + ) + item_amount = self._money(item.item_amount) + if item_amount is None or item_amount != original: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.ORIGINAL_AMOUNT_MISMATCH, + "历史原金额与当前 ExpenseClaimItem.item_amount 不一致。", + original=original, + ) + target = self._money(flag.get("reimbursable_amount")) + if target is None or target < Decimal("0"): + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.INVALID_TARGET_AMOUNT, + "目标报销金额缺失、格式错误或为负数。", + original=original, + ) + saving = (original - target).quantize(Decimal("0.01")) + absorbed = self._money(flag.get("employee_absorbed_amount")) + if saving <= Decimal("0") or absorbed is None or absorbed != saving: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.INVALID_SAVING_DIFFERENCE, + "差额不为正数，或员工承担金额与原金额减目标金额不一致。", + original=original, + target=target, + saving=saving, + ) + currency = str(claim.currency or "").strip().upper() + if len(currency) != 3 or not currency.isalpha(): + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.INVALID_CURRENCY, + "Claim 缺少合法三位币种，禁止默认猜测币种。", + original=original, + target=target, + saving=saving, + ) + expected_fingerprint = self._calculation_fingerprint(flag, original, target) + if expected_fingerprint is None: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.INVALID_POLICY_SNAPSHOT, + "政策天数、地点、职级、酒店标准或规则名称快照不完整。", + original=original, + target=target, + saving=saving, + ) + if not hmac.compare_digest(expected_fingerprint, calculation_fingerprint): + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.CALCULATION_FINGERPRINT_MISMATCH, + "按历史政策快照重算的 SHA-256 与存储指纹不一致。", + original=original, + target=target, + saving=saving, + ) + opportunity_key = self._opportunity_key( + claim_id=str(claim.id), + item_id=item_id, + policy_version=policy_version, + calculation_fingerprint=calculation_fingerprint, + ) + existing = context.opportunities_by_key.get(opportunity_key) + if existing is not None: + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.REPLAYED, + "相同稳定键的节省机会已存在。", + original=original, + target=target, + saving=saving, + opportunity_id=existing.id, + ) + return self._item( + claim, + flag, + StandardAdjustmentBackfillDisposition.ELIGIBLE, + "Case、租户、金额及服务端政策指纹均验证通过。", + original=original, + target=target, + saving=saving, + ) + + @staticmethod + def _standard_adjustment_flags(claim: ExpenseClaim) -> list[dict[str, Any]]: + return [ + flag + for flag in list(claim.risk_flags_json or []) + if isinstance(flag, dict) + and str(flag.get("source") or "").strip() == STANDARD_ADJUSTMENT_RISK_SOURCE + ] + + @classmethod + def _find_flag( + cls, + claim: ExpenseClaim, + *, + item: StandardAdjustmentBackfillItem, + ) -> dict[str, Any]: + matches = [ + flag + for flag in cls._standard_adjustment_flags(claim) + if str(flag.get("item_id") or "").strip() == item.item_id + and str(flag.get("calculation_fingerprint") or "").strip() + == item.calculation_fingerprint + ] + if len(matches) != 1: + raise RuntimeError("历史标准调整证据在锁定后发生歧义，当前批次必须回滚。") + return matches[0] + + @classmethod + def _calculation_fingerprint( + cls, + flag: dict[str, Any], + original: Decimal, + target: Decimal, + ) -> str | None: + try: + days = int(flag["policy_days"]) + location = cls._snapshot_text(flag, "policy_location") + matched_city = cls._snapshot_text(flag, "policy_matched_city") + grade = cls._snapshot_text(flag, "policy_grade") + grade_band = cls._snapshot_text(flag, "policy_grade_band") + hotel_rate = cls._snapshot_money(flag, "policy_hotel_rate") + hotel_amount = cls._snapshot_money(flag, "policy_hotel_amount") + rule_name = cls._snapshot_text(flag, "policy_rule_name") + rule_version = cls._snapshot_text(flag, "policy_rule_version") + except (KeyError, TypeError, ValueError, InvalidOperation): + return None + if days <= 0 or hotel_rate < Decimal("0") or hotel_amount < Decimal("0"): + return None + material = { + "item_id": str(flag.get("item_id") or "").strip(), + "original_amount": f"{original:.2f}", + "reimbursable_amount": f"{target:.2f}", + "policy_days": days, + "policy_location": location, + "policy_matched_city": matched_city, + "policy_grade": grade, + "policy_grade_band": grade_band, + "policy_hotel_rate": f"{hotel_rate:.2f}", + "policy_hotel_amount": f"{hotel_amount:.2f}", + "policy_rule_name": rule_name, + "policy_rule_version": rule_version, + } + canonical = json.dumps( + material, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ) + return f"sha256:{hashlib.sha256(canonical.encode('utf-8')).hexdigest()}" + + def _opportunity_key( + self, + *, + claim_id: str, + item_id: str, + policy_version: str, + calculation_fingerprint: str, + ) -> str: + material = { + "tenant_id": self.tenant_id, + "claim_id": claim_id, + "item_id": item_id, + "policy_version": policy_version, + "calculation_fingerprint": calculation_fingerprint, + } + canonical = json.dumps(material, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + return f"standard-adjustment:{hashlib.sha256(canonical.encode('utf-8')).hexdigest()}" + + def _item( + self, + claim: ExpenseClaim, + flag: dict[str, Any], + disposition: StandardAdjustmentBackfillDisposition, + reason: str, + *, + original: Decimal | None = None, + target: Decimal | None = None, + saving: Decimal | None = None, + opportunity_id: str | None = None, + ) -> StandardAdjustmentBackfillItem: + return StandardAdjustmentBackfillItem( + claim_id=str(claim.id), + claim_no=str(claim.claim_no or claim.id), + item_id=str(flag.get("item_id") or "").strip(), + disposition=disposition, + reason=reason, + calculation_fingerprint=str(flag.get("calculation_fingerprint") or "").strip(), + policy_version=str(flag.get("policy_rule_version") or "").strip(), + currency=str(claim.currency or "").strip().upper(), + original_amount=original, + target_amount=target, + saving_amount=saving, + opportunity_id=opportunity_id, + ) + + def _system_actor(self, *, run_id: str) -> CurrentUserContext: + return CurrentUserContext( + username=f"savings-backfill:{run_id}"[:120], + name="Savings 历史回填", + role_codes=["finance"], + is_admin=False, + tenant_id=self.tenant_id, + ) + + @staticmethod + def _snapshot_text(flag: dict[str, Any], key: str) -> str: + value = str(flag[key] or "").strip() + if not value: + raise ValueError(f"{key} is blank") + return value + + @classmethod + def _snapshot_money(cls, flag: dict[str, Any], key: str) -> Decimal: + value = cls._money(flag[key]) + if value is None: + raise ValueError(f"{key} is not money") + return value + + @staticmethod + def _money(value: Any) -> Decimal | None: + if isinstance(value, bool) or value is None or str(value).strip() == "": + return None + try: + parsed = Decimal(str(value)) + except (InvalidOperation, ValueError): + return None + if not parsed.is_finite(): + return None + return parsed.quantize(Decimal("0.01")) + + @staticmethod + def _required_text(value: str, *, field_name: str, max_length: int) -> str: + normalized = str(value or "").strip() + if not normalized: + raise ValueError(f"{field_name} must not be empty") + if len(normalized) > max_length: + raise ValueError(f"{field_name} must be at most {max_length} characters") + return normalized + + @staticmethod + def _aware(value: datetime, *, field_name: str) -> datetime: + if value.tzinfo is None or value.utcoffset() is None: + raise ValueError(f"{field_name} must include timezone") + return value.astimezone(UTC) + + @staticmethod + def _batch_size(value: int) -> int: + normalized = int(value) + if normalized < 1 or normalized > MAX_BATCH_SIZE: + raise ValueError(f"batch_size must be between 1 and {MAX_BATCH_SIZE}") + return normalized + + @staticmethod + def _next_cursor(claims: Sequence[ExpenseClaim]) -> StandardAdjustmentBackfillCursor | None: + if not claims: + return None + claim = claims[-1] + return StandardAdjustmentBackfillCursor(created_at=claim.created_at, claim_id=str(claim.id)) + + @staticmethod + def _count( + items: Sequence[StandardAdjustmentBackfillItem], + disposition: StandardAdjustmentBackfillDisposition, + ) -> int: + return sum(item.disposition is disposition for item in items) + + @staticmethod + def _reason_counts(items: Sequence[StandardAdjustmentBackfillItem]) -> dict[str, int]: + counts = Counter(item.disposition.value for item in items) + return dict(sorted(counts.items())) + + @staticmethod + def _skipped(items: Sequence[StandardAdjustmentBackfillItem]) -> int: + accepted = { + StandardAdjustmentBackfillDisposition.ELIGIBLE, + StandardAdjustmentBackfillDisposition.CREATED, + StandardAdjustmentBackfillDisposition.REPLAYED, + } + return sum(item.disposition not in accepted for item in items) diff --git a/server/src/app/core/agent_asset_scope.py b/server/src/app/core/agent_asset_scope.py new file mode 100644 index 0000000..393b967 --- /dev/null +++ b/server/src/app/core/agent_asset_scope.py @@ -0,0 +1,3 @@ +AGENT_ASSET_PLATFORM_SCOPE = "platform" +AGENT_ASSET_TENANT_SCOPE = "tenant" +AGENT_ASSET_PLATFORM_TENANT_ID = "platform" diff --git a/server/src/app/core/agent_release_telemetry_keys.py b/server/src/app/core/agent_release_telemetry_keys.py new file mode 100644 index 0000000..18d130f --- /dev/null +++ b/server/src/app/core/agent_release_telemetry_keys.py @@ -0,0 +1,70 @@ +"""Agent 发布遥测伪名密钥的本地版本化存储。""" + +from __future__ import annotations + +import base64 +import binascii +import os +import re +import secrets +from pathlib import Path + +from app.core.config import SERVER_DIR + +FINGERPRINT_KEY_DIRECTORY = SERVER_DIR / ".secrets" / "agent-release-telemetry" +ACTIVE_KEY_VERSION_ENV = "AGENT_RELEASE_TELEMETRY_KEY_VERSION" +DEFAULT_ACTIVE_KEY_VERSION = "v1" +KEY_BYTES = 32 +_VERSION_PATTERN = re.compile(r"^[a-zA-Z0-9._-]{1,32}$") + + +def active_agent_release_telemetry_key_version() -> str: + version = str(os.environ.get(ACTIVE_KEY_VERSION_ENV) or DEFAULT_ACTIVE_KEY_VERSION).strip() + if not _VERSION_PATTERN.fullmatch(version): + raise ValueError("Agent 发布遥测指纹密钥版本无效。") + return version + + +def available_agent_release_telemetry_key_versions() -> list[str]: + active = active_agent_release_telemetry_key_version() + versions = {active} + if FINGERPRINT_KEY_DIRECTORY.exists(): + for path in FINGERPRINT_KEY_DIRECTORY.glob("*.key"): + if path.is_file() and not path.is_symlink() and _VERSION_PATTERN.fullmatch(path.stem): + versions.add(path.stem) + return [active, *sorted(versions - {active})] + + +def get_agent_release_telemetry_key(version: str, *, create: bool) -> bytes: + normalized = str(version or "").strip() + if not _VERSION_PATTERN.fullmatch(normalized): + raise ValueError("Agent 发布遥测指纹密钥版本无效。") + key_path = FINGERPRINT_KEY_DIRECTORY / f"{normalized}.key" + if not key_path.exists(): + if not create: + raise ValueError("Agent 发布遥测指纹密钥版本不可用。") + _create_key_atomically(key_path) + if key_path.is_symlink() or not key_path.is_file(): + raise ValueError("Agent 发布遥测指纹密钥文件无效。") + os.chmod(key_path, 0o600) + try: + key = base64.urlsafe_b64decode(key_path.read_text(encoding="utf-8").strip()) + except (binascii.Error, ValueError, UnicodeError) as error: + raise ValueError("Agent 发布遥测指纹密钥内容无效。") from error + if len(key) != KEY_BYTES: + raise ValueError("Agent 发布遥测指纹密钥长度无效。") + return key + + +def _create_key_atomically(key_path: Path) -> None: + key_path.parent.mkdir(parents=True, exist_ok=True) + os.chmod(key_path.parent, 0o700) + encoded = base64.urlsafe_b64encode(secrets.token_bytes(KEY_BYTES)) + try: + descriptor = os.open(key_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + except FileExistsError: + return + with os.fdopen(descriptor, "wb") as stream: + stream.write(encoded) + stream.flush() + os.fsync(stream.fileno()) diff --git a/server/src/app/core/config.py b/server/src/app/core/config.py index f690b7c..2dbce6a 100644 --- a/server/src/app/core/config.py +++ b/server/src/app/core/config.py @@ -73,6 +73,10 @@ class Settings(BaseSettings): onlyoffice_backend_url: str = Field(default="", alias="ONLYOFFICE_BACKEND_URL") onlyoffice_jwt_secret: str = Field(default="", alias="ONLYOFFICE_JWT_SECRET") hermes_agent_shared_token: str = Field(default="", alias="HERMES_AGENT_SHARED_TOKEN") + financial_connector_hmac_keys_json: str = Field( + default="", + alias="FINANCIAL_CONNECTOR_HMAC_KEYS_JSON", + ) steward_agent_runtime: str = Field(default="langgraph", alias="STEWARD_AGENT_RUNTIME") log_level: str = Field(default="INFO", alias="LOG_LEVEL") diff --git a/server/src/app/db/base.py b/server/src/app/db/base.py index 316de32..653220a 100644 --- a/server/src/app/db/base.py +++ b/server/src/app/db/base.py @@ -6,6 +6,11 @@ from app.models.agent_asset import ( AgentAssetTestRun, AgentAssetVersion, ) +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) from app.models.agent_conversation import AgentConversation, AgentConversationMessage from app.models.agent_feedback import AgentOperationFeedback from app.models.agent_run import AgentRun, AgentToolCall, AgentTraceEvent, SemanticParseLog @@ -19,11 +24,28 @@ from app.models.attachment_association_job import AttachmentAssociationJob from app.models.audit_log import AuditLog from app.models.auth_session import AuthSession from app.models.budget import BudgetAllocation, BudgetReservation, BudgetTransaction +from app.models.commercial import ( + CommercialCostEvent, + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) +from app.models.commercial_billing import CommercialAdminEvent, CommercialBillingPeriod +from app.models.commercial_runtime import CommercialRuntimeReservation from app.models.employee import Employee from app.models.employee_behavior_profile import EmployeeBehaviorProfileSnapshot from app.models.employee_change_log import EmployeeChangeLog from app.models.expense_case import BusinessEvent, ExpenseCase, ExpenseCaseLink from app.models.few_shot_sample import FewShotSample +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorConfigEvent, + FinancialConnectorEvent, + FinancialConnectorOperationalEvent, + PaymentReconciliationCase, + PaymentReconciliationEvent, +) from app.models.financial_record import ( AccountsPayableRecord, AccountsReceivableRecord, @@ -33,6 +55,7 @@ from app.models.financial_record import ( from app.models.golden_case import GoldenCase from app.models.hermes_config import HermesTaskConfig, HermesTaskExecutionLog from app.models.hermes_report import HermesRiskReport +from app.models.knowledge_security import KnowledgeOnlyOfficeSession from app.models.notification_state import NotificationState from app.models.organization import OrganizationUnit from app.models.reimbursement import ReimbursementRequest @@ -42,6 +65,11 @@ from app.models.role import Role from app.models.system_model_setting import SystemModelSetting from app.models.system_setting import SystemSetting from app.models.system_setting_secret import SystemSettingSecret +from app.models.tenant import Tenant, TenantMembership +from app.models.tenant_finance_report import ( + TenantFinanceReportConfig, + TenantFinanceReportRun, +) from app.models.user_session_metric import UserSessionMetric __all__ = [ @@ -55,6 +83,9 @@ __all__ = [ "AgentAssetRuleFeedback", "AgentAssetTestRun", "AgentAssetVersion", + "AgentAssetReleaseAuditSample", + "AgentAssetReleaseLabel", + "AgentAssetReleaseObservation", "AgentOperationFeedback", "AgentRun", "AgentToolCall", @@ -72,6 +103,11 @@ __all__ = [ "BudgetAllocation", "BudgetReservation", "BudgetTransaction", + "CommercialCostEvent", + "CommercialAdminEvent", + "CommercialBillingPeriod", + "CommercialEntitlement", + "CommercialRuntimeReservation", "Employee", "ExpenseCase", "ExpenseCaseLink", @@ -81,13 +117,20 @@ __all__ = [ "ExpenseClaim", "FewShotSample", "ExpenseClaimItem", + "FinancialConnectorConfig", + "FinancialConnectorConfigEvent", + "FinancialConnectorEvent", + "FinancialConnectorOperationalEvent", "GoldenCase", "HermesTaskConfig", "HermesTaskExecutionLog", "HermesRiskReport", + "KnowledgeOnlyOfficeSession", "MemoryEntry", "MemoryEvidenceLink", "NotificationState", + "PaymentReconciliationCase", + "PaymentReconciliationEvent", "OrganizationUnit", "ReimbursementRequest", "RiskDisposition", @@ -99,6 +142,13 @@ __all__ = [ "SystemModelSetting", "SystemSetting", "SystemSettingSecret", + "Tenant", + "TenantCommercialPlan", + "TenantMembership", + "TenantFinanceReportConfig", + "TenantFinanceReportRun", + "TenantSubscription", + "UsageMeterEvent", "UserSessionMetric", "WorkflowOutcome", ] diff --git a/server/src/app/db/migration_preflight.py b/server/src/app/db/migration_preflight.py index 77b2c7d..cd4769a 100644 --- a/server/src/app/db/migration_preflight.py +++ b/server/src/app/db/migration_preflight.py @@ -238,7 +238,89 @@ MIGRATION_OWNED_TABLES_BY_REVISION: dict[str, frozenset[str]] = { } ), } -if MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0014"] != MIGRATION_OWNED_TABLES: +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0015"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0014"] + | frozenset( + { + "profile_baseline_snapshots", + "savings_opportunities", + "savings_realizations", + "savings_evidence_links", + "savings_events", + } + ) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0016"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0015"] + | frozenset( + { + "tenant_commercial_plans", + "tenant_subscriptions", + "commercial_entitlements", + "usage_meter_events", + "commercial_cost_events", + } + ) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0017"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0016"] + | frozenset( + { + "financial_connector_configs", + "financial_connector_events", + "payment_reconciliation_cases", + "payment_reconciliation_events", + } + ) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0018"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0017"] + | frozenset( + { + "agent_asset_release_observations", + "agent_asset_release_labels", + } + ) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0019"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0018"] + | frozenset({"commercial_runtime_reservations"}) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0020"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0019"] + | frozenset({"financial_connector_config_events"}) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0021"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0020"] + | frozenset({"commercial_admin_events", "commercial_billing_periods"}) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0022"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0021"] + | frozenset({"financial_connector_operational_events"}) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0023"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0022"] + | frozenset({"agent_asset_release_audit_samples"}) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0024"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0023"] +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0025"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0024"] + | frozenset({"tenants"}) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0026"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0025"] +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0027"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0026"] + | frozenset({"knowledge_onlyoffice_sessions"}) +) +MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0028"] = ( + MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0027"] + | frozenset({"tenant_finance_report_configs", "tenant_finance_report_runs"}) +) +if MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0028"] != MIGRATION_OWNED_TABLES: raise RuntimeError("latest Alembic revision must own the centralized migration table set") # 0008 之前这三张表由旧 bootstrap / 风险服务按需创建。迁移前置检查允许 @@ -314,6 +396,20 @@ def _validate_connection(connection: Connection) -> MigrationPreflightState: "20260716_0012", "20260716_0013", "20260716_0014", + "20260716_0015", + "20260716_0016", + "20260716_0017", + "20260716_0018", + "20260716_0019", + "20260716_0020", + "20260716_0021", + "20260716_0022", + "20260716_0023", + "20260717_0024", + "20260717_0025", + "20260717_0026", + "20260717_0027", + "20260717_0028", } else frozenset() ) diff --git a/server/src/app/db/schema_ownership.py b/server/src/app/db/schema_ownership.py index 9765015..53b1658 100644 --- a/server/src/app/db/schema_ownership.py +++ b/server/src/app/db/schema_ownership.py @@ -10,6 +10,9 @@ MIGRATION_OWNED_TABLES: frozenset[str] = frozenset( "approval_action_ledgers", "approval_task_events", "approval_tasks", + "agent_asset_release_audit_samples", + "agent_asset_release_labels", + "agent_asset_release_observations", "attachment_association_jobs", "ai_application_preview_decisions", "ai_decisions", @@ -17,12 +20,35 @@ MIGRATION_OWNED_TABLES: frozenset[str] = frozenset( "expense_cases", "expense_case_links", "business_events", + "financial_connector_configs", + "financial_connector_config_events", + "financial_connector_events", + "financial_connector_operational_events", + "commercial_cost_events", + "commercial_admin_events", + "commercial_billing_periods", + "commercial_entitlements", + "commercial_runtime_reservations", + "knowledge_onlyoffice_sessions", "memory_entries", "memory_evidence_links", "risk_observations", "risk_observation_feedback", "risk_dispositions", "risk_disposition_events", + "profile_baseline_snapshots", + "payment_reconciliation_cases", + "payment_reconciliation_events", + "savings_opportunities", + "savings_realizations", + "savings_evidence_links", + "savings_events", + "tenant_commercial_plans", + "tenant_finance_report_configs", + "tenant_finance_report_runs", + "tenant_subscriptions", + "tenants", + "usage_meter_events", "few_shot_samples", "workflow_outcomes", } diff --git a/server/src/app/main.py b/server/src/app/main.py index 25f568e..d736ac6 100644 --- a/server/src/app/main.py +++ b/server/src/app/main.py @@ -15,8 +15,10 @@ from app.core.openapi import API_DESCRIPTION, OPENAPI_TAGS from app.db.session import get_session_factory from app.middleware.logging import AccessLogMiddleware from app.schemas.common import RootStatusRead +from app.services.agent_asset_release_scheduler import agent_asset_release_scheduler from app.services.agent_foundation import prepare_agent_foundation from app.services.approval_task_scheduler import approval_task_scheduler +from app.services.commercial_rollover_scheduler import commercial_rollover_scheduler from app.services.digital_employee_reminder_scheduler import digital_employee_reminder_scheduler from app.services.employee import EmployeeService, prepare_employee_directory from app.services.employee_profile_scheduler import employee_profile_scheduler @@ -28,6 +30,7 @@ from app.services.knowledge_index_tasks import knowledge_index_task_manager from app.services.knowledge_rag import shutdown_knowledge_rag_runtime from app.services.knowledge_scheduler import knowledge_index_scheduler from app.services.settings import SettingsService +from app.services.tenant_registry import DEFAULT_TENANT_ID from app.services.user_session_metrics import UserSessionMetricService @@ -71,7 +74,10 @@ def _warm_startup_caches(logger: Logger) -> None: session_factory = get_session_factory() with session_factory() as db: SettingsService(db).ensure_settings_ready() - EmployeeService(db).ensure_directory_ready() + EmployeeService( + db, + tenant_id=DEFAULT_TENANT_ID, + ).ensure_directory_ready() UserSessionMetricService(db).ensure_storage_ready() logger.info("Startup cache warmup complete") except Exception: @@ -104,7 +110,9 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]: schedulers_started = _should_start_background_schedulers(settings) if schedulers_started: knowledge_index_scheduler.start() + agent_asset_release_scheduler.start() approval_task_scheduler.start() + commercial_rollover_scheduler.start() finance_dashboard_scheduler.start() employee_profile_scheduler.start() digital_employee_reminder_scheduler.start() @@ -123,7 +131,9 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]: ) yield if schedulers_started: + agent_asset_release_scheduler.shutdown() approval_task_scheduler.shutdown() + commercial_rollover_scheduler.shutdown() finance_report_scheduler.shutdown() digital_employee_reminder_scheduler.shutdown() employee_profile_scheduler.shutdown() diff --git a/server/src/app/models/__init__.py b/server/src/app/models/__init__.py index b5822bd..ce01524 100644 --- a/server/src/app/models/__init__.py +++ b/server/src/app/models/__init__.py @@ -4,6 +4,11 @@ from app.models.agent_asset import ( AgentAssetRuleFeedback, AgentAssetVersion, ) +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) from app.models.agent_conversation import AgentConversation, AgentConversationMessage from app.models.agent_feedback import AgentOperationFeedback from app.models.agent_run import AgentRun, AgentToolCall, AgentTraceEvent, SemanticParseLog @@ -17,11 +22,28 @@ from app.models.attachment_association_job import AttachmentAssociationJob from app.models.audit_log import AuditLog from app.models.auth_session import AuthSession from app.models.budget import BudgetAllocation, BudgetReservation, BudgetTransaction +from app.models.commercial import ( + CommercialCostEvent, + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) +from app.models.commercial_billing import CommercialAdminEvent, CommercialBillingPeriod +from app.models.commercial_runtime import CommercialRuntimeReservation from app.models.employee import Employee from app.models.employee_behavior_profile import EmployeeBehaviorProfileSnapshot from app.models.employee_change_log import EmployeeChangeLog from app.models.expense_case import BusinessEvent, ExpenseCase, ExpenseCaseLink from app.models.few_shot_sample import FewShotSample +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorConfigEvent, + FinancialConnectorEvent, + FinancialConnectorOperationalEvent, + PaymentReconciliationCase, + PaymentReconciliationEvent, +) from app.models.financial_record import ( AccountsPayableRecord, AccountsReceivableRecord, @@ -31,15 +53,28 @@ from app.models.financial_record import ( from app.models.golden_case import GoldenCase from app.models.hermes_config import HermesTaskConfig, HermesTaskExecutionLog from app.models.hermes_report import HermesRiskReport +from app.models.knowledge_security import KnowledgeOnlyOfficeSession from app.models.notification_state import NotificationState from app.models.organization import OrganizationUnit from app.models.reimbursement import ReimbursementRequest from app.models.risk_disposition import RiskDisposition, RiskDispositionEvent from app.models.risk_observation import RiskObservation, RiskObservationFeedback from app.models.role import Role +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) from app.models.system_model_setting import SystemModelSetting from app.models.system_setting import SystemSetting from app.models.system_setting_secret import SystemSettingSecret +from app.models.tenant import Tenant, TenantMembership +from app.models.tenant_finance_report import ( + TenantFinanceReportConfig, + TenantFinanceReportRun, +) from app.models.user_session_metric import UserSessionMetric __all__ = [ @@ -51,6 +86,9 @@ __all__ = [ "AgentAssetReview", "AgentAssetRuleFeedback", "AgentAssetVersion", + "AgentAssetReleaseAuditSample", + "AgentAssetReleaseLabel", + "AgentAssetReleaseObservation", "AgentOperationFeedback", "AgentRun", "AgentToolCall", @@ -68,6 +106,11 @@ __all__ = [ "BudgetAllocation", "BudgetReservation", "BudgetTransaction", + "CommercialCostEvent", + "CommercialAdminEvent", + "CommercialBillingPeriod", + "CommercialEntitlement", + "CommercialRuntimeReservation", "Employee", "ExpenseCase", "ExpenseCaseLink", @@ -76,14 +119,21 @@ __all__ = [ "EmployeeChangeLog", "ExpenseClaim", "ExpenseClaimItem", + "FinancialConnectorConfig", + "FinancialConnectorConfigEvent", + "FinancialConnectorEvent", + "FinancialConnectorOperationalEvent", "FewShotSample", "GoldenCase", "HermesTaskConfig", "HermesTaskExecutionLog", "HermesRiskReport", + "KnowledgeOnlyOfficeSession", "MemoryEntry", "MemoryEvidenceLink", "NotificationState", + "PaymentReconciliationCase", + "PaymentReconciliationEvent", "OrganizationUnit", "ReimbursementRequest", "RiskDisposition", @@ -91,10 +141,22 @@ __all__ = [ "RiskObservation", "RiskObservationFeedback", "Role", + "ProfileBaselineSnapshot", + "SavingsEvent", + "SavingsEvidenceLink", + "SavingsOpportunity", + "SavingsRealization", "SemanticParseLog", "SystemModelSetting", "SystemSetting", "SystemSettingSecret", + "Tenant", + "TenantCommercialPlan", + "TenantMembership", + "TenantFinanceReportConfig", + "TenantFinanceReportRun", + "TenantSubscription", + "UsageMeterEvent", "UserSessionMetric", "WorkflowOutcome", ] diff --git a/server/src/app/models/agent_asset.py b/server/src/app/models/agent_asset.py index e8f9482..766e42b 100644 --- a/server/src/app/models/agent_asset.py +++ b/server/src/app/models/agent_asset.py @@ -4,19 +4,64 @@ import uuid from datetime import datetime from typing import Any -from sqlalchemy import Boolean, DateTime, ForeignKey, Index, String, Text, UniqueConstraint, func +from sqlalchemy import ( + Boolean, + CheckConstraint, + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + String, + Text, + UniqueConstraint, + func, +) from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.types import JSON +from app.core.agent_asset_scope import AGENT_ASSET_PLATFORM_TENANT_ID from app.db.base_class import Base class AgentAsset(Base): __tablename__ = "agent_assets" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "scope", + "id", + name="uq_agent_assets_tenant_scope_id", + ), + UniqueConstraint( + "tenant_id", + "scope", + "code", + name="uq_agent_assets_tenant_scope_code", + ), + CheckConstraint( + "(scope = 'platform' AND tenant_id = 'platform') OR " + "(scope = 'tenant' AND tenant_id <> 'platform')", + name="ck_agent_assets_scope_tenant", + ), + Index("ix_agent_assets_scope_tenant", "scope", "tenant_id"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) + tenant_id: Mapped[str] = mapped_column( + String(64), + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + default=AGENT_ASSET_PLATFORM_TENANT_ID, + server_default=AGENT_ASSET_PLATFORM_TENANT_ID, + ) + scope: Mapped[str] = mapped_column( + String(16), + nullable=False, + default="platform", + server_default="platform", + ) asset_type: Mapped[str] = mapped_column(String(20), index=True) - code: Mapped[str] = mapped_column(String(100), unique=True, index=True) + code: Mapped[str] = mapped_column(String(100), index=True) name: Mapped[str] = mapped_column(String(200)) description: Mapped[str] = mapped_column(Text(), default="") domain: Mapped[str] = mapped_column(String(50), index=True) @@ -60,14 +105,94 @@ class AgentAsset(Base): ) +class AgentAssetOnlyOfficeSession(Base): + __tablename__ = "agent_asset_onlyoffice_sessions" + __table_args__ = ( + CheckConstraint( + "(resource_scope = 'platform' AND tenant_id = 'platform') OR " + "(resource_scope = 'tenant' AND tenant_id <> 'platform')", + name="ck_agent_asset_onlyoffice_sessions_scope_tenant", + ), + CheckConstraint( + "status IN ('active', 'processing', 'consumed', 'failed', 'revoked')", + name="ck_agent_asset_onlyoffice_sessions_status", + ), + CheckConstraint( + "(status = 'active' AND claimed_at IS NULL AND consumed_at IS NULL) OR " + "(status IN ('processing', 'failed') AND claimed_at IS NOT NULL " + "AND consumed_at IS NULL) OR " + "(status = 'consumed' AND claimed_at IS NOT NULL AND consumed_at IS NOT NULL) OR " + "(status = 'revoked' AND consumed_at IS NULL)", + name="ck_agent_asset_onlyoffice_sessions_lifecycle", + ), + Index( + "ix_agent_asset_onlyoffice_sessions_tenant_asset", + "tenant_id", + "resource_scope", + "asset_id", + "created_at", + ), + Index( + "ix_agent_asset_onlyoffice_sessions_status_expiry", + "status", + "expires_at", + ), + ) + + jti: Mapped[str] = mapped_column(String(36), primary_key=True) + tenant_id: Mapped[str] = mapped_column( + String(64), + ForeignKey("tenants.tenant_id", ondelete="CASCADE"), + nullable=False, + ) + resource_scope: Mapped[str] = mapped_column(String(16), nullable=False) + asset_id: Mapped[str] = mapped_column(String(100), nullable=False) + document_key: Mapped[str] = mapped_column(String(200), nullable=False) + document_version: Mapped[str] = mapped_column(String(30), nullable=False) + document_fingerprint: Mapped[str] = mapped_column(String(160), nullable=False) + audience: Mapped[str] = mapped_column(String(80), nullable=False) + writable: Mapped[bool] = mapped_column(Boolean, nullable=False) + status: Mapped[str] = mapped_column(String(16), nullable=False) + actor: Mapped[str] = mapped_column(String(160), nullable=False) + expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + consumed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + failure_reason: Mapped[str] = mapped_column(Text(), nullable=False, default="") + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), server_default=func.now(), nullable=False + ) + + class AgentAssetVersion(Base): __tablename__ = "agent_asset_versions" __table_args__ = ( UniqueConstraint("asset_id", "version", name="uq_agent_asset_versions_asset_version"), + CheckConstraint( + "(scope = 'platform' AND tenant_id = 'platform') OR " + "(scope = 'tenant' AND tenant_id <> 'platform')", + name="ck_agent_asset_versions_scope_tenant", + ), + ForeignKeyConstraint( + ["tenant_id", "scope", "asset_id"], + ["agent_assets.tenant_id", "agent_assets.scope", "agent_assets.id"], + ondelete="CASCADE", + name="fk_agent_asset_versions_tenant_asset", + ), + Index("ix_agent_asset_versions_tenant_asset", "tenant_id", "scope", "asset_id"), ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - asset_id: Mapped[str] = mapped_column(ForeignKey("agent_assets.id"), index=True) + tenant_id: Mapped[str] = mapped_column( + String(64), + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + default=AGENT_ASSET_PLATFORM_TENANT_ID, + server_default=AGENT_ASSET_PLATFORM_TENANT_ID, + ) + scope: Mapped[str] = mapped_column( + String(16), nullable=False, default="platform", server_default="platform" + ) + asset_id: Mapped[str] = mapped_column(String(36), index=True) version: Mapped[str] = mapped_column(String(30)) content: Mapped[str] = mapped_column(Text()) content_type: Mapped[str] = mapped_column(String(20)) @@ -80,9 +205,33 @@ class AgentAssetVersion(Base): class AgentAssetReview(Base): __tablename__ = "agent_asset_reviews" + __table_args__ = ( + CheckConstraint( + "(scope = 'platform' AND tenant_id = 'platform') OR " + "(scope = 'tenant' AND tenant_id <> 'platform')", + name="ck_agent_asset_reviews_scope_tenant", + ), + ForeignKeyConstraint( + ["tenant_id", "scope", "asset_id"], + ["agent_assets.tenant_id", "agent_assets.scope", "agent_assets.id"], + ondelete="CASCADE", + name="fk_agent_asset_reviews_tenant_asset", + ), + Index("ix_agent_asset_reviews_tenant_asset", "tenant_id", "scope", "asset_id"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - asset_id: Mapped[str] = mapped_column(ForeignKey("agent_assets.id"), index=True) + tenant_id: Mapped[str] = mapped_column( + String(64), + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + default=AGENT_ASSET_PLATFORM_TENANT_ID, + server_default=AGENT_ASSET_PLATFORM_TENANT_ID, + ) + scope: Mapped[str] = mapped_column( + String(16), nullable=False, default="platform", server_default="platform" + ) + asset_id: Mapped[str] = mapped_column(String(36), index=True) version: Mapped[str] = mapped_column(String(30)) reviewer: Mapped[str] = mapped_column(String(100)) review_status: Mapped[str] = mapped_column(String(20), index=True) @@ -95,9 +244,29 @@ class AgentAssetReview(Base): class AgentAssetTestRun(Base): __tablename__ = "agent_asset_test_runs" + __table_args__ = ( + CheckConstraint( + "(scope = 'platform' AND tenant_id = 'platform') OR " + "(scope = 'tenant' AND tenant_id <> 'platform')", + name="ck_agent_asset_test_runs_scope_tenant", + ), + Index("ix_agent_asset_test_runs_tenant_asset", "tenant_id", "scope", "asset_id"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - asset_id: Mapped[str] = mapped_column(ForeignKey("agent_assets.id"), index=True) + tenant_id: Mapped[str] = mapped_column( + String(64), + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + default=AGENT_ASSET_PLATFORM_TENANT_ID, + server_default=AGENT_ASSET_PLATFORM_TENANT_ID, + ) + scope: Mapped[str] = mapped_column( + String(16), nullable=False, default="platform", server_default="platform" + ) + asset_id: Mapped[str] = mapped_column( + ForeignKey("agent_assets.id", ondelete="CASCADE"), index=True + ) version: Mapped[str] = mapped_column(String(30), index=True) test_type: Mapped[str] = mapped_column(String(30), index=True) status: Mapped[str] = mapped_column(String(20), index=True) @@ -116,6 +285,17 @@ class AgentAssetRuleFeedback(Base): __table_args__ = ( Index("ix_agent_asset_rule_feedback_asset_version", "asset_id", "version"), Index("ix_agent_asset_rule_feedback_type_status", "feedback_type", "status"), + Index( + "ix_agent_asset_rule_feedback_tenant_asset", + "tenant_id", + "scope", + "asset_id", + ), + CheckConstraint( + "(scope = 'platform' AND tenant_id = 'platform') OR " + "(scope = 'tenant' AND tenant_id <> 'platform')", + name="ck_agent_asset_rule_feedback_scope_tenant", + ), ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) @@ -125,7 +305,19 @@ class AgentAssetRuleFeedback(Base): index=True, default=lambda: f"arf_{uuid.uuid4().hex[:16]}", ) - asset_id: Mapped[str] = mapped_column(ForeignKey("agent_assets.id"), index=True) + tenant_id: Mapped[str] = mapped_column( + String(64), + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + default=AGENT_ASSET_PLATFORM_TENANT_ID, + server_default=AGENT_ASSET_PLATFORM_TENANT_ID, + ) + scope: Mapped[str] = mapped_column( + String(16), nullable=False, default="platform", server_default="platform" + ) + asset_id: Mapped[str] = mapped_column( + ForeignKey("agent_assets.id", ondelete="CASCADE"), index=True + ) version: Mapped[str] = mapped_column(String(30), index=True) feedback_type: Mapped[str] = mapped_column(String(30), index=True) status: Mapped[str] = mapped_column(String(30), default="open", index=True) @@ -137,6 +329,8 @@ class AgentAssetRuleFeedback(Base): comment: Mapped[str | None] = mapped_column(Text(), nullable=True) payload_json: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict) created_by: Mapped[str] = mapped_column(String(100), default="", index=True) - created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), index=True) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), server_default=func.now(), index=True + ) asset = relationship("AgentAsset", back_populates="rule_feedback_items") diff --git a/server/src/app/models/agent_asset_release_telemetry.py b/server/src/app/models/agent_asset_release_telemetry.py new file mode 100644 index 0000000..dbdf074 --- /dev/null +++ b/server/src/app/models/agent_asset_release_telemetry.py @@ -0,0 +1,298 @@ +"""Agent 资产分阶段发布的只追加运行观察与人工标签。""" + +from __future__ import annotations + +import uuid +from datetime import datetime + +from sqlalchemy import ( + Boolean, + CheckConstraint, + DateTime, + ForeignKeyConstraint, + Index, + Integer, + String, + Text, + UniqueConstraint, + event, + func, +) +from sqlalchemy.orm import Mapped, mapped_column + +from app.db.base_class import Base + + +def _new_id() -> str: + return str(uuid.uuid4()) + + +class AgentAssetReleaseObservation(Base): + """一次真实规则执行的去敏事实；同一来源重放只能得到同一条记录。""" + + __tablename__ = "agent_asset_release_observations" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_agent_asset_release_observations_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_agent_asset_release_observations_tenant_idempotency", + ), + UniqueConstraint( + "tenant_id", + "id", + "asset_id", + "release_id", + "stage", + "version", + name="uq_agent_asset_release_observations_release_identity", + ), + CheckConstraint( + "stage IN ('shadow', 'canary', 'active')", + name="ck_agent_asset_release_observations_stage", + ), + CheckConstraint( + "runtime_status IN ('completed', 'failed')", + name="ck_agent_asset_release_observations_runtime_status", + ), + CheckConstraint( + "source_kind IN ('expense_claim_risk')", + name="ck_agent_asset_release_observations_source_kind", + ), + Index( + "ix_agent_asset_release_observations_release", + "tenant_id", + "asset_id", + "release_id", + "stage", + "version", + "created_at", + ), + Index( + "ix_agent_asset_release_observations_source", + "tenant_id", + "source_fingerprint", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + asset_id: Mapped[str] = mapped_column(String(36), nullable=False) + release_id: Mapped[str] = mapped_column(String(64), nullable=False) + stage: Mapped[str] = mapped_column(String(16), nullable=False) + version: Mapped[str] = mapped_column(String(30), nullable=False) + rule_code: Mapped[str] = mapped_column(String(100), nullable=False) + business_stage: Mapped[str] = mapped_column(String(40), nullable=False) + source_kind: Mapped[str] = mapped_column( + String(32), + nullable=False, + default="expense_claim_risk", + server_default="expense_claim_risk", + ) + # 只保存租户、单据和规则的不可逆指纹，不保存单号、事由或票据内容。 + source_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False) + candidate_hit: Mapped[bool] = mapped_column(Boolean, nullable=False) + baseline_hit: Mapped[bool | None] = mapped_column(Boolean, nullable=True) + runtime_status: Mapped[str] = mapped_column(String(16), nullable=False) + failure_code: Mapped[str] = mapped_column( + String(40), + nullable=False, + default="none", + server_default="none", + ) + idempotency_key: Mapped[str] = mapped_column(String(80), nullable=False) + payload_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + ) + + +class AgentAssetReleaseLabel(Base): + """对运行观察的人工真值标签；纠正通过追加新标签完成。""" + + __tablename__ = "agent_asset_release_labels" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_agent_asset_release_labels_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_agent_asset_release_labels_tenant_idempotency", + ), + CheckConstraint( + "label IN ('confirmed', 'false_positive', 'risk_present', 'risk_absent')", + name="ck_agent_asset_release_labels_label", + ), + CheckConstraint( + "verification_source IN ('typed_risk_disposition', 'release_review', " + "'blind_release_review')", + name="ck_agent_asset_release_labels_source", + ), + CheckConstraint( + "(verification_source = 'blind_release_review' " + "AND label IN ('risk_present', 'risk_absent')) OR " + "(verification_source IN ('typed_risk_disposition', 'release_review') " + "AND label IN ('confirmed', 'false_positive'))", + name="ck_agent_asset_release_labels_semantics", + ), + ForeignKeyConstraint( + [ + "tenant_id", + "observation_id", + "asset_id", + "release_id", + "stage", + "version", + ], + [ + "agent_asset_release_observations.tenant_id", + "agent_asset_release_observations.id", + "agent_asset_release_observations.asset_id", + "agent_asset_release_observations.release_id", + "agent_asset_release_observations.stage", + "agent_asset_release_observations.version", + ], + ondelete="RESTRICT", + name="fk_agent_asset_release_labels_release_observation", + ), + Index( + "ix_agent_asset_release_labels_observation_time", + "tenant_id", + "observation_id", + "created_at", + ), + Index( + "ix_agent_asset_release_labels_release", + "tenant_id", + "asset_id", + "release_id", + "stage", + "version", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + observation_id: Mapped[str] = mapped_column(String(36), nullable=False) + asset_id: Mapped[str] = mapped_column(String(36), nullable=False) + release_id: Mapped[str] = mapped_column(String(64), nullable=False) + stage: Mapped[str] = mapped_column(String(16), nullable=False) + version: Mapped[str] = mapped_column(String(30), nullable=False) + label: Mapped[str] = mapped_column(String(24), nullable=False) + verification_source: Mapped[str] = mapped_column(String(32), nullable=False) + # 事件和操作者只保留租户内稳定指纹，避免把账号标识带入评测数据集。 + source_event_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False) + actor_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(80), nullable=False) + payload_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + ) + + +class AgentAssetReleaseAuditSample(Base): + """发布盲审样本；模型结论和业务来源只在服务端受控解析。""" + + __tablename__ = "agent_asset_release_audit_samples" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_agent_asset_release_audit_samples_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "observation_id", + name="uq_agent_asset_release_audit_samples_observation", + ), + UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_agent_asset_release_audit_samples_idempotency", + ), + CheckConstraint( + "stratum IN ('candidate_positive_census', " + "'candidate_disagreement_census', 'candidate_negative_random')", + name="ck_agent_asset_release_audit_samples_stratum", + ), + CheckConstraint( + "sampling_probability_ppm BETWEEN 1 AND 1000000", + name="ck_agent_asset_release_audit_samples_probability", + ), + CheckConstraint( + "selection_score_ppm BETWEEN 0 AND 999999", + name="ck_agent_asset_release_audit_samples_score", + ), + ForeignKeyConstraint( + [ + "tenant_id", + "observation_id", + "asset_id", + "release_id", + "stage", + "version", + ], + [ + "agent_asset_release_observations.tenant_id", + "agent_asset_release_observations.id", + "agent_asset_release_observations.asset_id", + "agent_asset_release_observations.release_id", + "agent_asset_release_observations.stage", + "agent_asset_release_observations.version", + ], + ondelete="RESTRICT", + name="fk_agent_asset_release_audit_samples_observation", + ), + Index( + "ix_agent_asset_release_audit_samples_release", + "tenant_id", + "asset_id", + "release_id", + "stage", + "version", + "created_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + observation_id: Mapped[str] = mapped_column(String(36), nullable=False) + asset_id: Mapped[str] = mapped_column(String(36), nullable=False) + release_id: Mapped[str] = mapped_column(String(64), nullable=False) + stage: Mapped[str] = mapped_column(String(16), nullable=False) + version: Mapped[str] = mapped_column(String(30), nullable=False) + stratum: Mapped[str] = mapped_column(String(40), nullable=False) + sampling_probability_ppm: Mapped[int] = mapped_column(Integer, nullable=False) + selection_score_ppm: Mapped[int] = mapped_column(Integer, nullable=False) + source_reference_encrypted: Mapped[str] = mapped_column(Text, nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(80), nullable=False) + payload_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + ) + + +def _reject_mutation(_mapper: object, _connection: object, target: object) -> None: + raise ValueError(f"{type(target).__name__} is append-only and cannot be mutated.") + + +for _model in ( + AgentAssetReleaseObservation, + AgentAssetReleaseLabel, + AgentAssetReleaseAuditSample, +): + event.listen(_model, "before_update", _reject_mutation) + event.listen(_model, "before_delete", _reject_mutation) diff --git a/server/src/app/models/auth_session.py b/server/src/app/models/auth_session.py index c1b445d..92d1707 100644 --- a/server/src/app/models/auth_session.py +++ b/server/src/app/models/auth_session.py @@ -18,7 +18,7 @@ class AuthSession(Base): id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) token_hash: Mapped[str] = mapped_column(String(64), unique=True) - tenant_id: Mapped[str] = mapped_column(String(64), default="default", index=True) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True) principal_type: Mapped[str] = mapped_column(String(20), index=True) employee_id: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True) username: Mapped[str] = mapped_column(String(255), index=True) diff --git a/server/src/app/models/commercial.py b/server/src/app/models/commercial.py new file mode 100644 index 0000000..de58d5b --- /dev/null +++ b/server/src/app/models/commercial.py @@ -0,0 +1,607 @@ +from __future__ import annotations + +import uuid +from datetime import datetime +from decimal import Decimal +from typing import Any + +from sqlalchemy import ( + Boolean, + CheckConstraint, + DateTime, + ForeignKeyConstraint, + Index, + Integer, + Numeric, + String, + UniqueConstraint, + func, + text, +) +from sqlalchemy.orm import Mapped, mapped_column, relationship +from sqlalchemy.types import JSON + +from app.db.base_class import Base + + +def _new_id() -> str: + return str(uuid.uuid4()) + + +class TenantCommercialPlan(Base): + """租户已协商的商业套餐版本，不与平台内部成本或客户节省事实混用。""" + + __tablename__ = "tenant_commercial_plans" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_tenant_commercial_plans_tenant_id"), + UniqueConstraint( + "tenant_id", + "plan_code", + "version", + name="uq_tenant_commercial_plans_tenant_code_version", + ), + CheckConstraint( + "pricing_model IN ('subscription', 'usage', 'hybrid', 'pilot', 'custom')", + name="ck_tenant_commercial_plans_pricing_model", + ), + CheckConstraint( + "billing_interval IN ('monthly', 'quarterly', 'annual', 'contract')", + name="ck_tenant_commercial_plans_billing_interval", + ), + CheckConstraint( + "status IN ('draft', 'active', 'retired')", + name="ck_tenant_commercial_plans_status", + ), + CheckConstraint( + "base_fee >= 0 AND included_seats >= 0 AND version >= 1", + name="ck_tenant_commercial_plans_values", + ), + CheckConstraint( + "length(trim(plan_code)) > 0 AND length(trim(name)) > 0", + name="ck_tenant_commercial_plans_keys", + ), + CheckConstraint( + "length(trim(currency)) = 3", + name="ck_tenant_commercial_plans_currency", + ), + CheckConstraint( + "effective_to IS NULL OR effective_to > effective_from", + name="ck_tenant_commercial_plans_effective_window", + ), + Index( + "uq_tenant_commercial_plans_active_code", + "tenant_id", + "plan_code", + unique=True, + postgresql_where=text("status = 'active'"), + ), + Index( + "ix_tenant_commercial_plans_tenant_status", + "tenant_id", + "status", + "effective_from", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + plan_code: Mapped[str] = mapped_column(String(80), nullable=False) + name: Mapped[str] = mapped_column(String(160), nullable=False) + pricing_model: Mapped[str] = mapped_column(String(24), nullable=False) + billing_interval: Mapped[str] = mapped_column(String(20), nullable=False) + currency: Mapped[str] = mapped_column(String(3), nullable=False) + base_fee: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + included_seats: Mapped[int] = mapped_column( + Integer, nullable=False, default=0, server_default="0" + ) + overage_enabled: Mapped[bool] = mapped_column( + Boolean, nullable=False, default=False, server_default="false" + ) + status: Mapped[str] = mapped_column( + String(16), nullable=False, default="draft", server_default="draft" + ) + effective_from: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + effective_to: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1") + contract_terms_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + created_by: Mapped[str] = mapped_column(String(120), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now() + ) + + subscriptions = relationship("TenantSubscription", back_populates="plan", passive_deletes=True) + + +class TenantSubscription(Base): + """租户订阅及当期计费快照，避免套餐后续变化污染历史账期。""" + + __tablename__ = "tenant_subscriptions" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_tenant_subscriptions_tenant_id"), + UniqueConstraint( + "tenant_id", "subscription_key", name="uq_tenant_subscriptions_tenant_key" + ), + UniqueConstraint( + "tenant_id", + "external_provider", + "external_subscription_id", + name="uq_tenant_subscriptions_external_ref", + ), + ForeignKeyConstraint( + ["tenant_id", "plan_id"], + ["tenant_commercial_plans.tenant_id", "tenant_commercial_plans.id"], + name="fk_tenant_subscriptions_tenant_plan", + ondelete="RESTRICT", + ), + CheckConstraint( + "status IN ('trialing', 'active', 'past_due', 'suspended', 'canceled', 'expired')", + name="ck_tenant_subscriptions_status", + ), + CheckConstraint( + "billing_interval IN ('monthly', 'quarterly', 'annual', 'contract')", + name="ck_tenant_subscriptions_billing_interval", + ), + CheckConstraint( + "seats > 0 AND base_fee_snapshot >= 0 AND version >= 1", + name="ck_tenant_subscriptions_values", + ), + CheckConstraint( + "length(trim(subscription_key)) > 0 AND length(trim(currency)) = 3", + name="ck_tenant_subscriptions_keys", + ), + CheckConstraint( + "current_period_end > current_period_start", + name="ck_tenant_subscriptions_period", + ), + CheckConstraint( + "ends_at IS NULL OR ends_at > starts_at", + name="ck_tenant_subscriptions_contract_window", + ), + CheckConstraint( + "(external_provider IS NULL AND external_subscription_id IS NULL) OR " + "(external_provider IS NOT NULL AND external_subscription_id IS NOT NULL)", + name="ck_tenant_subscriptions_external_pair", + ), + CheckConstraint( + "status != 'canceled' OR canceled_at IS NOT NULL", + name="ck_tenant_subscriptions_cancellation", + ), + Index( + "uq_tenant_subscriptions_current", + "tenant_id", + unique=True, + postgresql_where=text("status IN ('trialing', 'active', 'past_due', 'suspended')"), + ), + Index( + "ix_tenant_subscriptions_tenant_status_period", + "tenant_id", + "status", + "current_period_end", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + subscription_key: Mapped[str] = mapped_column(String(120), nullable=False) + plan_id: Mapped[str] = mapped_column(String(36), nullable=False) + status: Mapped[str] = mapped_column(String(20), nullable=False) + starts_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + ends_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + current_period_start: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + current_period_end: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + seats: Mapped[int] = mapped_column(Integer, nullable=False) + base_fee_snapshot: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + currency: Mapped[str] = mapped_column(String(3), nullable=False) + billing_interval: Mapped[str] = mapped_column(String(20), nullable=False) + auto_renew: Mapped[bool] = mapped_column( + Boolean, nullable=False, default=False, server_default="false" + ) + external_provider: Mapped[str | None] = mapped_column(String(60)) + external_subscription_id: Mapped[str | None] = mapped_column(String(160)) + canceled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1") + metadata_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + created_by: Mapped[str] = mapped_column(String(120), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now() + ) + + plan = relationship("TenantCommercialPlan", back_populates="subscriptions") + entitlements = relationship( + "CommercialEntitlement", back_populates="subscription", passive_deletes=True + ) + + +class CommercialEntitlement(Base): + """订阅的功能权益与可计量配额定义。""" + + __tablename__ = "commercial_entitlements" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_commercial_entitlements_tenant_id"), + UniqueConstraint( + "tenant_id", + "subscription_id", + "id", + name="uq_commercial_entitlements_tenant_subscription_id", + ), + UniqueConstraint( + "tenant_id", + "subscription_id", + "entitlement_key", + name="uq_commercial_entitlements_subscription_key", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_entitlements_tenant_subscription", + ondelete="RESTRICT", + ), + CheckConstraint( + "entitlement_type IN ('feature', 'metered', 'unlimited')", + name="ck_commercial_entitlements_type", + ), + CheckConstraint( + "reset_interval IN ('none', 'monthly', 'quarterly', 'annual', 'contract')", + name="ck_commercial_entitlements_reset_interval", + ), + CheckConstraint( + "overage_policy IN ('block', 'allow', 'alert')", + name="ck_commercial_entitlements_overage_policy", + ), + CheckConstraint( + "status IN ('active', 'suspended', 'expired')", + name="ck_commercial_entitlements_status", + ), + CheckConstraint( + "version >= 1 AND (included_quantity IS NULL OR included_quantity >= 0) " + "AND (hard_limit_quantity IS NULL OR hard_limit_quantity >= 0)", + name="ck_commercial_entitlements_values", + ), + CheckConstraint( + "(entitlement_type = 'unlimited' AND included_quantity IS NULL " + "AND hard_limit_quantity IS NULL) OR " + "(entitlement_type = 'feature' AND included_quantity IN (0, 1) " + "AND (hard_limit_quantity IS NULL OR hard_limit_quantity IN (0, 1))) OR " + "(entitlement_type = 'metered' AND included_quantity IS NOT NULL " + "AND (hard_limit_quantity IS NULL OR hard_limit_quantity >= included_quantity))", + name="ck_commercial_entitlements_quota_shape", + ), + CheckConstraint( + "length(trim(entitlement_key)) > 0 AND length(trim(metric_key)) > 0 " + "AND length(trim(unit)) > 0", + name="ck_commercial_entitlements_keys", + ), + CheckConstraint( + "effective_to IS NULL OR effective_to > effective_from", + name="ck_commercial_entitlements_effective_window", + ), + Index( + "ix_commercial_entitlements_subscription_status", + "tenant_id", + "subscription_id", + "status", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + subscription_id: Mapped[str] = mapped_column(String(36), nullable=False) + entitlement_key: Mapped[str] = mapped_column(String(120), nullable=False) + metric_key: Mapped[str] = mapped_column(String(120), nullable=False) + entitlement_type: Mapped[str] = mapped_column(String(20), nullable=False) + unit: Mapped[str] = mapped_column(String(40), nullable=False) + included_quantity: Mapped[Decimal | None] = mapped_column(Numeric(20, 6)) + hard_limit_quantity: Mapped[Decimal | None] = mapped_column(Numeric(20, 6)) + reset_interval: Mapped[str] = mapped_column(String(20), nullable=False) + overage_policy: Mapped[str] = mapped_column(String(16), nullable=False) + status: Mapped[str] = mapped_column(String(16), nullable=False) + effective_from: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + effective_to: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1") + config_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now() + ) + + subscription = relationship("TenantSubscription", back_populates="entitlements") + + +class UsageMeterEvent(Base): + """追加只读的客户用量事实；同一来源幂等键只能产生一条事件。""" + + __tablename__ = "usage_meter_events" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_usage_meter_events_tenant_id"), + UniqueConstraint( + "tenant_id", + "subscription_id", + "id", + name="uq_usage_meter_events_tenant_subscription_id", + ), + UniqueConstraint( + "tenant_id", + "subscription_id", + "entitlement_id", + "id", + name="uq_usage_meter_events_entitlement_id", + ), + UniqueConstraint( + "tenant_id", + "source_system", + "idempotency_key", + name="uq_usage_meter_events_source_request", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_usage_meter_events_tenant_subscription", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id", "entitlement_id"], + [ + "commercial_entitlements.tenant_id", + "commercial_entitlements.subscription_id", + "commercial_entitlements.id", + ], + name="fk_usage_meter_events_tenant_entitlement", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id", "billing_period_id"], + [ + "commercial_billing_periods.tenant_id", + "commercial_billing_periods.subscription_id", + "commercial_billing_periods.id", + ], + name="fk_usage_meter_events_tenant_billing_period", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id", "entitlement_id", "reversal_of_event_id"], + [ + "usage_meter_events.tenant_id", + "usage_meter_events.subscription_id", + "usage_meter_events.entitlement_id", + "usage_meter_events.id", + ], + name="fk_usage_meter_events_tenant_reversal", + ondelete="RESTRICT", + ), + CheckConstraint( + "event_type IN ('usage', 'credit', 'adjustment', 'reversal')", + name="ck_usage_meter_events_type", + ), + CheckConstraint( + "(event_type = 'usage' AND quantity > 0) OR " + "(event_type = 'credit' AND quantity < 0) OR " + "(event_type IN ('adjustment', 'reversal') AND quantity <> 0)", + name="ck_usage_meter_events_quantity", + ), + CheckConstraint( + "(event_type = 'reversal' AND reversal_of_event_id IS NOT NULL) OR " + "(event_type != 'reversal' AND reversal_of_event_id IS NULL)", + name="ck_usage_meter_events_reversal", + ), + CheckConstraint( + "(subject_type IS NULL AND subject_id IS NULL) OR " + "(subject_type IS NOT NULL AND subject_id IS NOT NULL)", + name="ck_usage_meter_events_subject_pair", + ), + CheckConstraint( + "actor_type IN ('system', 'user', 'integration', 'admin')", + name="ck_usage_meter_events_actor_type", + ), + CheckConstraint( + "length(trim(metric_key)) > 0 AND length(trim(unit)) > 0 " + "AND length(trim(period_key)) > 0 AND length(trim(source_system)) > 0 " + "AND length(trim(quota_period_key)) > 0 " + "AND length(trim(idempotency_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + name="ck_usage_meter_events_keys", + ), + Index( + "ix_usage_meter_events_quota_window", + "tenant_id", + "subscription_id", + "metric_key", + "quota_period_key", + "occurred_at", + ), + Index( + "ix_usage_meter_events_billing_period", + "tenant_id", + "billing_period_id", + "occurred_at", + ), + Index( + "ix_usage_meter_events_correlation", + "tenant_id", + "correlation_id", + "occurred_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + subscription_id: Mapped[str] = mapped_column(String(36), nullable=False) + entitlement_id: Mapped[str] = mapped_column(String(36), nullable=False) + billing_period_id: Mapped[str] = mapped_column(String(36), nullable=False) + event_type: Mapped[str] = mapped_column(String(16), nullable=False) + metric_key: Mapped[str] = mapped_column(String(120), nullable=False) + quantity: Mapped[Decimal] = mapped_column(Numeric(20, 6), nullable=False) + unit: Mapped[str] = mapped_column(String(40), nullable=False) + period_key: Mapped[str] = mapped_column(String(64), nullable=False) + quota_period_key: Mapped[str] = mapped_column(String(64), nullable=False) + occurred_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + source_system: Mapped[str] = mapped_column(String(80), nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(160), nullable=False) + request_fingerprint: Mapped[str] = mapped_column(String(80), nullable=False) + reversal_of_event_id: Mapped[str | None] = mapped_column(String(36)) + subject_type: Mapped[str | None] = mapped_column(String(60)) + subject_id: Mapped[str | None] = mapped_column(String(160)) + actor_type: Mapped[str] = mapped_column(String(20), nullable=False) + actor_id: Mapped[str] = mapped_column(String(120), nullable=False) + correlation_id: Mapped[str | None] = mapped_column(String(120)) + trace_id: Mapped[str | None] = mapped_column(String(120)) + metadata_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + recorded_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + +class CommercialCostEvent(Base): + """平台内部成本事实；物理上独立于客户节省、价值机会和价值实现。""" + + __tablename__ = "commercial_cost_events" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_commercial_cost_events_tenant_id"), + UniqueConstraint( + "tenant_id", + "source_system", + "idempotency_key", + name="uq_commercial_cost_events_source_request", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_cost_events_tenant_subscription", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id", "usage_event_id"], + [ + "usage_meter_events.tenant_id", + "usage_meter_events.subscription_id", + "usage_meter_events.id", + ], + name="fk_commercial_cost_events_tenant_usage", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id", "billing_period_id"], + [ + "commercial_billing_periods.tenant_id", + "commercial_billing_periods.subscription_id", + "commercial_billing_periods.id", + ], + name="fk_commercial_cost_events_tenant_billing_period", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "reversal_of_cost_event_id"], + ["commercial_cost_events.tenant_id", "commercial_cost_events.id"], + name="fk_commercial_cost_events_tenant_reversal", + ondelete="RESTRICT", + ), + CheckConstraint( + "event_type IN ('incurred', 'credit', 'adjustment', 'reversal')", + name="ck_commercial_cost_events_type", + ), + CheckConstraint( + "cost_category IN ('ai_inference', 'ocr', 'storage', 'connector', " + "'support', 'implementation', 'infrastructure', 'payment', 'other')", + name="ck_commercial_cost_events_category", + ), + CheckConstraint( + "quantity > 0 AND unit_cost >= 0 AND fx_rate > 0", + name="ck_commercial_cost_events_values", + ), + CheckConstraint( + "(event_type = 'incurred' AND cost_amount >= 0 AND reporting_amount >= 0) OR " + "(event_type = 'credit' AND cost_amount <= 0 AND reporting_amount <= 0) OR " + "(event_type IN ('adjustment', 'reversal') AND cost_amount <> 0 " + "AND reporting_amount <> 0)", + name="ck_commercial_cost_events_amount_direction", + ), + CheckConstraint( + "(event_type = 'reversal' AND reversal_of_cost_event_id IS NOT NULL) OR " + "(event_type != 'reversal' AND reversal_of_cost_event_id IS NULL)", + name="ck_commercial_cost_events_reversal", + ), + CheckConstraint( + "usage_event_id IS NULL OR subscription_id IS NOT NULL", + name="ck_commercial_cost_events_usage_pair", + ), + CheckConstraint( + "(subscription_id IS NULL AND billing_period_id IS NULL) OR " + "(subscription_id IS NOT NULL AND billing_period_id IS NOT NULL)", + name="ck_commercial_cost_events_billing_period_pair", + ), + CheckConstraint( + "length(trim(unit)) > 0 AND length(trim(source_system)) > 0 " + "AND length(trim(idempotency_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0 " + "AND length(trim(original_currency)) = 3 " + "AND length(trim(reporting_currency)) = 3", + name="ck_commercial_cost_events_keys", + ), + Index( + "ix_commercial_cost_events_tenant_period", + "tenant_id", + "occurred_at", + "cost_category", + ), + Index( + "ix_commercial_cost_events_subscription_period", + "tenant_id", + "subscription_id", + "billing_period_id", + "occurred_at", + ), + Index( + "ix_commercial_cost_events_allocation", + "tenant_id", + "allocation_key", + "occurred_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + subscription_id: Mapped[str | None] = mapped_column(String(36)) + billing_period_id: Mapped[str | None] = mapped_column(String(36)) + usage_event_id: Mapped[str | None] = mapped_column(String(36)) + event_type: Mapped[str] = mapped_column(String(16), nullable=False) + cost_category: Mapped[str] = mapped_column(String(32), nullable=False) + quantity: Mapped[Decimal] = mapped_column(Numeric(20, 6), nullable=False) + unit: Mapped[str] = mapped_column(String(40), nullable=False) + unit_cost: Mapped[Decimal] = mapped_column(Numeric(20, 8), nullable=False) + cost_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + original_currency: Mapped[str] = mapped_column(String(3), nullable=False) + reporting_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + reporting_currency: Mapped[str] = mapped_column(String(3), nullable=False) + fx_rate: Mapped[Decimal] = mapped_column(Numeric(20, 8), nullable=False) + provider: Mapped[str | None] = mapped_column(String(120)) + sku: Mapped[str | None] = mapped_column(String(120)) + model_name: Mapped[str | None] = mapped_column(String(120)) + allocation_key: Mapped[str] = mapped_column(String(160), nullable=False) + occurred_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + source_system: Mapped[str] = mapped_column(String(80), nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(160), nullable=False) + request_fingerprint: Mapped[str] = mapped_column(String(80), nullable=False) + reversal_of_cost_event_id: Mapped[str | None] = mapped_column(String(36)) + correlation_id: Mapped[str | None] = mapped_column(String(120)) + trace_id: Mapped[str | None] = mapped_column(String(120)) + metadata_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + recorded_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + +__all__ = [ + "CommercialCostEvent", + "CommercialEntitlement", + "TenantCommercialPlan", + "TenantSubscription", + "UsageMeterEvent", +] diff --git a/server/src/app/models/commercial_billing.py b/server/src/app/models/commercial_billing.py new file mode 100644 index 0000000..931f0ad --- /dev/null +++ b/server/src/app/models/commercial_billing.py @@ -0,0 +1,238 @@ +from __future__ import annotations + +import uuid +from datetime import datetime +from decimal import Decimal +from typing import Any + +from sqlalchemy import ( + CheckConstraint, + DateTime, + ForeignKeyConstraint, + Index, + Integer, + Numeric, + String, + Text, + UniqueConstraint, + func, +) +from sqlalchemy.orm import Mapped, mapped_column +from sqlalchemy.types import JSON + +from app.db.base_class import Base + + +def _new_id() -> str: + return str(uuid.uuid4()) + + +class CommercialBillingPeriod(Base): + """不可变的订阅账期签发事实；时间态由窗口计算，不回写状态。""" + + __tablename__ = "commercial_billing_periods" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_commercial_billing_periods_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "subscription_id", + "id", + name="uq_commercial_billing_periods_tenant_subscription_id", + ), + UniqueConstraint( + "tenant_id", + "subscription_id", + "period_sequence", + name="uq_commercial_billing_periods_subscription_sequence", + ), + UniqueConstraint( + "tenant_id", + "subscription_id", + "period_key", + name="uq_commercial_billing_periods_subscription_key", + ), + UniqueConstraint( + "tenant_id", + "subscription_id", + "period_start", + name="uq_commercial_billing_periods_subscription_start", + ), + UniqueConstraint( + "tenant_id", + "subscription_id", + "idempotency_key", + name="uq_commercial_billing_periods_subscription_request", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_billing_periods_tenant_subscription", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "plan_id"], + ["tenant_commercial_plans.tenant_id", "tenant_commercial_plans.id"], + name="fk_commercial_billing_periods_tenant_plan", + ondelete="RESTRICT", + ), + CheckConstraint( + "status = 'issued'", + name="ck_commercial_billing_periods_status", + ), + CheckConstraint( + "subscription_status_snapshot IN ('trialing', 'active', 'past_due', " + "'suspended', 'canceled', 'expired')", + name="ck_commercial_billing_periods_subscription_status", + ), + CheckConstraint( + "pricing_model_snapshot IN ('subscription', 'usage', 'hybrid', 'pilot', 'custom')", + name="ck_commercial_billing_periods_pricing_model", + ), + CheckConstraint( + "billing_interval IN ('monthly', 'quarterly', 'annual', 'contract')", + name="ck_commercial_billing_periods_interval", + ), + CheckConstraint( + "source IN ('subscription_created', 'auto_renew', 'migration_backfill')", + name="ck_commercial_billing_periods_source", + ), + CheckConstraint( + "period_sequence >= 1 AND period_end > period_start " + "AND plan_version_snapshot >= 1 AND base_fee_snapshot >= 0 " + "AND seats_snapshot > 0", + name="ck_commercial_billing_periods_values", + ), + CheckConstraint( + "length(trim(period_key)) > 0 AND length(trim(plan_code_snapshot)) > 0 " + "AND length(trim(currency)) = 3 AND length(trim(idempotency_key)) > 0 " + "AND length(trim(created_by)) > 0", + name="ck_commercial_billing_periods_keys", + ), + Index( + "ix_commercial_billing_periods_tenant_window", + "tenant_id", + "period_start", + "period_end", + ), + Index( + "ix_commercial_billing_periods_subscription_window", + "tenant_id", + "subscription_id", + "period_start", + "period_end", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + subscription_id: Mapped[str] = mapped_column(String(36), nullable=False) + plan_id: Mapped[str] = mapped_column(String(36), nullable=False) + period_sequence: Mapped[int] = mapped_column(Integer, nullable=False) + period_key: Mapped[str] = mapped_column(String(64), nullable=False) + status: Mapped[str] = mapped_column( + String(16), nullable=False, default="issued", server_default="issued" + ) + period_start: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + period_end: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + subscription_status_snapshot: Mapped[str] = mapped_column(String(20), nullable=False) + plan_code_snapshot: Mapped[str] = mapped_column(String(80), nullable=False) + plan_version_snapshot: Mapped[int] = mapped_column(Integer, nullable=False) + pricing_model_snapshot: Mapped[str] = mapped_column(String(24), nullable=False) + billing_interval: Mapped[str] = mapped_column(String(20), nullable=False) + currency: Mapped[str] = mapped_column(String(3), nullable=False) + base_fee_snapshot: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + seats_snapshot: Mapped[int] = mapped_column(Integer, nullable=False) + source: Mapped[str] = mapped_column(String(32), nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(160), nullable=False) + created_by: Mapped[str] = mapped_column(String(120), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + +class CommercialAdminEvent(Base): + """商业配置与续期动作的脱敏追加式审计事实。""" + + __tablename__ = "commercial_admin_events" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_commercial_admin_events_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "request_id", + "action", + "resource_type", + "resource_id", + name="uq_commercial_admin_events_request_resource", + ), + CheckConstraint( + "actor_type IN ('user', 'system', 'migration')", + name="ck_commercial_admin_events_actor_type", + ), + CheckConstraint( + "action IN ('plan_created', 'plan_activated', 'plan_retired', " + "'subscription_created', 'subscription_activated', " + "'subscription_transitioned', 'entitlement_created', " + "'entitlement_updated', 'entitlement_activated', " + "'billing_period_created', 'subscription_rolled_over', " + "'legacy_state_imported')", + name="ck_commercial_admin_events_action", + ), + CheckConstraint( + "resource_type IN ('plan', 'subscription', 'entitlement', 'billing_period')", + name="ck_commercial_admin_events_resource_type", + ), + CheckConstraint( + "resource_version >= 1", + name="ck_commercial_admin_events_resource_version", + ), + CheckConstraint( + "length(trim(actor_id)) > 0 AND length(trim(request_id)) > 0 " + "AND length(trim(reason)) > 0 AND length(trim(resource_id)) > 0", + name="ck_commercial_admin_events_required_text", + ), + Index( + "ix_commercial_admin_events_tenant_time", + "tenant_id", + "occurred_at", + "id", + ), + Index( + "ix_commercial_admin_events_tenant_resource", + "tenant_id", + "resource_type", + "resource_id", + "occurred_at", + ), + Index( + "ix_commercial_admin_events_tenant_request", + "tenant_id", + "request_id", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + actor_type: Mapped[str] = mapped_column(String(16), nullable=False) + actor_id: Mapped[str] = mapped_column(String(120), nullable=False) + request_id: Mapped[str] = mapped_column(String(120), nullable=False) + reason: Mapped[str] = mapped_column(Text, nullable=False) + action: Mapped[str] = mapped_column(String(48), nullable=False) + resource_type: Mapped[str] = mapped_column(String(24), nullable=False) + resource_id: Mapped[str] = mapped_column(String(36), nullable=False) + resource_version: Mapped[int] = mapped_column(Integer, nullable=False) + before_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + after_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + occurred_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + +__all__ = ["CommercialAdminEvent", "CommercialBillingPeriod"] diff --git a/server/src/app/models/commercial_runtime.py b/server/src/app/models/commercial_runtime.py new file mode 100644 index 0000000..6286738 --- /dev/null +++ b/server/src/app/models/commercial_runtime.py @@ -0,0 +1,167 @@ +from __future__ import annotations + +import uuid +from datetime import datetime +from decimal import Decimal +from typing import Any + +from sqlalchemy import ( + CheckConstraint, + DateTime, + ForeignKeyConstraint, + Index, + Numeric, + String, + UniqueConstraint, + func, +) +from sqlalchemy.orm import Mapped, mapped_column +from sqlalchemy.types import JSON + +from app.db.base_class import Base + + +class CommercialRuntimeReservation(Base): + """工具执行前的额度占位；它是可结算状态，不是客户用量事实。""" + + __tablename__ = "commercial_runtime_reservations" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_commercial_runtime_reservations_tenant_id", + ), + UniqueConstraint( + "tool_call_id", + name="uq_commercial_runtime_reservations_tool_call", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id"], + ["tenant_subscriptions.tenant_id", "tenant_subscriptions.id"], + name="fk_commercial_runtime_reservations_tenant_subscription", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id", "entitlement_id"], + [ + "commercial_entitlements.tenant_id", + "commercial_entitlements.subscription_id", + "commercial_entitlements.id", + ], + name="fk_commercial_runtime_reservations_tenant_entitlement", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "subscription_id", "billing_period_id"], + [ + "commercial_billing_periods.tenant_id", + "commercial_billing_periods.subscription_id", + "commercial_billing_periods.id", + ], + name="fk_commercial_runtime_reservations_tenant_billing_period", + ondelete="RESTRICT", + ), + CheckConstraint( + "status IN ('reserved', 'committed', 'released', 'expired', " + "'reconciliation_required', 'committed_reconciliation_required')", + name="ck_commercial_runtime_reservations_status", + ), + CheckConstraint( + "quantity_basis IN ('call', 'input_tokens', 'output_tokens', " + "'total_tokens', 'duration_ms', 'bytes', 'pages', 'objects', 'events')", + name="ck_commercial_runtime_reservations_basis", + ), + CheckConstraint( + "reserved_quantity > 0 AND (actual_quantity IS NULL OR actual_quantity > 0)", + name="ck_commercial_runtime_reservations_quantity", + ), + CheckConstraint( + "expires_at > created_at", + name="ck_commercial_runtime_reservations_expiry", + ), + CheckConstraint( + "(status = 'reserved' AND actual_quantity IS NULL AND settled_at IS NULL " + "AND resolution_code IS NULL) OR " + "(status = 'committed' AND actual_quantity IS NOT NULL " + "AND actual_quantity <= reserved_quantity AND settled_at IS NOT NULL " + "AND resolution_code IS NULL) OR " + "(status IN ('released', 'expired') AND actual_quantity IS NULL " + "AND settled_at IS NOT NULL AND resolution_code IS NOT NULL) OR " + "(status = 'reconciliation_required' AND settled_at IS NULL " + "AND resolution_code IS NOT NULL) OR " + "(status = 'committed_reconciliation_required' " + "AND actual_quantity IS NOT NULL " + "AND actual_quantity <= reserved_quantity " + "AND settled_at IS NOT NULL AND resolution_code IS NOT NULL)", + name="ck_commercial_runtime_reservations_state", + ), + CheckConstraint( + "length(trim(run_id)) > 0 AND length(trim(tool_call_id)) > 0 " + "AND length(trim(tool_type)) > 0 AND length(trim(tool_name)) > 0 " + "AND length(trim(period_key)) > 0 " + "AND length(trim(quota_period_key)) > 0 " + "AND length(trim(request_fingerprint)) > 0", + name="ck_commercial_runtime_reservations_keys", + ), + Index( + "ix_commercial_runtime_reservations_quota", + "tenant_id", + "subscription_id", + "entitlement_id", + "quota_period_key", + "status", + ), + Index( + "ix_commercial_runtime_reservations_billing_period", + "tenant_id", + "billing_period_id", + "status", + ), + Index( + "ix_commercial_runtime_reservations_expiry", + "status", + "expires_at", + ), + Index( + "ix_commercial_runtime_reservations_run", + "tenant_id", + "run_id", + "created_at", + ), + ) + + id: Mapped[str] = mapped_column( + String(36), + primary_key=True, + default=lambda: str(uuid.uuid4()), + ) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + subscription_id: Mapped[str] = mapped_column(String(36), nullable=False) + entitlement_id: Mapped[str] = mapped_column(String(36), nullable=False) + billing_period_id: Mapped[str] = mapped_column(String(36), nullable=False) + run_id: Mapped[str] = mapped_column(String(50), nullable=False) + tool_call_id: Mapped[str] = mapped_column(String(36), nullable=False) + tool_type: Mapped[str] = mapped_column(String(30), nullable=False) + tool_name: Mapped[str] = mapped_column(String(100), nullable=False) + quantity_basis: Mapped[str] = mapped_column(String(20), nullable=False) + reserved_quantity: Mapped[Decimal] = mapped_column(Numeric(20, 6), nullable=False) + actual_quantity: Mapped[Decimal | None] = mapped_column(Numeric(20, 6)) + period_key: Mapped[str] = mapped_column(String(64), nullable=False) + quota_period_key: Mapped[str] = mapped_column(String(64), nullable=False) + status: Mapped[str] = mapped_column(String(32), nullable=False) + request_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False) + meter_config_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + resolution_code: Mapped[str | None] = mapped_column(String(64)) + expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + settled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + onupdate=func.now(), + ) diff --git a/server/src/app/models/employee.py b/server/src/app/models/employee.py index 8268017..896497c 100644 --- a/server/src/app/models/employee.py +++ b/server/src/app/models/employee.py @@ -3,8 +3,21 @@ from __future__ import annotations import uuid from datetime import date, datetime -from sqlalchemy import Boolean, Column, Date, DateTime, ForeignKey, Integer, String, Table, func -from sqlalchemy.orm import Mapped, mapped_column, relationship +from sqlalchemy import ( + Boolean, + Column, + Date, + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + Integer, + String, + Table, + UniqueConstraint, + func, +) +from sqlalchemy.orm import Mapped, foreign, mapped_column, relationship, remote from app.db.base_class import Base @@ -18,11 +31,43 @@ employee_role_links = Table( class Employee(Base): __tablename__ = "employees" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_employees_tenant_id"), + UniqueConstraint( + "tenant_id", + "employee_no", + name="uq_employees_tenant_employee_no", + ), + UniqueConstraint( + "tenant_id", + "email", + name="uq_employees_tenant_email", + ), + ForeignKeyConstraint( + ["tenant_id", "organization_unit_id"], + ["organization_units.tenant_id", "organization_units.id"], + name="fk_employees_tenant_organization_unit", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "manager_id"], + ["employees.tenant_id", "employees.id"], + name="fk_employees_tenant_manager", + ondelete="RESTRICT", + ), + Index("ix_employees_tenant_status", "tenant_id", "employment_status"), + Index("ix_employees_tenant_name", "tenant_id", "name"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - employee_no: Mapped[str] = mapped_column(String(50), unique=True, index=True) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + ) + employee_no: Mapped[str] = mapped_column(String(50), index=True) name: Mapped[str] = mapped_column(String(100), index=True) - email: Mapped[str] = mapped_column(String(255), unique=True, index=True) + email: Mapped[str] = mapped_column(String(255), index=True) gender: Mapped[str | None] = mapped_column(String(20), nullable=True) birth_date: Mapped[date | None] = mapped_column(Date(), nullable=True) phone: Mapped[str | None] = mapped_column(String(30), nullable=True) @@ -41,19 +86,51 @@ class Employee(Base): compliance_score: Mapped[int] = mapped_column(Integer, default=100) spotlight: Mapped[bool] = mapped_column(Boolean, default=False) last_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) - organization_unit_id: Mapped[str | None] = mapped_column( - ForeignKey("organization_units.id"), nullable=True, index=True - ) - manager_id: Mapped[str | None] = mapped_column(ForeignKey("employees.id"), nullable=True, index=True) + organization_unit_id: Mapped[str | None] = mapped_column(nullable=True, index=True) + manager_id: Mapped[str | None] = mapped_column(nullable=True, index=True) created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), onupdate=func.now() ) - organization_unit = relationship("OrganizationUnit", back_populates="employees") - manager = relationship("Employee", remote_side=[id], back_populates="reports") - reports = relationship("Employee", back_populates="manager") + organization_unit = relationship( + "OrganizationUnit", + back_populates="employees", + primaryjoin=( + "and_(Employee.tenant_id == OrganizationUnit.tenant_id, " + "foreign(Employee.organization_unit_id) == OrganizationUnit.id)" + ), + foreign_keys=[organization_unit_id], + overlaps="manager,reports", + ) + manager = relationship( + "Employee", + primaryjoin=lambda: ( + (Employee.tenant_id == remote(Employee.tenant_id)) + & (foreign(Employee.manager_id) == remote(Employee.id)) + ), + remote_side=[tenant_id, id], + foreign_keys=[manager_id], + back_populates="reports", + overlaps="organization_unit", + ) + reports = relationship( + "Employee", + primaryjoin=lambda: ( + (Employee.tenant_id == remote(Employee.tenant_id)) + & (Employee.id == foreign(remote(Employee.manager_id))) + ), + foreign_keys=[manager_id], + back_populates="manager", + overlaps="organization_unit", + ) roles = relationship("Role", secondary=employee_role_links, back_populates="employees") + tenant_memberships = relationship( + "TenantMembership", + back_populates="employee", + cascade="all, delete-orphan", + overlaps="memberships,tenant", + ) change_logs = relationship( "EmployeeChangeLog", back_populates="employee", diff --git a/server/src/app/models/employee_behavior_profile.py b/server/src/app/models/employee_behavior_profile.py index ca36a26..a4bf419 100644 --- a/server/src/app/models/employee_behavior_profile.py +++ b/server/src/app/models/employee_behavior_profile.py @@ -4,7 +4,16 @@ import uuid from datetime import datetime from typing import Any -from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, func +from sqlalchemy import ( + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + Integer, + String, + UniqueConstraint, + func, +) from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.types import JSON @@ -14,8 +23,20 @@ from app.db.base_class import Base class EmployeeBehaviorProfileSnapshot(Base): __tablename__ = "employee_behavior_profile_snapshots" __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_employee_behavior_profiles_tenant_id", + ), + ForeignKeyConstraint( + ["tenant_id", "subject_id"], + ["employees.tenant_id", "employees.id"], + name="fk_employee_behavior_profiles_tenant_employee", + ondelete="CASCADE", + ), Index( "ix_employee_behavior_profile_latest", + "tenant_id", "subject_id", "profile_type", "window_days", @@ -25,6 +46,12 @@ class EmployeeBehaviorProfileSnapshot(Base): ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + index=True, + ) subject_type: Mapped[str] = mapped_column(String(30), default="employee", index=True) subject_id: Mapped[str] = mapped_column(String(100), index=True) subject_name: Mapped[str] = mapped_column(String(100), index=True) diff --git a/server/src/app/models/financial_connector.py b/server/src/app/models/financial_connector.py new file mode 100644 index 0000000..e724632 --- /dev/null +++ b/server/src/app/models/financial_connector.py @@ -0,0 +1,476 @@ +from __future__ import annotations + +import uuid +from datetime import datetime +from decimal import Decimal +from typing import Any + +from sqlalchemy import ( + CheckConstraint, + DateTime, + ForeignKeyConstraint, + Index, + Integer, + Numeric, + String, + Text, + UniqueConstraint, + func, +) +from sqlalchemy.orm import Mapped, mapped_column +from sqlalchemy.types import JSON + +from app.db.base_class import Base + + +def _new_id() -> str: + return str(uuid.uuid4()) + + +class FinancialConnectorConfig(Base): + """租户绑定的连接器契约；只保存服务端密钥引用，不保存密钥。""" + + __tablename__ = "financial_connector_configs" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_financial_connector_configs_tenant_id"), + UniqueConstraint( + "tenant_id", + "provider", + "key_version", + name="uq_financial_connector_configs_tenant_provider_key", + ), + CheckConstraint( + "environment IN ('test', 'mock', 'staging', 'production')", + name="ck_financial_connector_configs_environment", + ), + CheckConstraint( + "status IN ('active', 'disabled', 'rotating')", + name="ck_financial_connector_configs_status", + ), + CheckConstraint( + "clock_skew_seconds BETWEEN 30 AND 900", + name="ck_financial_connector_configs_clock_skew", + ), + CheckConstraint( + "version >= 1", + name="ck_financial_connector_configs_version", + ), + CheckConstraint( + "length(trim(provider)) > 0 AND length(trim(key_version)) > 0 " + "AND length(trim(secret_ref)) > 0", + name="ck_financial_connector_configs_keys", + ), + Index( + "ix_financial_connector_configs_tenant_status", + "tenant_id", + "status", + "provider", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + provider: Mapped[str] = mapped_column(String(80), nullable=False) + environment: Mapped[str] = mapped_column(String(16), nullable=False) + key_version: Mapped[str] = mapped_column(String(40), nullable=False) + secret_ref: Mapped[str] = mapped_column(String(180), nullable=False) + allowed_event_types_json: Mapped[list[str]] = mapped_column(JSON, nullable=False, default=list) + clock_skew_seconds: Mapped[int] = mapped_column(Integer, nullable=False, default=300) + status: Mapped[str] = mapped_column(String(16), nullable=False, default="disabled") + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1) + last_success_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + last_error_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + last_error_code: Mapped[str | None] = mapped_column(String(80)) + created_by: Mapped[str] = mapped_column(String(120), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now() + ) + + +class FinancialConnectorConfigEvent(Base): + """连接器配置生命周期审计事实；不得保存密钥引用或密钥明文。""" + + __tablename__ = "financial_connector_config_events" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_financial_connector_config_events_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "request_id", + "action", + name="uq_financial_connector_config_events_tenant_request_action", + ), + ForeignKeyConstraint( + ["tenant_id", "config_id"], + ["financial_connector_configs.tenant_id", "financial_connector_configs.id"], + name="fk_financial_connector_config_events_tenant_config", + ondelete="RESTRICT", + ), + CheckConstraint( + "action IN ('created', 'activated', 'disabled', " + "'rotation_started', 'rotation_replacement_created')", + name="ck_financial_connector_config_events_action", + ), + CheckConstraint( + "expected_version IS NULL OR expected_version >= 1", + name="ck_financial_connector_config_events_expected_version", + ), + CheckConstraint( + "length(trim(actor_id)) > 0 AND length(trim(request_id)) > 0 " + "AND length(trim(reason)) > 0", + name="ck_financial_connector_config_events_required_text", + ), + Index( + "ix_financial_connector_config_events_tenant_config_time", + "tenant_id", + "config_id", + "occurred_at", + ), + Index( + "ix_financial_connector_config_events_tenant_request", + "tenant_id", + "request_id", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + config_id: Mapped[str] = mapped_column(String(36), nullable=False) + action: Mapped[str] = mapped_column(String(40), nullable=False) + actor_id: Mapped[str] = mapped_column(String(120), nullable=False) + request_id: Mapped[str] = mapped_column(String(120), nullable=False) + reason: Mapped[str] = mapped_column(Text(), nullable=False) + expected_version: Mapped[int | None] = mapped_column(Integer) + before_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + after_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + occurred_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + +class FinancialConnectorOperationalEvent(Base): + """连接器重放、认证失败和载荷冲突的最小化追加式运营事实。""" + + __tablename__ = "financial_connector_operational_events" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_financial_connector_operational_events_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_financial_connector_operational_events_tenant_request", + ), + ForeignKeyConstraint( + ["tenant_id", "config_id"], + ["financial_connector_configs.tenant_id", "financial_connector_configs.id"], + name="fk_financial_connector_operational_events_tenant_config", + ondelete="RESTRICT", + ), + CheckConstraint( + "event_type IN ('replay', 'auth_failure', 'payload_conflict')", + name="ck_financial_connector_operational_events_type", + ), + CheckConstraint( + "environment IN ('test', 'mock', 'staging', 'production')", + name="ck_financial_connector_operational_events_environment", + ), + CheckConstraint( + "length(trim(provider)) > 0 AND length(trim(reason_code)) > 0", + name="ck_financial_connector_operational_events_required_text", + ), + CheckConstraint( + "length(request_fingerprint) = 76 " + "AND request_fingerprint LIKE 'hmac-sha256:%' " + "AND length(external_event_fingerprint) = 76 " + "AND external_event_fingerprint LIKE 'hmac-sha256:%' " + "AND length(idempotency_key) = 71 " + "AND idempotency_key LIKE 'sha256:%'", + name="ck_financial_connector_operational_events_fingerprints", + ), + Index( + "ix_financial_connector_operational_events_tenant_config_time", + "tenant_id", + "config_id", + "occurred_at", + ), + Index( + "ix_financial_connector_operational_events_tenant_type_time", + "tenant_id", + "event_type", + "occurred_at", + ), + Index( + "ix_financial_connector_operational_events_tenant_provider_time", + "tenant_id", + "provider", + "occurred_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + config_id: Mapped[str] = mapped_column(String(36), nullable=False) + provider: Mapped[str] = mapped_column(String(80), nullable=False) + environment: Mapped[str] = mapped_column(String(16), nullable=False) + event_type: Mapped[str] = mapped_column(String(32), nullable=False) + reason_code: Mapped[str] = mapped_column(String(80), nullable=False) + request_fingerprint: Mapped[str] = mapped_column(String(76), nullable=False) + external_event_fingerprint: Mapped[str] = mapped_column(String(76), nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(71), nullable=False) + occurred_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + +class FinancialConnectorEvent(Base): + """经签名验证的最小化外部事实。表由 PostgreSQL 触发器强制只追加。""" + + __tablename__ = "financial_connector_events" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_financial_connector_events_tenant_id"), + UniqueConstraint( + "tenant_id", + "provider", + "external_event_id", + name="uq_financial_connector_events_external_id", + ), + ForeignKeyConstraint( + ["tenant_id", "config_id"], + ["financial_connector_configs.tenant_id", "financial_connector_configs.id"], + name="fk_financial_connector_events_tenant_config", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "origin_event_id"], + ["financial_connector_events.tenant_id", "financial_connector_events.id"], + name="fk_financial_connector_events_tenant_origin", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + name="fk_financial_connector_events_tenant_expense_case", + ondelete="RESTRICT", + ), + CheckConstraint( + "direction = 'inbound'", + name="ck_financial_connector_events_direction", + ), + CheckConstraint( + "event_type IN ('payment_settled', 'payment_failed', 'erp_posted', " + "'erp_posting_failed', 'payment_refunded', 'payment_reversed')", + name="ck_financial_connector_events_type", + ), + CheckConstraint( + "environment IN ('test', 'mock', 'staging', 'production')", + name="ck_financial_connector_events_environment", + ), + CheckConstraint( + "verification_level IN ('simulated', 'staging_verified', 'production_verified')", + name="ck_financial_connector_events_verification", + ), + CheckConstraint( + "processing_status IN ('processed', 'exception', 'pending')", + name="ck_financial_connector_events_processing_status", + ), + CheckConstraint( + "length(trim(external_event_id)) > 0 " + "AND length(trim(request_fingerprint)) >= 16 " + "AND length(trim(content_hash)) >= 16", + name="ck_financial_connector_events_fingerprints", + ), + CheckConstraint( + "(event_type IN ('payment_refunded', 'payment_reversed', " + "'erp_posted', 'erp_posting_failed') " + "AND (origin_event_id IS NOT NULL OR processing_status = 'exception')) " + "OR (event_type IN ('payment_settled', 'payment_failed') " + "AND origin_event_id IS NULL)", + name="ck_financial_connector_events_origin", + ), + Index( + "ix_financial_connector_events_tenant_received", + "tenant_id", + "received_at", + ), + Index( + "ix_financial_connector_events_tenant_claim", + "tenant_id", + "claim_id", + "occurred_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + config_id: Mapped[str] = mapped_column(String(36), nullable=False) + provider: Mapped[str] = mapped_column(String(80), nullable=False) + environment: Mapped[str] = mapped_column(String(16), nullable=False) + direction: Mapped[str] = mapped_column(String(12), nullable=False, default="inbound") + external_event_id: Mapped[str] = mapped_column(String(160), nullable=False) + event_type: Mapped[str] = mapped_column(String(40), nullable=False) + occurred_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + received_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + key_version: Mapped[str] = mapped_column(String(40), nullable=False) + verification_level: Mapped[str] = mapped_column(String(32), nullable=False) + request_fingerprint: Mapped[str] = mapped_column(String(80), nullable=False) + content_hash: Mapped[str] = mapped_column(String(80), nullable=False) + processing_status: Mapped[str] = mapped_column(String(20), nullable=False) + error_code: Mapped[str | None] = mapped_column(String(80)) + # expense_claims 由 legacy bootstrap 创建，迁移表只保存经过租户 Case 校验的软引用。 + claim_id: Mapped[str | None] = mapped_column(String(36)) + expense_case_id: Mapped[str | None] = mapped_column(String(36)) + origin_event_id: Mapped[str | None] = mapped_column(String(36)) + correlation_id: Mapped[str] = mapped_column(String(64), nullable=False) + external_reference_tail: Mapped[str | None] = mapped_column(String(8)) + normalized_payload_json: Mapped[dict[str, Any]] = mapped_column( + JSON, nullable=False, default=dict + ) + response_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + + +class PaymentReconciliationCase(Base): + """对账当前投影；所有历史变化由 PaymentReconciliationEvent 保存。""" + + __tablename__ = "payment_reconciliation_cases" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_payment_reconciliation_cases_tenant_id"), + UniqueConstraint( + "tenant_id", + "provider", + "claim_id", + name="uq_payment_reconciliation_cases_tenant_provider_claim", + ), + ForeignKeyConstraint( + ["tenant_id", "last_connector_event_id"], + ["financial_connector_events.tenant_id", "financial_connector_events.id"], + name="fk_payment_reconciliation_cases_tenant_last_event", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + name="fk_payment_reconciliation_cases_tenant_expense_case", + ondelete="RESTRICT", + ), + CheckConstraint( + "status IN ('pending', 'matched', 'exception', 'confirmed', " + "'rejected', 'reopened', 'closed')", + name="ck_payment_reconciliation_cases_status", + ), + CheckConstraint( + "erp_status IN ('pending_posting', 'posted', 'posting_failed')", + name="ck_payment_reconciliation_cases_erp_status", + ), + CheckConstraint( + "expected_amount >= 0 AND actual_amount >= 0", + name="ck_payment_reconciliation_cases_amounts", + ), + CheckConstraint( + "length(trim(expected_currency)) = 3 AND length(trim(actual_currency)) = 3", + name="ck_payment_reconciliation_cases_currencies", + ), + Index( + "ix_payment_reconciliation_cases_tenant_status", + "tenant_id", + "status", + "updated_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + provider: Mapped[str] = mapped_column(String(80), nullable=False) + # 租户边界由 expense_case_id 复合外键与服务查询共同保证。 + claim_id: Mapped[str] = mapped_column(String(36), nullable=False) + expense_case_id: Mapped[str | None] = mapped_column(String(36)) + expected_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + actual_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + amount_difference: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + expected_currency: Mapped[str] = mapped_column(String(3), nullable=False) + actual_currency: Mapped[str] = mapped_column(String(3), nullable=False) + expected_reference: Mapped[str] = mapped_column(String(160), nullable=False) + external_reference_tail: Mapped[str | None] = mapped_column(String(8)) + status: Mapped[str] = mapped_column(String(20), nullable=False) + exception_code: Mapped[str | None] = mapped_column(String(80)) + erp_status: Mapped[str] = mapped_column(String(20), nullable=False, default="pending_posting") + erp_document_tail: Mapped[str | None] = mapped_column(String(8)) + erp_document_hash: Mapped[str | None] = mapped_column(String(80)) + assigned_to: Mapped[str | None] = mapped_column(String(120)) + last_connector_event_id: Mapped[str] = mapped_column(String(36), nullable=False) + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now() + ) + + +class PaymentReconciliationEvent(Base): + """对账动作审计事实；数据库级禁止更新和删除。""" + + __tablename__ = "payment_reconciliation_events" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_payment_reconciliation_events_tenant_id"), + UniqueConstraint( + "tenant_id", + "connector_event_id", + "action", + name="uq_payment_reconciliation_events_connector_action", + ), + ForeignKeyConstraint( + ["tenant_id", "reconciliation_case_id"], + ["payment_reconciliation_cases.tenant_id", "payment_reconciliation_cases.id"], + name="fk_payment_reconciliation_events_tenant_case", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "connector_event_id"], + ["financial_connector_events.tenant_id", "financial_connector_events.id"], + name="fk_payment_reconciliation_events_tenant_connector_event", + ondelete="RESTRICT", + ), + CheckConstraint( + "action IN ('auto_matched', 'exception_created', 'erp_posted', " + "'erp_posting_failed', 'reopened', 'confirmed', 'rejected', 'closed')", + name="ck_payment_reconciliation_events_action", + ), + CheckConstraint( + "length(trim(request_fingerprint)) >= 16", + name="ck_payment_reconciliation_events_fingerprint", + ), + Index( + "ix_payment_reconciliation_events_tenant_case_time", + "tenant_id", + "reconciliation_case_id", + "occurred_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + reconciliation_case_id: Mapped[str] = mapped_column(String(36), nullable=False) + connector_event_id: Mapped[str] = mapped_column(String(36), nullable=False) + action: Mapped[str] = mapped_column(String(32), nullable=False) + actor_type: Mapped[str] = mapped_column(String(20), nullable=False) + actor_id: Mapped[str] = mapped_column(String(120), nullable=False) + request_fingerprint: Mapped[str] = mapped_column(String(80), nullable=False) + before_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + after_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + response_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + reason: Mapped[str | None] = mapped_column(Text()) + correlation_id: Mapped[str] = mapped_column(String(64), nullable=False) + occurred_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) diff --git a/server/src/app/models/financial_record.py b/server/src/app/models/financial_record.py index ebe53f7..1cb9e65 100644 --- a/server/src/app/models/financial_record.py +++ b/server/src/app/models/financial_record.py @@ -5,7 +5,20 @@ from datetime import date, datetime from decimal import Decimal from typing import Any -from sqlalchemy import Boolean, Date, DateTime, ForeignKey, Integer, Numeric, String, Text, func +from sqlalchemy import ( + Boolean, + Date, + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + Integer, + Numeric, + String, + Text, + UniqueConstraint, + func, +) from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.types import JSON @@ -14,16 +27,39 @@ from app.db.base_class import Base class ExpenseClaim(Base): __tablename__ = "expense_claims" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_expense_claims_tenant_id"), + UniqueConstraint( + "tenant_id", + "claim_no", + name="uq_expense_claims_tenant_claim_no", + ), + ForeignKeyConstraint( + ["tenant_id", "employee_id"], + ["employees.tenant_id", "employees.id"], + name="fk_expense_claims_tenant_employee", + ondelete="RESTRICT", + ), + ForeignKeyConstraint( + ["tenant_id", "department_id"], + ["organization_units.tenant_id", "organization_units.id"], + name="fk_expense_claims_tenant_department", + ondelete="RESTRICT", + ), + Index("ix_expense_claims_tenant_status", "tenant_id", "status"), + Index("ix_expense_claims_tenant_occurred", "tenant_id", "occurred_at"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - claim_no: Mapped[str] = mapped_column(String(50), unique=True, index=True) - employee_id: Mapped[str | None] = mapped_column( - ForeignKey("employees.id"), nullable=True, index=True + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", ) + claim_no: Mapped[str] = mapped_column(String(50), index=True) + employee_id: Mapped[str | None] = mapped_column(nullable=True, index=True) employee_name: Mapped[str] = mapped_column(String(100), index=True) - department_id: Mapped[str | None] = mapped_column( - ForeignKey("organization_units.id"), nullable=True, index=True - ) + department_id: Mapped[str | None] = mapped_column(nullable=True, index=True) department_name: Mapped[str] = mapped_column(String(100), index=True) project_code: Mapped[str | None] = mapped_column(String(50), nullable=True) expense_type: Mapped[str] = mapped_column(String(50), index=True) @@ -46,7 +82,10 @@ class ExpenseClaim(Base): DateTime(timezone=True), server_default=func.now(), onupdate=func.now() ) - employee = relationship("Employee", foreign_keys=[employee_id]) + employee = relationship( + "Employee", + foreign_keys=[tenant_id, employee_id], + ) items = relationship( "ExpenseClaimItem", back_populates="claim", @@ -119,9 +158,22 @@ class ExpenseClaimItem(Base): class AccountsReceivableRecord(Base): __tablename__ = "accounts_receivable" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "receivable_no", + name="uq_accounts_receivable_tenant_no", + ), + Index("ix_accounts_receivable_tenant_customer", "tenant_id", "customer_id"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - receivable_no: Mapped[str] = mapped_column(String(50), unique=True, index=True) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + ) + receivable_no: Mapped[str] = mapped_column(String(50), index=True) customer_id: Mapped[str] = mapped_column(String(64), index=True) customer_name: Mapped[str] = mapped_column(String(120), index=True) contract_no: Mapped[str | None] = mapped_column(String(100), nullable=True) @@ -143,9 +195,22 @@ class AccountsReceivableRecord(Base): class AccountsPayableRecord(Base): __tablename__ = "accounts_payable" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "payable_no", + name="uq_accounts_payable_tenant_no", + ), + Index("ix_accounts_payable_tenant_vendor", "tenant_id", "vendor_id"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - payable_no: Mapped[str] = mapped_column(String(50), unique=True, index=True) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + ) + payable_no: Mapped[str] = mapped_column(String(50), index=True) vendor_id: Mapped[str] = mapped_column(String(64), index=True) vendor_name: Mapped[str] = mapped_column(String(120), index=True) invoice_no: Mapped[str | None] = mapped_column(String(100), nullable=True) diff --git a/server/src/app/models/hermes_config.py b/server/src/app/models/hermes_config.py index 13cdc6b..7403782 100644 --- a/server/src/app/models/hermes_config.py +++ b/server/src/app/models/hermes_config.py @@ -4,7 +4,17 @@ import uuid from datetime import datetime from typing import Any -from sqlalchemy import Boolean, DateTime, ForeignKey, String, Text, func +from sqlalchemy import ( + Boolean, + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + String, + Text, + UniqueConstraint, + func, +) from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.types import JSON @@ -13,13 +23,22 @@ from app.db.base_class import Base class HermesTaskConfig(Base): __tablename__ = "hermes_task_configs" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_hermes_task_configs_tenant_id"), + Index("ix_hermes_task_configs_tenant_enabled", "tenant_id", "is_enabled"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + ) task_type: Mapped[str] = mapped_column(String(50), index=True) cron_expression: Mapped[str] = mapped_column(String(100)) is_enabled: Mapped[bool] = mapped_column(Boolean, default=True) payload_template: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict) - + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), onupdate=func.now() @@ -35,13 +54,32 @@ class HermesTaskConfig(Base): class HermesTaskExecutionLog(Base): __tablename__ = "hermes_task_execution_logs" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_hermes_task_execution_logs_tenant_id", + ), + ForeignKeyConstraint( + ["tenant_id", "config_id"], + ["hermes_task_configs.tenant_id", "hermes_task_configs.id"], + name="fk_hermes_task_logs_tenant_config", + ondelete="CASCADE", + ), + Index("ix_hermes_task_logs_tenant_started", "tenant_id", "started_at"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - config_id: Mapped[str] = mapped_column(String(36), ForeignKey("hermes_task_configs.id"), index=True) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + ) + config_id: Mapped[str] = mapped_column(String(36), index=True) status: Mapped[str] = mapped_column(String(30), index=True) result_summary: Mapped[str | None] = mapped_column(String(255), nullable=True) error_trace: Mapped[str | None] = mapped_column(Text(), nullable=True) - + started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) diff --git a/server/src/app/models/hermes_report.py b/server/src/app/models/hermes_report.py index 0f1b2af..5b391ee 100644 --- a/server/src/app/models/hermes_report.py +++ b/server/src/app/models/hermes_report.py @@ -2,9 +2,17 @@ from __future__ import annotations import uuid from datetime import datetime -from typing import Any -from sqlalchemy import DateTime, ForeignKey, String, Text, func +from sqlalchemy import ( + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + String, + Text, + UniqueConstraint, + func, +) from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.types import JSON @@ -13,22 +21,51 @@ from app.db.base_class import Base class HermesRiskReport(Base): __tablename__ = "hermes_risk_reports" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_hermes_risk_reports_tenant_id"), + ForeignKeyConstraint( + ["tenant_id", "claim_id"], + ["expense_claims.tenant_id", "expense_claims.id"], + name="fk_hermes_risk_reports_tenant_claim", + ondelete="CASCADE", + ), + ForeignKeyConstraint( + ["tenant_id", "execution_log_id"], + ["hermes_task_execution_logs.tenant_id", "hermes_task_execution_logs.id"], + name="fk_hermes_risk_reports_tenant_log", + ondelete="CASCADE", + ), + Index("ix_hermes_risk_reports_tenant_status", "tenant_id", "status"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - claim_id: Mapped[str] = mapped_column(ForeignKey("expense_claims.id"), index=True) - execution_log_id: Mapped[str] = mapped_column(ForeignKey("hermes_task_execution_logs.id"), index=True) - + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + ) + claim_id: Mapped[str] = mapped_column(index=True) + execution_log_id: Mapped[str] = mapped_column(index=True) + risk_level: Mapped[str] = mapped_column(String(20), index=True) risk_type: Mapped[str] = mapped_column(String(50), index=True) risk_description: Mapped[str] = mapped_column(Text()) - + related_claim_ids: Mapped[list[str]] = mapped_column(JSON, default=list) status: Mapped[str] = mapped_column(String(30), default="pending_review", index=True) - + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), onupdate=func.now() ) - claim = relationship("ExpenseClaim", foreign_keys=[claim_id]) - execution_log = relationship("HermesTaskExecutionLog", foreign_keys=[execution_log_id]) + claim = relationship( + "ExpenseClaim", + foreign_keys=[tenant_id, claim_id], + overlaps="execution_log", + ) + execution_log = relationship( + "HermesTaskExecutionLog", + foreign_keys=[tenant_id, execution_log_id], + overlaps="claim", + ) diff --git a/server/src/app/models/knowledge_security.py b/server/src/app/models/knowledge_security.py new file mode 100644 index 0000000..5ddaa73 --- /dev/null +++ b/server/src/app/models/knowledge_security.py @@ -0,0 +1,86 @@ +from __future__ import annotations + +import uuid +from datetime import datetime + +from sqlalchemy import ( + Boolean, + CheckConstraint, + DateTime, + ForeignKey, + Index, + Integer, + String, + Text, + func, +) +from sqlalchemy.orm import Mapped, mapped_column + +from app.db.base_class import Base + + +class KnowledgeOnlyOfficeSession(Base): + __tablename__ = "knowledge_onlyoffice_sessions" + __table_args__ = ( + CheckConstraint( + "resource_scope IN ('tenant', 'platform')", + name="ck_knowledge_onlyoffice_sessions_scope", + ), + CheckConstraint( + "status IN ('active', 'processing', 'consumed', 'failed', 'revoked')", + name="ck_knowledge_onlyoffice_sessions_status", + ), + CheckConstraint( + "tenant_id IS NOT NULL AND " + "(resource_scope = 'tenant' OR " + "(resource_scope = 'platform' AND editable = false))", + name="ck_knowledge_onlyoffice_sessions_scope_tenant", + ), + CheckConstraint( + "(status = 'active' AND claimed_at IS NULL AND consumed_at IS NULL) OR " + "(status IN ('processing', 'failed') AND claimed_at IS NOT NULL " + "AND consumed_at IS NULL) OR " + "(status = 'consumed' AND claimed_at IS NOT NULL AND consumed_at IS NOT NULL) OR " + "(status = 'revoked' AND consumed_at IS NULL)", + name="ck_knowledge_onlyoffice_sessions_lifecycle", + ), + Index( + "ix_knowledge_onlyoffice_sessions_tenant_document", + "tenant_id", + "document_id", + "created_at", + ), + Index( + "ix_knowledge_onlyoffice_sessions_status_expiry", + "status", + "expires_at", + ), + ) + + jti: Mapped[str] = mapped_column( + String(36), + primary_key=True, + default=lambda: str(uuid.uuid4()), + ) + tenant_id: Mapped[str] = mapped_column( + String(64), + ForeignKey("tenants.tenant_id", ondelete="CASCADE"), + nullable=False, + ) + resource_scope: Mapped[str] = mapped_column(String(16), nullable=False) + document_id: Mapped[str] = mapped_column(String(64), nullable=False) + document_key: Mapped[str] = mapped_column(String(160), nullable=False) + document_version: Mapped[int] = mapped_column(Integer, nullable=False) + audience: Mapped[str] = mapped_column(String(80), nullable=False) + editable: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False) + status: Mapped[str] = mapped_column(String(16), nullable=False, default="active") + created_by: Mapped[str] = mapped_column(String(100), nullable=False) + expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + claimed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + consumed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + failure_reason: Mapped[str] = mapped_column(Text, nullable=False, default="") + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + ) diff --git a/server/src/app/models/organization.py b/server/src/app/models/organization.py index e99fabc..77fa76f 100644 --- a/server/src/app/models/organization.py +++ b/server/src/app/models/organization.py @@ -3,7 +3,15 @@ from __future__ import annotations import uuid from datetime import datetime -from sqlalchemy import DateTime, ForeignKey, String, func +from sqlalchemy import ( + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + String, + UniqueConstraint, + func, +) from sqlalchemy.orm import Mapped, mapped_column, relationship from app.db.base_class import Base @@ -11,14 +19,36 @@ from app.db.base_class import Base class OrganizationUnit(Base): __tablename__ = "organization_units" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "id", + name="uq_organization_units_tenant_id", + ), + UniqueConstraint( + "tenant_id", + "unit_code", + name="uq_organization_units_tenant_code", + ), + ForeignKeyConstraint( + ["tenant_id", "parent_id"], + ["organization_units.tenant_id", "organization_units.id"], + name="fk_organization_units_tenant_parent", + ondelete="RESTRICT", + ), + Index("ix_organization_units_tenant_name", "tenant_id", "name"), + ) id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) - unit_code: Mapped[str] = mapped_column(String(50), unique=True, index=True) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + server_default="default", + ) + unit_code: Mapped[str] = mapped_column(String(50), index=True) name: Mapped[str] = mapped_column(String(100), index=True) unit_type: Mapped[str] = mapped_column(String(30), default="department", index=True) - parent_id: Mapped[str | None] = mapped_column( - ForeignKey("organization_units.id"), nullable=True, index=True - ) + parent_id: Mapped[str | None] = mapped_column(nullable=True, index=True) cost_center: Mapped[str | None] = mapped_column(String(50), nullable=True) location: Mapped[str | None] = mapped_column(String(100), nullable=True) manager_name: Mapped[str | None] = mapped_column(String(100), nullable=True) @@ -27,6 +57,24 @@ class OrganizationUnit(Base): DateTime(timezone=True), server_default=func.now(), onupdate=func.now() ) - parent = relationship("OrganizationUnit", remote_side=[id], back_populates="children") - children = relationship("OrganizationUnit", back_populates="parent") - employees = relationship("Employee", back_populates="organization_unit") + parent = relationship( + "OrganizationUnit", + remote_side=[tenant_id, id], + foreign_keys=[tenant_id, parent_id], + back_populates="children", + ) + children = relationship( + "OrganizationUnit", + foreign_keys=[tenant_id, parent_id], + back_populates="parent", + ) + employees = relationship( + "Employee", + primaryjoin=( + "and_(OrganizationUnit.tenant_id == Employee.tenant_id, " + "OrganizationUnit.id == foreign(Employee.organization_unit_id))" + ), + foreign_keys="[Employee.organization_unit_id]", + back_populates="organization_unit", + overlaps="manager,reports", + ) diff --git a/server/src/app/models/savings.py b/server/src/app/models/savings.py new file mode 100644 index 0000000..22659fd --- /dev/null +++ b/server/src/app/models/savings.py @@ -0,0 +1,800 @@ +from __future__ import annotations + +import uuid +from datetime import date, datetime +from decimal import Decimal +from typing import Any + +from sqlalchemy import ( + CheckConstraint, + Date, + DateTime, + ForeignKeyConstraint, + Index, + Integer, + Numeric, + String, + Text, + UniqueConstraint, + func, + text, +) +from sqlalchemy.orm import Mapped, mapped_column, relationship +from sqlalchemy.types import JSON + +from app.db.base_class import Base + + +def _new_id() -> str: + return str(uuid.uuid4()) + + +class ProfileBaselineSnapshot(Base): + """冻结的费用基线事实；机会始终引用快照而不是实时重算结果。""" + __tablename__ = "profile_baseline_snapshots" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_profile_baseline_snapshots_tenant_id"), + UniqueConstraint( + "tenant_id", "baseline_key", name="uq_profile_baseline_snapshots_tenant_key" + ), + CheckConstraint( + "baseline_type IN ('historical_cohort', 'policy_counterfactual', 'manual')", + name="ck_profile_baseline_snapshots_type", + ), + CheckConstraint( + "data_quality_status IN ('complete', 'partial', 'insufficient', 'invalid')", + name="ck_profile_baseline_snapshots_quality_status", + ), + CheckConstraint( + "baseline_value >= 0 AND sample_count >= 0", + name="ck_profile_baseline_snapshots_values", + ), + CheckConstraint( + "data_quality_score >= 0 AND data_quality_score <= 1", + name="ck_profile_baseline_snapshots_quality_score", + ), + CheckConstraint( + "window_end IS NULL OR window_start IS NOT NULL", + name="ck_profile_baseline_snapshots_window_pair", + ), + CheckConstraint( + "window_start IS NULL OR window_end IS NULL OR window_end >= window_start", + name="ck_profile_baseline_snapshots_window_order", + ), + CheckConstraint( + "baseline_type != 'historical_cohort' OR " + "(window_start IS NOT NULL AND window_end IS NOT NULL AND sample_count > 0)", + name="ck_profile_baseline_snapshots_historical_shape", + ), + CheckConstraint( + "baseline_type != 'policy_counterfactual' OR " + "(policy_version IS NOT NULL AND length(trim(policy_version)) > 0 " + "AND policy_effective_from IS NOT NULL AND target_resource_type IS NOT NULL " + "AND target_resource_id IS NOT NULL)", + name="ck_profile_baseline_snapshots_policy_shape", + ), + CheckConstraint( + "policy_effective_to IS NULL OR policy_effective_from IS NOT NULL", + name="ck_profile_baseline_snapshots_policy_pair", + ), + CheckConstraint( + "policy_effective_from IS NULL OR policy_effective_to IS NULL " + "OR policy_effective_to >= policy_effective_from", + name="ck_profile_baseline_snapshots_policy_order", + ), + CheckConstraint( + "valid_until IS NULL OR valid_until >= frozen_at", + name="ck_profile_baseline_snapshots_validity", + ), + CheckConstraint( + "length(trim(baseline_key)) > 0 AND length(trim(query_fingerprint)) > 0", + name="ck_profile_baseline_snapshots_keys", + ), + CheckConstraint("version >= 1", name="ck_profile_baseline_snapshots_version"), + Index( + "ix_profile_baseline_snapshots_lookup", "tenant_id", "baseline_type", + "dimension_type", "dimension_id", "metric_key", "frozen_at", + ), + Index( + "ix_profile_baseline_snapshots_quality", "tenant_id", + "data_quality_status", "frozen_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + baseline_key: Mapped[str] = mapped_column(String(160), nullable=False) + baseline_type: Mapped[str] = mapped_column(String(32), nullable=False) + dimension_type: Mapped[str] = mapped_column(String(50), nullable=False) + dimension_id: Mapped[str] = mapped_column(String(160), nullable=False) + metric_key: Mapped[str] = mapped_column(String(100), nullable=False) + unit: Mapped[str] = mapped_column(String(30), nullable=False) + original_currency: Mapped[str | None] = mapped_column(String(3), nullable=True) + baseline_value: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + window_start: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + window_end: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + sample_count: Mapped[int] = mapped_column( + Integer, nullable=False, default=0, server_default="0" + ) + method: Mapped[str] = mapped_column(String(80), nullable=False) + query_fingerprint: Mapped[str] = mapped_column(String(80), nullable=False) + data_quality_status: Mapped[str] = mapped_column(String(20), nullable=False) + data_quality_score: Mapped[Decimal] = mapped_column(Numeric(5, 4), nullable=False) + quality_issues_json: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list) + algorithm_version: Mapped[str] = mapped_column(String(80), nullable=False) + policy_version: Mapped[str | None] = mapped_column(String(120), nullable=True) + policy_effective_from: Mapped[date | None] = mapped_column(Date(), nullable=True) + policy_effective_to: Mapped[date | None] = mapped_column(Date(), nullable=True) + target_resource_type: Mapped[str | None] = mapped_column(String(50), nullable=True) + target_resource_id: Mapped[str | None] = mapped_column(String(160), nullable=True) + frozen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + frozen_by: Mapped[str] = mapped_column(String(120), nullable=False) + valid_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1") + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + + opportunities = relationship( + "SavingsOpportunity", back_populates="baseline_snapshot", passive_deletes=True + ) + evidence_links = relationship( + "SavingsEvidenceLink", + foreign_keys="SavingsEvidenceLink.baseline_snapshot_id", + back_populates="baseline_snapshot", + passive_deletes=True, + ) + events = relationship( + "SavingsEvent", + foreign_keys="SavingsEvent.baseline_snapshot_id", + back_populates="baseline_snapshot", + passive_deletes=True, + ) + + +class SavingsOpportunity(Base): + """从风险暴露中分离出的可执行价值机会投影。""" + __tablename__ = "savings_opportunities" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_savings_opportunities_tenant_id"), + UniqueConstraint( + "tenant_id", "opportunity_key", name="uq_savings_opportunities_tenant_key" + ), + ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + ondelete="RESTRICT", + name="fk_savings_opportunities_tenant_case", + ), + ForeignKeyConstraint( + ["tenant_id", "expense_case_id", "discovery_business_event_id"], + [ + "business_events.tenant_id", + "business_events.expense_case_id", + "business_events.id", + ], + ondelete="RESTRICT", + name="fk_savings_opportunities_tenant_event", + ), + ForeignKeyConstraint( + ["tenant_id", "baseline_snapshot_id"], + ["profile_baseline_snapshots.tenant_id", "profile_baseline_snapshots.id"], + ondelete="RESTRICT", + name="fk_savings_opportunities_tenant_baseline", + ), + ForeignKeyConstraint( + ["tenant_id", "expense_case_id", "ai_decision_id"], + ["ai_decisions.tenant_id", "ai_decisions.expense_case_id", "ai_decisions.id"], + ondelete="RESTRICT", + name="fk_savings_opportunities_tenant_ai_decision", + ), + CheckConstraint( + "value_kind IN ('cash', 'labor')", + name="ck_savings_opportunities_value_kind", + ), + CheckConstraint( + "status IN ('identified', 'accepted', 'in_progress', 'realized', " + "'verified', 'reversed', 'rejected', 'expired')", + name="ck_savings_opportunities_status", + ), + CheckConstraint( + "exposure_amount >= 0 AND baseline_amount >= 0 AND target_amount >= 0 " + "AND estimated_gross >= 0 AND estimated_cost >= 0 AND estimated_net >= 0 " + "AND estimated_low >= 0 AND estimated_high >= 0", + name="ck_savings_opportunities_amounts", + ), + CheckConstraint( + "estimated_net = estimated_gross - estimated_cost", + name="ck_savings_opportunities_net_math", + ), + CheckConstraint( + "estimated_low <= estimated_net AND estimated_net <= estimated_high", + name="ck_savings_opportunities_interval", + ), + CheckConstraint( + "confidence >= 0 AND confidence <= 1", + name="ck_savings_opportunities_confidence", + ), + CheckConstraint("version >= 1", name="ck_savings_opportunities_version"), + CheckConstraint( + "length(trim(benefit_key)) > 0 AND length(trim(opportunity_key)) > 0", + name="ck_savings_opportunities_keys", + ), + CheckConstraint( + "length(trim(currency)) = 3 AND length(trim(reporting_currency)) = 3", + name="ck_savings_opportunities_currencies", + ), + CheckConstraint( + "status NOT IN ('accepted', 'in_progress', 'realized', 'verified', 'reversed') " + "OR accepted_at IS NOT NULL", + name="ck_savings_opportunities_acceptance", + ), + CheckConstraint( + "status NOT IN ('in_progress', 'realized', 'verified', 'reversed') " + "OR started_at IS NOT NULL", + name="ck_savings_opportunities_started", + ), + CheckConstraint( + "status NOT IN ('realized', 'verified', 'reversed') OR realized_at IS NOT NULL", + name="ck_savings_opportunities_realized", + ), + CheckConstraint( + "status NOT IN ('verified', 'reversed') OR verified_at IS NOT NULL", + name="ck_savings_opportunities_verified", + ), + CheckConstraint( + "status NOT IN ('verified', 'reversed', 'rejected', 'expired') " + "OR closed_at IS NOT NULL", + name="ck_savings_opportunities_closed", + ), + Index( + "ix_savings_opportunities_tenant_status_due", + "tenant_id", + "status", + "due_at", + ), + Index( + "ix_savings_opportunities_tenant_case", + "tenant_id", + "expense_case_id", + "created_at", + ), + Index( + "ix_savings_opportunities_tenant_benefit", + "tenant_id", + "benefit_key", + ), + Index( + "ix_savings_opportunities_tenant_owner", + "tenant_id", + "owner_id", + "status", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + opportunity_key: Mapped[str] = mapped_column(String(180), nullable=False) + benefit_key: Mapped[str] = mapped_column(String(180), nullable=False) + expense_case_id: Mapped[str] = mapped_column(String(36), nullable=False) + claim_id: Mapped[str] = mapped_column(String(36), nullable=False) + claim_no_snapshot: Mapped[str] = mapped_column(String(80), nullable=False) + claim_item_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + discovery_business_event_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + source_type: Mapped[str] = mapped_column(String(50), nullable=False) + source_id: Mapped[str] = mapped_column(String(160), nullable=False) + category: Mapped[str] = mapped_column(String(60), nullable=False) + value_kind: Mapped[str] = mapped_column(String(20), nullable=False) + title: Mapped[str] = mapped_column(String(200), nullable=False) + description: Mapped[str] = mapped_column(Text(), nullable=False) + exposure_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + baseline_snapshot_id: Mapped[str] = mapped_column(String(36), nullable=False) + baseline_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + target_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + estimated_gross: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + estimated_cost: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + estimated_net: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + estimated_low: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + estimated_high: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + confidence: Mapped[Decimal] = mapped_column(Numeric(5, 4), nullable=False) + currency: Mapped[str] = mapped_column(String(3), nullable=False) + reporting_currency: Mapped[str] = mapped_column(String(3), nullable=False) + attribution_method: Mapped[str] = mapped_column(String(60), nullable=False) + ai_decision_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + suggested_action: Mapped[str] = mapped_column(Text(), nullable=False) + owner_id: Mapped[str] = mapped_column(String(120), nullable=False) + owner_name: Mapped[str] = mapped_column(String(120), nullable=False) + owner_role: Mapped[str] = mapped_column(String(60), nullable=False) + due_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + status: Mapped[str] = mapped_column( + String(24), nullable=False, default="identified", server_default="identified" + ) + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1") + dimension_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + baseline_snapshot_json: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict) + evidence_json: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list) + accepted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + realized_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + closed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now() + ) + + baseline_snapshot = relationship("ProfileBaselineSnapshot", back_populates="opportunities") + realizations = relationship( + "SavingsRealization", + back_populates="opportunity", + order_by="asc(SavingsRealization.realized_at)", + passive_deletes=True, + ) + evidence_links = relationship( + "SavingsEvidenceLink", + foreign_keys="SavingsEvidenceLink.opportunity_id", + back_populates="opportunity", + passive_deletes=True, + ) + events = relationship( + "SavingsEvent", + foreign_keys="SavingsEvent.opportunity_id", + back_populates="opportunity", + passive_deletes=True, + ) + + +class SavingsRealization(Base): + """实际金额事实及其财务确认投影；冲回以新行表达。""" + __tablename__ = "savings_realizations" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_savings_realizations_tenant_id"), + UniqueConstraint( + "tenant_id", "realization_key", name="uq_savings_realizations_tenant_key" + ), + UniqueConstraint( + "tenant_id", "opportunity_id", "benefit_key", "id", + name="uq_savings_realizations_tenant_opportunity_benefit_id", + ), + UniqueConstraint( + "tenant_id", "benefit_key", "id", + name="uq_savings_realizations_tenant_benefit_id", + ), + ForeignKeyConstraint( + ["tenant_id", "opportunity_id"], + ["savings_opportunities.tenant_id", "savings_opportunities.id"], + ondelete="RESTRICT", + name="fk_savings_realizations_tenant_opportunity", + ), + ForeignKeyConstraint( + ["tenant_id", "expense_case_id"], + ["expense_cases.tenant_id", "expense_cases.id"], + ondelete="RESTRICT", + name="fk_savings_realizations_tenant_case", + ), + ForeignKeyConstraint( + ["tenant_id", "expense_case_id", "business_event_id"], + [ + "business_events.tenant_id", + "business_events.expense_case_id", + "business_events.id", + ], + ondelete="RESTRICT", + name="fk_savings_realizations_tenant_event", + ), + ForeignKeyConstraint( + [ + "tenant_id", "opportunity_id", "benefit_key", "reversal_of_realization_id" + ], + [ + "savings_realizations.tenant_id", "savings_realizations.opportunity_id", + "savings_realizations.benefit_key", "savings_realizations.id", + ], + ondelete="RESTRICT", + name="fk_savings_realizations_tenant_reversal", + ), + ForeignKeyConstraint( + ["tenant_id", "benefit_key", "canonical_realization_id"], + [ + "savings_realizations.tenant_id", "savings_realizations.benefit_key", + "savings_realizations.id", + ], + ondelete="RESTRICT", + name="fk_savings_realizations_tenant_canonical", + ), + CheckConstraint( + "realization_type IN ('actual', 'reversal')", + name="ck_savings_realizations_type", + ), + CheckConstraint( + "dedupe_status IN ('pending_review', 'canonical', 'duplicate', 'excluded')", + name="ck_savings_realizations_dedupe_status", + ), + CheckConstraint( + "status IN ('pending_confirmation', 'finance_confirmed', 'rejected', 'reversed')", + name="ck_savings_realizations_status", + ), + CheckConstraint( + "attribution_ratio > 0 AND attribution_ratio <= 1", + name="ck_savings_realizations_attribution", + ), + CheckConstraint( + "incremental_cost >= 0 AND fx_rate > 0", + name="ck_savings_realizations_cost_fx", + ), + CheckConstraint( + "actual_net = actual_gross - incremental_cost", + name="ck_savings_realizations_net_math", + ), + CheckConstraint( + "(realization_type = 'actual' AND reversal_of_realization_id IS NULL " + "AND actual_gross >= 0 AND actual_net >= 0 AND reporting_amount >= 0) OR " + "(realization_type = 'reversal' AND reversal_of_realization_id IS NOT NULL " + "AND actual_gross <= 0 AND actual_net <= 0 AND reporting_amount <= 0)", + name="ck_savings_realizations_amount_direction", + ), + CheckConstraint( + "(dedupe_status = 'duplicate' AND canonical_realization_id IS NOT NULL " + "AND canonical_realization_id <> id) OR " + "(dedupe_status != 'duplicate' AND canonical_realization_id IS NULL)", + name="ck_savings_realizations_duplicate_target", + ), + CheckConstraint( + "status != 'finance_confirmed' OR " + "(finance_confirmer_id IS NOT NULL AND finance_confirmer_name IS NOT NULL " + "AND confirmed_at IS NOT NULL AND confirmation_note IS NOT NULL " + "AND (realization_type = 'reversal' OR finance_confirmer_id <> recorded_by_id) " + "AND dedupe_status = 'canonical')", + name="ck_savings_realizations_confirmation", + ), + CheckConstraint( + "status != 'rejected' OR (rejected_by_id IS NOT NULL " + "AND rejected_by_name IS NOT NULL AND rejected_at IS NOT NULL " + "AND rejection_reason IS NOT NULL)", + name="ck_savings_realizations_rejection", + ), + CheckConstraint( + "status != 'reversed' OR (reversed_by_id IS NOT NULL " + "AND reversed_by_name IS NOT NULL AND reversed_at IS NOT NULL " + "AND reversal_reason IS NOT NULL)", + name="ck_savings_realizations_reversal", + ), + CheckConstraint("version >= 1", name="ck_savings_realizations_version"), + CheckConstraint( + "length(trim(realization_key)) > 0 AND length(trim(benefit_key)) > 0", + name="ck_savings_realizations_keys", + ), + CheckConstraint( + "length(trim(original_currency)) = 3 " + "AND length(trim(reporting_currency)) = 3", + name="ck_savings_realizations_currencies", + ), + Index( + "uq_savings_realizations_actual_canonical_benefit", + "tenant_id", + "benefit_key", + unique=True, + postgresql_where=text( + "realization_type = 'actual' AND dedupe_status = 'canonical'" + ), + ).ddl_if(dialect="postgresql"), + Index( + "ix_savings_realizations_tenant_status_time", "tenant_id", "status", "realized_at" + ), + Index( + "ix_savings_realizations_tenant_opportunity", "tenant_id", + "opportunity_id", "realized_at", + ), + Index( + "ix_savings_realizations_tenant_benefit", "tenant_id", + "benefit_key", "dedupe_status", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + realization_key: Mapped[str] = mapped_column(String(180), nullable=False) + opportunity_id: Mapped[str] = mapped_column(String(36), nullable=False) + expense_case_id: Mapped[str] = mapped_column(String(36), nullable=False) + claim_id: Mapped[str] = mapped_column(String(36), nullable=False) + claim_item_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + business_event_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + realization_type: Mapped[str] = mapped_column(String(20), nullable=False) + reversal_of_realization_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + realized_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + recorded_by_id: Mapped[str] = mapped_column(String(120), nullable=False) + recorded_by_name: Mapped[str] = mapped_column(String(120), nullable=False) + actual_gross: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + incremental_cost: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + actual_net: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + original_currency: Mapped[str] = mapped_column(String(3), nullable=False) + reporting_amount: Mapped[Decimal] = mapped_column(Numeric(20, 4), nullable=False) + reporting_currency: Mapped[str] = mapped_column(String(3), nullable=False) + fx_rate: Mapped[Decimal] = mapped_column(Numeric(20, 8), nullable=False) + fx_source: Mapped[str] = mapped_column(String(80), nullable=False) + fx_date: Mapped[date] = mapped_column(Date(), nullable=False) + fx_version: Mapped[str] = mapped_column(String(80), nullable=False) + attribution_method: Mapped[str] = mapped_column(String(60), nullable=False) + attribution_ratio: Mapped[Decimal] = mapped_column(Numeric(7, 6), nullable=False) + benefit_key: Mapped[str] = mapped_column(String(180), nullable=False) + dedupe_status: Mapped[str] = mapped_column( + String(24), nullable=False, default="pending_review", server_default="pending_review" + ) + canonical_realization_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + status: Mapped[str] = mapped_column( + String(24), + nullable=False, + default="pending_confirmation", + server_default="pending_confirmation", + ) + finance_confirmer_id: Mapped[str | None] = mapped_column(String(120), nullable=True) + finance_confirmer_name: Mapped[str | None] = mapped_column(String(120), nullable=True) + confirmed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + confirmation_note: Mapped[str | None] = mapped_column(Text(), nullable=True) + rejected_by_id: Mapped[str | None] = mapped_column(String(120), nullable=True) + rejected_by_name: Mapped[str | None] = mapped_column(String(120), nullable=True) + rejected_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + rejection_reason: Mapped[str | None] = mapped_column(Text(), nullable=True) + reversed_by_id: Mapped[str | None] = mapped_column(String(120), nullable=True) + reversed_by_name: Mapped[str | None] = mapped_column(String(120), nullable=True) + reversed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + reversal_reason: Mapped[str | None] = mapped_column(Text(), nullable=True) + baseline_snapshot_json: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict) + final_snapshot_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + evidence_json: Mapped[list[dict[str, Any]]] = mapped_column(JSON, default=list) + version: Mapped[int] = mapped_column(Integer, nullable=False, default=1, server_default="1") + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now() + ) + + opportunity = relationship("SavingsOpportunity", back_populates="realizations") + evidence_links = relationship( + "SavingsEvidenceLink", + foreign_keys="SavingsEvidenceLink.realization_id", + back_populates="realization", + passive_deletes=True, + ) + events = relationship( + "SavingsEvent", + foreign_keys="SavingsEvent.realization_id", + back_populates="realization", + passive_deletes=True, + ) + + +class SavingsEvidenceLink(Base): + """基线、机会或实际结果与服务端可验证资源之间的租户安全链接。""" + __tablename__ = "savings_evidence_links" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_savings_evidence_links_tenant_id"), + UniqueConstraint( + "tenant_id", + "evidence_key", + name="uq_savings_evidence_links_tenant_key", + ), + ForeignKeyConstraint( + ["tenant_id", "baseline_snapshot_id"], + ["profile_baseline_snapshots.tenant_id", "profile_baseline_snapshots.id"], + ondelete="RESTRICT", + name="fk_savings_evidence_links_tenant_baseline", + ), + ForeignKeyConstraint( + ["tenant_id", "opportunity_id"], + ["savings_opportunities.tenant_id", "savings_opportunities.id"], + ondelete="RESTRICT", + name="fk_savings_evidence_links_tenant_opportunity", + ), + ForeignKeyConstraint( + ["tenant_id", "realization_id"], + ["savings_realizations.tenant_id", "savings_realizations.id"], + ondelete="RESTRICT", + name="fk_savings_evidence_links_tenant_realization", + ), + CheckConstraint( + "entity_type IN ('baseline', 'opportunity', 'realization')", + name="ck_savings_evidence_links_entity_type", + ), + CheckConstraint( + "(entity_type = 'baseline' AND baseline_snapshot_id = entity_id " + "AND opportunity_id IS NULL AND realization_id IS NULL) OR " + "(entity_type = 'opportunity' AND opportunity_id = entity_id " + "AND baseline_snapshot_id IS NULL AND realization_id IS NULL) OR " + "(entity_type = 'realization' AND realization_id = entity_id " + "AND baseline_snapshot_id IS NULL AND opportunity_id IS NULL)", + name="ck_savings_evidence_links_entity_shape", + ), + CheckConstraint( + "verification_status IN ('unverified', 'verified', 'rejected', 'unavailable')", + name="ck_savings_evidence_links_verification", + ), + CheckConstraint( + "verification_status != 'verified' OR " + "(verified_by IS NOT NULL AND verified_at IS NOT NULL)", + name="ck_savings_evidence_links_verifier", + ), + CheckConstraint( + "length(trim(evidence_key)) > 0 AND length(trim(content_hash)) > 0", + name="ck_savings_evidence_links_keys", + ), + Index( + "ix_savings_evidence_links_entity", + "tenant_id", + "entity_type", + "entity_id", + "collected_at", + ), + Index( + "ix_savings_evidence_links_resource", + "tenant_id", + "resource_type", + "resource_id", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + evidence_key: Mapped[str] = mapped_column(String(180), nullable=False) + entity_type: Mapped[str] = mapped_column(String(20), nullable=False) + entity_id: Mapped[str] = mapped_column(String(36), nullable=False) + baseline_snapshot_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + opportunity_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + realization_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + evidence_role: Mapped[str] = mapped_column(String(50), nullable=False) + resource_type: Mapped[str] = mapped_column(String(50), nullable=False) + resource_id: Mapped[str] = mapped_column(String(160), nullable=False) + source_system: Mapped[str] = mapped_column(String(80), nullable=False) + external_event_id: Mapped[str | None] = mapped_column(String(160), nullable=True) + content_hash: Mapped[str] = mapped_column(String(80), nullable=False) + occurred_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + collected_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + verification_status: Mapped[str] = mapped_column( + String(20), nullable=False, default="unverified", server_default="unverified" + ) + verified_by: Mapped[str | None] = mapped_column(String(120), nullable=True) + verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + metadata_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + baseline_snapshot = relationship( + "ProfileBaselineSnapshot", + foreign_keys=[baseline_snapshot_id], + back_populates="evidence_links", + ) + opportunity = relationship( + "SavingsOpportunity", + foreign_keys=[opportunity_id], + back_populates="evidence_links", + ) + realization = relationship( + "SavingsRealization", + foreign_keys=[realization_id], + back_populates="evidence_links", + ) + + +class SavingsEvent(Base): + """Savings Ledger 的不可变动作记录和幂等首次响应。""" + __tablename__ = "savings_events" + __table_args__ = ( + UniqueConstraint("tenant_id", "id", name="uq_savings_events_tenant_id"), + UniqueConstraint( + "tenant_id", + "actor_id", + "request_id", + name="uq_savings_events_actor_request", + ), + UniqueConstraint( + "tenant_id", + "aggregate_type", + "aggregate_id", + "result_version", + name="uq_savings_events_aggregate_version", + ), + ForeignKeyConstraint( + ["tenant_id", "baseline_snapshot_id"], + ["profile_baseline_snapshots.tenant_id", "profile_baseline_snapshots.id"], + ondelete="RESTRICT", + name="fk_savings_events_tenant_baseline", + ), + ForeignKeyConstraint( + ["tenant_id", "opportunity_id"], + ["savings_opportunities.tenant_id", "savings_opportunities.id"], + ondelete="RESTRICT", + name="fk_savings_events_tenant_opportunity", + ), + ForeignKeyConstraint( + ["tenant_id", "realization_id"], + ["savings_realizations.tenant_id", "savings_realizations.id"], + ondelete="RESTRICT", + name="fk_savings_events_tenant_realization", + ), + CheckConstraint( + "aggregate_type IN ('baseline', 'opportunity', 'realization')", + name="ck_savings_events_aggregate_type", + ), + CheckConstraint( + "(aggregate_type = 'baseline' AND baseline_snapshot_id = aggregate_id " + "AND opportunity_id IS NULL AND realization_id IS NULL) OR " + "(aggregate_type = 'opportunity' AND opportunity_id = aggregate_id " + "AND baseline_snapshot_id IS NULL AND realization_id IS NULL) OR " + "(aggregate_type = 'realization' AND realization_id = aggregate_id " + "AND baseline_snapshot_id IS NULL AND opportunity_id IS NULL)", + name="ck_savings_events_aggregate_shape", + ), + CheckConstraint( + "length(trim(action)) > 0", + name="ck_savings_events_action", + ), + CheckConstraint( + "actor_type IN ('user', 'system', 'agent', 'service')", + name="ck_savings_events_actor_type", + ), + CheckConstraint( + "expected_version >= 0 AND result_version >= 1 " + "AND result_version >= expected_version", + name="ck_savings_events_version", + ), + CheckConstraint( + "length(trim(request_id)) > 0 AND length(trim(payload_fingerprint)) > 0", + name="ck_savings_events_request", + ), + Index( + "ix_savings_events_tenant_aggregate_time", + "tenant_id", + "aggregate_type", + "aggregate_id", + "occurred_at", + ), + Index( + "ix_savings_events_tenant_correlation", + "tenant_id", + "correlation_id", + "occurred_at", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=_new_id) + tenant_id: Mapped[str] = mapped_column(String(64), nullable=False) + aggregate_type: Mapped[str] = mapped_column(String(20), nullable=False) + aggregate_id: Mapped[str] = mapped_column(String(36), nullable=False) + baseline_snapshot_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + opportunity_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + realization_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + action: Mapped[str] = mapped_column(String(60), nullable=False) + actor_id: Mapped[str] = mapped_column(String(120), nullable=False) + actor_name: Mapped[str] = mapped_column(String(120), nullable=False) + actor_type: Mapped[str] = mapped_column(String(20), nullable=False) + request_id: Mapped[str] = mapped_column(String(120), nullable=False) + expected_version: Mapped[int] = mapped_column(Integer, nullable=False) + result_version: Mapped[int] = mapped_column(Integer, nullable=False) + payload_fingerprint: Mapped[str] = mapped_column(String(80), nullable=False) + payload_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + before_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + after_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + response_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + correlation_id: Mapped[str | None] = mapped_column(String(64), nullable=True) + causation_id: Mapped[str | None] = mapped_column(String(64), nullable=True) + occurred_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + + baseline_snapshot = relationship( + "ProfileBaselineSnapshot", + foreign_keys=[baseline_snapshot_id], + back_populates="events", + ) + opportunity = relationship( + "SavingsOpportunity", + foreign_keys=[opportunity_id], + back_populates="events", + ) + realization = relationship( + "SavingsRealization", + foreign_keys=[realization_id], + back_populates="events", + ) diff --git a/server/src/app/models/tenant.py b/server/src/app/models/tenant.py new file mode 100644 index 0000000..dde3e22 --- /dev/null +++ b/server/src/app/models/tenant.py @@ -0,0 +1,128 @@ +from __future__ import annotations + +import uuid +from datetime import datetime + +from sqlalchemy import ( + Boolean, + CheckConstraint, + DateTime, + ForeignKey, + ForeignKeyConstraint, + Index, + String, + UniqueConstraint, + func, +) +from sqlalchemy.orm import Mapped, mapped_column, relationship + +from app.db.base_class import Base + + +class Tenant(Base): + """可信租户注册表；业务代码不得用任意字符串替代这里的记录。""" + + __tablename__ = "tenants" + __table_args__ = ( + CheckConstraint( + "status IN ('active', 'suspended', 'disabled')", + name="ck_tenants_status", + ), + CheckConstraint( + "length(trim(tenant_id)) > 0 AND length(trim(tenant_code)) > 0 " + "AND length(trim(name)) > 0", + name="ck_tenants_identity", + ), + Index("ix_tenants_status", "status"), + ) + + tenant_id: Mapped[str] = mapped_column(String(64), primary_key=True) + tenant_code: Mapped[str] = mapped_column(String(64), unique=True, nullable=False) + name: Mapped[str] = mapped_column(String(160), nullable=False) + status: Mapped[str] = mapped_column(String(20), nullable=False, default="active") + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + onupdate=func.now(), + ) + + memberships = relationship( + "TenantMembership", + back_populates="tenant", + cascade="all, delete-orphan", + overlaps="employee,tenant_memberships", + ) + + +class TenantMembership(Base): + """员工在租户中的可认证成员资格。""" + + __tablename__ = "tenant_memberships" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "employee_id", + name="uq_tenant_memberships_tenant_employee", + ), + ForeignKeyConstraint( + ["tenant_id", "employee_id"], + ["employees.tenant_id", "employees.id"], + name="fk_tenant_memberships_tenant_employee", + ondelete="CASCADE", + ), + CheckConstraint( + "status IN ('active', 'inactive')", + name="ck_tenant_memberships_status", + ), + Index( + "ix_tenant_memberships_employee_active", + "employee_id", + "status", + ), + Index( + "ix_tenant_memberships_tenant_active", + "tenant_id", + "status", + ), + ) + + id: Mapped[str] = mapped_column( + String(36), + primary_key=True, + default=lambda: str(uuid.uuid4()), + ) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="RESTRICT"), + nullable=False, + ) + employee_id: Mapped[str] = mapped_column(String(36), nullable=False) + status: Mapped[str] = mapped_column(String(20), nullable=False, default="active") + is_primary: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + onupdate=func.now(), + ) + + tenant = relationship( + "Tenant", + back_populates="memberships", + overlaps="employee,tenant_memberships", + ) + employee = relationship( + "Employee", + back_populates="tenant_memberships", + overlaps="memberships,tenant", + ) diff --git a/server/src/app/models/tenant_finance_report.py b/server/src/app/models/tenant_finance_report.py new file mode 100644 index 0000000..a26c1b3 --- /dev/null +++ b/server/src/app/models/tenant_finance_report.py @@ -0,0 +1,98 @@ +from __future__ import annotations + +import uuid +from datetime import date, datetime +from typing import Any + +from sqlalchemy import ( + Boolean, + CheckConstraint, + Date, + DateTime, + ForeignKey, + Index, + String, + Text, + UniqueConstraint, + func, +) +from sqlalchemy.orm import Mapped, mapped_column +from sqlalchemy.types import JSON + +from app.db.base_class import Base + + +class TenantFinanceReportConfig(Base): + """租户自有的报告收件配置；绝不回落到平台级管理员邮箱。""" + + __tablename__ = "tenant_finance_report_configs" + __table_args__ = ( + CheckConstraint( + "status IN ('active', 'disabled')", + name="ck_tenant_finance_report_configs_status", + ), + ) + + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="CASCADE"), + primary_key=True, + ) + status: Mapped[str] = mapped_column(String(20), nullable=False, default="disabled") + delivery_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False) + recipients_json: Mapped[list[Any]] = mapped_column(JSON, nullable=False, default=list) + updated_by: Mapped[str] = mapped_column(String(100), nullable=False, default="") + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + nullable=False, + server_default=func.now(), + onupdate=func.now(), + ) + + +class TenantFinanceReportRun(Base): + """按租户和报告周期占位，提供并发安全的定时幂等边界。""" + + __tablename__ = "tenant_finance_report_runs" + __table_args__ = ( + UniqueConstraint( + "tenant_id", + "idempotency_key", + name="uq_tenant_finance_report_runs_idempotency", + ), + CheckConstraint( + "report_type IN ('weekly', 'quarterly', 'annual')", + name="ck_tenant_finance_report_runs_type", + ), + CheckConstraint( + "status IN ('running', 'succeeded', 'failed')", + name="ck_tenant_finance_report_runs_status", + ), + Index( + "ix_tenant_finance_report_runs_period", + "tenant_id", + "report_type", + "period_start", + "period_end", + ), + ) + + id: Mapped[str] = mapped_column(String(36), primary_key=True, default=lambda: str(uuid.uuid4())) + tenant_id: Mapped[str] = mapped_column( + ForeignKey("tenants.tenant_id", ondelete="CASCADE"), nullable=False + ) + report_type: Mapped[str] = mapped_column(String(20), nullable=False) + period_start: Mapped[date] = mapped_column(Date, nullable=False) + period_end: Mapped[date] = mapped_column(Date, nullable=False) + idempotency_key: Mapped[str] = mapped_column(String(180), nullable=False) + status: Mapped[str] = mapped_column(String(20), nullable=False, default="running") + agent_run_id: Mapped[str | None] = mapped_column(String(50), nullable=True, index=True) + storage_key: Mapped[str] = mapped_column(String(512), nullable=False, default="") + result_json: Mapped[dict[str, Any]] = mapped_column(JSON, nullable=False, default=dict) + error_message: Mapped[str | None] = mapped_column(Text, nullable=True) + started_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), nullable=False, server_default=func.now() + ) + finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) diff --git a/server/src/app/repositories/agent_asset.py b/server/src/app/repositories/agent_asset.py index 71a7a9c..6bc97c6 100644 --- a/server/src/app/repositories/agent_asset.py +++ b/server/src/app/repositories/agent_asset.py @@ -10,12 +10,65 @@ from app.models.agent_asset import ( AgentAssetTestRun, AgentAssetVersion, ) +from app.services.agent_asset_access import AgentAssetAccessScope from app.services.pagination import PageResult, paginate_select class AgentAssetRepository: - def __init__(self, db: Session) -> None: + def __init__( + self, + db: Session, + *, + access_scope: AgentAssetAccessScope | None = None, + ) -> None: self.db = db + # 没有可信用户上下文时只允许系统维护平台资产，绝不能退回全表查询。 + self.access_scope = access_scope or AgentAssetAccessScope( + tenant_id="__internal_no_tenant__", + is_platform_admin=True, + ) + + def _visible(self, stmt, model): + return stmt.where(self.access_scope.visibility_clause(model)) + + def _require_write(self, asset: AgentAsset) -> None: + self.access_scope.require_write(asset) + + def _prepare_asset_child(self, child): + asset = self.get(str(child.asset_id or "")) + if asset is None: + raise LookupError("Asset not found") + self._require_write(asset) + tenant_id = str(getattr(child, "tenant_id", "") or "").strip() + scope = str(getattr(child, "scope", "") or "").strip() + if tenant_id and tenant_id != asset.tenant_id: + raise LookupError("Asset not found") + if scope and scope != asset.scope: + raise LookupError("Asset not found") + child.tenant_id = asset.tenant_id + child.scope = asset.scope + return child + + def _prepare_tenant_evidence(self, child): + asset = self.get(str(child.asset_id or "")) + if asset is None: + raise LookupError("Asset not found") + tenant_id = str(getattr(child, "tenant_id", "") or "").strip() + scope = str(getattr(child, "scope", "") or "").strip() + expected_tenant = self.access_scope.tenant_id + if expected_tenant != "__internal_no_tenant__": + if tenant_id and tenant_id != expected_tenant: + raise LookupError("Asset not found") + if scope and scope != "tenant": + raise LookupError("Asset not found") + child.tenant_id = expected_tenant + child.scope = "tenant" + return child + if asset.scope != "platform" or asset.tenant_id != "platform": + raise LookupError("Asset not found") + child.tenant_id = "platform" + child.scope = "platform" + return child def _list_stmt( self, @@ -25,7 +78,7 @@ class AgentAssetRepository: domain: str | None = None, keyword: str | None = None, ): - stmt = select(AgentAsset) + stmt = self._visible(select(AgentAsset), AgentAsset) if asset_type: stmt = stmt.where(AgentAsset.asset_type == asset_type) @@ -81,18 +134,25 @@ class AgentAssetRepository: return paginate_select(self.db, stmt, page=page, page_size=page_size) def get(self, asset_id: str) -> AgentAsset | None: - return self.db.get(AgentAsset, asset_id) + stmt = self._visible( + select(AgentAsset).where(AgentAsset.id == asset_id), + AgentAsset, + ) + return self.db.scalar(stmt) def get_by_code(self, code: str) -> AgentAsset | None: - stmt = select(AgentAsset).where(AgentAsset.code == code) + stmt = self._visible( + select(AgentAsset).where(AgentAsset.code == code), + AgentAsset, + ).order_by(AgentAsset.scope.desc()) return self.db.scalar(stmt) def list_versions(self, asset_id: str, *, limit: int | None = None) -> list[AgentAssetVersion]: - stmt = ( + stmt = self._visible( select(AgentAssetVersion) - .where(AgentAssetVersion.asset_id == asset_id) - .order_by(AgentAssetVersion.created_at.desc()) - ) + .where(AgentAssetVersion.asset_id == asset_id), + AgentAssetVersion, + ).order_by(AgentAssetVersion.created_at.desc()) if limit is not None: stmt = stmt.limit(limit) return list(self.db.scalars(stmt).all()) @@ -101,26 +161,29 @@ class AgentAssetRepository: if not asset_ids: return [] - stmt = ( + stmt = self._visible( select(AgentAssetVersion) - .where(AgentAssetVersion.asset_id.in_(asset_ids)) - .order_by(AgentAssetVersion.asset_id, AgentAssetVersion.created_at.desc()) - ) + .where(AgentAssetVersion.asset_id.in_(asset_ids)), + AgentAssetVersion, + ).order_by(AgentAssetVersion.asset_id, AgentAssetVersion.created_at.desc()) return list(self.db.scalars(stmt).all()) def get_version(self, asset_id: str, version: str) -> AgentAssetVersion | None: - stmt = select(AgentAssetVersion).where( - AgentAssetVersion.asset_id == asset_id, - AgentAssetVersion.version == version, + stmt = self._visible( + select(AgentAssetVersion).where( + AgentAssetVersion.asset_id == asset_id, + AgentAssetVersion.version == version, + ), + AgentAssetVersion, ) return self.db.scalar(stmt) def list_reviews(self, asset_id: str, *, limit: int | None = None) -> list[AgentAssetReview]: - stmt = ( + stmt = self._visible( select(AgentAssetReview) - .where(AgentAssetReview.asset_id == asset_id) - .order_by(AgentAssetReview.created_at.desc()) - ) + .where(AgentAssetReview.asset_id == asset_id), + AgentAssetReview, + ).order_by(AgentAssetReview.created_at.desc()) if limit is not None: stmt = stmt.limit(limit) return list(self.db.scalars(stmt).all()) @@ -129,19 +192,22 @@ class AgentAssetRepository: if not asset_ids: return [] - stmt = ( + stmt = self._visible( select(AgentAssetReview) - .where(AgentAssetReview.asset_id.in_(asset_ids)) - .order_by(AgentAssetReview.asset_id, AgentAssetReview.created_at.desc()) - ) + .where(AgentAssetReview.asset_id.in_(asset_ids)), + AgentAssetReview, + ).order_by(AgentAssetReview.asset_id, AgentAssetReview.created_at.desc()) return list(self.db.scalars(stmt).all()) def get_review( self, asset_id: str, version: str, review_status: str | None = None ) -> AgentAssetReview | None: - stmt = select(AgentAssetReview).where( - AgentAssetReview.asset_id == asset_id, - AgentAssetReview.version == version, + stmt = self._visible( + select(AgentAssetReview).where( + AgentAssetReview.asset_id == asset_id, + AgentAssetReview.version == version, + ), + AgentAssetReview, ) if review_status: stmt = stmt.where(AgentAssetReview.review_status == review_status) @@ -149,24 +215,28 @@ class AgentAssetRepository: return self.db.scalar(stmt) def create_asset(self, asset: AgentAsset) -> AgentAsset: + self._require_write(asset) self.db.add(asset) self.db.commit() self.db.refresh(asset) return asset def save_asset(self, asset: AgentAsset) -> AgentAsset: + self._require_write(asset) self.db.add(asset) self.db.commit() self.db.refresh(asset) return asset def create_version(self, version: AgentAssetVersion) -> AgentAssetVersion: + self._prepare_asset_child(version) self.db.add(version) self.db.commit() self.db.refresh(version) return version def create_review(self, review: AgentAssetReview) -> AgentAssetReview: + self._prepare_asset_child(review) self.db.add(review) self.db.commit() self.db.refresh(review) @@ -181,11 +251,11 @@ class AgentAssetRepository: status: str | None = None, limit: int | None = None, ) -> list[AgentAssetTestRun]: - stmt = ( + stmt = self._visible( select(AgentAssetTestRun) - .where(AgentAssetTestRun.asset_id == asset_id) - .order_by(AgentAssetTestRun.created_at.desc()) - ) + .where(AgentAssetTestRun.asset_id == asset_id), + AgentAssetTestRun, + ).order_by(AgentAssetTestRun.created_at.desc()) if version: stmt = stmt.where(AgentAssetTestRun.version == version) if test_type: @@ -214,6 +284,7 @@ class AgentAssetRepository: return items[0] if items else None def create_test_run(self, test_run: AgentAssetTestRun) -> AgentAssetTestRun: + self._prepare_tenant_evidence(test_run) self.db.add(test_run) self.db.commit() self.db.refresh(test_run) @@ -227,11 +298,11 @@ class AgentAssetRepository: status: str | None = None, limit: int | None = None, ) -> list[AgentAssetRuleFeedback]: - stmt = ( + stmt = self._visible( select(AgentAssetRuleFeedback) - .where(AgentAssetRuleFeedback.asset_id == asset_id) - .order_by(AgentAssetRuleFeedback.created_at.desc()) - ) + .where(AgentAssetRuleFeedback.asset_id == asset_id), + AgentAssetRuleFeedback, + ).order_by(AgentAssetRuleFeedback.created_at.desc()) if version: stmt = stmt.where(AgentAssetRuleFeedback.version == version) if status: @@ -244,11 +315,13 @@ class AgentAssetRepository: self, feedback: AgentAssetRuleFeedback, ) -> AgentAssetRuleFeedback: + self._prepare_tenant_evidence(feedback) self.db.add(feedback) self.db.commit() self.db.refresh(feedback) return feedback def delete_asset(self, asset: AgentAsset) -> None: + self._require_write(asset) self.db.delete(asset) self.db.commit() diff --git a/server/src/app/repositories/agent_run.py b/server/src/app/repositories/agent_run.py index 509e40c..14fb231 100644 --- a/server/src/app/repositories/agent_run.py +++ b/server/src/app/repositories/agent_run.py @@ -2,7 +2,7 @@ from __future__ import annotations from typing import Any -from sqlalchemy import select +from sqlalchemy import Select, select from sqlalchemy.orm import Session from app.models.agent_run import AgentRun, AgentToolCall, SemanticParseLog @@ -19,8 +19,13 @@ class AgentRunRepository: status: str | None = None, source: str | None = None, limit: int = 20, + tenant_id: str | None = None, + scope_clause: Any | None = None, ) -> list[AgentRun]: stmt = select(AgentRun) + stmt = self._apply_tenant_scope(stmt, tenant_id) + if scope_clause is not None: + stmt = stmt.where(scope_clause) if agent: stmt = stmt.where(AgentRun.agent == agent) if status: @@ -37,6 +42,8 @@ class AgentRunRepository: status: str | None = None, source: str | None = None, limit: int = 20, + tenant_id: str | None = None, + scope_clause: Any | None = None, ) -> list[dict[str, Any]]: stmt = select( AgentRun.id.label("id"), @@ -69,6 +76,9 @@ class AgentRunRepository: AgentRun.ontology_json["intent"].as_string().label("ontology_intent"), AgentRun.ontology_json["parse_strategy"].as_string().label("ontology_parse_strategy"), ) + stmt = self._apply_tenant_scope(stmt, tenant_id) + if scope_clause is not None: + stmt = stmt.where(scope_clause) if agent: stmt = stmt.where(AgentRun.agent == agent) if status: @@ -98,10 +108,63 @@ class AgentRunRepository: ) return [dict(item) for item in self.db.execute(stmt).mappings().all()] - def get_by_run_id(self, run_id: str) -> AgentRun | None: + def list_light_semantic_parses(self, run_ids: list[str]) -> dict[str, dict[str, Any]]: + if not run_ids: + return {} + + stmt = ( + select( + SemanticParseLog.id, + SemanticParseLog.run_id, + SemanticParseLog.user_id, + SemanticParseLog.raw_query, + SemanticParseLog.scenario, + SemanticParseLog.intent, + SemanticParseLog.entities_json, + SemanticParseLog.time_range_json, + SemanticParseLog.metrics_json, + SemanticParseLog.constraints_json, + SemanticParseLog.risk_flags_json, + SemanticParseLog.permission_json, + SemanticParseLog.confidence, + SemanticParseLog.created_at, + ) + .where(SemanticParseLog.run_id.in_(run_ids)) + .order_by(SemanticParseLog.created_at.asc()) + ) + first_by_run_id: dict[str, dict[str, Any]] = {} + for row in self.db.execute(stmt).mappings(): + payload = dict(row) + first_by_run_id.setdefault(str(payload["run_id"]), payload) + return first_by_run_id + + def get_by_run_id( + self, + run_id: str, + *, + tenant_id: str | None = None, + ) -> AgentRun | None: stmt = select(AgentRun).where(AgentRun.run_id == run_id) + stmt = self._apply_tenant_scope(stmt, tenant_id) return self.db.scalar(stmt) + @staticmethod + def _apply_tenant_scope( + stmt: Select[Any], + tenant_id: str | None, + ) -> Select[Any]: + """在排序和 limit 前收窄租户；None 仅供受信任的内部调用。""" + if tenant_id is None: + return stmt + + route_tenant = AgentRun.route_json["tenant_id"].as_string() + ontology_tenant = AgentRun.ontology_json["tenant_id"].as_string() + # 两份独立载荷都必须携带同一可信租户，缺一、空值或冲突均不进入窗口。 + return stmt.where( + route_tenant == tenant_id, + ontology_tenant == tenant_id, + ) + def create_run(self, run: AgentRun) -> AgentRun: self.db.add(run) self.db.commit() diff --git a/server/src/app/repositories/employee.py b/server/src/app/repositories/employee.py index 62e7291..bf85c12 100644 --- a/server/src/app/repositories/employee.py +++ b/server/src/app/repositories/employee.py @@ -7,11 +7,13 @@ from app.models.employee import Employee from app.models.organization import OrganizationUnit from app.models.role import Role from app.services.pagination import PageResult, paginate_select +from app.services.tenant_registry import required_tenant_id class EmployeeRepository: - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str) -> None: self.db = db + self.tenant_id = required_tenant_id(tenant_id) def _list_stmt(self, status: str | None = None, keyword: str | None = None): stmt = ( @@ -22,6 +24,7 @@ class EmployeeRepository: selectinload(Employee.roles), selectinload(Employee.change_logs), ) + .where(Employee.tenant_id == self.tenant_id) .order_by(Employee.updated_at.desc(), Employee.name.asc()) ) @@ -65,16 +68,25 @@ class EmployeeRepository: selectinload(Employee.roles), selectinload(Employee.change_logs), ) - .where(Employee.id == employee_id) + .where( + Employee.tenant_id == self.tenant_id, + Employee.id == employee_id, + ) ) return self.db.execute(stmt).scalars().unique().first() def get_by_employee_no(self, employee_no: str) -> Employee | None: - stmt = select(Employee).where(Employee.employee_no == employee_no) + stmt = select(Employee).where( + Employee.tenant_id == self.tenant_id, + Employee.employee_no == employee_no, + ) return self.db.execute(stmt).scalars().first() def get_by_email(self, email: str) -> Employee | None: - stmt = select(Employee).where(Employee.email == email) + stmt = select(Employee).where( + Employee.tenant_id == self.tenant_id, + Employee.email == email, + ) return self.db.execute(stmt).scalars().first() def list_roles(self) -> list[Role]: @@ -86,15 +98,24 @@ class EmployeeRepository: return self.db.execute(stmt).scalars().first() def list_organization_units(self) -> list[OrganizationUnit]: - stmt = select(OrganizationUnit) + stmt = select(OrganizationUnit).where( + OrganizationUnit.tenant_id == self.tenant_id + ) return list(self.db.execute(stmt).scalars().all()) def get_organization_by_code(self, unit_code: str) -> OrganizationUnit | None: - stmt = select(OrganizationUnit).where(OrganizationUnit.unit_code == unit_code) + stmt = select(OrganizationUnit).where( + OrganizationUnit.tenant_id == self.tenant_id, + OrganizationUnit.unit_code == unit_code, + ) return self.db.execute(stmt).scalars().first() def count_employees(self) -> int: - stmt = select(func.count()).select_from(Employee) + stmt = ( + select(func.count()) + .select_from(Employee) + .where(Employee.tenant_id == self.tenant_id) + ) return int(self.db.execute(stmt).scalar_one()) def count_roles(self) -> int: @@ -102,17 +123,27 @@ class EmployeeRepository: return int(self.db.execute(stmt).scalar_one()) def count_organization_units(self) -> int: - stmt = select(func.count()).select_from(OrganizationUnit) + stmt = ( + select(func.count()) + .select_from(OrganizationUnit) + .where(OrganizationUnit.tenant_id == self.tenant_id) + ) return int(self.db.execute(stmt).scalar_one()) def create(self, employee: Employee) -> Employee: + self._require_employee_tenant(employee) self.db.add(employee) self.db.commit() self.db.refresh(employee) return employee def save(self, employee: Employee) -> Employee: + self._require_employee_tenant(employee) self.db.add(employee) self.db.commit() self.db.refresh(employee) return employee + + def _require_employee_tenant(self, employee: Employee) -> None: + if required_tenant_id(employee.tenant_id) != self.tenant_id: + raise ValueError("员工记录不属于当前可信租户。") diff --git a/server/src/app/schemas/agent_asset.py b/server/src/app/schemas/agent_asset.py index 0ba9cd6..8984473 100644 --- a/server/src/app/schemas/agent_asset.py +++ b/server/src/app/schemas/agent_asset.py @@ -1,7 +1,7 @@ from __future__ import annotations from datetime import datetime -from typing import Any +from typing import Any, Literal from pydantic import BaseModel, ConfigDict, Field @@ -15,6 +15,7 @@ from app.core.agent_enums import ( class AgentAssetCreate(BaseModel): + scope: Literal["tenant", "platform"] = "tenant" asset_type: AgentAssetType code: str = Field(min_length=1, max_length=100) name: str = Field(min_length=1, max_length=200) @@ -67,6 +68,8 @@ class AgentAssetReviewRead(BaseModel): model_config = ConfigDict(from_attributes=True) id: str + tenant_id: str + scope: str asset_id: str version: str reviewer: str @@ -90,6 +93,7 @@ class AgentAssetOnlyOfficeCallbackWrite(BaseModel): status: int = Field(description="ONLYOFFICE 回调状态码。") url: str | None = Field(default=None, description="文档下载地址，状态为 2 或 6 时使用。") + key: str | None = Field(default=None, description="ONLYOFFICE 当前文档会话 key。") users: list[str] = Field(default_factory=list, description="当前编辑用户列表。") @@ -193,6 +197,7 @@ class AgentAssetRiskRuleSampleTestRequest(BaseModel): class AgentAssetRiskRuleScenarioTestRequest(BaseModel): + target_tenant_id: str = Field(min_length=1, max_length=64) version: str | None = Field(default=None, max_length=30) intent: str = Field(default="", max_length=1000) filters: dict[str, Any] = Field(default_factory=dict) @@ -320,6 +325,8 @@ class AgentAssetRiskRuleFeedbackRead(BaseModel): model_config = ConfigDict(from_attributes=True) id: str + tenant_id: str + scope: str feedback_id: str asset_id: str version: str @@ -340,6 +347,8 @@ class AgentAssetRiskRuleTestRunRead(BaseModel): model_config = ConfigDict(from_attributes=True) id: str + tenant_id: str + scope: str asset_id: str version: str test_type: str @@ -399,6 +408,8 @@ class AgentAssetVersionRead(BaseModel): model_config = ConfigDict(from_attributes=True) id: str + tenant_id: str + scope: str asset_id: str version: str content: Any @@ -416,6 +427,8 @@ class AgentAssetListItem(BaseModel): model_config = ConfigDict(from_attributes=True) id: str + tenant_id: str + scope: str asset_type: str code: str name: str diff --git a/server/src/app/schemas/agent_asset_release.py b/server/src/app/schemas/agent_asset_release.py new file mode 100644 index 0000000..bb4dbcb --- /dev/null +++ b/server/src/app/schemas/agent_asset_release.py @@ -0,0 +1,117 @@ +from __future__ import annotations + +from datetime import datetime +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field + + +class AgentAssetReleasePolicyWrite(BaseModel): + shadow_min_samples: int = Field(default=20, ge=1, le=1_000_000) + canary_min_samples: int = Field(default=100, ge=1, le=1_000_000) + max_error_rate: float = Field(default=0.02, ge=0, le=1) + min_precision: float = Field(default=0.98, ge=0, le=1) + max_precision_drop: float = Field(default=0.02, ge=0, le=1) + canary_traffic_percent: int = Field(default=5, ge=1, le=50) + reviewer_quorum: int = Field(default=1, ge=1, le=2) + recall_gate_enabled: bool = True + negative_sample_percent: int = Field(default=20, ge=1, le=100) + negative_min_reviewed: int = Field(default=5, ge=1, le=10_000) + min_recall: float = Field(default=0.95, ge=0, le=1) + recall_confidence_level: Literal[0.9, 0.95, 0.99] = 0.95 + + +class AgentAssetReleaseStartWrite(BaseModel): + candidate_version: str = Field(min_length=1, max_length=30) + policy: AgentAssetReleasePolicyWrite = Field(default_factory=AgentAssetReleasePolicyWrite) + + +class AgentAssetReleaseMonitorTriggerWrite(BaseModel): + """可信监控触发器不接受调用方提供的样本数或质量指标。""" + + model_config = ConfigDict(extra="forbid") + + +class AgentAssetReleaseRollbackWrite(BaseModel): + reason: str = Field(min_length=2, max_length=1000) + + +class AgentAssetReleaseStateRead(BaseModel): + stage: Literal["unmanaged", "shadow", "canary", "active", "rolled_back"] = "unmanaged" + release_id: str = "" + candidate_version: str = "" + previous_version: str = "" + policy: dict[str, Any] = Field(default_factory=dict) + started_at: str = "" + started_by: str = "" + updated_at: str = "" + history: list[dict[str, Any]] = Field(default_factory=list) + rollback: dict[str, Any] | None = None + + +class AgentAssetReleaseServingPlanRead(BaseModel): + stage: str + primary_version: str + candidate_version: str + candidate_traffic_percent: int + shadow_evaluation: bool + + +class AgentAssetReleaseMonitorRead(BaseModel): + asset_id: str + release_id: str + stage: str + version: str + telemetry_status: Literal["collecting", "ready"] + status: Literal["collecting", "passed", "failed"] + evaluation_submitted: bool + release_stage: str + rolled_back: bool + reasons: list[str] = Field(default_factory=list) + test_run_id: str | None = None + metrics: dict[str, Any] = Field(default_factory=dict) + alerts: list[dict[str, str]] = Field(default_factory=list) + + +class AgentAssetReleaseReviewItemRead(BaseModel): + sample_id: str + observation_id: str + source_document_id: str + rule_code: str + business_stage: Literal["expense_application", "reimbursement"] + prediction_blinded: Literal[True] = True + reviewer_count: int = 0 + required_reviewers: int = 1 + conflicted: bool = False + created_at: datetime + + +class AgentAssetReleaseReviewQueueRead(BaseModel): + asset_id: str + release_id: str + stage: Literal["shadow", "canary", "active"] + version: str + pending_total: int + telemetry_status: Literal["collecting", "ready"] + reasons: list[str] = Field(default_factory=list) + metrics: dict[str, Any] = Field(default_factory=dict) + alerts: list[dict[str, str]] = Field(default_factory=list) + items: list[AgentAssetReleaseReviewItemRead] = Field(default_factory=list) + + +class AgentAssetReleaseReviewLabelWrite(BaseModel): + model_config = ConfigDict(extra="forbid") + + label: Literal[ + "risk_present", + "risk_absent", + "confirmed", + "false_positive", + ] + + +class AgentAssetReleaseReviewLabelRead(BaseModel): + label_id: str + observation_id: str + label: Literal["risk_present", "risk_absent", "confirmed", "false_positive"] + monitor: AgentAssetReleaseMonitorRead diff --git a/server/src/app/schemas/auth.py b/server/src/app/schemas/auth.py index ceccda1..feafcc7 100644 --- a/server/src/app/schemas/auth.py +++ b/server/src/app/schemas/auth.py @@ -3,12 +3,20 @@ from __future__ import annotations from datetime import datetime from typing import Any -from pydantic import BaseModel, EmailStr, Field +from pydantic import BaseModel, ConfigDict, EmailStr, Field class LoginRequest(BaseModel): + model_config = ConfigDict(populate_by_name=True) + username: str = Field(min_length=1, max_length=255) password: str = Field(min_length=1, max_length=128) + tenant_id: str | None = Field( + default=None, + alias="tenantId", + min_length=1, + max_length=64, + ) class AuthUserRead(BaseModel): @@ -29,6 +37,7 @@ class AuthUserRead(BaseModel): email: EmailStr | str avatar: str isAdmin: bool = False + tenantId: str class LoginResponse(BaseModel): diff --git a/server/src/app/schemas/cfo_value.py b/server/src/app/schemas/cfo_value.py new file mode 100644 index 0000000..d097351 --- /dev/null +++ b/server/src/app/schemas/cfo_value.py @@ -0,0 +1,142 @@ +from __future__ import annotations + +from datetime import datetime +from decimal import Decimal +from typing import Literal + +from pydantic import BaseModel, Field + + +class CfoValueMoneyRead(BaseModel): + currency: str + amount: Decimal + + +class CfoValueWindowRead(BaseModel): + start: datetime + end: datetime + as_of: datetime + timezone: str = "Asia/Shanghai" + + +class CfoValueSourceRead(BaseModel): + system: Literal["savings_ledger"] = "savings_ledger" + ledger_revision: str = "20260716_0015" + generated_at: datetime + freshness_at: datetime | None = None + data_status: Literal["complete", "empty", "partial"] + opportunity_count: int = 0 + realization_count: int = 0 + coverage_notes: list[str] = Field(default_factory=list) + + +class CfoValueCashKpiRead(BaseModel): + key: Literal["verified_net_cash_savings"] = "verified_net_cash_savings" + label: str = "财务确认净现金节省" + status: Literal["available", "empty"] + values: list[CfoValueMoneyRead] = Field(default_factory=list) + confirmed_realization_count: int = 0 + definition: str + + +class CfoValueUnavailableKpiRead(BaseModel): + key: Literal[ + "verified_releasable_labor_value", + "safe_straight_through_rate", + ] + label: str + status: Literal["unavailable", "collecting"] + reason: str + required_inputs: list[str] = Field(default_factory=list) + + +class CfoValueKpisRead(BaseModel): + verified_cash: CfoValueCashKpiRead + releasable_labor: CfoValueUnavailableKpiRead + safe_straight_through: CfoValueUnavailableKpiRead + + +class CfoValueFunnelStageRead(BaseModel): + key: Literal[ + "estimated", + "in_progress", + "actual_pending", + "verified", + "reversed", + "rejected_or_expired", + ] + label: str + count: int + values: list[CfoValueMoneyRead] = Field(default_factory=list) + + +class CfoValueFunnelRead(BaseModel): + stages: list[CfoValueFunnelStageRead] = Field(default_factory=list) + mature_estimated_values: list[CfoValueMoneyRead] = Field(default_factory=list) + verified_values: list[CfoValueMoneyRead] = Field(default_factory=list) + realization_rate_by_currency: dict[str, Decimal | None] = Field(default_factory=dict) + + +class CfoValueTrendPointRead(BaseModel): + period: str + currency: str + verified_net: Decimal = Decimal("0") + actual_pending: Decimal = Decimal("0") + reversal: Decimal = Decimal("0") + + +class CfoValueBreakdownItemRead(BaseModel): + dimension: str + dimension_id: str + dimension_name: str + opportunity_count: int + verified_values: list[CfoValueMoneyRead] = Field(default_factory=list) + estimated_values: list[CfoValueMoneyRead] = Field(default_factory=list) + + +class CfoValueBreakdownGroupRead(BaseModel): + dimension: str + items: list[CfoValueBreakdownItemRead] = Field(default_factory=list) + + +class CfoValueGuardrailRead(BaseModel): + key: str + label: str + status: Literal["ok", "attention", "unavailable"] + count: int | None = None + values: list[CfoValueMoneyRead] = Field(default_factory=list) + rate: Decimal | None = None + reason: str = "" + + +class CfoValueDataQualityRead(BaseModel): + pending_confirmation_count: int = 0 + business_state_only_count: int = 0 + pending_dedupe_count: int = 0 + missing_fx_count: int = 0 + missing_evidence_count: int = 0 + actual_over_estimate_count: int = 0 + notes: list[str] = Field(default_factory=list) + + +class CfoValueFiltersRead(BaseModel): + department_id: str | None = None + project_code: str | None = None + expense_type: str | None = None + supplier_id: str | None = None + city: str | None = None + owner_id: str | None = None + source_type: str | None = None + value_kind: Literal["cash", "labor"] | None = None + + +class CfoValueDashboardRead(BaseModel): + window: CfoValueWindowRead + source: CfoValueSourceRead + filters: CfoValueFiltersRead + kpis: CfoValueKpisRead + funnel: CfoValueFunnelRead + trend: list[CfoValueTrendPointRead] = Field(default_factory=list) + breakdowns: list[CfoValueBreakdownGroupRead] = Field(default_factory=list) + guardrails: list[CfoValueGuardrailRead] = Field(default_factory=list) + data_quality: CfoValueDataQualityRead diff --git a/server/src/app/schemas/commercial.py b/server/src/app/schemas/commercial.py new file mode 100644 index 0000000..83cf196 --- /dev/null +++ b/server/src/app/schemas/commercial.py @@ -0,0 +1,516 @@ +from __future__ import annotations + +from datetime import datetime +from decimal import Decimal +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +PricingModel = Literal["subscription", "usage", "hybrid", "pilot", "custom"] +BillingInterval = Literal["monthly", "quarterly", "annual", "contract"] +SubscriptionStatus = Literal["trialing", "active", "past_due", "suspended", "canceled", "expired"] +EntitlementType = Literal["feature", "metered", "unlimited"] +EntitlementStatus = Literal["active", "suspended", "expired"] +ResetInterval = Literal["none", "monthly", "quarterly", "annual", "contract"] +OveragePolicy = Literal["block", "allow", "alert"] + + +class CommercialPlanCreate(BaseModel): + plan_code: str = Field(min_length=1, max_length=80) + name: str = Field(min_length=1, max_length=160) + pricing_model: PricingModel + billing_interval: BillingInterval + currency: str = Field(pattern=r"^[A-Za-z]{3}$") + base_fee: Decimal = Field(ge=0, max_digits=20, decimal_places=4) + included_seats: int = Field(default=0, ge=0) + overage_enabled: bool = False + effective_from: datetime + effective_to: datetime | None = None + contract_terms_json: dict[str, Any] = Field(default_factory=dict) + reason: str = Field( + default="平台管理员创建套餐版本", + min_length=2, + max_length=500, + ) + + @field_validator("effective_from", "effective_to") + @classmethod + def require_timezone(cls, value: datetime | None) -> datetime | None: + return _require_timezone(value) + + @model_validator(mode="after") + def validate_window(self) -> CommercialPlanCreate: + if self.effective_to is not None and self.effective_to <= self.effective_from: + raise ValueError("套餐失效时间必须晚于生效时间。") + return self + + +class CommercialPlanRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + plan_code: str + name: str + pricing_model: PricingModel + billing_interval: BillingInterval + currency: str + base_fee: Decimal + included_seats: int + overage_enabled: bool + status: Literal["draft", "active", "retired"] + effective_from: datetime + effective_to: datetime | None = None + version: int + contract_terms_json: dict[str, Any] = Field(default_factory=dict) + created_by: str + created_at: datetime + updated_at: datetime + + +class CommercialPlanActivationRead(BaseModel): + plan: CommercialPlanRead + retired_plan_ids: list[str] = Field(default_factory=list) + + +class CommercialVersionAction(BaseModel): + expected_version: int = Field(ge=1) + reason: str = Field(min_length=2, max_length=500) + + +class CommercialSubscriptionTransition(BaseModel): + expected_version: int = Field(ge=1) + target_status: Literal["past_due", "suspended", "canceled", "expired"] + reason: str = Field(min_length=2, max_length=500) + + +class CommercialSubscriptionCreate(BaseModel): + subscription_key: str = Field(min_length=1, max_length=120) + plan_id: str = Field(min_length=1, max_length=36) + status: Literal["trialing", "active"] = "active" + starts_at: datetime + ends_at: datetime | None = None + current_period_start: datetime + current_period_end: datetime + seats: int = Field(ge=1) + auto_renew: bool = False + external_provider: str | None = Field(default=None, max_length=60) + external_subscription_id: str | None = Field(default=None, max_length=160) + metadata_json: dict[str, Any] = Field(default_factory=dict) + reason: str = Field( + default="平台管理员创建订阅", + min_length=2, + max_length=500, + ) + + @field_validator( + "starts_at", + "ends_at", + "current_period_start", + "current_period_end", + ) + @classmethod + def require_timezone(cls, value: datetime | None) -> datetime | None: + return _require_timezone(value) + + @model_validator(mode="after") + def validate_windows_and_provider(self) -> CommercialSubscriptionCreate: + if self.current_period_end <= self.current_period_start: + raise ValueError("当前订阅周期结束时间必须晚于开始时间。") + if self.ends_at is not None and self.ends_at <= self.starts_at: + raise ValueError("订阅结束时间必须晚于开始时间。") + if bool(self.external_provider) != bool(self.external_subscription_id): + raise ValueError("外部订阅提供商和订阅编号必须同时填写或同时留空。") + return self + + +class CommercialSubscriptionRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + subscription_key: str + plan_id: str + status: SubscriptionStatus + starts_at: datetime + ends_at: datetime | None = None + current_period_start: datetime + current_period_end: datetime + seats: int + base_fee_snapshot: Decimal + currency: str + billing_interval: BillingInterval + auto_renew: bool + external_provider: str | None = None + external_subscription_id: str | None = None + canceled_at: datetime | None = None + version: int + metadata_json: dict[str, Any] = Field(default_factory=dict) + created_by: str + created_at: datetime + updated_at: datetime + + +class CommercialEntitlementUpsert(BaseModel): + subscription_id: str = Field(min_length=1, max_length=36) + entitlement_key: str = Field(min_length=1, max_length=120) + metric_key: str = Field(min_length=1, max_length=120) + entitlement_type: EntitlementType + unit: str = Field(min_length=1, max_length=40) + included_quantity: Decimal | None = Field(default=None, ge=0) + hard_limit_quantity: Decimal | None = Field(default=None, ge=0) + reset_interval: ResetInterval + overage_policy: OveragePolicy + status: EntitlementStatus = "active" + effective_from: datetime + effective_to: datetime | None = None + config_json: dict[str, Any] = Field(default_factory=dict) + reason: str = Field( + default="平台管理员维护商业权益", + min_length=2, + max_length=500, + ) + + @field_validator("effective_from", "effective_to") + @classmethod + def require_timezone(cls, value: datetime | None) -> datetime | None: + return _require_timezone(value) + + @model_validator(mode="after") + def validate_shape(self) -> CommercialEntitlementUpsert: + if self.effective_to is not None and self.effective_to <= self.effective_from: + raise ValueError("权益失效时间必须晚于生效时间。") + if self.entitlement_type == "unlimited": + if self.included_quantity is not None or self.hard_limit_quantity is not None: + raise ValueError("无限权益不能配置包含量或硬配额。") + elif self.entitlement_type == "feature": + if self.included_quantity not in {Decimal("0"), Decimal("1")}: + raise ValueError("功能权益包含量只能是 0 或 1。") + if self.hard_limit_quantity not in {None, Decimal("0"), Decimal("1")}: + raise ValueError("功能权益硬配额只能是 0 或 1。") + else: + if self.included_quantity is None: + raise ValueError("计量权益必须配置包含量。") + if ( + self.hard_limit_quantity is not None + and self.hard_limit_quantity < self.included_quantity + ): + raise ValueError("硬配额不能小于包含量。") + return self + + +class CommercialEntitlementRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + subscription_id: str + entitlement_key: str + metric_key: str + entitlement_type: EntitlementType + unit: str + included_quantity: Decimal | None = None + hard_limit_quantity: Decimal | None = None + reset_interval: ResetInterval + overage_policy: OveragePolicy + status: EntitlementStatus + effective_from: datetime + effective_to: datetime | None = None + version: int + config_json: dict[str, Any] = Field(default_factory=dict) + created_at: datetime + updated_at: datetime + + +class UsageMeterEventCreate(BaseModel): + subscription_id: str = Field(min_length=1, max_length=36) + entitlement_id: str = Field(min_length=1, max_length=36) + event_type: Literal["usage", "credit", "adjustment", "reversal"] = "usage" + quantity: Decimal = Field(max_digits=20, decimal_places=6) + occurred_at: datetime + source_system: str = Field(min_length=1, max_length=80) + idempotency_key: str = Field(min_length=1, max_length=160) + reversal_of_event_id: str | None = Field(default=None, max_length=36) + subject_type: str | None = Field(default=None, max_length=60) + subject_id: str | None = Field(default=None, max_length=160) + correlation_id: str | None = Field(default=None, max_length=120) + trace_id: str | None = Field(default=None, max_length=120) + metadata_json: dict[str, Any] = Field(default_factory=dict) + + @field_validator("occurred_at") + @classmethod + def require_timezone(cls, value: datetime) -> datetime: + return _require_timezone(value) + + @model_validator(mode="after") + def validate_usage_event(self) -> UsageMeterEventCreate: + if self.event_type == "usage" and self.quantity <= 0: + raise ValueError("usage 事件数量必须大于 0。") + if self.event_type == "credit" and self.quantity >= 0: + raise ValueError("credit 事件数量必须小于 0。") + if self.event_type in {"adjustment", "reversal"} and self.quantity == 0: + raise ValueError("adjustment/reversal 事件数量不能为 0。") + if (self.event_type == "reversal") != bool(self.reversal_of_event_id): + raise ValueError("只有 reversal 事件必须且只能填写被冲回事件。") + if bool(self.subject_type) != bool(self.subject_id): + raise ValueError("计量主体类型和编号必须同时填写或同时留空。") + return self + + +class UsageMeterEventRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + subscription_id: str + entitlement_id: str + billing_period_id: str + event_type: Literal["usage", "credit", "adjustment", "reversal"] + metric_key: str + quantity: Decimal + unit: str + period_key: str + quota_period_key: str + occurred_at: datetime + source_system: str + idempotency_key: str + request_fingerprint: str + reversal_of_event_id: str | None = None + subject_type: str | None = None + subject_id: str | None = None + actor_type: Literal["system", "user", "integration", "admin"] + actor_id: str + correlation_id: str | None = None + trace_id: str | None = None + metadata_json: dict[str, Any] = Field(default_factory=dict) + recorded_at: datetime + + +class CommercialCostEventCreate(BaseModel): + subscription_id: str | None = Field(default=None, max_length=36) + usage_event_id: str | None = Field(default=None, max_length=36) + event_type: Literal["incurred", "credit", "adjustment", "reversal"] = "incurred" + cost_category: Literal[ + "ai_inference", + "ocr", + "storage", + "connector", + "support", + "implementation", + "infrastructure", + "payment", + "other", + ] + quantity: Decimal = Field(gt=0, max_digits=20, decimal_places=6) + unit: str = Field(min_length=1, max_length=40) + unit_cost: Decimal = Field(ge=0, max_digits=20, decimal_places=8) + original_currency: str = Field(pattern=r"^[A-Za-z]{3}$") + reporting_currency: str = Field(pattern=r"^[A-Za-z]{3}$") + fx_rate: Decimal = Field(gt=0, max_digits=20, decimal_places=8) + provider: str | None = Field(default=None, max_length=120) + sku: str | None = Field(default=None, max_length=120) + model_name: str | None = Field(default=None, max_length=120) + allocation_key: str = Field(min_length=1, max_length=160) + occurred_at: datetime + source_system: str = Field(min_length=1, max_length=80) + idempotency_key: str = Field(min_length=1, max_length=160) + reversal_of_cost_event_id: str | None = Field(default=None, max_length=36) + correlation_id: str | None = Field(default=None, max_length=120) + trace_id: str | None = Field(default=None, max_length=120) + metadata_json: dict[str, Any] = Field(default_factory=dict) + + @field_validator("occurred_at") + @classmethod + def require_timezone(cls, value: datetime) -> datetime: + return _require_timezone(value) + + @model_validator(mode="after") + def validate_cost_event(self) -> CommercialCostEventCreate: + if (self.event_type == "reversal") != bool(self.reversal_of_cost_event_id): + raise ValueError("只有 reversal 成本事件必须且只能填写被冲回事件。") + if self.usage_event_id and not self.subscription_id: + raise ValueError("关联用量事件时必须填写订阅编号。") + return self + + +class CommercialCostEventRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + subscription_id: str | None = None + billing_period_id: str | None = None + usage_event_id: str | None = None + event_type: Literal["incurred", "credit", "adjustment", "reversal"] + cost_category: str + quantity: Decimal + unit: str + unit_cost: Decimal + cost_amount: Decimal + original_currency: str + reporting_amount: Decimal + reporting_currency: str + fx_rate: Decimal + provider: str | None = None + sku: str | None = None + model_name: str | None = None + allocation_key: str + occurred_at: datetime + source_system: str + idempotency_key: str + request_fingerprint: str + reversal_of_cost_event_id: str | None = None + correlation_id: str | None = None + trace_id: str | None = None + metadata_json: dict[str, Any] = Field(default_factory=dict) + recorded_at: datetime + + +class CommercialMutationRead(BaseModel): + created: bool + usage_event: UsageMeterEventRead | None = None + cost_event: CommercialCostEventRead | None = None + + +class CommercialQuotaRead(BaseModel): + entitlement: CommercialEntitlementRead + billing_period_id: str + period_key: str + quota_period_key: str + used_quantity: Decimal + reserved_quantity: Decimal = Decimal("0") + included_remaining: Decimal | None = None + hard_limit_remaining: Decimal | None = None + overage_quantity: Decimal + status: Literal["available", "approaching", "exhausted", "unlimited", "inactive"] + commercially_allowed: bool + reason: str + + +class CommercialAccountRead(BaseModel): + tenant_id: str + as_of: datetime + data_status: Literal["available", "partial", "unavailable"] + plan: CommercialPlanRead | None = None + subscription: CommercialSubscriptionRead | None = None + quotas: list[CommercialQuotaRead] = Field(default_factory=list) + notes: list[str] = Field(default_factory=list) + + +class EntitlementGateRead(BaseModel): + tenant_id: str + entitlement_key: str + requested_quantity: Decimal + commercial_allowed: bool + security_decision: Literal["allow", "deny", "human_review"] + final_allowed: bool + reason: str + quota: CommercialQuotaRead | None = None + + +class CommercialMoneyRead(BaseModel): + currency: str + amount: Decimal + basis: str + + +class CommercialRatioRead(BaseModel): + currency: str + ratio: Decimal + numerator: Decimal + denominator: Decimal + + +class CommercialMetricRead(BaseModel): + key: str + label: str + status: Literal["available", "partial", "unavailable"] + values: list[CommercialMoneyRead] = Field(default_factory=list) + ratios: list[CommercialRatioRead] = Field(default_factory=list) + reason: str + required_inputs: list[str] = Field(default_factory=list) + notes: list[str] = Field(default_factory=list) + + +class CommercialAnalyticsRead(BaseModel): + tenant_id: str + start: datetime + end: datetime + as_of: datetime + generated_at: datetime + customer_charges: CommercialMetricRead + internal_costs: CommercialMetricRead + contribution_margin: CommercialMetricRead + verified_cash_savings: CommercialMetricRead + customer_roi: CommercialMetricRead + customer_labor_value: CommercialMetricRead + data_quality_status: Literal["complete", "partial", "unavailable"] + data_quality_issues: list[str] = Field(default_factory=list) + + +class CommercialPricingScenarioWrite(BaseModel): + start: datetime + end: datetime + as_of: datetime + target_contribution_margin_rate: Decimal = Field( + default=Decimal("0.65"), + ge=0, + lt=Decimal("0.95"), + decimal_places=6, + ) + max_verified_savings_share: Decimal = Field( + default=Decimal("0.25"), + gt=0, + le=1, + decimal_places=6, + ) + + @field_validator("start", "end", "as_of") + @classmethod + def require_timezone(cls, value: datetime) -> datetime: + return _require_timezone(value) + + @model_validator(mode="after") + def validate_window(self) -> CommercialPricingScenarioWrite: + if self.end <= self.start: + raise ValueError("定价分析结束时间必须晚于开始时间。") + if self.as_of < self.start: + raise ValueError("定价分析 as_of 不能早于分析开始时间。") + return self + + +class CommercialPricingCurrencyScenarioRead(BaseModel): + currency: str + status: Literal["feasible", "insufficient_value", "cost_only", "unavailable"] + internal_cost: Decimal | None = None + verified_cash_savings: Decimal | None = None + minimum_sustainable_charge: Decimal | None = None + maximum_value_aligned_charge: Decimal | None = None + maximum_success_fee: Decimal | None = None + customer_roi_at_minimum_charge: Decimal | None = None + contribution_margin_at_value_ceiling: Decimal | None = None + reason: str + + +class CommercialPricingScenarioRead(BaseModel): + tenant_id: str + start: datetime + end: datetime + as_of: datetime + target_contribution_margin_rate: Decimal + max_verified_savings_share: Decimal + recommended_model: Literal[ + "hybrid", + "subscription", + "pilot_collecting", + "optimize_unit_economics", + ] + scenarios: list[CommercialPricingCurrencyScenarioRead] = Field(default_factory=list) + evidence_status: Literal["complete", "partial", "unavailable"] + notes: list[str] = Field(default_factory=list) + + +def _require_timezone(value: datetime | None) -> datetime | None: + if value is not None and (value.tzinfo is None or value.utcoffset() is None): + raise ValueError("商业事实时间必须显式包含时区。") + return value diff --git a/server/src/app/schemas/commercial_billing.py b/server/src/app/schemas/commercial_billing.py new file mode 100644 index 0000000..5e56104 --- /dev/null +++ b/server/src/app/schemas/commercial_billing.py @@ -0,0 +1,117 @@ +from __future__ import annotations + +from datetime import datetime +from decimal import Decimal +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field + + +class CommercialBillingPeriodRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + subscription_id: str + plan_id: str + period_sequence: int + period_key: str + status: Literal["issued"] + temporal_state: Literal["upcoming", "current", "elapsed"] + period_start: datetime + period_end: datetime + subscription_status_snapshot: Literal[ + "trialing", + "active", + "past_due", + "suspended", + "canceled", + "expired", + ] + plan_code_snapshot: str + plan_version_snapshot: int + pricing_model_snapshot: Literal[ + "subscription", + "usage", + "hybrid", + "pilot", + "custom", + ] + billing_interval: Literal["monthly", "quarterly", "annual", "contract"] + currency: str + base_fee_snapshot: Decimal + seats_snapshot: int + source: Literal["subscription_created", "auto_renew", "migration_backfill"] + idempotency_key: str + created_by: str + created_at: datetime + + +class CommercialBillingPeriodTenantRead(BaseModel): + """租户可见账期，不暴露平台操作人和内部幂等键。""" + + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + subscription_id: str + plan_id: str + period_sequence: int + period_key: str + status: Literal["issued"] + temporal_state: Literal["upcoming", "current", "elapsed"] + period_start: datetime + period_end: datetime + subscription_status_snapshot: str + plan_code_snapshot: str + plan_version_snapshot: int + pricing_model_snapshot: str + billing_interval: str + currency: str + base_fee_snapshot: Decimal + seats_snapshot: int + source: str + created_at: datetime + + +class CommercialAdminEventRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + actor_type: Literal["user", "system", "migration"] + actor_id: str + request_id: str + reason: str + action: Literal[ + "plan_created", + "plan_activated", + "plan_retired", + "subscription_created", + "subscription_activated", + "subscription_transitioned", + "entitlement_created", + "entitlement_updated", + "entitlement_activated", + "billing_period_created", + "subscription_rolled_over", + "legacy_state_imported", + ] + resource_type: Literal["plan", "subscription", "entitlement", "billing_period"] + resource_id: str + resource_version: int + before_json: dict[str, Any] = Field(default_factory=dict) + after_json: dict[str, Any] = Field(default_factory=dict) + occurred_at: datetime + + +class CommercialRolloverRead(BaseModel): + tenant_id: str + subscription_id: str + status: Literal["rolled_over", "not_due", "ineligible", "replayed"] + reason_code: str + reason: str + created_period_ids: list[str] = Field(default_factory=list) + current_period_start: datetime + current_period_end: datetime + subscription_version: int diff --git a/server/src/app/schemas/finance_report_config.py b/server/src/app/schemas/finance_report_config.py new file mode 100644 index 0000000..1c28b0b --- /dev/null +++ b/server/src/app/schemas/finance_report_config.py @@ -0,0 +1,19 @@ +from __future__ import annotations + +from datetime import datetime + +from pydantic import BaseModel, Field + + +class FinanceReportConfigUpdate(BaseModel): + recipients: list[str] = Field(default_factory=list, max_length=50) + delivery_enabled: bool = False + + +class FinanceReportConfigRead(BaseModel): + tenant_id: str + status: str + delivery_enabled: bool + recipients: list[str] = Field(default_factory=list) + updated_by: str = "" + updated_at: datetime | None = None diff --git a/server/src/app/schemas/financial_connector.py b/server/src/app/schemas/financial_connector.py new file mode 100644 index 0000000..96c87e4 --- /dev/null +++ b/server/src/app/schemas/financial_connector.py @@ -0,0 +1,381 @@ +from __future__ import annotations + +from datetime import datetime +from decimal import Decimal, InvalidOperation +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +FinancialEventType = Literal[ + "payment_settled", + "payment_failed", + "erp_posted", + "erp_posting_failed", + "payment_refunded", + "payment_reversed", +] + +FinancialConnectorSimulationScenario = Literal[ + "success", + "failure", + "out_of_order", + "duplicate", + "conflict", + "refund", + "erp_receipt", +] + + +class FinancialConnectorConfigCreate(BaseModel): + provider: str = Field(min_length=1, max_length=80) + environment: Literal["test", "mock", "staging", "production"] + key_version: str = Field(min_length=1, max_length=40) + secret_ref: str = Field(min_length=1, max_length=180) + allowed_event_types: list[FinancialEventType] = Field(min_length=1) + clock_skew_seconds: int = Field(default=300, ge=30, le=900) + status: Literal["disabled"] = "disabled" + request_id: str = Field(min_length=8, max_length=120) + reason: str = Field(min_length=4, max_length=1000) + + @field_validator( + "provider", + "key_version", + "secret_ref", + "request_id", + "reason", + mode="before", + ) + @classmethod + def normalize_text(cls, value: Any) -> str: + return str(value or "").strip() + + +class FinancialConnectorConfigRead(BaseModel): + """刻意不包含 secret_ref 或任何 HMAC 材料。""" + + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + provider: str + environment: Literal["test", "mock", "staging", "production"] + key_version: str + allowed_event_types_json: list[str] = Field(default_factory=list) + clock_skew_seconds: int + status: Literal["active", "disabled", "rotating"] + version: int + last_success_at: datetime | None = None + last_error_at: datetime | None = None + last_error_code: str | None = None + created_by: str + created_at: datetime + updated_at: datetime + + +class FinancialConnectorConfigLifecycleAction(BaseModel): + expected_version: int = Field(ge=1) + request_id: str = Field(min_length=8, max_length=120) + reason: str = Field(min_length=4, max_length=1000) + + @field_validator("request_id", "reason", mode="before") + @classmethod + def normalize_lifecycle_text(cls, value: Any) -> str: + return str(value or "").strip() + + +class FinancialConnectorConfigRotateAction(FinancialConnectorConfigLifecycleAction): + new_key_version: str = Field(min_length=1, max_length=40) + new_secret_ref: str = Field(min_length=1, max_length=180) + + @field_validator("new_key_version", "new_secret_ref", mode="before") + @classmethod + def normalize_rotation_text(cls, value: Any) -> str: + return str(value or "").strip() + + +class FinancialConnectorConfigRotationRead(BaseModel): + previous: FinancialConnectorConfigRead + replacement: FinancialConnectorConfigRead + + +class FinancialConnectorConfigEventRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + config_id: str + action: Literal[ + "created", + "activated", + "disabled", + "rotation_started", + "rotation_replacement_created", + ] + actor_id: str + request_id: str + reason: str + expected_version: int | None = None + before_json: dict[str, Any] = Field(default_factory=dict) + after_json: dict[str, Any] = Field(default_factory=dict) + occurred_at: datetime + + +class FinancialEventEnvelope(BaseModel): + model_config = ConfigDict(extra="forbid") + + tenant_id: str = Field(min_length=1, max_length=64) + external_event_id: str = Field(min_length=1, max_length=160) + event_type: FinancialEventType + occurred_at: datetime + correlation_id: str = Field(min_length=1, max_length=64) + payload: dict[str, Any] + + @field_validator("tenant_id", "external_event_id", "correlation_id", mode="before") + @classmethod + def normalize_text(cls, value: Any) -> str: + return str(value or "").strip() + + @model_validator(mode="after") + def validate_payload_shape(self) -> FinancialEventEnvelope: + allowed = { + "claim_id", + "claim_reference", + "amount", + "currency", + "external_payment_reference", + "origin_external_event_id", + "erp_document_number", + "accounting_period", + "failure_code", + } + unknown = sorted(str(key) for key in self.payload if key not in allowed) + if unknown: + raise ValueError(f"财务事件 payload 包含未允许字段：{', '.join(unknown)}") + required = {"claim_id", "claim_reference", "amount", "currency"} + if self.event_type in {"payment_settled", "payment_failed"}: + required.add("external_payment_reference") + if self.event_type in {"payment_refunded", "payment_reversed"}: + required.update({"external_payment_reference", "origin_external_event_id"}) + if self.event_type in {"erp_posted", "erp_posting_failed"}: + required.add("origin_external_event_id") + if self.event_type == "erp_posted": + required.add("erp_document_number") + missing = sorted(key for key in required if not str(self.payload.get(key) or "").strip()) + if missing: + raise ValueError(f"财务事件 payload 缺少字段：{', '.join(missing)}") + currency = str(self.payload.get("currency") or "").strip().upper() + if len(currency) != 3 or not currency.isalpha(): + raise ValueError("财务事件币种必须是三位字母代码。") + try: + amount = Decimal(str(self.payload.get("amount"))) + except (InvalidOperation, TypeError, ValueError) as error: + raise ValueError("财务事件金额无效。") from error + if not amount.is_finite() or amount < 0 or amount.as_tuple().exponent < -4: + raise ValueError("财务事件金额必须是非负且最多四位小数。") + if self.occurred_at.tzinfo is None: + raise ValueError("财务事件发生时间必须包含时区。") + return self + + +class FinancialEventIngestionRead(BaseModel): + accepted: bool = True + replayed: bool = False + event_id: str + external_event_id: str + processing_status: Literal["processed", "exception", "pending"] + reconciliation_case_id: str | None = None + reconciliation_status: str | None = None + claim_status: str | None = None + verification_level: Literal["simulated", "staging_verified", "production_verified"] + evidence_classification: Literal["simulated_connector", "staging_connector", "external_cash"] + projection_scope: Literal[ + "simulation_only", + "canonical", + "legacy_nonproduction_effect_unknown", + ] + error_code: str | None = None + + +class FinancialConnectorSimulationCreate(BaseModel): + claim_id: str = Field(min_length=1, max_length=36) + scenario: FinancialConnectorSimulationScenario + request_id: str = Field(min_length=8, max_length=120) + + @field_validator("claim_id", "request_id", mode="before") + @classmethod + def normalize_simulation_text(cls, value: Any) -> str: + return str(value or "").strip() + + +class FinancialConnectorSimulationStepRead(BaseModel): + name: str + event_type: FinancialEventType + outcome: Literal["accepted", "replayed", "expected_exception", "conflict"] + event_id: str | None = None + processing_status: Literal["processed", "exception", "pending"] | None = None + error_code: str | None = None + + +class FinancialConnectorSimulationRead(BaseModel): + tenant_id: str + config_id: str + provider: str + environment: Literal["test", "mock", "staging"] + scenario: FinancialConnectorSimulationScenario + request_fingerprint: str + evidence_classification: Literal["simulated_connector", "staging_connector"] + projection_scope: Literal["simulation_only"] = "simulation_only" + core_side_effects_allowed: Literal[False] = False + steps: list[FinancialConnectorSimulationStepRead] = Field(default_factory=list) + + +class FinancialConnectorMetricAvailabilityRead(BaseModel): + status: Literal["available", "unavailable"] + source: str + reason: str | None = None + + +class FinancialConnectorObservabilityItemRead(BaseModel): + config_id: str + provider: str + environment: Literal["test", "mock", "staging", "production"] + key_version: str + status: Literal["active", "disabled", "rotating"] + evidence_classification: Literal[ + "simulated_connector", + "staging_connector", + "external_cash", + ] + evidence_label: str + last_success_at: datetime | None = None + last_error_at: datetime | None = None + last_error_code: str | None = None + event_count: int = 0 + processed_event_count: int = 0 + failed_event_count: int = 0 + failure_rate: float = Field(default=0.0, ge=0.0, le=1.0) + backlog_count: int = 0 + reconciliation_anomaly_count: int = 0 + retry_count: int | None = 0 + auth_failure_count: int = 0 + signature_failure_count: int | None = 0 + payload_conflict_count: int = 0 + latest_replay_at: datetime | None = None + latest_auth_failure_at: datetime | None = None + latest_signature_failure_at: datetime | None = None + latest_payload_conflict_at: datetime | None = None + + +class FinancialConnectorObservabilitySummaryRead(BaseModel): + connector_count: int = 0 + active_connector_count: int = 0 + event_count: int = 0 + processed_event_count: int = 0 + failed_event_count: int = 0 + failure_rate: float = Field(default=0.0, ge=0.0, le=1.0) + backlog_count: int = 0 + reconciliation_anomaly_count: int = 0 + retry_count: int | None = 0 + auth_failure_count: int = 0 + signature_failure_count: int | None = 0 + payload_conflict_count: int = 0 + latest_replay_at: datetime | None = None + latest_auth_failure_at: datetime | None = None + latest_signature_failure_at: datetime | None = None + latest_payload_conflict_at: datetime | None = None + + +class FinancialConnectorObservabilityRead(BaseModel): + tenant_id: str + window_hours: int + window_started_at: datetime + as_of: datetime + generated_at: datetime + source_revision: str + summary: FinancialConnectorObservabilitySummaryRead + retry_metric: FinancialConnectorMetricAvailabilityRead + auth_failure_metric: FinancialConnectorMetricAvailabilityRead + signature_failure_metric: FinancialConnectorMetricAvailabilityRead + payload_conflict_metric: FinancialConnectorMetricAvailabilityRead + items: list[FinancialConnectorObservabilityItemRead] = Field(default_factory=list) + + +class FinancialPaymentEvidenceRead(BaseModel): + claim_id: str + claim_status: str + payment_state: Literal["not_paid", "paid"] + evidence_classification: Literal[ + "none", + "internal_manual_payment", + "external_cash", + "simulated_connector", + "staging_connector", + ] + evidence_label: str + trust_level: Literal["none", "low", "high"] + source_type: Literal["none", "manual_confirmation", "external_connector"] + provider: str | None = None + verification_level: str | None = None + external_reference_tail: str | None = None + recorded_at: datetime | None = None + disclaimer: str + + +class PaymentReconciliationEventRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + connector_event_id: str + action: str + actor_type: str + actor_id: str + reason: str | None = None + correlation_id: str + occurred_at: datetime + + +class PaymentReconciliationCaseRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + provider: str + claim_id: str + expense_case_id: str | None = None + expected_amount: Decimal + actual_amount: Decimal + amount_difference: Decimal + expected_currency: str + actual_currency: str + expected_reference: str + external_reference_tail: str | None = None + status: str + exception_code: str | None = None + erp_status: str + erp_document_tail: str | None = None + assigned_to: str | None = None + last_connector_event_id: str + version: int + created_at: datetime + updated_at: datetime + + +class PaymentReconciliationCaseDetailRead(PaymentReconciliationCaseRead): + timeline: list[PaymentReconciliationEventRead] = Field(default_factory=list) + + +class PaymentReconciliationListRead(BaseModel): + items: list[PaymentReconciliationCaseRead] = Field(default_factory=list) + total: int + page: int + page_size: int + + +class PaymentReconciliationActionCreate(BaseModel): + expected_version: int = Field(ge=1) + reason: str = Field(min_length=4, max_length=1000) + + @field_validator("reason", mode="before") + @classmethod + def normalize_reason(cls, value: Any) -> str: + return str(value or "").strip() diff --git a/server/src/app/schemas/knowledge.py b/server/src/app/schemas/knowledge.py index 9ae9a15..c5b646f 100644 --- a/server/src/app/schemas/knowledge.py +++ b/server/src/app/schemas/knowledge.py @@ -31,6 +31,8 @@ class KnowledgePreviewPageRead(BaseModel): class KnowledgeDocumentRead(BaseModel): id: str + scope: str = "tenant" + readOnly: bool = False name: str folder: str tag: str diff --git a/server/src/app/schemas/reimbursement.py b/server/src/app/schemas/reimbursement.py index 696f77a..481dcb8 100644 --- a/server/src/app/schemas/reimbursement.py +++ b/server/src/app/schemas/reimbursement.py @@ -128,14 +128,41 @@ class ExpenseClaimStandardAdjustmentRisk(BaseModel): item_id: str | None = Field(default=None, max_length=120) title: str | None = Field(default=None, max_length=120) risk: str | None = Field(default=None, max_length=500) - application_days: int | None = Field(default=None, ge=1, le=365) - original_amount: Decimal | None = None - reimbursable_amount: Decimal | None = None + application_days: int | None = Field( + default=None, + ge=1, + le=365, + description="旧客户端展示提示；服务端按单据、明细及关联申请重新确定政策天数。", + ) + original_amount: Decimal | None = Field( + default=None, + description="旧客户端展示提示；服务端仅使用数据库中的明细原金额。", + ) + reimbursable_amount: Decimal | None = Field( + default=None, + description="旧客户端展示提示；服务端仅使用规则中心计算的可报销金额。", + ) class ExpenseClaimStandardAdjustmentPayload(BaseModel): + request_id: str | None = Field( + default=None, + min_length=1, + max_length=120, + description="客户端生成的幂等请求号；相同请求号只能用于同一组明细。", + ) + expected_updated_at: datetime | None = Field( + default=None, + description="客户端最后读取到的单据更新时间，用于拒绝过期页面提交。", + ) risks: list[ExpenseClaimStandardAdjustmentRisk] = Field(default_factory=list, max_length=20) + @field_validator("request_id") + @classmethod + def normalize_standard_adjustment_request_id(cls, value: str | None) -> str | None: + normalized = str(value or "").strip() + return normalized or None + class ExpenseClaimPreReviewRemediationRead(BaseModel): action: str diff --git a/server/src/app/schemas/savings.py b/server/src/app/schemas/savings.py new file mode 100644 index 0000000..0da2174 --- /dev/null +++ b/server/src/app/schemas/savings.py @@ -0,0 +1,335 @@ +from __future__ import annotations + +from datetime import date, datetime +from decimal import Decimal +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +SavingsValueType = Literal["cash", "labor"] +SavingsOpportunityStatus = Literal[ + "identified", + "accepted", + "in_progress", + "realized", + "verified", + "reversed", + "rejected", + "expired", +] +SavingsOpportunityAction = Literal["accept", "start", "reject", "expire"] +SavingsOpportunityAvailableAction = Literal[ + "accept", + "start", + "reject", + "expire", + "record_realization", +] +SavingsRealizationAction = Literal["confirm", "reject", "reverse"] + + +class ProfileBaselineSnapshotRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + baseline_key: str + baseline_type: Literal["historical_cohort", "policy_counterfactual", "manual"] + dimension_type: str + dimension_id: str + metric_key: str + unit: str + original_currency: str | None = None + baseline_value: Decimal + window_start: datetime | None = None + window_end: datetime | None = None + sample_count: int + method: str + query_fingerprint: str + data_quality_status: Literal["complete", "partial", "insufficient", "invalid"] + data_quality_score: Decimal + quality_issues_json: list[dict[str, Any]] = Field(default_factory=list) + algorithm_version: str + policy_version: str | None = None + policy_effective_from: date | None = None + policy_effective_to: date | None = None + target_resource_type: str | None = None + target_resource_id: str | None = None + frozen_at: datetime + frozen_by: str + valid_until: datetime | None = None + version: int + created_at: datetime + + +class SavingsEvidenceLinkRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + evidence_key: str + entity_type: Literal["baseline", "opportunity", "realization"] + entity_id: str + evidence_role: str + resource_type: str + resource_id: str + source_system: str + external_event_id: str | None = None + content_hash: str + occurred_at: datetime + collected_at: datetime + verification_status: Literal["unverified", "verified", "rejected", "unavailable"] + verified_by: str | None = None + verified_at: datetime | None = None + metadata_json: dict[str, Any] = Field(default_factory=dict) + created_at: datetime + + +class SavingsEventRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + aggregate_type: Literal["baseline", "opportunity", "realization"] + aggregate_id: str + action: str + actor_id: str + actor_name: str + actor_type: Literal["user", "system", "agent", "service"] + request_id: str + expected_version: int + result_version: int + payload_fingerprint: str + payload_json: dict[str, Any] = Field(default_factory=dict) + before_json: dict[str, Any] = Field(default_factory=dict) + after_json: dict[str, Any] = Field(default_factory=dict) + response_json: dict[str, Any] = Field(default_factory=dict) + correlation_id: str | None = None + causation_id: str | None = None + occurred_at: datetime + + +class SavingsRealizationRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + realization_key: str + opportunity_id: str + expense_case_id: str + claim_id: str + claim_item_id: str | None = None + business_event_id: str | None = None + realization_type: Literal["actual", "reversal"] + reversal_of_realization_id: str | None = None + realized_at: datetime + recorded_by_id: str + recorded_by_name: str + actual_gross: Decimal + incremental_cost: Decimal + actual_net: Decimal + original_currency: str + reporting_amount: Decimal + reporting_currency: str + fx_rate: Decimal + fx_source: str + fx_date: date + fx_version: str + attribution_method: str + attribution_ratio: Decimal + benefit_key: str + dedupe_status: Literal["pending_review", "canonical", "duplicate", "excluded"] + canonical_realization_id: str | None = None + status: Literal["pending_confirmation", "finance_confirmed", "rejected", "reversed"] + finance_confirmer_id: str | None = None + finance_confirmer_name: str | None = None + confirmed_at: datetime | None = None + confirmation_note: str | None = None + rejected_by_id: str | None = None + rejected_by_name: str | None = None + rejected_at: datetime | None = None + rejection_reason: str | None = None + reversed_by_id: str | None = None + reversed_by_name: str | None = None + reversed_at: datetime | None = None + reversal_reason: str | None = None + baseline_snapshot_json: dict[str, Any] = Field(default_factory=dict) + final_snapshot_json: dict[str, Any] = Field(default_factory=dict) + evidence_json: list[dict[str, Any]] = Field(default_factory=list) + version: int + created_at: datetime + updated_at: datetime + available_actions: list[SavingsRealizationAction] = Field(default_factory=list) + read_only_reason: str = "" + + +class SavingsOpportunityRead(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: str + tenant_id: str + opportunity_key: str + expense_case_id: str + claim_id: str + claim_item_id: str | None = None + discovery_business_event_id: str | None = None + source_type: str + source_id: str + category: str + claim_no_snapshot: str + title: str + description: str + value_kind: SavingsValueType + exposure_amount: Decimal + baseline_snapshot_id: str + baseline_amount: Decimal + target_amount: Decimal + estimated_gross: Decimal + estimated_cost: Decimal + estimated_net: Decimal + estimated_low: Decimal + estimated_high: Decimal + confidence: Decimal + currency: str + reporting_currency: str + attribution_method: str + ai_decision_id: str | None = None + owner_id: str + owner_name: str + owner_role: str + due_at: datetime | None = None + status: SavingsOpportunityStatus + version: int + benefit_key: str + suggested_action: str + dimension_json: dict[str, Any] = Field(default_factory=dict) + baseline_snapshot_json: dict[str, Any] = Field(default_factory=dict) + evidence_json: list[dict[str, Any]] = Field(default_factory=list) + accepted_at: datetime | None = None + started_at: datetime | None = None + realized_at: datetime | None = None + verified_at: datetime | None = None + closed_at: datetime | None = None + created_at: datetime + updated_at: datetime + available_actions: list[SavingsOpportunityAvailableAction] = Field(default_factory=list) + read_only_reason: str = "" + baseline: ProfileBaselineSnapshotRead | None = None + realizations: list[SavingsRealizationRead] = Field(default_factory=list) + evidence: list[SavingsEvidenceLinkRead] = Field(default_factory=list) + events: list[SavingsEventRead] = Field(default_factory=list) + + +class SavingsOpportunityListRead(BaseModel): + items: list[SavingsOpportunityRead] = Field(default_factory=list) + total: int = 0 + page: int = 1 + page_size: int = 20 + total_pages: int = 0 + generated_at: datetime + + +class SavingsActionBase(BaseModel): + request_id: str = Field(min_length=8, max_length=120) + expected_version: int = Field(ge=1) + comment: str = Field(min_length=2, max_length=1000) + + @field_validator("request_id", "comment", mode="before") + @classmethod + def normalize_text(cls, value: Any) -> str: + return str(value or "").strip() + + +class SavingsOpportunityActionCreate(SavingsActionBase): + action: SavingsOpportunityAction + + +class SavingsEvidenceCreate(BaseModel): + evidence_key: str = Field(min_length=1, max_length=160) + evidence_role: str = Field(min_length=1, max_length=50) + resource_type: str = Field(min_length=1, max_length=50) + resource_id: str = Field(min_length=1, max_length=160) + source_system: str = Field(min_length=1, max_length=60) + external_event_id: str | None = Field(default=None, max_length=160) + content_hash: str = Field(min_length=16, max_length=80) + occurred_at: datetime + verification_status: Literal["unverified", "unavailable"] = "unverified" + metadata_json: dict[str, Any] = Field(default_factory=dict) + + @field_validator( + "evidence_key", + "evidence_role", + "resource_type", + "resource_id", + "source_system", + "external_event_id", + "content_hash", + mode="before", + ) + @classmethod + def normalize_optional_text(cls, value: Any) -> Any: + if value is None: + return None + normalized = str(value).strip() + return normalized or None + + +class SavingsRealizationCreate(SavingsActionBase): + actual_gross: Decimal = Field(gt=0, max_digits=16, decimal_places=2) + incremental_cost: Decimal = Field( + default=Decimal("0.00"), + ge=0, + max_digits=16, + decimal_places=2, + ) + currency: str = Field(default="CNY", min_length=3, max_length=10) + realized_at: datetime + attribution_method: str = Field(min_length=2, max_length=80) + attribution_ratio: Decimal = Field(default=Decimal("1.0000"), gt=0, le=1) + evidence_level: Literal["business_state", "external_document", "external_verified"] + evidence: list[SavingsEvidenceCreate] = Field(default_factory=list, max_length=30) + + @field_validator("currency", mode="before") + @classmethod + def normalize_currency(cls, value: Any) -> str: + return str(value or "CNY").strip().upper() + + @field_validator("attribution_method", mode="before") + @classmethod + def normalize_method(cls, value: Any) -> str: + return str(value or "").strip() + + @model_validator(mode="after") + def require_result_evidence(self) -> SavingsRealizationCreate: + if not self.evidence: + raise ValueError("实际节省结果必须至少提供一条可追溯证据。") + if self.incremental_cost > self.actual_gross: + raise ValueError("执行成本不能高于实际毛节省。") + return self + + +class SavingsRealizationActionCreate(SavingsActionBase): + action: SavingsRealizationAction + reversal_amount: Decimal | None = Field(default=None, gt=0, max_digits=16, decimal_places=2) + evidence: list[SavingsEvidenceCreate] = Field(default_factory=list, max_length=30) + + @model_validator(mode="after") + def validate_action_payload(self) -> SavingsRealizationActionCreate: + if self.action == "reverse" and self.reversal_amount is None: + raise ValueError("冲回动作必须提供冲回金额。") + if self.action != "reverse" and self.reversal_amount is not None: + raise ValueError("只有冲回动作可以提供冲回金额。") + return self + + +class SavingsOpportunityMutationRead(BaseModel): + opportunity: SavingsOpportunityRead + event: SavingsEventRead + replayed: bool = False + + +class SavingsRealizationMutationRead(BaseModel): + realization: SavingsRealizationRead + opportunity: SavingsOpportunityRead + event: SavingsEventRead + replayed: bool = False diff --git a/server/src/app/schemas/savings_insights.py b/server/src/app/schemas/savings_insights.py new file mode 100644 index 0000000..54b61e8 --- /dev/null +++ b/server/src/app/schemas/savings_insights.py @@ -0,0 +1,163 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal +from typing import Any, Literal + +from pydantic import BaseModel, Field, field_validator, model_validator + +from app.schemas.savings import ProfileBaselineSnapshotRead + +SavingsBaselineDimension = Literal[ + "employee", + "department", + "expense_type", + "city", + "project", + "workflow", + "supplier", +] +SavingsInsightType = Literal[ + "budget_forecast_variance", + "repeated_small_expense_pattern", + "historical_price_deviation", + "anomaly_driver_attribution", + "policy_simulation_candidate", +] + + +class SavingsAnalysisQualityIssue(BaseModel): + code: str + message: str + severity: Literal["info", "warning", "error"] = "warning" + dimension_type: str | None = None + dimension_id: str | None = None + metadata: dict[str, Any] = Field(default_factory=dict) + + +class SavingsWindowRequest(BaseModel): + request_id: str = Field(min_length=8, max_length=120) + window_start: datetime + window_end: datetime + as_of: datetime + + @field_validator("request_id", mode="before") + @classmethod + def normalize_request_id(cls, value: Any) -> str: + return str(value or "").strip() + + @field_validator("window_start", "window_end", "as_of") + @classmethod + def require_timezone(cls, value: datetime) -> datetime: + if value.tzinfo is None or value.utcoffset() is None: + raise ValueError("分析窗口时间必须包含时区。") + return value.astimezone(UTC) + + @model_validator(mode="after") + def validate_window(self) -> SavingsWindowRequest: + if self.window_end < self.window_start: + raise ValueError("分析窗口结束时间不能早于开始时间。") + if self.as_of < self.window_end: + raise ValueError("分析截止时间不能早于窗口结束时间。") + return self + + +class SavingsBaselineGenerateRequest(SavingsWindowRequest): + dimensions: list[SavingsBaselineDimension] = Field( + default_factory=lambda: [ + "employee", + "department", + "expense_type", + "city", + "project", + "workflow", + "supplier", + ], + min_length=1, + max_length=7, + ) + minimum_complete_samples: int = Field(default=5, ge=2, le=100) + + @field_validator("dimensions") + @classmethod + def deduplicate_dimensions( + cls, + value: list[SavingsBaselineDimension], + ) -> list[SavingsBaselineDimension]: + return list(dict.fromkeys(value)) + + +class SavingsBaselineGenerationRead(BaseModel): + request_id: str + request_fingerprint: str + tenant_id: str + data_scope: str + snapshots: list[ProfileBaselineSnapshotRead] = Field(default_factory=list) + quality_issues: list[SavingsAnalysisQualityIssue] = Field(default_factory=list) + source_claim_count: int = 0 + source_item_count: int = 0 + source_workflow_cycle_count: int = 0 + replayed: bool = False + generated_at: datetime + + +class SavingsInsightAnalyzeRequest(SavingsWindowRequest): + small_amount_threshold: Decimal = Field( + default=Decimal("200.00"), + gt=0, + max_digits=16, + decimal_places=2, + ) + minimum_repeat_count: int = Field(default=3, ge=3, le=20) + price_deviation_ratio: Decimal = Field( + default=Decimal("1.2500"), + gt=1, + le=10, + max_digits=8, + decimal_places=4, + ) + + +class SavingsInsightEvidenceRead(BaseModel): + evidence_role: str + resource_type: str + resource_id: str + content_hash: str + occurred_at: datetime + metadata: dict[str, Any] = Field(default_factory=dict) + + +class SavingsInsightCandidateRead(BaseModel): + candidate_key: str + insight_type: SavingsInsightType + title: str + description: str + dimension_json: dict[str, Any] = Field(default_factory=dict) + evidence: list[SavingsInsightEvidenceRead] = Field(default_factory=list) + evidence_sufficient_for_signal: bool + data_quality_status: Literal["complete", "partial", "insufficient"] + quality_issues: list[SavingsAnalysisQualityIssue] = Field(default_factory=list) + exposure_amount: Decimal | None = None + exposure_meaning: str + currency: str | None = None + estimated_savings: Decimal | None = None + monetization_status: Literal[ + "eligible", + "withheld_no_counterfactual", + "unavailable", + ] + baseline_snapshot_id: str | None = None + + +class SavingsInsightAnalysisRead(BaseModel): + request_id: str + request_fingerprint: str + tenant_id: str + data_scope: str + candidates: list[SavingsInsightCandidateRead] = Field(default_factory=list) + quality_issues: list[SavingsAnalysisQualityIssue] = Field(default_factory=list) + source_claim_count: int = 0 + source_item_count: int = 0 + created_opportunity_ids: list[str] = Field(default_factory=list) + monetized_opportunity_count: int = 0 + generated_at: datetime diff --git a/server/src/app/services/account_behavior_profile.py b/server/src/app/services/account_behavior_profile.py index 2f5552e..c14e0a1 100644 --- a/server/src/app/services/account_behavior_profile.py +++ b/server/src/app/services/account_behavior_profile.py @@ -18,11 +18,13 @@ from app.algorithem.employee_behavior_profile_tags import build_profile_radar, b from app.models.agent_run import AgentRun from app.schemas.employee_profile import EmployeeProfileLatestRead, EmployeeProfileRead from app.services.employee_behavior_profile_helpers import EmployeeBehaviorProfileMetricHelpers +from app.services.finance_report_tenant import require_report_tenant_id class AccountBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str = "default") -> None: self.db = db + self.tenant_id = require_report_tenant_id(tenant_id) def get_latest_account_profile( self, @@ -87,7 +89,10 @@ class AccountBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): profiles=[ EmployeeProfileRead( profile_type=payload["profile_type"], - profile_label=PROFILE_LABELS.get(payload["profile_type"], payload["profile_type"]), + profile_label=PROFILE_LABELS.get( + payload["profile_type"], + payload["profile_type"], + ), score=payload["score"], level=payload["level"], level_label=LEVEL_LABELS.get(payload["level"], payload["level"]), @@ -173,6 +178,11 @@ class AccountBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): stmt = ( select(AgentRun) .options(selectinload(AgentRun.tool_calls)) - .where(AgentRun.started_at >= cutoff, AgentRun.user_id.in_(normalized)) + .where( + AgentRun.route_json["tenant_id"].as_string() == self.tenant_id, + AgentRun.ontology_json["tenant_id"].as_string() == self.tenant_id, + AgentRun.started_at >= cutoff, + AgentRun.user_id.in_(normalized), + ) ) return list(self.db.scalars(stmt).all()) diff --git a/server/src/app/services/agent_asset_access.py b/server/src/app/services/agent_asset_access.py new file mode 100644 index 0000000..a580ade --- /dev/null +++ b/server/src/app/services/agent_asset_access.py @@ -0,0 +1,93 @@ +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any + +from sqlalchemy import and_, or_, select + +from app.api.deps import CurrentUserContext +from app.core.agent_asset_scope import ( + AGENT_ASSET_PLATFORM_SCOPE, + AGENT_ASSET_PLATFORM_TENANT_ID, + AGENT_ASSET_TENANT_SCOPE, +) + + +def stable_user_principal(current_user: CurrentUserContext) -> str: + """返回不可由显示名变化影响的审计主体。""" + + employee_id = str(current_user.employee_id or "").strip() + if employee_id: + return f"employee:{employee_id}" + username = str(current_user.username or "").strip().casefold() + if username: + return f"username:{username}" + raise PermissionError("当前登录用户缺少稳定身份标识。") + + +@dataclass(frozen=True, slots=True) +class AgentAssetAccessScope: + tenant_id: str + is_platform_admin: bool = False + + @classmethod + def from_user(cls, current_user: CurrentUserContext) -> AgentAssetAccessScope: + tenant_id = str(current_user.tenant_id or "").strip() + if not tenant_id or tenant_id == AGENT_ASSET_PLATFORM_TENANT_ID: + raise PermissionError("当前登录会话缺少有效租户。") + return cls(tenant_id=tenant_id, is_platform_admin=bool(current_user.is_admin)) + + def visibility_clause(self, model: Any) -> Any: + return or_( + and_( + model.scope == AGENT_ASSET_PLATFORM_SCOPE, + model.tenant_id == AGENT_ASSET_PLATFORM_TENANT_ID, + ), + and_( + model.scope == AGENT_ASSET_TENANT_SCOPE, + model.tenant_id == self.tenant_id, + ), + ) + + def can_write(self, resource: Any) -> bool: + scope = str(getattr(resource, "scope", "") or "").strip() + tenant_id = str(getattr(resource, "tenant_id", "") or "").strip() + if scope == AGENT_ASSET_PLATFORM_SCOPE: + return bool( + self.is_platform_admin and tenant_id == AGENT_ASSET_PLATFORM_TENANT_ID + ) + return scope == AGENT_ASSET_TENANT_SCOPE and tenant_id == self.tenant_id + + def require_write(self, resource: Any) -> None: + scope = str(getattr(resource, "scope", "") or "").strip() + tenant_id = str(getattr(resource, "tenant_id", "") or "").strip() + if scope == AGENT_ASSET_PLATFORM_SCOPE: + if tenant_id != AGENT_ASSET_PLATFORM_TENANT_ID: + raise LookupError("Asset not found") + if not self.is_platform_admin: + raise PermissionError("只有平台管理员可以修改平台资产。") + return + if scope != AGENT_ASSET_TENANT_SCOPE or tenant_id != self.tenant_id: + raise LookupError("Asset not found") + + +def tenant_resource_identity(tenant_id: str) -> tuple[str, str]: + normalized = str(tenant_id or "").strip() + if not normalized or normalized == AGENT_ASSET_PLATFORM_TENANT_ID: + raise ValueError("tenant_id 必须是有效的企业租户。") + return normalized, AGENT_ASSET_TENANT_SCOPE + + +def platform_resource_identity() -> tuple[str, str]: + return AGENT_ASSET_PLATFORM_TENANT_ID, AGENT_ASSET_PLATFORM_SCOPE + + +def platform_asset_statement(): + """平台初始化任务只能查询平台资产，不能按 code 命中租户覆盖项。""" + + from app.models.agent_asset import AgentAsset + + return select(AgentAsset).where( + AgentAsset.scope == AGENT_ASSET_PLATFORM_SCOPE, + AgentAsset.tenant_id == AGENT_ASSET_PLATFORM_TENANT_ID, + ) diff --git a/server/src/app/services/agent_asset_onlyoffice.py b/server/src/app/services/agent_asset_onlyoffice.py index 9b8164c..50a411e 100644 --- a/server/src/app/services/agent_asset_onlyoffice.py +++ b/server/src/app/services/agent_asset_onlyoffice.py @@ -4,8 +4,6 @@ from dataclasses import dataclass from datetime import UTC, datetime from pathlib import Path from typing import Any -from urllib.parse import quote -from urllib.request import Request, urlopen import jwt @@ -13,6 +11,14 @@ from app.api.deps import CurrentUserContext from app.core.config import get_settings from app.models.agent_asset import AgentAsset from app.schemas.agent_asset import AgentAssetOnlyOfficeConfigRead, AgentAssetRead +from app.services.agent_asset_access import stable_user_principal +from app.services.agent_asset_onlyoffice_security import ( + AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + AGENT_ASSET_ONLYOFFICE_CONTENT_SCOPE, + AgentAssetOnlyOfficeSecurityError, + AgentAssetOnlyOfficeSessionService, + AgentAssetOnlyOfficeValidatedSession, +) from app.services.agent_asset_spreadsheet import ( COMPANY_TRAVEL_EXPENSE_RULE_FILENAME, FINANCE_RULES_LIBRARY, @@ -20,6 +26,7 @@ from app.services.agent_asset_spreadsheet import ( AgentAssetSpreadsheetManager, RuleSpreadsheetMeta, ) +from app.services.knowledge_onlyoffice_security import download_onlyoffice_document from app.services.settings import resolve_onlyoffice_settings PREVIEW_RULE_ASSET_ID = "preview-rule-expense-company-travel-expense" @@ -35,7 +42,7 @@ PREVIEW_RULE_VERSION_FILENAMES = { class OnlyOfficeCallbackPayload: status: int download_url: str - users: list[str] + document_key: str class AgentAssetOnlyOfficeMixin: @@ -55,16 +62,27 @@ class AgentAssetOnlyOfficeMixin: resolved_version, metadata = self._ensure_preview_rule_spreadsheet(version=version) return self._build_onlyoffice_spreadsheet_config( asset_id=asset_id, + tenant_id="platform", + resource_scope="platform", + document_version=resolved_version, current_user=current_user, metadata=metadata, - editable=resolved_version == PREVIEW_RULE_CURRENT_VERSION, + editable=( + resolved_version == PREVIEW_RULE_CURRENT_VERSION + and current_user.is_admin + ), ) asset = self._require_spreadsheet_rule(asset_id) - _, metadata = self._resolve_current_spreadsheet_meta(asset) - editable = self._can_edit_current_spreadsheet(current_user) + resolved_version, metadata = self._resolve_current_spreadsheet_meta(asset) + editable = self._can_edit_current_spreadsheet(current_user) and ( + asset.scope == "tenant" or current_user.is_admin + ) return self._build_onlyoffice_spreadsheet_config( asset_id=asset.id, + tenant_id=asset.tenant_id, + resource_scope=asset.scope, + document_version=resolved_version, current_user=current_user, metadata=metadata, editable=editable, @@ -75,10 +93,21 @@ class AgentAssetOnlyOfficeMixin: asset_id: str, *, version: str | None = None, + validated_session: AgentAssetOnlyOfficeValidatedSession | None = None, ) -> tuple[Path, str, str]: self._ensure_ready() if asset_id == PREVIEW_RULE_ASSET_ID: - _, metadata = self._ensure_preview_rule_spreadsheet(version=version) + resolved_version, metadata = self._ensure_preview_rule_spreadsheet( + version=(validated_session.document_version if validated_session else version) + ) + if validated_session is not None: + self._require_matching_onlyoffice_document( + validated_session, + tenant_id="platform", + resource_scope="platform", + document_version=resolved_version, + metadata=metadata, + ) file_path = self.spreadsheet_manager.resolve_storage_path(metadata.storage_key) if not file_path.exists(): raise FileNotFoundError(metadata.file_name) @@ -87,9 +116,19 @@ class AgentAssetOnlyOfficeMixin: asset = self._require_spreadsheet_rule(asset_id) requested_version = str(version or "").strip() if requested_version and requested_version != "current": - _, metadata = self._resolve_spreadsheet_version_meta(asset, version=requested_version) + resolved_version, metadata = self._resolve_spreadsheet_version_meta( + asset, version=requested_version + ) else: - _, metadata = self._resolve_current_spreadsheet_meta(asset) + resolved_version, metadata = self._resolve_current_spreadsheet_meta(asset) + if validated_session is not None: + self._require_matching_onlyoffice_document( + validated_session, + tenant_id=asset.tenant_id, + resource_scope=asset.scope, + document_version=resolved_version, + metadata=metadata, + ) file_path = self.spreadsheet_manager.resolve_storage_path(metadata.storage_key) if not file_path.exists(): raise FileNotFoundError(metadata.file_name) @@ -99,22 +138,14 @@ class AgentAssetOnlyOfficeMixin: self, asset_id: str, access_token: str, - ) -> None: - onlyoffice_settings = self._resolve_onlyoffice_settings() - try: - payload = jwt.decode( - access_token, - onlyoffice_settings.jwt_secret, - algorithms=["HS256"], - ) - except jwt.PyJWTError as exc: - raise ValueError("ONLYOFFICE 文件访问令牌无效。") from exc - - if ( - payload.get("scope") != "agent-asset-spreadsheet" - or payload.get("asset_id") != asset_id - ): - raise ValueError("ONLYOFFICE 文件访问令牌无效。") + *, + expected_scope: str = AGENT_ASSET_ONLYOFFICE_CONTENT_SCOPE, + ) -> AgentAssetOnlyOfficeValidatedSession: + return self._onlyoffice_session_service().validate( + asset_id=asset_id, + token=access_token, + expected_scope=expected_scope, + ) def upload_rule_spreadsheet( self, @@ -210,42 +241,54 @@ class AgentAssetOnlyOfficeMixin: *, version: str | None = None, payload: dict[str, Any], - actor_name: str | None = None, + callback_token: str, ) -> None: self._ensure_ready() - if asset_id == PREVIEW_RULE_ASSET_ID: - self._handle_preview_rule_spreadsheet_onlyoffice_callback( - version=version, - payload=payload, + callback = self._parse_onlyoffice_callback(payload) + session_service = self._onlyoffice_session_service() + if callback.status not in {2, 6}: + session_service.validate( + asset_id=asset_id, + token=callback_token, + expected_scope=AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, ) return + if not callback.download_url: + raise ValueError("ONLYOFFICE 回写回调缺少下载 URL。") - asset = self._require_spreadsheet_rule(asset_id) - callback = self._parse_onlyoffice_callback(payload) - if callback.status not in {2, 6} or not callback.download_url: - return - - _, current_metadata = self._resolve_current_spreadsheet_meta(asset) - request = Request( - callback.download_url, - headers={"User-Agent": "x-financial-onlyoffice-agent-asset"}, - ) - with urlopen(request, timeout=30) as response: # noqa: S310 - content = response.read() - - if current_metadata.checksum and current_metadata.checksum == self._hash_bytes(content): - return - - resolved_actor_name = str(actor_name or "").strip() or ( - callback.users[0] if callback.users else "ONLYOFFICE" - ) - self.upload_rule_spreadsheet( - asset.id, - filename=current_metadata.file_name, - content=content, - actor=resolved_actor_name, - source="onlyoffice", + claimed = session_service.claim_callback( + asset_id=asset_id, + token=callback_token, + payload_document_key=callback.document_key, ) + if version and str(version).strip() not in {"current", claimed.document_version}: + session_service.finish_callback( + claimed.jti, + succeeded=False, + failure_reason="legacy_version_mismatch", + ) + raise AgentAssetOnlyOfficeSecurityError( + "ONLYOFFICE 回调版本与编辑会话不一致。" + ) + try: + if asset_id == PREVIEW_RULE_ASSET_ID: + self._save_preview_rule_spreadsheet_callback( + claimed=claimed, + download_url=callback.download_url, + ) + else: + self._save_current_rule_spreadsheet_callback( + claimed=claimed, + download_url=callback.download_url, + ) + except Exception as exc: + session_service.finish_callback( + claimed.jti, + succeeded=False, + failure_reason=type(exc).__name__, + ) + raise + session_service.finish_callback(claimed.jti, succeeded=True) @staticmethod @@ -265,28 +308,21 @@ class AgentAssetOnlyOfficeMixin: for character in raw_key ) - def _build_onlyoffice_access_token(self, asset_id: str) -> str: - onlyoffice_settings = self._resolve_onlyoffice_settings() - payload = { - "scope": "agent-asset-spreadsheet", - "asset_id": asset_id, - } - return jwt.encode(payload, onlyoffice_settings.jwt_secret, algorithm="HS256") - @staticmethod def _parse_onlyoffice_callback(payload: dict[str, Any]) -> OnlyOfficeCallbackPayload: return OnlyOfficeCallbackPayload( status=int(payload.get("status") or 0), download_url=str(payload.get("url") or "").strip(), - users=[str(item).strip() for item in payload.get("users") or [] if str(item).strip()], + document_key=str(payload.get("key") or "").strip(), ) - - def _build_onlyoffice_spreadsheet_config( self, *, asset_id: str, + tenant_id: str, + resource_scope: str, + document_version: str, current_user: CurrentUserContext, metadata: RuleSpreadsheetMeta, editable: bool, @@ -302,21 +338,32 @@ class AgentAssetOnlyOfficeMixin: backend_base_url = onlyoffice_settings.backend_url.rstrip("/") public_url = onlyoffice_settings.public_url.rstrip("/") - access_token = self._build_onlyoffice_access_token(asset_id) + actor = stable_user_principal(current_user) + document_key = self._build_onlyoffice_document_key(asset_id, metadata) + tokens = self._onlyoffice_session_service().issue( + tenant_id=tenant_id, + resource_scope=resource_scope, + asset_id=asset_id, + document_key=document_key, + document_version=document_version, + document_fingerprint=self._onlyoffice_document_fingerprint(metadata), + actor=actor, + writable=editable, + ) document_url = ( f"{backend_base_url}{settings.api_v1_prefix}/agent-assets/{asset_id}/spreadsheet/onlyoffice/content" - f"?access_token={access_token}" + f"?access_token={tokens.content_token}" ) callback_url = ( f"{backend_base_url}{settings.api_v1_prefix}/agent-assets/{asset_id}/spreadsheet/onlyoffice/callback" - f"?actor_name={quote(current_user.name)}" + f"?access_token={tokens.callback_token}" ) config: dict[str, Any] = { "documentType": "cell", "document": { "fileType": Path(metadata.file_name).suffix.lstrip(".").lower() or "xlsx", - "key": self._build_onlyoffice_document_key(asset_id, metadata), + "key": document_key, "title": metadata.file_name, "url": document_url, "permissions": { @@ -396,38 +443,101 @@ class AgentAssetOnlyOfficeMixin: ) return resolved_version, metadata - def _handle_preview_rule_spreadsheet_onlyoffice_callback( + def _save_current_rule_spreadsheet_callback( self, *, - version: str, - payload: dict[str, Any], + claimed: AgentAssetOnlyOfficeValidatedSession, + download_url: str, ) -> None: - callback = self._parse_onlyoffice_callback(payload) - if callback.status not in {2, 6} or not callback.download_url: - return - - resolved_version, metadata = self._ensure_preview_rule_spreadsheet(version=version) - request = Request( - callback.download_url, - headers={"User-Agent": "x-financial-onlyoffice-agent-asset-preview"}, + asset = self._require_spreadsheet_rule(claimed.asset_id) + resolved_version, metadata = self._resolve_current_spreadsheet_meta(asset) + self._require_matching_onlyoffice_document( + claimed, + tenant_id=asset.tenant_id, + resource_scope=asset.scope, + document_version=resolved_version, + metadata=metadata, + ) + content = download_onlyoffice_document( + download_url, + expected_filename=metadata.file_name, ) - with urlopen(request, timeout=30) as response: # noqa: S310 - content = response.read() - if metadata.checksum and metadata.checksum == self._hash_bytes(content): return + self.upload_rule_spreadsheet( + asset.id, + filename=metadata.file_name, + content=content, + actor=claimed.actor, + source="onlyoffice", + ) - actor_name = callback.users[0] if callback.users else "ONLYOFFICE" + def _save_preview_rule_spreadsheet_callback( + self, + *, + claimed: AgentAssetOnlyOfficeValidatedSession, + download_url: str, + ) -> None: + resolved_version, metadata = self._ensure_preview_rule_spreadsheet( + version=claimed.document_version + ) + self._require_matching_onlyoffice_document( + claimed, + tenant_id="platform", + resource_scope="platform", + document_version=resolved_version, + metadata=metadata, + ) + content = download_onlyoffice_document( + download_url, + expected_filename=metadata.file_name, + ) + if metadata.checksum and metadata.checksum == self._hash_bytes(content): + return self.spreadsheet_manager.store_rule_library_spreadsheet_snapshot( library=FINANCE_RULES_LIBRARY, asset_id=PREVIEW_RULE_ASSET_ID, version=resolved_version, file_name=metadata.file_name, content=content, - actor_name=actor_name, + actor_name=claimed.actor, source="onlyoffice-preview", ) + def _onlyoffice_session_service(self) -> AgentAssetOnlyOfficeSessionService: + settings = self._resolve_onlyoffice_settings() + return AgentAssetOnlyOfficeSessionService( + self.db, + jwt_secret=settings.jwt_secret, + ) + + @staticmethod + def _onlyoffice_document_fingerprint(metadata: RuleSpreadsheetMeta) -> str: + return str(metadata.checksum or metadata.updated_at or metadata.file_name) + + def _require_matching_onlyoffice_document( + self, + session: AgentAssetOnlyOfficeValidatedSession, + *, + tenant_id: str, + resource_scope: str, + document_version: str, + metadata: RuleSpreadsheetMeta, + ) -> None: + comparisons = ( + session.tenant_id == tenant_id, + session.resource_scope == resource_scope, + session.document_version == document_version, + session.document_key + == self._build_onlyoffice_document_key(session.asset_id, metadata), + session.document_fingerprint + == self._onlyoffice_document_fingerprint(metadata), + ) + if not all(comparisons): + raise AgentAssetOnlyOfficeSecurityError( + "ONLYOFFICE 编辑会话绑定的规则表版本已失效。" + ) + @staticmethod def _read_current_rule_document_meta(asset: AgentAsset) -> RuleSpreadsheetMeta | None: payload = (asset.config_json or {}).get("rule_document") diff --git a/server/src/app/services/agent_asset_onlyoffice_security.py b/server/src/app/services/agent_asset_onlyoffice_security.py new file mode 100644 index 0000000..883d269 --- /dev/null +++ b/server/src/app/services/agent_asset_onlyoffice_security.py @@ -0,0 +1,288 @@ +from __future__ import annotations + +from dataclasses import dataclass +from datetime import UTC, datetime, timedelta +from typing import Any +from uuid import uuid4 + +import jwt +from sqlalchemy import update +from sqlalchemy.orm import Session + +from app.models.agent_asset import AgentAssetOnlyOfficeSession + +AGENT_ASSET_ONLYOFFICE_ISSUER = "x-financial" +AGENT_ASSET_ONLYOFFICE_AUDIENCE = "onlyoffice-document-server" +AGENT_ASSET_ONLYOFFICE_CONTENT_SCOPE = "agent-asset-spreadsheet-content" +AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE = "agent-asset-spreadsheet-callback" +AGENT_ASSET_ONLYOFFICE_CONTENT_TTL_SECONDS = 900 +AGENT_ASSET_ONLYOFFICE_CALLBACK_TTL_SECONDS = 4 * 60 * 60 + + +class AgentAssetOnlyOfficeSecurityError(ValueError): + pass + + +class AgentAssetOnlyOfficeReplayError(AgentAssetOnlyOfficeSecurityError): + pass + + +@dataclass(frozen=True, slots=True) +class AgentAssetOnlyOfficeTokens: + content_token: str + callback_token: str + expires_at: datetime + + +@dataclass(frozen=True, slots=True) +class AgentAssetOnlyOfficeValidatedSession: + jti: str + tenant_id: str + resource_scope: str + asset_id: str + document_key: str + document_version: str + document_fingerprint: str + writable: bool + actor: str + status: str + + +class AgentAssetOnlyOfficeSessionService: + def __init__(self, db: Session, *, jwt_secret: str) -> None: + self.db = db + self.jwt_secret = str(jwt_secret or "").strip() + if not self.jwt_secret: + raise AgentAssetOnlyOfficeSecurityError("ONLYOFFICE JWT 密钥未配置。") + + def issue( + self, + *, + tenant_id: str, + resource_scope: str, + asset_id: str, + document_key: str, + document_version: str, + document_fingerprint: str, + writable: bool, + actor: str, + ) -> AgentAssetOnlyOfficeTokens: + normalized_tenant_id = str(tenant_id or "").strip() + normalized_scope = str(resource_scope or "").strip() + if not normalized_tenant_id or normalized_scope not in {"platform", "tenant"}: + raise AgentAssetOnlyOfficeSecurityError("ONLYOFFICE 会话租户范围不合法。") + if (normalized_scope == "platform") != (normalized_tenant_id == "platform"): + raise AgentAssetOnlyOfficeSecurityError("ONLYOFFICE 会话租户范围不一致。") + + now = datetime.now(UTC) + content_expires_at = now + timedelta( + seconds=AGENT_ASSET_ONLYOFFICE_CONTENT_TTL_SECONDS + ) + expires_at = now + timedelta( + seconds=AGENT_ASSET_ONLYOFFICE_CALLBACK_TTL_SECONDS + ) + jti = str(uuid4()) + row = AgentAssetOnlyOfficeSession( + jti=jti, + tenant_id=normalized_tenant_id, + resource_scope=normalized_scope, + asset_id=str(asset_id), + document_key=str(document_key), + document_version=str(document_version), + document_fingerprint=str(document_fingerprint), + audience=AGENT_ASSET_ONLYOFFICE_AUDIENCE, + writable=bool(writable), + status="active", + actor=str(actor), + expires_at=expires_at, + failure_reason="", + ) + self.db.add(row) + self.db.commit() + + common_claims = { + "iss": AGENT_ASSET_ONLYOFFICE_ISSUER, + "aud": AGENT_ASSET_ONLYOFFICE_AUDIENCE, + "sub": str(asset_id), + "jti": jti, + "iat": int(now.timestamp()), + "nbf": int(now.timestamp()), + "tenant_id": normalized_tenant_id, + "resource_scope": normalized_scope, + "asset_id": str(asset_id), + "document_key": str(document_key), + "document_version": str(document_version), + "document_fingerprint": str(document_fingerprint), + "writable": bool(writable), + "actor": str(actor), + } + return AgentAssetOnlyOfficeTokens( + content_token=self._encode( + { + **common_claims, + "scope": AGENT_ASSET_ONLYOFFICE_CONTENT_SCOPE, + "exp": int(content_expires_at.timestamp()), + } + ), + callback_token=self._encode( + { + **common_claims, + "scope": AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + "exp": int(expires_at.timestamp()), + } + ), + expires_at=expires_at, + ) + + def validate( + self, + *, + asset_id: str, + token: str, + expected_scope: str, + ) -> AgentAssetOnlyOfficeValidatedSession: + try: + claims = jwt.decode( + token, + self.jwt_secret, + algorithms=["HS256"], + audience=AGENT_ASSET_ONLYOFFICE_AUDIENCE, + issuer=AGENT_ASSET_ONLYOFFICE_ISSUER, + options={ + "require": [ + "iss", + "aud", + "sub", + "jti", + "iat", + "nbf", + "exp", + "scope", + "tenant_id", + "resource_scope", + "asset_id", + "document_key", + "document_version", + "document_fingerprint", + "writable", + "actor", + ] + }, + ) + except jwt.PyJWTError as exc: + raise AgentAssetOnlyOfficeSecurityError( + "ONLYOFFICE 会话令牌无效或已过期。" + ) from exc + + jti = str(claims.get("jti") or "").strip() + row = self.db.get(AgentAssetOnlyOfficeSession, jti) + if row is None: + raise AgentAssetOnlyOfficeSecurityError("ONLYOFFICE 会话不存在。") + now = datetime.now(UTC) + comparisons = ( + claims.get("scope") == expected_scope, + claims.get("sub") == asset_id, + claims.get("asset_id") == asset_id == row.asset_id, + claims.get("tenant_id") == row.tenant_id, + claims.get("resource_scope") == row.resource_scope, + claims.get("document_key") == row.document_key, + claims.get("document_version") == row.document_version, + claims.get("document_fingerprint") == row.document_fingerprint, + bool(claims.get("writable")) == row.writable, + claims.get("actor") == row.actor, + row.audience == AGENT_ASSET_ONLYOFFICE_AUDIENCE, + row.status == "active", + _as_utc(row.expires_at) > now, + ) + if not all(comparisons): + if row.status != "active": + raise AgentAssetOnlyOfficeReplayError( + "ONLYOFFICE 回调会话已被使用或已撤销。" + ) + raise AgentAssetOnlyOfficeSecurityError( + "ONLYOFFICE 会话与目标规则表不匹配。" + ) + return self._validated(row) + + def claim_callback( + self, + *, + asset_id: str, + token: str, + payload_document_key: str, + ) -> AgentAssetOnlyOfficeValidatedSession: + validated = self.validate( + asset_id=asset_id, + token=token, + expected_scope=AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + ) + if not validated.writable: + raise AgentAssetOnlyOfficeSecurityError("只读 ONLYOFFICE 会话禁止回写规则表。") + if not payload_document_key or payload_document_key != validated.document_key: + raise AgentAssetOnlyOfficeSecurityError( + "ONLYOFFICE 回调文档 key 与编辑会话不一致。" + ) + + now = datetime.now(UTC) + result = self.db.execute( + update(AgentAssetOnlyOfficeSession) + .where( + AgentAssetOnlyOfficeSession.jti == validated.jti, + AgentAssetOnlyOfficeSession.status == "active", + AgentAssetOnlyOfficeSession.expires_at > now, + ) + .values(status="processing", claimed_at=now) + ) + if result.rowcount != 1: + self.db.rollback() + raise AgentAssetOnlyOfficeReplayError( + "ONLYOFFICE 回调会话已被使用或已过期。" + ) + self.db.commit() + row = self.db.get(AgentAssetOnlyOfficeSession, validated.jti) + if row is None: # pragma: no cover - 数据库约束保证 + raise AgentAssetOnlyOfficeSecurityError("ONLYOFFICE 会话不存在。") + return self._validated(row) + + def finish_callback( + self, + jti: str, + *, + succeeded: bool, + failure_reason: str = "", + ) -> None: + row = self.db.get(AgentAssetOnlyOfficeSession, jti) + if row is None or row.status != "processing": + raise AgentAssetOnlyOfficeReplayError("ONLYOFFICE 回调会话状态不可更新。") + if succeeded: + row.status = "consumed" + row.consumed_at = datetime.now(UTC) + row.failure_reason = "" + else: + row.status = "failed" + row.failure_reason = str(failure_reason or "callback_failed")[:1000] + self.db.commit() + + def _encode(self, claims: dict[str, Any]) -> str: + return jwt.encode(claims, self.jwt_secret, algorithm="HS256") + + @staticmethod + def _validated(row: AgentAssetOnlyOfficeSession) -> AgentAssetOnlyOfficeValidatedSession: + return AgentAssetOnlyOfficeValidatedSession( + jti=row.jti, + tenant_id=row.tenant_id, + resource_scope=row.resource_scope, + asset_id=row.asset_id, + document_key=row.document_key, + document_version=row.document_version, + document_fingerprint=row.document_fingerprint, + writable=row.writable, + actor=row.actor, + status=row.status, + ) + + +def _as_utc(value: datetime) -> datetime: + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value.astimezone(UTC) diff --git a/server/src/app/services/agent_asset_release_aggregation.py b/server/src/app/services/agent_asset_release_aggregation.py new file mode 100644 index 0000000..b7f4f55 --- /dev/null +++ b/server/src/app/services/agent_asset_release_aggregation.py @@ -0,0 +1,451 @@ +"""发布遥测的精度、负样本证据与召回率聚合。""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) +from app.services.agent_asset_release_label_votes import ( + release_reviewer_quorum, + resolve_release_label_votes, +) +from app.services.agent_asset_release_recall import ( + ReleaseRecallEstimate, + estimate_release_recall, +) +from app.services.agent_asset_release_telemetry_values import precision + + +@dataclass(frozen=True, slots=True) +class _RecallEvidence: + status: str + estimate: ReleaseRecallEstimate + false_negative_count: int | None + negative_sample_count: int + negative_labeled_count: int + negative_pending_count: int + random_population_count: int + random_sample_count: int + random_labeled_count: int + random_false_negative_count: int + + +def build_release_telemetry_aggregate( + *, + db: Session, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + version: str, + release_state: dict[str, Any], +) -> Any: + from app.services.agent_asset_release_telemetry import ReleaseTelemetryAggregate + + observations = _observations(db, tenant_id, asset_id, release_id, stage, version) + labels = _labels(db, tenant_id, asset_id, release_id, stage, version) + samples = _samples(db, tenant_id, asset_id, release_id, stage, version) + vote_states = resolve_release_label_votes( + labels, + required_reviewers=release_reviewer_quorum(release_state), + required_reviewers_by_observation={ + item.observation_id: _sample_quorum(item, release_state) + for item in samples + }, + ) + latest_labels = { + observation_id: state.label + for observation_id, state in vote_states.items() + if state.label is not None + } + completed = [item for item in observations if item.runtime_status == "completed"] + failures = [item for item in observations if item.runtime_status == "failed"] + candidate_hits = [item for item in completed if item.candidate_hit] + baseline_hits = [item for item in completed if item.baseline_hit is True] + candidate_values = _values(candidate_hits, latest_labels) + baseline_values = _values(baseline_hits, latest_labels) + candidate_confirmed = candidate_values.count("confirmed") + candidate_false_positive = candidate_values.count("false_positive") + baseline_confirmed = baseline_values.count("confirmed") + baseline_false_positive = baseline_values.count("false_positive") + candidate_pending = len(candidate_hits) - len(candidate_values) + baseline_pending = len(baseline_hits) - len(baseline_values) + candidate_precision = precision(candidate_confirmed, candidate_false_positive) + baseline_precision = ( + precision(baseline_confirmed, baseline_false_positive) + if not baseline_pending + else None + ) + recall = _recall_evidence( + completed=completed, + samples=samples, + labels=latest_labels, + true_positive_count=candidate_confirmed, + candidate_pending_count=candidate_pending, + release_state=release_state, + ) + reasons = _reasons( + observations=observations, + candidate_hits=candidate_hits, + candidate_values=candidate_values, + candidate_pending=candidate_pending, + baseline_precision=baseline_precision, + recall=recall, + release_state=release_state, + ) + precision_ready = bool(candidate_hits and candidate_values and not candidate_pending) + recall_gate = _recall_gate_enabled(release_state) + recall_ready = recall.status.startswith("available") + min_precision = _policy_float(release_state, "min_precision", 0.98) + hard_precision_failure = ( + candidate_precision is not None and candidate_precision < min_precision + ) + ready = bool( + observations + and ( + failures + or ( + precision_ready + and (not recall_gate or recall_ready or hard_precision_failure) + ) + ) + ) + estimate = recall.estimate + return ReleaseTelemetryAggregate( + tenant_id=tenant_id, + asset_id=asset_id, + release_id=release_id, + stage=stage, + version=version, + status="ready" if ready else "collecting", + reasons=tuple(reasons), + observed_count=len(observations), + completed_count=len(completed), + runtime_failure_count=len(failures), + candidate_hit_count=len(candidate_hits), + candidate_labeled_count=len(candidate_values), + candidate_pending_label_count=candidate_pending, + candidate_confirmed_count=candidate_confirmed, + candidate_false_positive_count=candidate_false_positive, + precision=candidate_precision, + baseline_hit_count=len(baseline_hits), + baseline_labeled_count=len(baseline_values), + baseline_pending_label_count=baseline_pending, + baseline_confirmed_count=baseline_confirmed, + baseline_false_positive_count=baseline_false_positive, + baseline_precision=baseline_precision, + false_negative_count=recall.false_negative_count, + estimated_false_negative_count=estimate.estimated_false_negative_count, + false_negative_upper_bound=estimate.false_negative_upper_bound, + negative_sample_count=recall.negative_sample_count, + negative_labeled_count=recall.negative_labeled_count, + negative_pending_label_count=recall.negative_pending_count, + random_negative_population_count=recall.random_population_count, + random_negative_sample_count=recall.random_sample_count, + random_negative_labeled_count=recall.random_labeled_count, + random_negative_false_negative_count=recall.random_false_negative_count, + recall=estimate.recall, + recall_lower_bound=estimate.recall_lower_bound, + recall_confidence_level=estimate.confidence_level, + recall_method=estimate.method, + negative_ground_truth_status=recall.status, + ) + + +def _recall_evidence( + *, + completed: list[AgentAssetReleaseObservation], + samples: list[AgentAssetReleaseAuditSample], + labels: dict[str, str], + true_positive_count: int, + candidate_pending_count: int, + release_state: dict[str, Any], +) -> _RecallEvidence: + by_id = {item.id: item for item in completed} + negative_samples = [ + item + for item in samples + if item.observation_id in by_id and not by_id[item.observation_id].candidate_hit + ] + negative_labeled = [item for item in negative_samples if item.observation_id in labels] + disagreement = [ + item + for item in completed + if not item.candidate_hit and item.baseline_hit is True + ] + random_population = [ + item + for item in completed + if not item.candidate_hit and item.baseline_hit is not True + ] + random_samples = [ + item for item in negative_samples if item.stratum == "candidate_negative_random" + ] + random_labeled = [item for item in random_samples if item.observation_id in labels] + disagreement_false_negatives = sum( + 1 for item in disagreement if labels.get(item.id) == "confirmed" + ) + random_false_negatives = sum( + 1 for item in random_labeled if labels[item.observation_id] == "confirmed" + ) + observed_false_negatives = sum( + 1 for item in negative_labeled if labels[item.observation_id] == "confirmed" + ) + status = _negative_status( + candidate_pending_count=candidate_pending_count, + disagreement=disagreement, + random_population=random_population, + random_samples=random_samples, + random_labeled=random_labeled, + labels=labels, + minimum_random_reviews=_policy_int( + release_state, + "negative_min_reviewed", + 5, + low=1, + high=10_000, + ), + ) + confidence = _policy_confidence(release_state) + estimate = _estimate_if_ready( + status=status, + true_positive_count=true_positive_count, + disagreement_false_negatives=disagreement_false_negatives, + random_population_count=len(random_population), + random_reviewed_count=len(random_labeled), + random_false_negative_count=random_false_negatives, + confidence=confidence, + ) + return _RecallEvidence( + status=status, + estimate=estimate, + false_negative_count=( + observed_false_negatives if status.startswith("available") else None + ), + negative_sample_count=len(negative_samples), + negative_labeled_count=len(negative_labeled), + negative_pending_count=len(negative_samples) - len(negative_labeled), + random_population_count=len(random_population), + random_sample_count=len(random_samples), + random_labeled_count=len(random_labeled), + random_false_negative_count=random_false_negatives, + ) + + +def _negative_status( + *, + candidate_pending_count: int, + disagreement: list[AgentAssetReleaseObservation], + random_population: list[AgentAssetReleaseObservation], + random_samples: list[AgentAssetReleaseAuditSample], + random_labeled: list[AgentAssetReleaseAuditSample], + labels: dict[str, str], + minimum_random_reviews: int, +) -> str: + if candidate_pending_count: + return "collecting_candidate_labels" + if any(item.id not in labels for item in disagreement): + return "collecting_disagreement_labels" + if not random_population: + return "available_census" + if not random_samples: + return "unavailable_no_random_negative_samples" + if len(random_labeled) < len(random_samples): + return "collecting_random_negative_labels" + if len(random_labeled) < minimum_random_reviews: + return "insufficient_random_negative_reviews" + return "available_stratified_random_audit" + + +def _estimate_if_ready( + *, + status: str, + true_positive_count: int, + disagreement_false_negatives: int, + random_population_count: int, + random_reviewed_count: int, + random_false_negative_count: int, + confidence: float, +) -> ReleaseRecallEstimate: + if not status.startswith("available"): + return estimate_release_recall( + true_positive_count=true_positive_count, + disagreement_false_negative_count=disagreement_false_negatives, + random_negative_population_count=max(1, random_population_count), + random_reviewed_count=0, + random_false_negative_count=0, + confidence_level=confidence, + ) + return estimate_release_recall( + true_positive_count=true_positive_count, + disagreement_false_negative_count=disagreement_false_negatives, + random_negative_population_count=random_population_count, + random_reviewed_count=random_reviewed_count, + random_false_negative_count=random_false_negative_count, + confidence_level=confidence, + ) + + +def _reasons( + *, + observations: list[AgentAssetReleaseObservation], + candidate_hits: list[AgentAssetReleaseObservation], + candidate_values: list[str], + candidate_pending: int, + baseline_precision: float | None, + recall: _RecallEvidence, + release_state: dict[str, Any], +) -> list[str]: + reasons: list[str] = [] + if not observations: + reasons.append("no_runtime_observations") + if not candidate_hits: + reasons.append("no_candidate_positive_samples") + if not candidate_values: + reasons.append("no_candidate_labels") + if candidate_pending: + reasons.append("candidate_labels_pending") + if baseline_precision is None: + reasons.append("baseline_precision_unavailable") + if _recall_gate_enabled(release_state) and not recall.status.startswith("available"): + reasons.append(recall.status) + return reasons + + +def _observations( + db: Session, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + version: str, +) -> list[AgentAssetReleaseObservation]: + return list( + db.scalars( + select(AgentAssetReleaseObservation) + .where( + AgentAssetReleaseObservation.tenant_id == tenant_id, + AgentAssetReleaseObservation.asset_id == asset_id, + AgentAssetReleaseObservation.release_id == release_id, + AgentAssetReleaseObservation.stage == stage, + AgentAssetReleaseObservation.version == version, + ) + .order_by( + AgentAssetReleaseObservation.created_at.asc(), + AgentAssetReleaseObservation.id.asc(), + ) + ).all() + ) + + +def _labels( + db: Session, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + version: str, +) -> list[AgentAssetReleaseLabel]: + return list( + db.scalars( + select(AgentAssetReleaseLabel) + .where( + AgentAssetReleaseLabel.tenant_id == tenant_id, + AgentAssetReleaseLabel.asset_id == asset_id, + AgentAssetReleaseLabel.release_id == release_id, + AgentAssetReleaseLabel.stage == stage, + AgentAssetReleaseLabel.version == version, + ) + .order_by( + AgentAssetReleaseLabel.created_at.asc(), + AgentAssetReleaseLabel.id.asc(), + ) + ).all() + ) + + +def _samples( + db: Session, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + version: str, +) -> list[AgentAssetReleaseAuditSample]: + return list( + db.scalars( + select(AgentAssetReleaseAuditSample) + .where( + AgentAssetReleaseAuditSample.tenant_id == tenant_id, + AgentAssetReleaseAuditSample.asset_id == asset_id, + AgentAssetReleaseAuditSample.release_id == release_id, + AgentAssetReleaseAuditSample.stage == stage, + AgentAssetReleaseAuditSample.version == version, + ) + .order_by( + AgentAssetReleaseAuditSample.created_at.asc(), + AgentAssetReleaseAuditSample.id.asc(), + ) + ).all() + ) + + +def _values( + observations: list[AgentAssetReleaseObservation], + labels: dict[str, str], +) -> list[str]: + return [labels[item.id] for item in observations if item.id in labels] + + +def _recall_gate_enabled(release_state: dict[str, Any]) -> bool: + policy = release_state.get("policy") + return isinstance(policy, dict) and policy.get("recall_gate_enabled") is True + + +def _sample_quorum( + sample: AgentAssetReleaseAuditSample, + release_state: dict[str, Any], +) -> int: + if sample.stratum != "candidate_positive_census": + return 2 + return release_reviewer_quorum(release_state) + + +def _policy_int( + release_state: dict[str, Any], + key: str, + default: int, + *, + low: int, + high: int, +) -> int: + policy = release_state.get("policy") + source = policy if isinstance(policy, dict) else {} + try: + parsed = int(source.get(key, default)) + except (TypeError, ValueError, OverflowError): + return default + return max(low, min(high, parsed)) + + +def _policy_float(release_state: dict[str, Any], key: str, default: float) -> float: + policy = release_state.get("policy") + source = policy if isinstance(policy, dict) else {} + try: + parsed = float(source.get(key, default)) + except (TypeError, ValueError, OverflowError): + return default + return max(0.0, min(1.0, parsed)) + + +def _policy_confidence(release_state: dict[str, Any]) -> float: + value = _policy_float(release_state, "recall_confidence_level", 0.95) + return value if value in {0.9, 0.95, 0.99} else 0.95 diff --git a/server/src/app/services/agent_asset_release_alerts.py b/server/src/app/services/agent_asset_release_alerts.py new file mode 100644 index 0000000..e94a259 --- /dev/null +++ b/server/src/app/services/agent_asset_release_alerts.py @@ -0,0 +1,129 @@ +"""把发布遥测状态转换为不含业务正文的运营告警。""" + +from __future__ import annotations + +from typing import Any + + +def build_release_alerts( + *, + status: str, + rolled_back: bool, + reasons: list[str] | tuple[str, ...], + metrics: dict[str, Any], +) -> list[dict[str, str]]: + alerts: list[dict[str, str]] = [] + reason_set = {str(item) for item in reasons} + if bool(metrics.get("aggregation_failed")): + alerts.append( + _alert( + "release_aggregation_failed", + "error", + "发布遥测聚合失败，候选版本已停止晋级。", + "检查遥测存储和聚合作业；稳定版本会继续保护业务。", + ) + ) + if rolled_back: + alerts.append( + _alert( + "release_auto_rolled_back", + "critical", + "候选版本已因真实运行指标越界自动回滚。", + "检查失败样本和版本差异后重新发布。", + ) + ) + if int(metrics.get("runtime_failure_count") or 0) > 0: + alerts.append( + _alert( + "runtime_failures_detected", + "error", + "候选规则出现结构化运行失败。", + "优先修复 evaluator 或发布快照完整性。", + ) + ) + if int(metrics.get("candidate_pending_label_count") or 0) > 0: + alerts.append( + _alert( + "release_labels_pending", + "warning", + "候选命中仍有待人工复核样本，发布不会晋级。", + "由非发布发起人完成确认或误报标注。", + ) + ) + if int(metrics.get("candidate_oldest_pending_age_seconds") or 0) >= 86_400: + alerts.append( + _alert( + "release_labels_overdue", + "error", + "最早待复核样本已超过 24 小时。", + "安排独立复核人处理积压；完成前保持候选版本不晋级。", + ) + ) + if int(metrics.get("negative_pending_label_count") or 0) > 0: + alerts.append( + _alert( + "release_negative_audit_pending", + "warning", + "独立负样本盲审仍有积压，召回率证据尚未闭合。", + "完成抽样单据核验；复核界面不会展示候选规则的原始结论。", + ) + ) + negative_status = str(metrics.get("negative_ground_truth_status") or "") + if negative_status in { + "unavailable_no_random_negative_samples", + "insufficient_random_negative_reviews", + }: + alerts.append( + _alert( + "release_negative_audit_insufficient", + "info", + "随机负样本数量尚不足以形成可信召回率下界。", + "继续采集真实流量并完成系统选中的盲审样本。", + ) + ) + if "precision_below_threshold" in reason_set or "precision_regression_exceeded" in reason_set: + alerts.append( + _alert( + "release_precision_degraded", + "error", + "候选规则精度低于门禁或相对基线明显下降。", + "检查误报样本、规则条件和基线版本。", + ) + ) + if "recall_lower_bound_below_threshold" in reason_set: + alerts.append( + _alert( + "release_recall_degraded", + "critical", + "候选规则的召回率保守下界低于发布门禁。", + "检查已确认漏检样本，修正规则后重新进入影子发布。", + ) + ) + if metrics.get("baseline_precision") is None: + alerts.append( + _alert( + "baseline_precision_unavailable", + "info", + "当前没有完整的基线精度证据。", + "继续采集基线命中并完成可信标注。", + ) + ) + if status == "collecting" and int(metrics.get("observed_count") or 0) == 0: + alerts.append( + _alert( + "release_observations_pending", + "info", + "当前发布阶段尚未采集到真实运行样本。", + "保持稳定版本并等待真实流量覆盖。", + ) + ) + return alerts + + +def _alert(code: str, severity: str, message: str, action: str) -> dict[str, str]: + return { + "code": code, + "severity": severity, + "message": message, + "recommended_action": action, + } diff --git a/server/src/app/services/agent_asset_release_artifacts.py b/server/src/app/services/agent_asset_release_artifacts.py new file mode 100644 index 0000000..30071b2 --- /dev/null +++ b/server/src/app/services/agent_asset_release_artifacts.py @@ -0,0 +1,278 @@ +"""风险规则分阶段发布使用的不可变运行快照。""" + +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from datetime import UTC, datetime +from typing import Any + +from app.core.agent_enums import AgentAssetType +from app.models.agent_asset import AgentAsset +from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager +from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY +from app.services.risk_rule_manifest_normalizer import normalize_risk_rule_manifest + + +@dataclass(frozen=True) +class ReleaseArtifactBundle: + artifacts: dict[str, dict[str, Any]] + previous_config: dict[str, Any] + + +class AgentAssetReleaseArtifactService: + """读取并冻结候选与基线规则，避免发布期间文件被原地改写。""" + + _PROHIBITED_AUTOMATION_ACTIONS = { + "approve", + "auto_approve", + "auto_reject", + "execute_payment", + "pay", + "reject", + "transfer", + } + + def __init__( + self, + rule_library_manager: AgentAssetRuleLibraryManager | None = None, + ) -> None: + self.rule_library_manager = rule_library_manager or AgentAssetRuleLibraryManager() + + def capture( + self, + asset: AgentAsset, + candidate_version: str, + *, + previous_state: dict[str, Any] | None = None, + ) -> ReleaseArtifactBundle: + config = dict(asset.config_json or {}) + config.pop("release_guard", None) + if asset.asset_type != AgentAssetType.RULE.value: + return ReleaseArtifactBundle(artifacts={}, previous_config=config) + if str(config.get("detail_mode") or "").strip().lower() != "json_risk": + raise ValueError("分阶段发布当前只支持 JSON 风险规则资产。") + + library = str(config.get("rule_library") or RISK_RULES_LIBRARY).strip() + previous_version = str(asset.published_version or "").strip() + artifacts: dict[str, dict[str, Any]] = {} + if previous_version: + previous_document = config.get("rule_document") + artifact = self._capture_document( + asset, + version=previous_version, + library=library, + document=previous_document, + allow_disabled=False, + ) + artifacts[previous_version] = artifact + + candidate_document = self._candidate_document( + config, + candidate_version, + has_published_version=bool(previous_version), + ) + if candidate_document is None: + prior_artifacts = ( + previous_state.get("artifacts") + if isinstance(previous_state, dict) + and isinstance(previous_state.get("artifacts"), dict) + else {} + ) + prior_candidate = prior_artifacts.get(candidate_version) + if isinstance(prior_candidate, dict): + artifacts[candidate_version] = dict(prior_candidate) + else: + raise ValueError("候选版本缺少可执行规则快照，不能进入影子发布。") + else: + artifacts[candidate_version] = self._capture_document( + asset, + version=candidate_version, + library=library, + document=candidate_document, + allow_disabled=True, + ) + return ReleaseArtifactBundle(artifacts=artifacts, previous_config=config) + + def apply_candidate( + self, + asset: AgentAsset, + state: dict[str, Any], + *, + actor: str, + ) -> None: + candidate = str(state.get("candidate_version") or "").strip() + artifact = self.artifact(state, candidate) + manifest = dict(artifact["manifest"]) + config = dict(asset.config_json or {}) + config.update(self._runtime_config(manifest, artifact["rule_document"])) + revision = config.get("revision_draft") + if isinstance(revision, dict): + previous_config = state.get("previous_config") + previous_document = ( + previous_config.get("rule_document") + if isinstance(previous_config, dict) + and isinstance(previous_config.get("rule_document"), dict) + else {} + ) + history = list( + config.get("revision_history") + if isinstance(config.get("revision_history"), list) + else [] + ) + history.insert( + 0, + { + "version": candidate, + "base_version": revision.get("base_version"), + "change_reason": revision.get("change_reason"), + "published_by": actor, + "published_at": datetime.now(UTC).isoformat(), + "previous_rule_document": previous_document, + "rule_document": dict(artifact["rule_document"]), + }, + ) + config["revision_history"] = history[:20] + config.pop("revision_draft", None) + config["last_operation"] = { + "action": "activate_staged_release", + "actor": actor, + "at": datetime.now(UTC).isoformat(), + "target_version": candidate, + } + asset.name = str(manifest.get("name") or asset.name) + asset.description = str(manifest.get("description") or asset.description) + risk_category = str(manifest.get("risk_category") or "").strip() + if risk_category: + asset.scenario_json = [risk_category] + asset.config_json = config + + @staticmethod + def restore_previous(asset: AgentAsset, state: dict[str, Any]) -> None: + previous_config = state.get("previous_config") + if not isinstance(previous_config, dict): + raise ValueError("发布状态缺少基线配置快照，不能安全回滚。") + release_state = dict(state) + restored = dict(previous_config) + restored["release_guard"] = release_state + asset.config_json = restored + + @staticmethod + def artifact(state: dict[str, Any], version: str) -> dict[str, Any]: + artifacts = state.get("artifacts") + artifact = artifacts.get(version) if isinstance(artifacts, dict) else None + if not isinstance(artifact, dict) or not isinstance(artifact.get("manifest"), dict): + raise ValueError(f"发布版本 {version or ''} 缺少可信运行快照。") + manifest = artifact["manifest"] + expected = str(artifact.get("sha256") or "").strip() + if not expected or expected != _manifest_hash(manifest): + raise ValueError(f"发布版本 {version} 的运行快照完整性校验失败。") + return dict(artifact) + + def _capture_document( + self, + asset: AgentAsset, + *, + version: str, + library: str, + document: Any, + allow_disabled: bool, + ) -> dict[str, Any]: + if not isinstance(document, dict): + raise ValueError(f"规则版本 {version} 缺少 rule_document。") + file_name = str(document.get("file_name") or "").strip() + if not file_name: + raise ValueError(f"规则版本 {version} 缺少 JSON 文件名。") + manifest = normalize_risk_rule_manifest( + self.rule_library_manager.read_rule_library_json( + library=library, + file_name=file_name, + ) + ) + rule_code = str(manifest.get("rule_code") or "").strip() + if not rule_code or rule_code != str(asset.code or "").strip(): + raise ValueError("规则快照的 rule_code 与资产编码不一致。") + if manifest.get("enabled") is False and not allow_disabled: + raise PermissionError("当前基线规则已停用，不能作为分阶段发布的主版本。") + self._assert_no_prohibited_automation(manifest) + frozen = json.loads(json.dumps(manifest, ensure_ascii=False, sort_keys=True, default=str)) + return { + "version": version, + "rule_library": library, + "rule_document": dict(document), + "manifest": frozen, + "sha256": _manifest_hash(frozen), + } + + @staticmethod + def _candidate_document( + config: dict[str, Any], + version: str, + *, + has_published_version: bool, + ) -> dict[str, Any] | None: + revision = config.get("revision_draft") + if isinstance(revision, dict) and str(revision.get("version") or "").strip() == version: + document = revision.get("rule_document") + if isinstance(document, dict): + return dict(document) + document = config.get("rule_document") + if isinstance(document, dict) and not has_published_version: + return dict(document) + return None + + def _assert_no_prohibited_automation(self, manifest: dict[str, Any]) -> None: + outcomes = manifest.get("outcomes") + if not isinstance(outcomes, dict): + return + actions = { + str(outcome.get("action") or "").strip().lower() + for outcome in outcomes.values() + if isinstance(outcome, dict) + } + prohibited = sorted(actions & self._PROHIBITED_AUTOMATION_ACTIONS) + if prohibited: + raise PermissionError( + "风险规则包含禁止无人值守执行的高风险动作：" + ", ".join(prohibited) + ) + + @staticmethod + def _runtime_config( + manifest: dict[str, Any], + rule_document: dict[str, Any], + ) -> dict[str, Any]: + metadata = manifest.get("metadata") if isinstance(manifest.get("metadata"), dict) else {} + outcomes = manifest.get("outcomes") if isinstance(manifest.get("outcomes"), dict) else {} + fail = outcomes.get("fail") if isinstance(outcomes.get("fail"), dict) else {} + risk_level = str(metadata.get("risk_level") or fail.get("severity") or "medium") + risk_score = metadata.get("risk_score") or fail.get("risk_score") or 0 + try: + parsed_score = int(risk_score) + except (TypeError, ValueError): + parsed_score = 0 + return { + "severity": risk_level, + "risk_score": parsed_score, + "risk_level": risk_level, + "risk_level_label": metadata.get("risk_level_label"), + "risk_score_detail": metadata.get("risk_score_detail") or {}, + "enabled": True, + "detail_mode": "json_risk", + "rule_document": dict(rule_document), + "ontology_signal": manifest.get("ontology_signal"), + "evaluator": manifest.get("evaluator"), + "risk_category": manifest.get("risk_category"), + "flow_diagram_svg": manifest.get("flow_diagram_svg"), + } + + +def _manifest_hash(manifest: dict[str, Any]) -> str: + canonical = json.dumps( + manifest, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + default=str, + ) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() diff --git a/server/src/app/services/agent_asset_release_disposition_labels.py b/server/src/app/services/agent_asset_release_disposition_labels.py new file mode 100644 index 0000000..0905651 --- /dev/null +++ b/server/src/app/services/agent_asset_release_disposition_labels.py @@ -0,0 +1,147 @@ +"""把类型化风险处置解析为当前 Agent 发布样本的可信标签。""" + +from __future__ import annotations + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.agent_asset import AgentAsset +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) +from app.models.risk_disposition import RiskDispositionEvent +from app.models.risk_observation import RiskObservation +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, +) +from app.services.agent_asset_release_telemetry_crypto import release_source_fingerprints + +_RELEASE_STAGES = {"shadow", "canary", "active"} +_BUSINESS_STAGES = {"expense_application", "reimbursement"} + + +class AgentAssetReleaseDispositionLabelService: + """在当前 release/stage/version 内定位同租户、同单据、同规则样本。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def record_current_labels( + self, + *, + tenant_id: str, + disposition_event_id: str, + ) -> list[AgentAssetReleaseLabel]: + normalized_tenant = _required(tenant_id, "tenant_id", 64) + event = self.db.scalar( + select(RiskDispositionEvent).where( + RiskDispositionEvent.id == disposition_event_id, + RiskDispositionEvent.tenant_id == normalized_tenant, + ) + ) + if event is None: + raise LookupError("Risk disposition event not found.") + if event.action not in {"confirm", "false_positive"}: + return [] + risk_observation = self.db.scalar( + select(RiskObservation).where( + RiskObservation.id == event.observation_id, + RiskObservation.tenant_id == normalized_tenant, + ) + ) + if risk_observation is None: + raise LookupError("Risk observation not found.") + rule_code = _rule_code(risk_observation) + if not rule_code: + return [] + + asset = self._asset(normalized_tenant, rule_code) + state = _release_state(asset) + release_id = str(state.get("release_id") or "").strip() + stage = str(state.get("stage") or "").strip().lower() + version = str(state.get("candidate_version") or "").strip() + if not release_id or stage not in _RELEASE_STAGES or not version: + return [] + source_fingerprints = release_source_fingerprints( + tenant_id=normalized_tenant, + source_key=_required(risk_observation.claim_id, "claim_id", 100), + rule_code=rule_code, + ) + statement = select(AgentAssetReleaseObservation).where( + AgentAssetReleaseObservation.tenant_id == normalized_tenant, + AgentAssetReleaseObservation.asset_id == asset.id, + AgentAssetReleaseObservation.release_id == release_id, + AgentAssetReleaseObservation.stage == stage, + AgentAssetReleaseObservation.version == version, + AgentAssetReleaseObservation.rule_code == rule_code, + AgentAssetReleaseObservation.source_fingerprint.in_(source_fingerprints), + AgentAssetReleaseObservation.runtime_status == "completed", + ( + AgentAssetReleaseObservation.candidate_hit.is_(True) + | AgentAssetReleaseObservation.baseline_hit.is_(True) + ), + ) + business_stage = str(risk_observation.control_stage or "").strip().lower() + if business_stage in _BUSINESS_STAGES: + statement = statement.where( + AgentAssetReleaseObservation.business_stage == business_stage + ) + observations = list( + self.db.scalars( + statement.order_by( + AgentAssetReleaseObservation.created_at.asc(), + AgentAssetReleaseObservation.id.asc(), + ) + ).all() + ) + telemetry = AgentAssetReleaseTelemetryService(self.db) + return [ + telemetry.record_risk_disposition_label( + tenant_id=normalized_tenant, + observation_id=observation.id, + disposition_event_id=event.id, + ) + for observation in observations + ] + + def _asset(self, tenant_id: str, rule_code: str) -> AgentAsset: + asset = self.db.scalar( + select(AgentAsset) + .where( + AgentAsset.code == rule_code, + ( + ((AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == tenant_id)) + | ((AgentAsset.scope == "platform") & (AgentAsset.tenant_id == "platform")) + ), + ) + .order_by(AgentAsset.scope.desc()) + ) + if asset is None: + raise LookupError("Agent asset not found.") + configured_tenant = str((asset.config_json or {}).get("tenant_id") or "").strip() + if asset.scope == "tenant" and configured_tenant not in {"", tenant_id}: + raise LookupError("Agent asset not found.") + if asset.scope == "platform" and configured_tenant != tenant_id: + raise LookupError("Agent asset not found.") + return asset + + +def _rule_code(observation: RiskObservation) -> str: + value = str((observation.decision_trace_json or {}).get("rule_code") or "").strip() + if not value and observation.policy_refs_json: + value = str(observation.policy_refs_json[0] or "").strip() + return value + + +def _release_state(asset: AgentAsset) -> dict: + config = asset.config_json if isinstance(asset.config_json, dict) else {} + state = config.get("release_guard") + return dict(state) if isinstance(state, dict) else {} + + +def _required(value: object, field: str, maximum: int) -> str: + normalized = str(value or "").strip() + if not normalized or len(normalized) > maximum: + raise ValueError(f"{field} is required and must be at most {maximum} characters.") + return normalized diff --git a/server/src/app/services/agent_asset_release_guard.py b/server/src/app/services/agent_asset_release_guard.py new file mode 100644 index 0000000..71b2c0f --- /dev/null +++ b/server/src/app/services/agent_asset_release_guard.py @@ -0,0 +1,658 @@ +"""基于 AgentAsset 配置与测试记录的最小安全发布状态机。""" + +from __future__ import annotations + +import uuid +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.core.agent_enums import AgentAssetStatus, AgentAssetType, AgentReviewStatus +from app.models.agent_asset import ( + AgentAsset, + AgentAssetReview, + AgentAssetTestRun, + AgentAssetVersion, +) +from app.services.agent_asset_release_artifacts import AgentAssetReleaseArtifactService +from app.services.agent_asset_release_policy import ( + ReleaseEvaluationInput, + ReleaseGuardPolicy, + ReleaseStage, + evaluate_release, + normalize_release_policy, +) +from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager +from app.services.audit import AuditLogService + + +class AgentAssetReleaseGuardService: + """管理 shadow→canary→active，并在质量越界时自动回滚。""" + + CONFIG_KEY = "release_guard" + + def __init__( + self, + db: Session, + *, + rule_library_manager: AgentAssetRuleLibraryManager | None = None, + ) -> None: + self.db = db + self.artifacts = AgentAssetReleaseArtifactService(rule_library_manager) + self.audit_service = AuditLogService(db) + + def start_shadow( + self, + asset_id: str, + candidate_version: str, + *, + actor: str, + policy: ReleaseGuardPolicy | None = None, + tenant_id: str | None = None, + allow_global_management: bool = False, + request_id: str | None = None, + ) -> dict[str, Any]: + asset = self._asset( + asset_id, + tenant_id=tenant_id, + allow_global_management=allow_global_management, + lock=True, + ) + version = str(candidate_version or "").strip() + if not version or self._version(asset.id, version) is None: + raise ValueError("候选版本不存在，不能进入影子发布。") + if version == str(asset.published_version or "").strip(): + raise ValueError("候选版本已是当前发布版本，无需重复发布。") + current = self._state(asset) + if current.get("stage") in {"shadow", "canary"}: + if current.get("candidate_version") == version: + return current + raise ValueError("当前已有进行中的发布，请先完成或回滚。") + if current.get("stage") == "active" and current.get("candidate_version") == version: + return current + + if asset.asset_type == AgentAssetType.RULE.value: + self._require_rule_start_lifecycle(asset, version) + bundle = self.artifacts.capture(asset, version, previous_state=current) + if asset.asset_type == AgentAssetType.RULE.value: + self._require_rule_release_preconditions(asset, version, bundle.artifacts, actor=actor) + asset = self._asset( + asset_id, + tenant_id=tenant_id, + allow_global_management=allow_global_management, + lock=True, + ) + latest_state = self._state(asset) + if latest_state != current: + if ( + latest_state.get("stage") == "shadow" + and latest_state.get("candidate_version") == version + ): + return latest_state + raise ValueError("发布状态已被其他操作更新，请刷新后重试。") + + now = _now() + release_id = uuid.uuid4().hex + history = list(current.get("history") if isinstance(current.get("history"), list) else []) + history.append( + { + "from": current.get("stage") or "none", + "to": "shadow", + "actor": actor, + "at": now, + "reason": "release_started", + } + ) + state = { + "stage": "shadow", + "release_id": release_id, + "candidate_version": version, + "previous_version": str(asset.published_version or ""), + "policy": (policy or ReleaseGuardPolicy()).to_dict(), + "artifacts": bundle.artifacts, + "previous_config": bundle.previous_config, + "started_at": now, + "started_by": actor, + "updated_at": now, + "history": history[-20:], + } + self._save_state(asset, state, commit=False) + self._audit( + asset, + actor=actor, + action="start_agent_asset_shadow_release", + after={ + "stage": "shadow", + "release_id": release_id, + "candidate_version": version, + }, + request_id=request_id, + ) + self.db.commit() + return state + + def record_evaluation( + self, + asset_id: str, + evaluation: ReleaseEvaluationInput, + *, + actor: str, + tenant_id: str | None = None, + allow_global_management: bool = False, + request_id: str | None = None, + ) -> dict[str, Any]: + asset = self._asset( + asset_id, + tenant_id=tenant_id, + allow_global_management=allow_global_management, + lock=True, + ) + state = self._state(asset) + stage = str(state.get("stage") or "") + if asset.status == AgentAssetStatus.DISABLED.value: + raise ValueError("当前资产已停用，不能继续写入发布评测。") + if stage not in {"shadow", "canary", "active"}: + raise ValueError("当前资产不在可评测的发布阶段。") + candidate_version = str(state.get("candidate_version") or "") + release_id = str(state.get("release_id") or "") + policy = self._normalized_policy(state.get("policy")) + result = self._evaluate(stage, evaluation, policy) + requested_tenant = str(tenant_id or "").strip() + run = AgentAssetTestRun( + id=str(uuid.uuid4()), + tenant_id=requested_tenant or asset.tenant_id, + scope="tenant" if requested_tenant else asset.scope, + asset_id=asset.id, + version=candidate_version, + test_type=f"release_{stage}", + status=result["status"], + passed=result["status"] == "passed", + summary=result["summary"], + input_json={ + "total": evaluation.total, + "failure_count": evaluation.failure_count, + "precision": evaluation.precision, + "baseline_precision": evaluation.baseline_precision, + "release_id": release_id, + }, + result_json={**result, "details": dict(evaluation.details or {})}, + created_by=actor, + created_at=datetime.now(UTC), + ) + self.db.add(run) + if result["status"] == "failed": + self._apply_rollback( + asset, + state, + actor=actor, + reason=";".join(result["reasons"]), + automatic=True, + ) + self._audit( + asset, + actor=actor, + action="evaluate_agent_asset_release", + after={ + "stage": stage, + "status": result["status"], + "reasons": result["reasons"], + "automatic_rollback": result["status"] == "failed", + }, + request_id=request_id, + ) + self.db.commit() + self.db.refresh(run) + return { + "test_run_id": run.id, + "stage": stage, + **result, + "release_stage": self._state(asset).get("stage"), + } + + def promote( + self, + asset_id: str, + *, + actor: str, + tenant_id: str | None = None, + allow_global_management: bool = False, + request_id: str | None = None, + ) -> dict[str, Any]: + asset = self._asset( + asset_id, + tenant_id=tenant_id, + allow_global_management=allow_global_management, + lock=True, + ) + state = self._state(asset) + stage = str(state.get("stage") or "") + if stage not in {"shadow", "canary"}: + raise ValueError("只有 shadow 或 canary 阶段可以晋级。") + candidate_version = str(state.get("candidate_version") or "") + if asset.asset_type == AgentAssetType.RULE.value: + self._require_release_state_integrity(asset, state) + latest = self._latest_stage_run( + asset.id, + candidate_version, + stage, + release_id=str(state.get("release_id") or ""), + ) + if latest is None or latest.status != "passed" or not latest.passed: + raise PermissionError(f"{stage} 阶段尚无通过的质量评测，不能晋级。") + + target = "canary" if stage == "shadow" else "active" + if asset.asset_type == AgentAssetType.RULE.value: + self._require_approved_rule_version(asset, candidate_version) + if target == "active": + if asset.asset_type == AgentAssetType.RULE.value: + self.artifacts.apply_candidate(asset, state, actor=actor) + asset.published_version = candidate_version + asset.current_version = candidate_version + asset.working_version = candidate_version + asset.status = AgentAssetStatus.ACTIVE.value + self._transition(state, stage, target, actor=actor, reason="quality_gate_passed") + self._save_state(asset, state, commit=False) + self._audit( + asset, + actor=actor, + action="promote_agent_asset_release", + after={"from": stage, "to": target, "candidate_version": candidate_version}, + request_id=request_id, + ) + self.db.commit() + return state + + def rollback( + self, + asset_id: str, + *, + actor: str, + reason: str, + tenant_id: str | None = None, + allow_global_management: bool = False, + request_id: str | None = None, + ) -> dict[str, Any]: + asset = self._asset( + asset_id, + tenant_id=tenant_id, + allow_global_management=allow_global_management, + lock=True, + ) + state = self._state(asset) + if state.get("stage") not in {"shadow", "canary", "active"}: + raise ValueError("当前发布状态不能回滚。") + self._apply_rollback(asset, state, actor=actor, reason=reason, automatic=False) + self._audit( + asset, + actor=actor, + action="rollback_agent_asset_release", + after={"stage": "rolled_back", "reason": reason, "automatic": False}, + request_id=request_id, + ) + self.db.commit() + return state + + def get_state( + self, + asset_id: str, + *, + tenant_id: str | None = None, + allow_global_management: bool = False, + ) -> dict[str, Any]: + return self._state( + self._asset( + asset_id, + tenant_id=tenant_id, + allow_global_management=allow_global_management, + ) + ) + + def get_serving_plan( + self, + asset_id: str, + *, + tenant_id: str | None = None, + allow_global_management: bool = False, + ) -> dict[str, Any]: + """返回运行时可直接消费的候选/主版本路由计划。""" + + asset = self._asset( + asset_id, + tenant_id=tenant_id, + allow_global_management=allow_global_management, + ) + state = self._state(asset) + stage = str(state.get("stage") or "") + candidate = str(state.get("candidate_version") or "") + previous = str(state.get("previous_version") or asset.published_version or "") + policy = self._normalized_policy(state.get("policy")) + if stage == "shadow": + return { + "stage": stage, + "primary_version": previous, + "candidate_version": candidate, + "candidate_traffic_percent": 0, + "shadow_evaluation": True, + } + if stage == "canary": + return { + "stage": stage, + "primary_version": previous, + "candidate_version": candidate, + "candidate_traffic_percent": policy["canary_traffic_percent"], + "shadow_evaluation": False, + } + if stage == "active": + return { + "stage": stage, + "primary_version": candidate, + "candidate_version": candidate, + "candidate_traffic_percent": 100, + "shadow_evaluation": False, + } + return { + "stage": stage or "unmanaged", + "primary_version": previous or str(asset.published_version or ""), + "candidate_version": candidate, + "candidate_traffic_percent": 0, + "shadow_evaluation": False, + } + + def _evaluate( + self, + stage: str, + evaluation: ReleaseEvaluationInput, + policy: dict[str, Any], + ) -> dict[str, Any]: + return evaluate_release(stage, evaluation, policy) + + def _apply_rollback( + self, + asset: AgentAsset, + state: dict[str, Any], + *, + actor: str, + reason: str, + automatic: bool, + ) -> None: + previous = str(state.get("previous_version") or "") + candidate = str(state.get("candidate_version") or "") + asset.published_version = previous or None + if previous: + asset.current_version = previous + asset.working_version = previous + asset.status = AgentAssetStatus.ACTIVE.value + else: + asset.current_version = candidate or asset.current_version + asset.working_version = candidate or asset.working_version + asset.status = AgentAssetStatus.REVIEW.value + state["rollback"] = { + "automatic": automatic, + "reason": str(reason or "release_guard_triggered"), + "actor": actor, + "at": _now(), + "restored_version": previous, + } + self._transition( + state, + str(state.get("stage") or "unknown"), + "rolled_back", + actor=actor, + reason=str(reason or "release_guard_triggered"), + ) + if asset.asset_type == AgentAssetType.RULE.value: + self.artifacts.restore_previous(asset, state) + self._save_state(asset, state, commit=False) + + @staticmethod + def _transition( + state: dict[str, Any], + source: str, + target: ReleaseStage, + *, + actor: str, + reason: str, + ) -> None: + now = _now() + history = list(state.get("history") if isinstance(state.get("history"), list) else []) + history.append({"from": source, "to": target, "actor": actor, "at": now, "reason": reason}) + state["history"] = history[-20:] + state["stage"] = target + state["updated_at"] = now + + def _save_state( + self, + asset: AgentAsset, + state: dict[str, Any], + *, + commit: bool = True, + ) -> None: + config = dict(asset.config_json or {}) + config[self.CONFIG_KEY] = state + asset.config_json = config + self.db.add(asset) + if commit: + self.db.commit() + + @staticmethod + def _state(asset: AgentAsset) -> dict[str, Any]: + config = asset.config_json if isinstance(asset.config_json, dict) else {} + value = config.get(AgentAssetReleaseGuardService.CONFIG_KEY) + return dict(value) if isinstance(value, dict) else {} + + def _asset( + self, + asset_id: str, + *, + tenant_id: str | None = None, + allow_global_management: bool = False, + lock: bool = False, + ) -> AgentAsset: + requested_tenant = str(tenant_id or "").strip() + stmt = select(AgentAsset).where(AgentAsset.id == asset_id) + if requested_tenant: + stmt = stmt.where( + ( + (AgentAsset.scope == "tenant") + & (AgentAsset.tenant_id == requested_tenant) + ) + | ( + (AgentAsset.scope == "platform") + & (AgentAsset.tenant_id == "platform") + ) + ) + else: + stmt = stmt.where( + AgentAsset.scope == "platform", + AgentAsset.tenant_id == "platform", + ) + if lock: + stmt = stmt.with_for_update() + asset = self.db.scalar(stmt) + if asset is None: + raise LookupError("Agent asset not found") + configured_tenant = str((asset.config_json or {}).get("tenant_id") or "").strip() + if requested_tenant: + if asset.scope == "tenant" and configured_tenant not in { + "", + requested_tenant, + }: + raise LookupError("Agent asset not found") + if asset.scope == "platform" and not allow_global_management: + # 未绑定租户的规则是平台共享资产，租户管理员不得修改其发布状态。 + raise LookupError("Agent asset not found") + return asset + + def _version(self, asset_id: str, version: str) -> AgentAssetVersion | None: + return self.db.scalar( + select(AgentAssetVersion).where( + AgentAssetVersion.asset_id == asset_id, + AgentAssetVersion.version == version, + ) + ) + + def _latest_stage_run( + self, + asset_id: str, + version: str, + stage: str, + *, + release_id: str, + ) -> AgentAssetTestRun | None: + runs = list( + self.db.scalars( + select(AgentAssetTestRun) + .where( + AgentAssetTestRun.asset_id == asset_id, + AgentAssetTestRun.version == version, + AgentAssetTestRun.test_type == f"release_{stage}", + ) + .order_by(AgentAssetTestRun.created_at.desc(), AgentAssetTestRun.id.desc()) + .limit(50) + ).all() + ) + return next( + ( + run + for run in runs + if str((run.input_json or {}).get("release_id") or "") == release_id + ), + None, + ) + + def _require_approved_rule_version(self, asset: AgentAsset, version: str) -> None: + if asset.asset_type != AgentAssetType.RULE.value: + return + review = self.db.scalar( + select(AgentAssetReview) + .where( + AgentAssetReview.asset_id == asset.id, + AgentAssetReview.version == version, + ) + .order_by(AgentAssetReview.created_at.desc(), AgentAssetReview.id.desc()) + ) + if review is None or review.review_status != AgentReviewStatus.APPROVED.value: + raise PermissionError("风险规则候选版本未经审核批准，不能进入 Canary 或 active。") + + def _require_release_state_integrity( + self, + asset: AgentAsset, + state: dict[str, Any], + ) -> None: + candidate = str(state.get("candidate_version") or "").strip() + previous = str(state.get("previous_version") or "").strip() + AgentAssetReleaseArtifactService.artifact(state, candidate) + if previous: + AgentAssetReleaseArtifactService.artifact(state, previous) + if str(asset.published_version or "").strip() != previous: + raise PermissionError("发布期间基线版本发生变化，已停止晋级。") + if asset.status != AgentAssetStatus.ACTIVE.value: + raise PermissionError("发布期间基线规则已不再生效，已停止晋级。") + else: + if str(asset.published_version or "").strip(): + raise PermissionError("发布期间资产发布版本发生变化，已停止晋级。") + if asset.status != AgentAssetStatus.REVIEW.value: + raise PermissionError("候选规则已不在待审核状态，已停止晋级。") + config = asset.config_json if isinstance(asset.config_json, dict) else {} + if config.get("enabled") is False: + raise PermissionError("发布期间风险规则已被停用，已停止晋级。") + + def _require_rule_release_preconditions( + self, + asset: AgentAsset, + version: str, + artifacts: dict[str, dict[str, Any]], + *, + actor: str, + ) -> None: + report = self.db.scalar( + select(AgentAssetTestRun) + .where( + AgentAssetTestRun.asset_id == asset.id, + AgentAssetTestRun.version == version, + AgentAssetTestRun.test_type == "report", + ) + .order_by(AgentAssetTestRun.created_at.desc(), AgentAssetTestRun.id.desc()) + ) + if report is None or not report.passed or report.status != "passed": + raise PermissionError("候选风险规则尚无通过的测试报告，不能进入影子发布。") + artifact = artifacts.get(version) + manifest = artifact.get("manifest") if isinstance(artifact, dict) else None + if not isinstance(manifest, dict): + raise PermissionError("候选风险规则缺少可信运行快照。") + rule_code = str(manifest.get("rule_code") or "").strip() + from app.services.risk_rule_golden_evaluator import RiskRuleGoldenEvaluator + + RiskRuleGoldenEvaluator().require_pass( + self.db, + asset, + version, + manifest, + rule_code, + actor=actor, + ) + review = self.db.scalar( + select(AgentAssetReview) + .where( + AgentAssetReview.asset_id == asset.id, + AgentAssetReview.version == version, + ) + .order_by(AgentAssetReview.created_at.desc(), AgentAssetReview.id.desc()) + ) + if review is None or review.review_status != AgentReviewStatus.APPROVED.value: + self.db.add( + AgentAssetReview( + asset_id=asset.id, + version=version, + reviewer=actor, + review_status=AgentReviewStatus.APPROVED.value, + review_note="批准候选版本进入受控分阶段发布。", + reviewed_at=datetime.now(UTC), + created_at=datetime.now(UTC), + ) + ) + asset.reviewer = actor + + @staticmethod + def _require_rule_start_lifecycle(asset: AgentAsset, version: str) -> None: + working = str(asset.working_version or asset.current_version or "").strip() + published = str(asset.published_version or "").strip() + if version != working: + raise PermissionError("只能发布当前工作版本，历史版本不能直接进入分阶段发布。") + if published: + if asset.status != AgentAssetStatus.ACTIVE.value: + raise PermissionError("只有当前基线仍在线的规则才能发布修订候选。") + if version == published: + raise ValueError("候选版本已是当前发布版本，无需重复发布。") + return + if asset.status != AgentAssetStatus.REVIEW.value: + raise PermissionError("首次发布前必须先完成测试并提交审核。") + + def _audit( + self, + asset: AgentAsset, + *, + actor: str, + action: str, + after: dict[str, Any], + request_id: str | None, + ) -> None: + self.audit_service.log_action( + actor=actor, + action=action, + resource_type=asset.asset_type, + resource_id=asset.id, + after_json=after, + request_id=request_id, + commit=False, + ) + + @staticmethod + def _normalized_policy(value: Any) -> dict[str, Any]: + return normalize_release_policy(value) + + +def _now() -> str: + return datetime.now(UTC).isoformat() diff --git a/server/src/app/services/agent_asset_release_label_votes.py b/server/src/app/services/agent_asset_release_label_votes.py new file mode 100644 index 0000000..a6a756b --- /dev/null +++ b/server/src/app/services/agent_asset_release_label_votes.py @@ -0,0 +1,69 @@ +"""发布复核标签的独立人员票数归并。""" + +from __future__ import annotations + +from collections.abc import Iterable +from dataclasses import dataclass +from typing import Any + +_CANONICAL_LABELS = { + "confirmed": "confirmed", + "risk_present": "confirmed", + "false_positive": "false_positive", + "risk_absent": "false_positive", +} + + +@dataclass(frozen=True, slots=True) +class ReleaseLabelVoteState: + label: str | None + reviewer_count: int + required_reviewers: int + conflicted: bool + + +def resolve_release_label_votes( + labels: Iterable[Any], + *, + required_reviewers: int, + required_reviewers_by_observation: dict[str, int] | None = None, +) -> dict[str, ReleaseLabelVoteState]: + default_quorum = max(1, min(2, int(required_reviewers))) + quorum_by_observation = required_reviewers_by_observation or {} + latest_by_actor: dict[tuple[str, str], str] = {} + for item in labels: + observation_id = str(item.observation_id) + actor = str(item.actor_fingerprint) + label = _CANONICAL_LABELS.get(str(item.label)) + if label is not None: + latest_by_actor[(observation_id, actor)] = label + + votes: dict[str, dict[str, int]] = {} + reviewers: dict[str, int] = {} + for (observation_id, _actor), label in latest_by_actor.items(): + reviewers[observation_id] = reviewers.get(observation_id, 0) + 1 + bucket = votes.setdefault(observation_id, {}) + bucket[label] = bucket.get(label, 0) + 1 + + states: dict[str, ReleaseLabelVoteState] = {} + for observation_id, counts in votes.items(): + quorum = max( + 1, + min(2, int(quorum_by_observation.get(observation_id, default_quorum))), + ) + eligible = [label for label, count in counts.items() if count >= quorum] + states[observation_id] = ReleaseLabelVoteState( + label=eligible[0] if len(eligible) == 1 else None, + reviewer_count=reviewers[observation_id], + required_reviewers=quorum, + conflicted=len(counts) > 1, + ) + return states + + +def release_reviewer_quorum(state: dict[str, Any]) -> int: + policy = state.get("policy") if isinstance(state.get("policy"), dict) else {} + try: + return max(1, min(2, int(policy.get("reviewer_quorum") or 1))) + except (TypeError, ValueError, OverflowError): + return 1 diff --git a/server/src/app/services/agent_asset_release_monitor.py b/server/src/app/services/agent_asset_release_monitor.py new file mode 100644 index 0000000..c101c63 --- /dev/null +++ b/server/src/app/services/agent_asset_release_monitor.py @@ -0,0 +1,457 @@ +"""用数据库真实发布遥测驱动 Release Guard 的协调器。""" + +from __future__ import annotations + +import hashlib +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.models.agent_asset import AgentAsset, AgentAssetTestRun +from app.services.agent_asset_release_alerts import build_release_alerts +from app.services.agent_asset_release_guard import AgentAssetReleaseGuardService +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseTelemetryAggregate, + ReleaseTelemetryStaleRelease, +) + +_MONITORED_STAGES = {"shadow", "canary", "active"} +_MAX_BATCH_SIZE = 500 + + +class AgentAssetReleaseMonitor: + """只接受 release 身份，绝不接受调用方提供的质量汇总数字。""" + + def __init__( + self, + db: Session, + *, + telemetry_service: AgentAssetReleaseTelemetryService | None = None, + guard_service: AgentAssetReleaseGuardService | None = None, + ) -> None: + self.db = db + self.telemetry = telemetry_service or AgentAssetReleaseTelemetryService(db) + self.guard = guard_service or AgentAssetReleaseGuardService(db) + + def evaluate_current( + self, + *, + tenant_id: str, + asset_id: str, + actor: str, + allow_global_management: bool = False, + ) -> dict[str, Any]: + """聚合当前 release；只有真实标签完备后才调用 Release Guard。""" + + tenant = _required(tenant_id, "tenant_id", 64) + target_asset = _required(asset_id, "asset_id", 36) + normalized_actor = _required(actor, "actor", 100) + _require_bool(allow_global_management, "allow_global_management") + asset, state = self._load_target( + tenant_id=tenant, + asset_id=target_asset, + allow_global_management=allow_global_management, + lock=False, + ) + identity = self._release_identity(asset, state) + aggregate = self.telemetry.aggregate( + tenant_id=tenant, + asset_id=asset.id, + release_id=identity["release_id"], + stage=identity["stage"], + version=identity["version"], + ) + if aggregate.status == "collecting": + return self._collecting_result(aggregate) + + # 在把指标交给 Guard 前锁定资产并重验 release 身份，避免把旧阶段 + # 的真实指标写入刚刚晋级或重启的新 release。 + locked_asset, locked_state = self._load_target( + tenant_id=tenant, + asset_id=target_asset, + allow_global_management=allow_global_management, + lock=True, + ) + if self._release_identity(locked_asset, locked_state) != identity: + raise ReleaseTelemetryStaleRelease( + "Release changed after telemetry aggregation; evaluation was not submitted." + ) + + evaluation = aggregate.to_release_evaluation_input() + existing_run = self._existing_evaluation( + aggregate=aggregate, + evaluation_input={ + "total": evaluation.total, + "failure_count": evaluation.failure_count, + "precision": evaluation.precision, + "baseline_precision": evaluation.baseline_precision, + "release_id": identity["release_id"], + }, + ) + if existing_run is not None: + result = existing_run.result_json if isinstance(existing_run.result_json, dict) else {} + release_stage = str(locked_state.get("stage") or identity["stage"]) + payload = { + "asset_id": locked_asset.id, + "release_id": identity["release_id"], + "stage": identity["stage"], + "version": identity["version"], + "telemetry_status": "ready", + "status": str(existing_run.status or "unknown"), + "evaluation_submitted": True, + "release_stage": release_stage, + "rolled_back": release_stage == "rolled_back", + "reasons": list(result.get("reasons") or []), + "test_run_id": existing_run.id, + "metrics": self._metrics(aggregate), + } + payload["alerts"] = build_release_alerts( + status=payload["status"], + rolled_back=payload["rolled_back"], + reasons=payload["reasons"], + metrics=payload["metrics"], + ) + return payload + guard_result = self.guard.record_evaluation( + locked_asset.id, + evaluation, + actor=normalized_actor, + tenant_id=tenant, + allow_global_management=allow_global_management, + request_id=self._request_id(aggregate), + ) + release_stage = str(guard_result.get("release_stage") or identity["stage"]) + payload = { + "asset_id": locked_asset.id, + "release_id": identity["release_id"], + "stage": identity["stage"], + "version": identity["version"], + "telemetry_status": "ready", + "status": str(guard_result.get("status") or "unknown"), + "evaluation_submitted": True, + "release_stage": release_stage, + "rolled_back": release_stage == "rolled_back", + "reasons": list(guard_result.get("reasons") or []), + "test_run_id": guard_result.get("test_run_id"), + "metrics": self._metrics(aggregate), + } + payload["alerts"] = build_release_alerts( + status=payload["status"], + rolled_back=payload["rolled_back"], + reasons=payload["reasons"], + metrics=payload["metrics"], + ) + return payload + + def batch_evaluate( + self, + *, + tenant_id: str, + actor: str, + allow_global_management: bool = False, + limit: int = 100, + after_asset_id: str | None = None, + ) -> dict[str, Any]: + """有界扫描当前受控风险规则，并将每个资产的失败隔离。""" + + tenant = _required(tenant_id, "tenant_id", 64) + normalized_actor = _required(actor, "actor", 100) + _require_bool(allow_global_management, "allow_global_management") + normalized_limit = _batch_limit(limit) + normalized_cursor = str(after_asset_id or "").strip() or None + assets = self._batch_targets( + tenant_id=tenant, + limit=normalized_limit, + after_asset_id=normalized_cursor, + ) + results: list[dict[str, Any]] = [] + errors: list[dict[str, Any]] = [] + evaluated = 0 + collecting = 0 + rolled_back = 0 + for asset in assets: + try: + result = self.evaluate_current( + tenant_id=tenant, + asset_id=asset.id, + actor=normalized_actor, + allow_global_management=allow_global_management, + ) + except Exception as error: + # Guard 在成功路径自行提交；失败路径必须回滚当前 Session， + # 防止一个资产留下的失败事务污染后续资产。 + self.db.rollback() + errors.append( + { + "asset_id": asset.id, + "error_type": type(error).__name__, + "message": _safe_error(error), + "alerts": build_release_alerts( + status="collecting", + rolled_back=False, + reasons=("telemetry_aggregation_failed",), + metrics={"aggregation_failed": True}, + ), + } + ) + continue + results.append(result) + if result["evaluation_submitted"]: + evaluated += 1 + if result["status"] == "collecting": + collecting += 1 + if result["rolled_back"]: + rolled_back += 1 + return { + "tenant_id": tenant, + "limit": normalized_limit, + "scanned": len(assets), + "evaluated": evaluated, + "collecting": collecting, + "rolled_back": rolled_back, + "next_cursor": assets[-1].id if assets else normalized_cursor, + "errors": errors, + "results": results, + } + + def _existing_evaluation( + self, + *, + aggregate: ReleaseTelemetryAggregate, + evaluation_input: dict[str, Any], + ) -> AgentAssetTestRun | None: + """相同 release 与真实聚合快照只产生一次测试运行。""" + + runs = self.db.scalars( + select(AgentAssetTestRun) + .where( + AgentAssetTestRun.tenant_id == aggregate.tenant_id, + AgentAssetTestRun.scope == "tenant", + AgentAssetTestRun.asset_id == aggregate.asset_id, + AgentAssetTestRun.version == aggregate.version, + AgentAssetTestRun.test_type == f"release_{aggregate.stage}", + ) + .order_by(AgentAssetTestRun.created_at.desc(), AgentAssetTestRun.id.desc()) + ) + for run in runs: + if run.input_json == evaluation_input: + return run + return None + + def _load_target( + self, + *, + tenant_id: str, + asset_id: str, + allow_global_management: bool, + lock: bool, + ) -> tuple[AgentAsset, dict[str, Any]]: + statement = select(AgentAsset).where( + AgentAsset.id == asset_id, + ( + ((AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == tenant_id)) + | ((AgentAsset.scope == "platform") & (AgentAsset.tenant_id == "platform")) + ), + ) + if lock: + statement = statement.with_for_update().execution_options(populate_existing=True) + asset = self.db.scalar(statement) + if asset is None: + raise LookupError("Agent asset not found.") + config = asset.config_json if isinstance(asset.config_json, dict) else {} + configured_tenant = str(config.get("tenant_id") or "").strip() + if asset.scope == "tenant": + if configured_tenant not in {"", tenant_id}: + raise LookupError("Agent asset not found.") + else: + # 全局资产会命中多个租户；用某一个当前用户的 tenant_id 聚合会把 + # 局部样本误当成全局事实。跨租户聚合协议完成前，即使平台管理员 + # 也只能显式回滚，不能触发自动评测。 + if allow_global_management: + raise ValueError( + "Global release telemetry requires a cross-tenant aggregate and " + "cannot be evaluated by the tenant monitor." + ) + raise LookupError("Agent asset not found.") + if ( + asset.asset_type != AgentAssetType.RULE.value + or asset.domain != AgentAssetDomain.EXPENSE.value + or str(config.get("detail_mode") or "").strip().lower() != "json_risk" + ): + raise ValueError("Asset is not an expense JSON risk rule.") + if asset.status == AgentAssetStatus.DISABLED.value or config.get("enabled") is False: + raise ValueError("Disabled risk rules cannot be evaluated by the release monitor.") + state = config.get("release_guard") + if not isinstance(state, dict): + raise ValueError("Asset has no active release guard state.") + return asset, dict(state) + + @staticmethod + def _release_identity(asset: AgentAsset, state: dict[str, Any]) -> dict[str, str]: + stage = str(state.get("stage") or "").strip().lower() + if stage not in _MONITORED_STAGES: + raise ValueError("Asset is not in a monitored release stage.") + return { + "asset_id": asset.id, + "release_id": _required(state.get("release_id"), "release_id", 64), + "stage": stage, + "version": _required(state.get("candidate_version"), "candidate_version", 30), + } + + def _batch_targets( + self, + *, + tenant_id: str, + limit: int, + after_asset_id: str | None, + ) -> list[AgentAsset]: + config = AgentAsset.config_json + tenant_value = config["tenant_id"].as_string() + base = ( + select(AgentAsset) + .where( + AgentAsset.asset_type == AgentAssetType.RULE.value, + AgentAsset.domain == AgentAssetDomain.EXPENSE.value, + AgentAsset.scope == "tenant", + AgentAsset.tenant_id == tenant_id, + config["detail_mode"].as_string() == "json_risk", + config["release_guard"]["stage"].as_string().in_(sorted(_MONITORED_STAGES)), + tenant_value == tenant_id, + ) + .order_by(AgentAsset.id.asc()) + ) + if not after_asset_id: + return list(self.db.scalars(base.limit(limit)).all()) + + # 游标扫描到尾部后从头补足本轮，避免固定 limit 永远只评测 ID 最小的资产。 + after = list( + self.db.scalars( + base.where(AgentAsset.id > after_asset_id).limit(limit) + ).all() + ) + remaining = limit - len(after) + if remaining <= 0: + return after + wrapped = list( + self.db.scalars( + base.where(AgentAsset.id <= after_asset_id).limit(remaining) + ).all() + ) + return [*after, *wrapped] + + @staticmethod + def _collecting_result(aggregate: ReleaseTelemetryAggregate) -> dict[str, Any]: + payload = { + "asset_id": aggregate.asset_id, + "release_id": aggregate.release_id, + "stage": aggregate.stage, + "version": aggregate.version, + "telemetry_status": "collecting", + "status": "collecting", + "evaluation_submitted": False, + "release_stage": aggregate.stage, + "rolled_back": False, + "reasons": list(aggregate.reasons), + "test_run_id": None, + "metrics": AgentAssetReleaseMonitor._metrics(aggregate), + } + payload["alerts"] = build_release_alerts( + status=payload["status"], + rolled_back=False, + reasons=payload["reasons"], + metrics=payload["metrics"], + ) + return payload + + @staticmethod + def _metrics(aggregate: ReleaseTelemetryAggregate) -> dict[str, Any]: + observed_count = aggregate.observed_count + return { + "observed_count": observed_count, + "completed_count": aggregate.completed_count, + "runtime_failure_count": aggregate.runtime_failure_count, + "runtime_failure_rate": ( + aggregate.runtime_failure_count / observed_count if observed_count else None + ), + "candidate_hit_count": aggregate.candidate_hit_count, + "candidate_labeled_count": aggregate.candidate_labeled_count, + "candidate_pending_label_count": aggregate.candidate_pending_label_count, + "precision": aggregate.precision, + "baseline_hit_count": aggregate.baseline_hit_count, + "baseline_labeled_count": aggregate.baseline_labeled_count, + "baseline_pending_label_count": aggregate.baseline_pending_label_count, + "baseline_precision": aggregate.baseline_precision, + "false_negative_count": aggregate.false_negative_count, + "estimated_false_negative_count": aggregate.estimated_false_negative_count, + "false_negative_upper_bound": aggregate.false_negative_upper_bound, + "negative_sample_count": aggregate.negative_sample_count, + "negative_labeled_count": aggregate.negative_labeled_count, + "negative_pending_label_count": aggregate.negative_pending_label_count, + "random_negative_population_count": aggregate.random_negative_population_count, + "random_negative_sample_count": aggregate.random_negative_sample_count, + "random_negative_labeled_count": aggregate.random_negative_labeled_count, + "recall": aggregate.recall, + "recall_lower_bound": aggregate.recall_lower_bound, + "recall_confidence_level": aggregate.recall_confidence_level, + "recall_method": aggregate.recall_method, + "negative_ground_truth_status": aggregate.negative_ground_truth_status, + } + + @staticmethod + def _request_id(aggregate: ReleaseTelemetryAggregate) -> str: + source = "\x1f".join( + str(value) + for value in ( + aggregate.tenant_id, + aggregate.asset_id, + aggregate.release_id, + aggregate.stage, + aggregate.version, + aggregate.observed_count, + aggregate.runtime_failure_count, + aggregate.candidate_labeled_count, + aggregate.candidate_confirmed_count, + aggregate.candidate_false_positive_count, + aggregate.baseline_labeled_count, + aggregate.baseline_confirmed_count, + aggregate.baseline_false_positive_count, + aggregate.negative_sample_count, + aggregate.negative_labeled_count, + aggregate.false_negative_count, + aggregate.recall, + aggregate.recall_lower_bound, + ) + ) + return f"release-monitor:{hashlib.sha256(source.encode('utf-8')).hexdigest()[:32]}" + + +def _required(value: Any, field: str, maximum: int) -> str: + normalized = str(value or "").strip() + if not normalized or len(normalized) > maximum: + raise ValueError(f"{field} is required and must be at most {maximum} characters.") + return normalized + + +def _require_bool(value: Any, field: str) -> None: + if not isinstance(value, bool): + raise ValueError(f"{field} must be a boolean.") + + +def _batch_limit(value: Any) -> int: + if isinstance(value, bool): + raise ValueError("limit must be an integer.") + try: + parsed = int(value) + except (TypeError, ValueError, OverflowError) as error: + raise ValueError("limit must be an integer.") from error + if parsed < 1 or parsed > _MAX_BATCH_SIZE: + raise ValueError(f"limit must be between 1 and {_MAX_BATCH_SIZE}.") + return parsed + + +def _safe_error(error: Exception) -> str: + message = " ".join(str(error).split()) + return message[:240] or type(error).__name__ diff --git a/server/src/app/services/agent_asset_release_monitor_auth.py b/server/src/app/services/agent_asset_release_monitor_auth.py new file mode 100644 index 0000000..e20be15 --- /dev/null +++ b/server/src/app/services/agent_asset_release_monitor_auth.py @@ -0,0 +1,119 @@ +"""受控发布监控指标的 HMAC 认证。""" + +from __future__ import annotations + +import hashlib +import hmac +import json +import os +import time +from typing import Any + +RELEASE_MONITOR_SECRET_ENV = "AGENT_RELEASE_MONITOR_SECRET" +MAX_CLOCK_SKEW_SECONDS = 300 + + +class ReleaseMonitorConfigurationError(RuntimeError): + """发布监控认证尚未完成安全配置。""" + + +class ReleaseMonitorAuthenticationError(PermissionError): + """发布监控签名不可信。""" + + +def build_release_monitor_signature( + *, + timestamp: str, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + payload: dict[str, Any], + secret: str | None = None, +) -> str: + signing_secret = _resolve_secret(secret) + message = _signature_message( + timestamp=timestamp, + tenant_id=tenant_id, + asset_id=asset_id, + release_id=release_id, + stage=stage, + payload=payload, + ) + return hmac.new( + signing_secret.encode("utf-8"), + message.encode("utf-8"), + hashlib.sha256, + ).hexdigest() + + +def require_release_monitor_signature( + *, + timestamp: str | None, + signature: str | None, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + payload: dict[str, Any], + now_epoch_seconds: int | None = None, +) -> None: + normalized_timestamp = str(timestamp or "").strip() + normalized_signature = str(signature or "").strip().lower() + if not normalized_timestamp or not normalized_signature: + raise ReleaseMonitorAuthenticationError("发布评测缺少监控签名或时间戳。") + try: + parsed_timestamp = int(normalized_timestamp) + except (TypeError, ValueError, OverflowError) as exc: + raise ReleaseMonitorAuthenticationError("发布评测时间戳格式无效。") from exc + now = int(time.time()) if now_epoch_seconds is None else int(now_epoch_seconds) + if abs(now - parsed_timestamp) > MAX_CLOCK_SKEW_SECONDS: + raise ReleaseMonitorAuthenticationError("发布评测签名已过期或服务器时钟偏差过大。") + expected = build_release_monitor_signature( + timestamp=normalized_timestamp, + tenant_id=tenant_id, + asset_id=asset_id, + release_id=release_id, + stage=stage, + payload=payload, + ) + if not hmac.compare_digest(normalized_signature, expected): + raise ReleaseMonitorAuthenticationError("发布评测监控签名无效。") + + +def _signature_message( + *, + timestamp: str, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + payload: dict[str, Any], +) -> str: + canonical_payload = json.dumps( + payload, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + default=str, + ) + payload_hash = hashlib.sha256(canonical_payload.encode("utf-8")).hexdigest() + return "\n".join( + ( + str(timestamp).strip(), + str(tenant_id).strip(), + str(asset_id).strip(), + str(release_id).strip(), + str(stage).strip(), + payload_hash, + ) + ) + + +def _resolve_secret(value: str | None) -> str: + secret = str(value if value is not None else os.environ.get(RELEASE_MONITOR_SECRET_ENV, "")) + if len(secret.encode("utf-8")) < 32: + raise ReleaseMonitorConfigurationError( + f"{RELEASE_MONITOR_SECRET_ENV} 必须配置为至少 32 字节的独立密钥。" + ) + return secret diff --git a/server/src/app/services/agent_asset_release_policy.py b/server/src/app/services/agent_asset_release_policy.py new file mode 100644 index 0000000..d15f9d4 --- /dev/null +++ b/server/src/app/services/agent_asset_release_policy.py @@ -0,0 +1,257 @@ +"""Agent 资产分阶段发布的纯策略归一化与质量门禁计算。""" + +from __future__ import annotations + +from dataclasses import dataclass +from math import isfinite +from typing import Any, Literal + +ReleaseStage = Literal["shadow", "canary", "active", "rolled_back"] + + +@dataclass(frozen=True) +class ReleaseGuardPolicy: + shadow_min_samples: int = 20 + canary_min_samples: int = 100 + max_error_rate: float = 0.02 + min_precision: float = 0.98 + max_precision_drop: float = 0.02 + canary_traffic_percent: int = 5 + reviewer_quorum: int = 1 + recall_gate_enabled: bool = True + negative_sample_percent: int = 20 + negative_min_reviewed: int = 5 + min_recall: float = 0.95 + recall_confidence_level: float = 0.95 + + def to_dict(self) -> dict[str, Any]: + return { + "shadow_min_samples": max(1, _safe_int(self.shadow_min_samples, 20)), + "canary_min_samples": max(1, _safe_int(self.canary_min_samples, 100)), + "max_error_rate": _clamp01(_safe_float(self.max_error_rate, 0.02)), + "min_precision": _clamp01(_safe_float(self.min_precision, 0.98)), + "max_precision_drop": _clamp01(_safe_float(self.max_precision_drop, 0.02)), + "canary_traffic_percent": max( + 1, + min(50, _safe_int(self.canary_traffic_percent, 5)), + ), + "reviewer_quorum": max(1, min(2, _safe_int(self.reviewer_quorum, 1))), + "recall_gate_enabled": _safe_bool(self.recall_gate_enabled, True), + "negative_sample_percent": max( + 1, + min(100, _safe_int(self.negative_sample_percent, 20)), + ), + "negative_min_reviewed": max( + 1, + min(10_000, _safe_int(self.negative_min_reviewed, 5)), + ), + "min_recall": _clamp01(_safe_float(self.min_recall, 0.95)), + "recall_confidence_level": _confidence_level( + self.recall_confidence_level + ), + } + + +@dataclass(frozen=True) +class ReleaseEvaluationInput: + total: int + failure_count: int + precision: float | None + baseline_precision: float | None = None + details: dict[str, Any] | None = None + + +def normalize_release_policy(value: Any) -> dict[str, Any]: + source = value if isinstance(value, dict) else {} + return ReleaseGuardPolicy( + shadow_min_samples=_safe_int(source.get("shadow_min_samples"), 20), + canary_min_samples=_safe_int(source.get("canary_min_samples"), 100), + max_error_rate=_safe_float(source.get("max_error_rate"), 0.02), + min_precision=_safe_float(source.get("min_precision"), 0.98), + max_precision_drop=_safe_float(source.get("max_precision_drop"), 0.02), + canary_traffic_percent=_safe_int(source.get("canary_traffic_percent"), 5), + reviewer_quorum=_safe_int(source.get("reviewer_quorum"), 1), + recall_gate_enabled=_safe_bool(source.get("recall_gate_enabled"), True), + negative_sample_percent=_safe_int(source.get("negative_sample_percent"), 20), + negative_min_reviewed=_safe_int(source.get("negative_min_reviewed"), 5), + min_recall=_safe_float(source.get("min_recall"), 0.95), + recall_confidence_level=_safe_float( + source.get("recall_confidence_level"), + 0.95, + ), + ).to_dict() + + +def evaluate_release( + stage: str, + evaluation: ReleaseEvaluationInput, + policy: dict[str, Any], +) -> dict[str, Any]: + reasons: list[str] = [] + total = _optional_int(evaluation.total) + failures = _optional_int(evaluation.failure_count) + precision = _optional_probability(evaluation.precision) + baseline = _optional_probability(evaluation.baseline_precision) + invalid = total is None or failures is None + normalized_total = total if total is not None else 0 + normalized_failures = failures if failures is not None else 0 + if not invalid: + invalid = ( + normalized_total < 0 + or normalized_failures < 0 + or normalized_failures > normalized_total + ) + if evaluation.precision is not None and precision is None: + invalid = True + if evaluation.baseline_precision is not None and baseline is None: + invalid = True + if invalid: + reasons.append("invalid_evaluation_metrics") + return _evaluation_result("failed", normalized_total, 1.0, precision, reasons) + if normalized_total == 0: + return _evaluation_result( + "collecting", + normalized_total, + 0.0, + precision, + ["no_evaluation_samples"], + ) + error_rate = normalized_failures / normalized_total + if error_rate > float(policy["max_error_rate"]): + reasons.append("error_rate_exceeded") + if precision is not None and precision < float(policy["min_precision"]): + reasons.append("precision_below_threshold") + if baseline is not None and precision is not None: + if baseline - precision > float(policy["max_precision_drop"]): + reasons.append("precision_regression_exceeded") + details = evaluation.details if isinstance(evaluation.details, dict) else {} + pending_release_labels = ( + details.get("metric_source") == "release_runtime_telemetry" + and _safe_int(details.get("candidate_pending_label_count"), 0) > 0 + ) + if precision is None and not (pending_release_labels and not reasons): + reasons.append("precision_metric_missing") + recall_collecting_reason = "" + if policy.get("recall_gate_enabled") is True: + recall_status = str(details.get("negative_ground_truth_status") or "").strip() + recall_lower_raw = details.get("recall_lower_bound") + recall_lower = _optional_probability(recall_lower_raw) + if details.get("metric_source") != "release_runtime_telemetry": + recall_collecting_reason = "negative_ground_truth_evidence_missing" + elif recall_lower_raw is not None and recall_lower is None: + reasons.append("invalid_recall_metric") + elif not recall_status.startswith("available") or recall_lower is None: + recall_collecting_reason = recall_status or "recall_metric_missing" + elif recall_lower < float(policy["min_recall"]): + reasons.append("recall_lower_bound_below_threshold") + if reasons: + return _evaluation_result("failed", normalized_total, error_rate, precision, reasons) + if precision is None: + return _evaluation_result( + "collecting", + normalized_total, + error_rate, + precision, + ["precision_metric_missing"], + ) + if recall_collecting_reason: + return _evaluation_result( + "collecting", + normalized_total, + error_rate, + precision, + [recall_collecting_reason], + ) + + minimum = 1 + if stage == "shadow": + minimum = int(policy["shadow_min_samples"]) + elif stage == "canary": + minimum = int(policy["canary_min_samples"]) + if normalized_total < minimum: + return _evaluation_result( + "collecting", + normalized_total, + error_rate, + precision, + ["minimum_sample_not_reached"], + ) + return _evaluation_result( + "passed", + normalized_total, + error_rate, + precision, + ["release_quality_gate_passed"], + ) + + +def _evaluation_result( + status: str, + total: int, + error_rate: float, + precision: float | None, + reasons: list[str], +) -> dict[str, Any]: + return { + "status": status, + "summary": f"release evaluation {status}: {', '.join(reasons)}", + "sample_count": max(0, total), + "error_rate": round(error_rate, 6), + "precision": precision, + "reasons": reasons, + } + + +def _clamp01(value: float) -> float: + try: + parsed = float(value) + except (TypeError, ValueError): + return 0.0 + if not isfinite(parsed): + return 0.0 + return max(0.0, min(1.0, parsed)) + + +def _optional_probability(value: float | None) -> float | None: + if value is None: + return None + try: + parsed = float(value) + except (TypeError, ValueError): + return None + if not isfinite(parsed) or parsed < 0 or parsed > 1: + return None + return parsed + + +def _optional_int(value: Any) -> int | None: + if isinstance(value, bool): + return None + try: + return int(value) + except (TypeError, ValueError, OverflowError): + return None + + +def _safe_int(value: Any, default: int) -> int: + try: + return int(value) + except (TypeError, ValueError, OverflowError): + return default + + +def _safe_bool(value: Any, default: bool) -> bool: + return value if isinstance(value, bool) else default + + +def _confidence_level(value: Any) -> float: + parsed = _safe_float(value, 0.95) + return parsed if parsed in {0.9, 0.95, 0.99} else 0.95 + + +def _safe_float(value: Any, default: float) -> float: + try: + parsed = float(value) + except (TypeError, ValueError, OverflowError): + return default + return parsed if isfinite(parsed) else default diff --git a/server/src/app/services/agent_asset_release_recall.py b/server/src/app/services/agent_asset_release_recall.py new file mode 100644 index 0000000..377d0be --- /dev/null +++ b/server/src/app/services/agent_asset_release_recall.py @@ -0,0 +1,161 @@ +"""发布负样本抽检的保守召回率估计。""" + +from __future__ import annotations + +from dataclasses import dataclass +from math import isfinite, sqrt + + +@dataclass(frozen=True, slots=True) +class ReleaseRecallEstimate: + """候选规则召回率及其保守下界。 + + ``disagreement_false_negative_count`` 是基线已命中、候选未命中的全量复核结果； + ``random_*`` 是候选和基线均未命中人群中的独立随机抽检。两层证据不能直接 + 混成普通样本比例，因此先估算漏检总量，再计算召回率。 + """ + + recall: float | None + recall_lower_bound: float | None + estimated_false_negative_count: float | None + false_negative_upper_bound: float | None + random_false_negative_rate: float | None + random_false_negative_rate_upper_bound: float | None + confidence_level: float + method: str + + +def estimate_release_recall( + *, + true_positive_count: int, + disagreement_false_negative_count: int, + random_negative_population_count: int, + random_reviewed_count: int, + random_false_negative_count: int, + confidence_level: float = 0.95, +) -> ReleaseRecallEstimate: + """使用分层抽检和 Wilson 上界生成保守召回率。 + + 随机层的漏检率用 Wilson 单侧保守上界近似；该上界投影到完整候选负例人群 + 后，再反推召回率下界。调用方必须只传已经达到独立复核法定票数的样本。 + """ + + true_positives = _count(true_positive_count, "true_positive_count") + disagreement_false_negatives = _count( + disagreement_false_negative_count, + "disagreement_false_negative_count", + ) + random_population = _count( + random_negative_population_count, + "random_negative_population_count", + ) + random_reviewed = _count(random_reviewed_count, "random_reviewed_count") + random_false_negatives = _count( + random_false_negative_count, + "random_false_negative_count", + ) + if random_reviewed > random_population: + raise ValueError("random_reviewed_count cannot exceed its population.") + if random_false_negatives > random_reviewed: + raise ValueError("random_false_negative_count cannot exceed reviewed samples.") + + confidence = _confidence(confidence_level) + if random_population and not random_reviewed: + return _unavailable(confidence) + + random_rate = ( + random_false_negatives / random_reviewed if random_reviewed else 0.0 + ) + random_upper = ( + _wilson_upper_bound( + successes=random_false_negatives, + total=random_reviewed, + z=_z_score(confidence), + ) + if random_reviewed + else 0.0 + ) + estimated_false_negatives = ( + float(disagreement_false_negatives) + random_rate * random_population + ) + false_negative_upper = ( + float(disagreement_false_negatives) + random_upper * random_population + ) + recall = _recall(true_positives, estimated_false_negatives) + recall_lower_bound = _recall(true_positives, false_negative_upper) + return ReleaseRecallEstimate( + recall=_rounded(recall), + recall_lower_bound=_rounded(recall_lower_bound), + estimated_false_negative_count=_rounded(estimated_false_negatives), + false_negative_upper_bound=_rounded(false_negative_upper), + random_false_negative_rate=_rounded(random_rate), + random_false_negative_rate_upper_bound=_rounded(random_upper), + confidence_level=confidence, + method="stratified_random_audit_wilson_upper_bound", + ) + + +def _unavailable(confidence: float) -> ReleaseRecallEstimate: + return ReleaseRecallEstimate( + recall=None, + recall_lower_bound=None, + estimated_false_negative_count=None, + false_negative_upper_bound=None, + random_false_negative_rate=None, + random_false_negative_rate_upper_bound=None, + confidence_level=confidence, + method="stratified_random_audit_wilson_upper_bound", + ) + + +def _wilson_upper_bound(*, successes: int, total: int, z: float) -> float: + if total <= 0: + raise ValueError("Wilson interval requires at least one reviewed sample.") + probability = successes / total + z_squared = z * z + denominator = 1.0 + z_squared / total + centre = probability + z_squared / (2.0 * total) + margin = z * sqrt( + probability * (1.0 - probability) / total + + z_squared / (4.0 * total * total) + ) + return min(1.0, max(0.0, (centre + margin) / denominator)) + + +def _recall(true_positives: int, false_negatives: float) -> float | None: + denominator = float(true_positives) + false_negatives + if denominator <= 0: + return None + return true_positives / denominator + + +def _count(value: int, field: str) -> int: + if isinstance(value, bool): + raise ValueError(f"{field} must be a non-negative integer.") + try: + parsed = int(value) + except (TypeError, ValueError, OverflowError) as error: + raise ValueError(f"{field} must be a non-negative integer.") from error + if parsed != value or parsed < 0: + raise ValueError(f"{field} must be a non-negative integer.") + return parsed + + +def _confidence(value: float) -> float: + try: + parsed = float(value) + except (TypeError, ValueError, OverflowError) as error: + raise ValueError("confidence_level must be 0.90, 0.95 or 0.99.") from error + if not isfinite(parsed) or parsed not in {0.9, 0.95, 0.99}: + raise ValueError("confidence_level must be 0.90, 0.95 or 0.99.") + return parsed + + +def _z_score(confidence: float) -> float: + return {0.9: 1.6448536269514722, 0.95: 1.959963984540054, 0.99: 2.5758293035489004}[ + confidence + ] + + +def _rounded(value: float | None) -> float | None: + return None if value is None else round(value, 6) diff --git a/server/src/app/services/agent_asset_release_review.py b/server/src/app/services/agent_asset_release_review.py new file mode 100644 index 0000000..573d1a2 --- /dev/null +++ b/server/src/app/services/agent_asset_release_review.py @@ -0,0 +1,316 @@ +"""Agent 分阶段发布的人工复核队列。""" + +from __future__ import annotations + +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.agent_asset import AgentAsset +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) +from app.services.agent_asset_release_alerts import build_release_alerts +from app.services.agent_asset_release_label_votes import ( + release_reviewer_quorum, + resolve_release_label_votes, +) +from app.services.agent_asset_release_sampling import AgentAssetReleaseSamplingService +from app.services.agent_asset_release_telemetry import AgentAssetReleaseTelemetryService + +_REVIEWABLE_STAGES = {"shadow", "canary", "active"} + + +class AgentAssetReleaseReviewService: + """混排正负样本且隐藏模型结论，禁止发布发起人自审。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def list_pending( + self, + *, + tenant_id: str, + asset_id: str, + limit: int = 50, + ) -> dict[str, Any]: + tenant, asset, state = self._current_release(tenant_id, asset_id) + normalized_limit = _limit(limit) + samples = list( + self.db.scalars( + select(AgentAssetReleaseAuditSample) + .where( + AgentAssetReleaseAuditSample.tenant_id == tenant, + AgentAssetReleaseAuditSample.asset_id == asset.id, + AgentAssetReleaseAuditSample.release_id == state["release_id"], + AgentAssetReleaseAuditSample.stage == state["stage"], + AgentAssetReleaseAuditSample.version == state["candidate_version"], + ) + .order_by( + AgentAssetReleaseAuditSample.selection_score_ppm.asc(), + AgentAssetReleaseAuditSample.created_at.asc(), + AgentAssetReleaseAuditSample.id.asc(), + ) + ).all() + ) + observation_ids = [item.observation_id for item in samples] + observations = { + item.id: item + for item in self.db.scalars( + select(AgentAssetReleaseObservation).where( + AgentAssetReleaseObservation.tenant_id == tenant, + AgentAssetReleaseObservation.id.in_(observation_ids), + ) + ).all() + } if observation_ids else {} + labels: list[AgentAssetReleaseLabel] = [] + if observation_ids: + labels = list( + self.db.scalars( + select(AgentAssetReleaseLabel).where( + AgentAssetReleaseLabel.tenant_id == tenant, + AgentAssetReleaseLabel.observation_id.in_(observation_ids), + ) + .order_by( + AgentAssetReleaseLabel.created_at.asc(), + AgentAssetReleaseLabel.id.asc(), + ) + ).all() + ) + quorum = release_reviewer_quorum(state) + vote_states = resolve_release_label_votes( + labels, + required_reviewers=quorum, + required_reviewers_by_observation={ + item.observation_id: _sample_quorum(item, quorum) for item in samples + }, + ) + pending = [ + item + for item in samples + if vote_states.get(item.observation_id) is None + or vote_states[item.observation_id].label is None + ] + items = [ + self._item( + item, + observations[item.observation_id], + vote_states.get(item.observation_id), + _sample_quorum(item, quorum), + tenant, + ) + for item in pending[:normalized_limit] + if item.observation_id in observations + ] + aggregate = AgentAssetReleaseTelemetryService(self.db).aggregate( + tenant_id=tenant, + asset_id=asset.id, + release_id=state["release_id"], + stage=state["stage"], + version=state["candidate_version"], + ) + pending_observations = [ + observations[item.observation_id] + for item in pending + if item.observation_id in observations + ] + metrics = _metrics(aggregate, pending_observations) + alerts = build_release_alerts( + status=aggregate.status, + rolled_back=False, + reasons=aggregate.reasons, + metrics=metrics, + ) + return { + "asset_id": asset.id, + "release_id": state["release_id"], + "stage": state["stage"], + "version": state["candidate_version"], + "pending_total": len(pending), + "telemetry_status": aggregate.status, + "reasons": list(aggregate.reasons), + "metrics": metrics, + "alerts": alerts, + "items": items, + } + + def record_label( + self, + *, + tenant_id: str, + asset_id: str, + observation_id: str, + label: str, + actor_id: str, + request_id: str, + ) -> AgentAssetReleaseLabel: + tenant, asset, state = self._current_release(tenant_id, asset_id) + actor = _required(actor_id, "actor_id", 160) + if actor == str(state.get("started_by") or "").strip(): + raise PermissionError("发布发起人不能复核自己的候选版本。") + sample = self.db.scalar( + select(AgentAssetReleaseAuditSample).where( + AgentAssetReleaseAuditSample.tenant_id == tenant, + AgentAssetReleaseAuditSample.observation_id == _required( + observation_id, + "observation_id", + 36, + ), + AgentAssetReleaseAuditSample.asset_id == asset.id, + AgentAssetReleaseAuditSample.release_id == state["release_id"], + AgentAssetReleaseAuditSample.stage == state["stage"], + AgentAssetReleaseAuditSample.version == state["candidate_version"], + ) + ) + if sample is None: + raise LookupError("Release audit sample not found.") + ground_truth = { + "confirmed": "risk_present", + "false_positive": "risk_absent", + "risk_present": "risk_present", + "risk_absent": "risk_absent", + }.get(str(label or "").strip().lower()) + if ground_truth is None: + raise ValueError("Blind review requires risk_present or risk_absent.") + return AgentAssetReleaseTelemetryService(self.db).record_blind_review_label( + tenant_id=tenant, + observation_id=sample.observation_id, + ground_truth=ground_truth, # type: ignore[arg-type] + request_id=_required(request_id, "request_id", 160), + actor_id=actor, + ) + + def _current_release( + self, + tenant_id: str, + asset_id: str, + ) -> tuple[str, AgentAsset, dict[str, Any]]: + tenant = _required(tenant_id, "tenant_id", 64) + asset = self.db.scalar( + select(AgentAsset).where( + AgentAsset.id == _required(asset_id, "asset_id", 36), + ( + ((AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == tenant)) + | ((AgentAsset.scope == "platform") & (AgentAsset.tenant_id == "platform")) + ), + ) + ) + if asset is None: + raise LookupError("Agent asset not found.") + config = asset.config_json if isinstance(asset.config_json, dict) else {} + configured_tenant = str(config.get("tenant_id") or "").strip() + if asset.scope == "tenant" and configured_tenant not in {"", tenant}: + raise LookupError("Agent asset not found.") + if asset.scope == "platform" and configured_tenant != tenant: + raise LookupError("Agent asset not found.") + state = config.get("release_guard") + if not isinstance(state, dict): + raise ValueError("Asset has no active release guard state.") + normalized = { + **state, + "release_id": _required(state.get("release_id"), "release_id", 64), + "stage": str(state.get("stage") or "").strip().lower(), + "candidate_version": _required( + state.get("candidate_version"), + "candidate_version", + 30, + ), + } + if normalized["stage"] not in _REVIEWABLE_STAGES: + raise ValueError("Asset is not in a reviewable release stage.") + return tenant, asset, normalized + + def _item( + self, + sample: AgentAssetReleaseAuditSample, + observation: AgentAssetReleaseObservation, + votes: Any, + quorum: int, + tenant_id: str, + ) -> dict[str, Any]: + return { + "sample_id": sample.id, + "observation_id": observation.id, + "source_document_id": AgentAssetReleaseSamplingService( + self.db + ).source_reference(sample=sample, tenant_id=tenant_id), + "rule_code": observation.rule_code, + "business_stage": observation.business_stage, + "prediction_blinded": True, + "reviewer_count": int(votes.reviewer_count if votes is not None else 0), + "required_reviewers": quorum, + "conflicted": bool(votes.conflicted if votes is not None else False), + "created_at": observation.created_at, + } + + +def _required(value: Any, field: str, maximum: int) -> str: + normalized = str(value or "").strip() + if not normalized or len(normalized) > maximum: + raise ValueError(f"{field} is required and must be at most {maximum} characters.") + return normalized + + +def _limit(value: Any) -> int: + if isinstance(value, bool): + raise ValueError("limit must be an integer.") + try: + parsed = int(value) + except (TypeError, ValueError, OverflowError) as error: + raise ValueError("limit must be an integer.") from error + if parsed < 1 or parsed > 100: + raise ValueError("limit must be between 1 and 100.") + return parsed + + +def _sample_quorum(sample: AgentAssetReleaseAuditSample, default: int) -> int: + return 2 if sample.stratum != "candidate_positive_census" else default + + +def _metrics(aggregate: Any, pending: list[AgentAssetReleaseObservation]) -> dict[str, Any]: + observed_count = aggregate.observed_count + oldest_pending = min((item.created_at for item in pending), default=None) + return { + "observed_count": observed_count, + "runtime_failure_count": aggregate.runtime_failure_count, + "runtime_failure_rate": ( + aggregate.runtime_failure_count / observed_count if observed_count else None + ), + "candidate_hit_count": aggregate.candidate_hit_count, + "candidate_labeled_count": aggregate.candidate_labeled_count, + "candidate_pending_label_count": aggregate.candidate_pending_label_count, + "candidate_oldest_pending_at": ( + oldest_pending.isoformat() if oldest_pending is not None else None + ), + "candidate_oldest_pending_age_seconds": _age_seconds(oldest_pending), + "precision": aggregate.precision, + "baseline_hit_count": aggregate.baseline_hit_count, + "baseline_labeled_count": aggregate.baseline_labeled_count, + "baseline_pending_label_count": aggregate.baseline_pending_label_count, + "baseline_precision": aggregate.baseline_precision, + "false_negative_count": aggregate.false_negative_count, + "estimated_false_negative_count": aggregate.estimated_false_negative_count, + "false_negative_upper_bound": aggregate.false_negative_upper_bound, + "negative_sample_count": aggregate.negative_sample_count, + "negative_labeled_count": aggregate.negative_labeled_count, + "negative_pending_label_count": aggregate.negative_pending_label_count, + "random_negative_population_count": aggregate.random_negative_population_count, + "random_negative_sample_count": aggregate.random_negative_sample_count, + "random_negative_labeled_count": aggregate.random_negative_labeled_count, + "recall": aggregate.recall, + "recall_lower_bound": aggregate.recall_lower_bound, + "recall_confidence_level": aggregate.recall_confidence_level, + "recall_method": aggregate.recall_method, + "negative_ground_truth_status": aggregate.negative_ground_truth_status, + } + + +def _age_seconds(value: datetime | None) -> int | None: + if value is None: + return None + normalized = value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + return max(0, int((datetime.now(UTC) - normalized).total_seconds())) diff --git a/server/src/app/services/agent_asset_release_sampling.py b/server/src/app/services/agent_asset_release_sampling.py new file mode 100644 index 0000000..da6a5b5 --- /dev/null +++ b/server/src/app/services/agent_asset_release_sampling.py @@ -0,0 +1,159 @@ +"""发布运行观察的确定性分层抽样与加密来源绑定。""" + +from __future__ import annotations + +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.core.secret_box import decrypt_secret, encrypt_secret +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseObservation, +) +from app.services.agent_asset_release_telemetry_crypto import ( + deterministic_fingerprint, + json_fingerprint, + stable_uuid, +) + +_PPM = 1_000_000 + + +class AgentAssetReleaseSamplingService: + """正例全量、分歧全量、其余负例按不可预测稳定分数抽检。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def ensure_sample( + self, + *, + observation: AgentAssetReleaseObservation, + source_reference: str, + release_state: dict[str, Any], + ) -> AgentAssetReleaseAuditSample | None: + if observation.runtime_status != "completed": + return None + existing = self._sample_for_observation(observation) + if existing is not None: + return existing + + stratum, probability = _sampling_plan(observation, release_state) + score = _selection_score(observation) + if stratum == "candidate_negative_random" and score >= probability: + return None + identity = { + "tenant_id": observation.tenant_id, + "observation_id": observation.id, + "asset_id": observation.asset_id, + "release_id": observation.release_id, + "stage": observation.stage, + "version": observation.version, + "stratum": stratum, + "sampling_probability_ppm": probability, + "selection_score_ppm": score, + } + payload_fingerprint = json_fingerprint( + {**identity, "source_fingerprint": observation.source_fingerprint} + ) + idempotency_key = ( + "aras:" + + deterministic_fingerprint( + observation.tenant_id, + observation.id, + observation.release_id, + )[:59] + ) + item = AgentAssetReleaseAuditSample( + id=stable_uuid(idempotency_key), + **identity, + source_reference_encrypted=encrypt_secret( + _required(source_reference, "source_reference", 500) + ), + idempotency_key=idempotency_key, + payload_fingerprint=payload_fingerprint, + created_at=datetime.now(UTC), + ) + try: + with self.db.begin_nested(): + self.db.add(item) + self.db.flush() + except IntegrityError: + replay = self._sample_for_observation(observation) + if replay is None or replay.payload_fingerprint != payload_fingerprint: + raise + return replay + return item + + def source_reference( + self, + *, + sample: AgentAssetReleaseAuditSample, + tenant_id: str, + ) -> str: + if sample.tenant_id != _required(tenant_id, "tenant_id", 64): + raise LookupError("Release audit sample not found.") + return _required( + decrypt_secret(sample.source_reference_encrypted), + "source_reference", + 500, + ) + + def _sample_for_observation( + self, + observation: AgentAssetReleaseObservation, + ) -> AgentAssetReleaseAuditSample | None: + return self.db.scalar( + select(AgentAssetReleaseAuditSample).where( + AgentAssetReleaseAuditSample.tenant_id == observation.tenant_id, + AgentAssetReleaseAuditSample.observation_id == observation.id, + ) + ) + + +def _sampling_plan( + observation: AgentAssetReleaseObservation, + release_state: dict[str, Any], +) -> tuple[str, int]: + if observation.candidate_hit: + return "candidate_positive_census", _PPM + if observation.baseline_hit is True: + return "candidate_disagreement_census", _PPM + policy = release_state.get("policy") + source = policy if isinstance(policy, dict) else {} + percent = _bounded_int(source.get("negative_sample_percent"), default=20, low=1, high=100) + return "candidate_negative_random", percent * 10_000 + + +def _selection_score(observation: AgentAssetReleaseObservation) -> int: + digest = deterministic_fingerprint( + "release-audit-sample:v1", + observation.tenant_id, + observation.asset_id, + observation.release_id, + observation.stage, + observation.version, + observation.source_fingerprint, + ) + return int(digest[:16], 16) % _PPM + + +def _bounded_int(value: Any, *, default: int, low: int, high: int) -> int: + if isinstance(value, bool): + return default + try: + parsed = int(value) + except (TypeError, ValueError, OverflowError): + return default + return max(low, min(high, parsed)) + + +def _required(value: Any, field: str, maximum: int) -> str: + normalized = str(value or "").strip() + if not normalized or len(normalized) > maximum: + raise ValueError(f"{field} is required and must be at most {maximum} characters.") + return normalized diff --git a/server/src/app/services/agent_asset_release_scheduler.py b/server/src/app/services/agent_asset_release_scheduler.py new file mode 100644 index 0000000..3ba3e88 --- /dev/null +++ b/server/src/app/services/agent_asset_release_scheduler.py @@ -0,0 +1,213 @@ +"""租户级 Agent 资产真实发布遥测周期监控。""" + +from __future__ import annotations + +import os +import threading +from collections.abc import Callable +from typing import Any + +from sqlalchemy import select, text +from sqlalchemy.orm import Session + +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.core.logging import get_logger +from app.db.session import get_session_factory +from app.models.agent_asset import AgentAsset +from app.services.agent_asset_release_monitor import AgentAssetReleaseMonitor + +logger = get_logger("app.services.agent_asset_release_scheduler") + +_MONITORED_STAGES = {"shadow", "canary", "active"} +_SCHEDULER_LEASE_KEY = "x-financial:agent-asset-release-scheduler:v1" + + +class AgentAssetReleaseScheduler: + def __init__( + self, + *, + session_factory: Callable[[], Session] | None = None, + ) -> None: + self._interval_seconds = max( + 30, + _env_int("X_FINANCIAL_RELEASE_MONITOR_INTERVAL_SECONDS", 60), + ) + self._initial_delay_seconds = max( + 1, + _env_int("X_FINANCIAL_RELEASE_MONITOR_INITIAL_DELAY_SECONDS", 15), + ) + self._batch_size = min( + 500, + max(1, _env_int("X_FINANCIAL_RELEASE_MONITOR_BATCH_SIZE", 100)), + ) + self._session_factory = session_factory + self._tenant_cursors: dict[str, str] = {} + self._stop_event = threading.Event() + self._thread: threading.Thread | None = None + self._lock = threading.Lock() + + def start(self) -> None: + with self._lock: + if self._thread is not None and self._thread.is_alive(): + return + self._stop_event.clear() + self._thread = threading.Thread( + target=self._run_loop, + name="agent-asset-release-scheduler", + daemon=True, + ) + self._thread.start() + logger.info( + "Agent asset release scheduler started interval=%ss batch=%s", + self._interval_seconds, + self._batch_size, + ) + + def shutdown(self) -> None: + with self._lock: + thread = self._thread + self._thread = None + self._stop_event.set() + if thread is not None and thread.is_alive(): + thread.join(timeout=3) + logger.info("Agent asset release scheduler stopped") + + def _run_loop(self) -> None: + if self._stop_event.wait(self._initial_delay_seconds): + return + while not self._stop_event.is_set(): + try: + self._run_once() + except Exception: # pragma: no cover - 调度器保底日志 + logger.exception("Scheduled Agent asset release monitoring failed") + if self._stop_event.wait(self._interval_seconds): + break + + def _run_once(self) -> dict[str, Any]: + factory = self._session_factory or get_session_factory() + db = factory() + lease_acquired = False + try: + lease_acquired = self._try_acquire_lease(db) + if not lease_acquired: + logger.info("Agent asset release monitor cycle skipped: lease held by peer") + return { + "tenants": 0, + "scanned": 0, + "evaluated": 0, + "collecting": 0, + "rolled_back": 0, + "errors": 0, + "global_skipped": 0, + "leader_skipped": 1, + } + tenants, global_count = self._tenant_targets(db) + summary: dict[str, Any] = { + "tenants": len(tenants), + "scanned": 0, + "evaluated": 0, + "collecting": 0, + "rolled_back": 0, + "errors": 0, + "global_skipped": global_count, + } + monitor = AgentAssetReleaseMonitor(db) + for tenant_id in tenants: + result = monitor.batch_evaluate( + tenant_id=tenant_id, + actor="release-telemetry-scheduler", + limit=self._batch_size, + after_asset_id=self._tenant_cursors.get(tenant_id), + ) + next_cursor = str(result.get("next_cursor") or "").strip() + if next_cursor: + self._tenant_cursors[tenant_id] = next_cursor + for key in ("scanned", "evaluated", "collecting", "rolled_back"): + summary[key] += int(result[key]) + summary["errors"] += len(result["errors"]) + if any(summary[key] for key in ("evaluated", "rolled_back", "errors")): + logger.info("Agent asset release monitor cycle summary=%s", summary) + if global_count: + logger.warning( + "Global release assets skipped by tenant scheduler count=%s", + global_count, + ) + return summary + except Exception: + db.rollback() + raise + finally: + if lease_acquired: + try: + self._release_lease(db) + except Exception: # pragma: no cover - 数据库连接故障兜底 + db.rollback() + logger.exception("Failed to release Agent asset scheduler lease") + db.close() + + @staticmethod + def _try_acquire_lease(db: Session) -> bool: + bind = db.get_bind() + if bind is None or bind.dialect.name != "postgresql": + return True + return bool( + db.scalar( + text("SELECT pg_try_advisory_lock(hashtextextended(:lease_key, 0))"), + {"lease_key": _SCHEDULER_LEASE_KEY}, + ) + ) + + @staticmethod + def _release_lease(db: Session) -> None: + bind = db.get_bind() + if bind is None or bind.dialect.name != "postgresql": + return + released = db.scalar( + text("SELECT pg_advisory_unlock(hashtextextended(:lease_key, 0))"), + {"lease_key": _SCHEDULER_LEASE_KEY}, + ) + if not released: + logger.warning("Agent asset release scheduler lease was not owned at release time") + + def _tenant_targets(self, db: Session) -> tuple[list[str], int]: + rows = list( + db.execute( + select(AgentAsset.tenant_id, AgentAsset.scope, AgentAsset.config_json) + .where( + AgentAsset.asset_type == AgentAssetType.RULE.value, + AgentAsset.domain == AgentAssetDomain.EXPENSE.value, + AgentAsset.status != AgentAssetStatus.DISABLED.value, + AgentAsset.config_json["detail_mode"].as_string() == "json_risk", + AgentAsset.config_json["release_guard"]["stage"] + .as_string() + .in_(sorted(_MONITORED_STAGES)), + ) + ).all() + ) + tenants: set[str] = set() + global_count = 0 + for asset_tenant_id, asset_scope, value in rows: + config = value if isinstance(value, dict) else {} + state = config.get("release_guard") + if ( + str(config.get("detail_mode") or "").strip().lower() != "json_risk" + or config.get("enabled") is False + or not isinstance(state, dict) + or str(state.get("stage") or "").strip().lower() not in _MONITORED_STAGES + ): + continue + if asset_scope == "tenant" and asset_tenant_id != "platform": + tenants.add(str(asset_tenant_id)) + elif asset_scope == "platform" and asset_tenant_id == "platform": + global_count += 1 + return sorted(tenants), global_count + + +def _env_int(name: str, default: int) -> int: + try: + return int(str(os.environ.get(name) or default).strip()) + except (TypeError, ValueError, OverflowError): + return default + + +agent_asset_release_scheduler = AgentAssetReleaseScheduler() diff --git a/server/src/app/services/agent_asset_release_telemetry.py b/server/src/app/services/agent_asset_release_telemetry.py new file mode 100644 index 0000000..96bdffd --- /dev/null +++ b/server/src/app/services/agent_asset_release_telemetry.py @@ -0,0 +1,799 @@ +"""从真实风险规则运行与人工处置构建分阶段发布质量指标。""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import UTC, datetime +from typing import Any, Literal + +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.models.agent_asset import AgentAsset +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) +from app.models.risk_disposition import RiskDispositionEvent +from app.models.risk_observation import RiskObservation +from app.services.agent_asset_release_guard import ReleaseEvaluationInput +from app.services.agent_asset_release_sampling import AgentAssetReleaseSamplingService +from app.services.agent_asset_release_telemetry_crypto import ( + deterministic_fingerprint as _fingerprint, +) +from app.services.agent_asset_release_telemetry_crypto import ( + json_fingerprint as _json_fingerprint, +) +from app.services.agent_asset_release_telemetry_crypto import ( + release_pseudonym_fingerprint, + release_source_fingerprint, + release_source_fingerprints, +) +from app.services.agent_asset_release_telemetry_crypto import ( + stable_uuid as _stable_uuid, +) +from app.services.agent_asset_release_telemetry_values import ( + release_stage as _stage, +) +from app.services.agent_asset_release_telemetry_values import ( + required_value as _required, +) +from app.services.agent_asset_release_telemetry_values import ( + safe_code as _safe_code, +) +from app.services.agent_asset_release_telemetry_values import ( + strict_bool as _strict_bool, +) + +ReleaseTelemetryStage = Literal["shadow", "canary", "active"] +ReleaseTelemetryLabelValue = Literal[ + "confirmed", + "false_positive", + "risk_present", + "risk_absent", +] + +_BUSINESS_STAGES = {"expense_application", "reimbursement"} +_FAILURE_CODES = { + "none", + "evaluator_error", + "artifact_integrity_error", + "unsupported_evaluator", + "timeout", +} + + +class ReleaseTelemetryError(RuntimeError): + """发布遥测输入不可信或与当前发布状态冲突。""" + + +class ReleaseTelemetryIdempotencyConflict(ReleaseTelemetryError): + """幂等键已被不同事实使用。""" + + +class ReleaseTelemetryStaleRelease(ReleaseTelemetryError): + """运行观察不再属于资产当前的 release/stage/version。""" + + +class ReleaseTelemetryCollecting(ReleaseTelemetryError): + """真实标签证据尚未满足发布评测输入要求。""" + + +@dataclass(frozen=True, slots=True) +class ReleaseObservationInput: + tenant_id: str + asset_id: str + release_id: str + stage: ReleaseTelemetryStage + version: str + rule_code: str + source_key: str + candidate_hit: bool + baseline_hit: bool | None = None + runtime_status: Literal["completed", "failed"] = "completed" + failure_code: str = "none" + business_stage: str = "reimbursement" + + +@dataclass(frozen=True, slots=True) +class ReleaseTelemetryAggregate: + tenant_id: str + asset_id: str + release_id: str + stage: str + version: str + status: Literal["collecting", "ready"] + reasons: tuple[str, ...] + observed_count: int + completed_count: int + runtime_failure_count: int + candidate_hit_count: int + candidate_labeled_count: int + candidate_pending_label_count: int + candidate_confirmed_count: int + candidate_false_positive_count: int + precision: float | None + baseline_hit_count: int + baseline_labeled_count: int + baseline_pending_label_count: int + baseline_confirmed_count: int + baseline_false_positive_count: int + baseline_precision: float | None + false_negative_count: int | None = None + estimated_false_negative_count: float | None = None + false_negative_upper_bound: float | None = None + negative_sample_count: int = 0 + negative_labeled_count: int = 0 + negative_pending_label_count: int = 0 + random_negative_population_count: int = 0 + random_negative_sample_count: int = 0 + random_negative_labeled_count: int = 0 + random_negative_false_negative_count: int = 0 + recall: float | None = None + recall_lower_bound: float | None = None + recall_confidence_level: float = 0.95 + recall_method: str = "unavailable" + negative_ground_truth_status: str = "unavailable" + + def to_release_evaluation_input(self) -> ReleaseEvaluationInput: + """只在所有候选正例已有可信标签时交给 Release Guard。""" + + if self.status != "ready": + raise ReleaseTelemetryCollecting( + "发布遥测仍在采集真实标签，不能把未标注样本当作成功。" + ) + guarded_precision = ( + self.precision if self.candidate_pending_label_count == 0 else None + ) + if guarded_precision is None and self.runtime_failure_count == 0: + raise ReleaseTelemetryCollecting( + "发布遥测缺少完整可信标签，不能把部分精度当作成功。" + ) + return ReleaseEvaluationInput( + total=self.observed_count, + failure_count=self.runtime_failure_count, + precision=guarded_precision, + baseline_precision=self.baseline_precision, + details={ + "metric_source": "release_runtime_telemetry", + "release_id": self.release_id, + "stage": self.stage, + "version": self.version, + "completed_count": self.completed_count, + "candidate_hit_count": self.candidate_hit_count, + "candidate_labeled_count": self.candidate_labeled_count, + "candidate_pending_label_count": self.candidate_pending_label_count, + "baseline_hit_count": self.baseline_hit_count, + "baseline_labeled_count": self.baseline_labeled_count, + "baseline_precision_status": ( + "available" if self.baseline_precision is not None else "unavailable" + ), + "false_negative_count": self.false_negative_count, + "estimated_false_negative_count": self.estimated_false_negative_count, + "false_negative_upper_bound": self.false_negative_upper_bound, + "negative_sample_count": self.negative_sample_count, + "negative_labeled_count": self.negative_labeled_count, + "negative_pending_label_count": self.negative_pending_label_count, + "random_negative_population_count": self.random_negative_population_count, + "random_negative_sample_count": self.random_negative_sample_count, + "random_negative_labeled_count": self.random_negative_labeled_count, + "recall": self.recall, + "recall_lower_bound": self.recall_lower_bound, + "recall_confidence_level": self.recall_confidence_level, + "recall_method": self.recall_method, + "negative_ground_truth_status": self.negative_ground_truth_status, + }, + ) + + +class AgentAssetReleaseTelemetryService: + """生产真实运行样本，接收可信人工结论并生成保守发布指标。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def record_observation( + self, + payload: ReleaseObservationInput, + ) -> AgentAssetReleaseObservation: + tenant_id = _required(payload.tenant_id, "tenant_id", 64) + asset_id = _required(payload.asset_id, "asset_id", 36) + release_id = _safe_code(payload.release_id, "release_id", 64) + stage = _stage(payload.stage) + version = _safe_code(payload.version, "version", 30) + rule_code = _safe_code(payload.rule_code, "rule_code", 100) + source_key = _required(payload.source_key, "source_key", 500) + business_stage = str(payload.business_stage or "").strip().lower() + if business_stage not in _BUSINESS_STAGES: + raise ValueError("business_stage must be expense_application or reimbursement.") + runtime_status = str(payload.runtime_status or "").strip().lower() + if runtime_status not in {"completed", "failed"}: + raise ValueError("runtime_status must be completed or failed.") + failure_code = str(payload.failure_code or "none").strip().lower() + if failure_code not in _FAILURE_CODES: + raise ValueError("failure_code is not an approved structured code.") + if runtime_status == "completed" and failure_code != "none": + raise ValueError("completed observation cannot contain a failure code.") + if runtime_status == "failed" and failure_code == "none": + raise ValueError("failed observation requires a structured failure code.") + candidate_hit = _strict_bool(payload.candidate_hit, "candidate_hit") + baseline_hit = ( + None + if payload.baseline_hit is None + else _strict_bool(payload.baseline_hit, "baseline_hit") + ) + + _asset, release_state = self._require_current_release( + tenant_id=tenant_id, + asset_id=asset_id, + release_id=release_id, + stage=stage, + version=version, + rule_code=rule_code, + ) + source_fingerprint = release_source_fingerprint( + tenant_id=tenant_id, + source_key=source_key, + rule_code=rule_code, + ) + identity = { + "tenant_id": tenant_id, + "asset_id": asset_id, + "release_id": release_id, + "stage": stage, + "version": version, + "rule_code": rule_code, + "business_stage": business_stage, + "source_fingerprint": source_fingerprint, + "candidate_hit": candidate_hit, + "baseline_hit": baseline_hit, + "runtime_status": runtime_status, + "failure_code": failure_code, + } + payload_fingerprint = _json_fingerprint(identity) + observation_key_hash = _fingerprint( + "observation", + tenant_id, + asset_id, + release_id, + stage, + version, + rule_code, + business_stage, + source_fingerprint, + ) + idempotency_key = f"aro:{observation_key_hash}" + replay = self._observation_replay(tenant_id, idempotency_key, payload_fingerprint) + if replay is not None: + with self.db.begin_nested(): + AgentAssetReleaseSamplingService(self.db).ensure_sample( + observation=replay, + source_reference=source_key, + release_state=release_state, + ) + return replay + + item = AgentAssetReleaseObservation( + id=_stable_uuid(idempotency_key), + **identity, + source_kind="expense_claim_risk", + idempotency_key=idempotency_key, + payload_fingerprint=payload_fingerprint, + created_at=datetime.now(UTC), + ) + try: + with self.db.begin_nested(): + self.db.add(item) + self.db.flush() + AgentAssetReleaseSamplingService(self.db).ensure_sample( + observation=item, + source_reference=source_key, + release_state=release_state, + ) + except IntegrityError: + replay = self._observation_replay( + tenant_id, + idempotency_key, + payload_fingerprint, + ) + if replay is None: + raise + with self.db.begin_nested(): + AgentAssetReleaseSamplingService(self.db).ensure_sample( + observation=replay, + source_reference=source_key, + release_state=release_state, + ) + return replay + return item + + def record_expense_risk_result( + self, + *, + tenant_id: str, + claim_id: str, + result: dict[str, Any], + business_stage: str = "reimbursement", + ) -> list[AgentAssetReleaseObservation]: + """把现有 evaluate_platform_risk_rules 返回值转换成真实遥测样本。""" + + normalized_tenant = _required(tenant_id, "tenant_id", 64) + normalized_claim = _required(claim_id, "claim_id", 100) + flags = [item for item in result.get("flags", []) if isinstance(item, dict)] + recorded: list[AgentAssetReleaseObservation] = [] + for raw in result.get("shadow_evaluations", []): + if not isinstance(raw, dict): + raise ValueError("shadow_evaluations must contain structured objects.") + if str(raw.get("release_stage") or "").strip().lower() != "shadow": + raise ValueError("shadow_evaluations contains a non-shadow release sample.") + candidate_hit = _strict_bool(raw.get("hit"), "hit") + asset_id = _required(raw.get("asset_id"), "asset_id", 36) + rule_code = _safe_code(raw.get("rule_code"), "rule_code", 100) + version = _safe_code(raw.get("rule_version"), "rule_version", 30) + state = self._release_state_for_asset(normalized_tenant, asset_id) + baseline_version = str(state.get("previous_version") or "").strip() + baseline_hit = any( + str(flag.get("rule_code") or "").strip() == rule_code + and str(flag.get("rule_version") or "").strip() == baseline_version + and str(flag.get("release_mode") or "enforced").strip() == "enforced" + for flag in flags + ) + recorded.append( + self.record_observation( + ReleaseObservationInput( + tenant_id=normalized_tenant, + asset_id=asset_id, + release_id=_required(state.get("release_id"), "release_id", 64), + stage="shadow", + version=version, + rule_code=rule_code, + source_key=normalized_claim, + candidate_hit=candidate_hit, + baseline_hit=baseline_hit, + business_stage=business_stage, + ) + ) + ) + + for flag in flags: + stage = str(flag.get("release_stage") or "").strip().lower() + if stage not in {"canary", "active"}: + continue + rule_code = _safe_code(flag.get("rule_code"), "rule_code", 100) + asset = self._asset_for_rule_code(normalized_tenant, rule_code) + state = self._release_state(asset) + version = _safe_code(flag.get("rule_version"), "rule_version", 30) + recorded.append( + self.record_observation( + ReleaseObservationInput( + tenant_id=normalized_tenant, + asset_id=asset.id, + release_id=_required(state.get("release_id"), "release_id", 64), + stage=stage, # type: ignore[arg-type] + version=version, + rule_code=rule_code, + source_key=normalized_claim, + candidate_hit=True, + baseline_hit=None, + business_stage=business_stage, + ) + ) + ) + return recorded + + def record_manifest_evaluation( + self, + *, + tenant_id: str, + claim_id: str, + manifest: dict[str, Any], + hit: bool, + baseline_hit: bool | None = None, + runtime_status: Literal["completed", "failed"] = "completed", + failure_code: str = "none", + business_stage: str = "reimbursement", + ) -> AgentAssetReleaseObservation: + """在规则执行循环内记录候选命中或未命中，覆盖 Canary 的负样本。""" + + normalized_tenant = _required(tenant_id, "tenant_id", 64) + asset_id = _required(manifest.get("_rule_asset_id"), "asset_id", 36) + stage = _stage(manifest.get("_release_stage")) + mode = str(manifest.get("_release_mode") or "").strip().lower() + if (stage == "shadow" and mode != "shadow") or ( + stage in {"canary", "active"} and mode != "enforced" + ): + raise ValueError("Only the candidate route can emit release telemetry.") + version = _safe_code(manifest.get("_rule_version"), "rule_version", 30) + rule_code = _safe_code(manifest.get("rule_code"), "rule_code", 100) + state = self._release_state_for_asset(normalized_tenant, asset_id) + return self.record_observation( + ReleaseObservationInput( + tenant_id=normalized_tenant, + asset_id=asset_id, + release_id=_required(state.get("release_id"), "release_id", 64), + stage=stage, # type: ignore[arg-type] + version=version, + rule_code=rule_code, + source_key=_required(claim_id, "claim_id", 100), + candidate_hit=_strict_bool(hit, "hit") if runtime_status == "completed" else False, + baseline_hit=baseline_hit, + runtime_status=runtime_status, + failure_code=failure_code, + business_stage=business_stage, + ) + ) + + def record_review_label( + self, + *, + tenant_id: str, + observation_id: str, + label: Literal["confirmed", "false_positive"], + request_id: str, + actor_id: str, + ) -> AgentAssetReleaseLabel: + """记录专用发布复核队列给出的类型化结论，不接收评论或业务正文。""" + + return self._record_label( + tenant_id=tenant_id, + observation_id=observation_id, + label=label, + verification_source="release_review", + source_event_key=_required(request_id, "request_id", 160), + actor_key=_required(actor_id, "actor_id", 160), + ) + + def record_blind_review_label( + self, + *, + tenant_id: str, + observation_id: str, + ground_truth: Literal["risk_present", "risk_absent"], + request_id: str, + actor_id: str, + ) -> AgentAssetReleaseLabel: + """记录不披露候选结论的独立业务真值。""" + + return self._record_label( + tenant_id=tenant_id, + observation_id=observation_id, + label=ground_truth, + verification_source="blind_release_review", + source_event_key=_required(request_id, "request_id", 160), + actor_key=_required(actor_id, "actor_id", 160), + ) + + def record_risk_disposition_label( + self, + *, + tenant_id: str, + observation_id: str, + disposition_event_id: str, + ) -> AgentAssetReleaseLabel: + """只接受数据库中真实存在的 confirm/false_positive 处置事件。""" + + normalized_tenant = _required(tenant_id, "tenant_id", 64) + telemetry = self._observation(normalized_tenant, observation_id) + event = self.db.scalar( + select(RiskDispositionEvent).where( + RiskDispositionEvent.id == disposition_event_id, + RiskDispositionEvent.tenant_id == normalized_tenant, + ) + ) + if event is None: + raise LookupError("Risk disposition event not found.") + if event.action not in {"confirm", "false_positive"}: + raise ValueError("Only typed confirm/false_positive events can label release samples.") + risk_observation = self.db.scalar( + select(RiskObservation).where( + RiskObservation.id == event.observation_id, + RiskObservation.tenant_id == normalized_tenant, + ) + ) + if risk_observation is None: + raise LookupError("Risk observation not found.") + rule_code = str((risk_observation.decision_trace_json or {}).get("rule_code") or "").strip() + if not rule_code and risk_observation.policy_refs_json: + rule_code = str(risk_observation.policy_refs_json[0] or "").strip() + expected_sources = release_source_fingerprints( + tenant_id=normalized_tenant, + source_key=_required(risk_observation.claim_id, "claim_id", 100), + rule_code=rule_code, + ) + if rule_code != telemetry.rule_code or telemetry.source_fingerprint not in expected_sources: + raise PermissionError("Risk disposition does not belong to this release observation.") + state = self._release_state_for_asset(normalized_tenant, telemetry.asset_id) + trusted_versions = { + telemetry.version, + str(state.get("previous_version") or "").strip(), + } + if str(risk_observation.algorithm_version or "").strip() not in trusted_versions: + raise PermissionError("Risk disposition version does not match the release sample.") + return self._record_label( + tenant_id=normalized_tenant, + observation_id=telemetry.id, + label="confirmed" if event.action == "confirm" else "false_positive", + verification_source="typed_risk_disposition", + source_event_key=event.id, + actor_key=event.actor_id, + ) + + def aggregate( + self, + *, + tenant_id: str, + asset_id: str, + release_id: str, + stage: ReleaseTelemetryStage, + version: str, + ) -> ReleaseTelemetryAggregate: + from app.services.agent_asset_release_aggregation import ( + build_release_telemetry_aggregate, + ) + + normalized_tenant = _required(tenant_id, "tenant_id", 64) + normalized_asset = _required(asset_id, "asset_id", 36) + normalized_release = _safe_code(release_id, "release_id", 64) + normalized_stage = _stage(stage) + normalized_version = _safe_code(version, "version", 30) + _asset, release_state = self._require_current_release( + tenant_id=normalized_tenant, + asset_id=normalized_asset, + release_id=normalized_release, + stage=normalized_stage, + version=normalized_version, + ) + return build_release_telemetry_aggregate( + db=self.db, + tenant_id=normalized_tenant, + asset_id=normalized_asset, + release_id=normalized_release, + stage=normalized_stage, + version=normalized_version, + release_state=release_state, + ) + + def _record_label( + self, + *, + tenant_id: str, + observation_id: str, + label: str, + verification_source: str, + source_event_key: str, + actor_key: str, + ) -> AgentAssetReleaseLabel: + normalized_tenant = _required(tenant_id, "tenant_id", 64) + normalized_label = str(label or "").strip().lower() + if normalized_label not in { + "confirmed", + "false_positive", + "risk_present", + "risk_absent", + }: + raise ValueError("label is not an approved typed release ground truth.") + if verification_source == "blind_release_review": + if normalized_label not in {"risk_present", "risk_absent"}: + raise ValueError( + "Blind release review requires risk_present or risk_absent." + ) + elif normalized_label not in {"confirmed", "false_positive"}: + raise ValueError( + "Typed dispositions and release reviews require confirmed or " + "false_positive." + ) + observation = self._observation(normalized_tenant, observation_id) + self._require_current_release( + tenant_id=normalized_tenant, + asset_id=observation.asset_id, + release_id=observation.release_id, + stage=observation.stage, + version=observation.version, + rule_code=observation.rule_code, + lock=True, + ) + sample = self.db.scalar( + select(AgentAssetReleaseAuditSample).where( + AgentAssetReleaseAuditSample.tenant_id == normalized_tenant, + AgentAssetReleaseAuditSample.observation_id == observation.id, + ) + ) + if verification_source == "blind_release_review" and sample is None: + raise ValueError("Blind review requires a selected release audit sample.") + if not observation.candidate_hit and observation.baseline_hit is not True: + if verification_source != "blind_release_review": + raise ValueError( + "Negative executions require independent ground truth from a selected " + "blind-review sample." + ) + source_event_fingerprint = release_pseudonym_fingerprint( + "label-source", + normalized_tenant, + source_event_key, + ) + actor_fingerprint = release_pseudonym_fingerprint( + "label-actor", + normalized_tenant, + actor_key, + ) + identity = { + "tenant_id": normalized_tenant, + "observation_id": observation.id, + "asset_id": observation.asset_id, + "release_id": observation.release_id, + "stage": observation.stage, + "version": observation.version, + "label": normalized_label, + "verification_source": verification_source, + "source_event_fingerprint": source_event_fingerprint, + "actor_fingerprint": actor_fingerprint, + } + payload_fingerprint = _json_fingerprint(identity) + label_key_hash = _fingerprint( + "label", + normalized_tenant, + observation.id, + source_event_fingerprint, + ) + idempotency_key = f"arl:{label_key_hash}" + replay = self._label_replay(normalized_tenant, idempotency_key, payload_fingerprint) + if replay is not None: + return replay + item = AgentAssetReleaseLabel( + id=_stable_uuid(idempotency_key), + **identity, + idempotency_key=idempotency_key, + payload_fingerprint=payload_fingerprint, + created_at=datetime.now(UTC), + ) + try: + with self.db.begin_nested(): + self.db.add(item) + self.db.flush() + except IntegrityError: + replay = self._label_replay( + normalized_tenant, + idempotency_key, + payload_fingerprint, + ) + if replay is None: + raise + return replay + return item + + def _observation(self, tenant_id: str, observation_id: str) -> AgentAssetReleaseObservation: + item = self.db.scalar( + select(AgentAssetReleaseObservation).where( + AgentAssetReleaseObservation.tenant_id == tenant_id, + AgentAssetReleaseObservation.id == observation_id, + ) + ) + if item is None: + raise LookupError("Release observation not found.") + return item + + def _observation_replay( + self, + tenant_id: str, + idempotency_key: str, + payload_fingerprint: str, + ) -> AgentAssetReleaseObservation | None: + item = self.db.scalar( + select(AgentAssetReleaseObservation).where( + AgentAssetReleaseObservation.tenant_id == tenant_id, + AgentAssetReleaseObservation.idempotency_key == idempotency_key, + ) + ) + if item is not None and item.payload_fingerprint != payload_fingerprint: + raise ReleaseTelemetryIdempotencyConflict( + "Observation idempotency key was reused with different facts." + ) + return item + + def _label_replay( + self, + tenant_id: str, + idempotency_key: str, + payload_fingerprint: str, + ) -> AgentAssetReleaseLabel | None: + item = self.db.scalar( + select(AgentAssetReleaseLabel).where( + AgentAssetReleaseLabel.tenant_id == tenant_id, + AgentAssetReleaseLabel.idempotency_key == idempotency_key, + ) + ) + if item is not None and item.payload_fingerprint != payload_fingerprint: + raise ReleaseTelemetryIdempotencyConflict( + "Label idempotency key was reused with different facts." + ) + return item + + def _require_current_release( + self, + *, + tenant_id: str, + asset_id: str, + release_id: str, + stage: str, + version: str, + rule_code: str | None = None, + lock: bool = False, + ) -> tuple[AgentAsset, dict[str, Any]]: + statement = select(AgentAsset).where( + AgentAsset.id == asset_id, + ( + ((AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == tenant_id)) + | ((AgentAsset.scope == "platform") & (AgentAsset.tenant_id == "platform")) + ), + ) + bind = self.db.get_bind() + if lock and bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update().execution_options(populate_existing=True) + asset = self.db.scalar(statement) + if asset is None: + raise LookupError("Agent asset not found.") + configured_tenant = str((asset.config_json or {}).get("tenant_id") or "").strip() + if asset.scope == "tenant" and configured_tenant not in {"", tenant_id}: + raise LookupError("Agent asset not found.") + if asset.scope == "platform" and configured_tenant != tenant_id: + raise LookupError("Agent asset not found.") + if rule_code is not None and str(asset.code or "").strip() != rule_code: + raise ReleaseTelemetryStaleRelease("Rule code no longer matches the release asset.") + state = self._release_state(asset) + if ( + str(state.get("release_id") or "").strip() != release_id + or str(state.get("stage") or "").strip() != stage + or str(state.get("candidate_version") or "").strip() != version + ): + raise ReleaseTelemetryStaleRelease( + "Observation or label targets a stale release/stage/version." + ) + return asset, state + + def _release_state_for_asset(self, tenant_id: str, asset_id: str) -> dict[str, Any]: + asset = self.db.scalar( + select(AgentAsset).where( + AgentAsset.id == asset_id, + ( + ((AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == tenant_id)) + | ((AgentAsset.scope == "platform") & (AgentAsset.tenant_id == "platform")) + ), + ) + ) + if asset is None: + raise LookupError("Agent asset not found.") + configured_tenant = str((asset.config_json or {}).get("tenant_id") or "").strip() + if asset.scope == "tenant" and configured_tenant not in {"", tenant_id}: + raise LookupError("Agent asset not found.") + if asset.scope == "platform" and configured_tenant != tenant_id: + raise LookupError("Agent asset not found.") + return self._release_state(asset) + + def _asset_for_rule_code(self, tenant_id: str, rule_code: str) -> AgentAsset: + asset = self.db.scalar( + select(AgentAsset) + .where( + AgentAsset.code == rule_code, + ( + ((AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == tenant_id)) + | ((AgentAsset.scope == "platform") & (AgentAsset.tenant_id == "platform")) + ), + ) + .order_by(AgentAsset.scope.desc()) + ) + if asset is None: + raise LookupError("Agent asset not found.") + configured_tenant = str((asset.config_json or {}).get("tenant_id") or "").strip() + if asset.scope == "tenant" and configured_tenant not in {"", tenant_id}: + raise LookupError("Agent asset not found.") + if asset.scope == "platform" and configured_tenant != tenant_id: + raise LookupError("Agent asset not found.") + return asset + + @staticmethod + def _release_state(asset: AgentAsset) -> dict[str, Any]: + config = asset.config_json if isinstance(asset.config_json, dict) else {} + state = config.get("release_guard") + return dict(state) if isinstance(state, dict) else {} diff --git a/server/src/app/services/agent_asset_release_telemetry_crypto.py b/server/src/app/services/agent_asset_release_telemetry_crypto.py new file mode 100644 index 0000000..c3e2d1f --- /dev/null +++ b/server/src/app/services/agent_asset_release_telemetry_crypto.py @@ -0,0 +1,67 @@ +"""发布遥测的确定性标识与可轮换 HMAC 伪名。""" + +from __future__ import annotations + +import hashlib +import hmac +import json +import uuid +from typing import Any + +from app.core.agent_release_telemetry_keys import ( + active_agent_release_telemetry_key_version, + available_agent_release_telemetry_key_versions, + get_agent_release_telemetry_key, +) + + +def deterministic_fingerprint(*parts: Any) -> str: + payload = "\x1f".join(str(item) for item in parts) + return hashlib.sha256(payload.encode("utf-8")).hexdigest() + + +def release_pseudonym_fingerprint(domain: str, *parts: Any, version: str | None = None) -> str: + key_version = version or active_agent_release_telemetry_key_version() + key = get_agent_release_telemetry_key(key_version, create=version is None) + payload = "\x1f".join(("agent-release-telemetry:v1", domain, *(str(item) for item in parts))) + return hmac.new(key, payload.encode("utf-8"), hashlib.sha256).hexdigest() + + +def release_source_fingerprint(*, tenant_id: str, source_key: str, rule_code: str) -> str: + """使用当前版本密钥生成不可字典反推的租户内业务来源伪名。""" + + return release_pseudonym_fingerprint( + "expense-claim-risk-source", + tenant_id, + source_key, + rule_code, + ) + + +def release_source_fingerprints(*, tenant_id: str, source_key: str, rule_code: str) -> set[str]: + """轮换期间同时核验仍保留的历史密钥，避免在途样本失联。""" + + return { + release_pseudonym_fingerprint( + "expense-claim-risk-source", + tenant_id, + source_key, + rule_code, + version=version, + ) + for version in available_agent_release_telemetry_key_versions() + } + + +def json_fingerprint(payload: dict[str, Any]) -> str: + serialized = json.dumps( + payload, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ) + return hashlib.sha256(serialized.encode("utf-8")).hexdigest() + + +def stable_uuid(key: str) -> str: + return str(uuid.uuid5(uuid.NAMESPACE_URL, f"x-financial:{key}")) diff --git a/server/src/app/services/agent_asset_release_telemetry_values.py b/server/src/app/services/agent_asset_release_telemetry_values.py new file mode 100644 index 0000000..26070a3 --- /dev/null +++ b/server/src/app/services/agent_asset_release_telemetry_values.py @@ -0,0 +1,43 @@ +"""发布遥测输入的纯值校验。""" + +from __future__ import annotations + +import re +from typing import Any + +_SAFE_CODE = re.compile(r"^[A-Za-z0-9_.:-]+$") +_STAGES = {"shadow", "canary", "active"} + + +def required_value(value: Any, field: str, maximum: int) -> str: + normalized = str(value or "").strip() + if not normalized or len(normalized) > maximum: + raise ValueError(f"{field} is required and must be at most {maximum} characters.") + return normalized + + +def safe_code(value: Any, field: str, maximum: int) -> str: + normalized = required_value(value, field, maximum) + if not _SAFE_CODE.fullmatch(normalized): + raise ValueError(f"{field} contains unsupported characters.") + return normalized + + +def release_stage(value: Any) -> str: + normalized = str(value or "").strip().lower() + if normalized not in _STAGES: + raise ValueError("stage must be shadow, canary or active.") + return normalized + + +def strict_bool(value: Any, field: str) -> bool: + if not isinstance(value, bool): + raise ValueError(f"{field} must be a boolean.") + return value + + +def precision(confirmed: int, false_positive: int) -> float | None: + denominator = confirmed + false_positive + if denominator <= 0: + return None + return round(confirmed / denominator, 6) diff --git a/server/src/app/services/agent_asset_risk_rule_publish.py b/server/src/app/services/agent_asset_risk_rule_publish.py index 620671d..116191c 100644 --- a/server/src/app/services/agent_asset_risk_rule_publish.py +++ b/server/src/app/services/agent_asset_risk_rule_publish.py @@ -1,261 +1,44 @@ from __future__ import annotations -from datetime import UTC, datetime -from typing import Any - -from app.core.agent_enums import AgentAssetStatus, AgentAssetType, AgentReviewStatus -from app.models.agent_asset import AgentAsset, AgentAssetReview -from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY -from app.services.risk_rule_manifest_normalizer import normalize_risk_rule_manifest +from app.models.agent_asset import AgentAsset +from app.services.agent_asset_release_guard import AgentAssetReleaseGuardService class AgentAssetRiskRulePublishMixin: - """风险规则发布逻辑，支持普通待审核版本和已上线规则修订版本。""" + """把既有“发布”动作收口为强制受控的 shadow 发布入口。""" def publish_risk_rule( self, asset_id: str, *, actor: str, + tenant_id: str | None = None, + allow_global_management: bool = False, request_id: str | None = None, ) -> AgentAsset: asset = self._resolve_asset(asset_id) self._require_json_risk_asset(asset) - revision = self._resolve_publishable_revision(asset) - if revision is not None: - return self._publish_revision(asset, revision, actor=actor, request_id=request_id) - return self._publish_reviewed_working_version(asset, actor=actor, request_id=request_id) - - def _publish_reviewed_working_version( - self, - asset: AgentAsset, - *, - actor: str, - request_id: str | None, - ) -> AgentAsset: version = self._resolve_target_version(asset, None) - if asset.status != AgentAssetStatus.REVIEW.value: - raise ValueError("只有待审核风险规则可以发布上线。") - if not self.get_latest_risk_rule_test_summary(asset, version=version).test_passed: - raise PermissionError("当前规则版本尚未完成测试通过确认，不能发布。") - - # golden set 回归门禁：在 golden 用例集上跑规则，未 100% 通过则拦截发布。 - self._require_golden_set_passed(asset, version, actor=actor) - - before = self._asset_snapshot(asset) - self._ensure_approved_review(asset, version=version, actor=actor, note="发布上线前审核通过。") - asset.reviewer = actor - asset.published_version = version - asset.status = AgentAssetStatus.ACTIVE.value - self.db.add(asset) - self.db.commit() - self.audit_service.log_action( - actor=actor, - action="publish_agent_asset", - resource_type=AgentAssetType.RULE.value, - resource_id=asset.id, - before_json=before, - after_json=self._asset_snapshot(asset), - request_id=request_id, - ) - return self._refresh_asset(asset.id) - - def _publish_revision( - self, - asset: AgentAsset, - revision: dict[str, Any], - *, - actor: str, - request_id: str | None, - ) -> AgentAsset: - version = str(revision.get("version") or "").strip() - if not self.get_latest_risk_rule_test_summary(asset, version=version).test_passed: - raise PermissionError("当前修订版本尚未完成测试通过确认，不能发布。") - - rule_document = revision.get("rule_document") if isinstance(revision.get("rule_document"), dict) else {} - file_name = str(rule_document.get("file_name") or "").strip() - if not file_name: - raise ValueError("修订版本尚未生成可发布的 JSON 规则文件。") - - before = self._asset_snapshot(asset) - manifest = self.rule_library_manager.read_rule_library_json( - library=RISK_RULES_LIBRARY, - file_name=file_name, - ) - manifest = normalize_risk_rule_manifest(manifest) - manifest["enabled"] = True - self.rule_library_manager.write_rule_library_json( - library=RISK_RULES_LIBRARY, - file_name=file_name, - payload=manifest, - ) - - config = dict(asset.config_json or {}) - previous_rule_document = config.get("rule_document") if isinstance(config.get("rule_document"), dict) else {} - published_at = datetime.now(UTC).isoformat() - history = list(config.get("revision_history") if isinstance(config.get("revision_history"), list) else []) - history.insert( - 0, - { - "version": version, - "base_version": revision.get("base_version"), - "change_reason": revision.get("change_reason"), - "published_by": actor, - "published_at": published_at, - "previous_rule_document": previous_rule_document, - "rule_document": rule_document, - }, - ) - config.update(self._config_from_published_manifest(manifest, rule_document)) - config["revision_history"] = history[:20] - config.pop("revision_draft", None) - config["last_operation"] = { - "action": "publish_revision", - "actor": actor, - "at": published_at, - "target_version": version, - } - - asset.name = str(manifest.get("name") or asset.name) - asset.description = str(manifest.get("description") or asset.description) - risk_category = str(manifest.get("risk_category") or "").strip() - if risk_category: - asset.scenario_json = [risk_category] - asset.config_json = config - asset.current_version = version - asset.working_version = version - asset.published_version = version - asset.reviewer = actor - asset.status = AgentAssetStatus.ACTIVE.value - self._ensure_approved_review(asset, version=version, actor=actor, note="修订版本发布上线。") - self.db.add(asset) - self.db.commit() - self.audit_service.log_action( - actor=actor, - action="publish_risk_rule_revision", - resource_type=AgentAssetType.RULE.value, - resource_id=asset.id, - before_json=before, - after_json=self._asset_snapshot(asset), - request_id=request_id, - ) - return self._refresh_asset(asset.id) - - def _resolve_publishable_revision(self, asset: AgentAsset) -> dict[str, Any] | None: - config = dict(asset.config_json or {}) - revision = config.get("revision_draft") - if not isinstance(revision, dict): - return None - version = str(revision.get("version") or "").strip() - if not version or version != str(asset.working_version or "").strip(): - return None - if version == str(asset.published_version or "").strip(): - return None - if revision.get("generation_status") != "completed": - raise ValueError("修订版本尚未重新生成，不能发布上线。") - return dict(revision) - - def _ensure_approved_review( - self, - asset: AgentAsset, - *, - version: str, - actor: str, - note: str, - ) -> None: - approved_review = self.repository.get_review( - asset.id, version, AgentReviewStatus.APPROVED.value - ) - if approved_review is not None: - return - self.db.add( - AgentAssetReview( - asset_id=asset.id, - version=version, - reviewer=actor, - review_status=AgentReviewStatus.APPROVED.value, - review_note=note, - reviewed_at=datetime.now(UTC), - ) - ) - - def _require_golden_set_passed( - self, - asset: AgentAsset, - version: str, - *, - actor: str, - ) -> None: - """在 golden set 上跑当前规则 manifest，未 100% 通过则拦截发布。 - - 降级策略：feature flag 关闭 / 无 rule_document / 无 golden case / - evaluator 异常 → 一律放行，不阻塞发布主链路。 - """ - - import os - - if os.environ.get("GOLDEN_SET_GATE_ENABLED", "true").strip().lower() in {"0", "false", "no"}: - return config = asset.config_json if isinstance(asset.config_json, dict) else {} - rule_document = config.get("rule_document") if isinstance(config.get("rule_document"), dict) else {} - file_name = str(rule_document.get("file_name") or "").strip() - if not file_name: - return - try: - manifest = self.rule_library_manager.read_rule_library_json( - library=RISK_RULES_LIBRARY, - file_name=file_name, - ) - except Exception: - return - rule_code = str(manifest.get("rule_code") or "").strip() - if not rule_code: - return - from app.services.risk_rule_golden_evaluator import RiskRuleGoldenEvaluator + revision = config.get("revision_draft") + if isinstance(revision, dict) and str(revision.get("version") or "").strip() == version: + if revision.get("generation_status") != "completed": + raise ValueError("修订版本尚未重新生成，不能进入影子发布。") - RiskRuleGoldenEvaluator().require_pass( + AgentAssetReleaseGuardService( self.db, - asset, + rule_library_manager=self.rule_library_manager, + ).start_shadow( + asset.id, version, - manifest, - rule_code, actor=actor, + tenant_id=( + str(tenant_id or "").strip() or str(config.get("tenant_id") or "").strip() or None + ), + allow_global_management=allow_global_management, + request_id=request_id, ) - - @staticmethod - def _config_from_published_manifest( - manifest: dict[str, Any], - rule_document: dict[str, Any], - ) -> dict[str, Any]: - metadata = manifest.get("metadata") if isinstance(manifest.get("metadata"), dict) else {} - risk_score_detail = metadata.get("risk_score_detail") if isinstance(metadata.get("risk_score_detail"), dict) else {} - risk_level = str(metadata.get("risk_level") or manifest.get("outcomes", {}).get("fail", {}).get("severity") or "medium") - risk_score = int(metadata.get("risk_score") or manifest.get("outcomes", {}).get("fail", {}).get("risk_score") or 0) - return { - "severity": risk_level, - "risk_score": risk_score, - "risk_level": risk_level, - "risk_level_label": metadata.get("risk_level_label"), - "risk_score_detail": risk_score_detail, - "enabled": True, - "requires_attachment": bool(metadata.get("requires_attachment") or manifest.get("requires_attachment")), - "detail_mode": "json_risk", - "business_stage": metadata.get("business_stage"), - "business_stage_label": metadata.get("business_stage_label"), - "expense_category": metadata.get("expense_category"), - "expense_category_label": metadata.get("expense_category_label"), - "risk_category": manifest.get("risk_category"), - "rule_library": RISK_RULES_LIBRARY, - "rule_document": rule_document, - "ontology_signal": manifest.get("ontology_signal"), - "evaluator": manifest.get("evaluator"), - "generated_by": "natural_language", - "source_ref": "自然语言风险规则", - "flow_diagram_svg": manifest.get("flow_diagram_svg"), - } - - def _refresh_asset(self, asset_id: str) -> AgentAsset: - refreshed = self.repository.get(asset_id) + refreshed = self.repository.get(asset.id) if refreshed is None: raise LookupError("Asset not found") return refreshed diff --git a/server/src/app/services/agent_asset_risk_rule_regeneration.py b/server/src/app/services/agent_asset_risk_rule_regeneration.py index 89b1db0..e45bff9 100644 --- a/server/src/app/services/agent_asset_risk_rule_regeneration.py +++ b/server/src/app/services/agent_asset_risk_rule_regeneration.py @@ -5,6 +5,7 @@ from typing import Any from sqlalchemy.orm import Session +from app.api.deps import CurrentUserContext from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType from app.models.agent_asset import AgentAsset, AgentAssetVersion from app.repositories.agent_asset import AgentAssetRepository @@ -12,6 +13,7 @@ from app.schemas.agent_asset import ( AgentAssetRiskRuleGenerateRequest, AgentAssetRiskRuleRegenerateRequest, ) +from app.services.agent_asset_access import AgentAssetAccessScope from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.audit import AuditLogService @@ -36,9 +38,13 @@ class AgentAssetRiskRuleRegenerationService: *, rule_library_manager: AgentAssetRuleLibraryManager | None = None, runtime_chat_service: RuntimeChatService | None = None, + current_user: CurrentUserContext | None = None, ) -> None: self.db = db - self.repository = AgentAssetRepository(db) + self.access_scope = ( + AgentAssetAccessScope.from_user(current_user) if current_user is not None else None + ) + self.repository = AgentAssetRepository(db, access_scope=self.access_scope) self.rule_library_manager = rule_library_manager or AgentAssetRuleLibraryManager() self.generator = RiskRuleGenerationService( db, @@ -125,7 +131,9 @@ class AgentAssetRiskRuleRegenerationService: asset.name = str(payload["name"]) asset.description = str(payload["description"]) asset.domain = str(request.get("business_domain") or AgentAssetDomain.EXPENSE.value) - asset.scenario_json = [str(payload.get("risk_category") or BUSINESS_DOMAIN_LABELS[asset.domain])] + asset.scenario_json = [ + str(payload.get("risk_category") or BUSINESS_DOMAIN_LABELS[asset.domain]) + ] asset.status = AgentAssetStatus.DRAFT.value asset.current_version = version asset.working_version = version @@ -160,7 +168,11 @@ class AgentAssetRiskRuleRegenerationService: asset, config, body.model_dump(exclude_unset=True), - base=revision.get("generation_request") if isinstance(revision.get("generation_request"), dict) else {}, + base=( + revision.get("generation_request") + if isinstance(revision.get("generation_request"), dict) + else {} + ), ) payload, risk_score = self._compile_payload( request, @@ -303,8 +315,13 @@ class AgentAssetRiskRuleRegenerationService: asset = self.repository.get(asset_id) if asset is None: raise FileNotFoundError("风险规则不存在。") + if self.access_scope is not None: + self.access_scope.require_write(asset) config = asset.config_json or {} - if asset.asset_type != AgentAssetType.RULE.value or config.get("detail_mode") != "json_risk": + if ( + asset.asset_type != AgentAssetType.RULE.value + or config.get("detail_mode") != "json_risk" + ): raise ValueError("当前资产不是自然语言风险规则。") return asset @@ -394,6 +411,8 @@ class AgentAssetRiskRuleRegenerationService: if existing is None: self.db.add( AgentAssetVersion( + tenant_id=asset.tenant_id, + scope=asset.scope, asset_id=asset.id, version=version, content=content, diff --git a/server/src/app/services/agent_asset_risk_rule_revision.py b/server/src/app/services/agent_asset_risk_rule_revision.py index 3cc7386..8feee3d 100644 --- a/server/src/app/services/agent_asset_risk_rule_revision.py +++ b/server/src/app/services/agent_asset_risk_rule_revision.py @@ -5,6 +5,7 @@ from typing import Any from sqlalchemy.orm import Session +from app.api.deps import CurrentUserContext from app.core.agent_enums import AgentAssetStatus, AgentAssetType from app.models.agent_asset import AgentAsset, AgentAssetVersion from app.repositories.agent_asset import AgentAssetRepository @@ -12,6 +13,7 @@ from app.schemas.agent_asset import ( AgentAssetRiskRuleDraftUpdate, AgentAssetRiskRuleRevisionCreate, ) +from app.services.agent_asset_access import AgentAssetAccessScope from app.services.audit import AuditLogService from app.services.risk_rule_generation_ontology import EXPENSE_RISK_CATEGORY_LABELS @@ -19,9 +21,17 @@ from app.services.risk_rule_generation_ontology import EXPENSE_RISK_CATEGORY_LAB class AgentAssetRiskRuleRevisionService: """风险规则草稿编辑与已发布规则修订草稿服务。""" - def __init__(self, db: Session) -> None: + def __init__( + self, + db: Session, + *, + current_user: CurrentUserContext | None = None, + ) -> None: self.db = db - self.repository = AgentAssetRepository(db) + self.access_scope = ( + AgentAssetAccessScope.from_user(current_user) if current_user is not None else None + ) + self.repository = AgentAssetRepository(db, access_scope=self.access_scope) self.audit_service = AuditLogService(db) def update_unpublished_draft( @@ -95,6 +105,8 @@ class AgentAssetRiskRuleRevisionService: self.db.add(asset) self.db.add( AgentAssetVersion( + tenant_id=asset.tenant_id, + scope=asset.scope, asset_id=asset.id, version=revision_version, content=self._build_revision_content(asset, config), @@ -119,8 +131,13 @@ class AgentAssetRiskRuleRevisionService: asset = self.repository.get(asset_id) if asset is None: raise FileNotFoundError("风险规则不存在。") + if self.access_scope is not None: + self.access_scope.require_write(asset) config = asset.config_json or {} - if asset.asset_type != AgentAssetType.RULE.value or config.get("detail_mode") != "json_risk": + if ( + asset.asset_type != AgentAssetType.RULE.value + or config.get("detail_mode") != "json_risk" + ): raise ValueError("当前资产不是自然语言风险规则。") return asset @@ -136,8 +153,12 @@ class AgentAssetRiskRuleRevisionService: now = datetime.now(UTC).isoformat() rule_title = str(request.get("rule_title") or asset.name or "").strip() natural_language = str(request.get("natural_language") or asset.description or "").strip() - expense_category = str(request.get("expense_category") or config.get("expense_category") or "").strip() - category_label = EXPENSE_RISK_CATEGORY_LABELS.get(expense_category, config.get("risk_category") or "") + expense_category = str( + request.get("expense_category") or config.get("expense_category") or "" + ).strip() + category_label = EXPENSE_RISK_CATEGORY_LABELS.get( + expense_category, config.get("risk_category") or "" + ) asset.name = rule_title or asset.name asset.description = natural_language or asset.description if category_label: @@ -156,8 +177,14 @@ class AgentAssetRiskRuleRevisionService: asset.config_json = config @staticmethod - def _merged_generation_request(config: dict[str, Any], updates: dict[str, Any]) -> dict[str, Any]: - base = config.get("generation_request") if isinstance(config.get("generation_request"), dict) else {} + def _merged_generation_request( + config: dict[str, Any], updates: dict[str, Any] + ) -> dict[str, Any]: + base = ( + config.get("generation_request") + if isinstance(config.get("generation_request"), dict) + else {} + ) merged = dict(base) for key, value in updates.items(): if key == "change_reason": @@ -172,7 +199,12 @@ class AgentAssetRiskRuleRevisionService: return merged def _next_revision_version(self, asset: AgentAsset) -> str: - base = str(asset.working_version or asset.current_version or asset.published_version or "v0.1.0") + base = str( + asset.working_version + or asset.current_version + or asset.published_version + or "v0.1.0" + ) major, minor, patch = self._parse_version(base) existing = {version.version for version in self.repository.list_versions(asset.id)} while True: @@ -190,8 +222,16 @@ class AgentAssetRiskRuleRevisionService: @staticmethod def _build_revision_content(asset: AgentAsset, config: dict[str, Any]) -> str: - revision = config.get("revision_draft") if isinstance(config.get("revision_draft"), dict) else {} - request = revision.get("generation_request") if isinstance(revision.get("generation_request"), dict) else {} + revision = ( + config.get("revision_draft") + if isinstance(config.get("revision_draft"), dict) + else {} + ) + request = ( + revision.get("generation_request") + if isinstance(revision.get("generation_request"), dict) + else {} + ) return "\n".join( [ f"# {asset.name} 修订草稿", diff --git a/server/src/app/services/agent_asset_risk_rule_testing.py b/server/src/app/services/agent_asset_risk_rule_testing.py index 2f16019..99d0f34 100644 --- a/server/src/app/services/agent_asset_risk_rule_testing.py +++ b/server/src/app/services/agent_asset_risk_rule_testing.py @@ -24,9 +24,10 @@ from app.schemas.agent_asset import ( AgentAssetRiskRuleScenarioTestRequest, AgentAssetRiskRuleTestRunRead, ) +from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.expense_claims import ExpenseClaimService -from app.services.risk_rule_template_executor import RiskRuleTemplateExecutor from app.services.risk_rule_manifest_normalizer import normalize_risk_rule_manifest +from app.services.risk_rule_template_executor import RiskRuleTemplateExecutor class AgentAssetRiskRuleTestingMixin: @@ -92,8 +93,12 @@ class AgentAssetRiskRuleTestingMixin: if asset.domain != AgentAssetDomain.EXPENSE.value: raise ValueError("一期真实场景试运行仅支持报销业务域。") + target_tenant_id = self._require_scenario_target_tenant( + asset, + body.target_tenant_id, + ) parsed_scope = self._parse_scenario_scope(body.intent, body.filters) - claims = self._query_expense_claim_samples(parsed_scope) + claims = self._query_expense_claim_samples(target_tenant_id, parsed_scope) claim_results = [self._run_claim_scenario(manifest, claim) for claim in claims] hit_items = [item for item in claim_results if item["hit"]] severity_counts: dict[str, int] = {} @@ -114,6 +119,7 @@ class AgentAssetRiskRuleTestingMixin: passed=passed, summary=summary, input_json={ + "target_tenant_id": target_tenant_id, "intent": body.intent, "filters": body.filters, "parsed_scope": parsed_scope, @@ -126,6 +132,7 @@ class AgentAssetRiskRuleTestingMixin: }, actor=actor, request_id=request_id, + evidence_tenant_id=target_tenant_id, ) def confirm_risk_rule_test_report( @@ -209,9 +216,13 @@ class AgentAssetRiskRuleTestingMixin: version = self._resolve_target_version(asset, None) if asset.status != AgentAssetStatus.REVIEW.value: raise ValueError("只有待审核风险规则可以回退。") + if self.access_scope is not None: + self.access_scope.require_write(asset) before = self._asset_snapshot(asset) review = AgentAssetReview( + tenant_id=asset.tenant_id, + scope=asset.scope, asset_id=asset.id, version=version, reviewer=actor, @@ -235,55 +246,6 @@ class AgentAssetRiskRuleTestingMixin: ) return self.get_latest_risk_rule_test_summary(asset) - def publish_risk_rule( - self, - asset_id: str, - *, - actor: str, - request_id: str | None = None, - ) -> AgentAsset: - asset = self._resolve_asset(asset_id) - self._require_json_risk_asset(asset) - version = self._resolve_target_version(asset, None) - if asset.status != AgentAssetStatus.REVIEW.value: - raise ValueError("只有待审核风险规则可以发布上线。") - if not self.get_latest_risk_rule_test_summary(asset, version=version).test_passed: - raise PermissionError("当前规则版本尚未完成测试通过确认，不能发布。") - - before = self._asset_snapshot(asset) - approved_review = self.repository.get_review( - asset.id, version, AgentReviewStatus.APPROVED.value - ) - if approved_review is None: - self.db.add( - AgentAssetReview( - asset_id=asset.id, - version=version, - reviewer=actor, - review_status=AgentReviewStatus.APPROVED.value, - review_note="发布上线前审核通过。", - reviewed_at=datetime.now(UTC), - ) - ) - asset.reviewer = actor - asset.published_version = version - asset.status = AgentAssetStatus.ACTIVE.value - self.db.add(asset) - self.db.commit() - self.audit_service.log_action( - actor=actor, - action="publish_agent_asset", - resource_type=AgentAssetType.RULE.value, - resource_id=asset.id, - before_json=before, - after_json=self._asset_snapshot(asset), - request_id=request_id, - ) - refreshed = self.repository.get(asset.id) - if refreshed is None: - raise LookupError("Asset not found") - return refreshed - def set_risk_rule_enabled( self, asset_id: str, @@ -294,8 +256,26 @@ class AgentAssetRiskRuleTestingMixin: ) -> AgentAsset: asset = self._resolve_asset(asset_id) self._require_json_risk_asset(asset) + published_version = str(asset.published_version or "").strip() + if not published_version: + raise PermissionError("未发布风险规则不能直接启用，请先完成 shadow/Canary 发布。") + config_json = dict(asset.config_json or {}) + release_state = config_json.get("release_guard") + release_stage = ( + str(release_state.get("stage") or "").strip() if isinstance(release_state, dict) else "" + ) + if release_stage in {"shadow", "canary"}: + raise PermissionError("分阶段发布进行中，请先完成或回滚后再切换启用状态。") before = self._asset_snapshot(asset) - rule_library, file_name = self._resolve_json_risk_rule_document(asset) + rule_library = str(config_json.get("rule_library") or RISK_RULES_LIBRARY).strip() + rule_document = config_json.get("rule_document") + file_name = ( + str(rule_document.get("file_name") or "").strip() + if isinstance(rule_document, dict) + else "" + ) + if not file_name: + raise ValueError("已发布风险规则缺少运行文件，不能切换启用状态。") manifest = self.rule_library_manager.read_rule_library_json( library=rule_library, file_name=file_name, @@ -307,9 +287,9 @@ class AgentAssetRiskRuleTestingMixin: payload=manifest, ) - config_json = dict(asset.config_json or {}) config_json["enabled"] = bool(enabled) - self._set_risk_rule_status_for_online_toggle(asset, enabled=enabled, actor=actor) + asset.status = AgentAssetStatus.ACTIVE.value if enabled else AgentAssetStatus.DISABLED.value + asset.reviewer = actor config_json["last_operation"] = self._build_last_operation( action="online" if enabled else "offline", actor=actor, @@ -327,36 +307,6 @@ class AgentAssetRiskRuleTestingMixin: ) return updated - def _set_risk_rule_status_for_online_toggle( - self, - asset: AgentAsset, - *, - enabled: bool, - actor: str, - ) -> None: - if enabled: - version = self._resolve_target_version(asset, None) - approved_review = self.repository.get_review( - asset.id, version, AgentReviewStatus.APPROVED.value - ) - if approved_review is None: - self.db.add( - AgentAssetReview( - asset_id=asset.id, - version=version, - reviewer=actor, - review_status=AgentReviewStatus.APPROVED.value, - review_note="直接上线风险规则。", - reviewed_at=datetime.now(UTC), - ) - ) - asset.published_version = version - asset.reviewer = actor - asset.status = AgentAssetStatus.ACTIVE.value - return - - asset.status = AgentAssetStatus.DISABLED.value - def _mark_risk_rule_operation(self, asset: AgentAsset, *, action: str, actor: str) -> None: config_json = dict(asset.config_json or {}) config_json["last_operation"] = self._build_last_operation(action=action, actor=actor) @@ -400,10 +350,16 @@ class AgentAssetRiskRuleTestingMixin: result_json: dict[str, Any], actor: str, request_id: str | None, + evidence_tenant_id: str | None = None, ) -> AgentAssetRiskRuleTestRunRead: status = "passed" if passed else "failed" + scoped_tenant_id = str(evidence_tenant_id or "").strip() + if not scoped_tenant_id and self.access_scope is not None: + scoped_tenant_id = self.access_scope.tenant_id created = self.repository.create_test_run( AgentAssetTestRun( + tenant_id=scoped_tenant_id or asset.tenant_id, + scope="tenant" if scoped_tenant_id else asset.scope, asset_id=asset.id, version=version, test_type=test_type, @@ -432,7 +388,9 @@ class AgentAssetRiskRuleTestingMixin: case: AgentAssetRiskRuleSampleCase, ) -> dict[str, Any]: claim, contexts = self._build_synthetic_claim(case.values, manifest) - execution = RiskRuleTemplateExecutor().evaluate_with_trace(manifest, claim=claim, contexts=contexts) + execution = RiskRuleTemplateExecutor().evaluate_with_trace( + manifest, claim=claim, contexts=contexts + ) result = execution["result"] actual_hit = result is not None actual_severity = ( @@ -461,7 +419,9 @@ class AgentAssetRiskRuleTestingMixin: def _run_claim_scenario(self, manifest: dict[str, Any], claim: ExpenseClaim) -> dict[str, Any]: contexts = ExpenseClaimService(self.db)._build_claim_attachment_contexts(claim) - execution = RiskRuleTemplateExecutor().evaluate_with_trace(manifest, claim=claim, contexts=contexts) + execution = RiskRuleTemplateExecutor().evaluate_with_trace( + manifest, claim=claim, contexts=contexts + ) result = execution["result"] hit = result is not None return { @@ -621,8 +581,18 @@ class AgentAssetRiskRuleTestingMixin: template_key = str(manifest.get("template_key") or "").strip() params = manifest.get("params") if isinstance(manifest.get("params"), dict) else {} if template_key == "field_compare_v1": - if str(params.get("semantic_type") or "").strip() in {"travel_city_consistency", "travel_route_city_consistency"}: - values.update({"attachment.hotel_city": "上海" if hit else "北京", "attachment.route_cities": ["上海"] if hit else ["北京"], "claim.location": "北京", "item.item_location": "北京"}) + if str(params.get("semantic_type") or "").strip() in { + "travel_city_consistency", + "travel_route_city_consistency", + }: + values.update( + { + "attachment.hotel_city": "上海" if hit else "北京", + "attachment.route_cities": ["上海"] if hit else ["北京"], + "claim.location": "北京", + "item.item_location": "北京", + } + ) return values condition = next( (item for item in params.get("conditions", []) if isinstance(item, dict)), @@ -671,11 +641,19 @@ class AgentAssetRiskRuleTestingMixin: return "住宿费" return "测试值" - def _query_expense_claim_samples(self, parsed_scope: dict[str, Any]) -> list[ExpenseClaim]: + def _query_expense_claim_samples( + self, + target_tenant_id: str, + parsed_scope: dict[str, Any], + ) -> list[ExpenseClaim]: days = int(parsed_scope.get("days") or 30) limit = min(max(int(parsed_scope.get("limit") or 50), 1), 200) since = datetime.now(UTC) - timedelta(days=days) - stmt = select(ExpenseClaim).where(ExpenseClaim.created_at >= since) + # 租户谓词是查询构造的第一项，任何业务过滤与 limit 都只能在租户内生效。 + stmt = select(ExpenseClaim).where( + ExpenseClaim.tenant_id == target_tenant_id, + ExpenseClaim.created_at >= since, + ) expense_keyword = str(parsed_scope.get("expense_keyword") or "").strip() if expense_keyword: @@ -703,6 +681,22 @@ class AgentAssetRiskRuleTestingMixin: stmt = stmt.order_by(ExpenseClaim.created_at.desc()).limit(limit) return list(self.db.scalars(stmt).all()) + def _require_scenario_target_tenant( + self, + asset: AgentAsset, + target_tenant_id: str, + ) -> str: + target = str(target_tenant_id or "").strip() + if not target or target == "platform": + raise ValueError("真实场景试运行必须显式指定企业租户。") + if self.access_scope is None: + raise PermissionError("真实场景试运行需要可信登录租户上下文。") + if target != self.access_scope.tenant_id: + raise LookupError("Asset not found") + if asset.scope == "tenant" and asset.tenant_id != target: + raise LookupError("Asset not found") + return target + @staticmethod def _parse_scenario_scope(intent: str, filters: dict[str, Any]) -> dict[str, Any]: text = str(intent or "") diff --git a/server/src/app/services/agent_asset_serialization.py b/server/src/app/services/agent_asset_serialization.py new file mode 100644 index 0000000..ab8ad6c --- /dev/null +++ b/server/src/app/services/agent_asset_serialization.py @@ -0,0 +1,221 @@ +"""Agent 资产列表与版本响应的序列化辅助职责。""" + +from __future__ import annotations + +import json +from collections import defaultdict +from datetime import datetime +from typing import Any + +from app.core.agent_enums import AgentAssetContentType, AgentAssetType, AgentReviewStatus +from app.models.agent_asset import AgentAsset, AgentAssetVersion +from app.schemas.agent_asset import AgentAssetListItem, AgentAssetVersionRead + + +class AgentAssetSerializationMixin: + """集中处理资产/版本只读投影，不负责业务状态迁移。""" + + def _serialize_version( + self, version: AgentAssetVersion, asset: AgentAsset + ) -> AgentAssetVersionRead: + latest_review = self.repository.get_review(asset.id, version.version) + working_version = self._resolve_working_version(asset) + published_version = self._resolve_published_version(asset) + return AgentAssetVersionRead( + id=version.id, + tenant_id=version.tenant_id, + scope=version.scope, + asset_id=version.asset_id, + version=version.version, + content=self._deserialize_content(version), + content_type=version.content_type, + change_note=version.change_note, + created_by=version.created_by, + created_at=version.created_at, + is_current=version.version == working_version, + is_published=version.version == published_version, + is_working=version.version == working_version, + lifecycle_state=self._resolve_version_lifecycle_state( + version.version, + working_version=working_version, + published_version=published_version, + latest_review_status=latest_review.review_status if latest_review else "", + ), + ) + + def _collect_version_stats(self, assets: list[AgentAsset]) -> dict[str, dict[str, Any]]: + asset_ids = [item.id for item in assets] + versions = self.repository.list_versions_for_assets(asset_ids) + reviews = self.repository.list_reviews_for_assets(asset_ids) + spreadsheet_logs = self.audit_service.repository.list_for_resources( + resource_type=AgentAssetType.RULE.value, + resource_ids=[ + item.id + for item in assets + if item.asset_type == AgentAssetType.RULE.value + and str((item.config_json or {}).get("detail_mode") or "").strip().lower() + == "spreadsheet" + ], + action="edit_rule_spreadsheet", + ) + working_versions = {item.id: self._resolve_working_version(item) for item in assets} + version_counts: dict[str, int] = defaultdict(int) + modified_by: dict[str, str | None] = {item.id: None for item in assets} + published_versions = {item.id: self._resolve_published_version(item) for item in assets} + published_by: dict[str, str | None] = {} + published_at: dict[str, datetime | None] = {} + spreadsheet_edit_counts: dict[str, int] = defaultdict(int) + spreadsheet_last_actor: dict[str, str | None] = {} + spreadsheet_last_changed_at: dict[str, datetime] = {} + + for version in versions: + version_counts[version.asset_id] += 1 + if modified_by.get( + version.asset_id + ) is None and version.version == working_versions.get(version.asset_id): + modified_by[version.asset_id] = version.created_by + + for review in reviews: + if review.asset_id in published_at: + continue + if review.version != published_versions.get(review.asset_id): + continue + if review.review_status != AgentReviewStatus.APPROVED.value: + continue + published_by[review.asset_id] = review.reviewer + published_at[review.asset_id] = review.reviewed_at or review.created_at + + for log in spreadsheet_logs: + spreadsheet_edit_counts[log.resource_id] += 1 + last_changed_at = spreadsheet_last_changed_at.get(log.resource_id) + if last_changed_at is None or log.created_at >= last_changed_at: + spreadsheet_last_changed_at[log.resource_id] = log.created_at + spreadsheet_last_actor[log.resource_id] = log.actor + + return { + item.id: { + "change_count": ( + spreadsheet_edit_counts.get(item.id, 0) + if item.asset_type == AgentAssetType.RULE.value + and str((item.config_json or {}).get("detail_mode") or "").strip().lower() + == "spreadsheet" + and spreadsheet_edit_counts.get(item.id, 0) > 0 + else max(version_counts.get(item.id, 0) - 1, 0) + ), + "modified_by": ( + spreadsheet_last_actor.get(item.id) + if item.asset_type == AgentAssetType.RULE.value + and str((item.config_json or {}).get("detail_mode") or "").strip().lower() + == "spreadsheet" + and spreadsheet_last_actor.get(item.id) + else modified_by.get(item.id) + ), + "published_by": published_by.get(item.id), + "published_at": published_at.get(item.id), + } + for item in assets + } + + @staticmethod + def _serialize_list_item( + asset: AgentAsset, + version_stats: dict[str, int | str | None] | None = None, + ) -> AgentAssetListItem: + payload = AgentAssetListItem.model_validate(asset).model_dump() + payload["change_count"] = int((version_stats or {}).get("change_count") or 0) + payload["modified_by"] = str((version_stats or {}).get("modified_by") or "").strip() or None + payload["published_by"] = ( + str((version_stats or {}).get("published_by") or "").strip() or None + ) + payload["published_at"] = (version_stats or {}).get("published_at") + return AgentAssetListItem.model_validate(payload) + + @staticmethod + def _sort_versions( + versions: list[AgentAssetVersion], current_version: str | None + ) -> list[AgentAssetVersion]: + return sorted( + versions, + key=lambda item: (item.version == current_version, item.created_at), + reverse=True, + ) + + @staticmethod + def _serialize_content(content: Any, content_type: str) -> str: + if content_type == AgentAssetContentType.MARKDOWN.value: + return str(content) + return json.dumps(content, ensure_ascii=False, sort_keys=True, indent=2) + + @staticmethod + def _deserialize_content(version: AgentAssetVersion | None) -> Any: + if version is None: + return None + if version.content_type == AgentAssetContentType.MARKDOWN.value: + return version.content + return json.loads(version.content) + + @staticmethod + def _increment_version(version: str | None) -> str: + normalized = str(version or "").strip().removeprefix("v") + parts = normalized.split(".") + if len(parts) != 3 or not all(item.isdigit() for item in parts): + return "v1.0.0" + major, minor, patch = [int(item) for item in parts] + return f"v{major}.{minor}.{patch + 1}" + + @staticmethod + def _hash_bytes(content: bytes) -> str: + import hashlib + + return hashlib.sha256(content).hexdigest() + + @staticmethod + def _asset_snapshot(asset: AgentAsset) -> dict[str, Any]: + return { + "tenant_id": asset.tenant_id, + "scope": asset.scope, + "asset_type": asset.asset_type, + "code": asset.code, + "name": asset.name, + "status": asset.status, + "current_version": asset.current_version, + "published_version": asset.published_version, + "working_version": asset.working_version, + "domain": asset.domain, + "owner": asset.owner, + "reviewer": asset.reviewer, + } + + @staticmethod + def _resolve_working_version(asset: AgentAsset) -> str: + return str(asset.working_version or asset.current_version or "").strip() + + @staticmethod + def _resolve_published_version(asset: AgentAsset) -> str: + return str(asset.published_version or "").strip() + + @staticmethod + def _resolve_version_lifecycle_state( + version: str, + *, + working_version: str, + published_version: str, + latest_review_status: str, + ) -> str: + if version == published_version: + return "published" + if version != working_version: + return "history" + if latest_review_status == AgentReviewStatus.PENDING.value: + return "pending_review" + if latest_review_status == AgentReviewStatus.APPROVED.value: + return "approved" + if latest_review_status == AgentReviewStatus.REJECTED.value: + return "rejected" + return "draft" + + def _next_available_version(self, asset: AgentAsset) -> str: + candidate = self._increment_version(self._resolve_working_version(asset)) + while self.repository.get_version(asset.id, candidate) is not None: + candidate = self._increment_version(candidate) + return candidate diff --git a/server/src/app/services/agent_assets.py b/server/src/app/services/agent_assets.py index adb5ad2..83ca7f8 100644 --- a/server/src/app/services/agent_assets.py +++ b/server/src/app/services/agent_assets.py @@ -1,12 +1,10 @@ from __future__ import annotations -import json -from collections import defaultdict from datetime import UTC, datetime -from typing import Any from sqlalchemy.orm import Session +from app.api.deps import CurrentUserContext from app.core.agent_enums import ( AgentAssetContentType, AgentAssetStatus, @@ -26,6 +24,10 @@ from app.schemas.agent_asset import ( AgentAssetVersionCreate, AgentAssetVersionRead, ) +from app.services.agent_asset_access import ( + AgentAssetAccessScope, + platform_resource_identity, +) from app.services.agent_asset_json_rules import AgentAssetJsonRuleMixin from app.services.agent_asset_onlyoffice import AgentAssetOnlyOfficeMixin from app.services.agent_asset_risk_rule_feedback import AgentAssetRiskRuleFeedbackMixin @@ -34,6 +36,7 @@ from app.services.agent_asset_risk_rule_publish import AgentAssetRiskRulePublish from app.services.agent_asset_risk_rule_simulation import AgentAssetRiskRuleSimulationMixin from app.services.agent_asset_risk_rule_testing import AgentAssetRiskRuleTestingMixin from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager +from app.services.agent_asset_serialization import AgentAssetSerializationMixin from app.services.agent_asset_spreadsheet import AgentAssetSpreadsheetManager from app.services.agent_asset_spreadsheet_helpers import AgentAssetSpreadsheetHelperMixin from app.services.agent_asset_timeline import AgentAssetTimelineMixin @@ -142,212 +145,31 @@ class AgentAssetVersionMixin: ) return restored # type: ignore[return-value] - def _serialize_version( - self, version: AgentAssetVersion, asset: AgentAsset - ) -> AgentAssetVersionRead: - latest_review = self.repository.get_review(asset.id, version.version) - working_version = self._resolve_working_version(asset) - published_version = self._resolve_published_version(asset) - return AgentAssetVersionRead( - id=version.id, - asset_id=version.asset_id, - version=version.version, - content=self._deserialize_content(version), - content_type=version.content_type, - change_note=version.change_note, - created_by=version.created_by, - created_at=version.created_at, - is_current=version.version == working_version, - is_published=version.version == published_version, - is_working=version.version == working_version, - lifecycle_state=self._resolve_version_lifecycle_state( - version.version, - working_version=working_version, - published_version=published_version, - latest_review_status=latest_review.review_status if latest_review else "", - ), - ) - def _collect_version_stats(self, assets: list[AgentAsset]) -> dict[str, dict[str, Any]]: - asset_ids = [item.id for item in assets] - versions = self.repository.list_versions_for_assets(asset_ids) - reviews = self.repository.list_reviews_for_assets(asset_ids) - spreadsheet_logs = self.audit_service.repository.list_for_resources( - resource_type=AgentAssetType.RULE.value, - resource_ids=[ - item.id - for item in assets - if item.asset_type == AgentAssetType.RULE.value - and str((item.config_json or {}).get("detail_mode") or "").strip().lower() - == "spreadsheet" - ], - action="edit_rule_spreadsheet", - ) - working_versions = {item.id: self._resolve_working_version(item) for item in assets} - version_counts: dict[str, int] = defaultdict(int) - modified_by: dict[str, str | None] = {item.id: None for item in assets} - published_versions = {item.id: self._resolve_published_version(item) for item in assets} - published_by: dict[str, str | None] = {} - published_at: dict[str, datetime | None] = {} - spreadsheet_edit_counts: dict[str, int] = defaultdict(int) - spreadsheet_last_actor: dict[str, str | None] = {} - spreadsheet_last_changed_at: dict[str, datetime] = {} - - for version in versions: - version_counts[version.asset_id] += 1 - if modified_by.get( - version.asset_id - ) is None and version.version == working_versions.get(version.asset_id): - modified_by[version.asset_id] = version.created_by - - for review in reviews: - if review.asset_id in published_at: - continue - if review.version != published_versions.get(review.asset_id): - continue - if review.review_status != AgentReviewStatus.APPROVED.value: - continue - published_by[review.asset_id] = review.reviewer - published_at[review.asset_id] = review.reviewed_at or review.created_at - - for log in spreadsheet_logs: - spreadsheet_edit_counts[log.resource_id] += 1 - last_changed_at = spreadsheet_last_changed_at.get(log.resource_id) - if last_changed_at is None or log.created_at >= last_changed_at: - spreadsheet_last_changed_at[log.resource_id] = log.created_at - spreadsheet_last_actor[log.resource_id] = log.actor - - return { - item.id: { - "change_count": ( - spreadsheet_edit_counts.get(item.id, 0) - if item.asset_type == AgentAssetType.RULE.value - and str((item.config_json or {}).get("detail_mode") or "").strip().lower() - == "spreadsheet" - and spreadsheet_edit_counts.get(item.id, 0) > 0 - else max(version_counts.get(item.id, 0) - 1, 0) - ), - "modified_by": ( - spreadsheet_last_actor.get(item.id) - if item.asset_type == AgentAssetType.RULE.value - and str((item.config_json or {}).get("detail_mode") or "").strip().lower() - == "spreadsheet" - and spreadsheet_last_actor.get(item.id) - else modified_by.get(item.id) - ), - "published_by": published_by.get(item.id), - "published_at": published_at.get(item.id), - } - for item in assets - } - - @staticmethod - def _serialize_list_item( - asset: AgentAsset, - version_stats: dict[str, int | str | None] | None = None, - ) -> AgentAssetListItem: - payload = AgentAssetListItem.model_validate(asset).model_dump() - payload["change_count"] = int((version_stats or {}).get("change_count") or 0) - payload["modified_by"] = str((version_stats or {}).get("modified_by") or "").strip() or None - payload["published_by"] = ( - str((version_stats or {}).get("published_by") or "").strip() or None - ) - payload["published_at"] = (version_stats or {}).get("published_at") - return AgentAssetListItem.model_validate(payload) - - @staticmethod - def _sort_versions( - versions: list[AgentAssetVersion], current_version: str | None - ) -> list[AgentAssetVersion]: - return sorted( - versions, - key=lambda item: (item.version == current_version, item.created_at), - reverse=True, - ) - - @staticmethod - def _serialize_content(content: Any, content_type: str) -> str: - if content_type == AgentAssetContentType.MARKDOWN.value: - return str(content) - return json.dumps(content, ensure_ascii=False, sort_keys=True, indent=2) - - @staticmethod - def _deserialize_content(version: AgentAssetVersion | None) -> Any: - if version is None: - return None - if version.content_type == AgentAssetContentType.MARKDOWN.value: - return version.content - return json.loads(version.content) - - @staticmethod - def _increment_version(version: str | None) -> str: - normalized = str(version or "").strip().removeprefix("v") - parts = normalized.split(".") - if len(parts) != 3 or not all(item.isdigit() for item in parts): - return "v1.0.0" - major, minor, patch = [int(item) for item in parts] - return f"v{major}.{minor}.{patch + 1}" - - @staticmethod - def _hash_bytes(content: bytes) -> str: - import hashlib - - return hashlib.sha256(content).hexdigest() - - @staticmethod - def _asset_snapshot(asset: AgentAsset) -> dict[str, Any]: - return { - "asset_type": asset.asset_type, - "code": asset.code, - "name": asset.name, - "status": asset.status, - "current_version": asset.current_version, - "published_version": asset.published_version, - "working_version": asset.working_version, - "domain": asset.domain, - "owner": asset.owner, - "reviewer": asset.reviewer, - } - - @staticmethod - def _resolve_working_version(asset: AgentAsset) -> str: - return str(asset.working_version or asset.current_version or "").strip() - - @staticmethod - def _resolve_published_version(asset: AgentAsset) -> str: - return str(asset.published_version or "").strip() - - @staticmethod - def _resolve_version_lifecycle_state( - version: str, +class AgentAssetService( + AgentAssetSerializationMixin, + AgentAssetVersionMixin, + AgentAssetOnlyOfficeMixin, + AgentAssetSpreadsheetHelperMixin, + AgentAssetRiskRuleLevelMixin, + AgentAssetRiskRulePublishMixin, + AgentAssetRiskRuleFeedbackMixin, + AgentAssetRiskRuleTestingMixin, + AgentAssetRiskRuleSimulationMixin, + AgentAssetTimelineMixin, + AgentAssetJsonRuleMixin, +): + def __init__( + self, + db: Session, *, - working_version: str, - published_version: str, - latest_review_status: str, - ) -> str: - if version == published_version: - return "published" - if version != working_version: - return "history" - if latest_review_status == AgentReviewStatus.PENDING.value: - return "pending_review" - if latest_review_status == AgentReviewStatus.APPROVED.value: - return "approved" - if latest_review_status == AgentReviewStatus.REJECTED.value: - return "rejected" - return "draft" - - def _next_available_version(self, asset: AgentAsset) -> str: - candidate = self._increment_version(self._resolve_working_version(asset)) - while self.repository.get_version(asset.id, candidate) is not None: - candidate = self._increment_version(candidate) - return candidate - - -class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, AgentAssetSpreadsheetHelperMixin, AgentAssetRiskRuleLevelMixin, AgentAssetRiskRulePublishMixin, AgentAssetRiskRuleFeedbackMixin, AgentAssetRiskRuleTestingMixin, AgentAssetRiskRuleSimulationMixin, AgentAssetTimelineMixin, AgentAssetJsonRuleMixin): - def __init__(self, db: Session) -> None: + current_user: CurrentUserContext | None = None, + ) -> None: self.db = db - self.repository = AgentAssetRepository(db) + self.access_scope = ( + AgentAssetAccessScope.from_user(current_user) if current_user is not None else None + ) + self.repository = AgentAssetRepository(db, access_scope=self.access_scope) self.audit_service = AuditLogService(db) self.spreadsheet_manager = AgentAssetSpreadsheetManager() self.rule_library_manager = AgentAssetRuleLibraryManager() @@ -409,10 +231,13 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent if asset is None: return None try: - if backfill_missing_risk_rule_score(asset): + can_persist_backfill = self.access_scope is None or self.access_scope.can_write(asset) + if can_persist_backfill and backfill_missing_risk_rule_score(asset): asset = self.repository.save_asset(asset) except Exception: - logger.warning("Failed to backfill risk rule score asset_id=%s", asset_id, exc_info=True) + logger.warning( + "Failed to backfill risk rule score asset_id=%s", asset_id, exc_info=True + ) working_version = self._resolve_working_version(asset) recent_versions = self._sort_versions( @@ -450,7 +275,9 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent @staticmethod def _filter_excluded_risk_assets(assets: list[AgentAsset]) -> list[AgentAsset]: - return [asset for asset in assets if not AgentAssetService._is_excluded_budget_risk_asset(asset)] + return [ + asset for asset in assets if not AgentAssetService._is_excluded_budget_risk_asset(asset) + ] @staticmethod def _is_excluded_budget_risk_asset(asset: AgentAsset) -> bool: @@ -481,7 +308,19 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent if payload.status == AgentAssetStatus.ACTIVE: raise ValueError("请先创建资产并完成审核，再通过上线接口激活。") + if self.access_scope is None: + tenant_id, resource_scope = platform_resource_identity() + elif payload.scope == "platform": + if not self.access_scope.is_platform_admin: + raise PermissionError("只有平台管理员可以创建平台资产。") + tenant_id, resource_scope = platform_resource_identity() + else: + tenant_id = self.access_scope.tenant_id + resource_scope = "tenant" + asset = AgentAsset( + tenant_id=tenant_id, + scope=resource_scope, asset_type=payload.asset_type.value, code=payload.code, name=payload.name, @@ -521,6 +360,16 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent before = self._asset_snapshot(asset) + config_json = asset.config_json if isinstance(asset.config_json, dict) else {} + is_json_risk = ( + asset.asset_type == AgentAssetType.RULE.value + and str(config_json.get("detail_mode") or "").strip().lower() == "json_risk" + ) + if is_json_risk and payload.published_version is not None: + raise ValueError("JSON 风险规则发布版本只能由 shadow/Canary 发布流程变更。") + if is_json_risk and payload.config_json is not None: + raise ValueError("JSON 风险规则运行配置只能由专用规则接口变更。") + if payload.status == AgentAssetStatus.ACTIVE: raise ValueError("请使用上线接口激活资产。") @@ -607,7 +456,7 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent content=serialized_content, content_type=payload.content_type.value, change_note=payload.change_note, - created_by=payload.created_by, + created_by=actor, ) created = self.repository.create_version(version) @@ -675,17 +524,18 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent review = AgentAssetReview( asset_id=asset_id, version=payload.version, - reviewer=payload.reviewer, + reviewer=actor, review_status=payload.review_status.value, review_note=payload.review_note, reviewed_at=None if payload.review_status == AgentReviewStatus.PENDING else datetime.now(UTC), + created_at=datetime.now(UTC), ) created = self.repository.create_review(review) before = self._asset_snapshot(asset) - asset.reviewer = payload.reviewer + asset.reviewer = actor if payload.review_status == AgentReviewStatus.PENDING: if not asset.published_version: asset.status = AgentAssetStatus.REVIEW.value @@ -810,6 +660,9 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent raise ValueError("资产尚未设置工作版本，无法上线。") if asset.asset_type == AgentAssetType.RULE.value: + config_json = asset.config_json if isinstance(asset.config_json, dict) else {} + if str(config_json.get("detail_mode") or "").strip().lower() == "json_risk": + raise ValueError("JSON 风险规则必须通过 shadow/Canary 发布流程上线。") review = self.repository.get_review( asset.id, candidate_version, AgentReviewStatus.APPROVED.value ) @@ -846,4 +699,3 @@ class AgentAssetService(AgentAssetVersionMixin, AgentAssetOnlyOfficeMixin, Agent synced_count += foundation.sync_platform_risk_rules_from_library() self.db.commit() return synced_count - diff --git a/server/src/app/services/agent_foundation.py b/server/src/app/services/agent_foundation.py index 08c1c99..61ab777 100644 --- a/server/src/app/services/agent_foundation.py +++ b/server/src/app/services/agent_foundation.py @@ -19,6 +19,7 @@ from app.services.agent_foundation_financial_seed import AgentFoundationFinancia from app.services.agent_foundation_markdown import AgentFoundationMarkdownMixin from app.services.agent_foundation_risk_rules import AgentFoundationRiskRuleMixin from app.services.agent_foundation_spreadsheets import AgentFoundationSpreadsheetMixin +from app.services.tenant_registry import TenantRegistryService logger = get_logger("app.services.agent_foundation") _foundation_ready_lock = threading.RLock() @@ -63,6 +64,7 @@ class AgentFoundationService( def _prepare_foundation(self) -> None: try: create_legacy_schema(self.db.get_bind()) + TenantRegistryService(self.db).ensure_builtin() self._ensure_agent_asset_schema() self._ensure_financial_record_schema() self._seed_agent_assets() diff --git a/server/src/app/services/agent_foundation_asset_helpers.py b/server/src/app/services/agent_foundation_asset_helpers.py index f95e317..bafeb0d 100644 --- a/server/src/app/services/agent_foundation_asset_helpers.py +++ b/server/src/app/services/agent_foundation_asset_helpers.py @@ -1,66 +1,27 @@ from __future__ import annotations -import hashlib -import json -from datetime import UTC, date, datetime -from decimal import Decimal -from pathlib import Path +from datetime import datetime from sqlalchemy import inspect, select, text from app.core.agent_enums import ( - AgentAssetContentType, - AgentAssetDomain, AgentAssetStatus, - AgentAssetType, - AgentName, - AgentPermissionLevel, - AgentReviewStatus, - AgentRunSource, - AgentRunStatus, - AgentToolType, -) -from app.models.agent_asset import AgentAsset, AgentAssetReview, AgentAssetVersion -from app.models.agent_run import AgentRun, AgentToolCall, SemanticParseLog -from app.models.audit_log import AuditLog -from app.models.financial_record import ( - AccountsPayableRecord, - AccountsReceivableRecord, - ExpenseClaim, - ExpenseClaimItem, -) -from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager -from app.services.agent_asset_spreadsheet import ( - AgentAssetSpreadsheetManager, - COMPANY_COMMUNICATION_EXPENSE_RULE_CODE, - COMPANY_COMMUNICATION_EXPENSE_RULE_FILENAME, - COMPANY_TRAVEL_EXPENSE_RULE_CODE, - COMPANY_TRAVEL_EXPENSE_RULE_FILENAME, - FINANCE_RULES_LIBRARY, - RISK_RULES_LIBRARY, -) -from app.services.expense_rule_runtime import ( - build_scene_submission_standard_markdown, - build_travel_risk_control_standard_markdown, -) -from app.services.agent_foundation_constants import ( - ATTACHMENT_RULE_ASSET_CODE, - ATTACHMENT_RULE_RUNTIME_CONFIG, - COMPANY_COMMUNICATION_RULE_SCENARIO_JSON, - COMPANY_COMMUNICATION_RULE_VERSION, - COMPANY_TRAVEL_RULE_SCENARIO_JSON, - COMPANY_TRAVEL_RULE_VERSION, - DEMO_EXPENSE_CLAIM_SIGNATURES, - DEMO_PAYABLE_SIGNATURES, - DEMO_RECEIVABLE_SIGNATURES, - LEGACY_RULE_CODES, - PLATFORM_DESTINATION_LOCATION_RULE_FILENAME, ) from app.core.logging import get_logger +from app.models.agent_asset import AgentAsset, AgentAssetReview, AgentAssetVersion +from app.models.audit_log import AuditLog +from app.services.agent_asset_access import platform_asset_statement +from app.services.agent_foundation_constants import ( + LEGACY_RULE_CODES, +) logger = get_logger("app.services.agent_foundation") class AgentFoundationAssetHelperMixin: + @staticmethod + def _platform_asset_stmt(): + return platform_asset_statement() + def _create_seed_asset( self, @@ -247,7 +208,7 @@ class AgentFoundationAssetHelperMixin: self.db.scalars( - select(AgentAsset).where(AgentAsset.code.in_(LEGACY_RULE_CODES)) + self._platform_asset_stmt().where(AgentAsset.code.in_(LEGACY_RULE_CODES)) ).all() diff --git a/server/src/app/services/agent_foundation_asset_seed.py b/server/src/app/services/agent_foundation_asset_seed.py index 494c11c..263a189 100644 --- a/server/src/app/services/agent_foundation_asset_seed.py +++ b/server/src/app/services/agent_foundation_asset_seed.py @@ -34,7 +34,6 @@ from app.services.agent_foundation_constants import ( DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE, DIGITAL_EMPLOYEE_PROFILE_SCAN_TASK_CODE, DIGITAL_EMPLOYEE_RISK_GRAPH_SCAN_TASK_CODE, - DIGITAL_EMPLOYEE_RULE_DISCOVERY_TASK_CODE, DIGITAL_EMPLOYEE_SKILL_CATEGORIES, DIGITAL_EMPLOYEE_TASK_CATEGORY_MAP, ) @@ -177,7 +176,14 @@ class AgentFoundationAssetSeedMixin: def _seed_agent_assets(self) -> None: - existing_codes = set(self.db.scalars(select(AgentAsset.code)).all()) + existing_codes = set( + self.db.scalars( + select(AgentAsset.code).where( + AgentAsset.scope == "platform", + AgentAsset.tenant_id == "platform", + ) + ).all() + ) if existing_codes: @@ -513,7 +519,14 @@ class AgentFoundationAssetSeedMixin: self.db.flush() self._upsert_runtime_digital_employee_tasks( - set(self.db.scalars(select(AgentAsset.code)).all()) + set( + self.db.scalars( + select(AgentAsset.code).where( + AgentAsset.scope == "platform", + AgentAsset.tenant_id == "platform", + ) + ).all() + ) ) self.db.flush() diff --git a/server/src/app/services/agent_foundation_asset_topup.py b/server/src/app/services/agent_foundation_asset_topup.py index 1222319..8c1beb7 100644 --- a/server/src/app/services/agent_foundation_asset_topup.py +++ b/server/src/app/services/agent_foundation_asset_topup.py @@ -14,6 +14,7 @@ from app.core.agent_enums import ( from app.core.logging import get_logger from app.models.agent_asset import AgentAsset from app.models.agent_run import AgentRun +from app.services.agent_asset_access import platform_asset_statement from app.services.agent_asset_spreadsheet import ( COMPANY_COMMUNICATION_EXPENSE_RULE_CODE, COMPANY_PREAPPROVAL_RULE_CODE, @@ -43,7 +44,9 @@ class AgentFoundationAssetTopUpMixin: def _remove_legacy_digital_employee_assets(self) -> None: assets = list( self.db.scalars( - select(AgentAsset).where(AgentAsset.code.in_(DIGITAL_EMPLOYEE_LEGACY_TASK_CODES)) + platform_asset_statement().where( + AgentAsset.code.in_(DIGITAL_EMPLOYEE_LEGACY_TASK_CODES) + ) ).all() ) if not assets: @@ -65,7 +68,7 @@ class AgentFoundationAssetTopUpMixin: has_changes = False for code, category in DIGITAL_EMPLOYEE_TASK_CATEGORY_MAP.items(): - asset = self.db.scalar(select(AgentAsset).where(AgentAsset.code == code)) + asset = self.db.scalar(platform_asset_statement().where(AgentAsset.code == code)) if asset is None: continue @@ -95,31 +98,50 @@ class AgentFoundationAssetTopUpMixin: self._remove_legacy_rule_assets() self._remove_legacy_digital_employee_assets() - existing_codes = set(self.db.scalars(select(AgentAsset.code)).all()) + existing_codes = set( + self.db.scalars( + select(AgentAsset.code).where( + AgentAsset.scope == "platform", + AgentAsset.tenant_id == "platform", + ) + ).all() + ) self._sync_digital_employee_skill_categories() attachment_rule = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == ATTACHMENT_RULE_ASSET_CODE) + platform_asset_statement().where( + AgentAsset.code == ATTACHMENT_RULE_ASSET_CODE + ) ) scene_submission_rule = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == "rule.expense.scene_submission_standard") + platform_asset_statement().where( + AgentAsset.code == "rule.expense.scene_submission_standard" + ) ) travel_policy_rule = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == "rule.expense.travel_risk_control_standard") + platform_asset_statement().where( + AgentAsset.code == "rule.expense.travel_risk_control_standard" + ) ) company_travel_rule = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == COMPANY_TRAVEL_EXPENSE_RULE_CODE) + platform_asset_statement().where( + AgentAsset.code == COMPANY_TRAVEL_EXPENSE_RULE_CODE + ) ) company_communication_rule = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == COMPANY_COMMUNICATION_EXPENSE_RULE_CODE) + platform_asset_statement().where( + AgentAsset.code == COMPANY_COMMUNICATION_EXPENSE_RULE_CODE + ) ) company_preapproval_rule = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == COMPANY_PREAPPROVAL_RULE_CODE) + platform_asset_statement().where( + AgentAsset.code == COMPANY_PREAPPROVAL_RULE_CODE + ) ) if ATTACHMENT_RULE_ASSET_CODE not in existing_codes: @@ -752,7 +774,9 @@ class AgentFoundationAssetTopUpMixin: else: asset = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE) + platform_asset_statement().where( + AgentAsset.code == DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE + ) ) if asset is None: return diff --git a/server/src/app/services/agent_foundation_digital_employee_tasks.py b/server/src/app/services/agent_foundation_digital_employee_tasks.py index 50c861a..e022ce0 100644 --- a/server/src/app/services/agent_foundation_digital_employee_tasks.py +++ b/server/src/app/services/agent_foundation_digital_employee_tasks.py @@ -1,7 +1,5 @@ from __future__ import annotations -from sqlalchemy import select - from app.core.agent_enums import ( AgentAssetContentType, AgentAssetDomain, @@ -10,6 +8,7 @@ from app.core.agent_enums import ( AgentName, ) from app.models.agent_asset import AgentAsset +from app.services.agent_asset_access import platform_asset_statement from app.services.agent_foundation_constants import ( DIGITAL_EMPLOYEE_ALGORITHM_REPLAY_TASK_CODE, DIGITAL_EMPLOYEE_BUDGET_PRECONTROL_TASK_CODE, @@ -501,7 +500,7 @@ class AgentFoundationDigitalEmployeeTaskMixin: config_json=config, ) else: - asset = self.db.scalar(select(AgentAsset).where(AgentAsset.code == code)) + asset = self.db.scalar(platform_asset_statement().where(AgentAsset.code == code)) if asset is None: return self._refresh_runtime_digital_employee_asset(asset, spec) diff --git a/server/src/app/services/agent_foundation_financial_seed.py b/server/src/app/services/agent_foundation_financial_seed.py index a6eb6b0..6ab1373 100644 --- a/server/src/app/services/agent_foundation_financial_seed.py +++ b/server/src/app/services/agent_foundation_financial_seed.py @@ -1,26 +1,19 @@ from __future__ import annotations -import hashlib -import json from datetime import UTC, date, datetime from decimal import Decimal -from pathlib import Path -from sqlalchemy import inspect, select, text +from sqlalchemy import select from app.core.agent_enums import ( - AgentAssetContentType, - AgentAssetDomain, - AgentAssetStatus, - AgentAssetType, AgentName, AgentPermissionLevel, - AgentReviewStatus, AgentRunSource, AgentRunStatus, AgentToolType, ) -from app.models.agent_asset import AgentAsset, AgentAssetReview, AgentAssetVersion +from app.core.logging import get_logger +from app.models.agent_asset import AgentAsset from app.models.agent_run import AgentRun, AgentToolCall, SemanticParseLog from app.models.audit_log import AuditLog from app.models.financial_record import ( @@ -29,47 +22,33 @@ from app.models.financial_record import ( ExpenseClaim, ExpenseClaimItem, ) -from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager -from app.services.agent_asset_spreadsheet import ( - AgentAssetSpreadsheetManager, - COMPANY_COMMUNICATION_EXPENSE_RULE_CODE, - COMPANY_COMMUNICATION_EXPENSE_RULE_FILENAME, - COMPANY_TRAVEL_EXPENSE_RULE_CODE, - COMPANY_TRAVEL_EXPENSE_RULE_FILENAME, - FINANCE_RULES_LIBRARY, - RISK_RULES_LIBRARY, -) -from app.services.expense_rule_runtime import ( - build_scene_submission_standard_markdown, - build_travel_risk_control_standard_markdown, -) +from app.services.agent_asset_access import platform_asset_statement from app.services.agent_foundation_constants import ( ATTACHMENT_RULE_ASSET_CODE, - ATTACHMENT_RULE_RUNTIME_CONFIG, - COMPANY_COMMUNICATION_RULE_SCENARIO_JSON, - COMPANY_COMMUNICATION_RULE_VERSION, - COMPANY_TRAVEL_RULE_SCENARIO_JSON, - COMPANY_TRAVEL_RULE_VERSION, DEMO_EXPENSE_CLAIM_SIGNATURES, DEMO_PAYABLE_SIGNATURES, DEMO_RECEIVABLE_SIGNATURES, DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE, - LEGACY_RULE_CODES, - PLATFORM_DESTINATION_LOCATION_RULE_FILENAME, ) -from app.core.logging import get_logger +from app.services.tenant_registry import DEFAULT_TENANT_ID logger = get_logger("app.services.agent_foundation") class AgentFoundationFinancialSeedMixin: def _seed_financial_records(self) -> None: - if self.db.scalar(select(ExpenseClaim.id).limit(1)) is not None: + if self.db.scalar( + select(ExpenseClaim.id) + .where(ExpenseClaim.tenant_id == DEFAULT_TENANT_ID) + .limit(1) + ) is not None: return claim_1 = ExpenseClaim( + tenant_id=DEFAULT_TENANT_ID, + claim_no="EXP-202605-001", employee_name="张三", @@ -140,6 +119,8 @@ class AgentFoundationFinancialSeedMixin: claim_2 = ExpenseClaim( + tenant_id=DEFAULT_TENANT_ID, + claim_no="EXP-202605-002", employee_name="李四", @@ -174,6 +155,8 @@ class AgentFoundationFinancialSeedMixin: claim_3 = ExpenseClaim( + tenant_id=DEFAULT_TENANT_ID, + claim_no="EXP-202605-003", employee_name="王五", @@ -209,6 +192,7 @@ class AgentFoundationFinancialSeedMixin: ar_records = [ AccountsReceivableRecord( + tenant_id=DEFAULT_TENANT_ID, receivable_no="AR-202605-001", @@ -241,6 +225,7 @@ class AgentFoundationFinancialSeedMixin: ), AccountsReceivableRecord( + tenant_id=DEFAULT_TENANT_ID, receivable_no="AR-202605-002", @@ -277,6 +262,7 @@ class AgentFoundationFinancialSeedMixin: ap_records = [ AccountsPayableRecord( + tenant_id=DEFAULT_TENANT_ID, payable_no="AP-202605-001", @@ -307,6 +293,7 @@ class AgentFoundationFinancialSeedMixin: ), AccountsPayableRecord( + tenant_id=DEFAULT_TENANT_ID, payable_no="AP-202605-002", @@ -342,7 +329,13 @@ class AgentFoundationFinancialSeedMixin: def _purge_demo_financial_records(self) -> None: - demo_claims = list(self.db.scalars(select(ExpenseClaim)).all()) + demo_claims = list( + self.db.scalars( + select(ExpenseClaim).where( + ExpenseClaim.tenant_id == DEFAULT_TENANT_ID + ) + ).all() + ) for claim in demo_claims: @@ -364,7 +357,13 @@ class AgentFoundationFinancialSeedMixin: self.db.delete(claim) - demo_receivables = list(self.db.scalars(select(AccountsReceivableRecord)).all()) + demo_receivables = list( + self.db.scalars( + select(AccountsReceivableRecord).where( + AccountsReceivableRecord.tenant_id == DEFAULT_TENANT_ID + ) + ).all() + ) for record in demo_receivables: @@ -384,7 +383,13 @@ class AgentFoundationFinancialSeedMixin: self.db.delete(record) - demo_payables = list(self.db.scalars(select(AccountsPayableRecord)).all()) + demo_payables = list( + self.db.scalars( + select(AccountsPayableRecord).where( + AccountsPayableRecord.tenant_id == DEFAULT_TENANT_ID + ) + ).all() + ) for record in demo_payables: @@ -412,7 +417,9 @@ class AgentFoundationFinancialSeedMixin: task_asset = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE) + platform_asset_statement().where( + AgentAsset.code == DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE + ) ) diff --git a/server/src/app/services/agent_foundation_risk_rules.py b/server/src/app/services/agent_foundation_risk_rules.py index 27b9c52..0a85537 100644 --- a/server/src/app/services/agent_foundation_risk_rules.py +++ b/server/src/app/services/agent_foundation_risk_rules.py @@ -13,6 +13,7 @@ from app.core.agent_enums import ( ) from app.core.logging import get_logger from app.models.agent_asset import AgentAsset +from app.services.agent_asset_access import platform_asset_statement from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager from app.services.agent_asset_spreadsheet import ( RISK_RULES_LIBRARY, @@ -325,7 +326,14 @@ class AgentFoundationRiskRuleMixin: def sync_platform_risk_rules_from_library(self) -> int: - existing_codes = set(self.db.scalars(select(AgentAsset.code)).all()) + existing_codes = set( + self.db.scalars( + select(AgentAsset.code).where( + AgentAsset.scope == "platform", + AgentAsset.tenant_id == "platform", + ) + ).all() + ) before_count = len(existing_codes) @@ -339,7 +347,14 @@ class AgentFoundationRiskRuleMixin: self.db.flush() - after_codes = set(self.db.scalars(select(AgentAsset.code)).all()) + after_codes = set( + self.db.scalars( + select(AgentAsset.code).where( + AgentAsset.scope == "platform", + AgentAsset.tenant_id == "platform", + ) + ).all() + ) synced = max(len(after_codes) - before_count, 0) @@ -361,7 +376,9 @@ class AgentFoundationRiskRuleMixin: def _hide_stale_demo_risk_rules(self, manifest_codes: set[str]) -> None: assets = self.db.scalars( - select(AgentAsset).where(AgentAsset.asset_type == AgentAssetType.RULE.value) + platform_asset_statement().where( + AgentAsset.asset_type == AgentAssetType.RULE.value + ) ).all() for asset in assets: config = asset.config_json if isinstance(asset.config_json, dict) else {} @@ -400,7 +417,9 @@ class AgentFoundationRiskRuleMixin: scenario_json = self._platform_risk_scenario_json(manifest) - asset = self.db.scalar(select(AgentAsset).where(AgentAsset.code == rule_code)) + asset = self.db.scalar( + platform_asset_statement().where(AgentAsset.code == rule_code) + ) if asset is None and rule_code not in existing_codes: diff --git a/server/src/app/services/agent_foundation_spreadsheets.py b/server/src/app/services/agent_foundation_spreadsheets.py index 95a4c7a..db192c1 100644 --- a/server/src/app/services/agent_foundation_spreadsheets.py +++ b/server/src/app/services/agent_foundation_spreadsheets.py @@ -2,24 +2,23 @@ from __future__ import annotations from pathlib import Path -from sqlalchemy import select - from app.core.agent_enums import ( AgentAssetContentType, AgentAssetDomain, + AgentAssetStatus, AgentAssetType, AgentReviewStatus, - AgentAssetStatus, ) from app.core.logging import get_logger from app.models.agent_asset import AgentAsset +from app.services.agent_asset_access import platform_asset_statement from app.services.agent_asset_spreadsheet import ( COMPANY_COMMUNICATION_EXPENSE_RULE_CODE, COMPANY_COMMUNICATION_EXPENSE_RULE_FILENAME, - COMPANY_TRAVEL_ALLOWANCE_RULE_CODE, - COMPANY_TRAVEL_ALLOWANCE_RULE_FILENAME, COMPANY_PREAPPROVAL_RULE_CODE, COMPANY_PREAPPROVAL_RULE_FILENAME, + COMPANY_TRAVEL_ALLOWANCE_RULE_CODE, + COMPANY_TRAVEL_ALLOWANCE_RULE_FILENAME, COMPANY_TRAVEL_EXPENSE_RULE_CODE, COMPANY_TRAVEL_EXPENSE_RULE_FILENAME, COMPANY_TRAVEL_GRADE_MAPPING_RULE_CODE, @@ -41,13 +40,13 @@ from app.services.agent_foundation_constants import ( COMPANY_TRAVEL_RULE_SCENARIO_JSON, COMPANY_TRAVEL_RULE_VERSION, ) +from app.services.agent_foundation_preapproval_spreadsheet import ( + build_preapproval_rule_workbook_sheets, +) from app.services.finance_rule_catalog import ( DEPRECATED_FINANCE_RULE_CODES, DEPRECATED_FINANCE_RULE_REPLACEMENTS, ) -from app.services.agent_foundation_preapproval_spreadsheet import ( - build_preapproval_rule_workbook_sheets, -) logger = get_logger("app.services.agent_foundation") @@ -212,7 +211,7 @@ class AgentFoundationSpreadsheetMixin: tag: str = "基础规则", refresh_workbook_content: bool = False, ) -> bool: - asset = self.db.scalar(select(AgentAsset).where(AgentAsset.code == code)) + asset = self.db.scalar(platform_asset_statement().where(AgentAsset.code == code)) created_asset = asset is None if asset is None: asset = self._create_seed_asset( @@ -376,7 +375,7 @@ class AgentFoundationSpreadsheetMixin: def _hide_deprecated_finance_rule_assets(self) -> None: for code in DEPRECATED_FINANCE_RULE_CODES: - asset = self.db.scalar(select(AgentAsset).where(AgentAsset.code == code)) + asset = self.db.scalar(platform_asset_statement().where(AgentAsset.code == code)) if asset is None: continue asset.status = AgentAssetStatus.DISABLED.value diff --git a/server/src/app/services/agent_run_access_policy.py b/server/src/app/services/agent_run_access_policy.py new file mode 100644 index 0000000..3e3266b --- /dev/null +++ b/server/src/app/services/agent_run_access_policy.py @@ -0,0 +1,201 @@ +from __future__ import annotations + +from typing import Any + +from fastapi import HTTPException, status +from sqlalchemy import and_, func, not_, or_, select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.agent_run import AgentRun +from app.schemas.agent_run import AgentRunRead +from app.services.finance_dashboard_access_policy import FinanceDashboardAccessPolicy +from app.services.finance_dashboard_scope import ( + FINANCE_DASHBOARD_TASK_TYPE, + resolve_finance_dashboard_data_scope, +) + + +class AgentRunAccessPolicy: + """对租户边界和敏感领域权限做返回前的第二层校验。""" + + @classmethod + def build_query_scope(cls, current_user: CurrentUserContext) -> Any: + """把财务快照领域门禁下推到 limit 之前，避免不可见记录挤占窗口。""" + tenant_id = cls.require_current_tenant_id(current_user) + route_task_type = func.coalesce( + AgentRun.route_json["task_type"].as_string(), + "", + ) + route_job_type = func.coalesce( + AgentRun.route_json["job_type"].as_string(), + "", + ) + is_finance_snapshot = or_( + route_task_type == FINANCE_DASHBOARD_TASK_TYPE, + route_job_type == FINANCE_DASHBOARD_TASK_TYPE, + ) + if not FinanceDashboardAccessPolicy.can_read(current_user): + return not_(is_finance_snapshot) + + expected_data_scope = resolve_finance_dashboard_data_scope(tenant_id) + valid_finance_scope = and_( + AgentRun.route_json["tenant_id"].as_string() == tenant_id, + AgentRun.ontology_json["tenant_id"].as_string() == tenant_id, + AgentRun.route_json["data_scope"].as_string() == expected_data_scope, + AgentRun.ontology_json["data_scope"].as_string() == expected_data_scope, + ) + return or_(not_(is_finance_snapshot), valid_finance_scope) + + @classmethod + def filter_list_items( + cls, + runs: list[AgentRunRead], + current_user: CurrentUserContext, + db: Session, + ) -> list[AgentRunRead]: + payloads_by_run_id = cls._run_scope_payloads( + db, + [run.run_id for run in runs], + ) + current_tenant_id = cls.require_current_tenant_id(current_user) + visible: list[AgentRunRead] = [] + for run in runs: + payloads = payloads_by_run_id.get(run.run_id) + if payloads is None: + continue + route_json, ontology_json = payloads + if cls._tenant_scope_from_payloads(route_json, ontology_json) != current_tenant_id: + continue + if cls.is_finance_dashboard_snapshot(run) and not cls._can_read_finance_snapshot_scope( + cls._finance_scope_from_payloads(route_json, ontology_json), + current_user, + ): + continue + visible.append(run) + return visible + + @classmethod + def _run_scope_payloads( + cls, + db: Session, + run_ids: list[str], + ) -> dict[str, tuple[object, object]]: + if not run_ids: + return {} + rows = db.execute( + select(AgentRun.run_id, AgentRun.route_json, AgentRun.ontology_json).where( + AgentRun.run_id.in_(run_ids) + ) + ).all() + return { + str(run_id): (route_json, ontology_json) for run_id, route_json, ontology_json in rows + } + + @classmethod + def require_detail_read( + cls, + run: AgentRunRead, + current_user: CurrentUserContext, + ) -> None: + current_tenant_id = cls.require_current_tenant_id(current_user) + if cls._tenant_scope_from_payloads(run.route_json, run.ontology_json) != current_tenant_id: + cls._raise_not_found() + if not cls.is_finance_dashboard_snapshot(run): + return + + run_scope = cls._finance_scope_from_payloads(run.route_json, run.ontology_json) + expected_scope = ( + resolve_finance_dashboard_data_scope(current_tenant_id) if current_tenant_id else None + ) + if ( + current_tenant_id is None + or run_scope is None + or run_scope != (current_tenant_id, expected_scope) + ): + cls._raise_not_found() + FinanceDashboardAccessPolicy.require_read(current_user) + + @classmethod + def is_finance_dashboard_snapshot(cls, run: AgentRunRead) -> bool: + route = run.route_json if isinstance(run.route_json, dict) else {} + return any( + str(route.get(key) or "").strip() == FINANCE_DASHBOARD_TASK_TYPE + for key in ("task_type", "job_type") + ) + + @classmethod + def _can_read_finance_snapshot_scope( + cls, + run_scope: tuple[str, str] | None, + current_user: CurrentUserContext, + ) -> bool: + current_tenant_id = cls._normalized_tenant_id(current_user.tenant_id) + expected_scope = ( + resolve_finance_dashboard_data_scope(current_tenant_id) + if current_tenant_id is not None + else None + ) + return bool( + current_tenant_id + and run_scope + and run_scope == (current_tenant_id, expected_scope) + and FinanceDashboardAccessPolicy.can_read(current_user) + ) + + @classmethod + def _finance_scope_from_payloads( + cls, + *payloads: object, + ) -> tuple[str, str] | None: + tenant_ids: list[str] = [] + data_scopes: list[str] = [] + for payload in payloads: + if not isinstance(payload, dict): + return None + tenant_id = cls._normalized_tenant_id(payload.get("tenant_id")) + data_scope = str(payload.get("data_scope") or "").strip() + if tenant_id is None or not data_scope: + return None + tenant_ids.append(tenant_id) + data_scopes.append(data_scope) + if not tenant_ids or len(set(tenant_ids)) != 1 or len(set(data_scopes)) != 1: + return None + return tenant_ids[0], data_scopes[0] + + @classmethod + def _tenant_scope_from_payloads( + cls, + *payloads: object, + ) -> str | None: + tenant_ids: list[str] = [] + for payload in payloads: + if not isinstance(payload, dict) or "tenant_id" not in payload: + return None + tenant_id = cls._normalized_tenant_id(payload.get("tenant_id")) + if tenant_id is None: + return None + tenant_ids.append(tenant_id) + if len(tenant_ids) != len(payloads) or len(set(tenant_ids)) != 1: + return None + return tenant_ids[0] + + @classmethod + def require_current_tenant_id(cls, current_user: CurrentUserContext) -> str: + tenant_id = cls._normalized_tenant_id(current_user.tenant_id) + if tenant_id is None: + cls._raise_not_found() + return tenant_id + + @staticmethod + def _raise_not_found() -> None: + # 统一按不存在处理，避免 run_id 或作用域标记成为租户探针。 + raise HTTPException( + status_code=status.HTTP_404_NOT_FOUND, + detail="Run not found", + ) + + @staticmethod + def _normalized_tenant_id(value: object) -> str | None: + normalized = str(value or "").strip() + return normalized or None diff --git a/server/src/app/services/agent_runs.py b/server/src/app/services/agent_runs.py index fcf3f1e..80306e7 100644 --- a/server/src/app/services/agent_runs.py +++ b/server/src/app/services/agent_runs.py @@ -7,8 +7,8 @@ from typing import Any from sqlalchemy.orm import Session -from app.core.config import get_settings from app.core.agent_enums import AgentName, AgentPermissionLevel, AgentRunStatus +from app.core.config import get_settings from app.core.logging import get_logger from app.models.agent_run import AgentRun, AgentToolCall, SemanticParseLog from app.repositories.agent_run import AgentRunRepository @@ -19,6 +19,7 @@ from app.schemas.agent_run import ( SemanticParseRead, ) from app.services.agent_foundation import AgentFoundationService +from app.services.commercial_runtime_bridge import CommercialRuntimeBridge from app.services.knowledge_ingest_log import enrich_knowledge_ingest_route_json logger = get_logger("app.services.agent_runs") @@ -67,30 +68,94 @@ class AgentRunService: status: str | None = None, source: str | None = None, limit: int = 20, + ) -> list[AgentRunRead]: + """供后台任务等受信任内部流程读取全部作用域。""" + return self._list_runs( + agent=agent, + status=status, + source=source, + limit=limit, + tenant_id=None, + scope_clause=None, + ) + + def list_runs_for_tenant( + self, + *, + tenant_id: str, + agent: str | None = None, + status: str | None = None, + source: str | None = None, + limit: int = 20, + scope_clause: Any | None = None, + ) -> list[AgentRunRead]: + return self._list_runs( + agent=agent, + status=status, + source=source, + limit=limit, + tenant_id=self._require_tenant_id(tenant_id), + scope_clause=scope_clause, + ) + + def _list_runs( + self, + *, + agent: str | None, + status: str | None, + source: str | None, + limit: int, + tenant_id: str | None, + scope_clause: Any | None, ) -> list[AgentRunRead]: self._ensure_ready() - self._reconcile_stale_knowledge_index_runs() + self._reconcile_stale_knowledge_index_runs(tenant_id=tenant_id) rows = self.repository.list_light( agent=agent, status=status, source=source, limit=limit, + tenant_id=tenant_id, + scope_clause=scope_clause, ) + run_ids = [str(item["run_id"]) for item in rows] tool_calls_by_run_id = self._group_light_tool_calls( - self.repository.list_light_tool_calls([str(item["run_id"]) for item in rows]) + self.repository.list_light_tool_calls(run_ids) ) + semantic_parses_by_run_id = self.repository.list_light_semantic_parses(run_ids) return [ self._serialize_run_list_item( item, tool_calls_by_run_id.get(str(item["run_id"]), []), + semantic_parses_by_run_id.get(str(item["run_id"])), ) for item in rows ] def get_run(self, run_id: str) -> AgentRunRead | None: + """供持有可信 run_id 的内部流程跨作用域读取。""" + return self._get_run(run_id, tenant_id=None) + + def get_run_for_tenant( + self, + run_id: str, + *, + tenant_id: str, + ) -> AgentRunRead | None: + return self._get_run(run_id, tenant_id=self._require_tenant_id(tenant_id)) + + def _get_run( + self, + run_id: str, + *, + tenant_id: str | None, + ) -> AgentRunRead | None: self._ensure_ready() - self._reconcile_stale_knowledge_index_runs(target_run_id=run_id) - run = self.repository.get_by_run_id(run_id) + self._reconcile_stale_knowledge_index_runs( + target_run_id=run_id, + tenant_id=tenant_id, + ) + run = self.repository.get_by_run_id(run_id, tenant_id=tenant_id) if run is None: return None return self._serialize_run(run, enrich_knowledge_ingest=True) @@ -102,10 +167,56 @@ class AgentRunService: status: str | None = None, source: str | None = None, limit: int = 200, + ) -> AgentRunStatsRead: + """供后台诊断等受信任内部流程聚合全部作用域。""" + return self._summarize_runs( + agent=agent, + status=status, + source=source, + limit=limit, + tenant_id=None, + scope_clause=None, + ) + + def summarize_runs_for_tenant( + self, + *, + tenant_id: str, + agent: str | None = None, + status: str | None = None, + source: str | None = None, + limit: int = 200, + scope_clause: Any | None = None, + ) -> AgentRunStatsRead: + return self._summarize_runs( + agent=agent, + status=status, + source=source, + limit=limit, + tenant_id=self._require_tenant_id(tenant_id), + scope_clause=scope_clause, + ) + + def _summarize_runs( + self, + *, + agent: str | None, + status: str | None, + source: str | None, + limit: int, + tenant_id: str | None, + scope_clause: Any | None, ) -> AgentRunStatsRead: self._ensure_ready() - self._reconcile_stale_knowledge_index_runs() - runs = self.repository.list(agent=agent, status=status, source=source, limit=limit) + self._reconcile_stale_knowledge_index_runs(tenant_id=tenant_id) + runs = self.repository.list( + agent=agent, + status=status, + source=source, + limit=limit, + tenant_id=tenant_id, + scope_clause=scope_clause, + ) agents: dict[str, int] = {} statuses: dict[str, int] = {} tool_statuses: dict[str, int] = {} @@ -180,6 +291,7 @@ class AgentRunService: *, agent: str, source: str, + tenant_id: str | None = None, user_id: str | None = None, task_id: str | None = None, ontology_json: dict[str, Any] | None = None, @@ -192,14 +304,26 @@ class AgentRunService: finished_at: datetime | None = None, ) -> AgentRunRead: self._ensure_ready() + normalized_tenant_id = self._require_tenant_id(tenant_id) if tenant_id is not None else None + scoped_ontology_json = dict(ontology_json or {}) + scoped_route_json = dict(route_json or {}) + if normalized_tenant_id is not None: + scoped_ontology_json = self._stamp_tenant_scope( + scoped_ontology_json, + normalized_tenant_id, + ) + scoped_route_json = self._stamp_tenant_scope( + scoped_route_json, + normalized_tenant_id, + ) run = AgentRun( run_id=f"run_{uuid.uuid4().hex[:16]}", agent=agent, source=source, user_id=user_id, task_id=task_id, - ontology_json=ontology_json or {}, - route_json=route_json or {}, + ontology_json=scoped_ontology_json, + route_json=scoped_route_json, permission_level=permission_level, status=status, result_summary=result_summary, @@ -228,13 +352,22 @@ class AgentRunService: run = self.repository.get_by_run_id(run_id) if run is None: raise LookupError("Run not found") + existing_tenant_id = self._existing_tenant_id(run) if agent is not None: run.agent = agent if ontology_json is not None: - run.ontology_json = ontology_json + run.ontology_json = ( + self._stamp_tenant_scope(ontology_json, existing_tenant_id) + if existing_tenant_id is not None + else ontology_json + ) if route_json is not None: - run.route_json = route_json + run.route_json = ( + self._stamp_tenant_scope(route_json, existing_tenant_id) + if existing_tenant_id is not None + else route_json + ) if permission_level is not None: run.permission_level = permission_level if status is not None: @@ -267,6 +400,9 @@ class AgentRunService: route_json = dict(run.route_json or {}) route_json.update(route_patch or {}) + existing_tenant_id = self._existing_tenant_id(run) + if existing_tenant_id is not None: + route_json = self._stamp_tenant_scope(route_json, existing_tenant_id) run.route_json = route_json if status is not None: @@ -279,13 +415,18 @@ class AgentRunService: run.finished_at = finished_at updated = self.repository.save_run(run) - logger.info("Merged route_json for agent run run_id=%s status=%s", updated.run_id, updated.status) + logger.info( + "Merged route_json for agent run run_id=%s status=%s", + updated.run_id, + updated.status, + ) return self._serialize_run(updated) def record_tool_call( self, *, run_id: str, + tool_call_id: str | None = None, tool_type: str, tool_name: str, request_json: dict[str, Any] | None = None, @@ -296,6 +437,7 @@ class AgentRunService: ) -> AgentToolCallRead: self._ensure_ready() tool_call = AgentToolCall( + id=tool_call_id or str(uuid.uuid4()), run_id=run_id, tool_type=tool_type, tool_name=tool_name, @@ -307,7 +449,9 @@ class AgentRunService: ) created = self.repository.create_tool_call(tool_call) logger.info("Recorded tool call run_id=%s tool=%s", run_id, tool_name) - return AgentToolCallRead.model_validate(created) + result = AgentToolCallRead.model_validate(created) + CommercialRuntimeBridge(self.db).sync_tool_call(created.id) + return result def update_tool_call( self, @@ -336,7 +480,9 @@ class AgentRunService: updated = self.repository.save_tool_call(tool_call) logger.info("Updated tool call id=%s status=%s", updated.id, updated.status) - return AgentToolCallRead.model_validate(updated) + result = AgentToolCallRead.model_validate(updated) + CommercialRuntimeBridge(self.db).sync_tool_call(updated.id) + return result def record_semantic_parse( self, @@ -378,18 +524,28 @@ class AgentRunService: def _ensure_ready(self) -> None: AgentFoundationService(self.db).ensure_foundation_ready() - def _reconcile_stale_knowledge_index_runs(self, *, target_run_id: str | None = None) -> None: - runs = self.repository.list( - agent=AgentName.HERMES.value, - status=AgentRunStatus.RUNNING.value, - limit=200, - ) + def _reconcile_stale_knowledge_index_runs( + self, + *, + target_run_id: str | None = None, + tenant_id: str | None = None, + ) -> None: + if target_run_id is not None: + target = self.repository.get_by_run_id( + target_run_id, + tenant_id=tenant_id, + ) + runs = [target] if target is not None else [] + else: + runs = self.repository.list( + agent=AgentName.HERMES.value, + status=AgentRunStatus.RUNNING.value, + limit=200, + tenant_id=tenant_id, + ) now = datetime.now(UTC) for run in runs: - if target_run_id is not None and run.run_id != target_run_id: - continue - route_json = dict(run.route_json or {}) if str(route_json.get("job_type") or "").strip() not in KNOWLEDGE_SYNC_JOB_TYPES: continue @@ -415,11 +571,21 @@ class AgentRunService: KnowledgeService, ) - KnowledgeService(db=self.db).set_document_ingest_statuses( - stale_document_ids, - KNOWLEDGE_INGEST_STATUS_FAILED, - agent_run_id=run.run_id, - ) + run_tenant_id = self._existing_tenant_id(run) + if run_tenant_id is None: + logger.error( + "Refused stale knowledge status reconciliation without tenant run_id=%s", + run.run_id, + ) + else: + KnowledgeService( + db=self.db, + tenant_id=run_tenant_id, + ).set_document_ingest_statuses( + stale_document_ids, + KNOWLEDGE_INGEST_STATUS_FAILED, + agent_run_id=run.run_id, + ) route_json.update( { @@ -445,6 +611,39 @@ class AgentRunService: except ValueError: return None + @staticmethod + def _require_tenant_id(value: object) -> str: + normalized = str(value or "").strip() + if not normalized: + raise ValueError("tenant_id 不能为空。") + return normalized + + @classmethod + def _stamp_tenant_scope( + cls, + payload: dict[str, Any], + tenant_id: str, + ) -> dict[str, Any]: + scoped = dict(payload) + if "tenant_id" in scoped: + existing_tenant_id = cls._require_tenant_id(scoped.get("tenant_id")) + if existing_tenant_id != tenant_id: + raise ValueError("Agent Run tenant_id 与业务上下文冲突。") + scoped["tenant_id"] = tenant_id + return scoped + + @classmethod + def _existing_tenant_id(cls, run: AgentRun) -> str | None: + tenant_ids: list[str] = [] + for payload in (run.route_json, run.ontology_json): + if not isinstance(payload, dict) or "tenant_id" not in payload: + return None + try: + tenant_ids.append(cls._require_tenant_id(payload.get("tenant_id"))) + except ValueError: + return None + return tenant_ids[0] if len(set(tenant_ids)) == 1 else None + def _serialize_run( self, run: AgentRun, @@ -483,6 +682,7 @@ class AgentRunService: self, row: dict[str, Any], tool_calls: list[dict[str, Any]], + semantic_parse: dict[str, Any] | None, ) -> AgentRunRead: return AgentRunRead( id=str(row["id"]), @@ -500,7 +700,9 @@ class AgentRunService: started_at=row["started_at"], finished_at=row.get("finished_at"), tool_calls=[self._serialize_light_tool_call(item) for item in tool_calls], - semantic_parse=None, + semantic_parse=( + SemanticParseRead.model_validate(semantic_parse) if semantic_parse else None + ), ) def _build_list_route_json(self, row: dict[str, Any]) -> dict[str, Any]: diff --git a/server/src/app/services/approval_task_backfill.py b/server/src/app/services/approval_task_backfill.py index 911ba27..f85976f 100644 --- a/server/src/app/services/approval_task_backfill.py +++ b/server/src/app/services/approval_task_backfill.py @@ -512,7 +512,10 @@ class ApprovalTaskBackfillService: employee = claim_employee.manager if claim_employee is not None else None if employee is None: manager_name = self.claim_policy.resolve_claim_manager_name(claim) - employee = self.claim_policy.resolve_employee_by_identity_candidates([manager_name]) + employee = self.claim_policy.resolve_employee_by_identity_candidates( + [manager_name], + tenant_id=self.claim_policy.resolve_structured_claim_tenant_id(claim), + ) elif node_key == "budget_manager": employee = self.claim_policy.resolve_department_budget_manager(claim) elif node_key == "finance": diff --git a/server/src/app/services/approval_task_lifecycle.py b/server/src/app/services/approval_task_lifecycle.py index 7ea27f9..efea7ea 100644 --- a/server/src/app/services/approval_task_lifecycle.py +++ b/server/src/app/services/approval_task_lifecycle.py @@ -116,16 +116,19 @@ class ApprovalTaskLifecycleService: self._supersede_stale_root(existing, claim=claim) assignment = self.resolve_assignment(claim, node=node) - sequence = int( - self.db.scalar( - select(func.max(ApprovalTask.node_sequence)).where( - ApprovalTask.tenant_id == tenant, - ApprovalTask.claim_id == claim.id, - ApprovalTask.task_kind == "root", + sequence = ( + int( + self.db.scalar( + select(func.max(ApprovalTask.node_sequence)).where( + ApprovalTask.tenant_id == tenant, + ApprovalTask.claim_id == claim.id, + ApprovalTask.task_kind == "root", + ) ) + or 0 ) - or 0 - ) + 1 + + 1 + ) now = datetime.now(UTC) normalized_entered_at = _aware_utc(entered_at) task_id = str(uuid.uuid4()) @@ -206,9 +209,7 @@ class ApprovalTaskLifecycleService: }, before_json={}, business_event_id=business_event.id if business_event is not None else None, - correlation_id=( - business_event.correlation_id if business_event is not None else None - ), + correlation_id=(business_event.correlation_id if business_event is not None else None), occurred_at=now, ) return task @@ -301,11 +302,7 @@ class ApprovalTaskLifecycleService: next_task = self.ensure_root_task( claim, tenant_id=task.tenant_id, - entered_at=( - business_event.occurred_at - if business_event is not None - else now - ), + entered_at=(business_event.occurred_at if business_event is not None else now), entered_at_source=("workflow_event" if business_event is not None else "backfill"), business_event=business_event, actor_id="system", @@ -326,9 +323,7 @@ class ApprovalTaskLifecycleService: related_tasks=related_tasks, approval_action_ledger_id=ledger.id, business_event_id=business_event.id if business_event is not None else None, - correlation_id=( - business_event.correlation_id if business_event is not None else None - ), + correlation_id=(business_event.correlation_id if business_event is not None else None), occurred_at=now, ) return response @@ -390,7 +385,8 @@ class ApprovalTaskLifecycleService: manager = employee.manager if employee is not None else None if manager is None: manager = self.access.claim_policy.resolve_employee_by_identity_candidates( - [str(claim.manager_name or "").strip()] + [str(claim.manager_name or "").strip()], + tenant_id=self.access.claim_policy.resolve_structured_claim_tenant_id(claim), ) if manager is None: raise ApprovalTaskConfigurationError( diff --git a/server/src/app/services/auth.py b/server/src/app/services/auth.py index 3af9ee5..36ebe9c 100644 --- a/server/src/app/services/auth.py +++ b/server/src/app/services/auth.py @@ -14,11 +14,17 @@ from app.core.security import verify_password from app.models.auth_session import AuthSession from app.models.employee import Employee from app.models.financial_record import ExpenseClaim +from app.models.tenant import Tenant, TenantMembership from app.schemas.auth import AuthUserRead, LoginRequest, LoginResponse from app.services.auth_sessions import AuthSessionService from app.services.employee import EmployeeService from app.services.employee_seed import ROLE_DISPLAY_ORDER from app.services.settings import SettingsService +from app.services.tenant_registry import ( + DEFAULT_TENANT_ID, + PLATFORM_TENANT_ID, + required_tenant_id, +) from app.services.user_session_metrics import UserSessionMetricService logger = get_logger("app.services.auth") @@ -51,10 +57,10 @@ class AuthenticatedUser: role_codes: list[str] email: str avatar: str + tenant_id: str is_admin: bool = False employee_id: str | None = None department_id: str | None = None - tenant_id: str = "default" class AuthService: @@ -71,7 +77,11 @@ class AuthService: logger.info("Admin login succeeded identifier=%s", identifier) return self._build_login_response(admin_user) - employee_user = self._authenticate_employee(identifier, password) + employee_user = self._authenticate_employee( + identifier, + password, + requested_tenant=payload.tenant_id, + ) if employee_user is not None: logger.info( "Employee login succeeded identifier=%s role_codes=%s", @@ -127,6 +137,19 @@ class AuthService: selectinload(Employee.manager), selectinload(Employee.roles), ) + stmt = ( + stmt.join( + TenantMembership, + (TenantMembership.tenant_id == Employee.tenant_id) + & (TenantMembership.employee_id == Employee.id), + ) + .join(Tenant, Tenant.tenant_id == Employee.tenant_id) + .where( + Employee.tenant_id == required_tenant_id(auth_session.tenant_id), + TenantMembership.status == "active", + Tenant.status == "active", + ) + ) if auth_session.employee_id: stmt = stmt.where(Employee.id == auth_session.employee_id) else: @@ -140,21 +163,27 @@ class AuthService: def _restore_session_scope( user: AuthenticatedUser, auth_session: AuthSession, - ) -> AuthenticatedUser: - """会话恢复时以签发并认证过的会话租户为准，禁止回落到默认租户。""" + ) -> AuthenticatedUser | None: + """会话与当前主体租户不一致时立即失效，禁止覆盖或回落。""" - user.tenant_id = str(auth_session.tenant_id or "default").strip() or "default" + try: + session_tenant = required_tenant_id(auth_session.tenant_id) + user_tenant = required_tenant_id(user.tenant_id) + except ValueError: + return None + if session_tenant != user_tenant: + return None return user - def get_user_snapshot(self, identifier: str) -> AuthUserRead | None: + def get_user_snapshot(self, identifier: str, *, tenant_id: str) -> AuthUserRead | None: normalized = identifier.strip() if not normalized: return None - employee = self._find_employee_by_email(normalized) - if employee is None: - EmployeeService(self.db).ensure_directory_ready() - employee = self._find_employee_by_email(normalized) + employee = self._find_employee_by_email( + normalized, + requested_tenant=tenant_id, + ) if employee is None or employee.employment_status == "停用": return None @@ -189,22 +218,40 @@ class AuthService: role_codes=["manager"], email=admin_email or f"{admin_username}@local", avatar=display_name[:1].upper(), + tenant_id=PLATFORM_TENANT_ID, is_admin=True, ) - def _authenticate_employee(self, identifier: str, password: str) -> AuthenticatedUser | None: + def _authenticate_employee( + self, + identifier: str, + password: str, + *, + requested_tenant: str | None, + ) -> AuthenticatedUser | None: if not self.settings.setup_completed: return None try: - employee = self._find_employee_by_email(identifier) + employee = self._find_employee_by_email( + identifier, + requested_tenant=requested_tenant, + ) except SQLAlchemyError: self.db.rollback() employee = None if employee is None: - EmployeeService(self.db).ensure_directory_ready() - employee = self._find_employee_by_email(identifier) + normalized_request = str(requested_tenant or "").strip() + if normalized_request in {"", DEFAULT_TENANT_ID}: + EmployeeService( + self.db, + tenant_id=DEFAULT_TENANT_ID, + ).ensure_directory_ready() + employee = self._find_employee_by_email( + identifier, + requested_tenant=requested_tenant, + ) if employee is None or not employee.password_hash: return None @@ -218,7 +265,12 @@ class AuthService: return self._build_employee_user(employee) - def _find_employee_by_email(self, identifier: str) -> Employee | None: + def _find_employee_by_email( + self, + identifier: str, + *, + requested_tenant: str | None, + ) -> Employee | None: stmt = ( select(Employee) .options( @@ -226,9 +278,30 @@ class AuthService: selectinload(Employee.manager), selectinload(Employee.roles), ) - .where(func.lower(Employee.email) == identifier.lower()) + .join( + TenantMembership, + (TenantMembership.tenant_id == Employee.tenant_id) + & (TenantMembership.employee_id == Employee.id), + ) + .join(Tenant, Tenant.tenant_id == Employee.tenant_id) + .where( + func.lower(Employee.email) == identifier.lower(), + TenantMembership.status == "active", + Tenant.status == "active", + ) ) - return self.db.execute(stmt).scalars().first() + normalized_tenant = str(requested_tenant or "").strip() + if normalized_tenant: + stmt = stmt.where( + or_( + Tenant.tenant_id == normalized_tenant, + Tenant.tenant_code == normalized_tenant, + ) + ) + employees = list(self.db.execute(stmt.limit(2)).scalars().unique().all()) + if len(employees) > 1: + raise ValueError("该账号关联多个企业，请填写企业代码后再登录。") + return employees[0] if employees else None def _build_employee_user(self, employee: Employee) -> AuthenticatedUser: sorted_roles = sorted( @@ -258,6 +331,7 @@ class AuthService: role_codes=role_codes or ["user"], email=employee.email, avatar=(employee.name or "?")[:1].upper(), + tenant_id=required_tenant_id(employee.tenant_id), is_admin=False, employee_id=employee.id, department_id=employee.organization_unit_id, @@ -285,7 +359,11 @@ class AuthService: stmt = ( select(ExpenseClaim) - .where(or_(*conditions), ExpenseClaim.occurred_at >= since) + .where( + ExpenseClaim.tenant_id == required_tenant_id(employee.tenant_id), + or_(*conditions), + ExpenseClaim.occurred_at >= since, + ) .order_by(ExpenseClaim.occurred_at.desc()) .limit(30) ) @@ -334,4 +412,5 @@ class AuthService: email=user.email, avatar=user.avatar, isAdmin=user.is_admin, + tenantId=user.tenant_id, ) diff --git a/server/src/app/services/auth_sessions.py b/server/src/app/services/auth_sessions.py index d0232bb..b8cc40a 100644 --- a/server/src/app/services/auth_sessions.py +++ b/server/src/app/services/auth_sessions.py @@ -9,6 +9,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session from app.models.auth_session import AuthSession +from app.services.tenant_registry import required_tenant_id DEFAULT_SESSION_TIMEOUT_MINUTES = 30 MIN_SESSION_TIMEOUT_MINUTES = 5 @@ -35,9 +36,10 @@ class AuthSessionService: ), ) access_token = secrets.token_urlsafe(32) + tenant_id = required_tenant_id(getattr(user, "tenant_id", None)) auth_session = AuthSession( token_hash=self.hash_token(access_token), - tenant_id=str(getattr(user, "tenant_id", "default") or "default").strip() or "default", + tenant_id=tenant_id, principal_type="admin" if bool(getattr(user, "is_admin", False)) else "employee", employee_id=str(getattr(user, "employee_id", "") or "").strip() or None, username=str(getattr(user, "username", "") or "").strip(), diff --git a/server/src/app/services/automation_eligibility.py b/server/src/app/services/automation_eligibility.py new file mode 100644 index 0000000..52255ca --- /dev/null +++ b/server/src/app/services/automation_eligibility.py @@ -0,0 +1,349 @@ +"""动作级自动化资格计算。 + +该模块只计算一个动作在当前证据下允许达到的最高自动化级别，不执行动作。 +默认策略保持在人审/影子模式；只有系统硬白名单与企业白名单的交集才可能进入 +自动执行，资金支付和高风险制度动作始终需要人工确认。 +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from decimal import Decimal, InvalidOperation +from enum import IntEnum +from math import isfinite +from typing import Any, Literal + + +class AutomationLevel(IntEnum): + """与费用闭环契约一致的动作级别；L5 不是更高自治。""" + + L0_EXPLAIN = 0 + L1_RECOMMEND = 1 + L2_PREFILL = 2 + L3_REVERSIBLE_AUTO = 3 + L4_LOW_RISK_STRAIGHT_THROUGH = 4 + L5_HUMAN_CONTROLLED = 5 + + +ActionRisk = Literal["low", "medium", "high", "critical"] + + +SYSTEM_AUTO_WHITELIST = frozenset( + { + "expense.prefill_fields", + "expense.classify_receipt", + "expense.deduplicate_attachment", + "expense.request_missing_information", + "expense.route_for_review", + "expense.apply_user_preference", + } +) + +NEVER_AUTO_ACTIONS = frozenset( + { + "expense.execute_payment", + "expense.release_funds", + "expense.change_policy", + "expense.publish_policy", + "expense.disable_control", + "expense.waive_high_risk_control", + } +) + + +@dataclass(frozen=True) +class AutomationPolicy: + """企业自动化上限；企业白名单不能扩张系统硬白名单。""" + + enabled: bool = False + enterprise_max_level: AutomationLevel = AutomationLevel.L1_RECOMMEND + enterprise_whitelist: frozenset[str] = field(default_factory=frozenset) + amount_cap: Decimal = Decimal("0") + min_confidence: float = 0.95 + min_evidence_completeness: float = 0.95 + min_historical_precision: float = 0.98 + min_historical_samples: int = 100 + minimum_sampling_rate: float = 0.10 + release_stage: Literal["shadow", "canary", "active"] = "shadow" + + +@dataclass(frozen=True) +class AutomationEligibilityInput: + action_key: str + action_risk: ActionRisk = "medium" + amount: Decimal = Decimal("0") + confidence: float = 0.0 + evidence_completeness: float = 0.0 + reversible: bool = False + historical_precision: float | None = None + historical_samples: int = 0 + sampling_rate: float = 0.0 + moves_money: bool = False + changes_policy: bool = False + + +@dataclass(frozen=True) +class AutomationEligibilityDecision: + eligible_level: AutomationLevel + mode: str + eligible_for_auto_execution: bool + requires_human_confirmation: bool + sampling_rate: float + reasons: tuple[str, ...] + checks: dict[str, Any] + + def to_dict(self) -> dict[str, Any]: + return { + "eligible_level": int(self.eligible_level), + "level_name": self.eligible_level.name, + "mode": self.mode, + "eligible_for_auto_execution": self.eligible_for_auto_execution, + "requires_human_confirmation": self.requires_human_confirmation, + "sampling_rate": self.sampling_rate, + "reasons": list(self.reasons), + "checks": self.checks, + } + + +class AutomationEligibilityCalculator: + """按最弱控制项计算动作的最高资格，所有异常输入均收紧而非放宽。""" + + def calculate( + self, + candidate: AutomationEligibilityInput, + policy: AutomationPolicy | None = None, + ) -> AutomationEligibilityDecision: + policy = policy or AutomationPolicy() + action_key = str(candidate.action_key or "").strip() + amount, amount_valid = _validated_amount(candidate.amount) + confidence = _clamp01(candidate.confidence) + evidence = _clamp01(candidate.evidence_completeness) + precision = _optional_clamp01(candidate.historical_precision) + sampling_rate = _clamp01(candidate.sampling_rate) + historical_samples, historical_samples_valid = _safe_count( + candidate.historical_samples, + default=0, + ) + minimum_samples, minimum_samples_valid = _safe_count( + policy.min_historical_samples, + default=100, + minimum=1, + ) + enterprise_level = _safe_level(policy.enterprise_max_level) + enterprise_whitelist = ( + policy.enterprise_whitelist + if isinstance(policy.enterprise_whitelist, (set, frozenset, list, tuple)) + else frozenset() + ) + release_stage = str(policy.release_stage or "").strip().lower() + risk = ( + candidate.action_risk + if candidate.action_risk in {"low", "medium", "high", "critical"} + else "critical" + ) + system_whitelisted = action_key in SYSTEM_AUTO_WHITELIST + enterprise_whitelisted = action_key in enterprise_whitelist + explicitly_forbidden = ( + action_key in NEVER_AUTO_ACTIONS + or candidate.moves_money + or candidate.changes_policy + or risk in {"high", "critical"} + ) + checks = { + "policy_enabled": bool(policy.enabled), + "system_whitelisted": system_whitelisted, + "enterprise_whitelisted": enterprise_whitelisted, + "action_risk": risk, + "amount": str(amount), + "amount_valid": amount_valid, + "amount_cap": str(_non_negative_decimal(policy.amount_cap)), + "confidence": confidence, + "evidence_completeness": evidence, + "reversible": bool(candidate.reversible), + "historical_precision": precision, + "historical_samples": historical_samples, + "historical_samples_valid": historical_samples_valid, + "minimum_historical_samples": minimum_samples, + "minimum_historical_samples_valid": minimum_samples_valid, + "sampling_rate": sampling_rate, + "enterprise_max_level": int(enterprise_level), + "release_stage": release_stage + if release_stage in {"shadow", "canary", "active"} + else "shadow", + "money_or_policy_forbidden": explicitly_forbidden, + } + + if enterprise_level == AutomationLevel.L5_HUMAN_CONTROLLED: + enterprise_level = AutomationLevel.L0_EXPLAIN + if not policy.enabled: + return self._decision( + min(AutomationLevel.L1_RECOMMEND, enterprise_level), + sampling_rate, + ("enterprise_automation_disabled",), + checks, + release_stage="shadow", + ) + if explicitly_forbidden: + return self._decision( + AutomationLevel.L5_HUMAN_CONTROLLED, + sampling_rate, + ("money_policy_or_high_risk_action_requires_human",), + checks, + release_stage=release_stage, + ) + if not system_whitelisted or not enterprise_whitelisted: + return self._decision( + min(AutomationLevel.L1_RECOMMEND, enterprise_level), + sampling_rate, + ("action_not_in_hard_whitelist",), + checks, + release_stage="shadow", + ) + + reasons: list[str] = [] + level = AutomationLevel.L4_LOW_RISK_STRAIGHT_THROUGH + amount_cap = _non_negative_decimal(policy.amount_cap) + if not amount_valid: + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("amount_invalid") + elif amount > amount_cap: + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("amount_exceeds_enterprise_cap") + if risk != "low": + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("action_risk_not_low") + if not candidate.reversible: + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("action_not_reversible") + if confidence < _clamp01(policy.min_confidence): + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("confidence_below_threshold") + if evidence < _clamp01(policy.min_evidence_completeness): + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("evidence_incomplete") + if precision is None or precision < _clamp01(policy.min_historical_precision): + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("historical_precision_insufficient") + if not historical_samples_valid: + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("historical_samples_invalid") + if not minimum_samples_valid: + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("historical_sample_policy_invalid") + if historical_samples < minimum_samples: + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("historical_sample_insufficient") + if sampling_rate < _clamp01(policy.minimum_sampling_rate): + level = min(level, AutomationLevel.L2_PREFILL) + reasons.append("sampling_rate_below_threshold") + + if level > enterprise_level: + level = enterprise_level + reasons.append("clamped_by_enterprise_level") + if ( + level + in { + AutomationLevel.L3_REVERSIBLE_AUTO, + AutomationLevel.L4_LOW_RISK_STRAIGHT_THROUGH, + } + and not reasons + ): + reasons.append("all_automation_controls_passed") + return self._decision( + level, + sampling_rate, + tuple(reasons), + checks, + release_stage=release_stage, + ) + + @staticmethod + def _decision( + level: AutomationLevel, + sampling_rate: float, + reasons: tuple[str, ...], + checks: dict[str, Any], + *, + release_stage: str, + ) -> AutomationEligibilityDecision: + action_mode = { + AutomationLevel.L0_EXPLAIN: "explain_only", + AutomationLevel.L1_RECOMMEND: "recommend", + AutomationLevel.L2_PREFILL: "prefill", + AutomationLevel.L3_REVERSIBLE_AUTO: "reversible_auto", + AutomationLevel.L4_LOW_RISK_STRAIGHT_THROUGH: "low_risk_straight_through", + AutomationLevel.L5_HUMAN_CONTROLLED: "human_controlled", + }[level] + normalized_stage = ( + release_stage if release_stage in {"shadow", "canary", "active"} else "shadow" + ) + automatic = level in { + AutomationLevel.L3_REVERSIBLE_AUTO, + AutomationLevel.L4_LOW_RISK_STRAIGHT_THROUGH, + } and normalized_stage in {"canary", "active"} + return AutomationEligibilityDecision( + eligible_level=level, + mode=(f"shadow_{action_mode}" if normalized_stage == "shadow" else action_mode), + eligible_for_auto_execution=automatic, + requires_human_confirmation=not automatic, + sampling_rate=sampling_rate, + reasons=reasons, + checks=checks, + ) + + +def _safe_level(value: AutomationLevel | int) -> AutomationLevel: + try: + numeric = int(value) + except (TypeError, ValueError, OverflowError): + return AutomationLevel.L1_RECOMMEND + return AutomationLevel(max(0, min(5, numeric))) + + +def _non_negative_decimal(value: Decimal | int | float | str) -> Decimal: + try: + parsed = Decimal(str(value)) + except (InvalidOperation, TypeError, ValueError): + return Decimal("0") + if not parsed.is_finite(): + return Decimal("0") + return max(Decimal("0"), parsed) + + +def _validated_amount(value: Decimal | int | float | str) -> tuple[Decimal, bool]: + try: + parsed = Decimal(str(value)) + except (InvalidOperation, TypeError, ValueError): + return Decimal("0"), False + if not parsed.is_finite() or parsed < 0: + return Decimal("0"), False + return parsed, True + + +def _clamp01(value: float | int | Decimal | None) -> float: + try: + parsed = float(value or 0) + except (TypeError, ValueError): + return 0.0 + if not isfinite(parsed): + return 0.0 + return max(0.0, min(1.0, parsed)) + + +def _optional_clamp01(value: float | int | Decimal | None) -> float | None: + return None if value is None else _clamp01(value) + + +def _safe_count( + value: Any, + *, + default: int, + minimum: int = 0, +) -> tuple[int, bool]: + try: + parsed = int(value) + except (TypeError, ValueError, OverflowError): + return max(minimum, default), False + if parsed < minimum: + return max(minimum, default), False + return parsed, True diff --git a/server/src/app/services/cfo_value_analytics.py b/server/src/app/services/cfo_value_analytics.py new file mode 100644 index 0000000..1036eec --- /dev/null +++ b/server/src/app/services/cfo_value_analytics.py @@ -0,0 +1,605 @@ +from __future__ import annotations + +from collections import defaultdict +from datetime import UTC, datetime +from decimal import Decimal +from typing import Any + +from sqlalchemy import Select, select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.savings import SavingsOpportunity, SavingsRealization +from app.schemas.cfo_value import ( + CfoValueBreakdownGroupRead, + CfoValueBreakdownItemRead, + CfoValueCashKpiRead, + CfoValueDashboardRead, + CfoValueDataQualityRead, + CfoValueFiltersRead, + CfoValueFunnelRead, + CfoValueFunnelStageRead, + CfoValueGuardrailRead, + CfoValueKpisRead, + CfoValueMoneyRead, + CfoValueSourceRead, + CfoValueTrendPointRead, + CfoValueUnavailableKpiRead, + CfoValueWindowRead, +) +from app.services.savings_access_policy import SavingsAccessPolicy + +MoneyMap = dict[str, Decimal] + + +class CfoValueAnalyticsService: + """只从 Savings Ledger 聚合可追溯 CFO 价值，不推导演示收益。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def build_dashboard( + self, + current_user: CurrentUserContext, + *, + start: datetime, + end: datetime, + as_of: datetime, + filters: CfoValueFiltersRead, + ) -> CfoValueDashboardRead: + SavingsAccessPolicy.require_tenant_value_read(current_user) + start, end, as_of = self._normalize_window(start, end, as_of) + tenant_id = str(current_user.tenant_id or "").strip() + if not tenant_id: + raise PermissionError("当前登录上下文缺少租户，不能聚合经营价值。") + + opportunity_statement = select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == tenant_id, + SavingsOpportunity.created_at >= start, + SavingsOpportunity.created_at <= end, + SavingsOpportunity.created_at <= as_of, + ) + opportunity_statement = SavingsAccessPolicy.apply_opportunity_read_scope( + opportunity_statement, + current_user, + include_owner=False, + ) + opportunity_statement = self._apply_filters(opportunity_statement, filters) + opportunities = list(self.db.scalars(opportunity_statement).all()) + + realization_statement = ( + select(SavingsRealization, SavingsOpportunity) + .join( + SavingsOpportunity, + (SavingsOpportunity.tenant_id == SavingsRealization.tenant_id) + & (SavingsOpportunity.id == SavingsRealization.opportunity_id), + ) + .where( + SavingsRealization.tenant_id == tenant_id, + SavingsRealization.realized_at >= start, + SavingsRealization.realized_at <= end, + SavingsRealization.realized_at <= as_of, + ) + ) + realization_statement = SavingsAccessPolicy.apply_opportunity_read_scope( + realization_statement, + current_user, + include_owner=False, + ) + realization_statement = self._apply_filters(realization_statement, filters) + realization_rows = list(self.db.execute(realization_statement).all()) + opportunity_by_id = {item.id: item for item in opportunities} + for _, opportunity in realization_rows: + opportunity_by_id.setdefault(opportunity.id, opportunity) + relevant_opportunities = list(opportunity_by_id.values()) + + verified_rows = [ + (realization, opportunity) + for realization, opportunity in realization_rows + if self._is_verified(realization, opportunity, as_of=as_of) + ] + pending_rows = [ + (realization, opportunity) + for realization, opportunity in realization_rows + if realization.status == "pending_confirmation" + ] + verified_cash = self._sum_realizations(verified_rows) + pending_cash = self._sum_realizations(pending_rows) + reversal_rows = [row for row in verified_rows if row[0].realization_type == "reversal"] + reversal_money = self._sum_realizations(reversal_rows) + estimated = self._sum_opportunities(opportunities) + in_progress = self._sum_opportunities( + [item for item in opportunities if item.status == "in_progress"] + ) + rejected_or_expired = self._sum_opportunities( + [item for item in opportunities if item.status in {"rejected", "expired"}] + ) + mature = [ + item + for item in opportunities + if item.status in {"realized", "verified", "reversed", "rejected", "expired"} + or ( + item.status == "in_progress" + and item.due_at is not None + and self._utc(item.due_at) <= as_of + ) + ] + mature_estimated = self._sum_opportunities(mature) + + data_quality = self._data_quality(realization_rows) + freshness_at = self._freshness(opportunities, realization_rows) + source_status = ( + "empty" + if not opportunities and not realization_rows + else ( + "partial" + if data_quality.missing_evidence_count or data_quality.missing_fx_count + else "complete" + ) + ) + return CfoValueDashboardRead( + window=CfoValueWindowRead(start=start, end=end, as_of=as_of), + source=CfoValueSourceRead( + generated_at=datetime.now(UTC), + freshness_at=freshness_at, + data_status=source_status, + opportunity_count=len(relevant_opportunities), + realization_count=len(realization_rows), + coverage_notes=[ + "现金价值仅包含 canonical 且已独立财务确认的实际结果。", + "当前付款证据为平台业务状态，尚未接入银行或 ERP 回执。", + ], + ), + filters=filters, + kpis=CfoValueKpisRead( + verified_cash=CfoValueCashKpiRead( + status="available" if verified_cash else "empty", + values=self._money_read(verified_cash), + confirmed_realization_count=len(verified_rows), + definition=( + "按实际发生期间汇总 canonical、finance_confirmed 的现金结果，" + "包含只追加负向冲回并扣除执行成本。" + ), + ), + releasable_labor=CfoValueUnavailableKpiRead( + key="verified_releasable_labor_value", + label="财务确认可释放工时价值", + status="collecting", + reason="尚未形成客户认可的活跃工时、角色成本与可释放比例基线。", + required_inputs=[ + "上线前后每单人工活跃分钟", + "生效期内角色完全成本", + "客户确认的可释放比例", + ], + ), + safe_straight_through=CfoValueUnavailableKpiRead( + key="safe_straight_through_rate", + label="安全智能直通率", + status="collecting", + reason="尚未冻结 eligibility 队列并完成事后审计结果回填。", + required_inputs=[ + "创建时 eligibility 快照", + "必要审批完成事实", + "事后审计重大问题结果", + ], + ), + ), + funnel=self._funnel( + opportunities, + pending_rows=pending_rows, + verified_rows=verified_rows, + reversal_rows=reversal_rows, + estimated=estimated, + in_progress=in_progress, + pending_cash=pending_cash, + verified_cash=verified_cash, + reversal_money=reversal_money, + rejected_or_expired=rejected_or_expired, + mature_estimated=mature_estimated, + ), + trend=self._trend(realization_rows, as_of=as_of), + breakdowns=self._breakdowns(relevant_opportunities, verified_rows), + guardrails=self._guardrails( + realization_rows, + verified_rows=verified_rows, + reversal_rows=reversal_rows, + pending_cash=pending_cash, + data_quality=data_quality, + ), + data_quality=data_quality, + ) + + @staticmethod + def _apply_filters( + statement: Select[Any], + filters: CfoValueFiltersRead, + ) -> Select[Any]: + scalar_filters = ( + (SavingsOpportunity.owner_id, filters.owner_id), + (SavingsOpportunity.source_type, filters.source_type), + (SavingsOpportunity.value_kind, filters.value_kind), + ) + for column, value in scalar_filters: + normalized = str(value or "").strip() + if normalized: + statement = statement.where(column == normalized) + for key, value in ( + ("department_id", filters.department_id), + ("project_code", filters.project_code), + ("expense_type", filters.expense_type), + ("supplier_id", filters.supplier_id), + ("city", filters.city), + ): + normalized = str(value or "").strip() + if normalized: + statement = statement.where( + SavingsOpportunity.dimension_json[key].as_string() == normalized + ) + return statement + + def _funnel( + self, + opportunities: list[SavingsOpportunity], + *, + pending_rows: list[tuple[SavingsRealization, SavingsOpportunity]], + verified_rows: list[tuple[SavingsRealization, SavingsOpportunity]], + reversal_rows: list[tuple[SavingsRealization, SavingsOpportunity]], + estimated: MoneyMap, + in_progress: MoneyMap, + pending_cash: MoneyMap, + verified_cash: MoneyMap, + reversal_money: MoneyMap, + rejected_or_expired: MoneyMap, + mature_estimated: MoneyMap, + ) -> CfoValueFunnelRead: + rejected_count = sum(1 for item in opportunities if item.status in {"rejected", "expired"}) + rates: dict[str, Decimal | None] = {} + for currency in sorted(set(mature_estimated) | set(verified_cash)): + denominator = mature_estimated.get(currency, Decimal("0")) + rates[currency] = ( + (verified_cash.get(currency, Decimal("0")) / denominator).quantize( + Decimal("0.0001") + ) + if denominator > 0 + else None + ) + return CfoValueFunnelRead( + stages=[ + CfoValueFunnelStageRead( + key="estimated", + label="已识别预计机会", + count=len(opportunities), + values=self._money_read(estimated), + ), + CfoValueFunnelStageRead( + key="in_progress", + label="执行中", + count=sum(1 for item in opportunities if item.status == "in_progress"), + values=self._money_read(in_progress), + ), + CfoValueFunnelStageRead( + key="actual_pending", + label="实际待确认", + count=len(pending_rows), + values=self._money_read(pending_cash), + ), + CfoValueFunnelStageRead( + key="verified", + label="财务已确认", + count=sum( + 1 + for realization, _ in verified_rows + if realization.realization_type == "actual" + ), + values=self._money_read(verified_cash), + ), + CfoValueFunnelStageRead( + key="reversed", + label="已冲回", + count=len(reversal_rows), + values=self._money_read(reversal_money), + ), + CfoValueFunnelStageRead( + key="rejected_or_expired", + label="拒绝或到期", + count=rejected_count, + values=self._money_read(rejected_or_expired), + ), + ], + mature_estimated_values=self._money_read(mature_estimated), + verified_values=self._money_read(verified_cash), + realization_rate_by_currency=rates, + ) + + def _trend( + self, + rows: list[tuple[SavingsRealization, SavingsOpportunity]], + *, + as_of: datetime, + ) -> list[CfoValueTrendPointRead]: + buckets: dict[tuple[str, str], dict[str, Decimal]] = defaultdict( + lambda: { + "verified_net": Decimal("0"), + "actual_pending": Decimal("0"), + "reversal": Decimal("0"), + } + ) + for realization, opportunity in rows: + period = realization.realized_at.strftime("%Y-%m") + bucket = buckets[(period, realization.reporting_currency)] + if self._is_verified(realization, opportunity, as_of=as_of): + bucket["verified_net"] += Decimal(realization.reporting_amount) + if realization.realization_type == "reversal": + bucket["reversal"] += Decimal(realization.reporting_amount) + elif realization.status == "pending_confirmation": + bucket["actual_pending"] += Decimal(realization.reporting_amount) + return [ + CfoValueTrendPointRead( + period=period, + currency=currency, + **values, + ) + for (period, currency), values in sorted(buckets.items()) + ] + + def _breakdowns( + self, + opportunities: list[SavingsOpportunity], + verified_rows: list[tuple[SavingsRealization, SavingsOpportunity]], + ) -> list[CfoValueBreakdownGroupRead]: + verified_by_opportunity: dict[str, MoneyMap] = defaultdict(lambda: defaultdict(Decimal)) + for realization, opportunity in verified_rows: + verified_by_opportunity[opportunity.id][realization.reporting_currency] += Decimal( + realization.reporting_amount + ) + dimensions = ( + ("department", "department_id", "department_name"), + ("project", "project_code", "project_code"), + ("expense_type", "expense_type", "expense_type"), + ("supplier", "supplier_id", "supplier_name"), + ("city", "city", "city"), + ("owner", None, None), + ("source", None, None), + ) + groups: list[CfoValueBreakdownGroupRead] = [] + for dimension, id_key, name_key in dimensions: + aggregate: dict[str, dict[str, Any]] = {} + for opportunity in opportunities: + dimension_id, dimension_name = self._dimension_value( + opportunity, + dimension=dimension, + id_key=id_key, + name_key=name_key, + ) + if not dimension_id: + continue + item = aggregate.setdefault( + dimension_id, + { + "name": dimension_name or dimension_id, + "count": 0, + "estimated": defaultdict(Decimal), + "verified": defaultdict(Decimal), + }, + ) + item["count"] += 1 + item["estimated"][opportunity.reporting_currency] += Decimal( + opportunity.estimated_net + ) + for currency, amount in verified_by_opportunity.get(opportunity.id, {}).items(): + item["verified"][currency] += amount + items = [ + CfoValueBreakdownItemRead( + dimension=dimension, + dimension_id=dimension_id, + dimension_name=data["name"], + opportunity_count=data["count"], + verified_values=self._money_read(data["verified"]), + estimated_values=self._money_read(data["estimated"]), + ) + for dimension_id, data in aggregate.items() + ] + items.sort( + key=lambda item: sum(value.amount for value in item.verified_values), + reverse=True, + ) + groups.append(CfoValueBreakdownGroupRead(dimension=dimension, items=items[:20])) + return groups + + def _guardrails( + self, + rows: list[tuple[SavingsRealization, SavingsOpportunity]], + *, + verified_rows: list[tuple[SavingsRealization, SavingsOpportunity]], + reversal_rows: list[tuple[SavingsRealization, SavingsOpportunity]], + pending_cash: MoneyMap, + data_quality: CfoValueDataQualityRead, + ) -> list[CfoValueGuardrailRead]: + confirmed_actual_count = sum( + 1 for realization, _ in verified_rows if realization.realization_type == "actual" + ) + reversal_rate = ( + (Decimal(len(reversal_rows)) / Decimal(confirmed_actual_count)).quantize( + Decimal("0.0001") + ) + if confirmed_actual_count + else None + ) + return [ + CfoValueGuardrailRead( + key="pending_confirmation", + label="实际结果待财务确认", + status="attention" if data_quality.pending_confirmation_count else "ok", + count=data_quality.pending_confirmation_count, + values=self._money_read(pending_cash), + ), + CfoValueGuardrailRead( + key="post_confirmation_reversal_rate", + label="确认后冲回率", + status="attention" if reversal_rows else "ok", + count=len(reversal_rows), + rate=reversal_rate, + ), + CfoValueGuardrailRead( + key="actual_over_estimate", + label="实际超过预计异常", + status="attention" if data_quality.actual_over_estimate_count else "ok", + count=data_quality.actual_over_estimate_count, + ), + CfoValueGuardrailRead( + key="evidence_or_dedupe_gap", + label="证据或去重待补齐", + status=( + "attention" + if data_quality.missing_evidence_count or data_quality.pending_dedupe_count + else "ok" + ), + count=(data_quality.missing_evidence_count + data_quality.pending_dedupe_count), + ), + CfoValueGuardrailRead( + key="confirmed_high_risk_exposure", + label="开放且已确认的高危风险暴露", + status="unavailable", + reason="风险观察池尚未建立与经济义务一致的 canonical 去重键，不能汇总为金额。", + ), + ] + + @staticmethod + def _data_quality( + rows: list[tuple[SavingsRealization, SavingsOpportunity]], + ) -> CfoValueDataQualityRead: + pending = business_state = pending_dedupe = missing_fx = missing_evidence = over = 0 + for realization, opportunity in rows: + pending += realization.status == "pending_confirmation" + business_state += ( + str((realization.final_snapshot_json or {}).get("evidence_level") or "") + == "business_state" + ) + pending_dedupe += realization.dedupe_status == "pending_review" + missing_fx += ( + not realization.fx_source + or not realization.fx_rate + or ( + realization.original_currency != realization.reporting_currency + and realization.fx_source == "same_currency" + ) + ) + missing_evidence += not list(realization.evidence_json or []) + over += realization.realization_type == "actual" and Decimal( + realization.actual_net + ) > Decimal(opportunity.estimated_net) + notes = [] + if business_state: + notes.append("部分结果仅有平台付款业务状态，仍需独立财务确认或外部凭证。") + if pending_dedupe: + notes.append("待确认结果尚未完成 canonical 去重复核，不进入主 KPI。") + return CfoValueDataQualityRead( + pending_confirmation_count=pending, + business_state_only_count=business_state, + pending_dedupe_count=pending_dedupe, + missing_fx_count=missing_fx, + missing_evidence_count=missing_evidence, + actual_over_estimate_count=over, + notes=notes, + ) + + @staticmethod + def _is_verified( + realization: SavingsRealization, + opportunity: SavingsOpportunity, + *, + as_of: datetime, + ) -> bool: + confirmed_at = ( + CfoValueAnalyticsService._utc(realization.confirmed_at) + if realization.confirmed_at is not None + else None + ) + return bool( + opportunity.value_kind == "cash" + and realization.status == "finance_confirmed" + and realization.dedupe_status == "canonical" + and confirmed_at is not None + and confirmed_at <= as_of + ) + + @staticmethod + def _sum_realizations( + rows: list[tuple[SavingsRealization, SavingsOpportunity]], + ) -> MoneyMap: + result: MoneyMap = defaultdict(Decimal) + for realization, _ in rows: + result[realization.reporting_currency] += Decimal(realization.reporting_amount) + return dict(result) + + @staticmethod + def _sum_opportunities(opportunities: list[SavingsOpportunity]) -> MoneyMap: + result: MoneyMap = defaultdict(Decimal) + for opportunity in opportunities: + if opportunity.value_kind == "cash": + result[opportunity.reporting_currency] += Decimal(opportunity.estimated_net) + return dict(result) + + @staticmethod + def _money_read(values: MoneyMap) -> list[CfoValueMoneyRead]: + return [ + CfoValueMoneyRead(currency=currency, amount=amount.quantize(Decimal("0.0001"))) + for currency, amount in sorted(values.items()) + ] + + @staticmethod + def _dimension_value( + opportunity: SavingsOpportunity, + *, + dimension: str, + id_key: str | None, + name_key: str | None, + ) -> tuple[str, str]: + if dimension == "owner": + return opportunity.owner_id, opportunity.owner_name + if dimension == "source": + return opportunity.source_type, opportunity.source_type + dimensions = opportunity.dimension_json or {} + dimension_id = str(dimensions.get(id_key or "") or "").strip() + dimension_name = str(dimensions.get(name_key or "") or dimension_id).strip() + return dimension_id, dimension_name + + @staticmethod + def _freshness( + opportunities: list[SavingsOpportunity], + rows: list[tuple[SavingsRealization, SavingsOpportunity]], + ) -> datetime | None: + timestamps = [ + CfoValueAnalyticsService._utc(item.updated_at) + for item in opportunities + if item.updated_at + ] + timestamps.extend( + CfoValueAnalyticsService._utc(realization.updated_at) + for realization, _ in rows + if realization.updated_at is not None + ) + return max(timestamps) if timestamps else None + + @staticmethod + def _normalize_window( + start: datetime, + end: datetime, + as_of: datetime, + ) -> tuple[datetime, datetime, datetime]: + values = [] + for value in (start, end, as_of): + normalized = ( + value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + ) + values.append(normalized) + normalized_start, normalized_end, normalized_as_of = values + if normalized_start > normalized_end: + raise ValueError("开始时间不能晚于结束时间。") + if (normalized_end - normalized_start).days > 731: + raise ValueError("单次价值看板查询时间范围不能超过 731 天。") + return normalized_start, normalized_end, normalized_as_of + + @staticmethod + def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_access_policy.py b/server/src/app/services/commercial_access_policy.py new file mode 100644 index 0000000..46a7676 --- /dev/null +++ b/server/src/app/services/commercial_access_policy.py @@ -0,0 +1,65 @@ +from __future__ import annotations + +from app.api.deps import CurrentUserContext + +COMMERCIAL_READ_ROLES = {"finance", "executive"} + + +class CommercialPermissionError(PermissionError): + pass + + +class CommercialAccessPolicy: + """商业数据访问边界；租户身份只从服务端登录上下文取得。""" + + @staticmethod + def normalized_tenant_id(value: str | None) -> str: + tenant_id = str(value or "").strip() + if not tenant_id: + raise CommercialPermissionError("商业操作缺少可信租户上下文。") + return tenant_id + + @classmethod + def tenant_for_current_user(cls, current_user: CurrentUserContext) -> str: + return cls.normalized_tenant_id(current_user.tenant_id) + + @staticmethod + def role_codes(current_user: CurrentUserContext) -> set[str]: + return { + str(item or "").strip().casefold() + for item in current_user.role_codes + if str(item or "").strip() + } + + @classmethod + def require_account_read(cls, current_user: CurrentUserContext) -> str: + tenant_id = cls.tenant_for_current_user(current_user) + if current_user.is_admin or cls.role_codes(current_user) & COMMERCIAL_READ_ROLES: + return tenant_id + raise CommercialPermissionError("只有租户财务、管理层或平台管理员可以查看商业账户。") + + @classmethod + def require_platform_admin( + cls, + current_user: CurrentUserContext, + *, + target_tenant_id: str, + ) -> str: + if not current_user.is_admin: + raise CommercialPermissionError("只有平台 admin 管理员可以维护商业配置和内部成本。") + return cls.normalized_tenant_id(target_tenant_id) + + @classmethod + def assert_same_tenant(cls, expected_tenant_id: str, actual_tenant_id: str) -> None: + expected = cls.normalized_tenant_id(expected_tenant_id) + actual = cls.normalized_tenant_id(actual_tenant_id) + if expected != actual: + raise CommercialPermissionError("商业资源不属于目标租户。") + + +class CommercialConflictError(RuntimeError): + pass + + +class CommercialConfigurationError(ValueError): + pass diff --git a/server/src/app/services/commercial_admin.py b/server/src/app/services/commercial_admin.py new file mode 100644 index 0000000..3b68048 --- /dev/null +++ b/server/src/app/services/commercial_admin.py @@ -0,0 +1,560 @@ +from __future__ import annotations + +from datetime import UTC, datetime + +from sqlalchemy import func, select +from sqlalchemy.orm import Session + +from app.models.commercial import ( + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) +from app.schemas.commercial import ( + CommercialEntitlementUpsert, + CommercialPlanActivationRead, + CommercialPlanCreate, + CommercialSubscriptionCreate, + CommercialSubscriptionTransition, +) +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) +from app.services.commercial_admin_audit import ( + CommercialAdminAuditService, + service_request_id, + snapshot_resource, +) +from app.services.commercial_billing_periods import CommercialBillingPeriodService + +CURRENT_SUBSCRIPTION_STATUSES = {"trialing", "active", "past_due", "suspended"} + + +class CommercialAdminService: + """平台侧维护有版本的套餐、订阅快照和权益配置。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def create_plan( + self, + tenant_id: str, + payload: CommercialPlanCreate, + *, + actor_id: str, + request_id: str | None = None, + reason: str | None = None, + ) -> TenantCommercialPlan: + tenant_id = _required(tenant_id, "租户编号") + plan_code = _required(payload.plan_code, "套餐编码") + actor_id = _required(actor_id, "配置操作人") + version = ( + int( + self.db.scalar( + select(func.max(TenantCommercialPlan.version)).where( + TenantCommercialPlan.tenant_id == tenant_id, + TenantCommercialPlan.plan_code == plan_code, + ) + ) + or 0 + ) + + 1 + ) + row = TenantCommercialPlan( + tenant_id=tenant_id, + plan_code=plan_code, + name=_required(payload.name, "套餐名称"), + pricing_model=payload.pricing_model, + billing_interval=payload.billing_interval, + currency=payload.currency.upper(), + base_fee=payload.base_fee, + included_seats=payload.included_seats, + overage_enabled=payload.overage_enabled, + status="draft", + effective_from=_utc(payload.effective_from), + effective_to=_optional_utc(payload.effective_to), + version=version, + contract_terms_json=dict(payload.contract_terms_json), + created_by=actor_id, + ) + self.db.add(row) + self.db.flush() + request = request_id or service_request_id("plan-create") + CommercialAdminAuditService(self.db).record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor_id, + request_id=request, + reason=reason or payload.reason, + action="plan_created", + resource=row, + before={}, + after=snapshot_resource(row), + ) + return row + + def activate_plan( + self, + tenant_id: str, + plan_id: str, + *, + expected_version: int, + actor_id: str = "platform-admin", + request_id: str | None = None, + reason: str = "平台管理员激活套餐版本", + ) -> CommercialPlanActivationRead: + plan = self._plan(tenant_id, plan_id, for_update=True) + if plan is None: + raise LookupError("商业套餐不存在。") + if plan.version != expected_version: + raise CommercialConflictError(f"商业套餐版本冲突，当前版本为 {plan.version}。") + if plan.status == "retired": + raise CommercialConflictError("已退役套餐不能重新激活，请创建新版本。") + + request = request_id or service_request_id("plan-activate") + actor = _required(actor_id, "配置操作人") + audit = CommercialAdminAuditService(self.db) + before_plan = snapshot_resource(plan) + retired_ids: list[str] = [] + current_rows = list( + self.db.scalars( + select(TenantCommercialPlan) + .where( + TenantCommercialPlan.tenant_id == tenant_id, + TenantCommercialPlan.plan_code == plan.plan_code, + TenantCommercialPlan.status == "active", + TenantCommercialPlan.id != plan.id, + ) + .with_for_update() + ).all() + ) + for current in current_rows: + before_current = snapshot_resource(current) + if _utc(plan.effective_from) <= _utc(current.effective_from): + raise CommercialConfigurationError("新套餐版本的生效时间必须晚于当前激活版本。") + current.status = "retired" + current.effective_to = plan.effective_from + retired_ids.append(current.id) + audit.record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor, + request_id=request, + reason=reason, + action="plan_retired", + resource=current, + before=before_current, + after=snapshot_resource(current), + ) + plan.status = "active" + self.db.flush() + audit.record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor, + request_id=request, + reason=reason, + action="plan_activated", + resource=plan, + before=before_plan, + after=snapshot_resource(plan), + ) + return CommercialPlanActivationRead( + plan=plan, + retired_plan_ids=retired_ids, + ) + + def create_subscription( + self, + tenant_id: str, + payload: CommercialSubscriptionCreate, + *, + actor_id: str, + request_id: str | None = None, + reason: str | None = None, + ) -> TenantSubscription: + tenant_id = _required(tenant_id, "租户编号") + actor_id = _required(actor_id, "配置操作人") + plan = self._plan(tenant_id, payload.plan_id, for_update=True) + if plan is None: + raise LookupError("订阅引用的商业套餐不存在。") + if plan.status != "active": + raise CommercialConfigurationError("只能基于已激活的套餐创建订阅。") + starts_at = _utc(payload.starts_at) + ends_at = _optional_utc(payload.ends_at) + period_start = _utc(payload.current_period_start) + period_end = _utc(payload.current_period_end) + if starts_at < _utc(plan.effective_from) or ( + plan.effective_to is not None and starts_at >= _utc(plan.effective_to) + ): + raise CommercialConfigurationError("订阅开始时间不在套餐有效期内。") + if period_start < starts_at or (ends_at is not None and period_end > ends_at): + raise CommercialConfigurationError("当前计费周期必须位于订阅合同有效期内。") + current = self._current_subscription(tenant_id, for_update=True) + if current is not None: + raise CommercialConflictError( + f"租户已有当前订阅 {current.subscription_key}，请先结束原订阅。" + ) + row = TenantSubscription( + tenant_id=tenant_id, + subscription_key=_required(payload.subscription_key, "订阅键"), + plan_id=plan.id, + status=payload.status, + starts_at=starts_at, + ends_at=ends_at, + current_period_start=period_start, + current_period_end=period_end, + seats=payload.seats, + base_fee_snapshot=plan.base_fee, + currency=plan.currency, + billing_interval=plan.billing_interval, + auto_renew=payload.auto_renew, + external_provider=_blank_to_none(payload.external_provider), + external_subscription_id=_blank_to_none(payload.external_subscription_id), + version=1, + metadata_json=dict(payload.metadata_json), + created_by=actor_id, + ) + self.db.add(row) + self.db.flush() + request = request_id or service_request_id("subscription-create") + audit_reason = reason or payload.reason + audit = CommercialAdminAuditService(self.db) + audit.record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor_id, + request_id=request, + reason=audit_reason, + action="subscription_created", + resource=row, + before={}, + after=snapshot_resource(row), + ) + period, _ = CommercialBillingPeriodService(self.db).create_initial( + row, + plan, + actor_id=actor_id, + request_id=request, + ) + audit.record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor_id, + request_id=request, + reason=audit_reason, + action="billing_period_created", + resource=period, + before={}, + after=snapshot_resource(period), + ) + return row + + def activate_subscription( + self, + tenant_id: str, + subscription_id: str, + *, + expected_version: int, + actor_id: str = "platform-admin", + request_id: str | None = None, + reason: str = "平台管理员重新激活订阅", + ) -> TenantSubscription: + row = self._subscription(tenant_id, subscription_id, for_update=True) + if row is None: + raise LookupError("商业订阅不存在。") + if row.version != expected_version: + raise CommercialConflictError(f"订阅版本冲突，当前版本为 {row.version}。") + if row.status in {"canceled", "expired"}: + raise CommercialConflictError("已取消或过期订阅不能重新激活。") + current = self._current_subscription(tenant_id, for_update=True) + if current is not None and current.id != row.id: + raise CommercialConflictError("租户已有另一个当前订阅，不能激活该订阅。") + before = snapshot_resource(row) + row.status = "active" + row.version += 1 + self.db.flush() + CommercialAdminAuditService(self.db).record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor_id, + request_id=request_id or service_request_id("subscription-activate"), + reason=reason, + action="subscription_activated", + resource=row, + before=before, + after=snapshot_resource(row), + ) + return row + + def transition_subscription( + self, + tenant_id: str, + subscription_id: str, + payload: CommercialSubscriptionTransition, + *, + actor_id: str, + request_id: str | None = None, + ) -> TenantSubscription: + """暂停或终止订阅；终态只能通过新订阅恢复商业服务。""" + + row = self._subscription(tenant_id, subscription_id, for_update=True) + if row is None: + raise LookupError("商业订阅不存在。") + if row.version != payload.expected_version: + raise CommercialConflictError(f"订阅版本冲突，当前版本为 {row.version}。") + target = payload.target_status + if row.status == target: + return row + allowed = { + "trialing": {"past_due", "suspended", "canceled", "expired"}, + "active": {"past_due", "suspended", "canceled", "expired"}, + "past_due": {"suspended", "canceled", "expired"}, + "suspended": {"canceled", "expired"}, + "canceled": set(), + "expired": set(), + } + if target not in allowed.get(str(row.status), set()): + raise CommercialConflictError(f"订阅不能从 {row.status} 转换为 {target}。") + + before = snapshot_resource(row) + changed_at = datetime.now(UTC) + metadata = dict(row.metadata_json or {}) + history = list( + metadata.get("status_history") + if isinstance(metadata.get("status_history"), list) + else [] + ) + history.append( + { + "from": row.status, + "to": target, + "reason": _required(payload.reason, "状态变更原因"), + "actor_id": _required(actor_id, "配置操作人"), + "changed_at": changed_at.isoformat(), + } + ) + metadata["status_history"] = history[-50:] + row.metadata_json = metadata + row.status = target + if target == "canceled": + row.canceled_at = changed_at + if target in {"canceled", "expired"} and ( + row.ends_at is None or _utc(row.ends_at) > changed_at + ): + row.ends_at = changed_at + row.version += 1 + self.db.flush() + CommercialAdminAuditService(self.db).record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor_id, + request_id=request_id or service_request_id("subscription-transition"), + reason=payload.reason, + action="subscription_transitioned", + resource=row, + before=before, + after=snapshot_resource(row), + ) + return row + + def upsert_entitlement( + self, + tenant_id: str, + payload: CommercialEntitlementUpsert, + *, + actor_id: str = "platform-admin", + request_id: str | None = None, + reason: str | None = None, + ) -> CommercialEntitlement: + tenant_id = _required(tenant_id, "租户编号") + entitlement_key = _required(payload.entitlement_key, "权益键") + subscription = self._subscription( + tenant_id, + payload.subscription_id, + for_update=True, + ) + if subscription is None: + raise LookupError("权益引用的商业订阅不存在。") + effective_from = _utc(payload.effective_from) + effective_to = _optional_utc(payload.effective_to) + if effective_from < _utc(subscription.starts_at) or ( + subscription.ends_at is not None + and (effective_to is None or effective_to > _utc(subscription.ends_at)) + ): + raise CommercialConfigurationError("权益有效期必须位于订阅合同有效期内。") + row = self.db.scalar( + select(CommercialEntitlement) + .where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.subscription_id == subscription.id, + CommercialEntitlement.entitlement_key == entitlement_key, + ) + .with_for_update() + ) + values = { + "metric_key": _required(payload.metric_key, "计量键"), + "entitlement_type": payload.entitlement_type, + "unit": _required(payload.unit, "计量单位"), + "included_quantity": payload.included_quantity, + "hard_limit_quantity": payload.hard_limit_quantity, + "reset_interval": payload.reset_interval, + "overage_policy": payload.overage_policy, + "status": payload.status, + "effective_from": effective_from, + "effective_to": effective_to, + "config_json": dict(payload.config_json), + } + before = snapshot_resource(row) if row is not None else {} + created = row is None + if row is None: + row = CommercialEntitlement( + tenant_id=tenant_id, + subscription_id=subscription.id, + entitlement_key=entitlement_key, + version=1, + **values, + ) + self.db.add(row) + else: + if all(getattr(row, key) == value for key, value in values.items()): + return row + has_usage = self.db.scalar( + select(UsageMeterEvent.id) + .where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.entitlement_id == row.id, + ) + .limit(1) + ) + immutable_after_usage = {key: value for key, value in values.items() if key != "status"} + if has_usage and any( + getattr(row, key) != value for key, value in immutable_after_usage.items() + ): + raise CommercialConflictError( + "已有用量事实的权益不能回改配额、计价或有效期，" + "请创建新订阅版本；仅允许暂停或恢复当前权益。" + ) + for key, value in values.items(): + setattr(row, key, value) + row.version += 1 + self.db.flush() + CommercialAdminAuditService(self.db).record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor_id, + request_id=request_id or service_request_id("entitlement-upsert"), + reason=reason or payload.reason, + action="entitlement_created" if created else "entitlement_updated", + resource=row, + before=before, + after=snapshot_resource(row), + ) + return row + + def activate_entitlement( + self, + tenant_id: str, + entitlement_id: str, + *, + expected_version: int, + actor_id: str = "platform-admin", + request_id: str | None = None, + reason: str = "平台管理员激活商业权益", + ) -> CommercialEntitlement: + row = self.db.scalar( + select(CommercialEntitlement) + .where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.id == entitlement_id, + ) + .with_for_update() + ) + if row is None: + raise LookupError("商业权益不存在。") + if row.version != expected_version: + raise CommercialConflictError(f"权益版本冲突，当前版本为 {row.version}。") + if row.status == "expired": + raise CommercialConflictError("已过期权益不能重新激活，请创建新订阅版本。") + before = snapshot_resource(row) + row.status = "active" + row.version += 1 + self.db.flush() + CommercialAdminAuditService(self.db).record( + tenant_id=tenant_id, + actor_type="user", + actor_id=actor_id, + request_id=request_id or service_request_id("entitlement-activate"), + reason=reason, + action="entitlement_activated", + resource=row, + before=before, + after=snapshot_resource(row), + ) + return row + + def _plan( + self, + tenant_id: str, + plan_id: str, + *, + for_update: bool, + ) -> TenantCommercialPlan | None: + statement = select(TenantCommercialPlan).where( + TenantCommercialPlan.tenant_id == tenant_id, + TenantCommercialPlan.id == plan_id, + ) + if for_update: + statement = statement.with_for_update() + return self.db.scalar(statement) + + def _subscription( + self, + tenant_id: str, + subscription_id: str, + *, + for_update: bool, + ) -> TenantSubscription | None: + statement = select(TenantSubscription).where( + TenantSubscription.tenant_id == tenant_id, + TenantSubscription.id == subscription_id, + ) + if for_update: + statement = statement.with_for_update() + return self.db.scalar(statement) + + def _current_subscription( + self, + tenant_id: str, + *, + for_update: bool, + ) -> TenantSubscription | None: + statement = select(TenantSubscription).where( + TenantSubscription.tenant_id == tenant_id, + TenantSubscription.status.in_(CURRENT_SUBSCRIPTION_STATUSES), + ) + if for_update: + statement = statement.with_for_update() + return self.db.scalar(statement) + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + + +def _optional_utc(value: datetime | None) -> datetime | None: + return _utc(value) if value is not None else None + + +def _blank_to_none(value: str | None) -> str | None: + normalized = str(value or "").strip() + return normalized or None + + +def _required(value: str | None, field_name: str) -> str: + normalized = str(value or "").strip() + if not normalized: + raise CommercialConfigurationError(f"{field_name}不能为空。") + return normalized diff --git a/server/src/app/services/commercial_admin_audit.py b/server/src/app/services/commercial_admin_audit.py new file mode 100644 index 0000000..a61e221 --- /dev/null +++ b/server/src/app/services/commercial_admin_audit.py @@ -0,0 +1,249 @@ +"""商业配置与自动续期的脱敏追加式审计。""" + +from __future__ import annotations + +import uuid +from datetime import date, datetime +from decimal import Decimal +from typing import Any, Literal + +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.models.commercial import ( + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, +) +from app.models.commercial_billing import CommercialAdminEvent, CommercialBillingPeriod +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) + +AuditResource = ( + TenantCommercialPlan | TenantSubscription | CommercialEntitlement | CommercialBillingPeriod +) +ActorType = Literal["user", "system", "migration"] + +_RESOURCE_FIELDS: dict[type[Any], tuple[str, ...]] = { + TenantCommercialPlan: ( + "id", + "tenant_id", + "plan_code", + "name", + "pricing_model", + "billing_interval", + "currency", + "base_fee", + "included_seats", + "overage_enabled", + "status", + "effective_from", + "effective_to", + "version", + ), + TenantSubscription: ( + "id", + "tenant_id", + "subscription_key", + "plan_id", + "status", + "starts_at", + "ends_at", + "current_period_start", + "current_period_end", + "seats", + "base_fee_snapshot", + "currency", + "billing_interval", + "auto_renew", + "canceled_at", + "version", + ), + CommercialEntitlement: ( + "id", + "tenant_id", + "subscription_id", + "entitlement_key", + "metric_key", + "entitlement_type", + "unit", + "included_quantity", + "hard_limit_quantity", + "reset_interval", + "overage_policy", + "status", + "effective_from", + "effective_to", + "version", + ), + CommercialBillingPeriod: ( + "id", + "tenant_id", + "subscription_id", + "plan_id", + "period_sequence", + "period_key", + "status", + "period_start", + "period_end", + "subscription_status_snapshot", + "plan_code_snapshot", + "plan_version_snapshot", + "pricing_model_snapshot", + "billing_interval", + "currency", + "base_fee_snapshot", + "seats_snapshot", + "source", + ), +} +_RESOURCE_TYPES = { + TenantCommercialPlan: "plan", + TenantSubscription: "subscription", + CommercialEntitlement: "entitlement", + CommercialBillingPeriod: "billing_period", +} +_BANNED_KEY_PARTS = ( + "secret", + "token", + "password", + "signature", + "authorization", + "contract_terms", + "metadata_json", + "config_json", + "external_subscription", +) + + +class CommercialAdminAuditService: + def __init__(self, db: Session) -> None: + self.db = db + + def record( + self, + *, + tenant_id: str, + actor_type: ActorType, + actor_id: str, + request_id: str, + reason: str, + action: str, + resource: AuditResource, + before: dict[str, Any] | None, + after: dict[str, Any] | None, + ) -> tuple[CommercialAdminEvent, bool]: + resource_type = resource_type_for(resource) + resource_id = _required(resource.id, "审计资源编号") + version = resource_version_for(resource) + normalized_request = _required(request_id, "X-Request-Id")[:120] + normalized_before = _validate_snapshot(before or {}) + normalized_after = _validate_snapshot(after or {}) + existing = self.db.scalar( + select(CommercialAdminEvent).where( + CommercialAdminEvent.tenant_id == tenant_id, + CommercialAdminEvent.request_id == normalized_request, + CommercialAdminEvent.action == action, + CommercialAdminEvent.resource_type == resource_type, + CommercialAdminEvent.resource_id == resource_id, + ) + ) + if existing is not None: + self._assert_same(existing, normalized_before, normalized_after) + return existing, False + row = CommercialAdminEvent( + tenant_id=_required(tenant_id, "审计租户编号")[:64], + actor_type=actor_type, + actor_id=_required(actor_id, "审计操作人")[:120], + request_id=normalized_request, + reason=_required(reason, "审计原因")[:500], + action=action, + resource_type=resource_type, + resource_id=resource_id, + resource_version=version, + before_json=normalized_before, + after_json=normalized_after, + ) + try: + with self.db.begin_nested(): + self.db.add(row) + self.db.flush() + return row, True + except IntegrityError: + existing = self.db.scalar( + select(CommercialAdminEvent).where( + CommercialAdminEvent.tenant_id == tenant_id, + CommercialAdminEvent.request_id == normalized_request, + CommercialAdminEvent.action == action, + CommercialAdminEvent.resource_type == resource_type, + CommercialAdminEvent.resource_id == resource_id, + ) + ) + if existing is None: + raise + self._assert_same(existing, normalized_before, normalized_after) + return existing, False + + @staticmethod + def _assert_same( + row: CommercialAdminEvent, + before: dict[str, Any], + after: dict[str, Any], + ) -> None: + if row.before_json != before or row.after_json != after: + raise CommercialConflictError("X-Request-Id 已被不同商业审计动作占用。") + + +def snapshot_resource(resource: AuditResource) -> dict[str, Any]: + fields = _RESOURCE_FIELDS.get(type(resource)) + if fields is None: + raise CommercialConfigurationError("不支持的商业审计资源类型。") + return {name: _json_value(getattr(resource, name)) for name in fields} + + +def resource_type_for(resource: AuditResource) -> str: + resource_type = _RESOURCE_TYPES.get(type(resource)) + if resource_type is None: + raise CommercialConfigurationError("不支持的商业审计资源类型。") + return resource_type + + +def resource_version_for(resource: AuditResource) -> int: + if isinstance(resource, CommercialBillingPeriod): + return int(resource.period_sequence) + return int(resource.version) + + +def service_request_id(prefix: str) -> str: + normalized = "-".join(str(prefix or "commercial").strip().split())[:48] + return f"service:{normalized}:{uuid.uuid4().hex}" + + +def _validate_snapshot(value: dict[str, Any]) -> dict[str, Any]: + for key, item in value.items(): + normalized = str(key).casefold() + if any(part in normalized for part in _BANNED_KEY_PARTS): + raise CommercialConfigurationError("商业审计快照包含禁止记录的敏感字段。") + if isinstance(item, dict): + _validate_snapshot(item) + return dict(value) + + +def _json_value(value: Any) -> Any: + if isinstance(value, Decimal): + return str(value) + if isinstance(value, (datetime, date)): + return value.isoformat() + if isinstance(value, (str, int, float, bool)) or value is None: + return value + raise CommercialConfigurationError("商业审计快照包含不可序列化字段。") + + +def _required(value: str | None, field_name: str) -> str: + normalized = str(value or "").strip() + if not normalized: + raise CommercialConfigurationError(f"{field_name}不能为空。") + return normalized diff --git a/server/src/app/services/commercial_analytics.py b/server/src/app/services/commercial_analytics.py new file mode 100644 index 0000000..bb196fe --- /dev/null +++ b/server/src/app/services/commercial_analytics.py @@ -0,0 +1,576 @@ +from __future__ import annotations + +from collections import defaultdict +from datetime import UTC, datetime +from decimal import ROUND_HALF_UP, Decimal, InvalidOperation +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.commercial import ( + CommercialCostEvent, + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) +from app.models.commercial_billing import CommercialBillingPeriod +from app.models.savings import SavingsOpportunity, SavingsRealization +from app.schemas.commercial import ( + CommercialAnalyticsRead, + CommercialMetricRead, + CommercialMoneyRead, + CommercialRatioRead, +) + +MoneyMap = dict[str, Decimal] +FOUR_PLACES = Decimal("0.0001") +SIX_PLACES = Decimal("0.000001") + + +class CommercialAnalyticsService: + """平台商业账与客户价值账分离聚合，不做隐式汇率换算。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def build( + self, + tenant_id: str, + *, + start: datetime, + end: datetime, + as_of: datetime, + ) -> CommercialAnalyticsRead: + start, end, as_of = self._normalize_window(start, end, as_of) + effective_end = min(end, as_of) + subscriptions = list( + self.db.scalars( + select(TenantSubscription).where( + TenantSubscription.tenant_id == tenant_id, + ) + ).all() + ) + billing_periods = list( + self.db.scalars( + select(CommercialBillingPeriod).where( + CommercialBillingPeriod.tenant_id == tenant_id, + CommercialBillingPeriod.period_start < effective_end, + CommercialBillingPeriod.period_end > start, + ) + ).all() + ) + billing_period_by_id = {item.id: item for item in billing_periods} + plans = list( + self.db.scalars( + select(TenantCommercialPlan).where( + TenantCommercialPlan.tenant_id == tenant_id, + ) + ).all() + ) + entitlements = list( + self.db.scalars( + select(CommercialEntitlement).where( + CommercialEntitlement.tenant_id == tenant_id, + ) + ).all() + ) + entitlement_by_id = {item.id: item for item in entitlements} + + charge_values, charge_basis_count, charge_issues = self._customer_charges( + tenant_id, + start=start, + end=effective_end, + billing_periods=billing_periods, + billing_period_by_id=billing_period_by_id, + entitlement_by_id=entitlement_by_id, + ) + cost_values = self._internal_costs( + tenant_id, + start=start, + end=effective_end, + ) + savings_values = self._verified_savings( + tenant_id, + start=start, + end=effective_end, + as_of=as_of, + ) + labor_values, labor_issues = self._labor_value(plans, start=start, end=effective_end) + + customer_charges = self._charge_metric( + charge_values, + charge_basis_count=charge_basis_count, + issues=charge_issues, + ) + internal_costs = self._cost_metric(cost_values) + verified_cash = self._savings_metric(savings_values) + contribution = self._contribution_metric( + charge_values, + cost_values, + charge_available=charge_basis_count > 0, + charge_issues=charge_issues, + ) + customer_roi = self._roi_metric( + savings_values, + charge_values, + charge_available=charge_basis_count > 0, + labor_available=bool(labor_values), + ) + labor = self._labor_metric(labor_values, labor_issues) + + issues = list(dict.fromkeys(charge_issues + labor_issues)) + if not cost_values: + issues.append("查询窗口内缺少平台内部成本事件。") + if not savings_values: + issues.append("查询窗口内缺少 canonical 且财务确认的现金节省事实。") + if not subscriptions: + issues.append("租户尚未配置商业订阅。") + has_any = bool(charge_basis_count or cost_values or savings_values or labor_values) + data_quality = "partial" if has_any else "unavailable" + return CommercialAnalyticsRead( + tenant_id=tenant_id, + start=start, + end=end, + as_of=as_of, + generated_at=datetime.now(UTC), + customer_charges=customer_charges, + internal_costs=internal_costs, + contribution_margin=contribution, + verified_cash_savings=verified_cash, + customer_roi=customer_roi, + customer_labor_value=labor, + data_quality_status=data_quality, + data_quality_issues=list(dict.fromkeys(issues)), + ) + + def _customer_charges( + self, + tenant_id: str, + *, + start: datetime, + end: datetime, + billing_periods: list[CommercialBillingPeriod], + billing_period_by_id: dict[str, CommercialBillingPeriod], + entitlement_by_id: dict[str, CommercialEntitlement], + ) -> tuple[MoneyMap, int, list[str]]: + values: dict[str, Decimal] = defaultdict(Decimal) + basis_count = 0 + issues = ["缺少发票、贷项通知单和收款事实；客户收费仅为合同计费代理值。"] + for period in billing_periods: + period_start = _utc(period.period_start) + if start <= period_start < end: + values[period.currency] += Decimal(period.base_fee_snapshot) + basis_count += 1 + + usage_events = list( + self.db.scalars( + select(UsageMeterEvent).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.occurred_at < end, + ) + ).all() + ) + grouped: dict[tuple[str, str, str], list[UsageMeterEvent]] = defaultdict(list) + quota_groups: dict[tuple[str, str], list[UsageMeterEvent]] = defaultdict(list) + for event in usage_events: + grouped[(event.entitlement_id, event.billing_period_id, event.quota_period_key)].append( + event + ) + quota_groups[(event.entitlement_id, event.quota_period_key)].append(event) + for (entitlement_id, billing_period_id, quota_key), events in grouped.items(): + entitlement = entitlement_by_id.get(entitlement_id) + if entitlement is None: + issues.append("存在无法关联权益配置的用量事件，未计入合同收费代理值。") + continue + period = billing_period_by_id.get(billing_period_id) + if period is None: + issues.append("存在无法关联不可变账期的用量事件，未计入合同收费代理值。") + continue + if period.pricing_model_snapshot not in {"usage", "hybrid", "custom"}: + continue + segment_start = max(start, _utc(period.period_start)) + segment_end = min(end, _utc(period.period_end)) + window_events = [ + event for event in events if segment_start <= _utc(event.occurred_at) < segment_end + ] + if not window_events: + continue + config = entitlement.config_json or {} + unit_price = _decimal_or_none(config.get("billable_unit_price")) + billing_currency = str(config.get("billing_currency") or "").strip().upper() + billing_mode = str(config.get("billing_mode") or "").strip().casefold() + if unit_price is None or unit_price < 0 or billing_currency != period.currency: + issues.append( + f"权益 {entitlement.entitlement_key} 缺少可信同币种计费单价，" + "对应使用量未计入客户收费。" + ) + continue + if billing_mode not in {"all_usage", "overage"}: + issues.append( + f"权益 {entitlement.entitlement_key} 未配置 billing_mode，" + "对应使用量未计入客户收费。" + ) + continue + if billing_mode == "all_usage": + billable_quantity = sum( + (Decimal(event.quantity) for event in window_events), + Decimal("0"), + ) + else: + included = Decimal(entitlement.included_quantity or 0) + quota_events = quota_groups[(entitlement_id, quota_key)] + before = sum( + ( + Decimal(event.quantity) + for event in quota_events + if _utc(event.occurred_at) < segment_start + ), + Decimal("0"), + ) + through_end = sum( + ( + Decimal(event.quantity) + for event in quota_events + if _utc(event.occurred_at) < segment_end + ), + Decimal("0"), + ) + billable_quantity = max(Decimal("0"), through_end - included) - max( + Decimal("0"), before - included + ) + values[billing_currency] += (billable_quantity * unit_price).quantize( + FOUR_PLACES, + rounding=ROUND_HALF_UP, + ) + basis_count += 1 + return dict(values), basis_count, list(dict.fromkeys(issues)) + + def _internal_costs(self, tenant_id: str, *, start: datetime, end: datetime) -> MoneyMap: + values: dict[str, Decimal] = defaultdict(Decimal) + rows = self.db.scalars( + select(CommercialCostEvent).where( + CommercialCostEvent.tenant_id == tenant_id, + CommercialCostEvent.occurred_at >= start, + CommercialCostEvent.occurred_at < end, + ) + ).all() + for row in rows: + values[row.reporting_currency] += Decimal(row.reporting_amount) + return dict(values) + + def _verified_savings( + self, + tenant_id: str, + *, + start: datetime, + end: datetime, + as_of: datetime, + ) -> MoneyMap: + values: dict[str, Decimal] = defaultdict(Decimal) + rows = self.db.execute( + select(SavingsRealization, SavingsOpportunity) + .join( + SavingsOpportunity, + (SavingsOpportunity.tenant_id == SavingsRealization.tenant_id) + & (SavingsOpportunity.id == SavingsRealization.opportunity_id), + ) + .where( + SavingsRealization.tenant_id == tenant_id, + SavingsRealization.realized_at >= start, + SavingsRealization.realized_at < end, + SavingsRealization.status == "finance_confirmed", + SavingsRealization.dedupe_status == "canonical", + SavingsRealization.confirmed_at.is_not(None), + SavingsRealization.confirmed_at <= as_of, + SavingsOpportunity.value_kind == "cash", + ) + ).all() + for realization, _ in rows: + values[realization.reporting_currency] += Decimal(realization.reporting_amount) + return dict(values) + + @staticmethod + def _labor_value( + plans: list[TenantCommercialPlan], + *, + start: datetime, + end: datetime, + ) -> tuple[MoneyMap, list[str]]: + values: dict[str, Decimal] = defaultdict(Decimal) + issues: list[str] = [] + relevant = [ + plan + for plan in plans + if plan.status == "active" + and _utc(plan.effective_from) < end + and (plan.effective_to is None or _utc(plan.effective_to) >= start) + ] + for plan in relevant: + baseline = (plan.contract_terms_json or {}).get("customer_labor_baseline") + if not isinstance(baseline, dict): + continue + currency = str(baseline.get("currency") or "").strip().upper() + released_hours = _decimal_or_none(baseline.get("released_hours")) + hourly_cost = _decimal_or_none(baseline.get("fully_loaded_hourly_cost")) + approved_by = str(baseline.get("approved_by") or "").strip() + verified_at = str(baseline.get("verified_at") or "").strip() + if ( + len(currency) != 3 + or released_hours is None + or released_hours < 0 + or hourly_cost is None + or hourly_cost < 0 + or not approved_by + or not verified_at + ): + issues.append("客户工时基线字段不完整，未计入工时价值。") + continue + values[currency] += (released_hours * hourly_cost).quantize( + FOUR_PLACES, + rounding=ROUND_HALF_UP, + ) + if not values: + issues.append("缺少客户批准的工时基线、完全成本和可释放工时。") + return dict(values), issues + + @staticmethod + def _charge_metric( + values: MoneyMap, + *, + charge_basis_count: int, + issues: list[str], + ) -> CommercialMetricRead: + if not charge_basis_count: + return _metric( + "customer_charges", + "客户合同收费", + "unavailable", + {}, + "查询窗口内没有可归属的合同周期收费或可信用量计价配置。", + required=["账单/发票事实", "用量计价规则", "贷项通知单"], + notes=issues, + basis="contract_charge_proxy", + ) + return _metric( + "customer_charges", + "客户合同收费", + "partial", + values, + "金额来自订阅快照和管理员维护的用量计价规则，不代表已开票或已收款收入。", + required=["账单/发票事实", "收款回执"], + notes=issues, + basis="contract_charge_proxy", + ) + + @staticmethod + def _cost_metric(values: MoneyMap) -> CommercialMetricRead: + if not values: + return _metric( + "internal_costs", + "平台内部成本", + "unavailable", + {}, + "查询窗口内没有内部成本事件，不能用 0 代替未知成本。", + required=["AI/OCR/基础设施/支持等成本事件"], + basis="internal_cost_ledger", + ) + return _metric( + "internal_costs", + "平台内部成本", + "available", + values, + "金额来自独立的内部成本事件台账。", + basis="internal_cost_ledger", + ) + + @staticmethod + def _savings_metric(values: MoneyMap) -> CommercialMetricRead: + if not values: + return _metric( + "verified_cash_savings", + "客户财务确认现金节省", + "unavailable", + {}, + "查询窗口内没有 canonical 且财务确认的现金节省事实。", + required=["Savings Ledger 财务确认实际结果"], + basis="verified_savings_ledger", + ) + return _metric( + "verified_cash_savings", + "客户财务确认现金节省", + "available", + values, + "仅包含 Savings Ledger 的 canonical、finance_confirmed 现金结果。", + basis="verified_savings_ledger", + ) + + @staticmethod + def _contribution_metric( + charges: MoneyMap, + costs: MoneyMap, + *, + charge_available: bool, + charge_issues: list[str], + ) -> CommercialMetricRead: + common = sorted(set(charges) & set(costs)) + if not charge_available or not costs or not common: + return _metric( + "contribution_margin", + "贡献毛利", + "unavailable", + {}, + "收费与成本缺失或币种不匹配，不能跨币种相减。", + required=["同币种收入事实", "同币种内部成本事实"], + notes=charge_issues, + basis="contract_contribution_proxy", + ) + values = {currency: charges[currency] - costs[currency] for currency in common} + unmatched = sorted(set(charges) ^ set(costs)) + notes = list(charge_issues) + if unmatched: + notes.append("部分币种只有收费或成本，未计算对应贡献毛利。") + return _metric( + "contribution_margin", + "贡献毛利", + "partial", + values, + "以合同收费代理值减内部成本；缺少收入确认事实，不能视为会计毛利。", + required=["收入确认事实"], + notes=notes, + basis="contract_contribution_proxy", + ) + + @staticmethod + def _roi_metric( + savings: MoneyMap, + charges: MoneyMap, + *, + charge_available: bool, + labor_available: bool, + ) -> CommercialMetricRead: + common = [ + currency for currency in sorted(set(savings) & set(charges)) if charges[currency] > 0 + ] + if not savings or not charge_available or not common: + return CommercialMetricRead( + key="customer_roi", + label="客户现金 ROI", + status="unavailable", + reason="缺少同币种正向客户收费或财务确认现金节省，不能计算 ROI。", + required_inputs=["同币种客户收费", "财务确认现金节省"], + notes=["不同币种不会被合并或换算。"], + ) + ratios = [ + CommercialRatioRead( + currency=currency, + ratio=((savings[currency] - charges[currency]) / charges[currency]).quantize( + SIX_PLACES, + rounding=ROUND_HALF_UP, + ), + numerator=(savings[currency] - charges[currency]).quantize(FOUR_PLACES), + denominator=charges[currency].quantize(FOUR_PLACES), + ) + for currency in common + ] + notes = ["ROI 按币种分别计算，未做隐式汇率换算。"] + if not labor_available: + notes.append("客户工时价值缺失，因此现金 ROI 未叠加任何推测工时收益。") + return CommercialMetricRead( + key="customer_roi", + label="客户现金 ROI", + status="partial", + ratios=ratios, + reason=( + "按（财务确认现金节省－客户合同收费代理值）/ 客户合同收费代理值计算；" + "缺少已开票/收款事实。" + ), + required_inputs=["账单/收款事实"], + notes=notes, + ) + + @staticmethod + def _labor_metric(values: MoneyMap, issues: list[str]) -> CommercialMetricRead: + if not values: + return _metric( + "customer_labor_value", + "客户认可工时价值", + "unavailable", + {}, + "缺少客户批准的工时基线，不能将假设工时折算为价值。", + required=["基线工时", "完全成本", "可释放工时", "客户批准人"], + notes=issues, + basis="customer_approved_labor_baseline", + ) + return _metric( + "customer_labor_value", + "客户认可工时价值", + "partial", + values, + "来自版本化合同条款中的客户批准工时基线，尚非逐期工时事实台账。", + required=["逐期活跃工时事实"], + notes=issues, + basis="customer_approved_labor_baseline", + ) + + @staticmethod + def _normalize_window( + start: datetime, + end: datetime, + as_of: datetime, + ) -> tuple[datetime, datetime, datetime]: + for item in (start, end, as_of): + if item.tzinfo is None or item.utcoffset() is None: + raise ValueError("商业分析时间必须显式包含时区。") + normalized = tuple(_utc(item) for item in (start, end, as_of)) + normalized_start, normalized_end, normalized_as_of = normalized + if normalized_start >= normalized_end: + raise ValueError("商业分析开始时间必须早于结束时间。") + if normalized_as_of < normalized_start: + raise ValueError("商业分析截止时间不能早于开始时间。") + if (normalized_end - normalized_start).days > 731: + raise ValueError("单次商业分析时间范围不能超过 731 天。") + return normalized_start, normalized_end, normalized_as_of + + +def _metric( + key: str, + label: str, + status: str, + values: MoneyMap, + reason: str, + *, + required: list[str] | None = None, + notes: list[str] | None = None, + basis: str, +) -> CommercialMetricRead: + return CommercialMetricRead( + key=key, + label=label, + status=status, # type: ignore[arg-type] + values=[ + CommercialMoneyRead( + currency=currency, + amount=amount.quantize(FOUR_PLACES, rounding=ROUND_HALF_UP), + basis=basis, + ) + for currency, amount in sorted(values.items()) + ], + reason=reason, + required_inputs=list(required or []), + notes=list(notes or []), + ) + + +def _decimal_or_none(value: Any) -> Decimal | None: + if value is None or isinstance(value, bool): + return None + try: + return Decimal(str(value)) + except (InvalidOperation, ValueError): + return None + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_billing_periods.py b/server/src/app/services/commercial_billing_periods.py new file mode 100644 index 0000000..72850d9 --- /dev/null +++ b/server/src/app/services/commercial_billing_periods.py @@ -0,0 +1,240 @@ +"""不可变商业账期的签发、定位与历史读取。""" + +from __future__ import annotations + +from datetime import UTC, datetime + +from sqlalchemy import func, select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.models.commercial import TenantCommercialPlan, TenantSubscription +from app.models.commercial_billing import CommercialBillingPeriod +from app.schemas.commercial_billing import CommercialBillingPeriodRead +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) +from app.services.commercial_periods import billing_period_key + + +class CommercialBillingPeriodService: + def __init__(self, db: Session) -> None: + self.db = db + + def create_initial( + self, + subscription: TenantSubscription, + plan: TenantCommercialPlan, + *, + actor_id: str, + request_id: str, + ) -> tuple[CommercialBillingPeriod, bool]: + return self.issue( + subscription, + plan, + period_start=subscription.current_period_start, + period_end=subscription.current_period_end, + source="subscription_created", + actor_id=actor_id, + idempotency_key=f"{request_id}:initial-period", + ) + + def issue( + self, + subscription: TenantSubscription, + plan: TenantCommercialPlan, + *, + period_start: datetime, + period_end: datetime, + source: str, + actor_id: str, + idempotency_key: str, + ) -> tuple[CommercialBillingPeriod, bool]: + self._validate_scope(subscription, plan) + start = _utc(period_start) + end = _utc(period_end) + if end <= start: + raise CommercialConfigurationError("账期结束时间必须晚于开始时间。") + if start < _utc(subscription.starts_at): + raise CommercialConfigurationError("账期不能早于订阅合同开始时间。") + if subscription.ends_at is not None and end > _utc(subscription.ends_at): + raise CommercialConfigurationError("账期不能超过订阅合同结束时间。") + request_key = _required(idempotency_key, "账期幂等键")[:160] + existing = self._by_idempotency(subscription, request_key) + expected_key = billing_period_key( + subscription.tenant_id, + subscription.id, + start, + end, + ) + if existing is not None: + self._assert_same_window(existing, start, end, expected_key) + return existing, False + overlap = self.db.scalar( + select(CommercialBillingPeriod.id).where( + CommercialBillingPeriod.tenant_id == subscription.tenant_id, + CommercialBillingPeriod.subscription_id == subscription.id, + CommercialBillingPeriod.period_start < end, + CommercialBillingPeriod.period_end > start, + ) + ) + if overlap is not None: + raise CommercialConflictError("新账期与已签发不可变账期重叠。") + sequence = ( + int( + self.db.scalar( + select(func.max(CommercialBillingPeriod.period_sequence)).where( + CommercialBillingPeriod.tenant_id == subscription.tenant_id, + CommercialBillingPeriod.subscription_id == subscription.id, + ) + ) + or 0 + ) + + 1 + ) + row = CommercialBillingPeriod( + tenant_id=subscription.tenant_id, + subscription_id=subscription.id, + plan_id=plan.id, + period_sequence=sequence, + period_key=expected_key, + status="issued", + period_start=start, + period_end=end, + subscription_status_snapshot=subscription.status, + plan_code_snapshot=plan.plan_code, + plan_version_snapshot=plan.version, + pricing_model_snapshot=plan.pricing_model, + billing_interval=subscription.billing_interval, + currency=subscription.currency, + base_fee_snapshot=subscription.base_fee_snapshot, + seats_snapshot=subscription.seats, + source=source, + idempotency_key=request_key, + created_by=_required(actor_id, "账期签发人")[:120], + ) + try: + with self.db.begin_nested(): + self.db.add(row) + self.db.flush() + return row, True + except IntegrityError: + existing = self._by_idempotency(subscription, request_key) + if existing is None: + raise + self._assert_same_window(existing, start, end, expected_key) + return existing, False + + def resolve( + self, + tenant_id: str, + subscription_id: str, + occurred_at: datetime, + *, + for_update: bool = False, + ) -> CommercialBillingPeriod: + when = _utc(occurred_at) + statement = select(CommercialBillingPeriod).where( + CommercialBillingPeriod.tenant_id == _required(tenant_id, "租户编号"), + CommercialBillingPeriod.subscription_id == _required(subscription_id, "订阅编号"), + CommercialBillingPeriod.period_start <= when, + CommercialBillingPeriod.period_end > when, + ) + if for_update: + statement = statement.with_for_update() + rows = list(self.db.scalars(statement.limit(2)).all()) + if not rows: + raise CommercialConflictError("发生时间没有对应的已签发不可变账期。") + if len(rows) != 1: + raise CommercialConflictError("订阅存在重叠账期，已失败关闭。") + return rows[0] + + def list_periods( + self, + tenant_id: str, + *, + subscription_id: str | None = None, + limit: int = 200, + offset: int = 0, + ) -> list[CommercialBillingPeriod]: + statement = select(CommercialBillingPeriod).where( + CommercialBillingPeriod.tenant_id == _required(tenant_id, "租户编号") + ) + if subscription_id: + statement = statement.where( + CommercialBillingPeriod.subscription_id == subscription_id.strip() + ) + return list( + self.db.scalars( + statement.order_by( + CommercialBillingPeriod.period_start.desc(), + CommercialBillingPeriod.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + @staticmethod + def to_read( + row: CommercialBillingPeriod, + *, + as_of: datetime | None = None, + ) -> CommercialBillingPeriodRead: + when = _utc(as_of or datetime.now(UTC)) + start = _utc(row.period_start) + end = _utc(row.period_end) + temporal_state = "upcoming" if when < start else "elapsed" if when >= end else "current" + payload = { + column.name: getattr(row, column.name) + for column in CommercialBillingPeriod.__table__.columns + } + payload["temporal_state"] = temporal_state + return CommercialBillingPeriodRead.model_validate(payload) + + def _by_idempotency( + self, + subscription: TenantSubscription, + idempotency_key: str, + ) -> CommercialBillingPeriod | None: + return self.db.scalar( + select(CommercialBillingPeriod).where( + CommercialBillingPeriod.tenant_id == subscription.tenant_id, + CommercialBillingPeriod.subscription_id == subscription.id, + CommercialBillingPeriod.idempotency_key == idempotency_key, + ) + ) + + @staticmethod + def _validate_scope( + subscription: TenantSubscription, + plan: TenantCommercialPlan, + ) -> None: + if subscription.tenant_id != plan.tenant_id or subscription.plan_id != plan.id: + raise CommercialConfigurationError("账期套餐与订阅租户范围不一致。") + + @staticmethod + def _assert_same_window( + row: CommercialBillingPeriod, + start: datetime, + end: datetime, + period_key: str, + ) -> None: + if ( + _utc(row.period_start) != start + or _utc(row.period_end) != end + or row.period_key != period_key + ): + raise CommercialConflictError("账期幂等键已被不同周期占用。") + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + + +def _required(value: str | None, field_name: str) -> str: + normalized = str(value or "").strip() + if not normalized: + raise CommercialConfigurationError(f"{field_name}不能为空。") + return normalized diff --git a/server/src/app/services/commercial_direct_operation.py b/server/src/app/services/commercial_direct_operation.py new file mode 100644 index 0000000..98990cd --- /dev/null +++ b/server/src/app/services/commercial_direct_operation.py @@ -0,0 +1,694 @@ +"""绕过中央 AgentToolCall 的真实运行入口使用的独立事务商业桥。""" + +from __future__ import annotations + +import uuid +from collections.abc import Callable, Mapping +from dataclasses import dataclass +from datetime import UTC, datetime +from decimal import Decimal, InvalidOperation +from typing import Any, Literal + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.commercial import CommercialEntitlement, TenantSubscription +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.schemas.commercial import CommercialCostEventCreate, UsageMeterEventCreate +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) +from app.services.commercial_metering import CommercialMeteringService +from app.services.commercial_runtime_costs import runtime_cost_config +from app.services.commercial_runtime_policy import preflight_quantity +from app.services.commercial_runtime_registry import ( + ConfiguredRuntimeMeter, + matching_runtime_meters, +) +from app.services.commercial_runtime_reservations import ( + CommercialRuntimeReservationService, +) + +DirectOperationOutcome = Literal[ + "succeeded", + "empty", + "provider_rejected", + "postprocess_failed", + "not_sent", + "outcome_unknown", +] +DirectOperationStatus = Literal[ + "created", + "replayed", + "released", + "skipped", + "denied", + "reconciliation_required", +] + +_SUPPORTED_BASES = { + "call", + "bytes", + "events", + "input_tokens", + "objects", + "output_tokens", + "pages", + "total_tokens", + "duration_ms", +} +_SOURCE_SYSTEM = "direct-runtime-metering" +_METER_VERSION = "direct-operation-v1" + + +@dataclass(frozen=True, slots=True) +class DirectOperationIdentity: + tenant_id: str + operation_key: str + run_key: str + tool_type: str + tool_name: str + started_at: datetime + provider: str | None = None + model_name: str | None = None + + def __post_init__(self) -> None: + object.__setattr__(self, "tenant_id", _required(self.tenant_id, "tenant_id", 64)) + object.__setattr__( + self, + "operation_key", + _required(self.operation_key, "operation_key", 500), + ) + object.__setattr__(self, "run_key", _required(self.run_key, "run_key", 500)) + object.__setattr__(self, "tool_type", _required(self.tool_type, "tool_type", 30)) + object.__setattr__(self, "tool_name", _required(self.tool_name, "tool_name", 100)) + object.__setattr__(self, "started_at", _utc(self.started_at)) + object.__setattr__(self, "provider", _optional(self.provider, 80)) + object.__setattr__(self, "model_name", _optional(self.model_name, 160)) + + @property + def operation_call_id(self) -> str: + return _stable_uuid( + "direct-operation", + self.tenant_id, + self.operation_key, + self.tool_type, + self.tool_name, + ) + + @property + def metering_run_id(self) -> str: + return f"direct-{_stable_uuid('direct-run', self.tenant_id, self.run_key)}" + + +@dataclass(frozen=True, slots=True) +class DirectOperationPermit: + enforced: bool + allowed: bool + reason_code: str + reason: str + operation_call_id: str + reservation_id: str | None = None + entitlement_id: str | None = None + quantity_basis: str | None = None + reserved_quantity: Decimal | None = None + + +@dataclass(frozen=True, slots=True) +class DirectOperationResult: + status: DirectOperationStatus + reason_code: str + reason: str + operation_call_id: str + reservation_id: str | None = None + quantity_basis: str | None = None + quantity: Decimal | None = None + usage_event_id: str | None = None + cost_event_id: str | None = None + requires_reconciliation: bool = False + + +class CommercialDirectOperationBridge: + """每次 permit/completion 都使用自有 Session，不提交调用方业务事务。""" + + def __init__( + self, + session_factory: Callable[[], Session], + *, + lookup_session: Session | None = None, + ) -> None: + self.session_factory = session_factory + self.lookup_session = lookup_session + + def permit( + self, + identity: DirectOperationIdentity, + *, + requested_quantity: int | Decimal | None = None, + required_quantity_basis: str | None = None, + ) -> DirectOperationPermit: + if self.lookup_session is not None: + with self.lookup_session.no_autoflush: + configured = matching_runtime_meters( + self.lookup_session, + tenant_id=identity.tenant_id, + tool_type=identity.tool_type, + tool_name=identity.tool_name, + as_of=identity.started_at, + ) + if not configured: + return DirectOperationPermit( + enforced=False, + allowed=True, + reason_code="runtime_meter_not_configured", + reason="该真实运行入口未显式配置商业计量。", + operation_call_id=identity.operation_call_id, + ) + with self.session_factory() as db: + try: + meters = matching_runtime_meters( + db, + tenant_id=identity.tenant_id, + tool_type=identity.tool_type, + tool_name=identity.tool_name, + as_of=identity.started_at, + ) + if not meters: + db.rollback() + return DirectOperationPermit( + enforced=False, + allowed=True, + reason_code="runtime_meter_not_configured", + reason="该真实运行入口未显式配置商业计量。", + operation_call_id=identity.operation_call_id, + ) + meter = _unique_meter(meters) + basis, reserved = _preflight( + meter, + requested_quantity=requested_quantity, + required_quantity_basis=required_quantity_basis, + ) + reservation, _created = CommercialRuntimeReservationService(db).reserve( + tenant_id=identity.tenant_id, + entitlement_id=meter.entitlement.id, + run_id=identity.metering_run_id, + tool_call_id=identity.operation_call_id, + tool_type=identity.tool_type, + tool_name=identity.tool_name, + quantity_basis=basis, + reserved_quantity=reserved, + meter_config=meter.config, + as_of=identity.started_at, + ) + db.commit() + return DirectOperationPermit( + enforced=True, + allowed=True, + reason_code="reserved", + reason="真实运行调用已完成独立事务额度预占。", + operation_call_id=identity.operation_call_id, + reservation_id=reservation.id, + entitlement_id=reservation.entitlement_id, + quantity_basis=reservation.quantity_basis, + reserved_quantity=Decimal(reservation.reserved_quantity), + ) + except ( + CommercialConfigurationError, + CommercialConflictError, + LookupError, + ValueError, + ) as error: + db.rollback() + return DirectOperationPermit( + enforced=True, + allowed=False, + reason_code="quota_or_configuration_denied", + reason=str(error), + operation_call_id=identity.operation_call_id, + ) + except Exception: + db.rollback() + raise + + def complete( + self, + identity: DirectOperationIdentity, + *, + outcome: DirectOperationOutcome, + authoritative_quantities: Mapping[str, int | Decimal | None], + completed_at: datetime, + usage_source: str, + usage_availability: str, + ) -> DirectOperationResult: + completed = _utc(completed_at) + request_sent = outcome != "not_sent" + with self.session_factory() as db: + reservation = CommercialRuntimeReservationService(db).by_tool_call( + identity.operation_call_id + ) + if reservation is None: + db.rollback() + if not request_sent: + return _result( + identity, + status="skipped", + reason_code="not_sent_without_reservation", + reason="调用未发送且没有商业预占，无需计量。", + ) + return self._backlog_without_reservation( + identity, + outcome=outcome, + quantities=authoritative_quantities, + completed_at=completed, + ) + reservation_id = reservation.id + if not request_sent: + try: + CommercialRuntimeReservationService(db).release( + reservation_id, + reason_code="direct_operation_not_sent", + settled_at=completed, + ) + db.commit() + return _result( + identity, + status="released", + reason_code="not_sent_released", + reason="调用未发送，商业预占已释放。", + reservation=reservation, + ) + except CommercialConflictError: + db.rollback() + return self._terminal_replay(identity, reservation_id) + + actual = _actual_quantity( + reservation.quantity_basis, + authoritative_quantities, + started_at=identity.started_at, + completed_at=completed, + ) + if actual is None: + db.rollback() + return self._mark_reconciliation( + identity, + reservation_id, + reason_code="authoritative_usage_unavailable", + actual_quantity=None, + ) + if actual > Decimal(reservation.reserved_quantity): + db.rollback() + return self._mark_reconciliation( + identity, + reservation_id, + reason_code="actual_exceeds_preflight_reservation", + actual_quantity=actual, + ) + try: + locked = CommercialRuntimeReservationService(db).mark_committed( + reservation_id, + actual, + settled_at=completed, + ) + result = _record_facts( + db, + identity=identity, + reservation=locked, + outcome=outcome, + quantity=actual, + usage_source=usage_source, + usage_availability=usage_availability, + ) + db.commit() + return result + except Exception: + db.rollback() + return self._mark_reconciliation( + identity, + reservation_id, + reason_code="direct_operation_metering_failed", + actual_quantity=actual, + ) + + def _backlog_without_reservation( + self, + identity: DirectOperationIdentity, + *, + outcome: DirectOperationOutcome, + quantities: Mapping[str, int | Decimal | None], + completed_at: datetime, + ) -> DirectOperationResult: + del outcome + with self.session_factory() as db: + meters = matching_runtime_meters( + db, + tenant_id=identity.tenant_id, + tool_type=identity.tool_type, + tool_name=identity.tool_name, + as_of=identity.started_at, + ) + if not meters: + db.rollback() + return _result( + identity, + status="skipped", + reason_code="runtime_meter_not_configured", + reason="调用已发生，但该入口未显式启用商业计量。", + ) + try: + meter = _unique_meter(meters) + basis, reserved = _preflight(meter) + actual = _actual_quantity( + basis, + quantities, + started_at=identity.started_at, + completed_at=completed_at, + ) + backlog, _created = CommercialRuntimeReservationService( + db + ).record_reconciliation_backlog( + tenant_id=identity.tenant_id, + entitlement_id=meter.entitlement.id, + run_id=identity.metering_run_id, + tool_call_id=identity.operation_call_id, + tool_type=identity.tool_type, + tool_name=identity.tool_name, + quantity_basis=basis, + reserved_quantity=reserved, + actual_quantity=actual, + meter_config=meter.config, + resolution_code="missing_pre_execution_reservation", + as_of=identity.started_at, + ) + db.commit() + return _result( + identity, + status="reconciliation_required", + reason_code="missing_pre_execution_reservation", + reason="真实调用缺少执行前预占，已冻结额度并进入补偿队列。", + reservation=backlog, + quantity=actual, + requires_reconciliation=True, + ) + except Exception: + db.rollback() + raise + + def _mark_reconciliation( + self, + identity: DirectOperationIdentity, + reservation_id: str, + *, + reason_code: str, + actual_quantity: Decimal | None, + ) -> DirectOperationResult: + with self.session_factory() as db: + row = CommercialRuntimeReservationService(db).require_reconciliation( + reservation_id, + reason_code=reason_code, + actual_quantity=actual_quantity, + ) + db.commit() + return _result( + identity, + status="reconciliation_required", + reason_code=reason_code, + reason="真实调用尚不能安全结算，已保留可审计补偿状态。", + reservation=row, + quantity=actual_quantity, + requires_reconciliation=True, + ) + + def _terminal_replay( + self, + identity: DirectOperationIdentity, + reservation_id: str, + ) -> DirectOperationResult: + with self.session_factory() as db: + row = CommercialRuntimeReservationService(db).get_for_update(reservation_id) + if row.status == "released": + db.rollback() + return _result( + identity, + status="released", + reason_code="not_sent_release_replayed", + reason="调用未发送，原商业预占已释放。", + reservation=row, + ) + db.rollback() + raise CommercialConflictError(f"直接运行预占已处于不兼容终态 {row.status}。") + + +def _record_facts( + db: Session, + *, + identity: DirectOperationIdentity, + reservation: CommercialRuntimeReservation, + outcome: DirectOperationOutcome, + quantity: Decimal, + usage_source: str, + usage_availability: str, +) -> DirectOperationResult: + subscription = db.scalar( + select(TenantSubscription).where( + TenantSubscription.tenant_id == identity.tenant_id, + TenantSubscription.id == reservation.subscription_id, + ) + ) + entitlement = db.scalar( + select(CommercialEntitlement).where( + CommercialEntitlement.tenant_id == identity.tenant_id, + CommercialEntitlement.subscription_id == reservation.subscription_id, + CommercialEntitlement.id == reservation.entitlement_id, + ) + ) + if subscription is None or entitlement is None: + raise LookupError("直接运行预占引用的订阅或权益不存在。") + metadata = { + "meter_version": _METER_VERSION, + "operation_call_id": identity.operation_call_id, + "tool_type": identity.tool_type, + "tool_name": identity.tool_name, + "quantity_basis": reservation.quantity_basis, + "entitlement_key": entitlement.entitlement_key, + "reservation_id": reservation.id, + "provider": identity.provider, + "model_name": identity.model_name, + "outcome": outcome, + "usage_source": _optional(usage_source, 80), + "usage_availability": _optional(usage_availability, 40), + } + usage, usage_created = CommercialMeteringService(db).record_usage( + identity.tenant_id, + UsageMeterEventCreate( + subscription_id=subscription.id, + entitlement_id=entitlement.id, + quantity=quantity, + occurred_at=_utc(reservation.created_at), + source_system=_SOURCE_SYSTEM, + idempotency_key=_usage_key(identity.operation_call_id, entitlement.id), + subject_type="runtime_provider_attempt", + subject_id=identity.operation_call_id, + correlation_id=identity.metering_run_id, + metadata_json=metadata, + ), + actor_type="system", + actor_id="commercial-direct-runtime-meter", + ) + cost = runtime_cost_config( + reservation.meter_config_json.get("internal_cost"), + entitlement.unit, + ) + cost_event = None + cost_created = False + if cost is not None: + try: + with db.begin_nested(): + cost_event, cost_created = CommercialMeteringService(db).record_cost( + identity.tenant_id, + CommercialCostEventCreate( + subscription_id=subscription.id, + usage_event_id=usage.id, + cost_category=cost.cost_category, # type: ignore[arg-type] + quantity=quantity, + unit=cost.unit, + unit_cost=cost.unit_cost, + original_currency=cost.original_currency, + reporting_currency=cost.reporting_currency, + fx_rate=cost.fx_rate, + provider=cost.provider, + sku=cost.sku, + model_name=cost.model_name, + allocation_key=f"direct-runtime:{entitlement.id}", + occurred_at=_utc(reservation.created_at), + source_system=_SOURCE_SYSTEM, + idempotency_key=_cost_key( + identity.operation_call_id, + entitlement.id, + ), + correlation_id=identity.metering_run_id, + metadata_json=metadata, + ), + ) + except Exception: + CommercialRuntimeReservationService(db).require_committed_reconciliation( + reservation.id, + reason_code="direct_operation_cost_failed", + ) + return _result( + identity, + status="reconciliation_required", + reason_code="direct_operation_cost_failed", + reason="真实用量已持久化，内部成本等待幂等补偿。", + reservation=reservation, + quantity=quantity, + usage_event_id=usage.id, + requires_reconciliation=True, + ) + created = usage_created or cost_created + return _result( + identity, + status="created" if created else "replayed", + reason_code="metered" if created else "idempotent_replay", + reason="真实运行用量与已配置成本已追加。" if created else "相同真实运行事实已计量。", + reservation=reservation, + quantity=quantity, + usage_event_id=usage.id, + cost_event_id=cost_event.id if cost_event is not None else None, + ) + + +def _unique_meter(meters: list[ConfiguredRuntimeMeter]) -> ConfiguredRuntimeMeter: + if len(meters) != 1: + raise CommercialConfigurationError("多个权益匹配同一真实运行入口，拒绝重复计量。") + return meters[0] + + +def _preflight( + meter: ConfiguredRuntimeMeter, + *, + requested_quantity: int | Decimal | None = None, + required_quantity_basis: str | None = None, +) -> tuple[str, Decimal]: + basis = str(meter.config.get("quantity_basis") or "").strip() + if basis not in _SUPPORTED_BASES: + raise CommercialConfigurationError("runtime_meter.quantity_basis 未配置或不受支持。") + required_basis = str(required_quantity_basis or "").strip() + if required_basis and basis != required_basis: + raise CommercialConfigurationError( + f"该真实资源边界只接受 {required_basis} 计量，当前配置为 {basis}。" + ) + if basis == "call": + if requested_quantity is not None and _positive_optional(requested_quantity) != 1: + raise CommercialConfigurationError("call 计量的 requested_quantity 只能是 1。") + return basis, Decimal("1") + if "preflight_quantity" not in meter.config: + raise CommercialConfigurationError( + "变量用量的直接运行入口必须显式配置可强制的 preflight_quantity。" + ) + maximum = preflight_quantity([meter.config]) + if requested_quantity is None: + return basis, maximum + requested = _positive_optional(requested_quantity) + if requested is None: + raise CommercialConfigurationError("requested_quantity 必须是权威正数。") + if requested > maximum: + raise CommercialConfigurationError( + "requested_quantity 超过 runtime_meter.preflight_quantity 硬上限。" + ) + return basis, requested + + +def _actual_quantity( + basis: str, + quantities: Mapping[str, int | Decimal | None], + *, + started_at: datetime, + completed_at: datetime, +) -> Decimal | None: + if basis == "call": + return Decimal("1") + if basis == "duration_ms": + milliseconds = int((_utc(completed_at) - _utc(started_at)).total_seconds() * 1000) + return Decimal(milliseconds) if milliseconds > 0 else None + if basis == "input_tokens": + return _positive_optional(quantities.get("input_tokens")) + if basis == "output_tokens": + return _positive_optional(quantities.get("output_tokens")) + if basis in {"bytes", "pages", "objects", "events"}: + return _positive_optional(quantities.get(basis)) + total = _positive_optional(quantities.get("total_tokens")) + if total is not None: + return total + input_tokens = _positive_optional(quantities.get("input_tokens")) + output_tokens = _positive_optional(quantities.get("output_tokens")) + if input_tokens is None or output_tokens is None: + return None + return input_tokens + output_tokens + + +def _result( + identity: DirectOperationIdentity, + *, + status: DirectOperationStatus, + reason_code: str, + reason: str, + reservation: CommercialRuntimeReservation | None = None, + quantity: Decimal | None = None, + usage_event_id: str | None = None, + cost_event_id: str | None = None, + requires_reconciliation: bool = False, +) -> DirectOperationResult: + return DirectOperationResult( + status=status, + reason_code=reason_code, + reason=reason, + operation_call_id=identity.operation_call_id, + reservation_id=reservation.id if reservation is not None else None, + quantity_basis=reservation.quantity_basis if reservation is not None else None, + quantity=quantity, + usage_event_id=usage_event_id, + cost_event_id=cost_event_id, + requires_reconciliation=requires_reconciliation, + ) + + +def _stable_uuid(domain: str, *parts: str) -> str: + key = "\x1f".join((domain, *(str(item) for item in parts))) + return str(uuid.uuid5(uuid.NAMESPACE_URL, f"x-financial:{key}")) + + +def _usage_key(operation_call_id: str, entitlement_id: str) -> str: + return f"direct:{operation_call_id}:{entitlement_id}:usage:v1" + + +def _cost_key(operation_call_id: str, entitlement_id: str) -> str: + return f"direct:{operation_call_id}:{entitlement_id}:cost:v1" + + +def _positive_optional(value: Any) -> Decimal | None: + if value is None or isinstance(value, bool): + return None + try: + parsed = Decimal(str(value)) + except (InvalidOperation, ValueError): + return None + if not parsed.is_finite() or parsed <= 0: + return None + return parsed + + +def _required(value: Any, field: str, maximum: int) -> str: + normalized = str(value or "").strip() + if not normalized or len(normalized) > maximum: + raise ValueError(f"{field} is required and must be at most {maximum} characters.") + return normalized + + +def _optional(value: Any, maximum: int) -> str | None: + normalized = str(value or "").strip() + return normalized[:maximum] or None + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_entitlements.py b/server/src/app/services/commercial_entitlements.py new file mode 100644 index 0000000..7bf8fa7 --- /dev/null +++ b/server/src/app/services/commercial_entitlements.py @@ -0,0 +1,293 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal +from typing import Literal + +from sqlalchemy import func, select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.commercial import ( + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) +from app.models.commercial_billing import CommercialBillingPeriod +from app.schemas.commercial import ( + CommercialAccountRead, + CommercialQuotaRead, + EntitlementGateRead, +) +from app.services.commercial_access_policy import CommercialAccessPolicy +from app.services.commercial_billing_periods import CommercialBillingPeriodService +from app.services.commercial_periods import quota_period_key_for +from app.services.commercial_runtime_reservations import active_reserved_quantity + +CURRENT_SUBSCRIPTION_STATUSES = {"trialing", "active", "past_due", "suspended"} + + +class CommercialEntitlementService: + """计算租户可见配额，并把商业门禁与安全门禁做收紧式合并。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def get_account( + self, + current_user: CurrentUserContext, + *, + as_of: datetime | None = None, + ) -> CommercialAccountRead: + tenant_id = CommercialAccessPolicy.require_account_read(current_user) + return self.get_account_for_tenant(tenant_id, as_of=as_of) + + def get_account_for_tenant( + self, + tenant_id: str, + *, + as_of: datetime | None = None, + ) -> CommercialAccountRead: + normalized_as_of = _utc(as_of or datetime.now(UTC)) + subscription = self.db.scalar( + select(TenantSubscription).where( + TenantSubscription.tenant_id == tenant_id, + TenantSubscription.status.in_(CURRENT_SUBSCRIPTION_STATUSES), + ) + ) + if subscription is None: + return CommercialAccountRead( + tenant_id=tenant_id, + as_of=normalized_as_of, + data_status="unavailable", + notes=["租户尚未配置当前商业订阅。"], + ) + plan = self.db.scalar( + select(TenantCommercialPlan).where( + TenantCommercialPlan.tenant_id == tenant_id, + TenantCommercialPlan.id == subscription.plan_id, + ) + ) + if plan is None: + return CommercialAccountRead( + tenant_id=tenant_id, + as_of=normalized_as_of, + data_status="partial", + subscription=subscription, + notes=["订阅套餐快照引用缺失，商业账户数据不完整。"], + ) + entitlements = list( + self.db.scalars( + select(CommercialEntitlement) + .where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.subscription_id == subscription.id, + ) + .order_by(CommercialEntitlement.entitlement_key) + ).all() + ) + quotas = [ + self._quota(subscription, entitlement, normalized_as_of) for entitlement in entitlements + ] + notes: list[str] = [] + data_status: Literal["available", "partial", "unavailable"] = "available" + if not quotas: + data_status = "partial" + notes.append("当前订阅尚未配置任何商业权益。") + if subscription.status not in {"trialing", "active"}: + data_status = "partial" + notes.append("当前订阅不是可消费状态，所有商业权益将失败关闭。") + return CommercialAccountRead( + tenant_id=tenant_id, + as_of=normalized_as_of, + data_status=data_status, + plan=plan, + subscription=subscription, + quotas=quotas, + notes=notes, + ) + + def check( + self, + tenant_id: str, + *, + entitlement_key: str, + requested_quantity: Decimal, + security_decision: Literal["allow", "deny", "human_review"], + as_of: datetime | None = None, + ) -> EntitlementGateRead: + """最终允许必须同时满足商业门禁和上游安全门禁；商业配置不能放宽安全。""" + if requested_quantity <= 0: + raise ValueError("权益检查数量必须大于 0。") + account = self.get_account_for_tenant(tenant_id, as_of=as_of) + quota = next( + ( + item + for item in account.quotas + if item.entitlement.entitlement_key == entitlement_key + ), + None, + ) + commercial_allowed = bool(quota and self._allows(quota, requested_quantity)) + final_allowed = commercial_allowed and security_decision == "allow" + if security_decision == "deny": + reason = "安全或租户门禁拒绝；商业权益不能覆盖该拒绝。" + elif security_decision == "human_review": + reason = "高风险动作必须人工审核；商业权益不能改为自动执行。" + elif quota is None: + reason = "未配置对应商业权益。" + elif not commercial_allowed: + reason = quota.reason + else: + reason = "商业权益和安全门禁均允许。" + return EntitlementGateRead( + tenant_id=tenant_id, + entitlement_key=entitlement_key, + requested_quantity=requested_quantity, + commercial_allowed=commercial_allowed, + security_decision=security_decision, + final_allowed=final_allowed, + reason=reason, + quota=quota, + ) + + def _quota( + self, + subscription: TenantSubscription, + entitlement: CommercialEntitlement, + as_of: datetime, + ) -> CommercialQuotaRead: + billing_period = CommercialBillingPeriodService(self.db).resolve( + subscription.tenant_id, + subscription.id, + as_of, + ) + quota_period_key = quota_period_key_for(entitlement, subscription, as_of) + raw_used = Decimal( + self.db.scalar( + select(func.coalesce(func.sum(UsageMeterEvent.quantity), 0)).where( + UsageMeterEvent.tenant_id == subscription.tenant_id, + UsageMeterEvent.subscription_id == subscription.id, + UsageMeterEvent.entitlement_id == entitlement.id, + UsageMeterEvent.quota_period_key == quota_period_key, + UsageMeterEvent.occurred_at <= as_of, + ) + ) + or 0 + ) + used = max(Decimal("0"), raw_used) + reserved = max( + Decimal("0"), + active_reserved_quantity( + self.db, + tenant_id=subscription.tenant_id, + subscription_id=subscription.id, + entitlement_id=entitlement.id, + period_key=quota_period_key, + ), + ) + included = ( + Decimal(entitlement.included_quantity) + if entitlement.included_quantity is not None + else None + ) + hard_limit = ( + Decimal(entitlement.hard_limit_quantity) + if entitlement.hard_limit_quantity is not None + else None + ) + committed_and_reserved = used + reserved + included_remaining = ( + max(Decimal("0"), included - committed_and_reserved) if included is not None else None + ) + hard_remaining = ( + max(Decimal("0"), hard_limit - committed_and_reserved) + if hard_limit is not None + else None + ) + overage = max(Decimal("0"), used - included) if included is not None else Decimal("0") + active = self._active(subscription, entitlement, billing_period, as_of) + if not active: + status = "inactive" + allowed = False + reason = "订阅或权益当前不可用。" + elif entitlement.entitlement_type == "unlimited": + status = "unlimited" + allowed = True + reason = "权益为无限量。" + else: + effective_limit = hard_limit + if effective_limit is None and entitlement.overage_policy == "block": + effective_limit = included + exhausted = effective_limit is not None and committed_and_reserved >= effective_limit + if exhausted: + status = "exhausted" + allowed = False + reason = "商业硬配额已耗尽。" + else: + approaching = bool( + effective_limit is not None + and effective_limit > 0 + and committed_and_reserved / effective_limit >= Decimal("0.8") + ) + status = "approaching" if approaching else "available" + allowed = True + reason = "商业权益可用。" + return CommercialQuotaRead( + entitlement=entitlement, + billing_period_id=billing_period.id, + period_key=billing_period.period_key, + quota_period_key=quota_period_key, + used_quantity=used, + reserved_quantity=reserved, + included_remaining=included_remaining, + hard_limit_remaining=hard_remaining, + overage_quantity=overage, + status=status, + commercially_allowed=allowed, + reason=reason, + ) + + @staticmethod + def _allows(quota: CommercialQuotaRead, requested_quantity: Decimal) -> bool: + if not quota.commercially_allowed: + return False + if quota.status == "unlimited": + return True + entitlement = quota.entitlement + limit = entitlement.hard_limit_quantity + if limit is None and entitlement.overage_policy == "block": + limit = entitlement.included_quantity + return ( + limit is None + or quota.used_quantity + quota.reserved_quantity + requested_quantity <= limit + ) + + @staticmethod + def _active( + subscription: TenantSubscription, + entitlement: CommercialEntitlement, + billing_period: CommercialBillingPeriod, + as_of: datetime, + ) -> bool: + if subscription.status not in {"trialing", "active"} or entitlement.status != "active": + return False + subscription_start = _utc(subscription.starts_at) + subscription_end = _utc(subscription.ends_at) if subscription.ends_at else None + period_start = _utc(billing_period.period_start) + period_end = _utc(billing_period.period_end) + if ( + as_of < subscription_start + or (subscription_end is not None and as_of >= subscription_end) + or as_of < period_start + or as_of >= period_end + ): + return False + effective_from = _utc(entitlement.effective_from) + effective_to = _utc(entitlement.effective_to) if entitlement.effective_to else None + return as_of >= effective_from and (effective_to is None or as_of < effective_to) + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_metering.py b/server/src/app/services/commercial_metering.py new file mode 100644 index 0000000..52d8120 --- /dev/null +++ b/server/src/app/services/commercial_metering.py @@ -0,0 +1,549 @@ +from __future__ import annotations + +import hashlib +import json +from datetime import UTC, datetime, timedelta +from decimal import ROUND_HALF_UP, Decimal +from typing import Any, Literal + +from sqlalchemy import func, select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.models.commercial import ( + CommercialCostEvent, + CommercialEntitlement, + TenantSubscription, + UsageMeterEvent, +) +from app.models.commercial_billing import CommercialBillingPeriod +from app.schemas.commercial import CommercialCostEventCreate, UsageMeterEventCreate +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) +from app.services.commercial_billing_periods import CommercialBillingPeriodService +from app.services.commercial_periods import quota_period_key_for +from app.services.commercial_runtime_reservations import active_reserved_quantity + +FOUR_PLACES = Decimal("0.0001") + + +class CommercialMeteringService: + """写入追加式用量/成本事实，所有派生字段均由服务端生成。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def record_usage( + self, + tenant_id: str, + payload: UsageMeterEventCreate, + *, + actor_type: Literal["system", "user", "integration", "admin"], + actor_id: str, + ) -> tuple[UsageMeterEvent, bool]: + tenant_id = _required(tenant_id, "租户编号") + source_system = _required(payload.source_system, "来源系统") + idempotency_key = _required(payload.idempotency_key, "幂等键") + actor_id = _required(actor_id, "事件操作人") + fingerprint = _fingerprint(tenant_id, payload.model_dump(mode="json")) + existing = self._usage_by_idempotency( + tenant_id, + source_system, + idempotency_key, + ) + if existing is not None: + self._assert_same_fingerprint(existing.request_fingerprint, fingerprint) + return existing, False + + subscription = self._subscription(tenant_id, payload.subscription_id) + entitlement = self._entitlement( + tenant_id, + payload.subscription_id, + payload.entitlement_id, + ) + # 锁定权益后再查一次，避免并发重放被后续配额校验误判为超限。 + existing = self._usage_by_idempotency( + tenant_id, + source_system, + idempotency_key, + ) + if existing is not None: + self._assert_same_fingerprint(existing.request_fingerprint, fingerprint) + return existing, False + occurred_at = _utc(payload.occurred_at) + if occurred_at > datetime.now(UTC) + timedelta(minutes=5): + raise CommercialConfigurationError("不能写入未来发生的商业用量事实。") + billing_period = ( + None + if payload.event_type == "reversal" + else CommercialBillingPeriodService(self.db).resolve( + tenant_id, + subscription.id, + occurred_at, + ) + ) + if payload.event_type == "usage" and billing_period is not None: + self._assert_consumable( + subscription, + entitlement, + billing_period, + occurred_at, + ) + + quantity = payload.quantity + period_key = billing_period.period_key if billing_period is not None else "" + quota_period_key = quota_period_key_for(entitlement, subscription, occurred_at) + billing_period_id = billing_period.id if billing_period is not None else "" + if payload.event_type == "reversal": + original = self._usage_event( + tenant_id, + payload.subscription_id, + payload.entitlement_id, + str(payload.reversal_of_event_id), + ) + if original is None: + raise LookupError("被冲回的用量事件不存在。") + if original.event_type == "reversal": + raise CommercialConflictError("不能再次冲回 reversal 用量事件。") + if self.db.scalar( + select(UsageMeterEvent.id).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.reversal_of_event_id == original.id, + ) + ): + raise CommercialConflictError("该用量事件已经冲回。") + if quantity != -Decimal(original.quantity): + raise CommercialConfigurationError("冲回数量必须与原事件数量完全相反。") + period_key = original.period_key + quota_period_key = original.quota_period_key + billing_period_id = original.billing_period_id + + current_usage = Decimal( + self.db.scalar( + select(func.coalesce(func.sum(UsageMeterEvent.quantity), 0)).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.subscription_id == subscription.id, + UsageMeterEvent.entitlement_id == entitlement.id, + UsageMeterEvent.quota_period_key == quota_period_key, + ) + ) + or 0 + ) + active_reservations = active_reserved_quantity( + self.db, + tenant_id=tenant_id, + subscription_id=subscription.id, + entitlement_id=entitlement.id, + period_key=quota_period_key, + ) + resulting_usage = current_usage + quantity + if resulting_usage < 0: + raise CommercialConflictError("用量抵扣或冲回后不能形成负数配额消耗。") + if quantity > 0: + hard_limit = ( + Decimal(entitlement.hard_limit_quantity) + if entitlement.hard_limit_quantity is not None + else None + ) + if hard_limit is None and entitlement.overage_policy == "block": + hard_limit = ( + Decimal(entitlement.included_quantity) + if entitlement.included_quantity is not None + else None + ) + if hard_limit is not None and resulting_usage + active_reservations > hard_limit: + raise CommercialConflictError("用量事件将超过商业硬配额，已失败关闭。") + + row = UsageMeterEvent( + tenant_id=tenant_id, + subscription_id=subscription.id, + entitlement_id=entitlement.id, + billing_period_id=billing_period_id, + event_type=payload.event_type, + metric_key=entitlement.metric_key, + quantity=quantity, + unit=entitlement.unit, + period_key=period_key, + quota_period_key=quota_period_key, + occurred_at=occurred_at, + source_system=source_system, + idempotency_key=idempotency_key, + request_fingerprint=fingerprint, + reversal_of_event_id=payload.reversal_of_event_id, + subject_type=_blank_to_none(payload.subject_type), + subject_id=_blank_to_none(payload.subject_id), + actor_type=actor_type, + actor_id=actor_id, + correlation_id=_blank_to_none(payload.correlation_id), + trace_id=_blank_to_none(payload.trace_id), + metadata_json=dict(payload.metadata_json), + ) + return self._flush_usage_idempotently(row, fingerprint) + + def record_cost( + self, + tenant_id: str, + payload: CommercialCostEventCreate, + ) -> tuple[CommercialCostEvent, bool]: + tenant_id = _required(tenant_id, "租户编号") + source_system = _required(payload.source_system, "来源系统") + idempotency_key = _required(payload.idempotency_key, "幂等键") + fingerprint = _fingerprint(tenant_id, payload.model_dump(mode="json")) + existing = self._cost_by_idempotency( + tenant_id, + source_system, + idempotency_key, + ) + if existing is not None: + self._assert_same_fingerprint(existing.request_fingerprint, fingerprint) + return existing, False + + occurred_at = self._validated_cost_occurred_at(payload.occurred_at) + subscription: TenantSubscription | None = None + billing_period_id: str | None = None + if payload.subscription_id: + subscription = self._subscription(tenant_id, payload.subscription_id) + usage_event: UsageMeterEvent | None = None + if payload.usage_event_id: + usage_event = self.db.scalar( + select(UsageMeterEvent).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.subscription_id == payload.subscription_id, + UsageMeterEvent.id == payload.usage_event_id, + ) + ) + if usage_event is None: + raise LookupError("成本关联的用量事件不存在。") + billing_period_id = usage_event.billing_period_id + elif subscription is not None and payload.event_type != "reversal": + billing_period_id = ( + CommercialBillingPeriodService(self.db) + .resolve( + tenant_id, + subscription.id, + occurred_at, + ) + .id + ) + + quantity = payload.quantity + unit = _required(payload.unit, "成本计量单位") + unit_cost = payload.unit_cost + original_currency = payload.original_currency.upper() + reporting_currency = payload.reporting_currency.upper() + fx_rate = payload.fx_rate + cost_category = payload.cost_category + allocation_key = _required(payload.allocation_key, "成本分摊键") + direction = Decimal("-1") if payload.event_type == "credit" else Decimal("1") + cost_amount = (quantity * unit_cost * direction).quantize( + FOUR_PLACES, + rounding=ROUND_HALF_UP, + ) + reporting_amount = (cost_amount * fx_rate).quantize( + FOUR_PLACES, + rounding=ROUND_HALF_UP, + ) + + if payload.event_type == "reversal": + original = self._cost_event(tenant_id, str(payload.reversal_of_cost_event_id)) + if original is None: + raise LookupError("被冲回的成本事件不存在。") + existing = self._cost_by_idempotency( + tenant_id, + source_system, + idempotency_key, + ) + if existing is not None: + self._assert_same_fingerprint(existing.request_fingerprint, fingerprint) + return existing, False + if original.event_type == "reversal": + raise CommercialConflictError("不能再次冲回 reversal 成本事件。") + if self.db.scalar( + select(CommercialCostEvent.id).where( + CommercialCostEvent.tenant_id == tenant_id, + CommercialCostEvent.reversal_of_cost_event_id == original.id, + ) + ): + raise CommercialConflictError("该成本事件已经冲回。") + self._assert_cost_reversal_payload(payload, original) + subscription = ( + self._subscription(tenant_id, original.subscription_id) + if original.subscription_id + else None + ) + usage_event = ( + self.db.scalar( + select(UsageMeterEvent).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.id == original.usage_event_id, + ) + ) + if original.usage_event_id + else None + ) + quantity = Decimal(original.quantity) + unit = original.unit + unit_cost = Decimal(original.unit_cost) + original_currency = original.original_currency + reporting_currency = original.reporting_currency + fx_rate = Decimal(original.fx_rate) + cost_category = original.cost_category + allocation_key = original.allocation_key + cost_amount = -Decimal(original.cost_amount) + reporting_amount = -Decimal(original.reporting_amount) + billing_period_id = original.billing_period_id + + if cost_amount == 0 or reporting_amount == 0: + raise CommercialConfigurationError("零金额不能写入内部成本事实。") + row = CommercialCostEvent( + tenant_id=tenant_id, + subscription_id=subscription.id if subscription else None, + billing_period_id=billing_period_id, + usage_event_id=usage_event.id if usage_event else None, + event_type=payload.event_type, + cost_category=cost_category, + quantity=quantity, + unit=unit, + unit_cost=unit_cost, + cost_amount=cost_amount, + original_currency=original_currency, + reporting_amount=reporting_amount, + reporting_currency=reporting_currency, + fx_rate=fx_rate, + provider=_blank_to_none(payload.provider), + sku=_blank_to_none(payload.sku), + model_name=_blank_to_none(payload.model_name), + allocation_key=allocation_key, + occurred_at=occurred_at, + source_system=source_system, + idempotency_key=idempotency_key, + request_fingerprint=fingerprint, + reversal_of_cost_event_id=payload.reversal_of_cost_event_id, + correlation_id=_blank_to_none(payload.correlation_id), + trace_id=_blank_to_none(payload.trace_id), + metadata_json=dict(payload.metadata_json), + ) + return self._flush_cost_idempotently(row, fingerprint) + + def _subscription(self, tenant_id: str, subscription_id: str) -> TenantSubscription: + row = self.db.scalar( + select(TenantSubscription) + .where( + TenantSubscription.tenant_id == tenant_id, + TenantSubscription.id == subscription_id, + ) + .with_for_update() + ) + if row is None: + raise LookupError("商业订阅不存在。") + return row + + def _entitlement( + self, + tenant_id: str, + subscription_id: str, + entitlement_id: str, + ) -> CommercialEntitlement: + row = self.db.scalar( + select(CommercialEntitlement) + .where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.subscription_id == subscription_id, + CommercialEntitlement.id == entitlement_id, + ) + .with_for_update() + ) + if row is None: + raise LookupError("商业权益不存在或不属于该订阅。") + return row + + def _usage_event( + self, + tenant_id: str, + subscription_id: str, + entitlement_id: str, + event_id: str, + ) -> UsageMeterEvent | None: + return self.db.scalar( + select(UsageMeterEvent).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.subscription_id == subscription_id, + UsageMeterEvent.entitlement_id == entitlement_id, + UsageMeterEvent.id == event_id, + ) + ) + + def _cost_event(self, tenant_id: str, event_id: str) -> CommercialCostEvent | None: + return self.db.scalar( + select(CommercialCostEvent) + .where( + CommercialCostEvent.tenant_id == tenant_id, + CommercialCostEvent.id == event_id, + ) + .with_for_update() + ) + + @staticmethod + def _assert_consumable( + subscription: TenantSubscription, + entitlement: CommercialEntitlement, + billing_period: CommercialBillingPeriod, + occurred_at: datetime, + ) -> None: + if subscription.status not in {"trialing", "active"}: + raise CommercialConflictError("订阅当前不可消费新的商业用量。") + subscription_start = _utc(subscription.starts_at) + subscription_end = _optional_utc(subscription.ends_at) + period_start = _utc(billing_period.period_start) + period_end = _utc(billing_period.period_end) + if ( + occurred_at < subscription_start + or (subscription_end is not None and occurred_at >= subscription_end) + or occurred_at < period_start + or occurred_at >= period_end + ): + raise CommercialConflictError("用量发生时间不在当前订阅计费周期内。") + if entitlement.status != "active": + raise CommercialConflictError("权益当前不可消费新的商业用量。") + start = _utc(entitlement.effective_from) + end = _optional_utc(entitlement.effective_to) + if occurred_at < start or (end is not None and occurred_at >= end): + raise CommercialConflictError("用量发生时间不在权益有效期内。") + + @staticmethod + def _validated_cost_occurred_at(value: datetime) -> datetime: + occurred_at = _utc(value) + if occurred_at > datetime.now(UTC) + timedelta(minutes=5): + raise CommercialConfigurationError("不能写入未来发生的平台成本事实。") + return occurred_at + + @staticmethod + def _assert_same_fingerprint(existing: str, incoming: str) -> None: + if existing != incoming: + raise CommercialConflictError("幂等键已被不同商业事件请求占用。") + + @staticmethod + def _assert_cost_reversal_payload( + payload: CommercialCostEventCreate, + original: CommercialCostEvent, + ) -> None: + mismatched = ( + _blank_to_none(payload.subscription_id) != original.subscription_id + or _blank_to_none(payload.usage_event_id) != original.usage_event_id + or payload.cost_category != original.cost_category + or payload.quantity != Decimal(original.quantity) + or payload.unit.strip() != original.unit + or payload.unit_cost != Decimal(original.unit_cost) + or payload.original_currency.upper() != original.original_currency + or payload.reporting_currency.upper() != original.reporting_currency + or payload.fx_rate != Decimal(original.fx_rate) + or _blank_to_none(payload.provider) != original.provider + or _blank_to_none(payload.sku) != original.sku + or _blank_to_none(payload.model_name) != original.model_name + or payload.allocation_key.strip() != original.allocation_key + ) + if mismatched: + raise CommercialConfigurationError("成本冲回参数必须与原始成本事实完全一致。") + + def _usage_by_idempotency( + self, + tenant_id: str, + source_system: str, + idempotency_key: str, + ) -> UsageMeterEvent | None: + return self.db.scalar( + select(UsageMeterEvent).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.source_system == source_system.strip(), + UsageMeterEvent.idempotency_key == idempotency_key.strip(), + ) + ) + + def _cost_by_idempotency( + self, + tenant_id: str, + source_system: str, + idempotency_key: str, + ) -> CommercialCostEvent | None: + return self.db.scalar( + select(CommercialCostEvent).where( + CommercialCostEvent.tenant_id == tenant_id, + CommercialCostEvent.source_system == source_system.strip(), + CommercialCostEvent.idempotency_key == idempotency_key.strip(), + ) + ) + + def _flush_usage_idempotently( + self, + row: UsageMeterEvent, + fingerprint: str, + ) -> tuple[UsageMeterEvent, bool]: + try: + with self.db.begin_nested(): + self.db.add(row) + self.db.flush() + return row, True + except IntegrityError: + existing = self._usage_by_idempotency( + row.tenant_id, + row.source_system, + row.idempotency_key, + ) + if existing is None: + raise + self._assert_same_fingerprint(existing.request_fingerprint, fingerprint) + return existing, False + + def _flush_cost_idempotently( + self, + row: CommercialCostEvent, + fingerprint: str, + ) -> tuple[CommercialCostEvent, bool]: + try: + with self.db.begin_nested(): + self.db.add(row) + self.db.flush() + return row, True + except IntegrityError: + existing = self._cost_by_idempotency( + row.tenant_id, + row.source_system, + row.idempotency_key, + ) + if existing is None: + raise + self._assert_same_fingerprint(existing.request_fingerprint, fingerprint) + return existing, False + + +def _fingerprint(tenant_id: str, payload: dict[str, Any]) -> str: + canonical = json.dumps( + {"tenant_id": tenant_id, "payload": payload}, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + default=str, + ) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + + +def _optional_utc(value: datetime | None) -> datetime | None: + return _utc(value) if value is not None else None + + +def _blank_to_none(value: str | None) -> str | None: + normalized = str(value or "").strip() + return normalized or None + + +def _required(value: str | None, field_name: str) -> str: + normalized = str(value or "").strip() + if not normalized: + raise CommercialConfigurationError(f"{field_name}不能为空。") + return normalized diff --git a/server/src/app/services/commercial_periods.py b/server/src/app/services/commercial_periods.py new file mode 100644 index 0000000..1f8d96a --- /dev/null +++ b/server/src/app/services/commercial_periods.py @@ -0,0 +1,73 @@ +"""商业账期边界与配额重置周期键。""" + +from __future__ import annotations + +import calendar +import hashlib +from datetime import UTC, datetime + +from app.models.commercial import CommercialEntitlement, TenantSubscription + + +def quota_period_key_for( + entitlement: CommercialEntitlement, + subscription: TenantSubscription, + occurred_at: datetime, +) -> str: + occurred = _utc(occurred_at) + if entitlement.reset_interval == "monthly": + return occurred.strftime("%Y-%m") + if entitlement.reset_interval == "quarterly": + return f"{occurred.year}-Q{((occurred.month - 1) // 3) + 1}" + if entitlement.reset_interval == "annual": + return str(occurred.year) + if entitlement.reset_interval == "contract": + return f"contract-{_utc(subscription.starts_at).date().isoformat()}" + return "all-time" + + +def period_key_for( + entitlement: CommercialEntitlement, + subscription: TenantSubscription, + occurred_at: datetime, +) -> str: + """兼容旧调用方；该键只表示权益重置周期，不再表示账期。""" + + return quota_period_key_for(entitlement, subscription, occurred_at) + + +def billing_period_key( + tenant_id: str, + subscription_id: str, + period_start: datetime, + period_end: datetime, +) -> str: + canonical = ":".join( + ( + tenant_id.strip(), + subscription_id.strip(), + _utc(period_start).isoformat(), + _utc(period_end).isoformat(), + ) + ) + return f"bp-{hashlib.sha256(canonical.encode('utf-8')).hexdigest()[:40]}" + + +def add_billing_interval(value: datetime, billing_interval: str) -> datetime: + months = {"monthly": 1, "quarterly": 3, "annual": 12}.get(billing_interval) + if months is None: + raise ValueError("合同制账期不能自动推导下一周期，必须显式续签合同。") + current = _utc(value) + month_index = current.year * 12 + current.month - 1 + months + target_year, zero_based_month = divmod(month_index, 12) + target_month = zero_based_month + 1 + current_last_day = calendar.monthrange(current.year, current.month)[1] + target_last_day = calendar.monthrange(target_year, target_month)[1] + target_day = ( + target_last_day if current.day == current_last_day else min(current.day, target_last_day) + ) + return current.replace(year=target_year, month=target_month, day=target_day) + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_pricing.py b/server/src/app/services/commercial_pricing.py new file mode 100644 index 0000000..0eb901f --- /dev/null +++ b/server/src/app/services/commercial_pricing.py @@ -0,0 +1,168 @@ +"""基于真实成本与财务确认价值计算商业定价可行区间。""" + +from __future__ import annotations + +from decimal import ROUND_HALF_UP, Decimal + +from sqlalchemy.orm import Session + +from app.schemas.commercial import ( + CommercialPricingCurrencyScenarioRead, + CommercialPricingScenarioRead, + CommercialPricingScenarioWrite, +) +from app.services.commercial_analytics import CommercialAnalyticsService + +FOUR_PLACES = Decimal("0.0001") +SIX_PLACES = Decimal("0.000001") + + +class CommercialPricingService: + """输出价格走廊，不写合同，也不把风险暴露或预计节省当价值。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def build( + self, + tenant_id: str, + payload: CommercialPricingScenarioWrite, + ) -> CommercialPricingScenarioRead: + analytics = CommercialAnalyticsService(self.db).build( + tenant_id, + start=payload.start, + end=payload.end, + as_of=payload.as_of, + ) + costs = {item.currency: Decimal(item.amount) for item in analytics.internal_costs.values} + savings = { + item.currency: Decimal(item.amount) + for item in analytics.verified_cash_savings.values + } + currencies = sorted(set(costs) | set(savings)) + scenarios = [ + self._currency_scenario( + currency, + cost=costs.get(currency), + saving=savings.get(currency), + target_margin=payload.target_contribution_margin_rate, + max_value_share=payload.max_verified_savings_share, + ) + for currency in currencies + ] + recommended_model = self._recommended_model(scenarios) + if not scenarios: + evidence_status = "unavailable" + elif all(item.status == "feasible" for item in scenarios): + evidence_status = "complete" + else: + evidence_status = "partial" + notes = [ + "价格下限仅使用内部成本事实；价值上限仅使用独立财务确认、canonical、已计冲回的现金节省。", + "风险暴露、预计机会、未确认结果和工时估值不进入本定价走廊。", + "输出是合同谈判边界，不会自动创建、激活或修改客户套餐。", + ] + notes.extend(analytics.data_quality_issues) + return CommercialPricingScenarioRead( + tenant_id=tenant_id, + start=analytics.start, + end=analytics.end, + as_of=analytics.as_of, + target_contribution_margin_rate=payload.target_contribution_margin_rate, + max_verified_savings_share=payload.max_verified_savings_share, + recommended_model=recommended_model, + scenarios=scenarios, + evidence_status=evidence_status, + notes=list(dict.fromkeys(notes)), + ) + + @staticmethod + def _currency_scenario( + currency: str, + *, + cost: Decimal | None, + saving: Decimal | None, + target_margin: Decimal, + max_value_share: Decimal, + ) -> CommercialPricingCurrencyScenarioRead: + normalized_cost = max(Decimal("0"), cost) if cost is not None else None + normalized_saving = max(Decimal("0"), saving) if saving is not None else None + floor = ( + _money(normalized_cost / (Decimal("1") - target_margin)) + if normalized_cost is not None + else None + ) + ceiling = ( + _money(normalized_saving * max_value_share) + if normalized_saving is not None + else None + ) + if floor is None and ceiling is None: + return CommercialPricingCurrencyScenarioRead( + currency=currency, + status="unavailable", + reason="缺少同币种内部成本与财务确认现金节省。", + ) + if floor is not None and ceiling is None: + return CommercialPricingCurrencyScenarioRead( + currency=currency, + status="cost_only", + internal_cost=_money(normalized_cost), + minimum_sustainable_charge=floor, + reason="已有成本下限，但缺少财务确认价值，暂不能计算价值定价上限。", + ) + if floor is None: + return CommercialPricingCurrencyScenarioRead( + currency=currency, + status="unavailable", + verified_cash_savings=_money(normalized_saving), + maximum_value_aligned_charge=ceiling, + reason="已有客户价值，但缺少平台内部成本，不能给出可持续报价。", + ) + customer_roi = ( + _ratio((normalized_saving - floor) / floor) + if normalized_saving is not None and floor > 0 + else None + ) + margin_at_ceiling = ( + _ratio((ceiling - normalized_cost) / ceiling) + if ceiling is not None and ceiling > 0 and normalized_cost is not None + else None + ) + feasible = ceiling is not None and floor <= ceiling + return CommercialPricingCurrencyScenarioRead( + currency=currency, + status="feasible" if feasible else "insufficient_value", + internal_cost=_money(normalized_cost), + verified_cash_savings=_money(normalized_saving), + minimum_sustainable_charge=floor, + maximum_value_aligned_charge=ceiling, + maximum_success_fee=_money(max(Decimal("0"), ceiling - floor)), + customer_roi_at_minimum_charge=customer_roi, + contribution_margin_at_value_ceiling=margin_at_ceiling, + reason=( + "成本可持续下限低于客户价值上限，可采用基础订阅费加封顶成功费。" + if feasible + else "成本下限高于客户价值上限，应先降低交付成本或提高可验证价值，不能强行报价。" + ), + ) + + @staticmethod + def _recommended_model( + scenarios: list[CommercialPricingCurrencyScenarioRead], + ) -> str: + if any(item.status == "insufficient_value" for item in scenarios): + return "optimize_unit_economics" + if any(item.status == "feasible" for item in scenarios): + return "hybrid" + if any(item.status == "cost_only" for item in scenarios): + return "subscription" + return "pilot_collecting" + + +def _money(value: Decimal | None) -> Decimal | None: + return value.quantize(FOUR_PLACES, rounding=ROUND_HALF_UP) if value is not None else None + + +def _ratio(value: Decimal) -> Decimal: + return value.quantize(SIX_PLACES, rounding=ROUND_HALF_UP) diff --git a/server/src/app/services/commercial_queries.py b/server/src/app/services/commercial_queries.py new file mode 100644 index 0000000..cc29aa9 --- /dev/null +++ b/server/src/app/services/commercial_queries.py @@ -0,0 +1,259 @@ +"""平台商业配置与事实账本的租户安全查询。""" + +from __future__ import annotations + +from datetime import datetime + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.commercial import ( + CommercialCostEvent, + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) +from app.models.commercial_billing import CommercialAdminEvent, CommercialBillingPeriod +from app.services.commercial_access_policy import CommercialAccessPolicy + + +class CommercialQueryService: + def __init__(self, db: Session) -> None: + self.db = db + + def list_plans( + self, + tenant_id: str, + *, + status: str | None = None, + limit: int = 100, + offset: int = 0, + ) -> list[TenantCommercialPlan]: + tenant = CommercialAccessPolicy.normalized_tenant_id(tenant_id) + statement = select(TenantCommercialPlan).where(TenantCommercialPlan.tenant_id == tenant) + if status: + statement = statement.where(TenantCommercialPlan.status == status) + return list( + self.db.scalars( + statement.order_by( + TenantCommercialPlan.created_at.desc(), + TenantCommercialPlan.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + def list_subscriptions( + self, + tenant_id: str, + *, + status: str | None = None, + limit: int = 100, + offset: int = 0, + ) -> list[TenantSubscription]: + tenant = CommercialAccessPolicy.normalized_tenant_id(tenant_id) + statement = select(TenantSubscription).where(TenantSubscription.tenant_id == tenant) + if status: + statement = statement.where(TenantSubscription.status == status) + return list( + self.db.scalars( + statement.order_by( + TenantSubscription.created_at.desc(), + TenantSubscription.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + def list_entitlements( + self, + tenant_id: str, + *, + subscription_id: str | None = None, + billing_period_id: str | None = None, + status: str | None = None, + limit: int = 200, + offset: int = 0, + ) -> list[CommercialEntitlement]: + tenant = CommercialAccessPolicy.normalized_tenant_id(tenant_id) + statement = select(CommercialEntitlement).where(CommercialEntitlement.tenant_id == tenant) + if subscription_id: + statement = statement.where(CommercialEntitlement.subscription_id == subscription_id) + if billing_period_id: + period = self._billing_period(tenant, billing_period_id) + if period is None: + raise LookupError("商业账期不存在。") + if subscription_id and period.subscription_id != subscription_id: + return [] + statement = statement.where( + CommercialEntitlement.subscription_id == period.subscription_id, + CommercialEntitlement.effective_from < period.period_end, + ( + CommercialEntitlement.effective_to.is_(None) + | (CommercialEntitlement.effective_to > period.period_start) + ), + ) + if status: + statement = statement.where(CommercialEntitlement.status == status) + return list( + self.db.scalars( + statement.order_by( + CommercialEntitlement.created_at.desc(), + CommercialEntitlement.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + def list_usage_events( + self, + tenant_id: str, + *, + subscription_id: str | None = None, + billing_period_id: str | None = None, + start: datetime | None = None, + end: datetime | None = None, + limit: int = 200, + offset: int = 0, + ) -> list[UsageMeterEvent]: + tenant = CommercialAccessPolicy.normalized_tenant_id(tenant_id) + statement = select(UsageMeterEvent).where(UsageMeterEvent.tenant_id == tenant) + if subscription_id: + statement = statement.where(UsageMeterEvent.subscription_id == subscription_id) + if billing_period_id: + statement = statement.where(UsageMeterEvent.billing_period_id == billing_period_id) + statement = _window(statement, UsageMeterEvent.occurred_at, start=start, end=end) + return list( + self.db.scalars( + statement.order_by( + UsageMeterEvent.occurred_at.desc(), + UsageMeterEvent.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + def list_cost_events( + self, + tenant_id: str, + *, + subscription_id: str | None = None, + billing_period_id: str | None = None, + start: datetime | None = None, + end: datetime | None = None, + limit: int = 200, + offset: int = 0, + ) -> list[CommercialCostEvent]: + tenant = CommercialAccessPolicy.normalized_tenant_id(tenant_id) + statement = select(CommercialCostEvent).where(CommercialCostEvent.tenant_id == tenant) + if subscription_id: + statement = statement.where(CommercialCostEvent.subscription_id == subscription_id) + if billing_period_id: + statement = statement.where(CommercialCostEvent.billing_period_id == billing_period_id) + statement = _window(statement, CommercialCostEvent.occurred_at, start=start, end=end) + return list( + self.db.scalars( + statement.order_by( + CommercialCostEvent.occurred_at.desc(), + CommercialCostEvent.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + def list_billing_periods( + self, + tenant_id: str, + *, + subscription_id: str | None = None, + limit: int = 200, + offset: int = 0, + ) -> list[CommercialBillingPeriod]: + tenant = CommercialAccessPolicy.normalized_tenant_id(tenant_id) + statement = select(CommercialBillingPeriod).where( + CommercialBillingPeriod.tenant_id == tenant + ) + if subscription_id: + statement = statement.where(CommercialBillingPeriod.subscription_id == subscription_id) + return list( + self.db.scalars( + statement.order_by( + CommercialBillingPeriod.period_start.desc(), + CommercialBillingPeriod.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + def list_admin_events( + self, + tenant_id: str, + *, + action: str | None = None, + resource_type: str | None = None, + resource_id: str | None = None, + start: datetime | None = None, + end: datetime | None = None, + limit: int = 200, + offset: int = 0, + ) -> list[CommercialAdminEvent]: + tenant = CommercialAccessPolicy.normalized_tenant_id(tenant_id) + statement = select(CommercialAdminEvent).where(CommercialAdminEvent.tenant_id == tenant) + if action: + statement = statement.where(CommercialAdminEvent.action == action) + if resource_type: + statement = statement.where(CommercialAdminEvent.resource_type == resource_type) + if resource_id: + statement = statement.where(CommercialAdminEvent.resource_id == resource_id) + statement = _window( + statement, + CommercialAdminEvent.occurred_at, + start=start, + end=end, + ) + return list( + self.db.scalars( + statement.order_by( + CommercialAdminEvent.occurred_at.desc(), + CommercialAdminEvent.id.desc(), + ) + .offset(offset) + .limit(limit) + ).all() + ) + + def _billing_period( + self, + tenant_id: str, + billing_period_id: str, + ) -> CommercialBillingPeriod | None: + return self.db.scalar( + select(CommercialBillingPeriod).where( + CommercialBillingPeriod.tenant_id == tenant_id, + CommercialBillingPeriod.id == billing_period_id, + ) + ) + + +def _window(statement, column, *, start: datetime | None, end: datetime | None): + if start is not None: + _require_aware(start) + statement = statement.where(column >= start) + if end is not None: + _require_aware(end) + statement = statement.where(column < end) + if start is not None and end is not None and end <= start: + raise ValueError("商业事实查询结束时间必须晚于开始时间。") + return statement + + +def _require_aware(value: datetime) -> None: + if value.tzinfo is None or value.utcoffset() is None: + raise ValueError("商业事实查询时间必须显式包含时区。") diff --git a/server/src/app/services/commercial_rollover_scheduler.py b/server/src/app/services/commercial_rollover_scheduler.py new file mode 100644 index 0000000..e7015d7 --- /dev/null +++ b/server/src/app/services/commercial_rollover_scheduler.py @@ -0,0 +1,185 @@ +"""商业自动续期后台调度；PostgreSQL advisory lock 选举单一执行者。""" + +from __future__ import annotations + +import os +import threading +from collections.abc import Callable +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import select, text +from sqlalchemy.orm import Session + +from app.core.logging import get_logger +from app.db.session import get_session_factory +from app.models.commercial import TenantSubscription +from app.services.commercial_subscription_rollover import ( + CommercialSubscriptionRolloverService, +) + +logger = get_logger("app.services.commercial_rollover_scheduler") +_LEASE_KEY = "x-financial:commercial-rollover-scheduler:v1" + + +class CommercialRolloverScheduler: + def __init__( + self, + *, + session_factory: Callable[[], Session] | None = None, + ) -> None: + self._interval_seconds = max( + 30, + _env_int("X_FINANCIAL_COMMERCIAL_ROLLOVER_INTERVAL_SECONDS", 300), + ) + self._initial_delay_seconds = max( + 1, + _env_int("X_FINANCIAL_COMMERCIAL_ROLLOVER_INITIAL_DELAY_SECONDS", 20), + ) + self._batch_size = min( + 500, + max(1, _env_int("X_FINANCIAL_COMMERCIAL_ROLLOVER_BATCH_SIZE", 100)), + ) + self._session_factory = session_factory + self._stop_event = threading.Event() + self._thread: threading.Thread | None = None + self._lock = threading.Lock() + + def start(self) -> None: + with self._lock: + if self._thread is not None and self._thread.is_alive(): + return + self._stop_event.clear() + self._thread = threading.Thread( + target=self._run_loop, + name="commercial-rollover-scheduler", + daemon=True, + ) + self._thread.start() + logger.info("Commercial rollover scheduler started interval=%ss", self._interval_seconds) + + def shutdown(self) -> None: + with self._lock: + thread = self._thread + self._thread = None + self._stop_event.set() + if thread is not None and thread.is_alive(): + thread.join(timeout=3) + logger.info("Commercial rollover scheduler stopped") + + def _run_loop(self) -> None: + if self._stop_event.wait(self._initial_delay_seconds): + return + while not self._stop_event.is_set(): + try: + self._run_once() + except Exception: # pragma: no cover - 后台循环最后防线 + logger.exception("Scheduled commercial subscription rollover failed") + if self._stop_event.wait(self._interval_seconds): + break + + def _run_once(self, *, as_of: datetime | None = None) -> dict[str, Any]: + when = _utc(as_of or datetime.now(UTC)) + factory = self._session_factory or get_session_factory() + db = factory() + lease_acquired = False + summary: dict[str, Any] = { + "scanned": 0, + "rolled_over": 0, + "periods_created": 0, + "ineligible": 0, + "errors": [], + "leader_skipped": 0, + } + try: + lease_acquired = self._try_acquire_lease(db) + if not lease_acquired: + summary["leader_skipped"] = 1 + return summary + candidates = list( + db.execute( + select(TenantSubscription.tenant_id, TenantSubscription.id) + .where( + TenantSubscription.status.in_(("trialing", "active")), + TenantSubscription.auto_renew.is_(True), + TenantSubscription.current_period_end <= when, + ) + .order_by( + TenantSubscription.current_period_end, + TenantSubscription.id, + ) + .limit(self._batch_size) + ).all() + ) + summary["scanned"] = len(candidates) + for tenant_id, subscription_id in candidates: + try: + result = CommercialSubscriptionRolloverService(db).rollover_due( + tenant_id, + subscription_id, + as_of=when, + ) + db.commit() + if result.status in {"rolled_over", "replayed"}: + summary["rolled_over"] += 1 + summary["periods_created"] += len(result.created_period_ids) + elif result.status == "ineligible": + summary["ineligible"] += 1 + except Exception as error: # 单租户失败不阻断其他租户 + db.rollback() + summary["errors"].append( + {"subscription_id": subscription_id, "error": str(error)} + ) + logger.exception( + "Commercial rollover failed subscription_id=%s", + subscription_id, + ) + if summary["rolled_over"] or summary["errors"]: + logger.info("Commercial rollover cycle summary=%s", summary) + return summary + finally: + if lease_acquired: + try: + self._release_lease(db) + except Exception: # pragma: no cover - 连接故障兜底 + db.rollback() + logger.exception("Failed to release commercial rollover lease") + db.close() + + @staticmethod + def _try_acquire_lease(db: Session) -> bool: + bind = db.get_bind() + if bind is None or bind.dialect.name != "postgresql": + return True + return bool( + db.scalar( + text("SELECT pg_try_advisory_lock(hashtextextended(:lease_key, 0))"), + {"lease_key": _LEASE_KEY}, + ) + ) + + @staticmethod + def _release_lease(db: Session) -> None: + bind = db.get_bind() + if bind is None or bind.dialect.name != "postgresql": + return + released = db.scalar( + text("SELECT pg_advisory_unlock(hashtextextended(:lease_key, 0))"), + {"lease_key": _LEASE_KEY}, + ) + if not released: + logger.warning("Commercial rollover scheduler lease was not owned") + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + + +def _env_int(name: str, default: int) -> int: + try: + return int(str(os.environ.get(name) or default).strip()) + except (TypeError, ValueError, OverflowError): + return default + + +commercial_rollover_scheduler = CommercialRolloverScheduler() diff --git a/server/src/app/services/commercial_runtime_bridge.py b/server/src/app/services/commercial_runtime_bridge.py new file mode 100644 index 0000000..08bf646 --- /dev/null +++ b/server/src/app/services/commercial_runtime_bridge.py @@ -0,0 +1,639 @@ +"""把商业门禁和追加式计量接到真实 Agent 工具调用生命周期。""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import UTC, datetime +from decimal import Decimal + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.core.logging import get_logger +from app.models.agent_run import AgentRun, AgentToolCall +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) +from app.services.commercial_runtime_metering import ( + CommercialRuntimeMeteringService, + RuntimeMeteringResult, + RuntimeQuotaPreflight, + SecurityDecision, +) +from app.services.commercial_runtime_policy import ( + preflight_quantity, + tenant_from_agent_route, + tool_call_business_occurred, + tool_call_metering_state, +) +from app.services.commercial_runtime_registry import ( + ConfiguredRuntimeMeter as _ConfiguredMeter, +) +from app.services.commercial_runtime_registry import matching_runtime_meters +from app.services.commercial_runtime_reservations import ( + CommercialRuntimeReservationService, +) + +logger = get_logger("app.services.commercial_runtime_bridge") + + +@dataclass(frozen=True) +class CommercialRuntimeGate: + enforced: bool + allowed: bool + reason_code: str + reason: str + tenant_id: str | None + preflight: RuntimeQuotaPreflight | None = None + + +@dataclass(frozen=True) +class CommercialRuntimePermit: + gate: CommercialRuntimeGate + tool_call_id: str + reservation_id: str | None = None + + +class CommercialRuntimeBridge: + """未配置时兼容放行；显式配置后执行严格门禁并追加真实事实。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def preflight( + self, + run_id: str, + *, + tool_type: str, + tool_name: str, + requested_quantity: Decimal | int | None = None, + security_decision: SecurityDecision = "allow", + ) -> CommercialRuntimeGate: + run = self.db.scalar(select(AgentRun).where(AgentRun.run_id == str(run_id or "").strip())) + if run is None: + return CommercialRuntimeGate( + enforced=True, + allowed=False, + reason_code="run_not_found", + reason="Agent 运行记录不存在，不能执行商业权益检查。", + tenant_id=None, + ) + tenant_id = tenant_from_agent_route(run.route_json) + if tenant_id is None: + return CommercialRuntimeGate( + enforced=False, + allowed=True, + reason_code="legacy_run_without_tenant", + reason="运行记录没有可信租户归属，按未启用商业管控的兼容路径执行。", + tenant_id=None, + ) + meters = self._matching_meters(tenant_id, tool_type, tool_name) + if not meters: + return CommercialRuntimeGate( + enforced=False, + allowed=True, + reason_code="runtime_meter_not_configured", + reason="该工具尚未显式启用商业运行计量，兼容执行且不生成商业事实。", + tenant_id=tenant_id, + ) + try: + reserved_quantity = ( + requested_quantity + if requested_quantity is not None + else preflight_quantity([item.config for item in meters]) + ) + result = CommercialRuntimeMeteringService(self.db).preflight_run_tool( + run.run_id, + tool_type=tool_type, + tool_name=tool_name, + requested_quantity=reserved_quantity, + security_decision=security_decision, + ) + except CommercialConfigurationError as error: + return CommercialRuntimeGate( + enforced=True, + allowed=False, + reason_code="configuration_error", + reason=str(error), + tenant_id=tenant_id, + ) + return CommercialRuntimeGate( + enforced=True, + allowed=result.allowed, + reason_code=result.reason_code, + reason=result.reason, + tenant_id=tenant_id, + preflight=result, + ) + + def reserve_tool( + self, + run_id: str, + *, + tool_call_id: str, + tool_type: str, + tool_name: str, + requested_quantity: Decimal | int | None = None, + hard_max_confirmed: bool = False, + security_decision: SecurityDecision = "allow", + ) -> CommercialRuntimePermit: + existing = CommercialRuntimeReservationService(self.db).by_tool_call(tool_call_id) + if existing is not None: + return self._replay_reservation( + existing, + run_id=run_id, + tool_call_id=tool_call_id, + tool_type=tool_type, + tool_name=tool_name, + requested_quantity=requested_quantity, + hard_max_confirmed=hard_max_confirmed, + security_decision=security_decision, + ) + gate = self.preflight( + run_id, + tool_type=tool_type, + tool_name=tool_name, + requested_quantity=requested_quantity, + security_decision=security_decision, + ) + if not gate.enforced or not gate.allowed: + self.db.rollback() + return CommercialRuntimePermit(gate=gate, tool_call_id=tool_call_id) + preflight = gate.preflight + if preflight is None or preflight.entitlement_id is None: + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate(gate, "configuration_error", "商业预检缺少权益身份。"), + tool_call_id=tool_call_id, + ) + if preflight.quantity_basis != "call" and not hard_max_confirmed: + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate( + gate, + "hard_max_required", + "变量用量工具必须提供执行器可强制的最大数量后才能预占。", + ), + tool_call_id=tool_call_id, + ) + meter = next( + ( + item + for item in self._matching_meters(gate.tenant_id or "", tool_type, tool_name) + if item.entitlement.id == preflight.entitlement_id + ), + None, + ) + if meter is None: + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate( + gate, + "configuration_changed", + "商业计量配置在预检后发生变化，请重试。", + ), + tool_call_id=tool_call_id, + ) + try: + reservation, _ = CommercialRuntimeReservationService(self.db).reserve( + tenant_id=gate.tenant_id or "", + entitlement_id=preflight.entitlement_id, + run_id=run_id, + tool_call_id=tool_call_id, + tool_type=tool_type, + tool_name=tool_name, + quantity_basis=preflight.quantity_basis or "", + reserved_quantity=preflight.requested_quantity or Decimal("0"), + meter_config=meter.config, + ) + self.db.commit() + except Exception as error: + self.db.rollback() + reason = ( + str(error) + if isinstance( + error, + ( + CommercialConfigurationError, + CommercialConflictError, + LookupError, + ValueError, + ), + ) + else "额度预占失败。" + ) + return CommercialRuntimePermit( + gate=self._denied_gate(gate, "reservation_denied", reason), + tool_call_id=tool_call_id, + ) + return CommercialRuntimePermit( + gate=gate, + tool_call_id=tool_call_id, + reservation_id=reservation.id, + ) + + def _replay_reservation( + self, + existing: CommercialRuntimeReservation, + *, + run_id: str, + tool_call_id: str, + tool_type: str, + tool_name: str, + requested_quantity: Decimal | int | None, + hard_max_confirmed: bool, + security_decision: SecurityDecision, + ) -> CommercialRuntimePermit: + run = self.db.scalar(select(AgentRun).where(AgentRun.run_id == str(run_id).strip())) + tenant_id = tenant_from_agent_route(run.route_json) if run is not None else None + gate = CommercialRuntimeGate( + enforced=True, + allowed=True, + reason_code="reservation_replayed", + reason="相同工具调用已完成额度预占，本次复用原子占位。", + tenant_id=tenant_id, + preflight=RuntimeQuotaPreflight( + allowed=True, + reason_code="reservation_replayed", + reason="相同工具调用已完成额度预占。", + run_id=str(run_id), + tenant_id=tenant_id, + entitlement_id=existing.entitlement_id, + quantity_basis=existing.quantity_basis, + requested_quantity=Decimal(existing.reserved_quantity), + commercially_allowed=True, + security_decision=security_decision, + ), + ) + identity_matches = ( + run is not None + and tenant_id == existing.tenant_id + and str(run_id).strip() == existing.run_id + and str(tool_call_id).strip() == existing.tool_call_id + and str(tool_type).strip() == existing.tool_type + and str(tool_name).strip() == existing.tool_name + ) + if not identity_matches: + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate( + gate, + "reservation_identity_mismatch", + "工具调用编号已属于不同运行身份，拒绝复用额度预占。", + ), + tool_call_id=tool_call_id, + ) + expires_at = existing.expires_at + expires_at = expires_at.replace(tzinfo=UTC) if expires_at.tzinfo is None else expires_at + if existing.status != "reserved" or expires_at <= datetime.now(UTC): + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate( + gate, + "reservation_not_reservable", + "原额度预占已终结或过期，不能再次授权真实工具执行。", + ), + tool_call_id=tool_call_id, + ) + if security_decision != "allow": + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate( + gate, + "security_denied", + "当前安全门禁不允许复用原额度预占。", + ), + tool_call_id=tool_call_id, + ) + if existing.quantity_basis != "call" and not hard_max_confirmed: + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate( + gate, + "hard_max_required", + "变量用量工具重试时仍须确认执行器可强制的最大数量。", + ), + tool_call_id=tool_call_id, + ) + quantity = ( + requested_quantity + if requested_quantity is not None + else preflight_quantity([existing.meter_config_json]) + ) + try: + replayed, _ = CommercialRuntimeReservationService(self.db).reserve( + tenant_id=existing.tenant_id, + entitlement_id=existing.entitlement_id, + run_id=run_id, + tool_call_id=tool_call_id, + tool_type=tool_type, + tool_name=tool_name, + quantity_basis=existing.quantity_basis, + reserved_quantity=Decimal(str(quantity)), + meter_config=existing.meter_config_json, + ) + self.db.commit() + except ( + CommercialConfigurationError, + CommercialConflictError, + LookupError, + ValueError, + ) as error: + self.db.rollback() + return CommercialRuntimePermit( + gate=self._denied_gate(gate, "reservation_replay_conflict", str(error)), + tool_call_id=tool_call_id, + ) + return CommercialRuntimePermit( + gate=gate, + tool_call_id=tool_call_id, + reservation_id=replayed.id, + ) + + def sync_tool_call(self, tool_call_id: str) -> RuntimeMeteringResult: + pair = None + run_id = "" + tenant_id = None + try: + pair = self.db.execute( + select(AgentToolCall, AgentRun) + .join(AgentRun, AgentRun.run_id == AgentToolCall.run_id) + .where(AgentToolCall.id == str(tool_call_id or "").strip()) + ).one_or_none() + if pair is not None: + tool_call, run = pair + tenant_id = tenant_from_agent_route(run.route_json) + run_id = run.run_id + reservation = CommercialRuntimeReservationService(self.db).by_tool_call( + tool_call.id + ) + if reservation is not None: + result = self._sync_reserved(tool_call, run, reservation) + if result.status in {"created", "replayed"} or result.usage_event_id: + self.db.commit() + elif reservation.status in {"released", "expired"}: + self.db.commit() + else: + self.db.rollback() + self._log_result(result) + return result + meters = self._matching_meters( + tenant_id or "", + tool_call.tool_type, + tool_call.tool_name, + ) + if not meters: + result = RuntimeMeteringResult( + status="skipped", + reason_code=( + "legacy_run_without_tenant" + if tenant_id is None + else "runtime_meter_not_configured" + ), + reason=( + "运行记录没有可信租户归属，按兼容路径跳过商业事实。" + if tenant_id is None + else "该工具尚未显式启用商业运行计量。" + ), + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + business_call_occurred=tool_call_business_occurred(tool_call.status), + requires_reconciliation=False, + ) + self.db.rollback() + return result + if tool_call_metering_state(tool_call.status) != "billable": + self.db.rollback() + return self._non_billable_result(tool_call, run, tenant_id) + result = self._record_missing_reservation_backlog( + tool_call, + run, + tenant_id, + meters, + ) + self.db.commit() + self._log_result(result) + return result + result = CommercialRuntimeMeteringService(self.db).sync_tool_call(tool_call_id) + self.db.rollback() + except Exception: # 工具调用已提交，计量失败必须转成可补偿结果。 + self.db.rollback() + result = RuntimeMeteringResult( + status="error", + reason_code="metering_bridge_failed_after_business_call", + reason="运行时计量桥接失败，真实工具调用已保留并等待幂等补偿。", + tool_call_id=str(tool_call_id or ""), + run_id=run_id, + tenant_id=tenant_id, + business_call_occurred=True, + requires_reconciliation=True, + ) + logger.exception( + "Commercial runtime metering bridge failed tool_call_id=%s", + tool_call_id, + ) + self._log_result(result) + return result + + def _sync_reserved( + self, + tool_call: AgentToolCall, + run: AgentRun, + reservation: CommercialRuntimeReservation, + ) -> RuntimeMeteringResult: + state = tool_call_metering_state(tool_call.status) + reservations = CommercialRuntimeReservationService(self.db) + tenant_id = tenant_from_agent_route(run.route_json) + if state == "collecting": + return RuntimeMeteringResult( + status="skipped", + reason_code="tool_call_not_terminal", + reason="工具调用尚未完成，额度预占继续保留。", + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + entitlement_id=reservation.entitlement_id, + quantity_basis=reservation.quantity_basis, + collecting=True, + business_call_occurred=False, + requires_reconciliation=True, + ) + if state != "billable": + if reservation.status == "reserved": + reservations.release( + reservation.id, + reason_code=f"tool_{str(tool_call.status or 'unknown').lower()}", + ) + return self._non_billable_result(tool_call, run, tenant_id) + if reservation.status in {"released", "expired", "reconciliation_required"}: + return RuntimeMeteringResult( + status="error", + reason_code="reservation_not_settleable", + reason=f"额度预占状态 {reservation.status} 不能自动结算。", + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + entitlement_id=reservation.entitlement_id, + quantity_basis=reservation.quantity_basis, + business_call_occurred=True, + requires_reconciliation=True, + ) + if reservation.status == "reserved": + actual = CommercialRuntimeMeteringService.measured_quantity( + tool_call, + reservation.quantity_basis, # type: ignore[arg-type] + ) + reservation = reservations.mark_committed(reservation.id, actual) + result = CommercialRuntimeMeteringService(self.db).sync_reserved_tool_call( + tool_call.id, + reservation, + ) + if result.requires_reconciliation and result.usage_event_id: + reservations.require_committed_reconciliation( + reservation.id, + reason_code=result.reason_code, + ) + elif ( + not result.requires_reconciliation + and reservation.status == "committed_reconciliation_required" + ): + reservations.mark_reconciled(reservation.id) + return result + + def _record_missing_reservation_backlog( + self, + tool_call: AgentToolCall, + run: AgentRun, + tenant_id: str | None, + meters: list[_ConfiguredMeter], + ) -> RuntimeMeteringResult: + gate = self.preflight( + run.run_id, + tool_type=tool_call.tool_type, + tool_name=tool_call.tool_name, + ) + preflight = gate.preflight + if tenant_id is None: + raise CommercialConflictError("真实工具调用缺少可持久化的商业补偿归属。") + meter = ( + next( + ( + item + for item in meters + if preflight is not None + and item.entitlement.id == preflight.entitlement_id + ), + None, + ) + if preflight is not None and preflight.entitlement_id is not None + else (meters[0] if len(meters) == 1 else None) + ) + if meter is None: + raise CommercialConflictError("真实工具调用对应的商业计量配置缺失或存在歧义。") + quantity_basis = ( + preflight.quantity_basis + if preflight is not None and preflight.quantity_basis + else str(meter.config.get("quantity_basis") or "").strip() + ) + try: + actual = CommercialRuntimeMeteringService.measured_quantity( + tool_call, + quantity_basis, # type: ignore[arg-type] + ) + except Exception: + actual = None + requested = preflight.requested_quantity if preflight is not None else None + reserved = actual or requested or preflight_quantity([meter.config]) + backlog, _ = CommercialRuntimeReservationService( + self.db + ).record_reconciliation_backlog( + tenant_id=tenant_id, + entitlement_id=meter.entitlement.id, + run_id=run.run_id, + tool_call_id=tool_call.id, + tool_type=tool_call.tool_type, + tool_name=tool_call.tool_name, + quantity_basis=quantity_basis, + reserved_quantity=reserved, + actual_quantity=actual, + meter_config=meter.config, + resolution_code="missing_pre_execution_reservation", + as_of=tool_call.created_at, + ) + return RuntimeMeteringResult( + status="error", + reason_code="missing_pre_execution_reservation", + reason="真实工具调用没有执行前预占，已持久化补偿并冻结对应额度。", + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + entitlement_id=backlog.entitlement_id, + quantity_basis=backlog.quantity_basis, + quantity=actual, + business_call_occurred=True, + requires_reconciliation=True, + ) + + def _matching_meters( + self, + tenant_id: str, + tool_type: str, + tool_name: str, + ) -> list[_ConfiguredMeter]: + return matching_runtime_meters( + self.db, + tenant_id=tenant_id, + tool_type=tool_type, + tool_name=tool_name, + ) + + @staticmethod + def _denied_gate( + gate: CommercialRuntimeGate, + reason_code: str, + reason: str, + ) -> CommercialRuntimeGate: + return CommercialRuntimeGate( + enforced=True, + allowed=False, + reason_code=reason_code, + reason=reason, + tenant_id=gate.tenant_id, + preflight=gate.preflight, + ) + + @staticmethod + def _non_billable_result( + tool_call: AgentToolCall, + run: AgentRun, + tenant_id: str | None, + ) -> RuntimeMeteringResult: + return RuntimeMeteringResult( + status="skipped", + reason_code="tool_call_not_billable", + reason="只有成功完成的真实工具调用才进入商业用量与成本账本。", + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + business_call_occurred=tool_call_business_occurred(tool_call.status), + requires_reconciliation=False, + ) + + @staticmethod + def _log_result(result: RuntimeMeteringResult) -> None: + if result.requires_reconciliation: + logger.warning( + "Commercial runtime metering requires reconciliation " + "tool_call_id=%s reason_code=%s retry=sync_tool_call", + result.tool_call_id, + result.reason_code, + ) + elif result.status in {"created", "replayed"}: + logger.info( + "Commercial runtime metering persisted tool_call_id=%s status=%s", + result.tool_call_id, + result.status, + ) diff --git a/server/src/app/services/commercial_runtime_costs.py b/server/src/app/services/commercial_runtime_costs.py new file mode 100644 index 0000000..5a750e4 --- /dev/null +++ b/server/src/app/services/commercial_runtime_costs.py @@ -0,0 +1,61 @@ +"""校验并解析 Agent 运行时内部成本配置。""" + +from __future__ import annotations + +from dataclasses import dataclass +from decimal import Decimal +from typing import Any + +from app.services.commercial_access_policy import CommercialConfigurationError +from app.services.commercial_runtime_values import currency, optional_text, positive_decimal + +SUPPORTED_COST_CATEGORIES = { + "ai_inference", + "ocr", + "storage", + "connector", + "support", + "implementation", + "infrastructure", + "payment", + "other", +} + + +@dataclass(frozen=True) +class RuntimeCostConfig: + cost_category: str + unit: str + unit_cost: Decimal + original_currency: str + reporting_currency: str + fx_rate: Decimal + provider: str | None + sku: str | None + model_name: str | None + + +def runtime_cost_config(value: Any, default_unit: str) -> RuntimeCostConfig | None: + if not isinstance(value, dict) or value.get("enabled") is not True: + return None + category = str(value.get("cost_category") or "").strip() + if category not in SUPPORTED_COST_CATEGORIES: + raise CommercialConfigurationError("internal_cost.cost_category 未配置或不受支持。") + unit_cost = positive_decimal(value.get("unit_cost"), "internal_cost.unit_cost") + fx_rate = positive_decimal(value.get("fx_rate"), "internal_cost.fx_rate") + original_currency = currency(value.get("original_currency"), "原始币种") + reporting_currency = currency(value.get("reporting_currency"), "报告币种") + unit = str(value.get("unit") or default_unit or "").strip() + if not unit: + raise CommercialConfigurationError("internal_cost.unit 不能为空。") + return RuntimeCostConfig( + cost_category=category, + unit=unit, + unit_cost=unit_cost, + original_currency=original_currency, + reporting_currency=reporting_currency, + fx_rate=fx_rate, + provider=optional_text(value.get("provider")), + sku=optional_text(value.get("sku")), + model_name=optional_text(value.get("model_name")), + ) diff --git a/server/src/app/services/commercial_runtime_metering.py b/server/src/app/services/commercial_runtime_metering.py new file mode 100644 index 0000000..bd1ce2f --- /dev/null +++ b/server/src/app/services/commercial_runtime_metering.py @@ -0,0 +1,796 @@ +"""将已发生的 Agent 工具调用安全映射为商业用量与内部成本事实。""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +from datetime import UTC, datetime +from decimal import Decimal +from typing import Any, Literal + +from sqlalchemy import and_, or_, select +from sqlalchemy.orm import Session + +from app.models.agent_run import AgentRun, AgentToolCall +from app.models.commercial import CommercialEntitlement, TenantSubscription +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.schemas.commercial import CommercialCostEventCreate, UsageMeterEventCreate +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) +from app.services.commercial_entitlements import CommercialEntitlementService +from app.services.commercial_metering import CommercialMeteringService +from app.services.commercial_runtime_costs import RuntimeCostConfig as _CostConfig +from app.services.commercial_runtime_costs import runtime_cost_config as _cost_config +from app.services.commercial_runtime_policy import ( + runtime_meter_config, + runtime_meter_matches, + tenant_from_agent_route, + tool_call_business_occurred, + tool_call_metering_state, +) +from app.services.commercial_runtime_values import ( + cost_idempotency_key as _cost_idempotency_key, +) +from app.services.commercial_runtime_values import ( + decimal as _decimal, +) +from app.services.commercial_runtime_values import ( + decode_cursor as _decode_cursor, +) +from app.services.commercial_runtime_values import ( + encode_cursor as _encode_cursor, +) +from app.services.commercial_runtime_values import ( + positive_decimal as _positive_decimal, +) +from app.services.commercial_runtime_values import ( + required as _required, +) +from app.services.commercial_runtime_values import ( + safe_error_reason as _safe_error_reason, +) +from app.services.commercial_runtime_values import ( + usage_idempotency_key as _usage_idempotency_key, +) +from app.services.commercial_runtime_values import ( + utc as _utc, +) + +QuantityBasis = Literal[ + "call", + "bytes", + "events", + "input_tokens", + "objects", + "output_tokens", + "pages", + "total_tokens", + "duration_ms", +] +SecurityDecision = Literal["allow", "deny", "human_review"] +ResultStatus = Literal["created", "replayed", "skipped", "error"] + +SOURCE_SYSTEM = "agent-runtime-metering" +METER_VERSION = "agent-tool-call-v1" +MAX_BATCH_SIZE = 200 +SUPPORTED_BASES = { + "call", + "bytes", + "events", + "input_tokens", + "objects", + "output_tokens", + "pages", + "total_tokens", + "duration_ms", +} +INPUT_TOKEN_KEYS = ("input_tokens", "prompt_tokens") +OUTPUT_TOKEN_KEYS = ("output_tokens", "completion_tokens") +TOTAL_TOKEN_KEYS = ("total_tokens",) +TOKEN_CONTAINERS = ("usage", "token_usage", "metrics") +RESOURCE_QUANTITY_KEYS = { + "bytes": ("bytes", "byte_count", "content_length", "size_bytes"), + "events": ("events", "event_count"), + "objects": ("objects", "object_count"), + "pages": ("pages", "page_count", "processed_pages"), +} + + +@dataclass(frozen=True) +class RuntimeQuotaPreflight: + allowed: bool + reason_code: str + reason: str + run_id: str + tenant_id: str | None + entitlement_id: str | None + quantity_basis: str | None + requested_quantity: Decimal | None + commercially_allowed: bool + security_decision: SecurityDecision + + +@dataclass(frozen=True) +class RuntimeMeteringResult: + status: ResultStatus + reason_code: str + reason: str + tool_call_id: str + run_id: str + tenant_id: str | None + entitlement_id: str | None = None + quantity_basis: str | None = None + quantity: Decimal | None = None + usage_event_id: str | None = None + cost_event_id: str | None = None + usage_created: bool = False + cost_created: bool = False + collecting: bool = False + business_call_occurred: bool = True + requires_reconciliation: bool = False + + def to_dict(self) -> dict[str, Any]: + payload = asdict(self) + if self.quantity is not None: + payload["quantity"] = str(self.quantity) + payload.update( + created=self.status == "created", + replayed=self.status == "replayed", + skipped=self.status == "skipped", + error=self.status == "error", + ) + return payload + + +@dataclass(frozen=True) +class RuntimeMeteringBatchResult: + items: tuple[RuntimeMeteringResult, ...] + created: int + replayed: int + skipped: int + errors: int + limit: int + has_more: bool + next_cursor: str | None + + def to_dict(self) -> dict[str, Any]: + return { + "items": [item.to_dict() for item in self.items], + "created": self.created, + "replayed": self.replayed, + "skipped": self.skipped, + "errors": self.errors, + "limit": self.limit, + "has_more": self.has_more, + "next_cursor": self.next_cursor, + } + + +@dataclass(frozen=True) +class _ResolvedMeter: + subscription: TenantSubscription + entitlement: CommercialEntitlement + quantity_basis: QuantityBasis + cost: _CostConfig | None + + +class _SkipMetering(RuntimeError): + def __init__(self, code: str, reason: str, *, collecting: bool = False) -> None: + super().__init__(reason) + self.code = code + self.reason = reason + self.collecting = collecting + + +class CommercialRuntimeMeteringService: + def __init__(self, db: Session) -> None: + self.db = db + + def preflight_run_tool( + self, + run_id: str, + *, + tool_type: str, + tool_name: str, + requested_quantity: Decimal | int | None = None, + security_decision: SecurityDecision = "allow", + as_of: datetime | None = None, + ) -> RuntimeQuotaPreflight: + run = self.db.scalar(select(AgentRun).where(AgentRun.run_id == _required(run_id))) + if run is None: + return RuntimeQuotaPreflight( + allowed=False, + reason_code="run_not_found", + reason="Agent 运行记录不存在，不能建立可信租户归属。", + run_id=str(run_id or ""), + tenant_id=None, + entitlement_id=None, + quantity_basis=None, + requested_quantity=None, + commercially_allowed=False, + security_decision=security_decision, + ) + tenant_id = tenant_from_agent_route(run.route_json) + if tenant_id is None: + return self._preflight_denied( + run.run_id, + None, + "missing_tenant", + "AgentRun.route_json 未显式提供可信 tenant_id。", + security_decision, + ) + when = _utc(as_of or datetime.now(UTC)) + try: + meter = self._resolve_meter(tenant_id, tool_type, tool_name, when) + quantity = ( + Decimal("1") + if requested_quantity is None and meter.quantity_basis == "call" + else _positive_decimal(requested_quantity, "调用前预留数量") + ) + gate = CommercialEntitlementService(self.db).check( + tenant_id, + entitlement_key=meter.entitlement.entitlement_key, + requested_quantity=quantity, + security_decision=security_decision, + as_of=when, + ) + except _SkipMetering as error: + return self._preflight_denied( + run.run_id, + tenant_id, + error.code, + error.reason, + security_decision, + ) + except (CommercialConfigurationError, CommercialConflictError, ValueError) as error: + return self._preflight_denied( + run.run_id, + tenant_id, + "configuration_error", + _safe_error_reason(error), + security_decision, + ) + return RuntimeQuotaPreflight( + allowed=gate.final_allowed, + reason_code="allowed" if gate.final_allowed else "quota_or_security_denied", + reason=gate.reason, + run_id=run.run_id, + tenant_id=tenant_id, + entitlement_id=meter.entitlement.id, + quantity_basis=meter.quantity_basis, + requested_quantity=quantity, + commercially_allowed=gate.commercial_allowed, + security_decision=security_decision, + ) + + def assert_run_tool_allowed(self, *args: Any, **kwargs: Any) -> RuntimeQuotaPreflight: + result = self.preflight_run_tool(*args, **kwargs) + if not result.allowed: + raise CommercialConflictError(result.reason) + return result + + def sync_tool_call(self, tool_call_id: str) -> RuntimeMeteringResult: + pair = self._tool_call_pair(tool_call_id) + if pair is None: + return RuntimeMeteringResult( + status="error", + reason_code="tool_call_not_found", + reason="Agent 工具调用记录不存在。", + tool_call_id=str(tool_call_id or ""), + run_id="", + tenant_id=None, + business_call_occurred=False, + requires_reconciliation=False, + ) + return self._guarded_sync(*pair) + + def sync_reserved_tool_call( + self, + tool_call_id: str, + reservation: CommercialRuntimeReservation, + ) -> RuntimeMeteringResult: + pair = self._tool_call_pair(tool_call_id) + if pair is None: + return RuntimeMeteringResult( + status="error", + reason_code="tool_call_not_found", + reason="Agent 工具调用记录不存在。", + tool_call_id=str(tool_call_id or ""), + run_id="", + tenant_id=reservation.tenant_id, + business_call_occurred=False, + requires_reconciliation=False, + ) + if reservation.tool_call_id != pair[0].id or reservation.run_id != pair[1].run_id: + return RuntimeMeteringResult( + status="error", + reason_code="reservation_identity_mismatch", + reason="商业预占与 Agent 工具调用身份不一致。", + tool_call_id=pair[0].id, + run_id=pair[1].run_id, + tenant_id=reservation.tenant_id, + business_call_occurred=True, + requires_reconciliation=True, + ) + return self._guarded_sync(*pair, reservation=reservation) + + def sync_batch( + self, + *, + limit: int = 100, + cursor: str | None = None, + ) -> RuntimeMeteringBatchResult: + """按 created_at + id 稳定游标同步一页，单页硬上限 200。""" + + if ( + not isinstance(limit, int) + or isinstance(limit, bool) + or not 1 <= limit <= MAX_BATCH_SIZE + ): + raise ValueError(f"运行时计量批次大小必须介于 1 和 {MAX_BATCH_SIZE} 之间。") + cursor_value = _decode_cursor(cursor) if cursor else None + statement = ( + select(AgentToolCall, AgentRun) + .join(AgentRun, AgentRun.run_id == AgentToolCall.run_id) + .order_by(AgentToolCall.created_at.asc(), AgentToolCall.id.asc()) + ) + if cursor_value is not None: + created_at, call_id = cursor_value + statement = statement.where( + or_( + AgentToolCall.created_at > created_at, + and_( + AgentToolCall.created_at == created_at, + AgentToolCall.id > call_id, + ), + ) + ) + rows = list(self.db.execute(statement.limit(limit + 1)).all()) + has_more = len(rows) > limit + page = rows[:limit] + items = tuple(self._guarded_sync(tool_call, run) for tool_call, run in page) + next_cursor = _encode_cursor(page[-1][0].created_at, page[-1][0].id) if page else None + return RuntimeMeteringBatchResult( + items=items, + created=sum(item.status == "created" for item in items), + replayed=sum(item.status == "replayed" for item in items), + skipped=sum(item.status == "skipped" for item in items), + errors=sum(item.status == "error" for item in items), + limit=limit, + has_more=has_more, + next_cursor=next_cursor, + ) + + def _guarded_sync( + self, + tool_call: AgentToolCall, + run: AgentRun, + *, + reservation: CommercialRuntimeReservation | None = None, + ) -> RuntimeMeteringResult: + tenant_id = tenant_from_agent_route(run.route_json) + try: + with self.db.begin_nested(): + return self._sync_pair(tool_call, run, tenant_id, reservation=reservation) + except Exception as error: # 错误必须结构化返回，不能把已发生调用伪装为未发生。 + return RuntimeMeteringResult( + status="error", + reason_code="metering_failed_after_business_call", + reason=_safe_error_reason(error), + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + business_call_occurred=True, + requires_reconciliation=True, + ) + + def _sync_pair( + self, + tool_call: AgentToolCall, + run: AgentRun, + tenant_id: str | None, + *, + reservation: CommercialRuntimeReservation | None = None, + ) -> RuntimeMeteringResult: + metering_state = tool_call_metering_state(tool_call.status) + if metering_state != "billable": + return self._skipped( + tool_call, + run, + tenant_id, + ( + "tool_call_not_terminal" + if metering_state == "collecting" + else "tool_call_not_billable" + ), + "工具调用尚未完成，等待终态后再计量。" + if metering_state == "collecting" + else "只有成功完成的真实工具调用才进入商业用量与成本账本。", + collecting=metering_state == "collecting", + reconcile=metering_state == "collecting", + business_call_occurred=tool_call_business_occurred(tool_call.status), + ) + if tenant_id is None: + return self._skipped( + tool_call, + run, + None, + "missing_tenant", + "AgentRun.route_json 未显式提供可信 tenant_id，调用已发生但不能归属计量。", + reconcile=True, + ) + occurred_at = _utc( + reservation.created_at if reservation is not None else tool_call.created_at + ) + try: + meter = ( + self._reserved_meter(reservation, tenant_id) + if reservation is not None + else self._resolve_meter( + tenant_id, + tool_call.tool_type, + tool_call.tool_name, + occurred_at, + ) + ) + quantity = self.measured_quantity(tool_call, meter.quantity_basis) + if ( + reservation is not None + and reservation.actual_quantity is not None + and quantity != Decimal(reservation.actual_quantity) + ): + raise CommercialConflictError("商业预占结算数量与工具调用事实不一致。") + except _SkipMetering as error: + return self._skipped( + tool_call, + run, + tenant_id, + error.code, + error.reason, + collecting=error.collecting, + reconcile=True, + ) + + metadata = { + "meter_version": METER_VERSION, + "tool_call_id": tool_call.id, + "tool_type": tool_call.tool_type, + "tool_name": tool_call.tool_name, + "tool_status": tool_call.status, + "quantity_basis": meter.quantity_basis, + "entitlement_key": meter.entitlement.entitlement_key, + } + if reservation is not None: + metadata["reservation_id"] = reservation.id + usage_payload = UsageMeterEventCreate( + subscription_id=meter.subscription.id, + entitlement_id=meter.entitlement.id, + quantity=quantity, + occurred_at=occurred_at, + source_system=SOURCE_SYSTEM, + idempotency_key=_usage_idempotency_key(tool_call.id, meter.entitlement.id), + subject_type="agent_tool_call", + subject_id=tool_call.id, + correlation_id=run.run_id, + metadata_json=metadata, + ) + usage_event, usage_created = CommercialMeteringService(self.db).record_usage( + tenant_id, + usage_payload, + actor_type="system", + actor_id="commercial-runtime-meter", + ) + cost_event = None + cost_created = False + if meter.cost is not None: + try: + with self.db.begin_nested(): + cost_event, cost_created = CommercialMeteringService(self.db).record_cost( + tenant_id, + self._cost_payload( + tool_call, + run, + meter, + quantity, + usage_event.id, + occurred_at, + metadata, + ), + ) + except Exception as error: + return RuntimeMeteringResult( + status="error", + reason_code="cost_metering_failed", + reason=_safe_error_reason(error), + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + entitlement_id=meter.entitlement.id, + quantity_basis=meter.quantity_basis, + quantity=quantity, + usage_event_id=usage_event.id, + usage_created=usage_created, + business_call_occurred=True, + requires_reconciliation=True, + ) + created = usage_created or cost_created + return RuntimeMeteringResult( + status="created" if created else "replayed", + reason_code="metered" if created else "idempotent_replay", + reason=( + "Agent 工具调用已追加写入商业用量与成本事实。" + if created + else "相同 Agent 工具调用已计量，本次返回原事实。" + ), + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + entitlement_id=meter.entitlement.id, + quantity_basis=meter.quantity_basis, + quantity=quantity, + usage_event_id=usage_event.id, + cost_event_id=cost_event.id if cost_event is not None else None, + usage_created=usage_created, + cost_created=cost_created, + business_call_occurred=True, + requires_reconciliation=False, + ) + + def _resolve_meter( + self, + tenant_id: str, + tool_type: str, + tool_name: str, + occurred_at: datetime, + ) -> _ResolvedMeter: + subscriptions = list( + self.db.scalars( + select(TenantSubscription).where( + TenantSubscription.tenant_id == tenant_id, + TenantSubscription.status.in_(("active", "trialing")), + ) + ).all() + ) + subscriptions = [row for row in subscriptions if _subscription_active_at(row, occurred_at)] + if not subscriptions: + raise _SkipMetering( + "no_active_subscription", + "工具调用已发生，但租户没有覆盖该发生时间的 active/trialing 订阅。", + ) + if len(subscriptions) > 1: + raise CommercialConfigurationError("租户存在多个可消费订阅，拒绝重复归属计量。") + subscription = subscriptions[0] + entitlements = list( + self.db.scalars( + select(CommercialEntitlement).where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.subscription_id == subscription.id, + CommercialEntitlement.status == "active", + ) + ).all() + ) + matches: list[tuple[CommercialEntitlement, dict[str, Any]]] = [] + for entitlement in entitlements: + if not _entitlement_active_at(entitlement, occurred_at): + continue + runtime = runtime_meter_config(entitlement.config_json) + if runtime is not None and runtime_meter_matches(runtime, tool_type, tool_name): + matches.append((entitlement, runtime)) + if not matches: + raise _SkipMetering( + "no_matching_runtime_meter", + "没有 active 权益显式启用并匹配该工具类型与名称。", + ) + if len(matches) > 1: + raise CommercialConfigurationError("多个 active 权益匹配同一工具，拒绝重复计量。") + entitlement, runtime = matches[0] + basis = str(runtime.get("quantity_basis") or "").strip() + if basis not in SUPPORTED_BASES: + raise CommercialConfigurationError("runtime_meter.quantity_basis 未配置或不受支持。") + cost = _cost_config(runtime.get("internal_cost"), entitlement.unit) + return _ResolvedMeter( + subscription=subscription, + entitlement=entitlement, + quantity_basis=basis, # type: ignore[arg-type] + cost=cost, + ) + + @staticmethod + def measured_quantity(tool_call: AgentToolCall, basis: QuantityBasis) -> Decimal: + if basis == "call": + return Decimal("1") + if basis == "duration_ms": + duration = Decimal(int(tool_call.duration_ms or 0)) + if duration <= 0: + raise _SkipMetering( + "collecting_missing_duration", + "duration_ms 尚无真实正数值，等待工具调用完成后再计量。", + collecting=True, + ) + return duration + request = tool_call.request_json if isinstance(tool_call.request_json, dict) else {} + response = tool_call.response_json if isinstance(tool_call.response_json, dict) else {} + if basis in RESOURCE_QUANTITY_KEYS: + quantity = _read_token( + (response, request), + RESOURCE_QUANTITY_KEYS[basis], + ) + if quantity is None or quantity <= 0: + raise _SkipMetering( + f"collecting_missing_{basis}", + f"{basis} 缺少真实结构化正数，拒绝按正文或文件名估算用量。", + collecting=True, + ) + return quantity + input_tokens = _read_token((request, response), INPUT_TOKEN_KEYS) + output_tokens = _read_token((response, request), OUTPUT_TOKEN_KEYS) + if basis == "input_tokens": + quantity = input_tokens + elif basis == "output_tokens": + quantity = output_tokens + else: + quantity = _read_token((response, request), TOTAL_TOKEN_KEYS) + if quantity is None and input_tokens is not None and output_tokens is not None: + quantity = input_tokens + output_tokens + if quantity is None or quantity <= 0: + raise _SkipMetering( + "collecting_missing_tokens", + f"{basis} 缺少真实结构化数值，正文长度不会被用于估算 token。", + collecting=True, + ) + return quantity + + def _reserved_meter( + self, + reservation: CommercialRuntimeReservation, + tenant_id: str, + ) -> _ResolvedMeter: + if reservation.tenant_id != tenant_id: + raise CommercialConflictError("商业预占不属于 AgentRun 的可信租户。") + subscription = self.db.scalar( + select(TenantSubscription).where( + TenantSubscription.tenant_id == tenant_id, + TenantSubscription.id == reservation.subscription_id, + ) + ) + entitlement = self.db.scalar( + select(CommercialEntitlement).where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.subscription_id == reservation.subscription_id, + CommercialEntitlement.id == reservation.entitlement_id, + ) + ) + if subscription is None or entitlement is None: + raise LookupError("商业预占引用的订阅或权益不存在。") + basis = reservation.quantity_basis + if basis not in SUPPORTED_BASES: + raise CommercialConfigurationError("商业预占的计量基准不受支持。") + runtime = reservation.meter_config_json + cost = _cost_config(runtime.get("internal_cost"), entitlement.unit) + return _ResolvedMeter( + subscription=subscription, + entitlement=entitlement, + quantity_basis=basis, # type: ignore[arg-type] + cost=cost, + ) + + @staticmethod + def _cost_payload( + tool_call: AgentToolCall, + run: AgentRun, + meter: _ResolvedMeter, + quantity: Decimal, + usage_event_id: str, + occurred_at: datetime, + usage_metadata: dict[str, Any], + ) -> CommercialCostEventCreate: + cost = meter.cost + if cost is None: + raise CommercialConfigurationError("内部成本配置不存在。") + return CommercialCostEventCreate( + subscription_id=meter.subscription.id, + usage_event_id=usage_event_id, + cost_category=cost.cost_category, # type: ignore[arg-type] + quantity=quantity, + unit=cost.unit, + unit_cost=cost.unit_cost, + original_currency=cost.original_currency, + reporting_currency=cost.reporting_currency, + fx_rate=cost.fx_rate, + provider=cost.provider, + sku=cost.sku, + model_name=cost.model_name, + allocation_key=f"runtime:{meter.entitlement.id}", + occurred_at=occurred_at, + source_system=SOURCE_SYSTEM, + idempotency_key=_cost_idempotency_key(tool_call.id, meter.entitlement.id), + correlation_id=run.run_id, + metadata_json=dict(usage_metadata), + ) + + def _tool_call_pair(self, tool_call_id: str) -> tuple[AgentToolCall, AgentRun] | None: + return self.db.execute( + select(AgentToolCall, AgentRun) + .join(AgentRun, AgentRun.run_id == AgentToolCall.run_id) + .where(AgentToolCall.id == str(tool_call_id or "").strip()) + ).one_or_none() + + @staticmethod + def _skipped( + tool_call: AgentToolCall, + run: AgentRun, + tenant_id: str | None, + code: str, + reason: str, + *, + collecting: bool = False, + reconcile: bool = False, + business_call_occurred: bool = True, + ) -> RuntimeMeteringResult: + return RuntimeMeteringResult( + status="skipped", + reason_code=code, + reason=reason, + tool_call_id=tool_call.id, + run_id=run.run_id, + tenant_id=tenant_id, + collecting=collecting, + business_call_occurred=business_call_occurred, + requires_reconciliation=reconcile, + ) + + @staticmethod + def _preflight_denied( + run_id: str, + tenant_id: str | None, + code: str, + reason: str, + security_decision: SecurityDecision, + ) -> RuntimeQuotaPreflight: + return RuntimeQuotaPreflight( + allowed=False, + reason_code=code, + reason=reason, + run_id=run_id, + tenant_id=tenant_id, + entitlement_id=None, + quantity_basis=None, + requested_quantity=None, + commercially_allowed=False, + security_decision=security_decision, + ) + + +def _read_token(payloads: tuple[dict[str, Any], ...], keys: tuple[str, ...]) -> Decimal | None: + for payload in payloads: + containers = [payload] + containers.extend( + value for key in TOKEN_CONTAINERS if isinstance((value := payload.get(key)), dict) + ) + for container in containers: + for key in keys: + if key not in container: + continue + value = container[key] + if isinstance(value, bool) or not isinstance(value, (int, float, Decimal)): + raise CommercialConfigurationError(f"结构化 token 字段 {key} 必须是数值。") + quantity = _decimal(value, key) + if quantity < 0 or quantity != quantity.to_integral_value(): + raise CommercialConfigurationError(f"结构化 token 字段 {key} 必须是非负整数。") + return quantity + return None + + +def _subscription_active_at(row: TenantSubscription, when: datetime) -> bool: + start = _utc(row.starts_at) + end = _utc(row.ends_at) if row.ends_at is not None else None + period_start = _utc(row.current_period_start) + period_end = _utc(row.current_period_end) + return start <= when < period_end and period_start <= when and (end is None or when < end) + + +def _entitlement_active_at(row: CommercialEntitlement, when: datetime) -> bool: + start = _utc(row.effective_from) + end = _utc(row.effective_to) if row.effective_to is not None else None + return start <= when and (end is None or when < end) diff --git a/server/src/app/services/commercial_runtime_policy.py b/server/src/app/services/commercial_runtime_policy.py new file mode 100644 index 0000000..23c0829 --- /dev/null +++ b/server/src/app/services/commercial_runtime_policy.py @@ -0,0 +1,95 @@ +"""商业运行计量的轻量策略，避免执行桥接与账本服务互相依赖。""" + +from __future__ import annotations + +from decimal import Decimal, InvalidOperation +from typing import Any, Literal + +from app.services.commercial_access_policy import CommercialConfigurationError + +ToolCallMeteringState = Literal["billable", "collecting", "not_billable"] + +SUCCESSFUL_TOOL_STATUSES = {"completed", "ok", "success", "succeeded"} +COLLECTING_TOOL_STATUSES = {"pending", "queued", "running"} +NON_OCCURRED_TOOL_STATUSES = {"blocked", "cancelled", "canceled", "skipped"} + + +def tool_call_metering_state(status: Any) -> ToolCallMeteringState: + normalized = str(status or "").strip().lower() + if normalized in SUCCESSFUL_TOOL_STATUSES: + return "billable" + if normalized in COLLECTING_TOOL_STATUSES: + return "collecting" + return "not_billable" + + +def tool_call_business_occurred(status: Any) -> bool: + return str(status or "").strip().lower() not in NON_OCCURRED_TOOL_STATUSES + + +def runtime_meter_config(config: Any) -> dict[str, Any] | None: + if not isinstance(config, dict): + return None + runtime = config.get("runtime_meter") + if not isinstance(runtime, dict) or runtime.get("enabled") is not True: + return None + return runtime + + +def runtime_meter_matches(runtime: dict[str, Any], tool_type: str, tool_name: str) -> bool: + return _dimension_matches(runtime, "tool_type", "tool_types", tool_type) and _dimension_matches( + runtime, + "tool_name", + "tool_names", + tool_name, + ) + + +def preflight_quantity(runtime_configs: list[dict[str, Any]]) -> Decimal: + """返回调用前最小预留量；变动计量可由配置声明更保守的预留量。""" + + raw_values = [item.get("preflight_quantity", 1) for item in runtime_configs] + quantities = {_positive_decimal(value) for value in raw_values} + if len(quantities) != 1: + raise CommercialConfigurationError("多个匹配计量器的 preflight_quantity 不一致。") + return quantities.pop() + + +def tenant_from_agent_route(route: Any) -> str | None: + if not isinstance(route, dict): + return None + value = route.get("tenant_id") + if not isinstance(value, str): + return None + normalized = value.strip() + return normalized or None + + +def _dimension_matches( + config: dict[str, Any], + singular: str, + plural: str, + actual: str, +) -> bool: + expected = config.get(singular) + if isinstance(expected, str) and expected.strip(): + return expected.strip() == str(actual or "").strip() + choices = config.get(plural) + if not isinstance(choices, list) or not choices: + return False + normalized = {item.strip() for item in choices if isinstance(item, str) and item.strip()} + return str(actual or "").strip() in normalized + + +def _positive_decimal(value: Any) -> Decimal: + if value is None or isinstance(value, bool): + raise CommercialConfigurationError("runtime_meter.preflight_quantity 必须是正数。") + try: + result = Decimal(str(value)) + except (InvalidOperation, ValueError) as error: + raise CommercialConfigurationError( + "runtime_meter.preflight_quantity 必须是正数。" + ) from error + if not result.is_finite() or result <= 0: + raise CommercialConfigurationError("runtime_meter.preflight_quantity 必须是正数。") + return result diff --git a/server/src/app/services/commercial_runtime_reconciler.py b/server/src/app/services/commercial_runtime_reconciler.py new file mode 100644 index 0000000..0dacb85 --- /dev/null +++ b/server/src/app/services/commercial_runtime_reconciler.py @@ -0,0 +1,153 @@ +"""安全补偿过期的商业运行时额度预占。""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import UTC, datetime + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.agent_run import AgentRun, AgentToolCall +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.commercial_runtime_bridge import CommercialRuntimeBridge +from app.services.commercial_runtime_metering import RuntimeMeteringResult +from app.services.commercial_runtime_reservations import ( + CommercialRuntimeReservationService, +) + +TERMINAL_RUN_STATUSES = {"succeeded", "failed", "blocked", "cancelled", "canceled"} + + +@dataclass(frozen=True) +class RuntimeReservationReconciliationBatch: + items: tuple[RuntimeMeteringResult, ...] + settled: int + released: int + deferred: int + errors: int + + +class CommercialRuntimeReconciler: + """只在能证明工具终态时结算或释放；不按超时猜测业务结果。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def reconcile_expired( + self, + *, + as_of: datetime | None = None, + limit: int = 100, + ) -> RuntimeReservationReconciliationBatch: + when = _utc(as_of or datetime.now(UTC)) + candidates = CommercialRuntimeReservationService(self.db).expired_candidates( + as_of=when, + limit=limit, + ) + candidate_ids = [row.id for row in candidates] + self.db.rollback() + items = tuple(self._reconcile_one(row_id, when) for row_id in candidate_ids) + return RuntimeReservationReconciliationBatch( + items=items, + settled=sum(item.status in {"created", "replayed"} for item in items), + released=sum( + item.reason_code + in {"expired_without_business_call", "tool_call_not_billable"} + for item in items + ), + deferred=sum(item.collecting for item in items), + errors=sum(item.status == "error" for item in items), + ) + + def _reconcile_one( + self, + reservation_id: str, + when: datetime, + ) -> RuntimeMeteringResult: + tool_call_id = "" + run_id = "" + tenant_id = None + try: + reservation = CommercialRuntimeReservationService(self.db).get_for_update( + reservation_id + ) + tool_call_id = reservation.tool_call_id + run_id = reservation.run_id + tenant_id = reservation.tenant_id + if reservation.status != "reserved" or _utc(reservation.expires_at) > when: + self.db.rollback() + return _result( + reservation, + reason_code="reservation_no_longer_expired", + reason="预占已由其他补偿器处理或尚未过期。", + ) + tool_call = self.db.scalar( + select(AgentToolCall).where(AgentToolCall.id == reservation.tool_call_id) + ) + if tool_call is not None: + self.db.rollback() + return CommercialRuntimeBridge(self.db).sync_tool_call(tool_call.id) + run = self.db.scalar( + select(AgentRun).where(AgentRun.run_id == reservation.run_id) + ) + if run is not None and str(run.status or "").strip().lower() in TERMINAL_RUN_STATUSES: + CommercialRuntimeReservationService(self.db).release( + reservation.id, + reason_code="terminal_run_without_tool_call", + expired=True, + settled_at=when, + ) + self.db.commit() + return _result( + reservation, + reason_code="expired_without_business_call", + reason="运行已终止且没有真实工具调用，过期预占已安全释放。", + ) + self.db.rollback() + return _result( + reservation, + reason_code="expired_reservation_still_uncertain", + reason="运行尚未终止或运行记录缺失，继续保留额度并等待可验证终态。", + collecting=True, + requires_reconciliation=True, + ) + except Exception: + self.db.rollback() + return RuntimeMeteringResult( + status="error", + reason_code="reservation_reconciliation_failed", + reason="过期预占补偿失败，额度继续保留并等待幂等重试。", + tool_call_id=tool_call_id, + run_id=run_id, + tenant_id=tenant_id, + business_call_occurred=False, + requires_reconciliation=True, + ) + + +def _result( + reservation: CommercialRuntimeReservation, + *, + reason_code: str, + reason: str, + collecting: bool = False, + requires_reconciliation: bool = False, +) -> RuntimeMeteringResult: + return RuntimeMeteringResult( + status="skipped", + reason_code=reason_code, + reason=reason, + tool_call_id=reservation.tool_call_id, + run_id=reservation.run_id, + tenant_id=reservation.tenant_id, + entitlement_id=reservation.entitlement_id, + quantity_basis=reservation.quantity_basis, + collecting=collecting, + business_call_occurred=False, + requires_reconciliation=requires_reconciliation, + ) + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_runtime_registry.py b/server/src/app/services/commercial_runtime_registry.py new file mode 100644 index 0000000..6d69e9d --- /dev/null +++ b/server/src/app/services/commercial_runtime_registry.py @@ -0,0 +1,80 @@ +"""按可信租户和工具维度解析唯一运行时商业计量器。""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import or_, select +from sqlalchemy.orm import Session + +from app.models.commercial import CommercialEntitlement, TenantSubscription +from app.services.commercial_runtime_policy import ( + runtime_meter_config, + runtime_meter_matches, +) + + +@dataclass(frozen=True) +class ConfiguredRuntimeMeter: + entitlement: CommercialEntitlement + subscription: TenantSubscription + config: dict[str, Any] + + +def matching_runtime_meters( + db: Session, + *, + tenant_id: str, + tool_type: str, + tool_name: str, + as_of: datetime | None = None, +) -> list[ConfiguredRuntimeMeter]: + """返回当前窗口内显式配置的计量器，包括需失败关闭的暂停合同。""" + + tenant = str(tenant_id or "").strip() + if not tenant: + return [] + when = _utc(as_of or datetime.now(UTC)) + rows = db.execute( + select(CommercialEntitlement, TenantSubscription) + .join( + TenantSubscription, + TenantSubscription.id == CommercialEntitlement.subscription_id, + ) + .where( + CommercialEntitlement.tenant_id == tenant, + TenantSubscription.tenant_id == tenant, + TenantSubscription.status.in_( + ("trialing", "active", "past_due", "suspended") + ), + TenantSubscription.starts_at <= when, + or_(TenantSubscription.ends_at.is_(None), TenantSubscription.ends_at > when), + TenantSubscription.current_period_start <= when, + TenantSubscription.current_period_end > when, + CommercialEntitlement.status.in_(("active", "suspended")), + CommercialEntitlement.effective_from <= when, + or_( + CommercialEntitlement.effective_to.is_(None), + CommercialEntitlement.effective_to > when, + ), + ) + ).all() + meters: list[ConfiguredRuntimeMeter] = [] + for entitlement, subscription in rows: + runtime = runtime_meter_config(entitlement.config_json) + if runtime is None or not runtime_meter_matches(runtime, tool_type, tool_name): + continue + meters.append( + ConfiguredRuntimeMeter( + entitlement=entitlement, + subscription=subscription, + config=runtime, + ) + ) + return meters + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_runtime_reservations.py b/server/src/app/services/commercial_runtime_reservations.py new file mode 100644 index 0000000..079348e --- /dev/null +++ b/server/src/app/services/commercial_runtime_reservations.py @@ -0,0 +1,685 @@ +"""商业运行时额度预占、结算与补偿状态。""" + +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from datetime import UTC, datetime, timedelta +from decimal import Decimal +from typing import Any + +from sqlalchemy import func, select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.models.commercial import CommercialEntitlement, TenantSubscription, UsageMeterEvent +from app.models.commercial_billing import CommercialBillingPeriod +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) +from app.services.commercial_billing_periods import CommercialBillingPeriodService +from app.services.commercial_periods import quota_period_key_for +from app.services.commercial_runtime_policy import runtime_meter_matches + +HOLDING_RESERVATION_STATUSES = ("reserved", "reconciliation_required") +DEFAULT_RESERVATION_TTL = timedelta(minutes=15) +MAX_RESERVATION_TTL = timedelta(hours=1) + + +@dataclass(frozen=True) +class RuntimeReservationUsage: + used_quantity: Decimal + held_quantity: Decimal + effective_limit: Decimal | None + + +class CommercialRuntimeReservationService: + def __init__(self, db: Session) -> None: + self.db = db + + def reserve( + self, + *, + tenant_id: str, + entitlement_id: str, + run_id: str, + tool_call_id: str, + tool_type: str, + tool_name: str, + quantity_basis: str, + reserved_quantity: Decimal, + meter_config: dict[str, Any], + as_of: datetime | None = None, + ttl: timedelta = DEFAULT_RESERVATION_TTL, + ) -> tuple[CommercialRuntimeReservation, bool]: + when = _utc(as_of or datetime.now(UTC)) + ttl = _validated_ttl(ttl) + identity = _identity(tenant_id, run_id, tool_call_id, tool_type, tool_name) + quantity = _positive_decimal(reserved_quantity, "预占数量") + fingerprint = _fingerprint( + { + **identity, + "entitlement_id": entitlement_id, + "quantity_basis": quantity_basis, + "reserved_quantity": str(quantity), + "meter_config": meter_config, + } + ) + existing = self._by_tool_call(identity["tenant_id"], identity["tool_call_id"]) + if existing is not None: + self._assert_fingerprint(existing, fingerprint) + if existing.status == "released": + return self._reactivate_released( + existing, + fingerprint=fingerprint, + meter_config=meter_config, + identity=identity, + quantity_basis=quantity_basis, + quantity=quantity, + when=when, + ttl=ttl, + ) + return existing, False + + subscription, entitlement, billing_period = self._locked_contract( + identity["tenant_id"], + _required(entitlement_id, "权益编号"), + when, + ) + self._assert_meter_snapshot( + entitlement, + meter_config, + identity["tool_type"], + identity["tool_name"], + quantity_basis, + ) + existing = self._by_tool_call(identity["tenant_id"], identity["tool_call_id"]) + if existing is not None: + self._assert_fingerprint(existing, fingerprint) + return existing, False + + quota_period_key = quota_period_key_for(entitlement, subscription, when) + usage = self._locked_usage(subscription, entitlement, quota_period_key) + if usage.effective_limit is not None: + resulting = usage.used_quantity + usage.held_quantity + quantity + if resulting > usage.effective_limit: + raise CommercialConflictError("运行时预占将超过商业硬配额，真实工具未执行。") + row = CommercialRuntimeReservation( + tenant_id=identity["tenant_id"], + subscription_id=subscription.id, + entitlement_id=entitlement.id, + billing_period_id=billing_period.id, + run_id=identity["run_id"], + tool_call_id=identity["tool_call_id"], + tool_type=identity["tool_type"], + tool_name=identity["tool_name"], + quantity_basis=quantity_basis, + reserved_quantity=quantity, + actual_quantity=None, + period_key=billing_period.period_key, + quota_period_key=quota_period_key, + status="reserved", + request_fingerprint=fingerprint, + meter_config_json=dict(meter_config), + resolution_code=None, + expires_at=when + ttl, + settled_at=None, + created_at=when, + updated_at=when, + ) + return self._flush_idempotently(row, fingerprint) + + def _reactivate_released( + self, + existing: CommercialRuntimeReservation, + *, + fingerprint: str, + meter_config: dict[str, Any], + identity: dict[str, str], + quantity_basis: str, + quantity: Decimal, + when: datetime, + ttl: timedelta, + ) -> tuple[CommercialRuntimeReservation, bool]: + row = self._locked_reservation(existing.id) + self._assert_fingerprint(row, fingerprint) + if row.status != "released": + return row, False + subscription, entitlement, billing_period = self._locked_contract( + identity["tenant_id"], + row.entitlement_id, + when, + ) + self._assert_meter_snapshot( + entitlement, + meter_config, + identity["tool_type"], + identity["tool_name"], + quantity_basis, + ) + quota_period_key = quota_period_key_for(entitlement, subscription, when) + if ( + row.subscription_id != subscription.id + or row.billing_period_id != billing_period.id + or row.quota_period_key != quota_period_key + ): + raise CommercialConflictError( + "已释放预占不属于当前商业账期，请使用新的业务操作标识重试。" + ) + usage = self._locked_usage(subscription, entitlement, quota_period_key) + if ( + usage.effective_limit is not None + and usage.used_quantity + usage.held_quantity + quantity + > usage.effective_limit + ): + raise CommercialConflictError("运行时预占将超过商业硬配额，真实工具未执行。") + row.status = "reserved" + row.actual_quantity = None + row.resolution_code = None + row.expires_at = when + ttl + row.settled_at = None + row.updated_at = when + self.db.flush() + return row, False + + def record_reconciliation_backlog( + self, + *, + tenant_id: str, + entitlement_id: str, + run_id: str, + tool_call_id: str, + tool_type: str, + tool_name: str, + quantity_basis: str, + reserved_quantity: Decimal, + actual_quantity: Decimal | None, + meter_config: dict[str, Any], + resolution_code: str, + as_of: datetime | None = None, + ) -> tuple[CommercialRuntimeReservation, bool]: + when = _utc(as_of or datetime.now(UTC)) + identity = _identity(tenant_id, run_id, tool_call_id, tool_type, tool_name) + reserved = _positive_decimal(reserved_quantity, "补偿占位数量") + actual = ( + _positive_decimal(actual_quantity, "真实调用数量") + if actual_quantity is not None + else None + ) + fingerprint = _fingerprint( + { + **identity, + "entitlement_id": entitlement_id, + "quantity_basis": quantity_basis, + "reserved_quantity": str(reserved), + "actual_quantity": str(actual) if actual is not None else None, + "resolution_code": resolution_code, + "meter_config": meter_config, + } + ) + existing = self._by_tool_call(identity["tenant_id"], identity["tool_call_id"]) + if existing is not None: + self._assert_fingerprint(existing, fingerprint) + return existing, False + subscription, entitlement, billing_period = self._locked_contract( + identity["tenant_id"], + _required(entitlement_id, "权益编号"), + when, + require_consumable=False, + ) + self._assert_meter_snapshot( + entitlement, + meter_config, + identity["tool_type"], + identity["tool_name"], + quantity_basis, + require_active=False, + ) + row = CommercialRuntimeReservation( + tenant_id=identity["tenant_id"], + subscription_id=subscription.id, + entitlement_id=entitlement.id, + billing_period_id=billing_period.id, + run_id=identity["run_id"], + tool_call_id=identity["tool_call_id"], + tool_type=identity["tool_type"], + tool_name=identity["tool_name"], + quantity_basis=quantity_basis, + reserved_quantity=reserved, + actual_quantity=actual, + period_key=billing_period.period_key, + quota_period_key=quota_period_key_for(entitlement, subscription, when), + status="reconciliation_required", + request_fingerprint=fingerprint, + meter_config_json=dict(meter_config), + resolution_code=_required(resolution_code, "补偿原因")[:64], + expires_at=when + DEFAULT_RESERVATION_TTL, + settled_at=None, + created_at=when, + updated_at=when, + ) + return self._flush_idempotently(row, fingerprint) + + def by_tool_call(self, tool_call_id: str) -> CommercialRuntimeReservation | None: + return self.db.scalar( + select(CommercialRuntimeReservation).where( + CommercialRuntimeReservation.tool_call_id == _required(tool_call_id, "工具调用编号") + ) + ) + + def get_for_update(self, reservation_id: str) -> CommercialRuntimeReservation: + return self._locked_reservation(reservation_id) + + def mark_committed( + self, + reservation_id: str, + actual_quantity: Decimal, + *, + settled_at: datetime | None = None, + ) -> CommercialRuntimeReservation: + row = self._locked_reservation(reservation_id) + quantity = _positive_decimal(actual_quantity, "真实调用数量") + if row.status == "committed": + if Decimal(row.actual_quantity or 0) != quantity: + raise CommercialConflictError("预占已按不同真实数量完成结算。") + return row + if row.status != "reserved": + raise CommercialConflictError(f"预占状态 {row.status} 不允许自动结算。") + if quantity > Decimal(row.reserved_quantity): + raise CommercialConflictError("真实调用数量超过执行前可信预占，拒绝伪造结算。") + now = _utc(settled_at or datetime.now(UTC)) + row.status = "committed" + row.actual_quantity = quantity + row.resolution_code = None + row.settled_at = now + row.updated_at = now + self.db.flush() + return row + + def require_committed_reconciliation( + self, + reservation_id: str, + *, + reason_code: str, + ) -> CommercialRuntimeReservation: + row = self._locked_reservation(reservation_id) + if row.status == "committed_reconciliation_required": + return row + if row.status != "committed" or row.actual_quantity is None: + raise CommercialConflictError("只有已写入用量的预占可进入提交后补偿。") + row.status = "committed_reconciliation_required" + row.resolution_code = _required(reason_code, "提交后补偿原因")[:64] + row.updated_at = datetime.now(UTC) + self.db.flush() + return row + + def require_reconciliation( + self, + reservation_id: str, + *, + reason_code: str, + actual_quantity: Decimal | None = None, + ) -> CommercialRuntimeReservation: + """真实调用已发生但尚不能安全结算时冻结预占并进入补偿队列。""" + + row = self._locked_reservation(reservation_id) + actual = ( + _positive_decimal(actual_quantity, "真实调用数量") + if actual_quantity is not None + else None + ) + if row.status == "reconciliation_required": + persisted = ( + Decimal(row.actual_quantity) if row.actual_quantity is not None else None + ) + if persisted != actual: + raise CommercialConflictError("补偿预占已绑定不同真实调用数量。") + return row + if row.status != "reserved": + raise CommercialConflictError( + f"预占状态 {row.status} 不允许进入执行后补偿。" + ) + row.status = "reconciliation_required" + row.actual_quantity = actual + row.resolution_code = _required(reason_code, "补偿原因")[:64] + row.settled_at = None + row.updated_at = datetime.now(UTC) + self.db.flush() + return row + + def mark_reconciled(self, reservation_id: str) -> CommercialRuntimeReservation: + row = self._locked_reservation(reservation_id) + if row.status == "committed": + return row + if row.status != "committed_reconciliation_required": + raise CommercialConflictError("当前预占不在提交后补偿状态。") + row.status = "committed" + row.resolution_code = None + row.updated_at = datetime.now(UTC) + self.db.flush() + return row + + def release( + self, + reservation_id: str, + *, + reason_code: str, + expired: bool = False, + settled_at: datetime | None = None, + ) -> CommercialRuntimeReservation: + row = self._locked_reservation(reservation_id) + target_status = "expired" if expired else "released" + if row.status == target_status: + return row + if row.status != "reserved": + raise CommercialConflictError(f"预占状态 {row.status} 不允许释放。") + now = _utc(settled_at or datetime.now(UTC)) + row.status = target_status + row.actual_quantity = None + row.resolution_code = _required(reason_code, "释放原因")[:64] + row.settled_at = now + row.updated_at = now + self.db.flush() + return row + + def expired_candidates( + self, + *, + as_of: datetime | None = None, + limit: int = 100, + ) -> list[CommercialRuntimeReservation]: + if not isinstance(limit, int) or isinstance(limit, bool) or not 1 <= limit <= 500: + raise ValueError("预占补偿批次大小必须介于 1 和 500 之间。") + when = _utc(as_of or datetime.now(UTC)) + return list( + self.db.scalars( + select(CommercialRuntimeReservation) + .where( + CommercialRuntimeReservation.status == "reserved", + CommercialRuntimeReservation.expires_at <= when, + ) + .order_by( + CommercialRuntimeReservation.expires_at, + CommercialRuntimeReservation.id, + ) + .limit(limit) + ).all() + ) + + def reconciliation_candidates( + self, + *, + limit: int = 100, + ) -> list[CommercialRuntimeReservation]: + if not isinstance(limit, int) or isinstance(limit, bool) or not 1 <= limit <= 500: + raise ValueError("运行时补偿队列批次大小必须介于 1 和 500 之间。") + return list( + self.db.scalars( + select(CommercialRuntimeReservation) + .where( + CommercialRuntimeReservation.status.in_( + ( + "reconciliation_required", + "committed_reconciliation_required", + ) + ) + ) + .order_by( + CommercialRuntimeReservation.created_at, + CommercialRuntimeReservation.id, + ) + .limit(limit) + ).all() + ) + + def _locked_contract( + self, + tenant_id: str, + entitlement_id: str, + when: datetime, + *, + require_consumable: bool = True, + ) -> tuple[TenantSubscription, CommercialEntitlement, CommercialBillingPeriod]: + hint = self.db.scalar( + select(CommercialEntitlement).where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.id == entitlement_id, + ) + ) + if hint is None: + raise LookupError("商业运行权益不存在。") + subscription = self.db.scalar( + select(TenantSubscription) + .where( + TenantSubscription.tenant_id == tenant_id, + TenantSubscription.id == hint.subscription_id, + ) + .with_for_update() + .execution_options(populate_existing=True) + ) + entitlement = self.db.scalar( + select(CommercialEntitlement) + .where( + CommercialEntitlement.tenant_id == tenant_id, + CommercialEntitlement.subscription_id == hint.subscription_id, + CommercialEntitlement.id == entitlement_id, + ) + .with_for_update() + .execution_options(populate_existing=True) + ) + if subscription is None or entitlement is None: + raise LookupError("商业运行订阅或权益不存在。") + billing_period = CommercialBillingPeriodService(self.db).resolve( + tenant_id, + subscription.id, + when, + ) + if require_consumable: + _assert_consumable(subscription, entitlement, billing_period, when) + return subscription, entitlement, billing_period + + def _locked_usage( + self, + subscription: TenantSubscription, + entitlement: CommercialEntitlement, + quota_period_key: str, + ) -> RuntimeReservationUsage: + used = Decimal( + self.db.scalar( + select(func.coalesce(func.sum(UsageMeterEvent.quantity), 0)).where( + UsageMeterEvent.tenant_id == subscription.tenant_id, + UsageMeterEvent.subscription_id == subscription.id, + UsageMeterEvent.entitlement_id == entitlement.id, + UsageMeterEvent.quota_period_key == quota_period_key, + ) + ) + or 0 + ) + held = active_reserved_quantity( + self.db, + tenant_id=subscription.tenant_id, + subscription_id=subscription.id, + entitlement_id=entitlement.id, + period_key=quota_period_key, + ) + return RuntimeReservationUsage( + used_quantity=max(Decimal("0"), used), + held_quantity=max(Decimal("0"), held), + effective_limit=_effective_limit(entitlement), + ) + + @staticmethod + def _assert_meter_snapshot( + entitlement: CommercialEntitlement, + meter_config: dict[str, Any], + tool_type: str, + tool_name: str, + quantity_basis: str, + *, + require_active: bool = True, + ) -> None: + current_config = ( + entitlement.config_json if isinstance(entitlement.config_json, dict) else {} + ) + if current_config.get("runtime_meter") != meter_config: + raise CommercialConflictError("运行时计量配置在额度预占前发生变化，请重试。") + if not runtime_meter_matches(meter_config, tool_type, tool_name): + raise CommercialConfigurationError("运行时计量器与工具维度不匹配。") + if str(meter_config.get("quantity_basis") or "").strip() != quantity_basis: + raise CommercialConfigurationError("运行时计量基准与权益配置不一致。") + if require_active and entitlement.status != "active": + raise CommercialConflictError("商业运行权益当前不可用。") + + def _by_tool_call( + self, + tenant_id: str, + tool_call_id: str, + ) -> CommercialRuntimeReservation | None: + return self.db.scalar( + select(CommercialRuntimeReservation).where( + CommercialRuntimeReservation.tenant_id == tenant_id, + CommercialRuntimeReservation.tool_call_id == tool_call_id, + ) + ) + + def _locked_reservation(self, reservation_id: str) -> CommercialRuntimeReservation: + row = self.db.scalar( + select(CommercialRuntimeReservation) + .where(CommercialRuntimeReservation.id == _required(reservation_id, "预占编号")) + .with_for_update() + ) + if row is None: + raise LookupError("商业运行预占不存在。") + return row + + def _flush_idempotently( + self, + row: CommercialRuntimeReservation, + fingerprint: str, + ) -> tuple[CommercialRuntimeReservation, bool]: + try: + with self.db.begin_nested(): + self.db.add(row) + self.db.flush() + return row, True + except IntegrityError: + existing = self._by_tool_call(row.tenant_id, row.tool_call_id) + if existing is None: + raise + self._assert_fingerprint(existing, fingerprint) + return existing, False + + @staticmethod + def _assert_fingerprint(row: CommercialRuntimeReservation, fingerprint: str) -> None: + if row.request_fingerprint != fingerprint: + raise CommercialConflictError("工具调用编号已被不同商业预占请求占用。") + + +def active_reserved_quantity( + db: Session, + *, + tenant_id: str, + subscription_id: str, + entitlement_id: str, + period_key: str, +) -> Decimal: + return Decimal( + db.scalar( + select( + func.coalesce( + func.sum(CommercialRuntimeReservation.reserved_quantity), + 0, + ) + ).where( + CommercialRuntimeReservation.tenant_id == tenant_id, + CommercialRuntimeReservation.subscription_id == subscription_id, + CommercialRuntimeReservation.entitlement_id == entitlement_id, + CommercialRuntimeReservation.quota_period_key == period_key, + CommercialRuntimeReservation.status.in_(HOLDING_RESERVATION_STATUSES), + ) + ) + or 0 + ) + + +def _assert_consumable( + subscription: TenantSubscription, + entitlement: CommercialEntitlement, + billing_period: CommercialBillingPeriod, + when: datetime, +) -> None: + if subscription.status not in {"trialing", "active"}: + raise CommercialConflictError("订阅当前不可预占商业用量。") + start = _utc(subscription.starts_at) + end = _utc(subscription.ends_at) if subscription.ends_at is not None else None + period_start = _utc(billing_period.period_start) + period_end = _utc(billing_period.period_end) + if when < start or (end is not None and when >= end) or not period_start <= when < period_end: + raise CommercialConflictError("预占时间不在当前订阅计费周期内。") + effective_from = _utc(entitlement.effective_from) + effective_to = _utc(entitlement.effective_to) if entitlement.effective_to else None + if entitlement.status != "active" or when < effective_from: + raise CommercialConflictError("商业运行权益当前不可预占。") + if effective_to is not None and when >= effective_to: + raise CommercialConflictError("商业运行权益已经过期。") + + +def _effective_limit(entitlement: CommercialEntitlement) -> Decimal | None: + if entitlement.entitlement_type == "unlimited": + return None + if entitlement.hard_limit_quantity is not None: + return Decimal(entitlement.hard_limit_quantity) + if entitlement.overage_policy == "block" and entitlement.included_quantity is not None: + return Decimal(entitlement.included_quantity) + return None + + +def _identity( + tenant_id: str, + run_id: str, + tool_call_id: str, + tool_type: str, + tool_name: str, +) -> dict[str, str]: + return { + "tenant_id": _required(tenant_id, "租户编号"), + "run_id": _required(run_id, "运行编号"), + "tool_call_id": _required(tool_call_id, "工具调用编号"), + "tool_type": _required(tool_type, "工具类型"), + "tool_name": _required(tool_name, "工具名称"), + } + + +def _fingerprint(payload: dict[str, Any]) -> str: + canonical = json.dumps(payload, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() + + +def _validated_ttl(value: timedelta) -> timedelta: + if not isinstance(value, timedelta) or value <= timedelta(0) or value > MAX_RESERVATION_TTL: + raise CommercialConfigurationError("运行时预占有效期必须大于 0 且不超过 1 小时。") + return value + + +def _positive_decimal(value: Any, label: str) -> Decimal: + try: + result = Decimal(str(value)) + except Exception as error: + raise CommercialConfigurationError(f"{label}必须是正数。") from error + if not result.is_finite() or result <= 0: + raise CommercialConfigurationError(f"{label}必须是正数。") + return result + + +def _required(value: Any, label: str) -> str: + normalized = str(value or "").strip() + if not normalized: + raise CommercialConfigurationError(f"{label}不能为空。") + return normalized + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/commercial_runtime_values.py b/server/src/app/services/commercial_runtime_values.py new file mode 100644 index 0000000..366fe2f --- /dev/null +++ b/server/src/app/services/commercial_runtime_values.py @@ -0,0 +1,90 @@ +"""商业运行计量的标量校验、游标与安全错误文案。""" + +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal, InvalidOperation +from typing import Any + +from pydantic import ValidationError + +from app.services.commercial_access_policy import ( + CommercialConfigurationError, + CommercialConflictError, +) + + +def encode_cursor(created_at: datetime, tool_call_id: str) -> str: + return f"{utc(created_at).isoformat()}|{tool_call_id}" + + +def usage_idempotency_key(tool_call_id: str, entitlement_id: str) -> str: + return f"agent-tool:{tool_call_id}:{entitlement_id}:usage:v1" + + +def cost_idempotency_key(tool_call_id: str, entitlement_id: str) -> str: + return f"agent-tool:{tool_call_id}:{entitlement_id}:cost:v1" + + +def decode_cursor(value: str) -> tuple[datetime, str]: + try: + raw_time, tool_call_id = str(value or "").rsplit("|", maxsplit=1) + created_at = datetime.fromisoformat(raw_time) + except (TypeError, ValueError) as error: + raise ValueError("运行时计量游标格式无效。") from error + if not tool_call_id.strip() or created_at.tzinfo is None or created_at.utcoffset() is None: + raise ValueError("运行时计量游标必须包含显式时区和工具调用 ID。") + return utc(created_at), tool_call_id.strip() + + +def currency(value: Any, label: str) -> str: + normalized = str(value or "").strip().upper() + if len(normalized) != 3 or not normalized.isalpha() or not normalized.isascii(): + raise CommercialConfigurationError(f"{label}必须是三位英文字母。") + return normalized + + +def positive_decimal(value: Any, label: str) -> Decimal: + quantity = decimal(value, label) + if quantity <= 0: + raise CommercialConfigurationError(f"{label}必须大于 0。") + return quantity + + +def decimal(value: Any, label: str) -> Decimal: + if value is None or isinstance(value, bool): + raise CommercialConfigurationError(f"{label}必须是数值。") + try: + result = Decimal(str(value)) + except (InvalidOperation, ValueError) as error: + raise CommercialConfigurationError(f"{label}必须是数值。") from error + if not result.is_finite(): + raise CommercialConfigurationError(f"{label}必须是有限数值。") + return result + + +def required(value: Any) -> str: + normalized = str(value or "").strip() + if not normalized: + raise ValueError("标识不能为空。") + return normalized + + +def optional_text(value: Any) -> str | None: + normalized = str(value or "").strip() + return normalized or None + + +def utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + + +def safe_error_reason(error: Exception) -> str: + if isinstance(error, ValidationError): + return "运行时计量载荷不满足商业事实约束，未写入伪事实。" + if isinstance( + error, + (CommercialConfigurationError, CommercialConflictError, LookupError, ValueError), + ): + return str(error) + return "运行时计量发生内部错误，业务调用已发生并需要补偿核对。" diff --git a/server/src/app/services/commercial_subscription_rollover.py b/server/src/app/services/commercial_subscription_rollover.py new file mode 100644 index 0000000..24d9209 --- /dev/null +++ b/server/src/app/services/commercial_subscription_rollover.py @@ -0,0 +1,194 @@ +"""自动续期账期签发；订阅行锁保证并发下只滚动一次。""" + +from __future__ import annotations + +from datetime import UTC, datetime + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.commercial import TenantCommercialPlan, TenantSubscription +from app.schemas.commercial_billing import CommercialRolloverRead +from app.services.commercial_admin_audit import ( + CommercialAdminAuditService, + snapshot_resource, +) +from app.services.commercial_billing_periods import CommercialBillingPeriodService +from app.services.commercial_periods import add_billing_interval + + +class CommercialSubscriptionRolloverService: + def __init__(self, db: Session) -> None: + self.db = db + + def rollover_due( + self, + tenant_id: str, + subscription_id: str, + *, + as_of: datetime | None = None, + actor_id: str = "commercial-rollover-scheduler", + max_periods: int = 24, + ) -> CommercialRolloverRead: + if not 1 <= max_periods <= 120: + raise ValueError("单次自动续期账期数量必须介于 1 和 120 之间。") + when = _utc(as_of or datetime.now(UTC)) + subscription = self.db.scalar( + select(TenantSubscription) + .where( + TenantSubscription.tenant_id == tenant_id.strip(), + TenantSubscription.id == subscription_id.strip(), + ) + .with_for_update() + .execution_options(populate_existing=True) + ) + if subscription is None: + raise LookupError("自动续期订阅不存在。") + ineligible = self._ineligible_reason(subscription) + if ineligible is not None: + return self._result(subscription, "ineligible", *ineligible) + if _utc(subscription.current_period_end) > when: + return self._result( + subscription, + "not_due", + "period_not_due", + "当前账期尚未结束。", + ) + plan = self.db.scalar( + select(TenantCommercialPlan).where( + TenantCommercialPlan.tenant_id == subscription.tenant_id, + TenantCommercialPlan.id == subscription.plan_id, + ) + ) + if plan is None: + raise LookupError("自动续期订阅引用的套餐不存在。") + + period_service = CommercialBillingPeriodService(self.db) + audit = CommercialAdminAuditService(self.db) + created_period_ids: list[str] = [] + replayed = False + stop_reason: tuple[str, str] | None = None + for _ in range(max_periods): + start = _utc(subscription.current_period_end) + if start > when: + break + contract_end = _optional_utc(subscription.ends_at) + if contract_end is not None and start >= contract_end: + stop_reason = ( + "contract_ended", + "订阅合同已经结束，拒绝自动生成合同外账期。", + ) + break + end = add_billing_interval(start, subscription.billing_interval) + if contract_end is not None and end > contract_end: + stop_reason = ( + "contract_boundary_requires_renewal", + "下一完整账期会越过合同结束时间，必须显式续签。", + ) + break + request_id = f"rollover:{subscription.id}:{start.isoformat()}"[:120] + reason = "自动续期服务在合同边界内签发下一不可变账期。" + before = snapshot_resource(subscription) + period, created = period_service.issue( + subscription, + plan, + period_start=start, + period_end=end, + source="auto_renew", + actor_id=actor_id, + idempotency_key=request_id, + ) + if created: + created_period_ids.append(period.id) + else: + replayed = True + audit.record( + tenant_id=subscription.tenant_id, + actor_type="system", + actor_id=actor_id, + request_id=request_id, + reason=reason, + action="billing_period_created", + resource=period, + before={}, + after=snapshot_resource(period), + ) + subscription.current_period_start = start + subscription.current_period_end = end + subscription.version += 1 + self.db.flush() + audit.record( + tenant_id=subscription.tenant_id, + actor_type="system", + actor_id=actor_id, + request_id=request_id, + reason=reason, + action="subscription_rolled_over", + resource=subscription, + before=before, + after=snapshot_resource(subscription), + ) + if created_period_ids: + return self._result( + subscription, + "rolled_over", + "periods_created", + f"已签发 {len(created_period_ids)} 个不可变账期。", + created_period_ids, + ) + if replayed: + return self._result( + subscription, + "replayed", + "period_already_exists", + "账期已由并发或先前请求签发，订阅投影已对齐。", + ) + if stop_reason is not None: + return self._result(subscription, "ineligible", *stop_reason) + return self._result( + subscription, + "not_due", + "period_not_due", + "当前账期尚未结束。", + ) + + @staticmethod + def _ineligible_reason(subscription: TenantSubscription) -> tuple[str, str] | None: + if subscription.status not in {"trialing", "active"}: + return "subscription_not_consumable", "只有试用中或生效订阅可以自动续期。" + if not subscription.auto_renew: + return "auto_renew_disabled", "订阅未启用自动续期。" + if subscription.billing_interval == "contract": + return ( + "contract_interval_requires_explicit_renewal", + "合同制账期不能自动推导，必须显式续签合同。", + ) + return None + + @staticmethod + def _result( + subscription: TenantSubscription, + status: str, + reason_code: str, + reason: str, + created_period_ids: list[str] | None = None, + ) -> CommercialRolloverRead: + return CommercialRolloverRead( + tenant_id=subscription.tenant_id, + subscription_id=subscription.id, + status=status, + reason_code=reason_code, + reason=reason, + created_period_ids=list(created_period_ids or []), + current_period_start=subscription.current_period_start, + current_period_end=subscription.current_period_end, + subscription_version=subscription.version, + ) + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + + +def _optional_utc(value: datetime | None) -> datetime | None: + return _utc(value) if value is not None else None diff --git a/server/src/app/services/commercial_transaction_callbacks.py b/server/src/app/services/commercial_transaction_callbacks.py new file mode 100644 index 0000000..b9ef516 --- /dev/null +++ b/server/src/app/services/commercial_transaction_callbacks.py @@ -0,0 +1,67 @@ +"""把独立事务商业结算绑定到调用方业务事务终态。""" + +from __future__ import annotations + +import logging +from collections.abc import Callable + +from sqlalchemy import event +from sqlalchemy.orm import Session + +logger = logging.getLogger(__name__) +_CALLBACK_BATCH_KEY = "commercial_transaction_callback_batch" + + +def bind_commercial_transaction_outcome( + db: Session, + *, + on_commit: Callable[[], None], + on_rollback: Callable[[], None], + operation_name: str, +) -> None: + """业务提交后才结算；业务回滚只释放预占，不生成用量事实。""" + + # 纯文件删除可能发生在尚未执行 SQL 的 Session 中。显式取得连接以建立 + # 外层事务，确保随后 rollback/commit 一定产生可观察的终态回调。 + db.connection() + batch = db.info.get(_CALLBACK_BATCH_KEY) + if not isinstance(batch, dict) or batch.get("settled") is True: + batch = {"settled": False, "callbacks": []} + db.info[_CALLBACK_BATCH_KEY] = batch + _listen_for_transaction_outcome(db, batch) + callbacks = batch["callbacks"] + if not isinstance(callbacks, list): # pragma: no cover - defensive corruption guard + raise RuntimeError("商业事务回调批次状态损坏。") + callbacks.append((operation_name, on_commit, on_rollback)) + + +def _listen_for_transaction_outcome(db: Session, batch: dict) -> None: + def settle(outcome: str) -> None: + if batch["settled"]: + return + batch["settled"] = True + entries = list(batch["callbacks"]) + if outcome == "rolled_back": + entries.reverse() + callback_index = 1 if outcome == "committed" else 2 + for operation_name, on_commit, on_rollback in entries: + callback = (on_commit, on_rollback)[callback_index - 1] + try: + callback() + except Exception: + # 业务事务已经进入不可逆终态。商业预占保持可审计状态，交由补偿器处理； + # 不能把结算异常伪装成业务事务失败并诱发客户端重复写入。 + logger.exception( + "commercial_transaction_callback_failed operation=%s outcome=%s", + operation_name, + outcome, + ) + + def after_commit(_session: Session) -> None: + settle("committed") + + def after_rollback(_session: Session) -> None: + settle("rolled_back") + + event.listen(db, "after_commit", after_commit, once=True) + event.listen(db, "after_rollback", after_rollback, once=True) diff --git a/server/src/app/services/digital_employee_dashboard.py b/server/src/app/services/digital_employee_dashboard.py index d07ca6a..d10562a 100644 --- a/server/src/app/services/digital_employee_dashboard.py +++ b/server/src/app/services/digital_employee_dashboard.py @@ -10,6 +10,7 @@ from app.core.agent_enums import AgentName, AgentRunSource from app.db.schema_ownership import create_legacy_schema from app.models.agent_run import AgentRun, AgentToolCall from app.schemas.digital_employee_dashboard import DigitalEmployeeDashboardRead +from app.services.finance_report_tenant import require_report_tenant_id SUCCESS_STATUSES = {"success", "succeeded", "ok", "done", "completed"} FAILED_STATUSES = {"failed", "failure", "error", "errored"} @@ -159,15 +160,22 @@ CATEGORY_SPECS = { class DigitalEmployeeDashboardService: - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str = "default") -> None: self.db = db + self.tenant_id = require_report_tenant_id(tenant_id) - def build_dashboard(self, *, days: int = 7, limit: int = 300) -> DigitalEmployeeDashboardRead: + def build_dashboard( + self, + *, + days: int = 7, + limit: int = 300, + now: datetime | None = None, + ) -> DigitalEmployeeDashboardRead: window_days = max(1, min(int(days or 7), 30)) window_limit = max(1, min(int(limit or 300), 1000)) self._ensure_storage_ready() - now = datetime.now(UTC) - start = now - timedelta(days=window_days - 1) + generated_at = self._as_utc(now or datetime.now(UTC)) + start = generated_at - timedelta(days=window_days - 1) labels = self._date_labels(start.date(), window_days) all_runs = self._fetch_runs(start=start, limit=window_limit) @@ -176,7 +184,7 @@ class DigitalEmployeeDashboardService: return DigitalEmployeeDashboardRead( window_days=window_days, - generated_at=now.isoformat(), + generated_at=generated_at.isoformat(), has_real_data=bool(runs), totals=totals, daily_work=self._daily_work(labels, runs), @@ -193,6 +201,8 @@ class DigitalEmployeeDashboardService: select(AgentRun) .options(selectinload(AgentRun.tool_calls)) .where( + AgentRun.route_json["tenant_id"].as_string() == self.tenant_id, + AgentRun.ontology_json["tenant_id"].as_string() == self.tenant_id, AgentRun.started_at >= start, or_( AgentRun.agent == AgentName.HERMES.value, @@ -486,8 +496,7 @@ class DigitalEmployeeDashboardService: if str(route_json.get("selected_agent") or "").strip() == AgentName.HERMES.value: return True return any( - str(tool.tool_name or "").startswith("digital_employee.") - for tool in run.tool_calls + str(tool.tool_name or "").startswith("digital_employee.") for tool in run.tool_calls ) def _resolve_task_type(self, run: AgentRun) -> str: diff --git a/server/src/app/services/digital_employee_finance_report_task.py b/server/src/app/services/digital_employee_finance_report_task.py index 6f60b6e..2fa49cf 100644 --- a/server/src/app/services/digital_employee_finance_report_task.py +++ b/server/src/app/services/digital_employee_finance_report_task.py @@ -4,6 +4,7 @@ from datetime import UTC, date, datetime from time import perf_counter from typing import Any +from sqlalchemy import select from sqlalchemy.orm import Session from app.core.agent_enums import ( @@ -13,10 +14,16 @@ from app.core.agent_enums import ( AgentRunStatus, AgentToolType, ) +from app.models.agent_run import AgentRun from app.services.agent_runs import AgentRunService +from app.services.commercial_runtime_policy import tenant_from_agent_route from app.services.finance_report_context import FinanceReportContextService, FinanceReportType from app.services.finance_report_mailer import FinanceReportMailer from app.services.finance_report_renderer import FinanceReportRenderer +from app.services.finance_report_tenant import ( + TenantFinanceReportRunService, + require_report_tenant_id, +) FINANCE_REPORT_TASK_TYPE = "finance_report_orchestration" FINANCE_REPORT_TOOL_NAME = "digital_employee.finance_report.orchestrate" @@ -38,29 +45,69 @@ class DigitalEmployeeFinanceReportTaskService: source: str = AgentRunSource.SCHEDULE.value, run_id: str | None = None, record_tool_call: bool = True, + tenant_id: str = "default", ) -> dict[str, Any]: + tenant = require_report_tenant_id(tenant_id) + if run_id is not None: + self._require_run_tenant(run_id=run_id, tenant_id=tenant) + period = FinanceReportContextService.resolve_period( + report_type=report_type, + start_date=start_date, + end_date=end_date, + now=datetime.now(UTC), + ) + ledger_service = TenantFinanceReportRunService(self.db) + ledger, acquired = ledger_service.reserve( + tenant_id=tenant, + report_type=report_type, + period_start=period.start_date, + period_end=period.end_date, + ) + if not acquired: + existing = dict(ledger.result_json or {}) + if existing: + return {**existing, "idempotent_replay": True} + return { + "task_type": FINANCE_REPORT_TASK_TYPE, + "tenant_id": tenant, + "report_type": report_type, + "status": "already_running", + "period": period.to_dict(), + "idempotent_replay": True, + } + run_service = AgentRunService(self.db) run = None - if run_id is None: - run = run_service.create_run( - agent=AgentName.HERMES.value, - source=source, - user_id="digital_employee", - ontology_json={"scenario": "finance_report", "intent": report_type}, - route_json={ - "task_type": FINANCE_REPORT_TASK_TYPE, - "report_type": report_type, - "phase": "running", - "heartbeat_at": datetime.now(UTC).isoformat(), - }, - permission_level=AgentPermissionLevel.READ.value, - status=AgentRunStatus.RUNNING.value, - ) - run_id = run.run_id + try: + if run_id is None: + run = run_service.create_run( + agent=AgentName.HERMES.value, + source=source, + tenant_id=tenant, + user_id="digital_employee", + ontology_json={"scenario": "finance_report", "intent": report_type}, + route_json={ + "task_type": FINANCE_REPORT_TASK_TYPE, + "report_type": report_type, + "report_period_key": ledger.idempotency_key, + "phase": "running", + "heartbeat_at": datetime.now(UTC).isoformat(), + }, + permission_level=AgentPermissionLevel.READ.value, + status=AgentRunStatus.RUNNING.value, + ) + run_id = run.run_id + ledger_service.attach_agent_run(ledger, run_id) + except Exception as exc: + ledger_service.fail(ledger, exc) + raise timer = perf_counter() try: - context = FinanceReportContextService(self.db).build_context( + context = FinanceReportContextService( + self.db, + tenant_id=tenant, + ).build_context( report_type=report_type, start_date=start_date, end_date=end_date, @@ -70,6 +117,7 @@ class DigitalEmployeeFinanceReportTaskService: FinanceReportMailer(self.db).send_report( context=context, pdf_path=rendered.pdf_path, + tenant_id=tenant, recipients=recipients, dry_run=dry_run_email, ) @@ -90,6 +138,7 @@ class DigitalEmployeeFinanceReportTaskService: request_json={ "task_type": FINANCE_REPORT_TASK_TYPE, "report_type": report_type, + "tenant_id": tenant, "send_email": send_email, }, response_json=result, @@ -102,6 +151,7 @@ class DigitalEmployeeFinanceReportTaskService: "phase": "succeeded", "task_type": FINANCE_REPORT_TASK_TYPE, "report_type": report_type, + "report_period_key": ledger.idempotency_key, "report_delivery": result, "heartbeat_at": datetime.now(UTC).isoformat(), }, @@ -109,20 +159,24 @@ class DigitalEmployeeFinanceReportTaskService: result_summary=self._summary_text(result), finished_at=datetime.now(UTC), ) + ledger_service.succeed(ledger, result) return result except Exception as exc: - run_service.merge_route_json( - run_id, - { - "phase": "failed", - "task_type": FINANCE_REPORT_TASK_TYPE, - "report_type": report_type, - "heartbeat_at": datetime.now(UTC).isoformat(), - }, - status=AgentRunStatus.FAILED.value, - error_message=str(exc), - finished_at=datetime.now(UTC), - ) + ledger_service.fail(ledger, exc) + if run_id: + run_service.merge_route_json( + run_id, + { + "phase": "failed", + "task_type": FINANCE_REPORT_TASK_TYPE, + "report_type": report_type, + "report_period_key": ledger.idempotency_key, + "heartbeat_at": datetime.now(UTC).isoformat(), + }, + status=AgentRunStatus.FAILED.value, + error_message=str(exc), + finished_at=datetime.now(UTC), + ) raise @staticmethod @@ -136,6 +190,7 @@ class DigitalEmployeeFinanceReportTaskService: summary = context.get("summary") if isinstance(context.get("summary"), dict) else {} return { "task_type": FINANCE_REPORT_TASK_TYPE, + "tenant_id": context.get("tenant_id"), "report_type": context.get("report_type"), "title": period.get("title"), "period": period, @@ -151,6 +206,11 @@ class DigitalEmployeeFinanceReportTaskService: "delivery": delivery, } + def _require_run_tenant(self, *, run_id: str, tenant_id: str) -> None: + run = self.db.scalar(select(AgentRun).where(AgentRun.run_id == run_id)) + if run is None or tenant_from_agent_route(run.route_json) != tenant_id: + raise LookupError("财务报告运行不存在或不属于当前租户。") + @staticmethod def _summary_text(result: dict[str, Any]) -> str: summary = result.get("summary") if isinstance(result.get("summary"), dict) else {} diff --git a/server/src/app/services/digital_employee_reminder_scheduler.py b/server/src/app/services/digital_employee_reminder_scheduler.py index f224fb3..404aec9 100644 --- a/server/src/app/services/digital_employee_reminder_scheduler.py +++ b/server/src/app/services/digital_employee_reminder_scheduler.py @@ -5,9 +5,12 @@ import threading from datetime import datetime, time, timedelta from zoneinfo import ZoneInfo +from sqlalchemy import select + from app.core.agent_enums import AgentRunSource from app.core.logging import get_logger from app.db.session import get_session_factory +from app.models.tenant import Tenant from app.services.digital_employee_reminder_task import DigitalEmployeeReminderTaskService logger = get_logger("app.services.digital_employee_reminder_scheduler") @@ -64,16 +67,27 @@ class DigitalEmployeeReminderScheduler: def _refresh_reminders(self, *, reason: str) -> None: db = get_session_factory()() try: - result = DigitalEmployeeReminderTaskService(db).refresh_reminders( - source=AgentRunSource.SCHEDULE.value - ) - summary = result.get("summary") or {} - logger.info( - "Digital employee reminder scan generated reason=%s recipients=%s reminders=%s", - reason, - summary.get("recipient_count"), - summary.get("reminder_count"), + tenant_ids = list( + db.scalars( + select(Tenant.tenant_id) + .where(Tenant.status == "active") + .order_by(Tenant.tenant_id.asc()) + ).all() ) + for tenant_id in tenant_ids: + result = DigitalEmployeeReminderTaskService( + db, + tenant_id=tenant_id, + ).refresh_reminders(source=AgentRunSource.SCHEDULE.value) + summary = result.get("summary") or {} + logger.info( + "Digital employee reminder scan generated tenant=%s " + "reason=%s recipients=%s reminders=%s", + tenant_id, + reason, + summary.get("recipient_count"), + summary.get("reminder_count"), + ) except Exception: db.rollback() logger.exception("Scheduled digital employee reminder scan failed") diff --git a/server/src/app/services/digital_employee_reminder_task.py b/server/src/app/services/digital_employee_reminder_task.py index 592b729..f2bbc23 100644 --- a/server/src/app/services/digital_employee_reminder_task.py +++ b/server/src/app/services/digital_employee_reminder_task.py @@ -21,6 +21,7 @@ from app.models.employee import Employee from app.models.financial_record import ExpenseClaim from app.models.role import Role from app.services.agent_runs import AgentRunService +from app.services.tenant_registry import required_tenant_id DIGITAL_EMPLOYEE_REMINDER_TASK_TYPE = "digital_employee_reminder_scan" DIGITAL_EMPLOYEE_REMINDER_TOOL_NAME = "digital_employee.reminder.scan" @@ -35,8 +36,9 @@ DEFAULT_WINDOW_DAYS = 14 class DigitalEmployeeReminderTaskService: - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str) -> None: self.db = db + self.tenant_id = required_tenant_id(tenant_id) def refresh_reminders( self, @@ -50,6 +52,7 @@ class DigitalEmployeeReminderTaskService: run = run_service.create_run( agent=AgentName.HERMES.value, source=source, + tenant_id=self.tenant_id, user_id="digital_employee", ontology_json={"scenario": "financial_reminder", "intent": "scan"}, route_json={ @@ -165,7 +168,10 @@ class DigitalEmployeeReminderTaskService: stmt = ( select(ExpenseClaim) .options(selectinload(ExpenseClaim.employee).selectinload(Employee.manager)) - .where(ExpenseClaim.status.in_(APPROVAL_PENDING_STATUSES)) + .where( + ExpenseClaim.tenant_id == self.tenant_id, + ExpenseClaim.status.in_(APPROVAL_PENDING_STATUSES), + ) .order_by(ExpenseClaim.submitted_at.asc().nullslast(), ExpenseClaim.updated_at.asc()) .limit(200) ) @@ -191,19 +197,25 @@ class DigitalEmployeeReminderTaskService: fiscal_year = now.astimezone(UTC).year period_key = self._current_quarter_key(now) active_statuses = {"active", "published"} - year_count = self.db.scalar( - select(func.count(BudgetAllocation.id)).where( - BudgetAllocation.fiscal_year == fiscal_year, - BudgetAllocation.status.in_(active_statuses), + year_count = ( + self.db.scalar( + select(func.count(BudgetAllocation.id)).where( + BudgetAllocation.fiscal_year == fiscal_year, + BudgetAllocation.status.in_(active_statuses), + ) ) - ) or 0 - period_count = self.db.scalar( - select(func.count(BudgetAllocation.id)).where( - BudgetAllocation.fiscal_year == fiscal_year, - BudgetAllocation.period_key == period_key, - BudgetAllocation.status.in_(active_statuses), + or 0 + ) + period_count = ( + self.db.scalar( + select(func.count(BudgetAllocation.id)).where( + BudgetAllocation.fiscal_year == fiscal_year, + BudgetAllocation.period_key == period_key, + BudgetAllocation.status.in_(active_statuses), + ) ) - ) or 0 + or 0 + ) if year_count and period_count: return [] @@ -250,6 +262,7 @@ class DigitalEmployeeReminderTaskService: stmt = ( select(ExpenseClaim) .options(selectinload(ExpenseClaim.employee)) + .where(ExpenseClaim.tenant_id == self.tenant_id) .where(ExpenseClaim.expense_type.like("%_application")) .where(ExpenseClaim.status.in_(APPLICATION_ACTIVE_STATUSES)) .where(ExpenseClaim.occurred_at <= now) @@ -288,6 +301,7 @@ class DigitalEmployeeReminderTaskService: stmt = ( select(ExpenseClaim) .options(selectinload(ExpenseClaim.employee)) + .where(ExpenseClaim.tenant_id == self.tenant_id) .where(ExpenseClaim.status.in_(statuses)) .where(ExpenseClaim.updated_at >= cutoff) .order_by(ExpenseClaim.updated_at.asc()) @@ -430,7 +444,10 @@ class DigitalEmployeeReminderTaskService: select(Employee) .options(selectinload(Employee.roles)) .join(Employee.roles) - .where(Role.role_code.in_(("budget_monitor", "executive"))) + .where( + Employee.tenant_id == self.tenant_id, + Role.role_code.in_(("budget_monitor", "executive")), + ) .order_by(Employee.name.asc()) .limit(20) ) @@ -486,7 +503,10 @@ class DigitalEmployeeReminderTaskService: return True stmt = ( select(ExpenseClaim) - .where(ExpenseClaim.expense_type.not_like("%_application")) + .where( + ExpenseClaim.tenant_id == self.tenant_id, + ExpenseClaim.expense_type.not_like("%_application"), + ) .order_by(ExpenseClaim.created_at.desc()) .limit(300) ) diff --git a/server/src/app/services/employee.py b/server/src/app/services/employee.py index 05fecc3..c524760 100644 --- a/server/src/app/services/employee.py +++ b/server/src/app/services/employee.py @@ -29,13 +29,13 @@ from app.schemas.employee import ( EmployeeUpdate, ) from app.services.employee_bank_info import apply_default_bank_info +from app.services.employee_directory_maintenance import EmployeeDirectoryMaintenance from app.services.employee_import import EmployeeImportCoordinator from app.services.employee_schema import ensure_employee_schema from app.services.employee_seed import ( CANONICAL_DEPARTMENT_CODES, EMPLOYEE_DEFINITIONS, EMPLOYEE_PROFILE_REPAIRS, - LEGACY_ORGANIZATION_UNIT_CODE_MAP, ORGANIZATION_DEFINITIONS, ROLE_DEFINITIONS, ROLE_DISPLAY_ORDER, @@ -52,6 +52,11 @@ from app.services.employee_time import ( parse_date, parse_datetime, ) +from app.services.tenant_registry import ( + DEFAULT_TENANT_ID, + TenantRegistryService, + required_tenant_id, +) logger = get_logger("app.services.employee") DEFAULT_EMPLOYEE_PASSWORD = "123456" @@ -76,31 +81,38 @@ def prepare_employee_directory() -> None: session_factory = get_session_factory() with session_factory() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id=DEFAULT_TENANT_ID) service.ensure_directory_ready() service.apply_profile_repairs() class EmployeeService: _directory_ready_lock = threading.Lock() - _directory_ready_keys: set[tuple[str, int]] = set() + _directory_ready_keys: set[tuple[str, int, str]] = set() - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str) -> None: self.db = db - self.repository = EmployeeRepository(db) + self.tenant_id = required_tenant_id(tenant_id) + self.repository = EmployeeRepository(db, tenant_id=self.tenant_id) + self.directory_maintenance = EmployeeDirectoryMaintenance(db, self.repository) - @staticmethod - def _bind_cache_key(db: Session) -> tuple[str, int]: + def _bind_cache_key(self, db: Session) -> tuple[str, int, str]: bind = db.get_bind() - return (bind.url.render_as_string(hide_password=True), id(bind.pool)) + return ( + bind.url.render_as_string(hide_password=True), + id(bind.pool), + self.tenant_id, + ) def ensure_directory_ready(self) -> None: cache_key = self._bind_cache_key(self.db) if cache_key in self._directory_ready_keys: + self.directory_maintenance.normalize_after_cache_hit() return with self._directory_ready_lock: if cache_key in self._directory_ready_keys: + self.directory_maintenance.normalize_after_cache_hit() return self._ensure_directory_ready_uncached() @@ -110,22 +122,30 @@ class EmployeeService: try: create_legacy_schema(self.db.get_bind()) ensure_employee_schema(self.db) + tenant_registry = TenantRegistryService(self.db) + tenant_registry.ensure_builtin() + tenant_registry.require_active(self.tenant_id) self._prune_extra_seed_employees() self._seed_roles() self._seed_organization_units() self._seed_employees() - self._normalize_legacy_employee_departments() + self.directory_maintenance.normalize_legacy_departments() self._backfill_employee_bank_info() + self._ensure_tenant_memberships() self.db.commit() except Exception: self.db.rollback() logger.exception("Failed to prepare employee directory") raise - def list_employees(self, status: str | None = None, keyword: str | None = None) -> list[EmployeeRead]: + def list_employees( + self, status: str | None = None, keyword: str | None = None + ) -> list[EmployeeRead]: self.ensure_directory_ready() employees = self.repository.list(status=status, keyword=keyword) - logger.info("Listed employees (count=%d, status=%s, keyword=%s)", len(employees), status, keyword) + logger.info( + "Listed employees (count=%d, status=%s, keyword=%s)", len(employees), status, keyword + ) return [self._serialize_employee(item) for item in employees] def get_employee(self, employee_id: str) -> EmployeeRead | None: @@ -203,6 +223,7 @@ class EmployeeService: raise ValueError(f"邮箱 {payload.email} 已存在") employee = Employee( + tenant_id=self.tenant_id, employee_no=payload.employee_no, name=payload.name, email=str(payload.email), @@ -240,7 +261,12 @@ class EmployeeService: ] employee.roles = self._sorted_roles(roles) - created = self.repository.create(employee) + self.db.add(employee) + self.db.flush() + TenantRegistryService(self.db).ensure_employee_membership(employee) + self.db.commit() + self.db.refresh(employee) + created = employee logger.info( "Created employee id=%s no=%s name=%s", created.id, created.employee_no, created.name ) @@ -406,7 +432,9 @@ class EmployeeService: sorted_roles = self._sorted_roles(roles) next_role_codes = [role.role_code for role in sorted_roles] - current_role_codes = [role.role_code for role in self._sorted_roles(list(employee.roles))] + current_role_codes = [ + role.role_code for role in self._sorted_roles(list(employee.roles)) + ] if next_role_codes != current_role_codes: employee.roles = sorted_roles role_changed = True @@ -464,6 +492,8 @@ class EmployeeService: now = datetime.now(UTC) employee.employment_status = "停用" + for membership in employee.tenant_memberships: + membership.status = "inactive" employee.sync_state = "已同步" employee.last_sync_at = now employee.spotlight = False @@ -485,6 +515,7 @@ class EmployeeService: now = datetime.now(UTC) employee.employment_status = "在职" + TenantRegistryService(self.db).ensure_employee_membership(employee) employee.sync_state = "已同步" employee.last_sync_at = now self._append_change_log(employee, action="启用员工账号", occurred_at=now) @@ -501,9 +532,20 @@ class EmployeeService: self.ensure_directory_ready() return self._import_coordinator().export_employees(status=status, keyword=keyword) - def import_employees(self, content: bytes, actor: str = "系统管理员") -> EmployeeImportResultRead: + def import_employees( + self, content: bytes, actor: str = "系统管理员" + ) -> EmployeeImportResultRead: self.ensure_directory_ready() - return self._import_coordinator().import_employees(content, actor=actor) + try: + result = self._import_coordinator().import_employees(content, actor=actor) + if not result.success: + return result + self._ensure_tenant_memberships() + self.db.commit() + return result + except Exception: + self.db.rollback() + raise def _import_coordinator(self) -> EmployeeImportCoordinator: return EmployeeImportCoordinator( @@ -549,6 +591,7 @@ class EmployeeService: organization = existing_by_code.get(definition["unit_code"]) if organization is None: organization = OrganizationUnit( + tenant_id=self.tenant_id, unit_code=definition["unit_code"], name=definition["name"], unit_type=definition["unit_type"], @@ -579,31 +622,8 @@ class EmployeeService: self.db.flush() - def _normalize_legacy_employee_departments(self) -> None: - if not LEGACY_ORGANIZATION_UNIT_CODE_MAP: - return - - organizations_by_code = { - unit.unit_code: unit for unit in self.repository.list_organization_units() - } - for employee in self.repository.list(): - current_code = ( - employee.organization_unit.unit_code if employee.organization_unit else None - ) - next_code = normalize_organization_unit_code(current_code) - if not next_code or next_code == current_code: - continue - - organization = organizations_by_code.get(next_code) - if organization is not None: - employee.organization_unit = organization - - self.db.flush() - def _seed_employees(self) -> None: - employees_by_no = { - employee.employee_no: employee for employee in self.repository.list() - } + employees_by_no = {employee.employee_no: employee for employee in self.repository.list()} roles_by_code = {role.role_code: role for role in self.repository.list_roles()} organizations_by_code = { unit.unit_code: unit for unit in self.repository.list_organization_units() @@ -615,6 +635,7 @@ class EmployeeService: continue employee = Employee( + tenant_id=self.tenant_id, employee_no=employee_no, name=definition["name"], email=definition["email"], @@ -670,6 +691,11 @@ class EmployeeService: self.db.flush() + def _ensure_tenant_memberships(self) -> None: + registry = TenantRegistryService(self.db) + for employee in self.repository.list(): + registry.ensure_employee_membership(employee) + def apply_profile_repairs(self) -> None: """Apply one-off demo profile repairs. Intended for startup/bootstrap only.""" try: @@ -686,8 +712,12 @@ class EmployeeService: return employees = self.repository.list() - employees_by_email = {employee.email.lower(): employee for employee in employees if employee.email} - employees_by_no = {employee.employee_no: employee for employee in employees if employee.employee_no} + employees_by_email = { + employee.email.lower(): employee for employee in employees if employee.email + } + employees_by_no = { + employee.employee_no: employee for employee in employees if employee.employee_no + } roles_by_code = {role.role_code: role for role in self.repository.list_roles()} organizations_by_code = { unit.unit_code: unit for unit in self.repository.list_organization_units() @@ -729,7 +759,9 @@ class EmployeeService: apply_default_bank_info(employee) - role_codes = [item for item in definition.get("role_codes", []) if item in roles_by_code] + role_codes = [ + item for item in definition.get("role_codes", []) if item in roles_by_code + ] if role_codes: merged_roles = {role.role_code: role for role in employee.roles} for role_code in role_codes: @@ -847,4 +879,6 @@ class EmployeeService: ) def _sorted_roles(self, roles: list[Role]) -> list[Role]: - return sorted(roles, key=lambda item: (ROLE_DISPLAY_ORDER.get(item.role_code, 999), item.name)) + return sorted( + roles, key=lambda item: (ROLE_DISPLAY_ORDER.get(item.role_code, 999), item.name) + ) diff --git a/server/src/app/services/employee_behavior_profile_helpers.py b/server/src/app/services/employee_behavior_profile_helpers.py index 2b34b92..1ed7f25 100644 --- a/server/src/app/services/employee_behavior_profile_helpers.py +++ b/server/src/app/services/employee_behavior_profile_helpers.py @@ -5,6 +5,8 @@ from collections import defaultdict from decimal import Decimal from typing import Any +from sqlalchemy import select + from app.algorithem.employee_behavior_profile import ALGORITHM_VERSION from app.models.agent_run import AgentRun from app.models.employee import Employee @@ -128,7 +130,12 @@ class EmployeeBehaviorProfileMetricHelpers: normalized = str(expense_type_scope or "overall").strip() or "overall" if normalized != "overall" or not claim_id: return normalized - claim = self.db.get(ExpenseClaim, claim_id) + claim = self.db.scalar( + select(ExpenseClaim).where( + ExpenseClaim.tenant_id == self.tenant_id, + ExpenseClaim.id == claim_id, + ) + ) return str(claim.expense_type or "overall").strip() if claim is not None else normalized def _is_claim_in_scope(self, claim: ExpenseClaim, expense_type_scope: str) -> bool: diff --git a/server/src/app/services/employee_behavior_profile_service.py b/server/src/app/services/employee_behavior_profile_service.py index f603ddd..2b333be 100644 --- a/server/src/app/services/employee_behavior_profile_service.py +++ b/server/src/app/services/employee_behavior_profile_service.py @@ -4,8 +4,7 @@ from datetime import UTC, datetime, timedelta from decimal import Decimal from typing import Any -from sqlalchemy import func, or_, select -from sqlalchemy.orm import Session, selectinload +from sqlalchemy.orm import Session from app.algorithem.employee_behavior_profile import ( ALGORITHM_VERSION, @@ -22,8 +21,6 @@ from app.algorithem.employee_behavior_profile import ( ) from app.algorithem.employee_behavior_profile_tags import build_profile_radar, build_profile_tags from app.db.base import Base -from app.models.agent_run import AgentRun -from app.models.approval import ApprovalRecord from app.models.employee import Employee from app.models.employee_behavior_profile import EmployeeBehaviorProfileSnapshot from app.models.financial_record import ExpenseClaim @@ -39,16 +36,22 @@ from app.services.employee_behavior_profile_response import ( build_latest_review_suggestions, build_profile_payloads, ) +from app.services.employee_behavior_profile_storage import ( + ATTENTION_LEVELS, + EmployeeBehaviorProfileStorageMixin, +) +from app.services.finance_report_tenant import require_report_tenant_id -PROFILE_TYPES_FOR_APPROVAL = {"expense", "process_quality"} -ATTENTION_LEVELS = {"watch", "review", "escalation"} -PENDING_CLAIM_STATUSES = {"submitted", "review", "in_progress", "pending", "pending_review"} DEFAULT_WINDOWS = (30, 90, 180) -class EmployeeBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): - def __init__(self, db: Session) -> None: +class EmployeeBehaviorProfileService( + EmployeeBehaviorProfileStorageMixin, + EmployeeBehaviorProfileMetricHelpers, +): + def __init__(self, db: Session, *, tenant_id: str = "default") -> None: self.db = db + self.tenant_id = require_report_tenant_id(tenant_id) def ensure_storage_ready(self) -> None: Base.metadata.create_all( @@ -106,6 +109,8 @@ class EmployeeBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): employee = self._resolve_employee_by_identifier(requested_employee_id) if employee is None: return [] + if claim_id and not self._claim_belongs_to_employee(claim_id, employee.id): + raise LookupError("单据不存在或不属于当前租户员工。") now = datetime.now(UTC) snapshots: list[EmployeeBehaviorProfileSnapshot] = [] @@ -124,6 +129,7 @@ class EmployeeBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): self._calculate_approval_behavior_profile(context), ): snapshot = EmployeeBehaviorProfileSnapshot( + tenant_id=self.tenant_id, subject_type="employee", subject_id=employee.id, subject_name=employee.name, @@ -172,6 +178,8 @@ class EmployeeBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): expense_type_scope=expense_type_scope, empty_reason="员工不存在或尚未同步。", ) + if claim_id and not self._claim_belongs_to_employee(claim_id, employee.id): + raise LookupError("单据不存在或不属于当前租户员工。") resolved_scope = self._resolve_scope_from_claim(claim_id, expense_type_scope) resolved_employee_id = employee.id @@ -204,31 +212,6 @@ class EmployeeBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): expense_type_scope=resolved_scope, ) - def _resolve_employee_by_identifier(self, identifier: str) -> Employee | None: - normalized = str(identifier or "").strip() - if not normalized: - return None - - employee = self.db.get(Employee, normalized) - if employee is not None: - return employee - - normalized_email = normalized.lower() - conditions = [ - Employee.name == normalized, - Employee.employee_no == normalized, - ] - if "@" in normalized_email: - conditions.append(func.lower(Employee.email) == normalized_email) - - stmt = ( - select(Employee) - .where(or_(*conditions)) - .order_by(Employee.created_at.asc()) - .limit(1) - ) - return self.db.scalars(stmt).first() - def _build_window_context( self, *, @@ -628,56 +611,6 @@ class EmployeeBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): }, ) - def _load_latest_snapshots( - self, - *, - employee_id: str, - window_days: int, - expense_type_scope: str, - scene: str, - ) -> list[EmployeeBehaviorProfileSnapshot]: - allowed_types = PROFILE_TYPES_FOR_APPROVAL if scene == "approval" else None - rows = self._query_latest_rows( - employee_id=employee_id, - window_days=window_days, - expense_type_scope=expense_type_scope, - allowed_types=allowed_types, - ) - if rows or expense_type_scope == "overall": - return rows - return self._query_latest_rows( - employee_id=employee_id, - window_days=window_days, - expense_type_scope="overall", - allowed_types=allowed_types, - ) - - def _query_latest_rows( - self, - *, - employee_id: str, - window_days: int, - expense_type_scope: str, - allowed_types: set[str] | None, - ) -> list[EmployeeBehaviorProfileSnapshot]: - stmt = select(EmployeeBehaviorProfileSnapshot).where( - EmployeeBehaviorProfileSnapshot.subject_id == employee_id, - EmployeeBehaviorProfileSnapshot.window_days == window_days, - EmployeeBehaviorProfileSnapshot.expense_type_scope == expense_type_scope, - ) - if allowed_types: - stmt = stmt.where(EmployeeBehaviorProfileSnapshot.profile_type.in_(allowed_types)) - - rows = list( - self.db.scalars( - stmt.order_by(EmployeeBehaviorProfileSnapshot.calculated_at.desc()) - ).all() - ) - latest_by_type: dict[str, EmployeeBehaviorProfileSnapshot] = {} - for row in rows: - latest_by_type.setdefault(row.profile_type, row) - return list(latest_by_type.values()) - def _serialize_latest_profile( self, *, @@ -759,52 +692,6 @@ class EmployeeBehaviorProfileService(EmployeeBehaviorProfileMetricHelpers): review_suggestions=suggestions, ) - def _resolve_target_employee_ids(self, *, limit: int) -> list[str]: - cutoff = datetime.now(UTC) - timedelta(days=180) - claim_stmt = select(ExpenseClaim.employee_id).where( - ExpenseClaim.employee_id.is_not(None), - or_( - ExpenseClaim.occurred_at >= cutoff, - ExpenseClaim.status.in_(PENDING_CLAIM_STATUSES), - ), - ) - snapshot_stmt = select(EmployeeBehaviorProfileSnapshot.subject_id).where( - EmployeeBehaviorProfileSnapshot.profile_level.in_(ATTENTION_LEVELS) - ) - ordered: list[str] = [] - for value in [*self.db.scalars(claim_stmt).all(), *self.db.scalars(snapshot_stmt).all()]: - employee_id = str(value or "").strip() - if employee_id and employee_id not in ordered: - ordered.append(employee_id) - if len(ordered) >= limit: - break - return ordered - - def _fetch_claims_since(self, cutoff: datetime) -> list[ExpenseClaim]: - stmt = ( - select(ExpenseClaim) - .options(selectinload(ExpenseClaim.items), selectinload(ExpenseClaim.employee)) - .where(ExpenseClaim.occurred_at >= cutoff) - ) - return list(self.db.scalars(stmt).all()) - - def _fetch_agent_runs(self, identifiers: set[str], cutoff: datetime) -> list[AgentRun]: - if not identifiers: - return [] - stmt = ( - select(AgentRun) - .options(selectinload(AgentRun.tool_calls)) - .where(AgentRun.started_at >= cutoff, AgentRun.user_id.in_(identifiers)) - ) - return list(self.db.scalars(stmt).all()) - - def _fetch_approval_records(self, employee_id: str, cutoff: datetime) -> list[ApprovalRecord]: - stmt = select(ApprovalRecord).where( - ApprovalRecord.approver_id == employee_id, - ApprovalRecord.created_at >= cutoff, - ) - return list(self.db.scalars(stmt).all()) - def _resolve_peer_claims( self, *, diff --git a/server/src/app/services/employee_behavior_profile_storage.py b/server/src/app/services/employee_behavior_profile_storage.py new file mode 100644 index 0000000..23b4105 --- /dev/null +++ b/server/src/app/services/employee_behavior_profile_storage.py @@ -0,0 +1,187 @@ +from __future__ import annotations + +from datetime import UTC, datetime, timedelta + +from sqlalchemy import func, or_, select +from sqlalchemy.orm import selectinload + +from app.models.agent_run import AgentRun +from app.models.approval import ApprovalRecord +from app.models.employee import Employee +from app.models.employee_behavior_profile import EmployeeBehaviorProfileSnapshot +from app.models.financial_record import ExpenseClaim + +PROFILE_TYPES_FOR_APPROVAL = {"expense", "process_quality"} +ATTENTION_LEVELS = {"watch", "review", "escalation"} +PENDING_CLAIM_STATUSES = { + "submitted", + "review", + "in_progress", + "pending", + "pending_review", +} + + +class EmployeeBehaviorProfileStorageMixin: + """所有画像数据入口都在首个 SQL 收窄到可信租户。""" + + def _resolve_employee_by_identifier(self, identifier: str) -> Employee | None: + normalized = str(identifier or "").strip() + if not normalized: + return None + normalized_email = normalized.lower() + conditions = [ + Employee.id == normalized, + Employee.name == normalized, + Employee.employee_no == normalized, + ] + if "@" in normalized_email: + conditions.append(func.lower(Employee.email) == normalized_email) + return self.db.scalars( + select(Employee) + .where( + Employee.tenant_id == self.tenant_id, + or_(*conditions), + ) + .order_by(Employee.created_at.asc()) + .limit(1) + ).first() + + def _load_latest_snapshots( + self, + *, + employee_id: str, + window_days: int, + expense_type_scope: str, + scene: str, + ) -> list[EmployeeBehaviorProfileSnapshot]: + allowed_types = PROFILE_TYPES_FOR_APPROVAL if scene == "approval" else None + rows = self._query_latest_rows( + employee_id=employee_id, + window_days=window_days, + expense_type_scope=expense_type_scope, + allowed_types=allowed_types, + ) + if rows or expense_type_scope == "overall": + return rows + return self._query_latest_rows( + employee_id=employee_id, + window_days=window_days, + expense_type_scope="overall", + allowed_types=allowed_types, + ) + + def _query_latest_rows( + self, + *, + employee_id: str, + window_days: int, + expense_type_scope: str, + allowed_types: set[str] | None, + ) -> list[EmployeeBehaviorProfileSnapshot]: + stmt = select(EmployeeBehaviorProfileSnapshot).where( + EmployeeBehaviorProfileSnapshot.tenant_id == self.tenant_id, + EmployeeBehaviorProfileSnapshot.subject_id == employee_id, + EmployeeBehaviorProfileSnapshot.window_days == window_days, + EmployeeBehaviorProfileSnapshot.expense_type_scope == expense_type_scope, + ) + if allowed_types: + stmt = stmt.where(EmployeeBehaviorProfileSnapshot.profile_type.in_(allowed_types)) + rows = list( + self.db.scalars( + stmt.order_by(EmployeeBehaviorProfileSnapshot.calculated_at.desc()) + ).all() + ) + latest_by_type: dict[str, EmployeeBehaviorProfileSnapshot] = {} + for row in rows: + latest_by_type.setdefault(row.profile_type, row) + return list(latest_by_type.values()) + + def _claim_belongs_to_employee(self, claim_id: str, employee_id: str) -> bool: + if not str(claim_id or "").strip(): + return True + return bool( + self.db.scalar( + select(ExpenseClaim.id).where( + ExpenseClaim.tenant_id == self.tenant_id, + ExpenseClaim.id == str(claim_id).strip(), + ExpenseClaim.employee_id == str(employee_id).strip(), + ) + ) + ) + + def _resolve_target_employee_ids(self, *, limit: int) -> list[str]: + cutoff = datetime.now(UTC) - timedelta(days=180) + claim_stmt = select(ExpenseClaim.employee_id).where( + ExpenseClaim.tenant_id == self.tenant_id, + ExpenseClaim.employee_id.is_not(None), + or_( + ExpenseClaim.occurred_at >= cutoff, + ExpenseClaim.status.in_(PENDING_CLAIM_STATUSES), + ), + ) + snapshot_stmt = select(EmployeeBehaviorProfileSnapshot.subject_id).where( + EmployeeBehaviorProfileSnapshot.tenant_id == self.tenant_id, + EmployeeBehaviorProfileSnapshot.profile_level.in_(ATTENTION_LEVELS), + ) + ordered: list[str] = [] + values = [ + *self.db.scalars(claim_stmt).all(), + *self.db.scalars(snapshot_stmt).all(), + ] + for value in values: + employee_id = str(value or "").strip() + if employee_id and employee_id not in ordered: + ordered.append(employee_id) + if len(ordered) >= limit: + break + return ordered + + def _fetch_claims_since(self, cutoff: datetime) -> list[ExpenseClaim]: + stmt = ( + select(ExpenseClaim) + .options( + selectinload(ExpenseClaim.items), + selectinload(ExpenseClaim.employee), + ) + .where( + ExpenseClaim.tenant_id == self.tenant_id, + ExpenseClaim.occurred_at >= cutoff, + ) + ) + return list(self.db.scalars(stmt).all()) + + def _fetch_agent_runs( + self, + identifiers: set[str], + cutoff: datetime, + ) -> list[AgentRun]: + if not identifiers: + return [] + stmt = ( + select(AgentRun) + .options(selectinload(AgentRun.tool_calls)) + .where( + AgentRun.route_json["tenant_id"].as_string() == self.tenant_id, + AgentRun.ontology_json["tenant_id"].as_string() == self.tenant_id, + AgentRun.started_at >= cutoff, + AgentRun.user_id.in_(identifiers), + ) + ) + return list(self.db.scalars(stmt).all()) + + def _fetch_approval_records( + self, + employee_id: str, + cutoff: datetime, + ) -> list[ApprovalRecord]: + stmt = ( + select(ApprovalRecord) + .join(Employee, Employee.id == ApprovalRecord.approver_id) + .where( + Employee.tenant_id == self.tenant_id, + ApprovalRecord.approver_id == employee_id, + ApprovalRecord.created_at >= cutoff, + ) + ) + return list(self.db.scalars(stmt).all()) diff --git a/server/src/app/services/employee_directory_maintenance.py b/server/src/app/services/employee_directory_maintenance.py new file mode 100644 index 0000000..5b99027 --- /dev/null +++ b/server/src/app/services/employee_directory_maintenance.py @@ -0,0 +1,64 @@ +from __future__ import annotations + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.core.logging import get_logger +from app.models.employee import Employee +from app.repositories.employee import EmployeeRepository +from app.services.employee_seed import LEGACY_ORGANIZATION_UNIT_CODE_MAP + +logger = get_logger("app.services.employee_directory_maintenance") + + +class EmployeeDirectoryMaintenance: + """维护可能在目录初始化完成后出现的历史组织数据。""" + + def __init__(self, db: Session, repository: EmployeeRepository) -> None: + self.db = db + self.repository = repository + self.tenant_id = repository.tenant_id + + def normalize_after_cache_hit(self) -> None: + """建表与种子可缓存，外部同步产生的旧部门编码仍需持续归一化。""" + try: + if self.normalize_legacy_departments(): + self.db.commit() + except Exception: + self.db.rollback() + logger.exception("Failed to normalize cached employee departments") + raise + + def normalize_legacy_departments(self) -> bool: + if not LEGACY_ORGANIZATION_UNIT_CODE_MAP: + return False + + organizations_by_code = { + unit.unit_code: unit for unit in self.repository.list_organization_units() + } + legacy_target_by_id = { + organization.id: organizations_by_code.get(target_code) + for legacy_code, target_code in LEGACY_ORGANIZATION_UNIT_CODE_MAP.items() + if (organization := organizations_by_code.get(legacy_code)) is not None + } + if not legacy_target_by_id: + return False + + employees = list( + self.db.scalars( + select(Employee).where( + Employee.tenant_id == self.tenant_id, + Employee.organization_unit_id.in_(legacy_target_by_id), + ) + ).all() + ) + changed = False + for employee in employees: + organization = legacy_target_by_id.get(employee.organization_unit_id) + if organization is None or employee.organization_unit_id == organization.id: + continue + employee.organization_unit_id = organization.id + changed = True + + self.db.flush() + return changed diff --git a/server/src/app/services/employee_import.py b/server/src/app/services/employee_import.py index 8bbb7dd..3034a74 100644 --- a/server/src/app/services/employee_import.py +++ b/server/src/app/services/employee_import.py @@ -15,14 +15,14 @@ from app.schemas.employee import ( EmployeeImportResultRead, EmployeeImportSummaryRead, ) +from app.services.employee_bank_info import apply_default_bank_info +from app.services.employee_seed import normalize_organization_unit_code from app.services.employee_spreadsheet import ( EmployeeImportRow, EmployeeSpreadsheetError, build_export_workbook_bytes, parse_employee_workbook, ) -from app.services.employee_seed import normalize_organization_unit_code -from app.services.employee_bank_info import apply_default_bank_info logger = get_logger("app.services.employee") @@ -53,7 +53,9 @@ class EmployeeImportCoordinator: for employee in employees: organization = employee.organization_unit - role_codes = ",".join(role.role_code for role in self.sorted_roles(list(employee.roles))) + role_codes = ",".join( + role.role_code for role in self.sorted_roles(list(employee.roles)) + ) organization_code = ( normalize_organization_unit_code(organization.unit_code) if organization else "" ) @@ -83,7 +85,9 @@ class EmployeeImportCoordinator: return build_export_workbook_bytes(rows) - def import_employees(self, content: bytes, actor: str = "系统管理员") -> EmployeeImportResultRead: + def import_employees( + self, content: bytes, actor: str = "系统管理员" + ) -> EmployeeImportResultRead: parsed_rows, parse_errors = parse_employee_workbook(content) if parse_errors: return self._build_import_failure(parse_errors, total_rows=len(parsed_rows)) @@ -131,9 +135,7 @@ class EmployeeImportCoordinator: organizations_by_code = { unit.unit_code: unit for unit in self.repository.list_organization_units() } - employees_by_no = { - employee.employee_no: employee for employee in self.repository.list() - } + employees_by_no = {employee.employee_no: employee for employee in self.repository.list()} import_employee_nos = {row.employee_no for row in rows} for row in rows: @@ -169,8 +171,7 @@ class EmployeeImportCoordinator: column="邮箱*", employee_no=row.employee_no, message=( - f"邮箱 {row.email} 已被员工 " - f"{existing_by_email.employee_no} 使用。" + f"邮箱 {row.email} 已被员工 {existing_by_email.employee_no} 使用。" ), ) ) @@ -210,9 +211,7 @@ class EmployeeImportCoordinator: ) ) - invalid_role_codes = [ - code for code in row.role_codes if code not in roles_by_code - ] + invalid_role_codes = [code for code in row.role_codes if code not in roles_by_code] if invalid_role_codes: errors.append( EmployeeSpreadsheetError( @@ -235,80 +234,72 @@ class EmployeeImportCoordinator: organizations_by_code = { unit.unit_code: unit for unit in self.repository.list_organization_units() } - employees_by_no = { - employee.employee_no: employee for employee in self.repository.list() - } + employees_by_no = {employee.employee_no: employee for employee in self.repository.list()} created = 0 updated = 0 now = datetime.now(UTC) - try: - for row in rows: - employee = employees_by_no.get(row.employee_no) - is_new = employee is None + for row in rows: + employee = employees_by_no.get(row.employee_no) + is_new = employee is None - if is_new: - employee = Employee( - employee_no=row.employee_no, - name=row.name, - email=row.email, - password_hash=hash_password(self.default_password), - ) - self.db.add(employee) - employees_by_no[row.employee_no] = employee - created += 1 - else: - updated += 1 - - employee.name = row.name - employee.email = row.email - employee.gender = row.gender - employee.birth_date = row.birth_date - employee.phone = row.phone - employee.join_date = row.join_date - employee.location = row.location - employee.position = row.position - employee.grade = row.grade - employee.finance_owner_name = row.finance_owner_name - employee.cost_center = row.cost_center - employee.bank_account_name = row.bank_account_name - employee.bank_name = row.bank_name - employee.bank_account_no = row.bank_account_no - employee.employment_status = row.employment_status - employee.sync_state = "已同步" - employee.last_sync_at = now - apply_default_bank_info(employee) - - organization_code = normalize_organization_unit_code(row.organization_unit_code) - if organization_code: - employee.organization_unit = organizations_by_code[organization_code] - else: - employee.organization_unit = None - - employee.roles = self.sorted_roles( - [roles_by_code[code] for code in row.role_codes if code in roles_by_code] + if is_new: + employee = Employee( + tenant_id=self.repository.tenant_id, + employee_no=row.employee_no, + name=row.name, + email=row.email, + password_hash=hash_password(self.default_password), ) + self.db.add(employee) + employees_by_no[row.employee_no] = employee + created += 1 + else: + updated += 1 - action = ( - "通过 Excel 导入新建员工档案" - if is_new - else "通过 Excel 导入更新员工档案" - ) - self.append_change_log(employee, action=action, owner=actor, occurred_at=now) + employee.name = row.name + employee.email = row.email + employee.gender = row.gender + employee.birth_date = row.birth_date + employee.phone = row.phone + employee.join_date = row.join_date + employee.location = row.location + employee.position = row.position + employee.grade = row.grade + employee.finance_owner_name = row.finance_owner_name + employee.cost_center = row.cost_center + employee.bank_account_name = row.bank_account_name + employee.bank_name = row.bank_name + employee.bank_account_no = row.bank_account_no + employee.employment_status = row.employment_status + employee.sync_state = "已同步" + employee.last_sync_at = now + apply_default_bank_info(employee) - self.db.flush() + organization_code = normalize_organization_unit_code(row.organization_unit_code) + if organization_code: + employee.organization_unit = organizations_by_code[organization_code] + else: + employee.organization_unit = None - for row in rows: - employee = employees_by_no[row.employee_no] - if row.manager_employee_no: - employee.manager = employees_by_no.get(row.manager_employee_no) - else: - employee.manager = None + employee.roles = self.sorted_roles( + [roles_by_code[code] for code in row.role_codes if code in roles_by_code] + ) - self.db.commit() - except Exception: - self.db.rollback() - raise + action = "通过 Excel 导入新建员工档案" if is_new else "通过 Excel 导入更新员工档案" + self.append_change_log(employee, action=action, owner=actor, occurred_at=now) + + self.db.flush() + + for row in rows: + employee = employees_by_no[row.employee_no] + if row.manager_employee_no: + employee.manager = employees_by_no.get(row.manager_employee_no) + else: + employee.manager = None + + # 导入数据和租户成员资格由 EmployeeService 在同一事务统一提交。 + self.db.flush() return {"created": created, "updated": updated} @@ -342,4 +333,3 @@ class EmployeeImportCoordinator: errors=error_reads, importedAt=None, ) - diff --git a/server/src/app/services/employee_pagination.py b/server/src/app/services/employee_pagination.py index 0bfbd1c..cd40a7f 100644 --- a/server/src/app/services/employee_pagination.py +++ b/server/src/app/services/employee_pagination.py @@ -11,8 +11,8 @@ logger = get_logger("app.services.employee") class EmployeePaginationService: - def __init__(self, db: Session) -> None: - self.service = EmployeeService(db) + def __init__(self, db: Session, *, tenant_id: str) -> None: + self.service = EmployeeService(db, tenant_id=tenant_id) def list_employees_page( self, diff --git a/server/src/app/services/employee_profile_scan_task.py b/server/src/app/services/employee_profile_scan_task.py index 66ae7ba..80e9bd1 100644 --- a/server/src/app/services/employee_profile_scan_task.py +++ b/server/src/app/services/employee_profile_scan_task.py @@ -14,6 +14,7 @@ from app.core.agent_enums import ( AgentToolType, ) from app.services.agent_runs import AgentRunService +from app.services.finance_report_tenant import require_report_tenant_id from app.services.hermes_employee_profile_scanner import HermesEmployeeProfileScannerService EMPLOYEE_PROFILE_SCAN_TASK_TYPE = "employee_behavior_profile_scan" @@ -24,11 +25,18 @@ class EmployeeProfileScanTaskService: def __init__(self, db: Session) -> None: self.db = db - def refresh_profiles(self, *, source: str = AgentRunSource.SCHEDULE.value) -> dict[str, Any]: + def refresh_profiles( + self, + *, + source: str = AgentRunSource.SCHEDULE.value, + tenant_id: str = "default", + ) -> dict[str, Any]: + tenant = require_report_tenant_id(tenant_id) run_service = AgentRunService(self.db) run = run_service.create_run( agent=AgentName.HERMES.value, source=source, + tenant_id=tenant, user_id="digital_employee", ontology_json={ "scenario": "employee_behavior_profile", @@ -49,7 +57,8 @@ class EmployeeProfileScanTaskService: # 画像快照表的 source_task_log_id 外键指向 Hermes 任务日志。 # 这里用 agent_runs 记录数字员工轨迹，因此不写入该外键，避免错误关联。 summary = HermesEmployeeProfileScannerService(self.db).scan_employee_profiles( - log_id=None + log_id=None, + tenant_id=tenant, ) duration_ms = int((perf_counter() - timer) * 1000) report = self._build_report(summary) @@ -62,7 +71,10 @@ class EmployeeProfileScanTaskService: run_id=run.run_id, tool_type=AgentToolType.DATABASE.value, tool_name=EMPLOYEE_PROFILE_SCAN_TOOL_NAME, - request_json={"task_type": EMPLOYEE_PROFILE_SCAN_TASK_TYPE}, + request_json={ + "task_type": EMPLOYEE_PROFILE_SCAN_TASK_TYPE, + "tenant_id": tenant, + }, response_json=response, status=AgentRunStatus.SUCCEEDED.value, duration_ms=duration_ms, @@ -90,7 +102,10 @@ class EmployeeProfileScanTaskService: run_id=run.run_id, tool_type=AgentToolType.DATABASE.value, tool_name=EMPLOYEE_PROFILE_SCAN_TOOL_NAME, - request_json={"task_type": EMPLOYEE_PROFILE_SCAN_TASK_TYPE}, + request_json={ + "task_type": EMPLOYEE_PROFILE_SCAN_TASK_TYPE, + "tenant_id": tenant, + }, response_json={}, status=AgentRunStatus.FAILED.value, duration_ms=int((perf_counter() - timer) * 1000), @@ -114,9 +129,7 @@ class EmployeeProfileScanTaskService: "title": "员工财务行为画像扫描报告", "targetEmployeeCount": int(summary.get("target_employee_count") or 0), "profileSnapshotCount": int(summary.get("snapshot_count") or 0), - "highAttentionEmployeeCount": int( - summary.get("high_attention_employee_count") or 0 - ), + "highAttentionEmployeeCount": int(summary.get("high_attention_employee_count") or 0), "windowDays": list(summary.get("window_days") or []), "algorithmVersion": str(summary.get("algorithm_version") or ""), "baselineSummary": summary.get("baseline_summary") or {}, diff --git a/server/src/app/services/employee_profile_scheduler.py b/server/src/app/services/employee_profile_scheduler.py index 32e1e05..b8d07be 100644 --- a/server/src/app/services/employee_profile_scheduler.py +++ b/server/src/app/services/employee_profile_scheduler.py @@ -5,9 +5,12 @@ import threading from datetime import datetime from zoneinfo import ZoneInfo +from sqlalchemy import select + from app.core.agent_enums import AgentRunSource from app.core.logging import get_logger from app.db.session import get_session_factory +from app.models.tenant import Tenant from app.services.employee_profile_scan_task import EmployeeProfileScanTaskService logger = get_logger("app.services.employee_profile_scheduler") @@ -67,17 +70,24 @@ class EmployeeProfileScheduler: def _refresh_profiles(self) -> None: db = get_session_factory()() try: - result = EmployeeProfileScanTaskService(db).refresh_profiles( - source=AgentRunSource.SCHEDULE.value - ) - summary = result.get("summary") or {} - logger.info( - "Employee profile scan generated at=%s employees=%s snapshots=%s attention=%s", - datetime.now(self._timezone).isoformat(), - summary.get("target_employee_count"), - summary.get("snapshot_count"), - summary.get("high_attention_employee_count"), + tenant_ids = list( + db.scalars(select(Tenant.tenant_id).where(Tenant.status == "active")).all() ) + for tenant_id in tenant_ids: + result = EmployeeProfileScanTaskService(db).refresh_profiles( + source=AgentRunSource.SCHEDULE.value, + tenant_id=tenant_id, + ) + summary = result.get("summary") or {} + logger.info( + "Employee profile scan generated tenant=%s at=%s " + "employees=%s snapshots=%s attention=%s", + tenant_id, + datetime.now(self._timezone).isoformat(), + summary.get("target_employee_count"), + summary.get("snapshot_count"), + summary.get("high_attention_employee_count"), + ) except Exception: db.rollback() raise diff --git a/server/src/app/services/expense_application_learning.py b/server/src/app/services/expense_application_learning.py index 6e162e6..4fb3355 100644 --- a/server/src/app/services/expense_application_learning.py +++ b/server/src/app/services/expense_application_learning.py @@ -28,6 +28,7 @@ from app.services.expense_application_snapshot import ( snapshot_reference, ) from app.services.expense_cases import ExpenseCaseService +from app.services.expense_workflow_learning import ExpenseWorkflowLearningService logger = logging.getLogger(__name__) @@ -85,6 +86,11 @@ class ExpenseApplicationLearningService: preview_decision, final_values, ) + explicit_rejection = self._all_server_suggestions_cleared( + preview_decision, + final_values, + changed_field_references, + ) suggestion_reference = { "field_keys": list(preview_decision.field_keys_json or []), "value_fingerprint": preview_decision.snapshot_fingerprint, @@ -95,6 +101,10 @@ class ExpenseApplicationLearningService: else: final_reference = self._snapshot_reference(final_values) changed_fields = self._safe_changed_fields(payload, final_values) + explicit_rejection = bool(changed_fields) and all( + item["suggested_value"] and not item["final_value"] + for item in changed_fields + ) suggested_values = dict(final_values) for item in changed_fields: suggested_values[item["field_key"]] = item["suggested_value"] @@ -107,7 +117,11 @@ class ExpenseApplicationLearningService: ) verification_status = "client_observed" - feedback_type = "edited" if changed_field_references else "accepted" + feedback_type = ( + "rejected" + if explicit_rejection + else ("edited" if changed_field_references else "accepted") + ) correlation_id = ExpenseCaseService.normalize_correlation_id(payload.run_id) fingerprint_payload = { "tenant_id": tenant_id, @@ -126,6 +140,9 @@ class ExpenseApplicationLearningService: ) existing = self._find_existing(tenant_id, idempotency_key) if existing is not None: + ExpenseWorkflowLearningService(self.db).record_prior_events_for_decision( + existing.decision + ) self._record_transport_memory_evidence( current_user=current_user, records=existing, @@ -236,6 +253,7 @@ class ExpenseApplicationLearningService: ) self.db.add_all([decision, feedback, outcome]) self.db.flush() + ExpenseWorkflowLearningService(self.db).record_prior_events_for_decision(decision) records = ExpenseApplicationLearningRecords(decision, feedback, outcome) self._record_transport_memory_evidence( current_user=current_user, @@ -290,20 +308,23 @@ class ExpenseApplicationLearningService: ) if decision is None: return None - feedback = self.db.scalar( - select(AIDecisionFeedback).where( - AIDecisionFeedback.tenant_id == tenant_id, - AIDecisionFeedback.decision_id == decision.id, - ) + feedback = self.db.get( + AIDecisionFeedback, + self._stable_id("feedback", tenant_id, idempotency_key), ) - outcome = self.db.scalar( - select(WorkflowOutcome).where( - WorkflowOutcome.tenant_id == tenant_id, - WorkflowOutcome.decision_id == decision.id, - ) + outcome = self.db.get( + WorkflowOutcome, + self._stable_id("outcome", tenant_id, idempotency_key), ) if feedback is None or outcome is None: raise RuntimeError("AI 学习账本幂等状态不完整，请重试。") + if ( + feedback.tenant_id != tenant_id + or feedback.decision_id != decision.id + or outcome.tenant_id != tenant_id + or outcome.decision_id != decision.id + ): + raise RuntimeError("AI 学习账本幂等记录关联不一致，请联系管理员。") return ExpenseApplicationLearningRecords(decision, feedback, outcome) @classmethod @@ -422,6 +443,21 @@ class ExpenseApplicationLearningService: for item in changes ] + @staticmethod + def _all_server_suggestions_cleared( + decision: AIApplicationPreviewDecision, + final_values: dict[str, str], + changes: list[dict[str, str]], + ) -> bool: + if not changes: + return False + suggested_keys = set(decision.field_keys_json or []) + return all( + item["field_key"] in suggested_keys + and not str(final_values.get(item["field_key"]) or "").strip() + for item in changes + ) + @staticmethod def _decision_source(preview: dict[str, Any], facts: dict[str, str]) -> str: if preview.get("modelRefined") and facts.get("rule_version"): diff --git a/server/src/app/services/expense_cases.py b/server/src/app/services/expense_cases.py index 147f90a..cd1cd32 100644 --- a/server/src/app/services/expense_cases.py +++ b/server/src/app/services/expense_cases.py @@ -315,6 +315,7 @@ class ExpenseCaseService: ) ) if existing_event is not None: + self._record_workflow_learning(existing_event) return expense_case, existing_event event = BusinessEvent( @@ -336,8 +337,18 @@ class ExpenseCaseService: ) self.db.add(event) self.db.flush() + self._record_workflow_learning(event) return expense_case, event + def _record_workflow_learning(self, event: BusinessEvent) -> None: + """在业务事件事务内同步生成可信 AI 结果证据，未命中事件会立即返回。""" + + from app.services.expense_workflow_learning import ( + ExpenseWorkflowLearningService, + ) + + ExpenseWorkflowLearningService(self.db).record_event(event) + @staticmethod def _normalize_idempotency_key(value: str) -> str: text = str(value or "").strip() diff --git a/server/src/app/services/expense_claim_access_policy.py b/server/src/app/services/expense_claim_access_policy.py index 86b9c34..ec836d0 100644 --- a/server/src/app/services/expense_claim_access_policy.py +++ b/server/src/app/services/expense_claim_access_policy.py @@ -12,6 +12,7 @@ from app.models.financial_record import ExpenseClaim from app.models.organization import OrganizationUnit from app.models.role import Role from app.services.document_numbering import is_application_claim_no +from app.services.expense_claim_employee_resolver import ExpenseClaimEmployeeResolverMixin from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin from app.services.expense_claim_workflow_constants import ( APPLICATION_ARCHIVE_STAGE, @@ -39,7 +40,10 @@ ARCHIVED_REIMBURSEMENT_STAGES = ( ) -class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): +class ExpenseClaimAccessPolicy( + ExpenseClaimEmployeeResolverMixin, + ExpenseClaimTenantScopeMixin, +): def __init__(self, db: Session) -> None: self.db = db @@ -55,13 +59,18 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): def has_privileged_claim_access(current_user: CurrentUserContext) -> bool: if current_user.is_admin: return True - return bool(ExpenseClaimAccessPolicy.normalize_role_codes(current_user) & PRIVILEGED_CLAIM_ROLE_CODES) + return bool( + ExpenseClaimAccessPolicy.normalize_role_codes(current_user) + & PRIVILEGED_CLAIM_ROLE_CODES + ) @staticmethod def has_archive_center_access(current_user: CurrentUserContext) -> bool: if current_user.is_admin: return True - return bool(ExpenseClaimAccessPolicy.normalize_role_codes(current_user) & ARCHIVE_CENTER_ROLE_CODES) + return bool( + ExpenseClaimAccessPolicy.normalize_role_codes(current_user) & ARCHIVE_CENTER_ROLE_CODES + ) @staticmethod def build_archived_claim_condition() -> Any: @@ -116,10 +125,16 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): or normalized_type.endswith("_application") ) if is_application_claim: - return normalized_status in ARCHIVED_CLAIM_STATUSES and stage in APPLICATION_ARCHIVED_STAGES + return ( + normalized_status in ARCHIVED_CLAIM_STATUSES + and stage in APPLICATION_ARCHIVED_STAGES + ) if stage in set(ARCHIVED_REIMBURSEMENT_STAGES): return True - return normalized_status in ARCHIVED_CLAIM_STATUSES and stage in {"", *ARCHIVED_REIMBURSEMENT_STAGES} + return normalized_status in ARCHIVED_CLAIM_STATUSES and stage in { + "", + *ARCHIVED_REIMBURSEMENT_STAGES, + } def can_return_claim(self, current_user: CurrentUserContext, claim: ExpenseClaim) -> bool: normalized_status = str(claim.status or "").strip().lower() @@ -132,7 +147,9 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): if stage == BUDGET_MANAGER_APPROVAL_STAGE: return self.is_budget_manager_approver(current_user, claim) if stage == FINANCE_APPROVAL_STAGE: - return self.has_privileged_claim_access(current_user) and not self.is_claim_owned_by_current_user( + return self.has_privileged_claim_access( + current_user + ) and not self.is_claim_owned_by_current_user( claim, current_user, ) @@ -147,9 +164,8 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): if stage == FINANCE_APPROVAL_STAGE: role_codes = self.normalize_role_codes(current_user) return ( - (current_user.is_admin or "finance" in role_codes) - and not self.is_claim_owned_by_current_user(claim, current_user) - ) + current_user.is_admin or "finance" in role_codes + ) and not self.is_claim_owned_by_current_user(claim, current_user) return False def can_mark_claim_paid(self, current_user: CurrentUserContext, claim: ExpenseClaim) -> bool: @@ -161,7 +177,9 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): return True return bool(self.normalize_role_codes(current_user) & PRIVILEGED_CLAIM_ROLE_CODES) - def is_current_direct_manager_approver(self, current_user: CurrentUserContext, claim: ExpenseClaim) -> bool: + def is_current_direct_manager_approver( + self, current_user: CurrentUserContext, claim: ExpenseClaim + ) -> bool: role_codes = self.normalize_role_codes(current_user) if not (role_codes & APPROVAL_VISIBLE_CLAIM_ROLE_CODES): return False @@ -171,25 +189,35 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): return False current_employee = self.resolve_current_employee(current_user) - if current_employee is not None and str(claim.employee_id or "").strip() == current_employee.id: + if ( + current_employee is not None + and str(claim.employee_id or "").strip() == current_employee.id + ): return False claim_employee = claim.employee if current_employee is not None and claim_employee is not None: if claim_employee.manager_id == current_employee.id: return True - if claim_employee.manager is not None and claim_employee.manager.id == current_employee.id: + if ( + claim_employee.manager is not None + and claim_employee.manager.id == current_employee.id + ): return True approver_name = str( - current_employee.name if current_employee is not None and current_employee.name else current_user.name or "" + current_employee.name + if current_employee is not None and current_employee.name + else current_user.name or "" ).strip() if not approver_name: return False return self.resolve_claim_manager_name(claim) == approver_name - def is_budget_manager_approver(self, current_user: CurrentUserContext, claim: ExpenseClaim) -> bool: + def is_budget_manager_approver( + self, current_user: CurrentUserContext, claim: ExpenseClaim + ) -> bool: if str(claim.status or "").strip().lower() != "submitted": return False if str(claim.approval_stage or "").strip() != BUDGET_MANAGER_APPROVAL_STAGE: @@ -206,10 +234,14 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): role_codes = self.normalize_role_codes(current_user) return bool(role_codes & BUDGET_APPROVAL_ROLE_CODES) - def is_department_p8_budget_monitor(self, current_user: CurrentUserContext, claim: ExpenseClaim) -> bool: + def is_department_p8_budget_monitor( + self, current_user: CurrentUserContext, claim: ExpenseClaim + ) -> bool: return self.is_department_budget_approver(current_user, claim) - def is_department_budget_approver(self, current_user: CurrentUserContext, claim: ExpenseClaim) -> bool: + def is_department_budget_approver( + self, current_user: CurrentUserContext, claim: ExpenseClaim + ) -> bool: role_codes = self.normalize_role_codes(current_user) current_employee = self.resolve_current_employee(current_user) if current_employee is None: @@ -223,12 +255,15 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): return self._employee_matches_claim_department(current_employee, current_user, claim) def resolve_department_budget_manager(self, claim: ExpenseClaim) -> Employee | None: + tenant_id = self.resolve_structured_claim_tenant_id(claim) department_ids, department_names = self._collect_claim_department_identity(claim) department_conditions = [] if department_ids: department_conditions.append(Employee.organization_unit_id.in_(department_ids)) if department_names: - department_conditions.append(Employee.organization_unit.has(OrganizationUnit.name.in_(department_names))) + department_conditions.append( + Employee.organization_unit.has(OrganizationUnit.name.in_(department_names)) + ) if not department_conditions: return None @@ -236,6 +271,7 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): select(Employee) .options(selectinload(Employee.organization_unit), selectinload(Employee.roles)) .where( + Employee.tenant_id == tenant_id, func.upper(func.coalesce(Employee.grade, "")) == BUDGET_MONITOR_APPROVAL_GRADE, Employee.roles.any(Role.role_code.in_(BUDGET_APPROVAL_ROLE_CODES)), or_(*department_conditions), @@ -263,11 +299,15 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): return "" def resolve_finance_approver(self, claim: ExpenseClaim) -> Employee | None: + tenant_id = self.resolve_structured_claim_tenant_id(claim) claim_employee_id = str(claim.employee_id or "").strip() base_stmt = ( select(Employee) .options(selectinload(Employee.roles)) - .where(Employee.roles.any(Role.role_code == "finance")) + .where( + Employee.tenant_id == tenant_id, + Employee.roles.any(Role.role_code == "finance"), + ) ) if claim_employee_id: base_stmt = base_stmt.where(Employee.id != claim_employee_id) @@ -275,8 +315,7 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): finance_owner_name = self.resolve_claim_finance_owner_name(claim) if finance_owner_name: named_finance = self.db.scalar( - base_stmt - .where(Employee.name == finance_owner_name) + base_stmt.where(Employee.name == finance_owner_name) .order_by(Employee.name.asc(), Employee.employee_no.asc()) .limit(1) ) @@ -284,15 +323,19 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): return named_finance owner_matched_finance = self.db.scalar( - base_stmt - .where(func.lower(func.coalesce(Employee.finance_owner_name, "")) == finance_owner_name.lower()) + base_stmt.where( + func.lower(func.coalesce(Employee.finance_owner_name, "")) + == finance_owner_name.lower() + ) .order_by(Employee.name.asc(), Employee.employee_no.asc()) .limit(1) ) if owner_matched_finance is not None: return owner_matched_finance - return self.db.scalar(base_stmt.order_by(Employee.name.asc(), Employee.employee_no.asc()).limit(1)) + return self.db.scalar( + base_stmt.order_by(Employee.name.asc(), Employee.employee_no.asc()).limit(1) + ) def attach_budget_approval_snapshot(self, claim: ExpenseClaim | None) -> ExpenseClaim | None: if claim is None: @@ -304,13 +347,9 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): if budget_manager is None: return claim - setattr(claim, "budget_approver_name", str(budget_manager.name or "").strip()) - setattr(claim, "budget_approver_grade", str(budget_manager.grade or "").strip()) - setattr( - claim, - "budget_approver_role_code", - self.resolve_budget_approval_role_code(budget_manager), - ) + claim.budget_approver_name = str(budget_manager.name or "").strip() + claim.budget_approver_grade = str(budget_manager.grade or "").strip() + claim.budget_approver_role_code = self.resolve_budget_approval_role_code(budget_manager) return claim def attach_finance_approval_snapshot(self, claim: ExpenseClaim | None) -> ExpenseClaim | None: @@ -321,7 +360,7 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): finance_approver = self.resolve_finance_approver(claim) if finance_approver is not None and finance_approver.name: - setattr(claim, "finance_approver_name", str(finance_approver.name).strip()) + claim.finance_approver_name = str(finance_approver.name).strip() return claim def attach_approval_snapshot(self, claim: ExpenseClaim | None) -> ExpenseClaim | None: @@ -336,11 +375,7 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): @staticmethod def normalize_role_codes(current_user: CurrentUserContext) -> set[str]: - return { - str(item).strip().lower() - for item in current_user.role_codes - if str(item).strip() - } + return {str(item).strip().lower() for item in current_user.role_codes if str(item).strip()} @staticmethod def _collect_employee_role_codes(employee: Employee | None) -> set[str]: @@ -362,7 +397,9 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): current_user: CurrentUserContext, claim: ExpenseClaim, ) -> bool: - claim_department_ids, claim_department_names = self._collect_claim_department_identity(claim) + claim_department_ids, claim_department_names = self._collect_claim_department_identity( + claim + ) employee_department_ids = { str(employee.organization_unit_id or "").strip(), } @@ -397,28 +434,9 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): department_names.discard("") return department_ids, department_names - def resolve_current_employee(self, current_user: CurrentUserContext) -> Employee | None: - return self.resolve_employee_by_identity_candidates( - [ - str(current_user.username or "").strip(), - str(current_user.name or "").strip(), - str(current_user.employee_no or "").strip(), - ] - ) - - def resolve_current_user_display_name(self, current_user: CurrentUserContext) -> str: - current_employee = self.resolve_current_employee(current_user) - if current_employee is not None and str(current_employee.name or "").strip(): - return str(current_employee.name).strip() - - for candidate in (current_user.name, current_user.username): - normalized = str(candidate or "").strip() - if normalized and not self.is_email_like(normalized): - return normalized - - return str(current_user.username or current_user.name or "anonymous").strip() or "anonymous" - - def is_claim_owned_by_current_user(self, claim: ExpenseClaim, current_user: CurrentUserContext) -> bool: + def is_claim_owned_by_current_user( + self, claim: ExpenseClaim, current_user: CurrentUserContext + ) -> bool: claim_employee_id = str(claim.employee_id or "").strip() current_employee = self.resolve_current_employee(current_user) if current_employee is not None: @@ -448,155 +466,6 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): def is_email_like(value: str) -> bool: return bool(re.match(r"^[^@\s]+@[^@\s]+\.[^@\s]+$", str(value or "").strip())) - def resolve_claim_employee_for_backfill(self, claim: ExpenseClaim) -> Employee | None: - if claim.employee is not None: - employee = self.db.scalar( - select(Employee) - .options( - selectinload(Employee.organization_unit), - selectinload(Employee.manager), - selectinload(Employee.roles), - ) - .where(Employee.id == claim.employee.id) - .limit(1) - ) - return employee or claim.employee - - employee_id = str(claim.employee_id or "").strip() - if employee_id: - employee = self.db.scalar( - select(Employee) - .options( - selectinload(Employee.organization_unit), - selectinload(Employee.manager), - selectinload(Employee.roles), - ) - .where(Employee.id == employee_id) - .limit(1) - ) - if employee is not None: - return employee - - return self.resolve_employee_by_identity_candidates([str(claim.employee_name or "").strip()]) - - def resolve_employee_by_identity_candidates(self, candidates: list[str]) -> Employee | None: - normalized_candidates = [ - item - for item in dict.fromkeys(str(candidate or "").strip() for candidate in candidates) - if item - ] - if not normalized_candidates: - return None - - load_options = ( - selectinload(Employee.organization_unit), - selectinload(Employee.manager), - selectinload(Employee.roles), - ) - - for candidate in normalized_candidates: - employee = self.db.scalar( - select(Employee) - .options(*load_options) - .where( - or_( - func.lower(Employee.email) == candidate.lower(), - func.lower(Employee.employee_no) == candidate.lower(), - ) - ) - .limit(1) - ) - if employee is not None: - return employee - - for candidate in normalized_candidates: - if self.is_email_like(candidate): - continue - matches = list( - self.db.scalars( - select(Employee) - .options(*load_options) - .where(func.lower(Employee.email).like(f"{candidate.lower()}@%")) - .limit(2) - ).all() - ) - if len(matches) == 1: - return matches[0] - - for candidate in normalized_candidates: - matches = list( - self.db.scalars( - select(Employee) - .options(*load_options) - .where(Employee.name == candidate) - .limit(2) - ).all() - ) - if len(matches) == 1: - return matches[0] - - return None - - def backfill_claim_identity_from_current_user( - self, - claim: ExpenseClaim, - current_user: CurrentUserContext, - ) -> None: - employee = self.resolve_claim_employee_for_backfill(claim) or self.resolve_current_employee(current_user) - - if employee is not None: - claim_employee_id = str(claim.employee_id or "").strip() - claim_employee_name = str(claim.employee_name or "").strip() - employee_names = { - str(employee.name or "").strip(), - str(employee.email or "").strip(), - str(employee.employee_no or "").strip(), - } - employee_names.discard("") - - can_apply_employee = ( - not claim_employee_id - or claim_employee_id == employee.id - or self.is_missing_value(claim_employee_name) - or claim_employee_name in employee_names - ) - - if can_apply_employee: - claim.employee = employee - claim.employee_id = employee.id - if employee.name: - claim.employee_name = employee.name - if employee.organization_unit is not None: - claim.department_id = employee.organization_unit_id - claim.department_name = employee.organization_unit.name - return - - context_department = str( - getattr(current_user, "department_name", "") - or getattr(current_user, "department", "") - or getattr(current_user, "departmentName", "") - or "" - ).strip() - if context_department and self.is_missing_value(claim.department_name): - claim.department_name = context_department - - context_name = str(current_user.name or current_user.username or "").strip() - if context_name and self.is_missing_value(claim.employee_name): - claim.employee_name = context_name - - def employee_name_is_unique(self, employee: Employee) -> bool: - normalized_name = str(employee.name or "").strip() - if not normalized_name: - return False - - same_name_count = int( - self.db.scalar( - select(func.count()).select_from(Employee).where(Employee.name == normalized_name) - ) - or 0 - ) - return same_name_count == 1 - def build_personal_claim_conditions(self, current_user: CurrentUserContext) -> list[Any]: conditions = [] username = str(current_user.username or "").strip() @@ -649,20 +518,43 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): conditions = [] if employee is not None: - subordinate_ids = select(Employee.id).where(Employee.manager_id == employee.id) - conditions.append(and_(pending_leader_approval, ExpenseClaim.employee_id.in_(subordinate_ids))) + subordinate_ids = select(Employee.id).where( + Employee.tenant_id == self.normalize_tenant_id(current_user.tenant_id), + Employee.manager_id == employee.id, + ) + conditions.append( + and_(pending_leader_approval, ExpenseClaim.employee_id.in_(subordinate_ids)) + ) if manager_name: - managed_department_ids = select(OrganizationUnit.id).where(OrganizationUnit.manager_name == manager_name) - managed_department_names = select(OrganizationUnit.name).where(OrganizationUnit.manager_name == manager_name) - conditions.append(and_(pending_leader_approval, ExpenseClaim.department_id.in_(managed_department_ids))) - conditions.append(and_(pending_leader_approval, ExpenseClaim.department_name.in_(managed_department_names))) + tenant_id = self.normalize_tenant_id(current_user.tenant_id) + managed_department_ids = select(OrganizationUnit.id).where( + OrganizationUnit.tenant_id == tenant_id, + OrganizationUnit.manager_name == manager_name, + ) + managed_department_names = select(OrganizationUnit.name).where( + OrganizationUnit.tenant_id == tenant_id, + OrganizationUnit.manager_name == manager_name, + ) + conditions.append( + and_( + pending_leader_approval, ExpenseClaim.department_id.in_(managed_department_ids) + ) + ) + conditions.append( + and_( + pending_leader_approval, + ExpenseClaim.department_name.in_(managed_department_names), + ) + ) return conditions def build_budget_approval_claim_conditions(self, current_user: CurrentUserContext) -> list[Any]: employee = self.resolve_current_employee(current_user) - role_codes = self.normalize_role_codes(current_user) | self._collect_employee_role_codes(employee) + role_codes = self.normalize_role_codes(current_user) | self._collect_employee_role_codes( + employee + ) if not role_codes & BUDGET_APPROVAL_ROLE_CODES: return [] if employee is None or not self._employee_has_budget_approval_grade(employee): @@ -681,13 +573,20 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): department_conditions = [] department_name = str(current_user.department_name or "").strip() if employee.organization_unit_id: - department_conditions.append(ExpenseClaim.department_id == employee.organization_unit_id) - subordinate_department_employee_ids = select(Employee.id).where( - Employee.organization_unit_id == employee.organization_unit_id + department_conditions.append( + ExpenseClaim.department_id == employee.organization_unit_id + ) + subordinate_department_employee_ids = select(Employee.id).where( + Employee.tenant_id == self.normalize_tenant_id(current_user.tenant_id), + Employee.organization_unit_id == employee.organization_unit_id, + ) + department_conditions.append( + ExpenseClaim.employee_id.in_(subordinate_department_employee_ids) ) - department_conditions.append(ExpenseClaim.employee_id.in_(subordinate_department_employee_ids)) if employee.organization_unit is not None and employee.organization_unit.name: - department_conditions.append(ExpenseClaim.department_name == employee.organization_unit.name) + department_conditions.append( + ExpenseClaim.department_name == employee.organization_unit.name + ) if department_name: department_conditions.append(ExpenseClaim.department_name == department_name) if not department_conditions: @@ -703,10 +602,12 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): return stmt.where(ExpenseClaim.status == "submitted") conditions = [] if "finance" in role_codes: - conditions.append(and_( - ExpenseClaim.status == "submitted", - ExpenseClaim.approval_stage == FINANCE_APPROVAL_STAGE, - )) + conditions.append( + and_( + ExpenseClaim.status == "submitted", + ExpenseClaim.approval_stage == FINANCE_APPROVAL_STAGE, + ) + ) conditions.extend(self.build_budget_approval_claim_conditions(current_user)) conditions.extend(self.build_approval_claim_conditions(current_user)) @@ -748,7 +649,9 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): if include_approval_scope: if current_user.is_admin or "executive" in role_codes: - conditions.append(ExpenseClaim.status.in_(("submitted", PAYMENT_PENDING_STATUS, "returned"))) + conditions.append( + ExpenseClaim.status.in_(("submitted", PAYMENT_PENDING_STATUS, "returned")) + ) elif "finance" in role_codes: conditions.append( or_( @@ -785,7 +688,10 @@ class ExpenseClaimAccessPolicy(ExpenseClaimTenantScopeMixin): if claim.employee is not None: if claim.employee.manager is not None and claim.employee.manager.name: return str(claim.employee.manager.name).strip() - if claim.employee.organization_unit is not None and claim.employee.organization_unit.manager_name: + if ( + claim.employee.organization_unit is not None + and claim.employee.organization_unit.manager_name + ): return str(claim.employee.organization_unit.manager_name).strip() return "" diff --git a/server/src/app/services/expense_claim_application_handoff.py b/server/src/app/services/expense_claim_application_handoff.py index 35f9614..f3de032 100644 --- a/server/src/app/services/expense_claim_application_handoff.py +++ b/server/src/app/services/expense_claim_application_handoff.py @@ -90,6 +90,7 @@ class ExpenseClaimApplicationHandoffMixin: occurred_at = application_claim.occurred_at or datetime.now(UTC) created_at = datetime.now(UTC) draft_claim = ExpenseClaim( + tenant_id=application_claim.tenant_id, claim_no=self._generate_claim_no(occurred_at), employee_id=application_claim.employee_id, employee_name=application_claim.employee_name, diff --git a/server/src/app/services/expense_claim_approval_flow.py b/server/src/app/services/expense_claim_approval_flow.py index b34b070..9e38887 100644 --- a/server/src/app/services/expense_claim_approval_flow.py +++ b/server/src/app/services/expense_claim_approval_flow.py @@ -24,6 +24,8 @@ from app.services.expense_claim_workflow_constants import ( PAYMENT_PENDING_STAGE, PAYMENT_PENDING_STATUS, ) +from app.services.savings_payment_reversal import SavingsPaymentReversalService +from app.services.savings_realization import SavingsRealizationService class ExpenseClaimApprovalFlowMixin: @@ -428,8 +430,9 @@ class ExpenseClaimApprovalFlowMixin: claim: ExpenseClaim, current_user: CurrentUserContext, *, - ledger: ApprovalActionLedger, + ledger: ApprovalActionLedger | None, request_id: str, + payment_context: dict[str, Any] | None = None, ) -> ExpenseClaim: normalized_status = str(claim.status or "").strip().lower() @@ -443,14 +446,28 @@ class ExpenseClaimApprovalFlowMixin: before_json = self._serialize_claim(claim) operator = self._access_policy.resolve_current_user_display_name(current_user) previous_stage = str(claim.approval_stage or "").strip() + external_context = dict(payment_context or {}) + payment_event_id = str( + external_context.get("connector_event_id") or uuid.uuid4() + ).strip() + is_external_payment = bool(external_context) payment_flag = with_risk_business_stage( { - "source": "payment", - "event_type": "expense_claim_payment_completed", - "payment_event_id": str(uuid.uuid4()), + "source": "external_payment" if is_external_payment else "payment", + "event_type": ( + "expense_claim_external_payment_settled" + if is_external_payment + else "expense_claim_payment_completed" + ), + "payment_event_id": payment_event_id, "severity": "info", - "label": "付款完成", - "message": f"{operator} 已确认付款，报销单进入已付款。", + "label": "外部结算完成" if is_external_payment else "付款完成", + "message": ( + f"已通过 {external_context.get('provider')} 的可信回执完成付款匹配，" + "报销单进入已付款。" + if is_external_payment + else f"{operator} 已确认付款，报销单进入已付款。" + ), "operator": operator, "operator_username": current_user.username, "operator_role_codes": [ @@ -463,6 +480,7 @@ class ExpenseClaimApprovalFlowMixin: "next_status": PAYMENT_PAID_STATUS, "next_approval_stage": PAYMENT_PAID_STAGE, "created_at": datetime.now(UTC).isoformat(), + **external_context, }, "reimbursement", ) @@ -480,7 +498,11 @@ class ExpenseClaimApprovalFlowMixin: claim.approval_stage = PAYMENT_PAID_STAGE claim.risk_flags_json = [*list(claim.risk_flags_json or []), payment_flag] - payment_correlation_id = str(payment_flag.get("payment_event_id") or uuid.uuid4()) + payment_correlation_id = str( + external_context.get("correlation_id") + or payment_flag.get("payment_event_id") + or uuid.uuid4() + ) expense_case, payment_event = self._expense_cases.record_claim_event( claim, event_type="payment_completed", @@ -490,7 +512,19 @@ class ExpenseClaimApprovalFlowMixin: idempotency_key=payment_correlation_id, previous_status=str(before_json.get("status") or ""), previous_approval_stage=previous_stage, - extra_payload={"archived_applications": archived_applications}, + extra_payload={ + "archived_applications": archived_applications, + **( + {"external_payment_evidence": external_context} + if external_context + else {} + ), + }, + ) + SavingsRealizationService(self.db).realize_paid_claim( + claim, + payment_event, + current_user, ) for archived_application in archived_applications: application_claim = self.db.get( @@ -534,6 +568,134 @@ class ExpenseClaimApprovalFlowMixin: return claim + def mark_claim_paid_from_connector( + self, + claim: ExpenseClaim, + *, + tenant_id: str, + provider: str, + connector_event_id: str, + external_event_id: str, + content_hash: str, + verification_level: str, + evidence_classification: str, + external_reference_tail: str, + correlation_id: str, + ) -> ExpenseClaim: + """可信连接器专用入口；调用方必须已完成签名和精确对账。""" + + connector_user = CurrentUserContext( + username=f"financial-connector:{provider}", + name=f"财务连接器 {provider}", + role_codes=["finance"], + is_admin=False, + tenant_id=tenant_id, + ) + return self._mark_claim_paid_once( + claim, + connector_user, + ledger=None, + request_id=f"connector:{external_event_id}"[:120], + payment_context={ + "provider": provider, + "connector_event_id": connector_event_id, + "external_event_id": external_event_id, + "content_hash": content_hash, + "verification_level": verification_level, + "evidence_classification": evidence_classification, + "external_reference_tail": external_reference_tail, + "correlation_id": correlation_id, + }, + ) + + def reopen_claim_after_connector_reversal( + self, + claim: ExpenseClaim, + *, + tenant_id: str, + provider: str, + connector_event_id: str, + external_event_id: str, + origin_connector_event_id: str, + content_hash: str, + verification_level: str, + evidence_classification: str, + correlation_id: str, + ) -> ExpenseClaim: + """退款/冲回以追加事实重开付款，不删除原付款或申请归档事实。""" + + if str(claim.status or "").strip().lower() != PAYMENT_PAID_STATUS: + raise ValueError("只有已付款报销单可以由可信退款/冲回事件重开。") + before_json = self._serialize_claim(claim) + previous_stage = str(claim.approval_stage or "").strip() + reversal_flag = with_risk_business_stage( + { + "source": "external_payment_reversal", + "event_type": "expense_claim_external_payment_reversed", + "reversal_event_id": connector_event_id, + "origin_payment_event_id": origin_connector_event_id, + "external_event_id": external_event_id, + "provider": provider, + "content_hash": content_hash, + "severity": "warning", + "label": "外部付款已冲回", + "message": "可信退款/冲回回执已到达，报销单重开为待付款。", + "previous_status": str(claim.status or "").strip(), + "previous_approval_stage": previous_stage, + "next_status": PAYMENT_PENDING_STATUS, + "next_approval_stage": PAYMENT_PENDING_STAGE, + "created_at": datetime.now(UTC).isoformat(), + }, + "reimbursement", + ) + claim.status = PAYMENT_PENDING_STATUS + claim.approval_stage = PAYMENT_PENDING_STAGE + claim.risk_flags_json = [*list(claim.risk_flags_json or []), reversal_flag] + _, reversal_event = self._expense_cases.record_claim_event( + claim, + event_type="payment_reversed", + actor_id=f"financial-connector:{provider}", + tenant_id=tenant_id, + correlation_id=correlation_id, + idempotency_key=connector_event_id, + causation_id=origin_connector_event_id, + previous_status=str(before_json.get("status") or ""), + previous_approval_stage=previous_stage, + extra_payload={ + "connector_event_id": connector_event_id, + "external_event_id": external_event_id, + "origin_connector_event_id": origin_connector_event_id, + "content_hash": content_hash, + "provider": provider, + "verification_level": verification_level, + "evidence_classification": evidence_classification, + "savings_reversal_candidate": True, + }, + ) + connector_user = CurrentUserContext( + username=f"financial-connector:{provider}", + name=f"财务连接器 {provider}", + role_codes=["finance"], + is_admin=False, + tenant_id=tenant_id, + ) + SavingsPaymentReversalService(self.db).reconcile( + claim, + reversal_event, + connector_user, + ) + self.audit_service.log_action( + actor=f"financial-connector:{provider}", + action="expense_claim.reopen_after_payment_reversal", + resource_type="expense_claim", + resource_id=claim.id, + before_json=before_json, + after_json=self._serialize_claim(claim), + request_id=f"connector:{external_event_id}"[:120], + commit=False, + ) + return claim + @staticmethod def _resolve_latest_approval_opinion(claim, *, source: str) -> str: for flag in reversed(list(claim.risk_flags_json or [])): diff --git a/server/src/app/services/expense_claim_attachment_commercial.py b/server/src/app/services/expense_claim_attachment_commercial.py new file mode 100644 index 0000000..d74f06b --- /dev/null +++ b/server/src/app/services/expense_claim_attachment_commercial.py @@ -0,0 +1,325 @@ +"""报销附件源文件写入的商业预占与文件事务补偿。""" + +from __future__ import annotations + +import hashlib +import logging +import shutil +import uuid +from dataclasses import dataclass, field +from datetime import UTC, datetime +from pathlib import Path + +from sqlalchemy.orm import Session, sessionmaker + +from app.api.deps import CurrentUserContext +from app.models.financial_record import ExpenseClaimItem +from app.services.commercial_direct_operation import ( + CommercialDirectOperationBridge, + DirectOperationIdentity, + DirectOperationResult, +) +from app.services.commercial_transaction_callbacks import ( + bind_commercial_transaction_outcome, +) +from app.services.expense_claim_attachment_storage import ExpenseClaimAttachmentStorage +from app.services.tenant_registry import required_tenant_id + +logger = logging.getLogger(__name__) + + +class ExpenseClaimAttachmentCommercialAccessDenied(RuntimeError): + pass + + +@dataclass(frozen=True, slots=True) +class ExpenseClaimAttachmentCommercialAttempt: + identity: DirectOperationIdentity + size_bytes: int + + +class ExpenseClaimAttachmentCommercialObserver: + def __init__(self, bridge: CommercialDirectOperationBridge) -> None: + self.bridge = bridge + + def permit( + self, + *, + current_user: CurrentUserContext, + claim_id: str, + item_id: str, + content: bytes, + request_id: str, + source_receipt_id: str, + started_at: datetime, + ) -> ExpenseClaimAttachmentCommercialAttempt: + if not content: + raise ValueError("附件商业计量需要非空源文件。") + tenant_id = required_tenant_id(current_user.tenant_id) + content_hash = hashlib.sha256(content).hexdigest() + request_token = _request_token( + current_user=current_user, + claim_id=claim_id, + item_id=item_id, + content_hash=content_hash, + request_id=request_id, + source_receipt_id=source_receipt_id, + ) + identity = DirectOperationIdentity( + tenant_id=tenant_id, + operation_key=f"attachment-upload:{request_token}", + run_key=f"attachment-claim:{_digest(claim_id)}", + tool_type="storage", + tool_name="attachment.upload", + provider="local-filesystem", + started_at=_utc(started_at), + ) + permit = self.bridge.permit( + identity, + requested_quantity=len(content), + required_quantity_basis="bytes", + ) + if not permit.allowed: + raise ExpenseClaimAttachmentCommercialAccessDenied(permit.reason) + return ExpenseClaimAttachmentCommercialAttempt( + identity=identity, + size_bytes=len(content), + ) + + def complete( + self, + attempt: ExpenseClaimAttachmentCommercialAttempt, + ) -> DirectOperationResult: + return self.bridge.complete( + attempt.identity, + outcome="succeeded", + authoritative_quantities={"bytes": attempt.size_bytes}, + completed_at=datetime.now(UTC), + usage_source="persisted_attachment_source_bytes", + usage_availability="available", + ) + + def release( + self, + attempt: ExpenseClaimAttachmentCommercialAttempt, + ) -> DirectOperationResult: + return self.bridge.complete( + attempt.identity, + outcome="not_sent", + authoritative_quantities={}, + completed_at=datetime.now(UTC), + usage_source="business_transaction_rolled_back", + usage_availability="unavailable", + ) + + +@dataclass(slots=True) +class ExpenseClaimAttachmentFileTransaction: + db: Session + storage: ExpenseClaimAttachmentStorage + item: ExpenseClaimItem | None + attachment_dir: Path + observer: ExpenseClaimAttachmentCommercialObserver | None = None + attempt: ExpenseClaimAttachmentCommercialAttempt | None = None + staged_paths: list[tuple[Path, Path]] = field(default_factory=list) + + def stage_replacement(self) -> None: + self._stage_existing_item_files() + self._bind( + on_commit=self._commit_replacement, + on_rollback=self._rollback_replacement, + operation_name="expense_claim_attachment_upload", + ) + + def stage_deletion(self) -> None: + self._stage_existing_item_files() + self._bind( + on_commit=self._discard_staged_paths, + on_rollback=self._restore_staged_paths, + operation_name="expense_claim_attachment_delete", + ) + + def stage_claim_deletion(self, claim_id: str) -> None: + claim_root = (self.storage.root() / str(claim_id or "").strip()).resolve() + claim_root.relative_to(self.storage.root()) + self._stage_path(claim_root) + self._bind( + on_commit=self._discard_staged_paths, + on_rollback=self._restore_staged_paths, + operation_name="expense_claim_files_delete", + ) + + def _bind(self, *, on_commit, on_rollback, operation_name: str) -> None: + bind_commercial_transaction_outcome( + self.db, + on_commit=on_commit, + on_rollback=on_rollback, + operation_name=operation_name, + ) + + def _stage_existing_item_files(self) -> None: + if self.item is None: + raise ValueError("附件文件事务缺少费用明细。") + file_path = self.storage.resolve_item_path(self.item) + if file_path is None: + return + if file_path.parent == self.storage.root(): + self._stage_path(file_path) + self._stage_path(self.storage.meta_path(file_path)) + return + self._stage_path(file_path.parent) + + def _stage_path(self, original: Path) -> None: + if not original.exists(): + return + backup = original.with_name(f".{original.name}.txn-{uuid.uuid4().hex}") + original.replace(backup) + self.staged_paths.append((original, backup)) + + def _commit_replacement(self) -> None: + try: + if self.observer is not None and self.attempt is not None: + self.observer.complete(self.attempt) + finally: + self._discard_staged_paths() + + def _rollback_replacement(self) -> None: + try: + self._delete_path(self.attachment_dir) + self._restore_staged_paths() + finally: + if self.observer is not None and self.attempt is not None: + self.observer.release(self.attempt) + + def _restore_staged_paths(self) -> None: + for original, backup in reversed(self.staged_paths): + if not backup.exists(): + continue + self._delete_path(original) + backup.replace(original) + self.staged_paths.clear() + + def _discard_staged_paths(self) -> None: + for _original, backup in self.staged_paths: + self._delete_path(backup) + self.staged_paths.clear() + + @staticmethod + def _delete_path(path: Path) -> None: + if not path.exists(): + return + if path.is_dir(): + shutil.rmtree(path) + else: + path.unlink() + + +def stage_attachment_replacement( + db: Session, + *, + storage: ExpenseClaimAttachmentStorage, + item: ExpenseClaimItem, + current_user: CurrentUserContext, + claim_id: str, + content: bytes, + request_id: str = "", + source_receipt_id: str = "", +) -> ExpenseClaimAttachmentFileTransaction: + observer = _build_observer(db) + attempt = observer.permit( + current_user=current_user, + claim_id=claim_id, + item_id=item.id, + content=content, + request_id=request_id, + source_receipt_id=source_receipt_id, + started_at=datetime.now(UTC), + ) + transaction = ExpenseClaimAttachmentFileTransaction( + db=db, + storage=storage, + item=item, + attachment_dir=storage.build_item_dir(claim_id, item.id), + observer=observer, + attempt=attempt, + ) + try: + transaction.stage_replacement() + except Exception: + observer.release(attempt) + raise + return transaction + + +def stage_attachment_deletion( + db: Session, + *, + storage: ExpenseClaimAttachmentStorage, + item: ExpenseClaimItem, +) -> ExpenseClaimAttachmentFileTransaction: + transaction = ExpenseClaimAttachmentFileTransaction( + db=db, + storage=storage, + item=item, + attachment_dir=storage.build_item_dir(item.claim_id, item.id), + ) + transaction.stage_deletion() + return transaction + + +def stage_claim_attachment_deletion( + db: Session, + *, + storage: ExpenseClaimAttachmentStorage, + claim_id: str, +) -> ExpenseClaimAttachmentFileTransaction: + claim_root = (storage.root() / str(claim_id or "").strip()).resolve() + transaction = ExpenseClaimAttachmentFileTransaction( + db=db, + storage=storage, + item=None, + attachment_dir=claim_root, + ) + transaction.stage_claim_deletion(claim_id) + return transaction + + +def _build_observer(db: Session) -> ExpenseClaimAttachmentCommercialObserver: + factory = sessionmaker(bind=db.get_bind(), expire_on_commit=False) + return ExpenseClaimAttachmentCommercialObserver( + CommercialDirectOperationBridge(factory, lookup_session=db) + ) + + +def _request_token( + *, + current_user: CurrentUserContext, + claim_id: str, + item_id: str, + content_hash: str, + request_id: str, + source_receipt_id: str, +) -> str: + explicit = str(request_id or "").strip() + receipt = str(source_receipt_id or "").strip() + source = explicit or (f"receipt:{receipt}" if receipt else "content-bound") + return _digest( + "\x1f".join( + ( + required_tenant_id(current_user.tenant_id), + str(current_user.auth_session_id or "").strip(), + str(claim_id or "").strip(), + str(item_id or "").strip(), + content_hash, + source, + ) + ) + ) + + +def _digest(value: str) -> str: + return hashlib.sha256(str(value or "").encode("utf-8")).hexdigest() + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/expense_claim_attachment_operations.py b/server/src/app/services/expense_claim_attachment_operations.py index 297ac1e..ea376b1 100644 --- a/server/src/app/services/expense_claim_attachment_operations.py +++ b/server/src/app/services/expense_claim_attachment_operations.py @@ -1,6 +1,5 @@ from __future__ import annotations -import shutil from datetime import UTC, datetime from pathlib import Path from types import SimpleNamespace @@ -9,7 +8,12 @@ from typing import Any from app.api.deps import CurrentUserContext from app.models.financial_record import ExpenseClaim, ExpenseClaimItem from app.services.document_preview import DocumentPreviewAssets +from app.services.expense_claim_attachment_commercial import ( + stage_attachment_deletion, + stage_attachment_replacement, +) from app.services.ocr import OcrService +from app.services.ocr_commercial import content_digest, trusted_ocr_operation_context from app.services.receipt_folder import ReceiptFolderService @@ -24,6 +28,7 @@ class ExpenseClaimAttachmentOperationsMixin: media_type: str | None, current_user: CurrentUserContext, source_receipt_id: str = "", + request_id: str = "", commit: bool = True, link_source_receipt: bool = True, write_audit: bool = True, @@ -43,9 +48,18 @@ class ExpenseClaimAttachmentOperationsMixin: if not content: raise ValueError("上传文件不能为空。") + stage_attachment_replacement( + self.db, + storage=self._attachment_storage, + item=item, + current_user=current_user, + claim_id=claim.id, + content=content, + request_id=request_id, + source_receipt_id=source_receipt_id, + ) before_json = self._serialize_claim(claim) attachment_dir = self._attachment_storage.build_item_dir(claim.id, item.id) - shutil.rmtree(attachment_dir, ignore_errors=True) attachment_dir.mkdir(parents=True, exist_ok=True) file_path = attachment_dir / normalized_name @@ -72,7 +86,16 @@ class ExpenseClaimAttachmentOperationsMixin: ocr_error = "" upload_ocr_document = None try: - ocr_result = OcrService(self.db).recognize_files( + operation_context = trusted_ocr_operation_context( + current_user, + operation_scope=f"expense-claim-attachment:{claim.id}:{item.id}", + content_digests=[content_digest(content)], + run_id=f"claim-{claim.id}", + ) + ocr_result = OcrService( + self.db, + operation_context=operation_context, + ).recognize_files( [(normalized_name, content, media_type or "application/octet-stream")] ) documents = list(ocr_result.documents or []) @@ -154,8 +177,12 @@ class ExpenseClaimAttachmentOperationsMixin: "ocr_summary": str(getattr(ocr_document, "summary", "") or ""), "ocr_avg_score": float(getattr(ocr_document, "avg_score", 0.0) or 0.0), "ocr_line_count": int(getattr(ocr_document, "line_count", 0) or 0), - "ocr_classification_source": str(getattr(ocr_document, "classification_source", "") or ""), - "ocr_classification_confidence": float(getattr(ocr_document, "classification_confidence", 0.0) or 0.0), + "ocr_classification_source": str( + getattr(ocr_document, "classification_source", "") or "" + ), + "ocr_classification_confidence": float( + getattr(ocr_document, "classification_confidence", 0.0) or 0.0 + ), "ocr_classification_evidence": [ str(item) for item in getattr(ocr_document, "classification_evidence", []) or [] @@ -242,9 +269,11 @@ class ExpenseClaimAttachmentOperationsMixin: preview_file_name = "" if str(raw_meta.get("preview_kind") or "").strip() == "image": try: - preview_source_path, preview_media_type, preview_file_name = receipt_service.resolve_preview( - normalized_receipt_id, - current_user, + preview_source_path, preview_media_type, preview_file_name = ( + receipt_service.resolve_preview( + normalized_receipt_id, + current_user, + ) ) except FileNotFoundError: preview_source_path = None @@ -253,7 +282,9 @@ class ExpenseClaimAttachmentOperationsMixin: document = SimpleNamespace( filename=str(receipt.file_name or fallback_filename or "").strip(), - media_type=str(receipt.media_type or fallback_media_type or "application/octet-stream").strip(), + media_type=str( + receipt.media_type or fallback_media_type or "application/octet-stream" + ).strip(), engine=str(receipt.engine or raw_meta.get("engine") or ""), model=str(receipt.model or raw_meta.get("model") or ""), text=str(receipt.ocr_text or raw_meta.get("ocr_text") or ""), @@ -261,13 +292,19 @@ class ExpenseClaimAttachmentOperationsMixin: avg_score=float(receipt.avg_score or raw_meta.get("ocr_avg_score") or 0.0), line_count=int(receipt.line_count or raw_meta.get("ocr_line_count") or 0), page_count=max(1, int(receipt.page_count or raw_meta.get("page_count") or 1)), - document_type=str(receipt.document_type or raw_meta.get("document_type") or "other").strip(), + document_type=str( + receipt.document_type or raw_meta.get("document_type") or "other" + ).strip(), document_type_label=str( receipt.document_type_label or raw_meta.get("document_type_label") or "其他单据" ).strip(), scene_code=str(receipt.scene_code or raw_meta.get("scene_code") or "other").strip(), - scene_label=str(receipt.scene_label or raw_meta.get("scene_label") or "其他票据").strip(), - classification_source=str(raw_meta.get("ocr_classification_source") or "receipt_folder"), + scene_label=str( + receipt.scene_label or raw_meta.get("scene_label") or "其他票据" + ).strip(), + classification_source=str( + raw_meta.get("ocr_classification_source") or "receipt_folder" + ), classification_confidence=float( receipt.classification_confidence or raw_meta.get("ocr_classification_confidence") @@ -340,11 +377,9 @@ class ExpenseClaimAttachmentOperationsMixin: and source_score >= upload_score ): return source_receipt_document - if ( - source_type == upload_type - and cls._attachment_document_field_count(source_receipt_document) - > cls._attachment_document_field_count(upload_ocr_document) - ): + if source_type == upload_type and cls._attachment_document_field_count( + source_receipt_document + ) > cls._attachment_document_field_count(upload_ocr_document): return source_receipt_document if source_score > upload_score + 2: return source_receipt_document @@ -457,7 +492,11 @@ class ExpenseClaimAttachmentOperationsMixin: before_json = self._serialize_claim(claim) previous_invoice_id = str(item.invoice_id or "").strip() previous_name = self._attachment_presentation.resolve_display_name(item.invoice_id) - self._attachment_storage.delete_item_files(item) + stage_attachment_deletion( + self.db, + storage=self._attachment_storage, + item=item, + ) item.invoice_id = None claim.risk_flags_json = self._remove_deleted_attachment_risk_flags( claim.risk_flags_json, @@ -511,7 +550,9 @@ class ExpenseClaimAttachmentOperationsMixin: flag_item_id = str(flag.get("item_id") or flag.get("itemId") or "").strip() flag_invoice_id = str(flag.get("invoice_id") or flag.get("invoiceId") or "").strip() matches_deleted_item = bool(normalized_item_id and flag_item_id == normalized_item_id) - matches_deleted_invoice = bool(normalized_invoice_id and flag_invoice_id == normalized_invoice_id) + matches_deleted_invoice = bool( + normalized_invoice_id and flag_invoice_id == normalized_invoice_id + ) if matches_deleted_item or matches_deleted_invoice: continue @@ -557,7 +598,10 @@ class ExpenseClaimAttachmentOperationsMixin: if not metadata: return metadata - media_type = str(metadata.get("media_type") or self._attachment_presentation.resolve_media_type(file_path.name)).strip() + media_type = str( + metadata.get("media_type") + or self._attachment_presentation.resolve_media_type(file_path.name) + ).strip() if media_type != "application/pdf": return metadata @@ -586,7 +630,11 @@ class ExpenseClaimAttachmentOperationsMixin: scene_code="", scene_label="", document_fields=[], - warnings=[str(value) for value in list(metadata.get("ocr_warnings") or []) if str(value).strip()], + warnings=[ + str(value) + for value in list(metadata.get("ocr_warnings") or []) + if str(value).strip() + ], ) document_info = self._build_attachment_document_info(document) document.document_type = document_info.get("document_type", "") @@ -640,18 +688,26 @@ class ExpenseClaimAttachmentOperationsMixin: return metadata preview_storage_key = str(metadata.get("preview_storage_key") or "").strip() - preview_path = self._attachment_storage.resolve_path(preview_storage_key) if preview_storage_key else None + preview_path = ( + self._attachment_storage.resolve_path(preview_storage_key) + if preview_storage_key + else None + ) if ( preview_path is not None and preview_path.exists() and str(metadata.get("preview_kind") or "").strip() == "image" - and str(metadata.get("preview_media_type") or "").strip() == DocumentPreviewAssets.PDF_PREVIEW_MEDIA_TYPE - and str(metadata.get("preview_rendered_with") or "").strip() == DocumentPreviewAssets.PDF_RENDERER_ID + and str(metadata.get("preview_media_type") or "").strip() + == DocumentPreviewAssets.PDF_PREVIEW_MEDIA_TYPE + and str(metadata.get("preview_rendered_with") or "").strip() + == DocumentPreviewAssets.PDF_RENDERER_ID ): return metadata preview_name = str(metadata.get("preview_file_name") or "").strip() - if not preview_name or not preview_name.lower().endswith(DocumentPreviewAssets.PDF_PREVIEW_SUFFIX): + if not preview_name or not preview_name.lower().endswith( + DocumentPreviewAssets.PDF_PREVIEW_SUFFIX + ): preview_name = f"{file_path.stem}.preview{DocumentPreviewAssets.PDF_PREVIEW_SUFFIX}" preview_path = file_path.parent / preview_name diff --git a/server/src/app/services/expense_claim_attachment_storage.py b/server/src/app/services/expense_claim_attachment_storage.py index b6168e9..57bd16f 100644 --- a/server/src/app/services/expense_claim_attachment_storage.py +++ b/server/src/app/services/expense_claim_attachment_storage.py @@ -2,11 +2,10 @@ from __future__ import annotations import json import re -import shutil from pathlib import Path from app.core.config import get_settings -from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.financial_record import ExpenseClaimItem class ExpenseClaimAttachmentStorage: @@ -18,15 +17,6 @@ class ExpenseClaimAttachmentStorage: def build_item_dir(self, claim_id: str, item_id: str) -> Path: return (self.root() / claim_id / item_id).resolve() - def delete_claim_files(self, claim: ExpenseClaim) -> None: - for item in list(claim.items or []): - self.delete_item_files(item) - self.delete_claim_root(claim.id) - - def delete_claim_root(self, claim_id: str) -> None: - claim_root = self._assert_child(self.root() / claim_id) - self._delete_path(claim_root) - @staticmethod def normalize_filename(filename: str | None) -> str: normalized = Path(str(filename or "").strip()).name @@ -71,19 +61,6 @@ class ExpenseClaimAttachmentStorage: def to_storage_key(self, file_path: Path) -> str: return file_path.resolve().relative_to(self.root()).as_posix() - def delete_item_files(self, item: ExpenseClaimItem) -> None: - file_path = self.resolve_item_path(item) - if file_path is None: - return - - root = self.root() - if file_path.parent == root: - self._delete_path(file_path) - self._delete_path(self.meta_path(file_path)) - return - - self._delete_path(file_path.parent) - @staticmethod def meta_path(file_path: Path) -> Path: return file_path.with_name(f"{file_path.name}.meta.json") @@ -102,28 +79,3 @@ class ExpenseClaimAttachmentStorage: except (json.JSONDecodeError, OSError): return {} return payload if isinstance(payload, dict) else {} - - def _assert_child(self, path: Path) -> Path: - root = self.root() - resolved = path.resolve() - try: - resolved.relative_to(root) - except ValueError as exc: - raise FileNotFoundError("Attachment path is invalid") from exc - return resolved - - def _delete_path(self, path: Path | None) -> None: - if path is None: - return - - target = self._assert_child(path) - if not target.exists(): - return - - if target.is_dir(): - shutil.rmtree(target) - else: - target.unlink() - - if target.exists(): - raise OSError(f"Attachment path was not deleted: {target}") diff --git a/server/src/app/services/expense_claim_document_item_builder.py b/server/src/app/services/expense_claim_document_item_builder.py index 5617203..65e1be8 100644 --- a/server/src/app/services/expense_claim_document_item_builder.py +++ b/server/src/app/services/expense_claim_document_item_builder.py @@ -1,117 +1,21 @@ from __future__ import annotations -import json import re -import shutil -import uuid -from collections import defaultdict -from datetime import UTC, date, datetime, timedelta -from decimal import Decimal, InvalidOperation -from pathlib import Path -from types import SimpleNamespace +from datetime import date, datetime, timedelta +from decimal import Decimal from typing import Any -from sqlalchemy import func, or_, select -from sqlalchemy import inspect as sqlalchemy_inspect -from sqlalchemy.exc import IntegrityError -from sqlalchemy.orm import Session, selectinload - from app.api.deps import CurrentUserContext -from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType -from app.models.agent_asset import AgentAsset -from app.models.employee import Employee -from app.models.financial_record import ExpenseClaim, ExpenseClaimItem -from app.schemas.ontology import OntologyEntity, OntologyParseResult from app.schemas.reimbursement import ( - ExpenseClaimItemCreate, - ExpenseClaimItemUpdate, - ExpenseClaimUpdate, TravelReimbursementCalculatorRequest, ) -from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager -from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY -from app.services.agent_foundation import AgentFoundationService -from app.services.audit import AuditLogService -from app.services.document_intelligence import build_document_insight -from app.services.expense_claim_access_policy import ExpenseClaimAccessPolicy -from app.services.expense_claim_attachment_presentation import ExpenseClaimAttachmentPresentation -from app.services.expense_claim_attachment_storage import ExpenseClaimAttachmentStorage from app.services.expense_claim_constants import ( - EXPENSE_TYPE_LABELS, - MAX_DRAFT_CLAIMS_PER_USER, - EDITABLE_CLAIM_STATUSES, - SYSTEM_GENERATED_ITEM_TYPES, - TRAVEL_DETAIL_ITEM_TYPES, - TRAVEL_ALLOWANCE_TRIGGER_ITEM_TYPES, DOCUMENT_TYPE_ITEM_TYPE_MAP, - DOCUMENT_TYPE_SCENE_MAP, - DOCUMENT_FACT_ITEM_TYPES, - ROUTE_DESCRIPTION_ITEM_TYPES, - DOCUMENT_TRIP_DATE_LABELS, - DOCUMENT_TRIP_DATE_REQUIREMENT_LABELS, - DOCUMENT_TRIP_DATE_KEYS, - DOCUMENT_GENERIC_DATE_KEYS, - DOCUMENT_INVOICE_DATE_KEYS, - DOCUMENT_TRIP_DATE_LABEL_TOKENS, - DOCUMENT_GENERIC_DATE_LABEL_TOKENS, - DOCUMENT_INVOICE_DATE_LABEL_TOKENS, - DOCUMENT_ROUTE_FORMAT_PATTERN, - DOCUMENT_ROUTE_TEXT_PATTERN, - DOCUMENT_ROUTE_ORIGIN_LABELS, - DOCUMENT_ROUTE_DESTINATION_LABELS, - GENERIC_ATTACHMENT_BACKFILL_ITEM_TYPES, - LOCATION_REQUIRED_EXPENSE_TYPES, - EXPENSE_SCENE_KEYWORDS, - EXPENSE_TYPE_ALLOWED_DOCUMENT_SCENES, - DOCUMENT_SCENE_LABELS, - DOCUMENT_ASSOCIATION_REVIEW_ACTIONS, - PERSISTENT_EXPENSE_REVIEW_ACTIONS, - RETURN_REASON_OPTIONS, - MAX_CLAIM_NO_RETRY_ATTEMPTS, - DOCUMENT_DATE_PATTERN, - SYSTEM_GENERATED_REASON_PREFIXES, - LEADING_REASON_TIME_PATTERNS, - AI_REVIEW_LOOKBACK_DAYS, - AI_REVIEW_REPEAT_RISK_WARNING_COUNT, - AI_REVIEW_REPEAT_RISK_BLOCK_COUNT, - TRAVEL_REVIEW_RELEVANT_EXPENSE_TYPES, - TRAVEL_REVIEW_LONG_DISTANCE_DOCUMENT_TYPES, - TRAVEL_POLICY_CITY_TIERS, - TRAVEL_POLICY_CITY_MATCH_ORDER, - TRAVEL_POLICY_BAND_LABELS, - TRAVEL_POLICY_HOTEL_LIMITS, - TRAVEL_POLICY_ALLOWED_TRANSPORT_LEVELS, - TRAVEL_POLICY_ROUTE_EXCEPTION_KEYWORDS, - TRAVEL_POLICY_STANDARD_EXCEPTION_KEYWORDS, - TRAVEL_POLICY_FLIGHT_CLASS_PATTERNS, - TRAVEL_POLICY_TRAIN_CLASS_PATTERNS, - TRAVEL_POLICY_HOTEL_NIGHT_PATTERN, ) from app.services.expense_claim_platform_context_tools import ( collect_invoice_keys_from_document_info, ) -from app.services.expense_claim_risk_review import ExpenseClaimRiskReviewMixin -from app.services.expense_amounts import ( - extract_amount_candidates, - format_decimal_amount, - is_amount_match_date_fragment, - is_date_like_amount_candidate, - is_probable_year_amount, - parse_document_amount_value, - parse_plain_document_amount_value, - resolve_document_field_amount, - resolve_document_item_amount, - resolve_document_text_amount, -) -from app.services.expense_rule_runtime import ( - DEFAULT_SCENE_RULE_ASSET_CODE, - ExpenseRuleRuntimeService, - RuntimeTravelPolicy, - build_default_expense_rule_catalog, - resolve_document_type_label, -) from app.services.ontology_field_registry import normalize_ontology_form_values -from app.services.ocr import OcrService class ExpenseClaimDocumentItemBuilderMixin: @@ -210,9 +114,15 @@ class ExpenseClaimDocumentItemBuilderMixin: for document in context_documents: specs.append( { - "item_date": self._resolve_document_item_date(document, fallback=occurred_at.date()), - "item_type": self._resolve_document_item_type(document, fallback=expense_type), - "item_reason": self._resolve_document_item_reason(document, fallback=reason), + "item_date": self._resolve_document_item_date( + document, fallback=occurred_at.date() + ), + "item_type": self._resolve_document_item_type( + document, fallback=expense_type + ), + "item_reason": self._resolve_document_item_reason( + document, fallback=reason + ), "item_location": location, "item_amount": self._resolve_document_item_amount(document), "invoice_id": str(document.get("filename") or "").strip() or None, @@ -258,7 +168,11 @@ class ExpenseClaimDocumentItemBuilderMixin: user_id=user_id, ) if allowance_spec is not None: - specs = [spec for spec in specs if str(spec.get("item_type") or "").strip() != "travel_allowance"] + specs = [ + spec + for spec in specs + if str(spec.get("item_type") or "").strip() != "travel_allowance" + ] specs.append(allowance_spec) return specs @@ -313,13 +227,18 @@ class ExpenseClaimDocumentItemBuilderMixin: name="", role_codes=[], is_admin=False, + tenant_id=self.normalize_context_tenant_id(context_json), ), ) except ValueError: return None - allowance_amount = Decimal(result.allowance_amount or Decimal("0.00")).quantize(Decimal("0.01")) - allowance_rate = Decimal(result.total_allowance_rate or Decimal("0.00")).quantize(Decimal("0.01")) + allowance_amount = Decimal(result.allowance_amount or Decimal("0.00")).quantize( + Decimal("0.01") + ) + allowance_rate = Decimal(result.total_allowance_rate or Decimal("0.00")).quantize( + Decimal("0.01") + ) if allowance_amount <= Decimal("0.00") or allowance_rate <= Decimal("0.00"): return None @@ -327,8 +246,7 @@ class ExpenseClaimDocumentItemBuilderMixin: "item_date": end_date, "item_type": "travel_allowance", "item_reason": ( - f"系统自动计算出差补贴：{result.matched_city}，{days}天，" - f"{allowance_rate:.2f}元/天" + f"系统自动计算出差补贴：{result.matched_city}，{days}天，{allowance_rate:.2f}元/天" ), "item_location": str(result.allowance_region or allowance_location).strip(), "item_amount": allowance_amount, @@ -350,9 +268,7 @@ class ExpenseClaimDocumentItemBuilderMixin: if isinstance(review_form_values, dict): review_form_values = normalize_ontology_form_values(review_form_values) review_type = str( - review_form_values.get("expense_type") - or review_form_values.get("reason") - or "" + review_form_values.get("expense_type") or review_form_values.get("reason") or "" ) if any(keyword in review_type for keyword in ("差旅", "出差")): return True @@ -376,8 +292,12 @@ class ExpenseClaimDocumentItemBuilderMixin: business_time_context = context_json.get("business_time_context") if isinstance(business_time_context, dict): - start_date = self._parse_iso_date_or_default(business_time_context.get("start_date"), start_date) - end_date = self._parse_iso_date_or_default(business_time_context.get("end_date"), start_date) + start_date = self._parse_iso_date_or_default( + business_time_context.get("start_date"), start_date + ) + end_date = self._parse_iso_date_or_default( + business_time_context.get("end_date"), start_date + ) else: review_form_values = context_json.get("review_form_values") if isinstance(review_form_values, dict): @@ -519,7 +439,10 @@ class ExpenseClaimDocumentItemBuilderMixin: document_type = str(document.get("document_type") or "").strip().lower() item_type = self._resolve_document_item_type(document, fallback="") - if document_type in {"train_ticket", "flight_itinerary"} or item_type in {"train_ticket", "flight_ticket"}: + if document_type in {"train_ticket", "flight_itinerary"} or item_type in { + "train_ticket", + "flight_ticket", + }: route = self._resolve_document_route_value(document) trip_no = self._resolve_document_fact_field( document, diff --git a/server/src/app/services/expense_claim_draft_flow.py b/server/src/app/services/expense_claim_draft_flow.py index 4bc3362..28c9dd4 100644 --- a/server/src/app/services/expense_claim_draft_flow.py +++ b/server/src/app/services/expense_claim_draft_flow.py @@ -443,6 +443,7 @@ class ExpenseClaimDraftFlowMixin(ExpenseClaimApplicationLinkMixin, ExpenseClaimD try: if claim is None: claim = ExpenseClaim( + tenant_id=self.normalize_context_tenant_id(context_json), claim_no=self._generate_claim_no(final_occurred_at), employee_id=employee.id if employee is not None else None, employee_name=draft_owner_name, diff --git a/server/src/app/services/expense_claim_employee_resolver.py b/server/src/app/services/expense_claim_employee_resolver.py new file mode 100644 index 0000000..eb45f28 --- /dev/null +++ b/server/src/app/services/expense_claim_employee_resolver.py @@ -0,0 +1,223 @@ +from __future__ import annotations + +from sqlalchemy import func, or_, select +from sqlalchemy.orm import selectinload + +from app.api.deps import CurrentUserContext +from app.models.employee import Employee +from app.models.financial_record import ExpenseClaim + + +class ExpenseClaimEmployeeResolverMixin: + """集中处理报销单与员工身份的租户内解析和回填。""" + + def resolve_structured_claim_tenant_id(self, claim: ExpenseClaim) -> str: + tenant_id = str(claim.tenant_id or "").strip() + if tenant_id: + return self.normalize_tenant_id(tenant_id) + return self.resolve_claim_tenant_id(self.db, claim.id) + + def resolve_current_employee(self, current_user: CurrentUserContext) -> Employee | None: + return self.resolve_employee_by_identity_candidates( + [ + str(current_user.employee_id or "").strip(), + str(current_user.username or "").strip(), + str(current_user.name or "").strip(), + str(current_user.employee_no or "").strip(), + ], + tenant_id=current_user.tenant_id, + ) + + def resolve_current_user_display_name(self, current_user: CurrentUserContext) -> str: + current_employee = self.resolve_current_employee(current_user) + if current_employee is not None and str(current_employee.name or "").strip(): + return str(current_employee.name).strip() + + for candidate in (current_user.name, current_user.username): + normalized = str(candidate or "").strip() + if normalized and not self.is_email_like(normalized): + return normalized + + return str(current_user.username or current_user.name or "anonymous").strip() or "anonymous" + + def resolve_claim_employee_for_backfill(self, claim: ExpenseClaim) -> Employee | None: + tenant_id = self.resolve_structured_claim_tenant_id(claim) + if claim.employee is not None: + employee = self.db.scalar( + select(Employee) + .options( + selectinload(Employee.organization_unit), + selectinload(Employee.manager), + selectinload(Employee.roles), + ) + .where( + Employee.tenant_id == tenant_id, + Employee.id == claim.employee.id, + ) + .limit(1) + ) + if employee is not None: + return employee + if self.normalize_tenant_id(claim.employee.tenant_id) == tenant_id: + return claim.employee + return None + + employee_id = str(claim.employee_id or "").strip() + if employee_id: + employee = self.db.scalar( + select(Employee) + .options( + selectinload(Employee.organization_unit), + selectinload(Employee.manager), + selectinload(Employee.roles), + ) + .where( + Employee.tenant_id == tenant_id, + Employee.id == employee_id, + ) + .limit(1) + ) + if employee is not None: + return employee + + return self.resolve_employee_by_identity_candidates( + [str(claim.employee_name or "").strip()], + tenant_id=tenant_id, + ) + + def resolve_employee_by_identity_candidates( + self, + candidates: list[str], + *, + tenant_id: str, + ) -> Employee | None: + trusted_tenant_id = self.normalize_tenant_id(tenant_id) + normalized_candidates = [ + item + for item in dict.fromkeys(str(candidate or "").strip() for candidate in candidates) + if item + ] + if not normalized_candidates: + return None + + load_options = ( + selectinload(Employee.organization_unit), + selectinload(Employee.manager), + selectinload(Employee.roles), + ) + + for candidate in normalized_candidates: + employee = self.db.scalar( + select(Employee) + .options(*load_options) + .where( + Employee.tenant_id == trusted_tenant_id, + or_( + Employee.id == candidate, + func.lower(Employee.email) == candidate.lower(), + func.lower(Employee.employee_no) == candidate.lower(), + ), + ) + .limit(1) + ) + if employee is not None: + return employee + + for candidate in normalized_candidates: + if self.is_email_like(candidate): + continue + matches = list( + self.db.scalars( + select(Employee) + .options(*load_options) + .where( + Employee.tenant_id == trusted_tenant_id, + func.lower(Employee.email).like(f"{candidate.lower()}@%"), + ) + .limit(2) + ).all() + ) + if len(matches) == 1: + return matches[0] + + for candidate in normalized_candidates: + matches = list( + self.db.scalars( + select(Employee) + .options(*load_options) + .where( + Employee.tenant_id == trusted_tenant_id, + Employee.name == candidate, + ) + .limit(2) + ).all() + ) + if len(matches) == 1: + return matches[0] + + return None + + def backfill_claim_identity_from_current_user( + self, + claim: ExpenseClaim, + current_user: CurrentUserContext, + ) -> None: + employee = self.resolve_claim_employee_for_backfill(claim) or self.resolve_current_employee( + current_user + ) + + if employee is not None: + claim_employee_id = str(claim.employee_id or "").strip() + claim_employee_name = str(claim.employee_name or "").strip() + employee_names = { + str(employee.name or "").strip(), + str(employee.email or "").strip(), + str(employee.employee_no or "").strip(), + } + employee_names.discard("") + + can_apply_employee = ( + not claim_employee_id + or claim_employee_id == employee.id + or self.is_missing_value(claim_employee_name) + or claim_employee_name in employee_names + ) + + if can_apply_employee: + claim.employee = employee + claim.employee_id = employee.id + if employee.name: + claim.employee_name = employee.name + if employee.organization_unit is not None: + claim.department_id = employee.organization_unit_id + claim.department_name = employee.organization_unit.name + return + + context_department = str( + getattr(current_user, "department_name", "") + or getattr(current_user, "department", "") + or getattr(current_user, "departmentName", "") + or "" + ).strip() + if context_department and self.is_missing_value(claim.department_name): + claim.department_name = context_department + + context_name = str(current_user.name or current_user.username or "").strip() + if context_name and self.is_missing_value(claim.employee_name): + claim.employee_name = context_name + + def employee_name_is_unique(self, employee: Employee) -> bool: + normalized_name = str(employee.name or "").strip() + if not normalized_name: + return False + + same_name_count = int( + self.db.scalar( + select(func.count()) + .select_from(Employee) + .where(Employee.name == normalized_name) + .where(Employee.tenant_id == self.normalize_tenant_id(employee.tenant_id)) + ) + or 0 + ) + return same_name_count == 1 diff --git a/server/src/app/services/expense_claim_item_sync.py b/server/src/app/services/expense_claim_item_sync.py index 4ddb1c2..5483c65 100644 --- a/server/src/app/services/expense_claim_item_sync.py +++ b/server/src/app/services/expense_claim_item_sync.py @@ -6,33 +6,24 @@ from decimal import Decimal, InvalidOperation from types import SimpleNamespace from typing import Any -from sqlalchemy import or_, select from sqlalchemy import inspect as sqlalchemy_inspect +from sqlalchemy import select from app.api.deps import CurrentUserContext -from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType -from app.models.agent_asset import AgentAsset from app.models.employee import Employee from app.models.financial_record import ExpenseClaim, ExpenseClaimItem from app.schemas.reimbursement import TravelReimbursementCalculatorRequest -from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager -from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.expense_claim_constants import ( - AI_REVIEW_LOOKBACK_DAYS, - AI_REVIEW_REPEAT_RISK_BLOCK_COUNT, - AI_REVIEW_REPEAT_RISK_WARNING_COUNT, DOCUMENT_FACT_ITEM_TYPES, LOCATION_REQUIRED_EXPENSE_TYPES, OPTIONAL_ATTACHMENT_ITEM_TYPES, STANDARD_ADJUSTMENT_RISK_SOURCE, SYSTEM_GENERATED_ITEM_TYPES, TRAVEL_ALLOWANCE_TRIGGER_ITEM_TYPES, - TRAVEL_POLICY_HOTEL_NIGHT_PATTERN, ) from app.services.expense_claim_risk_stage import with_risk_business_stage from app.services.expense_rule_runtime import ( ExpenseRuleRuntimeService, - RuntimeTravelPolicy, build_default_expense_rule_catalog, ) @@ -41,10 +32,14 @@ class ExpenseClaimItemSyncMixin: def _sync_travel_allowance_item(self, claim: ExpenseClaim) -> None: items = list(claim.items or []) allowance_items = [ - item for item in items if str(item.item_type or "").strip().lower() == "travel_allowance" + item + for item in items + if str(item.item_type or "").strip().lower() == "travel_allowance" ] business_items = [ - item for item in items if str(item.item_type or "").strip().lower() != "travel_allowance" + item + for item in items + if str(item.item_type or "").strip().lower() != "travel_allowance" ] business_types = {str(item.item_type or "").strip().lower() for item in business_items} is_travel_claim = str(claim.expense_type or "").strip().lower() == "travel" @@ -90,13 +85,18 @@ class ExpenseClaimItemSyncMixin: name=str(claim.employee_name or ""), role_codes=[], is_admin=False, + tenant_id=self.normalize_tenant_id(claim.tenant_id), ), ) except ValueError: return - allowance_amount = Decimal(result.allowance_amount or Decimal("0.00")).quantize(Decimal("0.01")) - allowance_rate = Decimal(result.total_allowance_rate or Decimal("0.00")).quantize(Decimal("0.01")) + allowance_amount = Decimal(result.allowance_amount or Decimal("0.00")).quantize( + Decimal("0.01") + ) + allowance_rate = Decimal(result.total_allowance_rate or Decimal("0.00")).quantize( + Decimal("0.01") + ) if allowance_amount <= Decimal("0.00") or allowance_rate <= Decimal("0.00"): return @@ -112,8 +112,7 @@ class ExpenseClaimItemSyncMixin: item.item_date = end_date item.item_type = "travel_allowance" item.item_reason = ( - f"系统自动计算出差补贴：{result.matched_city}，{days}天，" - f"{allowance_rate:.2f}元/天" + f"系统自动计算出差补贴：{result.matched_city}，{days}天，{allowance_rate:.2f}元/天" ) item.item_location = str(result.allowance_region or allowance_location).strip() item.item_amount = allowance_amount @@ -170,7 +169,11 @@ class ExpenseClaimItemSyncMixin: if explicit_days > 0: days = explicit_days end_date = start_date + timedelta(days=days - 1) - if application_days is not None and application_days[0] > days and len(unique_dates) <= 1: + if ( + application_days is not None + and application_days[0] > days + and len(unique_dates) <= 1 + ): return application_days return max(1, days), start_date, end_date existing_days = self._extract_travel_allowance_days(existing_allowance) @@ -207,7 +210,13 @@ class ExpenseClaimItemSyncMixin: ) if days <= 0: return None - start_date = dates[0] if dates else claim.occurred_at.date() if claim.occurred_at is not None else date.today() + start_date = ( + dates[0] + if dates + else claim.occurred_at.date() + if claim.occurred_at is not None + else date.today() + ) end_date = start_date + timedelta(days=days - 1) return days, start_date, end_date @@ -216,7 +225,10 @@ class ExpenseClaimItemSyncMixin: for flag in list(claim.risk_flags_json or []): if not isinstance(flag, dict): continue - if str(flag.get("source") or "").strip() not in {"application_link", "application_handoff"}: + if str(flag.get("source") or "").strip() not in { + "application_link", + "application_handoff", + }: continue for source in ( flag.get("expense_scene_selection"), @@ -238,7 +250,10 @@ class ExpenseClaimItemSyncMixin: detail: dict[str, Any] = {} for flag in list(application_claim.risk_flags_json or []): - if not isinstance(flag, dict) or str(flag.get("source") or "").strip() != "application_detail": + if ( + not isinstance(flag, dict) + or str(flag.get("source") or "").strip() != "application_detail" + ): continue payload = flag.get("application_detail") or flag.get("applicationDetail") or {} if isinstance(payload, dict): @@ -262,9 +277,7 @@ class ExpenseClaimItemSyncMixin: def _find_linked_application_claim(self, values: dict[str, Any]) -> ExpenseClaim | None: application_claim_id = str( - values.get("application_claim_id") - or values.get("applicationClaimId") - or "" + values.get("application_claim_id") or values.get("applicationClaimId") or "" ).strip() if application_claim_id: linked_claim = self.db.get(ExpenseClaim, application_claim_id) @@ -272,9 +285,7 @@ class ExpenseClaimItemSyncMixin: return linked_claim application_claim_no = str( - values.get("application_claim_no") - or values.get("applicationClaimNo") - or "" + values.get("application_claim_no") or values.get("applicationClaimNo") or "" ).strip() if not application_claim_no: return None @@ -316,7 +327,10 @@ class ExpenseClaimItemSyncMixin: sorted_items = sorted( business_items, - key=lambda item: (item.item_date or date.max, self._normalize_sort_datetime(item.created_at)), + key=lambda item: ( + item.item_date or date.max, + self._normalize_sort_datetime(item.created_at), + ), ) for item in sorted_items: location = str(item.item_location or "").strip() @@ -388,7 +402,10 @@ class ExpenseClaimItemSyncMixin: primary_item = ordered_items[0] adjusted_amounts = self._collect_standard_adjusted_amounts(claim) total_amount = sum( - (self._resolve_item_amount_for_claim_total(item, adjusted_amounts) for item in ordered_items), + ( + self._resolve_item_amount_for_claim_total(item, adjusted_amounts) + for item in ordered_items + ), Decimal("0.00"), ) @@ -402,16 +419,21 @@ class ExpenseClaimItemSyncMixin: ) claim.expense_type = self._resolve_claim_expense_type_from_items( ordered_items, - fallback=str(primary_item.item_type or claim.expense_type or "other").strip() or "other", + fallback=str(primary_item.item_type or claim.expense_type or "other").strip() + or "other", ) primary_item_type = str(primary_item.item_type or "").strip() if primary_item_type not in DOCUMENT_FACT_ITEM_TYPES: claim.reason = ( - self._normalize_optional_text(primary_item.item_reason, fallback=claim.reason or "待补充") + self._normalize_optional_text( + primary_item.item_reason, fallback=claim.reason or "待补充" + ) or "待补充" ) claim.location = ( - self._normalize_optional_text(primary_item.item_location, fallback=claim.location or "待补充") + self._normalize_optional_text( + primary_item.item_location, fallback=claim.location or "待补充" + ) or "待补充" ) claim.risk_flags_json = self._merge_claim_attachment_risk_flags( @@ -440,7 +462,10 @@ class ExpenseClaimItemSyncMixin: return metadata = self._attachment_storage.read_meta(file_path) - media_type = str(metadata.get("media_type") or self._attachment_presentation.resolve_media_type(file_path.name)).strip() + media_type = str( + metadata.get("media_type") + or self._attachment_presentation.resolve_media_type(file_path.name) + ).strip() ocr_status = str(metadata.get("ocr_status") or "").strip().lower() if ocr_status == "failed": @@ -471,7 +496,11 @@ class ExpenseClaimItemSyncMixin: scene_code=str(stored_document_info.get("scene_code") or ""), scene_label=str(stored_document_info.get("scene_label") or ""), document_fields=list(stored_document_info.get("fields") or []), - warnings=[str(value) for value in list(metadata.get("ocr_warnings") or []) if str(value).strip()], + warnings=[ + str(value) + for value in list(metadata.get("ocr_warnings") or []) + if str(value).strip() + ], ) document_info = self._build_attachment_document_info(document) requirement_check = self._build_attachment_requirement_check( @@ -550,7 +579,7 @@ class ExpenseClaimItemSyncMixin: catalog = build_default_expense_rule_catalog() else: catalog = ExpenseRuleRuntimeService(db).load_catalog() - setattr(self, "_expense_rule_catalog", catalog) + self._expense_rule_catalog = catalog return catalog def _get_expense_scene_policy(self, expense_type: str | None) -> Any | None: @@ -565,11 +594,15 @@ class ExpenseClaimItemSyncMixin: @staticmethod def _is_attachment_required_item_type(item_type: str | None) -> bool: normalized = str(item_type or "").strip().lower() - return normalized not in SYSTEM_GENERATED_ITEM_TYPES and normalized not in OPTIONAL_ATTACHMENT_ITEM_TYPES + return ( + normalized not in SYSTEM_GENERATED_ITEM_TYPES + and normalized not in OPTIONAL_ATTACHMENT_ITEM_TYPES + ) def _resolve_claim_required_attachment_count(self, claim: ExpenseClaim) -> int: required_items = [ - item for item in list(claim.items or []) + item + for item in list(claim.items or []) if self._is_attachment_required_item_type(item.item_type) ] if not required_items: @@ -591,7 +624,10 @@ class ExpenseClaimItemSyncMixin: preserved_flags = [ flag for flag in list(claim.risk_flags_json or []) - if not (isinstance(flag, dict) and str(flag.get("source") or "").strip() == "attachment_analysis") + if not ( + isinstance(flag, dict) + and str(flag.get("source") or "").strip() == "attachment_analysis" + ) ] return preserved_flags + attachment_risk_flags @@ -629,7 +665,9 @@ class ExpenseClaimItemSyncMixin: @staticmethod def _format_submission_blocked_message(issues: list[str]) -> str: - normalized_issues = [str(issue or "").strip() for issue in issues if str(issue or "").strip()] + normalized_issues = [ + str(issue or "").strip() for issue in issues if str(issue or "").strip() + ] if not normalized_issues: return "自动检测未通过，但没有返回明确原因，请刷新草稿后重试。" @@ -669,10 +707,14 @@ class ExpenseClaimItemSyncMixin: for index, item in enumerate(claim.items, start=1): prefix = f"费用明细第 {index} 条" - is_system_generated = str(item.item_type or "").strip().lower() in SYSTEM_GENERATED_ITEM_TYPES + is_system_generated = ( + str(item.item_type or "").strip().lower() in SYSTEM_GENERATED_ITEM_TYPES + ) if is_system_generated or self._is_submission_placeholder_item(item): continue - item_location_required = self._is_location_required_expense_type(item.item_type or claim.expense_type) + item_location_required = self._is_location_required_expense_type( + item.item_type or claim.expense_type + ) item_has_attachment = not self._is_missing_value(item.invoice_id) if not item_has_attachment and item.item_date is None: issues.append(f"{prefix}缺少日期") @@ -680,9 +722,15 @@ class ExpenseClaimItemSyncMixin: issues.append(f"{prefix}缺少费用项目") if not item_has_attachment and self._is_missing_value(item.item_reason): issues.append(f"{prefix}缺少说明") - if not item_has_attachment and item_location_required and self._is_missing_value(item.item_location): + if ( + not item_has_attachment + and item_location_required + and self._is_missing_value(item.item_location) + ): issues.append(f"{prefix}缺少地点") - if not item_has_attachment and (item.item_amount is None or item.item_amount <= Decimal("0.00")): + if not item_has_attachment and ( + item.item_amount is None or item.item_amount <= Decimal("0.00") + ): issues.append(f"{prefix}缺少金额") if self._is_attachment_required_item_type(item.item_type) and not item_has_attachment: issues.append(f"{prefix}缺少票据标识") diff --git a/server/src/app/services/expense_claim_platform_risk.py b/server/src/app/services/expense_claim_platform_risk.py index 5a1e96a..8b17ecf 100644 --- a/server/src/app/services/expense_claim_platform_risk.py +++ b/server/src/app/services/expense_claim_platform_risk.py @@ -5,11 +5,8 @@ from typing import Any from sqlalchemy import select -from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType -from app.models.agent_asset import AgentAsset from app.models.financial_record import ExpenseClaim, ExpenseClaimItem from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager -from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.budget import BudgetService from app.services.expense_claim_platform_context_tools import ( collect_attachment_cities, @@ -25,35 +22,52 @@ from app.services.expense_claim_platform_route_risk import resolve_multi_city_re from app.services.expense_claim_platform_text_risk import ( collect_vague_goods_description_evidence, ) +from app.services.expense_claim_release_telemetry import ( + SUPPORTED_PLATFORM_RISK_EVALUATORS, + ExpenseClaimReleaseTelemetryRecorder, +) from app.services.expense_claim_risk_flags import dedupe_claim_risk_flags +from app.services.expense_claim_risk_rule_loader import ExpenseClaimRiskRuleLoader +from app.services.expense_claim_risk_stage import ( + build_shadow_release_evaluation, + normalize_risk_business_stage, + risk_manifest_matches_business_stage, +) from app.services.expense_claim_rule_fingerprint import build_risk_manifest_fingerprint +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin from app.services.expense_type_keywords import resolve_expense_type_code_from_text -from app.services.risk_rule_manifest_classifier import is_budget_risk_manifest -from app.services.risk_rule_manifest_normalizer import normalize_risk_rule_manifest from app.services.risk_rule_template_executor import RiskRuleTemplateExecutor class ExpenseClaimPlatformRiskMixin: - _DEFAULT_RISK_BUSINESS_STAGE = "reimbursement" - _SUPPORTED_RISK_BUSINESS_STAGES = {"expense_application", "reimbursement"} def evaluate_platform_risk_rules( self, claim: ExpenseClaim, *, rule_codes: list[str] | None = None, business_stage: str | None = None, + tenant_id: str | None = None, ) -> dict[str, Any]: - normalized_stage = self._normalize_platform_risk_business_stage(business_stage) + normalized_stage = normalize_risk_business_stage(business_stage) + normalized_tenant = ExpenseClaimTenantScopeMixin.normalize_tenant_id( + tenant_id or ExpenseClaimTenantScopeMixin.resolve_claim_tenant_id(self.db, claim.id) + ) manifests = self._load_platform_risk_rule_manifests( rule_codes=rule_codes, business_stage=normalized_stage, + routing_key=str(claim.id or claim.claim_no or "").strip(), + tenant_id=normalized_tenant, ) - rule_set_fingerprint = build_risk_manifest_fingerprint(manifests) + enforced_manifests = [ + item for item in manifests if str(item.get("_release_mode") or "enforced") != "shadow" + ] + rule_set_fingerprint = build_risk_manifest_fingerprint(enforced_manifests) if not manifests: return { "flags": [], "blocking_reasons": [], "rule_set_fingerprint": rule_set_fingerprint, + "shadow_evaluations": [], } contexts = self._build_claim_attachment_contexts(claim) @@ -69,25 +83,81 @@ class ExpenseClaimPlatformRiskMixin: ) flags: list[dict[str, Any]] = [] blocking_reasons: list[str] = [] + shadow_evaluations: list[dict[str, Any]] = [] + enforced_outcomes: dict[tuple[str, str], bool] = {} + release_telemetry = ExpenseClaimReleaseTelemetryRecorder(self.db) for manifest in manifests: + release_key = release_telemetry.manifest_key(manifest) if not self._risk_manifest_applies_to_claim(manifest, claim=claim, contexts=contexts): + if str(manifest.get("_release_mode") or "enforced") != "shadow": + enforced_outcomes[release_key] = False + release_telemetry.record( + tenant_id=normalized_tenant, + claim=claim, + manifest=manifest, + hit=False, + baseline_hit=release_telemetry.baseline_hit( + manifest, + enforced_outcomes=enforced_outcomes, + release_key=release_key, + ), + business_stage=normalized_stage, + ) + if str(manifest.get("_release_mode") or "") == "shadow": + shadow_evaluations.append(build_shadow_release_evaluation(manifest, None)) continue - flag = self._evaluate_platform_risk_manifest( - manifest, + evaluator = str(manifest.get("evaluator") or "").strip().lower() + runtime_status = "completed" + failure_code = "none" + if evaluator not in SUPPORTED_PLATFORM_RISK_EVALUATORS: + flag = None + runtime_status = "failed" + failure_code = "unsupported_evaluator" + else: + try: + flag = self._evaluate_platform_risk_manifest( + manifest, + claim=claim, + contexts=contexts, + ) + except Exception: + release_telemetry.record_failure_durably( + tenant_id=normalized_tenant, + claim=claim, + manifest=manifest, + failure_code="evaluator_error", + business_stage=normalized_stage, + ) + raise + if evaluator == "release_guard_integrity_failure": + runtime_status = "failed" + failure_code = "artifact_integrity_error" + if str(manifest.get("_release_mode") or "enforced") != "shadow": + enforced_outcomes[release_key] = flag is not None + release_telemetry.record( + tenant_id=normalized_tenant, claim=claim, - contexts=contexts, + manifest=manifest, + hit=flag is not None, + baseline_hit=release_telemetry.baseline_hit( + manifest, + enforced_outcomes=enforced_outcomes, + release_key=release_key, + ), + runtime_status=runtime_status, + failure_code=failure_code, + business_stage=normalized_stage, ) + if str(manifest.get("_release_mode") or "") == "shadow": + shadow_evaluations.append(build_shadow_release_evaluation(manifest, flag)) + continue if flag is None: continue flags.append(flag) - flags = [ - flag - for flag in dedupe_claim_risk_flags(flags) - if isinstance(flag, dict) - ] + flags = [flag for flag in dedupe_claim_risk_flags(flags) if isinstance(flag, dict)] for flag in flags: severity = str(flag.get("severity") or "").strip().lower() action = str(flag.get("action") or "").strip().lower() @@ -99,149 +169,43 @@ class ExpenseClaimPlatformRiskMixin: "flags": flags, "blocking_reasons": deduplicated_reasons, "rule_set_fingerprint": rule_set_fingerprint, + "shadow_evaluations": shadow_evaluations, } def platform_risk_rule_set_fingerprint( self, *, business_stage: str, + tenant_id: str | None = None, ) -> str: manifests = self._load_platform_risk_rule_manifests( rule_codes=None, - business_stage=self._normalize_platform_risk_business_stage(business_stage), + business_stage=normalize_risk_business_stage(business_stage), + routing_key="", + tenant_id=ExpenseClaimTenantScopeMixin.normalize_tenant_id(tenant_id), + ) + return build_risk_manifest_fingerprint( + [item for item in manifests if item.get("_release_mode") != "shadow"] ) - return build_risk_manifest_fingerprint(manifests) def _load_platform_risk_rule_manifests( self, *, rule_codes: list[str] | None, business_stage: str | None, + routing_key: str = "", + tenant_id: str = "default", ) -> list[dict[str, Any]]: - code_filter = { - str(code or "").strip() for code in list(rule_codes or []) if str(code or "").strip() - } - manifests_by_code: dict[str, dict[str, Any]] = {} - - assets = list( - self.db.scalars( - select(AgentAsset) - .where(AgentAsset.asset_type == AgentAssetType.RULE.value) - .where(AgentAsset.status == AgentAssetStatus.ACTIVE.value) - .where(AgentAsset.domain == AgentAssetDomain.EXPENSE.value) - .order_by(AgentAsset.updated_at.desc(), AgentAsset.created_at.desc()) - ).all() + return ExpenseClaimRiskRuleLoader( + self.db, + rule_library_manager=AgentAssetRuleLibraryManager(), + ).load( + rule_codes=rule_codes, + business_stage=business_stage, + routing_key=routing_key, + tenant_id=tenant_id, + stage_matcher=risk_manifest_matches_business_stage, ) - library_manager = AgentAssetRuleLibraryManager() - - for asset in assets: - config_json = asset.config_json if isinstance(asset.config_json, dict) else {} - if str(config_json.get("detail_mode") or "").strip().lower() != "json_risk": - continue - rule_code = str(asset.code or "").strip() - if code_filter and rule_code not in code_filter: - continue - - rule_document = config_json.get("rule_document") - if not isinstance(rule_document, dict): - continue - file_name = str(rule_document.get("file_name") or "").strip() - rule_library = ( - str(config_json.get("rule_library") or RISK_RULES_LIBRARY).strip() - or RISK_RULES_LIBRARY - ) - if not file_name: - continue - - try: - payload = library_manager.read_rule_library_json( - library=rule_library, - file_name=file_name, - ) - except (FileNotFoundError, ValueError): - continue - - payload = normalize_risk_rule_manifest(payload) - manifest_code = str(payload.get("rule_code") or rule_code).strip() - if not manifest_code or (code_filter and manifest_code not in code_filter): - continue - if is_budget_risk_manifest(payload): - continue - if payload.get("enabled") is False or not self._risk_manifest_matches_business_stage( - payload, - business_stage=business_stage, - ): - continue - - payload = dict(payload) - payload.setdefault("rule_code", manifest_code) - payload["_rule_version"] = str( - asset.published_version or asset.current_version or "v1.0.0" - ) - payload["_rule_asset_id"] = asset.id - manifests_by_code[manifest_code] = payload - - missing_codes = code_filter - set(manifests_by_code) - should_load_fallback = not code_filter or bool(missing_codes) - if should_load_fallback: - try: - files = library_manager.list_rule_library_json_files(library=RISK_RULES_LIBRARY) - except ValueError: - files = [] - for file_name in files: - try: - payload = library_manager.read_rule_library_json( - library=RISK_RULES_LIBRARY, - file_name=file_name, - ) - except (FileNotFoundError, ValueError): - continue - payload = normalize_risk_rule_manifest(payload) - rule_code = str(payload.get("rule_code") or "").strip() - if not rule_code or rule_code in manifests_by_code: - continue - if code_filter and rule_code not in missing_codes: - continue - if is_budget_risk_manifest(payload): - continue - if payload.get("enabled") is False or not self._risk_manifest_matches_business_stage( - payload, - business_stage=business_stage, - ): - continue - payload = dict(payload) - payload["_rule_version"] = "v1.0.0" - manifests_by_code[rule_code] = payload - - return list(manifests_by_code.values()) - - @classmethod - def _normalize_platform_risk_business_stage(cls, value: str | None) -> str: - normalized = str(value or cls._DEFAULT_RISK_BUSINESS_STAGE).strip().lower() - if not normalized or normalized not in cls._SUPPORTED_RISK_BUSINESS_STAGES: - return cls._DEFAULT_RISK_BUSINESS_STAGE - return normalized - - @classmethod - def _risk_manifest_matches_business_stage( - cls, - manifest: dict[str, Any], - *, - business_stage: str | None, - ) -> bool: - if not business_stage: - return True - applies_to = manifest.get("applies_to") if isinstance(manifest.get("applies_to"), dict) else {} - raw_stages = applies_to.get("business_stages") - if not isinstance(raw_stages, list): - metadata = manifest.get("metadata") if isinstance(manifest.get("metadata"), dict) else {} - raw_stages = [manifest.get("business_stage") or metadata.get("business_stage") or cls._DEFAULT_RISK_BUSINESS_STAGE] - stages = { - cls._normalize_platform_risk_business_stage(str(item)) - for item in raw_stages - if str(item or "").strip() - } - return business_stage in (stages or {cls._DEFAULT_RISK_BUSINESS_STAGE}) def _risk_manifest_applies_to_claim( self, @@ -372,6 +336,17 @@ class ExpenseClaimPlatformRiskMixin: contexts: list[dict[str, Any]], ) -> dict[str, Any] | None: evaluator = str(manifest.get("evaluator") or "").strip().lower() + if evaluator == "release_guard_integrity_failure": + integrity_error = manifest.get("_release_integrity_error") + return self._build_platform_risk_flag( + manifest, + message="风险规则发布快照完整性校验失败，已暂停自动流转，请平台管理员恢复稳定版本。", + evidence=( + dict(integrity_error) + if isinstance(integrity_error, dict) + else {"reason": "release_artifact_unavailable"} + ), + ) if evaluator == "reason_too_brief": return self._evaluate_reason_too_brief_risk(manifest, claim=claim) if evaluator == "entertainment_reason_missing": @@ -430,7 +405,9 @@ class ExpenseClaimPlatformRiskMixin: self._build_platform_risk_flag( manifest, message=str(result.get("message") or "自然语言风险规则命中。"), - evidence=result.get("evidence") if isinstance(result.get("evidence"), dict) else {}, + evidence=result.get("evidence") + if isinstance(result.get("evidence"), dict) + else {}, ), self._context_item_ids(contexts), ) @@ -574,9 +551,7 @@ class ExpenseClaimPlatformRiskMixin: contexts: list[dict[str, Any]], ) -> dict[str, Any] | None: invoice_keys = collect_invoice_keys_from_contexts(contexts) - duplicate_keys = [ - key for key, count in count_values(invoice_keys).items() if count > 1 - ] + duplicate_keys = [key for key, count in count_values(invoice_keys).items() if count > 1] if duplicate_keys: return self._build_platform_risk_flag( manifest, @@ -780,7 +755,7 @@ class ExpenseClaimPlatformRiskMixin: manifest, message=message, evidence=evidence, - default_business_stage=self._DEFAULT_RISK_BUSINESS_STAGE, + default_business_stage="reimbursement", ) @staticmethod @@ -790,7 +765,11 @@ class ExpenseClaimPlatformRiskMixin: @staticmethod def _with_related_item_ids(flag: dict[str, Any], item_ids: list[str]) -> dict[str, Any]: normalized_item_ids = list( - dict.fromkeys(str(item_id or "").strip() for item_id in list(item_ids or []) if str(item_id or "").strip()) + dict.fromkeys( + str(item_id or "").strip() + for item_id in list(item_ids or []) + if str(item_id or "").strip() + ) ) if not normalized_item_ids: return flag diff --git a/server/src/app/services/expense_claim_platform_risk_flag.py b/server/src/app/services/expense_claim_platform_risk_flag.py index 3d6a01e..a126fe7 100644 --- a/server/src/app/services/expense_claim_platform_risk_flag.py +++ b/server/src/app/services/expense_claim_platform_risk_flag.py @@ -4,8 +4,8 @@ from typing import Any from app.services.expense_claim_risk_stage import ( infer_risk_domain, - normalize_risk_business_stage, normalize_risk_actionability, + normalize_risk_business_stage, normalize_risk_visibility_scope, with_risk_business_stage, ) @@ -97,14 +97,10 @@ def build_platform_risk_flag( or "" ).strip() basic_rule_code = str( - manifest.get("basic_rule_code") - or metadata.get("basic_rule_code") - or finance_rule_code + manifest.get("basic_rule_code") or metadata.get("basic_rule_code") or finance_rule_code ).strip() basic_rule_sheet = str( - manifest.get("basic_rule_sheet") - or metadata.get("basic_rule_sheet") - or finance_rule_sheet + manifest.get("basic_rule_sheet") or metadata.get("basic_rule_sheet") or finance_rule_sheet ).strip() basic_rule_refs = _normalize_basic_rule_refs( manifest.get("basic_rule_refs") or metadata.get("basic_rule_refs") @@ -119,26 +115,31 @@ def build_platform_risk_flag( } ] + flag = { + "source": "submission_review", + "hit_source": "rule_center", + "rule_type": "risk", + "rule_code": str(manifest.get("rule_code") or "").strip(), + "rule_version": str(manifest.get("_rule_version") or "v1.0.0").strip(), + "basic_rule_code": basic_rule_code, + "basic_rule_sheet": basic_rule_sheet, + "basic_rule_refs": basic_rule_refs, + "finance_rule_code": finance_rule_code, + "finance_rule_sheet": finance_rule_sheet, + "severity": severity, + "action": action, + "label": label, + "message": message, + "evidence": evidence, + "risk_domain": risk_domain, + "visibility_scope": visibility_scope, + "actionability": actionability, + } + if manifest.get("_release_stage"): + flag["release_stage"] = str(manifest.get("_release_stage") or "") + if manifest.get("_release_mode"): + flag["release_mode"] = str(manifest.get("_release_mode") or "") return with_risk_business_stage( - { - "source": "submission_review", - "hit_source": "rule_center", - "rule_type": "risk", - "rule_code": str(manifest.get("rule_code") or "").strip(), - "rule_version": str(manifest.get("_rule_version") or "v1.0.0").strip(), - "basic_rule_code": basic_rule_code, - "basic_rule_sheet": basic_rule_sheet, - "basic_rule_refs": basic_rule_refs, - "finance_rule_code": finance_rule_code, - "finance_rule_sheet": finance_rule_sheet, - "severity": severity, - "action": action, - "label": label, - "message": message, - "evidence": evidence, - "risk_domain": risk_domain, - "visibility_scope": visibility_scope, - "actionability": actionability, - }, + flag, business_stage, ) diff --git a/server/src/app/services/expense_claim_pre_review.py b/server/src/app/services/expense_claim_pre_review.py index 15b233f..9771c57 100644 --- a/server/src/app/services/expense_claim_pre_review.py +++ b/server/src/app/services/expense_claim_pre_review.py @@ -98,9 +98,9 @@ class ExpenseClaimPreReviewMixin: "source": "ai_pre_review", "event_type": "expense_claim_ai_pre_review", "severity": "info" if passed else "high", - "label": "自动检测通过" if decision == "ready" else ( - "自动检测待复核" if passed else "自动检测未通过" - ), + "label": "自动检测通过" + if decision == "ready" + else ("自动检测待复核" if passed else "自动检测未通过"), "message": str(decision_payload.get("message") or ""), "status": "passed" if passed else "failed", "passed": passed, @@ -122,8 +122,7 @@ class ExpenseClaimPreReviewMixin: flag for flag in list(risk_flags or []) if not ( - isinstance(flag, dict) - and str(flag.get("source") or "").strip() == "ai_pre_review" + isinstance(flag, dict) and str(flag.get("source") or "").strip() == "ai_pre_review" ) ] return [*preserved_flags, next_flag] @@ -172,6 +171,7 @@ class ExpenseClaimPreReviewMixin: application_review = self.evaluate_platform_risk_rules( claim, business_stage="expense_application", + tenant_id=tenant_id, ) review_flags = dedupe_claim_risk_flags( [*preserved_flags, *list(application_review.get("flags") or [])] @@ -182,9 +182,7 @@ class ExpenseClaimPreReviewMixin: else: review_result = self._run_ai_submission_review(claim) review_flags = list(review_result.get("risk_flags") or []) - platform_rule_set_fingerprint = str( - review_result.get("rule_set_fingerprint") or "" - ) + platform_rule_set_fingerprint = str(review_result.get("rule_set_fingerprint") or "") business_stage = risk_business_stage_for_claim( is_application_claim=is_application_claim, @@ -196,9 +194,7 @@ class ExpenseClaimPreReviewMixin: platform_rule_set_fingerprint=platform_rule_set_fingerprint, reviewed_at=reviewed_at, ) - historical_case_evidence = ExpenseClaimHistoricalEvidenceService( - self.db - ).retrieve( + historical_case_evidence = ExpenseClaimHistoricalEvidenceService(self.db).retrieve( claim, tenant_id=tenant_id, business_stage=business_stage, @@ -245,18 +241,14 @@ class ExpenseClaimPreReviewMixin: extra_payload={ "review_id": review_id, "input_fingerprint": str(pre_review_flag.get("input_fingerprint") or ""), - "rule_set_fingerprint": str( - pre_review_flag.get("rule_set_fingerprint") or "" - ), + "rule_set_fingerprint": str(pre_review_flag.get("rule_set_fingerprint") or ""), "review_context_fingerprint": str( pre_review_flag.get("review_context_fingerprint") or "" ), "decision": str(pre_review_flag.get("decision") or ""), "review_status": str(pre_review_flag.get("status") or ""), "passed": bool(pre_review_flag.get("passed")), - "blocking_risk_count": int( - pre_review_flag.get("blocking_risk_count") or 0 - ), + "blocking_risk_count": int(pre_review_flag.get("blocking_risk_count") or 0), "business_stage": str(pre_review_flag.get("business_stage") or ""), "message": str(pre_review_flag.get("message") or ""), }, diff --git a/server/src/app/services/expense_claim_release_telemetry.py b/server/src/app/services/expense_claim_release_telemetry.py new file mode 100644 index 0000000..0185bb9 --- /dev/null +++ b/server/src/app/services/expense_claim_release_telemetry.py @@ -0,0 +1,176 @@ +"""把报销规则执行结果安全写入 Agent 资产发布遥测。""" + +from __future__ import annotations + +from typing import Any, Literal + +from sqlalchemy.engine import Connection, Engine +from sqlalchemy.orm import Session, sessionmaker + +from app.core.logging import get_logger +from app.models.financial_record import ExpenseClaim +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseTelemetryStaleRelease, +) + +logger = get_logger("app.services.expense_claim_release_telemetry") + +SUPPORTED_PLATFORM_RISK_EVALUATORS = frozenset( + { + "cross_year_invoice", + "document_expense_mismatch", + "duplicate_invoice", + "entertainment_reason_missing", + "identity_consistency", + "location_consistency", + "multi_city_reason_required", + "reason_too_brief", + "release_guard_integrity_failure", + "template_rule", + "vague_goods_description", + "void_or_red_invoice", + } +) + + +class ExpenseClaimReleaseTelemetryRecorder: + """只记录当前候选路由；遥测故障不改变报销风险结论。""" + + def __init__(self, db: Session) -> None: + self.db = db + + @staticmethod + def manifest_key(manifest: dict[str, Any]) -> tuple[str, str]: + return ( + str(manifest.get("_rule_asset_id") or "").strip(), + str(manifest.get("rule_code") or "").strip(), + ) + + @staticmethod + def baseline_hit( + manifest: dict[str, Any], + *, + enforced_outcomes: dict[tuple[str, str], bool], + release_key: tuple[str, str], + ) -> bool | None: + if ( + str(manifest.get("_release_stage") or "").strip().lower() == "shadow" + and str(manifest.get("_release_mode") or "").strip().lower() == "shadow" + ): + return enforced_outcomes.get(release_key) + return None + + def record( + self, + *, + tenant_id: str, + claim: ExpenseClaim, + manifest: dict[str, Any], + hit: bool, + baseline_hit: bool | None, + runtime_status: Literal["completed", "failed"] = "completed", + failure_code: str = "none", + business_stage: str, + ) -> None: + stage = str(manifest.get("_release_stage") or "").strip().lower() + mode = str(manifest.get("_release_mode") or "").strip().lower() + integrity_failure = ( + str(manifest.get("evaluator") or "").strip().lower() + == "release_guard_integrity_failure" + ) + if stage not in {"shadow", "canary", "active"} or not str( + manifest.get("_rule_asset_id") or "" + ).strip(): + return + if stage == "shadow" and mode != "shadow" and not integrity_failure: + return + telemetry_manifest = manifest + if integrity_failure: + integrity = manifest.get("_release_integrity_error") + candidate_version = str( + (integrity if isinstance(integrity, dict) else {}).get("candidate_version") + or "" + ).strip() + if not candidate_version: + return + telemetry_manifest = { + **manifest, + "_rule_version": candidate_version, + "_release_mode": "shadow" if stage == "shadow" else "enforced", + } + try: + AgentAssetReleaseTelemetryService(self.db).record_manifest_evaluation( + tenant_id=tenant_id, + claim_id=str(claim.id or claim.claim_no or "").strip(), + manifest=telemetry_manifest, + hit=hit, + baseline_hit=baseline_hit, + runtime_status=runtime_status, + failure_code=failure_code, + business_stage=business_stage, + ) + except ReleaseTelemetryStaleRelease: + # Canary 未命中路由使用稳定版本，不属于候选样本。 + return + except Exception: + # 发布门禁会保持 collecting，不能因为监控旁路故障改变报销结论。 + logger.exception( + "Failed to persist release telemetry tenant=%s asset=%s stage=%s", + tenant_id, + manifest.get("_rule_asset_id"), + stage, + ) + + def record_failure_durably( + self, + *, + tenant_id: str, + claim: ExpenseClaim, + manifest: dict[str, Any], + failure_code: str, + business_stage: str, + ) -> None: + """把 evaluator 崩溃样本写入独立事务，避免随报销请求一起回滚。""" + + bind = self.db.get_bind() + if bind is None or bind.dialect.name != "postgresql": + # SQLite 仅用于本地单元测试，无法在同一连接上可靠模拟两个并行事务。 + self.record( + tenant_id=tenant_id, + claim=claim, + manifest=manifest, + hit=False, + baseline_hit=None, + runtime_status="failed", + failure_code=failure_code, + business_stage=business_stage, + ) + return + + engine: Engine = bind.engine if isinstance(bind, Connection) else bind + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + try: + with factory() as durable_db: + AgentAssetReleaseTelemetryService(durable_db).record_manifest_evaluation( + tenant_id=tenant_id, + claim_id=str(claim.id or claim.claim_no or "").strip(), + manifest=manifest, + hit=False, + baseline_hit=None, + runtime_status="failed", + failure_code=failure_code, + business_stage=business_stage, + ) + durable_db.commit() + except ReleaseTelemetryStaleRelease: + # 阶段已经推进时，旧失败样本不能污染新 release。 + return + except Exception: + # 不用遥测旁路异常替换真正的 evaluator 异常，但必须留下可观测日志。 + logger.exception( + "Failed to durably persist release failure tenant=%s asset=%s stage=%s", + tenant_id, + manifest.get("_rule_asset_id"), + manifest.get("_release_stage"), + ) diff --git a/server/src/app/services/expense_claim_risk_rule_loader.py b/server/src/app/services/expense_claim_risk_rule_loader.py new file mode 100644 index 0000000..3cf6434 --- /dev/null +++ b/server/src/app/services/expense_claim_risk_rule_loader.py @@ -0,0 +1,398 @@ +"""从普通发布或 release guard 快照解析实际运行的风险规则。""" + +from __future__ import annotations + +import hashlib +from dataclasses import dataclass +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.models.agent_asset import AgentAsset +from app.services.agent_asset_release_artifacts import AgentAssetReleaseArtifactService +from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager +from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY +from app.services.risk_rule_manifest_classifier import is_budget_risk_manifest +from app.services.risk_rule_manifest_normalizer import normalize_risk_rule_manifest + + +@dataclass(frozen=True) +class _ReleaseRoute: + version: str + mode: str + + +class ExpenseClaimRiskRuleLoader: + """按租户、业务阶段与稳定路由键返回强制/影子规则。""" + + def __init__( + self, + db: Session, + *, + rule_library_manager: AgentAssetRuleLibraryManager, + ) -> None: + self.db = db + self.rule_library_manager = rule_library_manager + + def load( + self, + *, + rule_codes: list[str] | None, + business_stage: str | None, + routing_key: str, + tenant_id: str, + stage_matcher: Any, + ) -> list[dict[str, Any]]: + code_filter = { + str(code or "").strip() for code in list(rule_codes or []) if str(code or "").strip() + } + assets = list( + self.db.scalars( + select(AgentAsset) + .where( + ( + ((AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == tenant_id)) + | ( + (AgentAsset.scope == "platform") + & (AgentAsset.tenant_id == "platform") + ) + ), + AgentAsset.asset_type == AgentAssetType.RULE.value, + ) + .where(AgentAsset.domain == AgentAssetDomain.EXPENSE.value) + .order_by(AgentAsset.updated_at.desc(), AgentAsset.created_at.desc()) + ).all() + ) + assets.sort(key=lambda item: self._tenant_priority(item, tenant_id), reverse=True) + manifests_by_code: dict[str, dict[str, Any]] = {} + shadow_manifests: list[dict[str, Any]] = [] + reserved_codes: set[str] = set() + for asset in assets: + config = asset.config_json if isinstance(asset.config_json, dict) else {} + rule_code = str(asset.code or "").strip() + if rule_code and str(config.get("detail_mode") or "").strip().lower() == "json_risk": + reserved_codes.add(rule_code) + if not self._asset_is_visible(asset, config, tenant_id=tenant_id): + continue + if code_filter and rule_code not in code_filter: + continue + state = config.get("release_guard") + release_state = dict(state) if isinstance(state, dict) else {} + routes = self._release_routes(asset, release_state, routing_key=routing_key) + if not routes: + if asset.status != AgentAssetStatus.ACTIVE.value: + continue + manifest = self._load_document_manifest(asset, config) + if manifest is not None: + self._accept_manifest( + manifests_by_code, + manifest, + code_filter=code_filter, + business_stage=business_stage, + stage_matcher=stage_matcher, + ) + continue + + for route in routes: + manifest = self._load_release_manifest(asset, release_state, route) + if manifest is None and route.mode == "enforced": + manifest = self._safe_previous_manifest(asset, release_state) + if manifest is None and route.version: + # 受控发布的候选和稳定快照都不可用时不能静默跳过规则， + # 否则损坏配置会把高风险规则变成“未命中”。 + manifest = self._release_integrity_failure_manifest( + asset, + release_state, + failed_version=route.version, + ) + if manifest is None or not self._manifest_is_eligible( + manifest, + code_filter=code_filter, + business_stage=business_stage, + stage_matcher=stage_matcher, + ): + continue + if route.mode == "shadow": + shadow_manifests.append(manifest) + else: + manifests_by_code.setdefault(str(manifest["rule_code"]), manifest) + + self._load_library_fallbacks( + manifests_by_code, + reserved_codes=reserved_codes, + code_filter=code_filter, + business_stage=business_stage, + stage_matcher=stage_matcher, + ) + return [*manifests_by_code.values(), *shadow_manifests] + + @staticmethod + def _tenant_priority(asset: AgentAsset, tenant_id: str) -> tuple[int, Any, Any]: + return ( + 2 if asset.scope == "tenant" and asset.tenant_id == tenant_id else 1, + asset.updated_at, + asset.created_at, + ) + + @staticmethod + def _asset_is_visible( + asset: AgentAsset, + config: dict[str, Any], + *, + tenant_id: str, + ) -> bool: + if str(config.get("detail_mode") or "").strip().lower() != "json_risk": + return False + if asset.scope == "tenant" and asset.tenant_id != tenant_id: + return False + if asset.scope == "platform" and asset.tenant_id != "platform": + return False + if asset.status == AgentAssetStatus.DISABLED.value or config.get("enabled") is False: + return False + state = config.get("release_guard") + stage = str(state.get("stage") or "") if isinstance(state, dict) else "" + return asset.status == AgentAssetStatus.ACTIVE.value or stage in { + "shadow", + "canary", + "active", + "rolled_back", + } + + def _load_document_manifest( + self, + asset: AgentAsset, + config: dict[str, Any], + ) -> dict[str, Any] | None: + document = config.get("rule_document") + if not isinstance(document, dict): + return None + file_name = str(document.get("file_name") or "").strip() + library = str(config.get("rule_library") or RISK_RULES_LIBRARY).strip() + if not file_name: + return None + try: + payload = self.rule_library_manager.read_rule_library_json( + library=library, + file_name=file_name, + ) + except (FileNotFoundError, ValueError): + return None + payload = normalize_risk_rule_manifest(payload) + if payload.get("enabled") is False: + return None + return self._decorate( + payload, + asset=asset, + version=str(asset.published_version or asset.current_version or "v1.0.0"), + stage="unmanaged", + mode="enforced", + ) + + def _load_release_manifest( + self, + asset: AgentAsset, + state: dict[str, Any], + route: _ReleaseRoute, + ) -> dict[str, Any] | None: + try: + artifact = AgentAssetReleaseArtifactService.artifact(state, route.version) + except ValueError: + return None + manifest = dict(artifact["manifest"]) + manifest["enabled"] = True + return self._decorate( + manifest, + asset=asset, + version=route.version, + stage=str(state.get("stage") or "unmanaged"), + mode=route.mode, + ) + + def _safe_previous_manifest( + self, + asset: AgentAsset, + state: dict[str, Any], + ) -> dict[str, Any] | None: + previous = str(state.get("previous_version") or "").strip() + if not previous: + return None + return self._load_release_manifest(asset, state, _ReleaseRoute(previous, "enforced")) + + @staticmethod + def _release_integrity_failure_manifest( + asset: AgentAsset, + state: dict[str, Any], + *, + failed_version: str, + ) -> dict[str, Any]: + """把不可恢复的发布快照故障转换为强制人工阻断信号。""" + + stage = str(state.get("stage") or "unmanaged").strip() or "unmanaged" + return { + "rule_code": str(asset.code or "release_guard_integrity_failure").strip(), + "name": "风险规则发布快照完整性异常", + "description": "受控发布快照无法校验，已停止该申请自动流转。", + "evaluator": "release_guard_integrity_failure", + "enabled": True, + "applies_to": { + "domains": ["expense"], + "business_stages": ["expense_application", "reimbursement"], + }, + "outcomes": { + "fail": { + "severity": "critical", + "action": "block", + } + }, + "metadata": { + "risk_domain": "policy", + "visibility_scope": "leader", + "actionability": "review_decision", + }, + "_rule_version": failed_version, + "_rule_asset_id": asset.id, + "_release_stage": stage, + "_release_mode": "enforced", + "_release_integrity_error": { + "asset_id": asset.id, + "release_id": str(state.get("release_id") or ""), + "failed_version": failed_version, + "candidate_version": str(state.get("candidate_version") or ""), + "previous_version": str(state.get("previous_version") or ""), + }, + } + + @staticmethod + def _release_routes( + asset: AgentAsset, + state: dict[str, Any], + *, + routing_key: str, + ) -> list[_ReleaseRoute]: + stage = str(state.get("stage") or "").strip() + candidate = str(state.get("candidate_version") or "").strip() + previous = str(state.get("previous_version") or asset.published_version or "").strip() + if stage == "shadow": + routes = [_ReleaseRoute(previous, "enforced")] if previous else [] + if candidate: + routes.append(_ReleaseRoute(candidate, "shadow")) + return routes + if stage == "canary": + policy = state.get("policy") if isinstance(state.get("policy"), dict) else {} + traffic = _safe_percent(policy.get("canary_traffic_percent"), 5) + if candidate and _is_candidate_route(asset.id, routing_key, traffic): + return [_ReleaseRoute(candidate, "enforced")] + return [_ReleaseRoute(previous, "enforced")] if previous else [] + if stage == "active" and candidate: + return [_ReleaseRoute(candidate, "enforced")] + if stage == "rolled_back" and previous: + return [_ReleaseRoute(previous, "enforced")] + return [] + + @staticmethod + def _decorate( + manifest: dict[str, Any], + *, + asset: AgentAsset, + version: str, + stage: str, + mode: str, + ) -> dict[str, Any]: + payload = dict(manifest) + payload.setdefault("rule_code", str(asset.code or "").strip()) + payload["_rule_version"] = version + payload["_rule_asset_id"] = asset.id + payload["_release_stage"] = stage + payload["_release_mode"] = mode + return payload + + def _accept_manifest( + self, + target: dict[str, dict[str, Any]], + manifest: dict[str, Any], + *, + code_filter: set[str], + business_stage: str | None, + stage_matcher: Any, + ) -> None: + if self._manifest_is_eligible( + manifest, + code_filter=code_filter, + business_stage=business_stage, + stage_matcher=stage_matcher, + ): + target.setdefault(str(manifest["rule_code"]), manifest) + + @staticmethod + def _manifest_is_eligible( + manifest: dict[str, Any], + *, + code_filter: set[str], + business_stage: str | None, + stage_matcher: Any, + ) -> bool: + rule_code = str(manifest.get("rule_code") or "").strip() + return bool( + rule_code + and (not code_filter or rule_code in code_filter) + and not is_budget_risk_manifest(manifest) + and manifest.get("enabled") is not False + and stage_matcher(manifest, business_stage=business_stage) + ) + + def _load_library_fallbacks( + self, + target: dict[str, dict[str, Any]], + *, + reserved_codes: set[str], + code_filter: set[str], + business_stage: str | None, + stage_matcher: Any, + ) -> None: + try: + files = self.rule_library_manager.list_rule_library_json_files( + library=RISK_RULES_LIBRARY + ) + except ValueError: + return + for file_name in files: + try: + payload = normalize_risk_rule_manifest( + self.rule_library_manager.read_rule_library_json( + library=RISK_RULES_LIBRARY, + file_name=file_name, + ) + ) + except (FileNotFoundError, ValueError): + continue + payload = dict(payload) + rule_code = str(payload.get("rule_code") or "").strip() + if rule_code in reserved_codes: + continue + payload["_rule_version"] = "v1.0.0" + payload["_release_stage"] = "library_fallback" + payload["_release_mode"] = "enforced" + self._accept_manifest( + target, + payload, + code_filter=code_filter, + business_stage=business_stage, + stage_matcher=stage_matcher, + ) + + +def _safe_percent(value: Any, default: int) -> int: + try: + parsed = int(value) + except (TypeError, ValueError, OverflowError): + parsed = default + return max(0, min(100, parsed)) + + +def _is_candidate_route(asset_id: str, routing_key: str, traffic_percent: int) -> bool: + if not routing_key or traffic_percent <= 0: + return False + digest = hashlib.sha256(f"{asset_id}:{routing_key}".encode()).digest() + return int.from_bytes(digest[:8], "big") % 100 < traffic_percent diff --git a/server/src/app/services/expense_claim_risk_stage.py b/server/src/app/services/expense_claim_risk_stage.py index 068d15f..7e3fe2b 100644 --- a/server/src/app/services/expense_claim_risk_stage.py +++ b/server/src/app/services/expense_claim_risk_stage.py @@ -80,7 +80,49 @@ def normalize_risk_actionability(value: Any, default: str = "review_decision") - def risk_business_stage_for_claim(*, is_application_claim: bool) -> str: - return EXPENSE_APPLICATION_BUSINESS_STAGE if is_application_claim else REIMBURSEMENT_BUSINESS_STAGE + return ( + EXPENSE_APPLICATION_BUSINESS_STAGE if is_application_claim else REIMBURSEMENT_BUSINESS_STAGE + ) + + +def build_shadow_release_evaluation( + manifest: dict[str, Any], + flag: dict[str, Any] | None, +) -> dict[str, Any]: + """生成不含业务正文的 shadow 运行摘要。""" + + return { + "asset_id": str(manifest.get("_rule_asset_id") or ""), + "rule_code": str(manifest.get("rule_code") or ""), + "rule_version": str(manifest.get("_rule_version") or ""), + "release_stage": str(manifest.get("_release_stage") or "shadow"), + "hit": flag is not None, + "severity": str((flag or {}).get("severity") or "none"), + } + + +def risk_manifest_matches_business_stage( + manifest: dict[str, Any], + *, + business_stage: str | None, +) -> bool: + """判断规则声明的业务阶段是否覆盖当前申请或报销流程。""" + + if not business_stage: + return True + applies_to = manifest.get("applies_to") + applies_to = applies_to if isinstance(applies_to, dict) else {} + raw_stages = applies_to.get("business_stages") + if not isinstance(raw_stages, list): + metadata = manifest.get("metadata") + metadata = metadata if isinstance(metadata, dict) else {} + raw_stages = [ + manifest.get("business_stage") + or metadata.get("business_stage") + or REIMBURSEMENT_BUSINESS_STAGE + ] + stages = {normalize_risk_business_stage(item) for item in raw_stages if str(item or "").strip()} + return business_stage in (stages or {REIMBURSEMENT_BUSINESS_STAGE}) def risk_flag_business_stage(flag: dict[str, Any], default: str = "") -> str: @@ -94,11 +136,7 @@ def risk_flag_business_stage(flag: dict[str, Any], default: str = "") -> str: def infer_risk_domain(flag: dict[str, Any]) -> str: - explicit_domain = ( - flag.get("risk_domain") - or flag.get("riskDomain") - or flag.get("domain") - ) + explicit_domain = flag.get("risk_domain") or flag.get("riskDomain") or flag.get("domain") if explicit_domain: return normalize_risk_domain(explicit_domain) @@ -129,7 +167,9 @@ def infer_risk_domain(flag: dict[str, Any]) -> str: ).lower() if any(token in corpus for token in ["预算", "budget"]): return "budget" - if any(token in corpus for token in ["发票", "票据", "单据", "附件", "ocr", "invoice", "receipt"]): + if any( + token in corpus for token in ["发票", "票据", "单据", "附件", "ocr", "invoice", "receipt"] + ): return "invoice" trip_tokens = [ "行程", @@ -149,11 +189,17 @@ def infer_risk_domain(flag: dict[str, Any]) -> str: ] if any(token in corpus for token in trip_tokens): return "trip" - if any(token in corpus for token in ["金额", "超标", "阈值", "额度", "标准", "amount", "limit", "over"]): + if any( + token in corpus + for token in ["金额", "超标", "阈值", "额度", "标准", "amount", "limit", "over"] + ): return "amount" if any(token in corpus for token in ["历史", "画像", "异常关系", "profile", "baseline"]): return "profile" - if any(token in corpus for token in ["审批", "退回", "流程", "付款", "routing", "approval", "return", "payment"]): + if any( + token in corpus + for token in ["审批", "退回", "流程", "付款", "routing", "approval", "return", "payment"] + ): return "workflow" return "policy" @@ -206,7 +252,9 @@ def enrich_risk_flag_semantics( flag, business_stage=stage, ) - domain = normalize_risk_domain(risk_domain or flag.get("risk_domain") or flag.get("riskDomain"), inferred_domain) + domain = normalize_risk_domain( + risk_domain or flag.get("risk_domain") or flag.get("riskDomain"), inferred_domain + ) scope = normalize_risk_visibility_scope( visibility_scope or flag.get("visibility_scope") or flag.get("visibilityScope"), inferred_scope, diff --git a/server/src/app/services/expense_claim_standard_adjustment.py b/server/src/app/services/expense_claim_standard_adjustment.py index a3f21a3..42ead4e 100644 --- a/server/src/app/services/expense_claim_standard_adjustment.py +++ b/server/src/app/services/expense_claim_standard_adjustment.py @@ -1,10 +1,17 @@ from __future__ import annotations +import hashlib +import json import re +import threading +from collections.abc import Iterator +from contextlib import contextmanager from datetime import UTC, datetime from decimal import Decimal, InvalidOperation from typing import Any +from sqlalchemy import select, text + from app.api.deps import CurrentUserContext from app.models.financial_record import ExpenseClaim, ExpenseClaimItem from app.schemas.reimbursement import ( @@ -14,9 +21,107 @@ from app.schemas.reimbursement import ( from app.services.expense_claim_constants import STANDARD_ADJUSTMENT_RISK_SOURCE from app.services.expense_claim_risk_flags import dedupe_claim_risk_flags from app.services.expense_claim_risk_stage import with_risk_business_stage +from app.services.savings_discovery import SavingsDiscoveryService class ExpenseClaimStandardAdjustmentMixin: + _standard_adjustment_lock_registry_guard = threading.Lock() + _standard_adjustment_locks: dict[str, tuple[threading.RLock, int]] = {} + + @staticmethod + def _signed_standard_adjustment_lock_id(value: str) -> int: + unsigned = int.from_bytes( + hashlib.sha256(value.encode("utf-8")).digest()[:8], + byteorder="big", + signed=False, + ) + return unsigned - (1 << 64) if unsigned >= (1 << 63) else unsigned + + @classmethod + @contextmanager + def _serialize_standard_adjustment_fallback(cls, name: str) -> Iterator[None]: + with cls._standard_adjustment_lock_registry_guard: + lock, references = cls._standard_adjustment_locks.get( + name, + (threading.RLock(), 0), + ) + cls._standard_adjustment_locks[name] = (lock, references + 1) + try: + with lock: + yield + finally: + with cls._standard_adjustment_lock_registry_guard: + current = cls._standard_adjustment_locks.get(name) + if current is None or current[0] is not lock: + pass + elif current[1] <= 1: + cls._standard_adjustment_locks.pop(name, None) + else: + cls._standard_adjustment_locks[name] = (lock, current[1] - 1) + + @contextmanager + def _serialize_standard_adjustment( + self, + *, + claim_id: str, + current_user: CurrentUserContext, + ) -> Iterator[None]: + tenant_id = str(current_user.tenant_id or "default").strip() or "default" + lock_name = f"expense-standard-adjustment:{tenant_id}:{claim_id}" + bind = self.db.get_bind() + dialect_name = str(bind.dialect.name if bind is not None else "").lower() + if dialect_name == "postgresql": + self.db.execute( + text("SELECT pg_advisory_xact_lock(:lock_id)"), + {"lock_id": self._signed_standard_adjustment_lock_id(lock_name)}, + ) + yield + return + + with self._serialize_standard_adjustment_fallback(lock_name): + yield + + def _supports_standard_adjustment_row_lock(self) -> bool: + """SQLite 不支持行锁；生产数据库则锁住单据及明细后再读取金额事实。""" + + bind = self.db.get_bind() + return str(bind.dialect.name or "").strip().lower() != "sqlite" + + def _get_locked_standard_adjustment_claim( + self, + *, + claim_id: str, + current_user: CurrentUserContext, + ) -> tuple[ExpenseClaim, dict[str, ExpenseClaimItem]] | None: + claim_stmt = select(ExpenseClaim).where(ExpenseClaim.id == claim_id) + claim_stmt = self._access_policy.apply_claim_scope( + claim_stmt, + current_user, + include_approval_scope=True, + ) + supports_row_lock = self._supports_standard_adjustment_row_lock() + if supports_row_lock: + claim_stmt = claim_stmt.with_for_update() + + claim = self.db.scalar(claim_stmt.execution_options(populate_existing=True)) + if claim is None: + return None + + item_stmt = ( + select(ExpenseClaimItem) + .where(ExpenseClaimItem.claim_id == claim.id) + .order_by(ExpenseClaimItem.item_date.asc(), ExpenseClaimItem.id.asc()) + ) + if supports_row_lock: + item_stmt = item_stmt.with_for_update() + locked_items = list( + self.db.scalars(item_stmt.execution_options(populate_existing=True)).all() + ) + item_map = { + str(item.id or "").strip(): item for item in locked_items if str(item.id or "").strip() + } + return self._access_policy.attach_approval_snapshot(claim), item_map + @staticmethod def _normalize_standard_adjustment_amount(value: Any) -> Decimal | None: try: @@ -63,6 +168,11 @@ class ExpenseClaimStandardAdjustmentMixin: for flag in list(claim.risk_flags_json or []): if not isinstance(flag, dict): continue + if str(flag.get("source") or "").strip() not in { + "application_link", + "application_handoff", + }: + continue detail = flag.get("application_detail") or flag.get("applicationDetail") if isinstance(detail, dict): details.append(detail) @@ -75,14 +185,8 @@ class ExpenseClaimStandardAdjustmentMixin: self, claim: ExpenseClaim, item: ExpenseClaimItem, - entry: Any, ) -> int: - direct_days = self._normalize_standard_adjustment_days( - getattr(entry, "application_days", None) - ) - if direct_days is not None: - return direct_days - + # 关联申请由服务端持久化，优先于本次请求里可被篡改的 application_days。 for detail in self._iter_standard_adjustment_application_details(claim): for key in ("application_days", "applicationDays", "days"): detail_days = self._normalize_standard_adjustment_days(detail.get(key)) @@ -90,13 +194,12 @@ class ExpenseClaimStandardAdjustmentMixin: return detail_days candidates = [ - getattr(entry, "risk", None), - getattr(entry, "title", None), item.item_reason, + item.item_note, claim.reason, ] - for text in candidates: - match = re.search(r"(\d{1,3})\s*(?:天|晚|夜)", str(text or "")) + for candidate_text in candidates: + match = re.search(r"(\d{1,3})\s*(?:天|晚|夜)", str(candidate_text or "")) if match: days = self._normalize_standard_adjustment_days(match.group(1)) if days is not None: @@ -125,9 +228,8 @@ class ExpenseClaimStandardAdjustmentMixin: *, claim: ExpenseClaim, item: ExpenseClaimItem, - entry: Any, current_user: CurrentUserContext, - ) -> Decimal | None: + ) -> dict[str, Any] | None: item_type = str(item.item_type or "").strip().lower() if item_type not in {"hotel", "hotel_ticket"}: return None @@ -144,39 +246,190 @@ class ExpenseClaimStandardAdjustmentMixin: result = TravelReimbursementCalculatorService(self.db).calculate( TravelReimbursementCalculatorRequest( - days=self._resolve_standard_adjustment_days(claim, item, entry), + days=self._resolve_standard_adjustment_days(claim, item), location=location, grade=grade, ), current_user, ) - except Exception: + except ValueError: return None - return self._normalize_standard_adjustment_amount(result.hotel_amount) + policy_amount = self._normalize_standard_adjustment_amount(result.hotel_amount) + if policy_amount is None or policy_amount <= Decimal("0.00"): + return None + rule_name = str(result.rule_name or "公司差旅费报销规则").strip() + published_rule_version = str(result.rule_version or "").strip() + version_material = { + "rule_name": rule_name, + "days": int(result.days), + "location": str(result.location or location).strip(), + "matched_city": str(result.matched_city or "").strip(), + "grade": str(result.grade or grade).strip(), + "grade_band": str(result.grade_band or "").strip(), + "hotel_rate": self._format_adjustment_money(Decimal(result.hotel_rate)), + "hotel_amount": self._format_adjustment_money(policy_amount), + } + calculated_version = ( + "content-sha256:" + + hashlib.sha256( + json.dumps( + version_material, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + ).hexdigest()[:16] + ) + return { + "amount": policy_amount, + "days": int(result.days), + "location": str(result.location or location).strip(), + "matched_city": str(result.matched_city or "").strip(), + "grade": str(result.grade or grade).strip(), + "grade_band": str(result.grade_band or "").strip(), + "grade_band_label": str(result.grade_band_label or "").strip(), + "hotel_rate": self._normalize_standard_adjustment_amount(result.hotel_rate), + "hotel_amount": policy_amount, + "rule_name": rule_name, + "rule_version": published_rule_version or calculated_version, + "rule_version_source": "published" if published_rule_version else "content_fingerprint", + } + + @staticmethod + def _standard_adjustment_timestamp(value: datetime | None) -> datetime | None: + if value is None: + return None + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value.astimezone(UTC) + + def _ensure_standard_adjustment_version( + self, + claim: ExpenseClaim, + expected_updated_at: datetime | None, + ) -> None: + if expected_updated_at is None: + return + expected = self._standard_adjustment_timestamp(expected_updated_at) + actual = self._standard_adjustment_timestamp(claim.updated_at) + if actual is None or expected != actual: + raise ValueError("报销单内容已被其他操作更新，请刷新页面后重新确认报销标准。") + + @staticmethod + def _standard_adjustment_request_fingerprint( + *, + claim_id: str, + item_ids: list[str], + ) -> str: + material = { + "action": "standard_adjustment_accept", + "claim_id": str(claim_id), + "item_ids": sorted(item_ids), + } + return ( + "sha256:" + + hashlib.sha256( + json.dumps(material, sort_keys=True, separators=(",", ":")).encode("utf-8") + ).hexdigest() + ) + + @staticmethod + def _standard_adjustment_calculation_fingerprint( + *, + item_id: str, + original_amount: Decimal, + reimbursable_amount: Decimal, + policy_result: dict[str, Any], + ) -> str: + material = { + "item_id": item_id, + "original_amount": f"{original_amount:.2f}", + "reimbursable_amount": f"{reimbursable_amount:.2f}", + "policy_days": int(policy_result["days"]), + "policy_location": str(policy_result["location"]), + "policy_matched_city": str(policy_result["matched_city"]), + "policy_grade": str(policy_result["grade"]), + "policy_grade_band": str(policy_result["grade_band"]), + "policy_hotel_rate": f"{Decimal(policy_result['hotel_rate']):.2f}", + "policy_hotel_amount": f"{Decimal(policy_result['hotel_amount']):.2f}", + "policy_rule_name": str(policy_result["rule_name"]), + "policy_rule_version": str(policy_result["rule_version"]), + } + return ( + "sha256:" + + hashlib.sha256( + json.dumps( + material, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + ).hexdigest() + ) + + @staticmethod + def _standard_adjustment_flags_by_item(claim: ExpenseClaim) -> dict[str, dict[str, Any]]: + flags: dict[str, dict[str, Any]] = {} + for flag in list(claim.risk_flags_json or []): + if not isinstance(flag, dict): + continue + if str(flag.get("source") or "").strip() != STANDARD_ADJUSTMENT_RISK_SOURCE: + continue + item_id = str(flag.get("item_id") or "").strip() + if item_id: + flags[item_id] = flag + return flags + + def _is_standard_adjustment_request_replay( + self, + *, + claim: ExpenseClaim, + request_id: str | None, + request_fingerprint: str, + item_ids: set[str], + ) -> bool: + if not request_id: + return False + matched_flags = [ + flag + for flag in self._standard_adjustment_flags_by_item(claim).values() + if str(flag.get("request_id") or "").strip() == request_id + ] + if not matched_flags: + return False + fingerprints = { + str(flag.get("request_fingerprint") or "").strip() for flag in matched_flags + } + matched_item_ids = { + str(flag.get("item_id") or "").strip() + for flag in matched_flags + if str(flag.get("item_id") or "").strip() + } + if fingerprints != {request_fingerprint} or matched_item_ids != item_ids: + raise ValueError("该 request_id 已用于另一组报销标准调整，请生成新的 request_id。") + return True def _resolve_standard_adjustment_reimbursable_amount( self, *, claim: ExpenseClaim, item: ExpenseClaimItem, - entry: Any, original_amount: Decimal, current_user: CurrentUserContext, - ) -> Decimal: - policy_amount = self._resolve_policy_standard_reimbursable_amount( + ) -> tuple[Decimal, dict[str, Any]] | None: + policy_result = self._resolve_policy_standard_reimbursable_amount( claim=claim, item=item, - entry=entry, current_user=current_user, ) - if policy_amount is not None: - return min(max(policy_amount, Decimal("0.00")), original_amount) - - entry_amount = self._normalize_standard_adjustment_amount(entry.reimbursable_amount) - if entry_amount is not None: - return min(max(entry_amount, Decimal("0.00")), original_amount) - return original_amount + if policy_result is None: + return None + policy_amount = Decimal(policy_result["amount"]).quantize(Decimal("0.01")) + return ( + min(max(policy_amount, Decimal("0.00")), original_amount), + policy_result, + ) def accept_standard_adjustment( self, @@ -185,9 +438,46 @@ class ExpenseClaimStandardAdjustmentMixin: payload: ExpenseClaimStandardAdjustmentPayload, current_user: CurrentUserContext, ) -> ExpenseClaim | None: - claim = self.get_claim(claim_id, current_user) - if claim is None: + try: + with self._serialize_standard_adjustment( + claim_id=claim_id, + current_user=current_user, + ): + claim = self._accept_standard_adjustment_once( + claim_id=claim_id, + payload=payload, + current_user=current_user, + ) + if claim is None: + # 只读未命中也要结束事务，避免 PostgreSQL advisory/row lock + # 留到 Session 关闭时才释放。 + self.db.rollback() + return claim + except Exception: + self.db.rollback() + raise + + def _complete_standard_adjustment_replay(self, claim: ExpenseClaim) -> ExpenseClaim: + """结束只读幂等重放事务，确保数据库锁立即释放。""" + + self.db.commit() + self.db.refresh(claim) + return self._access_policy.attach_approval_snapshot(claim) + + def _accept_standard_adjustment_once( + self, + *, + claim_id: str, + payload: ExpenseClaimStandardAdjustmentPayload, + current_user: CurrentUserContext, + ) -> ExpenseClaim | None: + locked_claim = self._get_locked_standard_adjustment_claim( + claim_id=claim_id, + current_user=current_user, + ) + if locked_claim is None: return None + claim, item_map = locked_claim self._ensure_draft_claim(claim) if self._is_expense_application_claim(claim): @@ -197,36 +487,62 @@ class ExpenseClaimStandardAdjustmentMixin: if not risk_entries: raise ValueError("请至少选择一条需要按职级标准重算的风险。") + entries_by_item: dict[str, Any] = {} + for entry in risk_entries: + item_id = str(entry.item_id or "").strip() + if not item_id: + raise ValueError("需要按报销标准重算的风险缺少费用明细标识。") + entries_by_item.setdefault(item_id, entry) + selected_item_ids = set(entries_by_item) + missing_item_ids = sorted(selected_item_ids - set(item_map)) + if missing_item_ids: + raise ValueError("费用明细已变化，请刷新页面后重新选择需要调整的风险。") + + request_fingerprint = self._standard_adjustment_request_fingerprint( + claim_id=claim.id, + item_ids=list(selected_item_ids), + ) + request_id = str(payload.request_id or "").strip() or None + if self._is_standard_adjustment_request_replay( + claim=claim, + request_id=request_id, + request_fingerprint=request_fingerprint, + item_ids=selected_item_ids, + ): + return self._complete_standard_adjustment_replay(claim) + self._ensure_standard_adjustment_version(claim, payload.expected_updated_at) + before_json = self._serialize_claim(claim) - item_map = {str(item.id or "").strip(): item for item in list(claim.items or [])} now_text = datetime.now(UTC).isoformat() adjustment_flags: list[dict[str, Any]] = [] + unavailable_item_labels: list[str] = [] - for index, entry in enumerate(risk_entries, start=1): - item_id = str(entry.item_id or "").strip() + for index, (item_id, entry) in enumerate(entries_by_item.items(), start=1): item = item_map.get(item_id) - if item is None: - continue + if item is None: # pragma: no cover - item map was validated above + raise ValueError("费用明细已变化，请刷新页面后重试。") - original_amount = ( - self._normalize_standard_adjustment_amount(entry.original_amount) - or Decimal(item.item_amount or Decimal("0.00")).quantize(Decimal("0.01")) - ) - reimbursable_amount = self._resolve_standard_adjustment_reimbursable_amount( - claim=claim, - item=item, - entry=entry, - original_amount=original_amount, - current_user=current_user, - ) - employee_absorbed_amount = (original_amount - reimbursable_amount).quantize( - Decimal("0.01") - ) + original_amount = self._normalize_standard_adjustment_amount(item.item_amount) + if original_amount is None: + raise ValueError("费用明细金额异常，无法按服务端报销标准重算。") item_label = ( str(item.item_reason or "").strip() or str(entry.title or "").strip() or f"费用明细第 {index} 条" ) + adjustment_result = self._resolve_standard_adjustment_reimbursable_amount( + claim=claim, + item=item, + original_amount=original_amount, + current_user=current_user, + ) + if adjustment_result is None: + unavailable_item_labels.append(item_label) + continue + reimbursable_amount, policy_result = adjustment_result + employee_absorbed_amount = (original_amount - reimbursable_amount).quantize( + Decimal("0.01") + ) source_risk = str(entry.risk or entry.title or "原风险未补充异常说明").strip() message = ( f"提交人已选择按职级最高报销标准审核：{item_label} 原票据金额 " @@ -243,18 +559,46 @@ class ExpenseClaimStandardAdjustmentMixin: "label": "接受职级标准审核", "title": "提交人接受职级最高报销标准", "message": message, - "summary": "提交人未补充异常说明，已选择按职级最高报销标准重算实际报销金额。", - "suggestion": "领导和财务审批时请确认该差额由员工自行承担，并按实际报销金额入账。", + "summary": ( + "提交人未补充异常说明，已选择按职级最高报销标准重算实际报销金额。" + ), + "suggestion": ( + "领导和财务审批时请确认该差额由员工自行承担，并按实际报销金额入账。" + ), "risk_id": str(entry.risk_id or "").strip(), "source_risk": source_risk, "item_id": item_id, "original_amount": self._format_adjustment_money(original_amount), - "reimbursable_amount": self._format_adjustment_money( - reimbursable_amount - ), + "reimbursable_amount": self._format_adjustment_money(reimbursable_amount), "employee_absorbed_amount": self._format_adjustment_money( employee_absorbed_amount ), + "calculation_source": "server_policy", + "policy_days": int(policy_result["days"]), + "policy_location": str(policy_result["location"]), + "policy_matched_city": str(policy_result["matched_city"]), + "policy_grade": str(policy_result["grade"]), + "policy_grade_band": str(policy_result["grade_band"]), + "policy_grade_band_label": str(policy_result["grade_band_label"]), + "policy_hotel_rate": self._format_adjustment_money( + Decimal(policy_result["hotel_rate"] or Decimal("0.00")) + ), + "policy_hotel_amount": self._format_adjustment_money( + Decimal(policy_result["hotel_amount"] or Decimal("0.00")) + ), + "policy_rule_name": str(policy_result["rule_name"]), + "policy_rule_version": str(policy_result["rule_version"]), + "policy_rule_version_source": str(policy_result["rule_version_source"]), + "calculation_fingerprint": ( + self._standard_adjustment_calculation_fingerprint( + item_id=item_id, + original_amount=original_amount, + reimbursable_amount=reimbursable_amount, + policy_result=policy_result, + ) + ), + "request_id": request_id, + "request_fingerprint": request_fingerprint, "risk_domain": "amount", "actionability": "review_decision", "visibility_scope": "leader", @@ -264,26 +608,46 @@ class ExpenseClaimStandardAdjustmentMixin: ) ) + if unavailable_item_labels: + labels = "、".join(dict.fromkeys(unavailable_item_labels)) + raise ValueError( + f"{labels} 暂未取得服务端可验证的报销标准，本次未执行金额调整；" + "请完善职级、地点或关联申请后重试，也可以补充异常说明。" + ) + if not adjustment_flags: raise ValueError("未找到可按职级标准重算的费用明细。") + existing_by_item = self._standard_adjustment_flags_by_item(claim) + if all( + str(existing_by_item.get(item_id, {}).get("calculation_fingerprint") or "") + == str(flag.get("calculation_fingerprint") or "") + for item_id, flag in ( + (str(flag.get("item_id") or ""), flag) for flag in adjustment_flags + ) + ): + return self._complete_standard_adjustment_replay(claim) + preserved_flags = [ flag for flag in list(claim.risk_flags_json or []) if not ( isinstance(flag, dict) - and str(flag.get("source") or "").strip() - == STANDARD_ADJUSTMENT_RISK_SOURCE + and str(flag.get("source") or "").strip() == STANDARD_ADJUSTMENT_RISK_SOURCE + and str(flag.get("item_id") or "").strip() in selected_item_ids ) ] - claim.risk_flags_json = dedupe_claim_risk_flags( - [*preserved_flags, *adjustment_flags] - ) + claim.risk_flags_json = dedupe_claim_risk_flags([*preserved_flags, *adjustment_flags]) self._sync_claim_from_items(claim) self.refresh_claim_pre_review_state(claim, is_application_claim=False) - self.db.commit() - self.db.refresh(claim) + SavingsDiscoveryService(self.db).discover_standard_adjustments( + claim=claim, + items_by_id=item_map, + adjustment_flags=adjustment_flags, + current_user=current_user, + request_id=request_id, + ) self.audit_service.log_action( actor=current_user.name or current_user.username, @@ -292,6 +656,11 @@ class ExpenseClaimStandardAdjustmentMixin: resource_id=claim.id, before_json=before_json, after_json=self._serialize_claim(claim), + request_id=request_id, + commit=False, ) + self.db.commit() + self.db.refresh(claim) + return claim diff --git a/server/src/app/services/expense_claim_tenant_scope.py b/server/src/app/services/expense_claim_tenant_scope.py index faccb48..2eb06cc 100644 --- a/server/src/app/services/expense_claim_tenant_scope.py +++ b/server/src/app/services/expense_claim_tenant_scope.py @@ -2,19 +2,17 @@ from __future__ import annotations from typing import Any -from sqlalchemy import or_, select +from sqlalchemy import select from app.api.deps import CurrentUserContext -from app.models.expense_case import ExpenseCaseLink from app.models.financial_record import ExpenseClaim - -DEFAULT_TENANT_ID = "default" +from app.services.tenant_registry import required_tenant_id class ExpenseClaimTenantScopeMixin: @staticmethod def normalize_tenant_id(value: str | None) -> str: - return str(value or DEFAULT_TENANT_ID).strip() or DEFAULT_TENANT_ID + return required_tenant_id(value) @classmethod def normalize_context_tenant_id(cls, context_json: dict[str, Any] | None) -> str: @@ -25,44 +23,22 @@ class ExpenseClaimTenantScopeMixin: @classmethod def build_claim_tenant_condition(cls, tenant_id: str | None) -> Any: - """按 Expense Case Link 隔离 Claim；默认租户兼容尚未回填的旧单。""" + """Claim 自身保存结构化租户，任何查询都直接在首个 SQL 过滤。""" - normalized_tenant = cls.normalize_tenant_id(tenant_id) - same_tenant_link = ( - select(ExpenseCaseLink.id) - .where( - ExpenseCaseLink.resource_type == "expense_claim", - ExpenseCaseLink.resource_id == ExpenseClaim.id, - ExpenseCaseLink.tenant_id == normalized_tenant, - ) - .exists() - ) - if normalized_tenant != DEFAULT_TENANT_ID: - return same_tenant_link - - any_tenant_link = ( - select(ExpenseCaseLink.id) - .where( - ExpenseCaseLink.resource_type == "expense_claim", - ExpenseCaseLink.resource_id == ExpenseClaim.id, - ) - .exists() - ) - return or_(same_tenant_link, ~any_tenant_link) + return ExpenseClaim.tenant_id == cls.normalize_tenant_id(tenant_id) @classmethod def resolve_claim_tenant_id(cls, db: Any, claim_id: str | None) -> str: - """从 Case Link 解析 Claim 租户；无 Link 的历史单仍归 default。""" + """从 Claim 结构化字段解析租户；资源缺失时拒绝猜测归属。""" normalized_claim_id = str(claim_id or "").strip() if not normalized_claim_id: - return DEFAULT_TENANT_ID + raise ValueError("claim_id 不能为空。") tenant_id = db.scalar( - select(ExpenseCaseLink.tenant_id).where( - ExpenseCaseLink.resource_type == "expense_claim", - ExpenseCaseLink.resource_id == normalized_claim_id, - ) + select(ExpenseClaim.tenant_id).where(ExpenseClaim.id == normalized_claim_id) ) + if tenant_id is None: + raise LookupError("报销单不存在。") return cls.normalize_tenant_id(tenant_id) def apply_tenant_scope( diff --git a/server/src/app/services/expense_claims.py b/server/src/app/services/expense_claims.py index 99a3d29..26aab71 100644 --- a/server/src/app/services/expense_claims.py +++ b/server/src/app/services/expense_claims.py @@ -29,6 +29,10 @@ from app.services.expense_claim_application_handoff import ExpenseClaimApplicati from app.services.expense_claim_approval_flow import ExpenseClaimApprovalFlowMixin from app.services.expense_claim_approval_routing import ExpenseClaimApprovalRoutingMixin from app.services.expense_claim_attachment_analysis import ExpenseClaimAttachmentAnalysisMixin +from app.services.expense_claim_attachment_commercial import ( + stage_attachment_deletion, + stage_claim_attachment_deletion, +) from app.services.expense_claim_attachment_document import ExpenseClaimAttachmentDocumentMixin from app.services.expense_claim_attachment_operations import ExpenseClaimAttachmentOperationsMixin from app.services.expense_claim_attachment_presentation import ExpenseClaimAttachmentPresentation @@ -212,7 +216,11 @@ class ExpenseClaimItemActionMixin: item.item_type ) - self._attachment_storage.delete_item_files(item) + stage_attachment_deletion( + self.db, + storage=self._attachment_storage, + item=item, + ) claim.items = [entry for entry in claim.items if entry.id != item.id] self.db.delete(item) @@ -331,6 +339,7 @@ class ExpenseClaimItemActionMixin: platform_review = self.evaluate_platform_risk_rules( claim, business_stage="expense_application", + tenant_id=current_user.tenant_id, ) platform_flags = list(platform_review.get("flags") or []) submit_flag = with_risk_business_stage( @@ -457,7 +466,11 @@ class ExpenseClaimItemActionMixin: self._release_budget_for_delete(claim, current_user) self._delete_claim_analysis_records(resource_id) - self._attachment_storage.delete_claim_files(claim) + stage_claim_attachment_deletion( + self.db, + storage=self._attachment_storage, + claim_id=claim.id, + ) ReceiptFolderService().unlink_receipts_for_claim(resource_id) self.db.delete(claim) self.db.commit() diff --git a/server/src/app/services/expense_rule_runtime.py b/server/src/app/services/expense_rule_runtime.py index a00c9f9..9c78ff6 100644 --- a/server/src/app/services/expense_rule_runtime.py +++ b/server/src/app/services/expense_rule_runtime.py @@ -69,14 +69,24 @@ __all__ = [ class ExpenseRuleRuntimeService: - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str | None = None) -> None: self.db = db + self.tenant_id = str(tenant_id or "").strip() + + def _asset_scope_clause(self): + platform = (AgentAsset.scope == "platform") & (AgentAsset.tenant_id == "platform") + if not self.tenant_id: + return platform + return platform | ( + (AgentAsset.scope == "tenant") & (AgentAsset.tenant_id == self.tenant_id) + ) def load_catalog(self) -> ExpenseRuleCatalog: catalog = build_default_expense_rule_catalog() assets = list( self.db.scalars( select(AgentAsset) + .where(self._asset_scope_clause()) .where(AgentAsset.asset_type == AgentAssetType.RULE.value) .where(AgentAsset.status == AgentAssetStatus.ACTIVE.value) .where(AgentAsset.domain == AgentAssetDomain.EXPENSE.value) @@ -90,6 +100,7 @@ class ExpenseRuleRuntimeService: travel_spreadsheet_assets = list( self.db.scalars( select(AgentAsset) + .where(self._asset_scope_clause()) .where(AgentAsset.asset_type == AgentAssetType.RULE.value) .where(AgentAsset.domain == AgentAssetDomain.EXPENSE.value) .where(AgentAsset.code.in_(TRAVEL_SPREADSHEET_RULE_CODES)) diff --git a/server/src/app/services/expense_rule_runtime_defaults.py b/server/src/app/services/expense_rule_runtime_defaults.py index 5ba4cd1..06bbb99 100644 --- a/server/src/app/services/expense_rule_runtime_defaults.py +++ b/server/src/app/services/expense_rule_runtime_defaults.py @@ -63,13 +63,26 @@ DEFAULT_SCENE_MATRIX_CONFIG: dict[str, Any] = { "location_required": False, "min_attachment_count": 1, "allowed_scene_codes": ["transport"], - "allowed_document_types": ["taxi_receipt", "parking_toll_receipt", "vat_invoice", "receipt"], + "allowed_document_types": [ + "taxi_receipt", + "parking_toll_receipt", + "vat_invoice", + "receipt", + ], "attachment_mismatch_severity": "high", "item_amount_limit": { "scope": "item_amount", "warn_amount": "300.00", "block_amount": "800.00", - "exception_keywords": ["跨城", "夜间", "应急", "无公共交通", "机场", "火车站", "超标说明"], + "exception_keywords": [ + "跨城", + "夜间", + "应急", + "无公共交通", + "机场", + "火车站", + "超标说明", + ], "metric_label": "单笔交通金额", }, }, @@ -186,7 +199,9 @@ DEFAULT_SCENE_MATRIX_CONFIG: dict[str, Any] = { "allowed_document_types": ["vat_invoice", "receipt"], "attachment_mismatch_severity": "medium", "always_warn": True, - "always_warn_message": "其他费用默认进入人工重点复核，请补充清晰用途说明并由审批人重点确认。", + "always_warn_message": ( + "其他费用默认进入人工重点复核，请补充清晰用途说明并由审批人重点确认。" + ), "claim_amount_limit": { "scope": "claim_total", "warn_amount": "1000.00", @@ -281,6 +296,128 @@ DEFAULT_TRAVEL_POLICY_CONFIG: dict[str, Any] = { "P7": {"tier_1": "900.00", "tier_2": "820.00", "tier_3": "720.00"}, "P8": {"tier_1": "1200.00", "tier_2": "1000.00", "tier_3": "900.00"}, }, + # 与内置《公司差旅费报销规则》模板保持一致；数据库尚未初始化规则资产时， + # 计算器也必须得到同一套城市级标准，不能退回到更宽松的城市层级上限。 + "hotel_city_limits": { + "北京": { + "P0": "450.00", + "P1": "450.00", + "P2": "450.00", + "P3": "450.00", + "P4": "450.00", + "P5": "450.00", + "P6": "450.00", + "P7": "500.00", + "P8": "500.00", + }, + "上海": { + "P0": "450.00", + "P1": "450.00", + "P2": "450.00", + "P3": "450.00", + "P4": "450.00", + "P5": "450.00", + "P6": "450.00", + "P7": "500.00", + "P8": "500.00", + }, + "广州": { + "P0": "430.00", + "P1": "430.00", + "P2": "430.00", + "P3": "430.00", + "P4": "450.00", + "P5": "450.00", + "P6": "450.00", + "P7": "500.00", + "P8": "500.00", + }, + "深圳": { + "P0": "430.00", + "P1": "430.00", + "P2": "430.00", + "P3": "430.00", + "P4": "450.00", + "P5": "450.00", + "P6": "450.00", + "P7": "500.00", + "P8": "500.00", + }, + "杭州": { + "P0": "380.00", + "P1": "380.00", + "P2": "380.00", + "P3": "380.00", + "P4": "430.00", + "P5": "430.00", + "P6": "430.00", + "P7": "480.00", + "P8": "480.00", + }, + "南京": { + "P0": "380.00", + "P1": "380.00", + "P2": "380.00", + "P3": "380.00", + "P4": "430.00", + "P5": "430.00", + "P6": "430.00", + "P7": "480.00", + "P8": "480.00", + }, + "成都": { + "P0": "380.00", + "P1": "380.00", + "P2": "380.00", + "P3": "380.00", + "P4": "430.00", + "P5": "430.00", + "P6": "430.00", + "P7": "480.00", + "P8": "480.00", + }, + "武汉": { + "P0": "380.00", + "P1": "380.00", + "P2": "380.00", + "P3": "380.00", + "P4": "430.00", + "P5": "430.00", + "P6": "430.00", + "P7": "480.00", + "P8": "480.00", + }, + }, + # 数据库规则资产尚未初始化时仍提供可审计的只读兜底；租户规则表加载后会覆盖这些值。 + "allowance_limits": { + "meal": { + "直辖市/特区": "65.00", + "其他地区": "55.00", + "新疆-乌鲁木齐": "75.00", + "新疆-其他": "65.00", + "西藏": "80.00", + "港澳台": "120.00", + "国外": "180.00", + }, + "basic": { + "直辖市/特区": "35.00", + "其他地区": "35.00", + "新疆-乌鲁木齐": "45.00", + "新疆-其他": "40.00", + "西藏": "50.00", + "港澳台": "80.00", + "国外": "120.00", + }, + "total": { + "直辖市/特区": "100.00", + "其他地区": "90.00", + "新疆-乌鲁木齐": "120.00", + "新疆-其他": "105.00", + "西藏": "130.00", + "港澳台": "200.00", + "国外": "300.00", + }, + }, "transport_limits": { "P0": {"flight": 1, "train": 1}, "P1": {"flight": 1, "train": 1}, diff --git a/server/src/app/services/expense_workflow_learning.py b/server/src/app/services/expense_workflow_learning.py new file mode 100644 index 0000000..6c7d6a2 --- /dev/null +++ b/server/src/app/services/expense_workflow_learning.py @@ -0,0 +1,464 @@ +from __future__ import annotations + +import hashlib +import json +import uuid +from dataclasses import dataclass +from typing import Any + +from sqlalchemy import and_, or_, select +from sqlalchemy.orm import Session, aliased + +from app.models.ai_learning import AIDecision, AIDecisionFeedback, WorkflowOutcome +from app.models.expense_case import BusinessEvent, ExpenseCase, ExpenseCaseLink + +SUBMISSION_EVENT_TYPES = {"application_submitted", "claim_submitted"} +AUDIT_EVENT_TYPES = { + "application_pre_review_completed", + "claim_pre_review_completed", + "audit_conclusion_recorded", +} +SUPPORTED_EVENT_TYPES = { + *AUDIT_EVENT_TYPES, + "application_returned", + "claim_returned", + "application_approved", + "claim_approved", + "approval_stage_completed", + "approval_overridden", + "payment_completed", +} + + +@dataclass(frozen=True, slots=True) +class WorkflowLearningSpec: + outcome_type: str + feedback_type: str | None = None + verification_status: str = "server_verified" + training_eligible: bool = False + + +@dataclass(frozen=True, slots=True) +class WorkflowLearningRecords: + outcome: WorkflowOutcome + feedback: AIDecisionFeedback | None + + +class ExpenseWorkflowLearningService: + """把服务端费用事件转成可追溯、可幂等复放的 AI 学习证据。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def record_event(self, event: BusinessEvent) -> list[WorkflowLearningRecords]: + """只接受数据库中真实存在且租户、Case、单据关联一致的业务事件。""" + + if str(event.event_type or "").strip() not in SUPPORTED_EVENT_TYPES: + return [] + trusted_event = self._trusted_event(event) + records = [ + self._record_for_decision(trusted_event, decision) + for decision in self._eligible_decisions(trusted_event) + ] + self.db.flush() + return records + + def record_prior_events_for_decision( + self, + decision: AIDecision, + ) -> list[WorkflowLearningRecords]: + """补录提交决策创建前、同一请求内已落库的预审结论。""" + + trusted_decision = self.db.scalar( + select(AIDecision).where( + AIDecision.id == decision.id, + AIDecision.tenant_id == decision.tenant_id, + AIDecision.expense_case_id == decision.expense_case_id, + ) + ) + if trusted_decision is None: + raise PermissionError("AI 决策不存在或不属于指定租户与费用 Case。") + anchor = self._decision_anchor(trusted_decision) + if anchor is None: + return [] + + events = list( + self.db.scalars( + select(BusinessEvent) + .where( + BusinessEvent.tenant_id == trusted_decision.tenant_id, + BusinessEvent.expense_case_id == trusted_decision.expense_case_id, + BusinessEvent.aggregate_type == "expense_claim", + BusinessEvent.aggregate_id == anchor.aggregate_id, + BusinessEvent.event_type.in_(AUDIT_EVENT_TYPES), + BusinessEvent.correlation_id == anchor.correlation_id, + ) + .order_by(BusinessEvent.occurred_at.asc(), BusinessEvent.id.asc()) + ).all() + ) + records = [ + self._record_for_decision(self._trusted_event(event), trusted_decision) + for event in events + ] + self.db.flush() + return records + + def _eligible_decisions(self, event: BusinessEvent) -> list[AIDecision]: + anchor = aliased(BusinessEvent) + return list( + self.db.scalars( + select(AIDecision) + .join( + anchor, + and_( + anchor.tenant_id == AIDecision.tenant_id, + anchor.expense_case_id == AIDecision.expense_case_id, + anchor.id == AIDecision.business_event_id, + ), + ) + .where( + AIDecision.tenant_id == event.tenant_id, + AIDecision.expense_case_id == event.expense_case_id, + AIDecision.subject_type == "expense_claim", + AIDecision.status.in_(["accepted", "edited", "executed"]), + anchor.event_type.in_(SUBMISSION_EVENT_TYPES), + or_( + anchor.occurred_at <= event.occurred_at, + and_( + event.event_type in AUDIT_EVENT_TYPES, + anchor.correlation_id == event.correlation_id, + ), + ), + ) + .order_by( + anchor.occurred_at.desc(), + AIDecision.created_at.desc(), + AIDecision.id.desc(), + ) + .limit(1) + ).all() + ) + + def _trusted_event(self, event: BusinessEvent) -> BusinessEvent: + trusted = self.db.scalar( + select(BusinessEvent).where( + BusinessEvent.id == event.id, + BusinessEvent.tenant_id == event.tenant_id, + BusinessEvent.expense_case_id == event.expense_case_id, + ) + ) + if trusted is None: + raise PermissionError("业务事件不存在或不属于指定租户与费用 Case。") + if any( + trusted_value != supplied_value + for trusted_value, supplied_value in ( + (trusted.aggregate_type, event.aggregate_type), + (trusted.aggregate_id, event.aggregate_id), + (trusted.event_type, event.event_type), + (trusted.idempotency_key, event.idempotency_key), + ) + ): + raise PermissionError("业务事件身份与数据库可信事实不一致。") + if trusted.aggregate_type != "expense_claim": + raise ValueError("只有费用单据业务事件可以形成 AI 工作流学习证据。") + + expense_case = self.db.scalar( + select(ExpenseCase).where( + ExpenseCase.id == trusted.expense_case_id, + ExpenseCase.tenant_id == trusted.tenant_id, + ) + ) + claim_link = self.db.scalar( + select(ExpenseCaseLink).where( + ExpenseCaseLink.tenant_id == trusted.tenant_id, + ExpenseCaseLink.expense_case_id == trusted.expense_case_id, + ExpenseCaseLink.resource_type == "expense_claim", + ExpenseCaseLink.resource_id == trusted.aggregate_id, + ) + ) + if expense_case is None or claim_link is None: + raise PermissionError("业务事件缺少同租户费用 Case 与单据关联。") + return trusted + + def _decision_anchor(self, decision: AIDecision) -> BusinessEvent | None: + if not decision.business_event_id: + return None + return self.db.scalar( + select(BusinessEvent).where( + BusinessEvent.id == decision.business_event_id, + BusinessEvent.tenant_id == decision.tenant_id, + BusinessEvent.expense_case_id == decision.expense_case_id, + BusinessEvent.event_type.in_(SUBMISSION_EVENT_TYPES), + ) + ) + + def _record_for_decision( + self, + event: BusinessEvent, + decision: AIDecision, + ) -> WorkflowLearningRecords: + if ( + decision.tenant_id != event.tenant_id + or decision.expense_case_id != event.expense_case_id + ): + raise PermissionError("工作流结果不能关联其他租户或费用 Case 的 AI 决策。") + spec = self._event_spec(event) + safe_result = self._safe_result(event, spec) + outcome_key = f"workflow-outcome:{decision.id}:{event.id}" + outcome = self.db.scalar( + select(WorkflowOutcome).where( + WorkflowOutcome.tenant_id == event.tenant_id, + WorkflowOutcome.idempotency_key == outcome_key, + ) + ) + if outcome is None: + outcome = WorkflowOutcome( + id=self._stable_id("workflow-outcome", event.tenant_id, decision.id, event.id), + tenant_id=event.tenant_id, + expense_case_id=event.expense_case_id, + decision_id=decision.id, + business_event_id=event.id, + expense_claim_id=decision.expense_claim_id, + correlation_id=event.correlation_id, + outcome_type=spec.outcome_type, + outcome_status="verified", + actor_id=str(event.actor_id or "system").strip() or "system", + actor_type=str(event.actor_type or "system").strip() or "system", + result_json=safe_result, + idempotency_key=outcome_key, + content_fingerprint=self._fingerprint( + { + "decision_id": decision.id, + "business_event_id": event.id, + "outcome_type": spec.outcome_type, + "result": safe_result, + } + ), + effective_at=event.occurred_at, + ) + self.db.add(outcome) + else: + self._validate_outcome(outcome, event=event, decision=decision, spec=spec) + + training_eligible = spec.training_eligible and self._is_server_verified_decision( + decision + ) + feedback = self._feedback( + event, + decision, + spec, + safe_result, + training_eligible=training_eligible, + ) + if feedback is not None and training_eligible: + decision.training_eligible = True + return WorkflowLearningRecords(outcome=outcome, feedback=feedback) + + def _feedback( + self, + event: BusinessEvent, + decision: AIDecision, + spec: WorkflowLearningSpec, + safe_result: dict[str, Any], + *, + training_eligible: bool, + ) -> AIDecisionFeedback | None: + if spec.feedback_type is None: + return None + feedback_key = f"workflow-feedback:{decision.id}:{event.id}" + feedback = self.db.scalar( + select(AIDecisionFeedback).where( + AIDecisionFeedback.tenant_id == event.tenant_id, + AIDecisionFeedback.idempotency_key == feedback_key, + ) + ) + if feedback is not None: + if ( + feedback.decision_id != decision.id + or feedback.feedback_type != spec.feedback_type + or feedback.verification_status != spec.verification_status + or bool(feedback.training_eligible) != training_eligible + ): + raise RuntimeError("AI 工作流反馈幂等状态与可信事件不一致。") + return feedback + + feedback = AIDecisionFeedback( + id=self._stable_id("workflow-feedback", event.tenant_id, decision.id, event.id), + tenant_id=event.tenant_id, + decision_id=decision.id, + expense_claim_id=decision.expense_claim_id, + correlation_id=event.correlation_id, + feedback_type=spec.feedback_type, + action_type=spec.outcome_type[:30], + actor_id=str(event.actor_id or "system").strip() or "system", + actor_type=str(event.actor_type or "system").strip() or "system", + evidence_source="server_business_event", + verification_status=spec.verification_status, + training_eligible=training_eligible, + final_value_json={ + "source_event_id": event.id, + "outcome_type": spec.outcome_type, + "evidence_fingerprint": safe_result["evidence_fingerprint"], + }, + changed_fields_json=[], + idempotency_key=feedback_key, + content_fingerprint=self._fingerprint( + { + "decision_id": decision.id, + "business_event_id": event.id, + "feedback_type": spec.feedback_type, + "outcome_type": spec.outcome_type, + } + ), + created_at=event.occurred_at, + ) + self.db.add(feedback) + return feedback + + @staticmethod + def _event_spec(event: BusinessEvent) -> WorkflowLearningSpec: + event_type = str(event.event_type or "").strip() + payload = event.payload_json if isinstance(event.payload_json, dict) else {} + if event_type in {"application_returned", "claim_returned"}: + return WorkflowLearningSpec( + "workflow_returned", + "rejected", + "human_verified", + True, + ) + if event_type in { + "application_approved", + "claim_approved", + "approval_stage_completed", + "approval_overridden", + }: + overridden = event_type == "approval_overridden" or ( + payload.get("ai_decision_overridden") is True + ) + return WorkflowLearningSpec( + "approval_overridden" if overridden else "approval_passed", + "rejected" if overridden else "accepted", + "human_verified", + True, + ) + if event_type == "payment_completed": + return WorkflowLearningSpec("payment_completed") + if event_type in AUDIT_EVENT_TYPES: + audit_decision = str( + payload.get("audit_decision") or payload.get("decision") or "" + ).strip().lower() + if audit_decision in { + "ready", + "pass", + "cleared", + "confirm", + "resolve", + "resolved", + }: + return WorkflowLearningSpec( + "audit_cleared" if audit_decision != "confirm" else "audit_confirmed", + "accepted", + "server_verified", + True, + ) + if audit_decision in {"needs_fix", "blocked", "flagged"}: + return WorkflowLearningSpec( + "audit_flagged", + "rejected", + "server_verified", + True, + ) + if audit_decision == "false_positive": + return WorkflowLearningSpec( + "audit_false_positive", + "rejected", + "human_verified", + True, + ) + return WorkflowLearningSpec("audit_conclusion_recorded") + raise ValueError("该业务事件不属于可学习的费用工作流结果。") + + @classmethod + def _safe_result( + cls, + event: BusinessEvent, + spec: WorkflowLearningSpec, + ) -> dict[str, Any]: + payload = event.payload_json if isinstance(event.payload_json, dict) else {} + audit_decision = str( + payload.get("audit_decision") or payload.get("decision") or "" + ).strip().lower() + if audit_decision not in { + "ready", + "pass", + "cleared", + "confirm", + "resolve", + "resolved", + "needs_fix", + "blocked", + "flagged", + "false_positive", + }: + audit_decision = "" + evidence = { + "source_event_id": event.id, + "source_event_type": event.event_type, + "source_event_version": int(event.event_version or 1), + "outcome_type": spec.outcome_type, + "next_status": cls._safe_state(payload.get("next_status")), + "next_approval_stage": cls._safe_state(payload.get("next_approval_stage")), + "audit_decision": audit_decision, + "ai_decision_overridden": payload.get("ai_decision_overridden") is True, + } + return { + **evidence, + "evidence_source": "server_business_event", + "evidence_fingerprint": cls._fingerprint(evidence), + } + + @staticmethod + def _safe_state(value: object) -> str: + return str(value or "").strip()[:50] + + @staticmethod + def _is_server_verified_decision(decision: AIDecision) -> bool: + evidence = decision.evidence_json if isinstance(decision.evidence_json, dict) else {} + return str(evidence.get("trust_level") or "").strip() in { + "server_snapshot_verified", + "server_rule_verified", + "server_model_verified", + } + + @staticmethod + def _validate_outcome( + outcome: WorkflowOutcome, + *, + event: BusinessEvent, + decision: AIDecision, + spec: WorkflowLearningSpec, + ) -> None: + if ( + outcome.decision_id != decision.id + or outcome.business_event_id != event.id + or outcome.expense_case_id != event.expense_case_id + or outcome.outcome_type != spec.outcome_type + or outcome.outcome_status != "verified" + ): + raise RuntimeError("AI 工作流结果幂等状态与可信事件不一致。") + + @staticmethod + def _stable_id(kind: str, *parts: object) -> str: + value = ":".join([kind, *(str(part) for part in parts)]) + return str(uuid.uuid5(uuid.NAMESPACE_URL, value)) + + @staticmethod + def _fingerprint(payload: dict[str, Any]) -> str: + serialized = json.dumps( + payload, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + default=str, + ) + return f"sha256:{hashlib.sha256(serialized.encode('utf-8')).hexdigest()}" diff --git a/server/src/app/services/finance_dashboard.py b/server/src/app/services/finance_dashboard.py index 230bf79..7fcaf21 100644 --- a/server/src/app/services/finance_dashboard.py +++ b/server/src/app/services/finance_dashboard.py @@ -15,6 +15,8 @@ from app.schemas.finance_dashboard import FinanceDashboardRead from app.services.budget_support import BudgetSupportMixin from app.services.demo_company_simulation_filters import is_finance_reimbursement_claim from app.services.expense_claim_constants import EXPENSE_TYPE_LABELS +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.finance_dashboard_budget import FinanceDashboardBudgetMixin from app.services.finance_dashboard_constants import ( CHART_COLORS, EMPTY_DONUT, @@ -26,14 +28,24 @@ from app.services.finance_dashboard_constants import ( STAGE_LABELS, SUCCESS_STATUSES, ) +from app.services.finance_dashboard_scope import ( + finance_dashboard_includes_legacy_budget, + normalize_finance_dashboard_tenant_id, +) class FinanceDashboardMetricMixin: def _fetch_claims(self) -> list[ExpenseClaim]: - stmt = select(ExpenseClaim).order_by(ExpenseClaim.created_at.asc()) + stmt = ( + select(ExpenseClaim) + .where(ExpenseClaimTenantScopeMixin.build_claim_tenant_condition(self.tenant_id)) + .order_by(ExpenseClaim.created_at.asc()) + ) return list(self.db.scalars(stmt).all()) def _fetch_budget_allocations(self, fiscal_year: int) -> list[BudgetAllocation]: + if not self.include_legacy_budget: + return [] stmt = ( select(BudgetAllocation) .where(BudgetAllocation.fiscal_year == fiscal_year) @@ -122,7 +134,9 @@ class FinanceDashboardMetricMixin: return datetime(1970, 1, 1, tzinfo=UTC), self._day_after(today) if key == "本季度": quarter_month = ((today.month - 1) // 3) * 3 + 1 - return self._day_start(today.replace(month=quarter_month, day=1)), self._day_after(today) + return self._day_start(today.replace(month=quarter_month, day=1)), self._day_after( + today + ) if key == "本年": return self._day_start(today.replace(month=1, day=1)), self._day_after(today) if key == "本月": @@ -171,9 +185,7 @@ class FinanceDashboardMetricMixin: ) budget_summary = self._budget_summary(datetime.now(UTC).year) avg_amount = ( - total_amount / Decimal(str(len(spend_claims))) - if spend_claims - else Decimal("0.00") + total_amount / Decimal(str(len(spend_claims))) if spend_claims else Decimal("0.00") ) return { @@ -233,7 +245,10 @@ class FinanceDashboardMetricMixin: category = self._expense_type_label(claim.expense_type) claim_count[bucket] += 1 claim_amount[bucket] += amount - category_amounts.setdefault(category, [Decimal("0.00") for _ in labels])[bucket] += amount + category_amounts.setdefault( + category, + [Decimal("0.00") for _ in labels], + )[bucket] += amount category_totals[category] += amount if self._status(claim) in SUCCESS_STATUSES: success_count[bucket] += 1 @@ -252,7 +267,11 @@ class FinanceDashboardMetricMixin: "total": self._decimal_number(category_totals[name]), } for index, name in enumerate( - sorted(category_amounts, key=lambda item: category_totals[item], reverse=True)[:6] + sorted( + category_amounts, + key=lambda item: category_totals[item], + reverse=True, + )[:6] ) ], "successCount": success_count, @@ -393,7 +412,7 @@ class FinanceDashboardMetricMixin: ] -class FinanceDashboardBudgetAndLabelMixin: +class FinanceDashboardLabelMixin: def _top_claims(self, claims: list[ExpenseClaim]) -> list[dict[str, Any]]: spend_claims = [ claim for claim in claims if self._status(claim) not in EXCLUDED_SPEND_STATUSES @@ -475,172 +494,6 @@ class FinanceDashboardBudgetAndLabelMixin: priority = {"danger": 0, "warning": 1, "success": 2} return sorted(rows, key=lambda item: priority.get(str(item.get("tone")), 3))[:6] - def _budget_summary(self, fiscal_year: int) -> dict[str, Any]: - allocations = self._fetch_budget_allocations(fiscal_year) - total = Decimal("0.00") - used = Decimal("0.00") - available = Decimal("0.00") - - for allocation in allocations: - balance = self.get_balance(allocation) - total += balance.total_amount - used += balance.reserved_amount + balance.consumed_amount - available += balance.available_amount - - ratio = Decimal("0.00") - if total > Decimal("0.00"): - ratio = (used / total) * Decimal("100") - - return { - "ratio": self._decimal_number(ratio), - "total": self._currency(total), - "used": self._currency(used), - "left": self._currency(available), - } - - def _budget_metrics(self, fiscal_year: int) -> list[dict[str, Any]]: - allocations = self._fetch_budget_allocations(fiscal_year) - total = Decimal("0.00") - consumed = Decimal("0.00") - reserved = Decimal("0.00") - available = Decimal("0.00") - over_count = 0 - warning_count = 0 - - for allocation in allocations: - balance = self.get_balance(allocation) - total += balance.total_amount - consumed += balance.consumed_amount - reserved += balance.reserved_amount - available += balance.available_amount - if balance.available_amount < Decimal("0.00"): - over_count += 1 - continue - if balance.usage_rate >= Decimal(str(allocation.warning_threshold or 80)): - warning_count += 1 - - used = consumed + reserved - usage_rate = Decimal("0.00") - if total > Decimal("0.00"): - usage_rate = (used / total) * Decimal("100") - - return [ - self._budget_metric( - label="预算池数量", - value=f"{len(allocations)} 个", - detail="年度有效预算池", - tone="neutral", - icon="mdi mdi-database-outline", - ), - self._budget_metric( - label="总预算", - value=self._currency(total), - detail="原始预算 + 调整", - tone="neutral", - icon="mdi mdi-cash-register", - ), - self._budget_metric( - label="已用预算", - value=self._currency(used), - detail=f"使用率 {self._decimal_number(usage_rate):.1f}%", - tone="warning" if usage_rate >= Decimal("80") else "success", - icon="mdi mdi-chart-arc", - ), - self._budget_metric( - label="预占预算", - value=self._currency(reserved), - detail="待流转单据占用", - tone="warning" if reserved > Decimal("0.00") else "success", - icon="mdi mdi-lock-outline", - ), - self._budget_metric( - label="可用预算", - value=self._currency(available), - detail="可继续使用额度", - tone="danger" if available < Decimal("0.00") else "success", - icon="mdi mdi-wallet-outline", - ), - self._budget_metric( - label="预警预算池", - value=f"{warning_count} 个", - detail=f"超支 {over_count} 个", - tone="danger" if over_count else "warning" if warning_count else "success", - icon="mdi mdi-alert-outline", - ), - ] - - def _budget_metric( - self, - *, - label: str, - value: str, - detail: str, - tone: str, - icon: str, - ) -> dict[str, Any]: - return { - "label": label, - "value": value, - "detail": detail, - "tone": tone, - "icon": icon, - } - - def _budget_focus_rows(self) -> list[dict[str, Any]]: - allocations = self._fetch_budget_allocations(datetime.now(UTC).year) - over_count = 0 - warning_count = 0 - over_amount = Decimal("0.00") - warning_used = Decimal("0.00") - - for allocation in allocations: - balance = self.get_balance(allocation) - if balance.available_amount < Decimal("0.00"): - over_count += 1 - over_amount += abs(balance.available_amount) - continue - if balance.usage_rate >= Decimal(str(allocation.warning_threshold or 80)): - warning_count += 1 - warning_used += balance.reserved_amount + balance.consumed_amount - - return [ - self._focus_item( - name="预算超支", - role="预算控制", - duration=f"{over_count} 个池", - status=self._currency(over_amount), - tone="danger" if over_count else "success", - avatar="超", - ), - self._focus_item( - name="预算预警", - role="预算控制", - duration=f"{warning_count} 个池", - status=self._currency(warning_used), - tone="warning" if warning_count else "success", - avatar="预", - ), - ] - - def _focus_item( - self, - *, - name: str, - role: str, - duration: str, - status: str, - tone: str, - avatar: str, - ) -> dict[str, Any]: - return { - "name": name, - "role": role, - "duration": duration, - "status": status, - "tone": tone, - "avatar": avatar, - } - def _claim_time(self, claim: ExpenseClaim) -> datetime: return self._as_utc(claim.submitted_at or claim.occurred_at or claim.created_at) @@ -822,9 +675,16 @@ class FinanceDashboardBudgetAndLabelMixin: return f"{prefix}{amount:,.0f}" -class FinanceDashboardService(FinanceDashboardMetricMixin, FinanceDashboardBudgetAndLabelMixin, BudgetSupportMixin): - def __init__(self, db: Session) -> None: +class FinanceDashboardService( + FinanceDashboardMetricMixin, + FinanceDashboardLabelMixin, + FinanceDashboardBudgetMixin, + BudgetSupportMixin, +): + def __init__(self, db: Session, *, tenant_id: str = "default") -> None: self.db = db + self.tenant_id = normalize_finance_dashboard_tenant_id(tenant_id) + self.include_legacy_budget = finance_dashboard_includes_legacy_budget(self.tenant_id) def build_dashboard( self, @@ -856,9 +716,7 @@ class FinanceDashboardService(FinanceDashboardMetricMixin, FinanceDashboardBudge fallback_end=end, ) - claims = [ - claim for claim in self._fetch_claims() if is_finance_reimbursement_claim(claim) - ] + claims = [claim for claim in self._fetch_claims() if is_finance_reimbursement_claim(claim)] scope_claims = self._claims_between(claims, start, end) previous_claims = self._claims_between(claims, previous_start, start) trend_claims = self._claims_between(claims, trend_start, trend_end) @@ -886,4 +744,3 @@ class FinanceDashboardService(FinanceDashboardMetricMixin, FinanceDashboardBudge budget_summary=self._budget_summary(now.year), budget_metrics=self._budget_metrics(now.year), ) - diff --git a/server/src/app/services/finance_dashboard_access_policy.py b/server/src/app/services/finance_dashboard_access_policy.py new file mode 100644 index 0000000..80b4acc --- /dev/null +++ b/server/src/app/services/finance_dashboard_access_policy.py @@ -0,0 +1,31 @@ +from __future__ import annotations + +from fastapi import HTTPException, status + +from app.api.deps import CurrentUserContext +from app.services.expense_claim_access_policy import ExpenseClaimAccessPolicy + +FINANCE_DASHBOARD_READ_ROLE_CODES = frozenset({"finance", "executive"}) + + +class FinanceDashboardAccessPolicy: + """财务看板只提供租户内的财务只读视图。""" + + @staticmethod + def can_read(current_user: CurrentUserContext) -> bool: + # 租户上下文异常时必须拒绝，不能把空值隐式解释成 default 租户。 + if not str(current_user.tenant_id or "").strip(): + return False + if current_user.is_admin: + return True + role_codes = ExpenseClaimAccessPolicy.normalize_role_codes(current_user) + return bool(role_codes & FINANCE_DASHBOARD_READ_ROLE_CODES) + + @classmethod + def require_read(cls, current_user: CurrentUserContext) -> None: + if cls.can_read(current_user): + return + raise HTTPException( + status_code=status.HTTP_403_FORBIDDEN, + detail="只有财务人员、高级财务人员或 admin 管理员可以查看财务看板。", + ) diff --git a/server/src/app/services/finance_dashboard_budget.py b/server/src/app/services/finance_dashboard_budget.py new file mode 100644 index 0000000..813f1a7 --- /dev/null +++ b/server/src/app/services/finance_dashboard_budget.py @@ -0,0 +1,185 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal +from typing import Any + + +class FinanceDashboardBudgetMixin: + """构建预算摘要、预算卡片和预算瓶颈展示数据。""" + + def _budget_summary(self, fiscal_year: int) -> dict[str, Any]: + allocations = self._fetch_budget_allocations(fiscal_year) + total = Decimal("0.00") + used = Decimal("0.00") + available = Decimal("0.00") + + for allocation in allocations: + balance = self.get_balance(allocation) + total += balance.total_amount + used += balance.reserved_amount + balance.consumed_amount + available += balance.available_amount + + ratio = Decimal("0.00") + if total > Decimal("0.00"): + ratio = (used / total) * Decimal("100") + + payload = { + "ratio": self._decimal_number(ratio), + "total": self._currency(total), + "used": self._currency(used), + "left": self._currency(available), + "included": self.include_legacy_budget, + "scope": "legacy_default_budget" if self.include_legacy_budget else "unavailable", + } + if not self.include_legacy_budget: + payload["reason"] = "当前租户尚未接入独立预算池，预算指标未纳入统计。" + return payload + + def _budget_metrics(self, fiscal_year: int) -> list[dict[str, Any]]: + allocations = self._fetch_budget_allocations(fiscal_year) + total = Decimal("0.00") + consumed = Decimal("0.00") + reserved = Decimal("0.00") + available = Decimal("0.00") + over_count = 0 + warning_count = 0 + + for allocation in allocations: + balance = self.get_balance(allocation) + total += balance.total_amount + consumed += balance.consumed_amount + reserved += balance.reserved_amount + available += balance.available_amount + if balance.available_amount < Decimal("0.00"): + over_count += 1 + continue + if balance.usage_rate >= Decimal(str(allocation.warning_threshold or 80)): + warning_count += 1 + + used = consumed + reserved + usage_rate = Decimal("0.00") + if total > Decimal("0.00"): + usage_rate = (used / total) * Decimal("100") + + return [ + self._budget_metric( + label="预算池数量", + value=f"{len(allocations)} 个", + detail="年度有效预算池", + tone="neutral", + icon="mdi mdi-database-outline", + ), + self._budget_metric( + label="总预算", + value=self._currency(total), + detail="原始预算 + 调整", + tone="neutral", + icon="mdi mdi-cash-register", + ), + self._budget_metric( + label="已用预算", + value=self._currency(used), + detail=f"使用率 {self._decimal_number(usage_rate):.1f}%", + tone="warning" if usage_rate >= Decimal("80") else "success", + icon="mdi mdi-chart-arc", + ), + self._budget_metric( + label="预占预算", + value=self._currency(reserved), + detail="待流转单据占用", + tone="warning" if reserved > Decimal("0.00") else "success", + icon="mdi mdi-lock-outline", + ), + self._budget_metric( + label="可用预算", + value=self._currency(available), + detail="可继续使用额度", + tone="danger" if available < Decimal("0.00") else "success", + icon="mdi mdi-wallet-outline", + ), + self._budget_metric( + label="预警预算池", + value=f"{warning_count} 个", + detail=f"超支 {over_count} 个", + tone="danger" if over_count else "warning" if warning_count else "success", + icon="mdi mdi-alert-outline", + ), + ] + + def _budget_metric( + self, + *, + label: str, + value: str, + detail: str, + tone: str, + icon: str, + ) -> dict[str, Any]: + if not self.include_legacy_budget: + detail = "当前租户未接入独立预算池" + tone = "neutral" + return { + "label": label, + "value": value, + "detail": detail, + "tone": tone, + "icon": icon, + } + + def _budget_focus_rows(self) -> list[dict[str, Any]]: + if not self.include_legacy_budget: + return [] + allocations = self._fetch_budget_allocations(datetime.now(UTC).year) + over_count = 0 + warning_count = 0 + over_amount = Decimal("0.00") + warning_used = Decimal("0.00") + + for allocation in allocations: + balance = self.get_balance(allocation) + if balance.available_amount < Decimal("0.00"): + over_count += 1 + over_amount += abs(balance.available_amount) + continue + if balance.usage_rate >= Decimal(str(allocation.warning_threshold or 80)): + warning_count += 1 + warning_used += balance.reserved_amount + balance.consumed_amount + + return [ + self._focus_item( + name="预算超支", + role="预算控制", + duration=f"{over_count} 个池", + status=self._currency(over_amount), + tone="danger" if over_count else "success", + avatar="超", + ), + self._focus_item( + name="预算预警", + role="预算控制", + duration=f"{warning_count} 个池", + status=self._currency(warning_used), + tone="warning" if warning_count else "success", + avatar="预", + ), + ] + + @staticmethod + def _focus_item( + *, + name: str, + role: str, + duration: str, + status: str, + tone: str, + avatar: str, + ) -> dict[str, Any]: + return { + "name": name, + "role": role, + "duration": duration, + "status": status, + "tone": tone, + "avatar": avatar, + } diff --git a/server/src/app/services/finance_dashboard_scheduler.py b/server/src/app/services/finance_dashboard_scheduler.py index 1571029..4b4f595 100644 --- a/server/src/app/services/finance_dashboard_scheduler.py +++ b/server/src/app/services/finance_dashboard_scheduler.py @@ -8,6 +8,7 @@ from zoneinfo import ZoneInfo from app.core.logging import get_logger from app.db.session import get_session_factory from app.services.finance_dashboard_snapshot import FinanceDashboardSnapshotService +from app.services.tenant_registry import DEFAULT_TENANT_ID logger = get_logger("app.services.finance_dashboard_scheduler") @@ -66,7 +67,10 @@ class FinanceDashboardScheduler: def _refresh_snapshot(self) -> None: db = get_session_factory()() try: - dashboard = FinanceDashboardSnapshotService(db).refresh_default_snapshot() + dashboard = FinanceDashboardSnapshotService( + db, + tenant_id=DEFAULT_TENANT_ID, + ).refresh_default_snapshot() db.commit() totals = dashboard.totals or {} logger.info( diff --git a/server/src/app/services/finance_dashboard_scope.py b/server/src/app/services/finance_dashboard_scope.py new file mode 100644 index 0000000..0d797c9 --- /dev/null +++ b/server/src/app/services/finance_dashboard_scope.py @@ -0,0 +1,28 @@ +from __future__ import annotations + +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.tenant_registry import DEFAULT_TENANT_ID + +FINANCE_DASHBOARD_CLAIM_SCOPE = "expense_case_tenant" +FINANCE_DASHBOARD_DEFAULT_BUDGET_SCOPE = "legacy_default_budget" +FINANCE_DASHBOARD_NO_BUDGET_SCOPE = "budget_unavailable" +FINANCE_DASHBOARD_TASK_TYPE = "finance_dashboard_snapshot" + + +def normalize_finance_dashboard_tenant_id(value: str | None) -> str: + return ExpenseClaimTenantScopeMixin.normalize_tenant_id(value) + + +def finance_dashboard_includes_legacy_budget(tenant_id: str | None) -> bool: + """旧预算表没有 tenant_id，只允许 default 租户读取。""" + + return normalize_finance_dashboard_tenant_id(tenant_id) == DEFAULT_TENANT_ID + + +def resolve_finance_dashboard_data_scope(tenant_id: str | None) -> str: + budget_scope = ( + FINANCE_DASHBOARD_DEFAULT_BUDGET_SCOPE + if finance_dashboard_includes_legacy_budget(tenant_id) + else FINANCE_DASHBOARD_NO_BUDGET_SCOPE + ) + return f"claims:{FINANCE_DASHBOARD_CLAIM_SCOPE};budget:{budget_scope}" diff --git a/server/src/app/services/finance_dashboard_snapshot.py b/server/src/app/services/finance_dashboard_snapshot.py index 0dc8d30..4b56695 100644 --- a/server/src/app/services/finance_dashboard_snapshot.py +++ b/server/src/app/services/finance_dashboard_snapshot.py @@ -18,16 +18,23 @@ from app.models.agent_run import AgentRun from app.schemas.finance_dashboard import FinanceDashboardRead from app.services.agent_runs import AgentRunService from app.services.finance_dashboard import FinanceDashboardService +from app.services.finance_dashboard_scope import ( + FINANCE_DASHBOARD_TASK_TYPE, + normalize_finance_dashboard_tenant_id, + resolve_finance_dashboard_data_scope, +) +from app.services.tenant_registry import DEFAULT_TENANT_ID -FINANCE_DASHBOARD_TASK_TYPE = "finance_dashboard_snapshot" FINANCE_DASHBOARD_TOOL_NAME = "digital_employee.finance_dashboard.snapshot" SNAPSHOT_TTL_SECONDS = 120 -SNAPSHOT_SCHEMA_VERSION = "finance-dashboard-ranking-v2" +SNAPSHOT_SCHEMA_VERSION = "finance-dashboard-tenant-v3" class FinanceDashboardSnapshotService: - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str = "default") -> None: self.db = db + self.tenant_id = normalize_finance_dashboard_tenant_id(tenant_id) + self.data_scope = resolve_finance_dashboard_data_scope(self.tenant_id) def build_dashboard( self, @@ -39,6 +46,8 @@ class FinanceDashboardSnapshotService: department_range: str = "本月", ) -> FinanceDashboardRead: key = self._cache_key( + tenant_id=self.tenant_id, + data_scope=self.data_scope, range_key=range_key, start_date=start_date, end_date=end_date, @@ -58,6 +67,8 @@ class FinanceDashboardSnapshotService: ) def refresh_default_snapshot(self) -> FinanceDashboardRead: + if self.tenant_id != DEFAULT_TENANT_ID: + raise ValueError("后台默认财务快照只能在 default 系统租户范围内生成。") return self.refresh_snapshot( range_key="近30日", trend_range="近12天", @@ -76,6 +87,8 @@ class FinanceDashboardSnapshotService: source: str, ) -> FinanceDashboardRead: key = self._cache_key( + tenant_id=self.tenant_id, + data_scope=self.data_scope, range_key=range_key, start_date=start_date, end_date=end_date, @@ -86,16 +99,21 @@ class FinanceDashboardSnapshotService: run = run_service.create_run( agent=AgentName.HERMES.value, source=source, + tenant_id=self.tenant_id, user_id="digital_employee", ontology_json={ "scenario": "finance_dashboard", "intent": "snapshot", + "tenant_id": self.tenant_id, + "data_scope": self.data_scope, }, route_json={ "task_type": FINANCE_DASHBOARD_TASK_TYPE, "job_type": FINANCE_DASHBOARD_TASK_TYPE, "selected_agent": AgentName.HERMES.value, "snapshot_key": key, + "tenant_id": self.tenant_id, + "data_scope": self.data_scope, "params": { "range_key": range_key, "start_date": self._date_text(start_date), @@ -111,7 +129,10 @@ class FinanceDashboardSnapshotService: ) timer = perf_counter() try: - dashboard = FinanceDashboardService(self.db).build_dashboard( + dashboard = FinanceDashboardService( + self.db, + tenant_id=self.tenant_id, + ).build_dashboard( range_key=range_key, start_date=start_date, end_date=end_date, @@ -128,6 +149,8 @@ class FinanceDashboardSnapshotService: request_json={ "task_type": FINANCE_DASHBOARD_TASK_TYPE, "snapshot_key": key, + "tenant_id": self.tenant_id, + "data_scope": self.data_scope, }, response_json={ "task_type": FINANCE_DASHBOARD_TASK_TYPE, @@ -142,6 +165,8 @@ class FinanceDashboardSnapshotService: { "phase": "succeeded", "snapshot_key": key, + "tenant_id": self.tenant_id, + "data_scope": self.data_scope, "snapshot_payload": payload, "summary": summary, "expires_at": ( @@ -163,6 +188,8 @@ class FinanceDashboardSnapshotService: { "phase": "failed", "snapshot_key": key, + "tenant_id": self.tenant_id, + "data_scope": self.data_scope, "heartbeat_at": datetime.now(UTC).isoformat(), }, status=AgentRunStatus.FAILED.value, @@ -173,10 +200,12 @@ class FinanceDashboardSnapshotService: def _latest_fresh_snapshot(self, key: str) -> FinanceDashboardRead | None: now = datetime.now(UTC) - for run in self._recent_snapshot_runs(): + for run in self._recent_snapshot_runs( + tenant_id=self.tenant_id, + data_scope=self.data_scope, + snapshot_key=key, + ): route_json = run.route_json or {} - if str(route_json.get("snapshot_key") or "") != key: - continue payload = route_json.get("snapshot_payload") if not isinstance(payload, dict): payload = self._payload_from_tool_call(run) @@ -188,23 +217,29 @@ class FinanceDashboardSnapshotService: return FinanceDashboardRead.model_validate(payload) return None - def _recent_snapshot_runs(self) -> list[AgentRun]: + def _recent_snapshot_runs( + self, + *, + tenant_id: str, + data_scope: str, + snapshot_key: str, + ) -> list[AgentRun]: stmt = ( select(AgentRun) .options(selectinload(AgentRun.tool_calls)) .where( AgentRun.agent == AgentName.HERMES.value, AgentRun.status == AgentRunStatus.SUCCEEDED.value, + AgentRun.route_json["task_type"].as_string() == FINANCE_DASHBOARD_TASK_TYPE, + AgentRun.route_json["tenant_id"].as_string() == tenant_id, + AgentRun.route_json["data_scope"].as_string() == data_scope, + AgentRun.route_json["snapshot_key"].as_string() == snapshot_key, ) .order_by(AgentRun.started_at.desc()) .limit(80) ) runs = list(self.db.scalars(stmt).all()) - return [ - run - for run in runs - if str((run.route_json or {}).get("task_type") or "") == FINANCE_DASHBOARD_TASK_TYPE - ] + return runs @staticmethod def _payload_from_tool_call(run: AgentRun) -> dict[str, Any] | None: @@ -231,6 +266,8 @@ class FinanceDashboardSnapshotService: def _cache_key( cls, *, + tenant_id: str, + data_scope: str, range_key: str, start_date: Any, end_date: Any, @@ -239,15 +276,22 @@ class FinanceDashboardSnapshotService: ) -> str: return "|".join( [ - SNAPSHOT_SCHEMA_VERSION, - str(range_key or ""), - cls._date_text(start_date), - cls._date_text(end_date), - str(trend_range or ""), - str(department_range or ""), + cls._key_part(SNAPSHOT_SCHEMA_VERSION), + cls._key_part(normalize_finance_dashboard_tenant_id(tenant_id)), + cls._key_part(data_scope), + cls._key_part(range_key), + cls._key_part(cls._date_text(start_date)), + cls._key_part(cls._date_text(end_date)), + cls._key_part(trend_range), + cls._key_part(department_range), ] ) + @staticmethod + def _key_part(value: Any) -> str: + text = str(value or "") + return f"{len(text)}:{text}" + @staticmethod def _date_text(value: Any) -> str: if value is None: diff --git a/server/src/app/services/finance_report_context.py b/server/src/app/services/finance_report_context.py index 35930f2..90a61cb 100644 --- a/server/src/app/services/finance_report_context.py +++ b/server/src/app/services/finance_report_context.py @@ -13,6 +13,7 @@ from app.models.agent_run import AgentRun from app.models.employee_behavior_profile import EmployeeBehaviorProfileSnapshot from app.models.risk_observation import RiskObservation from app.services.finance_dashboard import FinanceDashboardService +from app.services.finance_report_tenant import require_report_tenant_id FinanceReportType = Literal["weekly", "quarterly", "annual"] @@ -33,8 +34,9 @@ class FinanceReportPeriod: class FinanceReportContextService: - def __init__(self, db: Session) -> None: + def __init__(self, db: Session, *, tenant_id: str) -> None: self.db = db + self.tenant_id = require_report_tenant_id(tenant_id) def build_context( self, @@ -51,7 +53,10 @@ class FinanceReportContextService: end_date=end_date, now=generated_at, ) - dashboard = FinanceDashboardService(self.db).build_dashboard( + dashboard = FinanceDashboardService( + self.db, + tenant_id=self.tenant_id, + ).build_dashboard( range_key="自定义", start_date=period.start_date, end_date=period.end_date, @@ -66,6 +71,7 @@ class FinanceReportContextService: insights = self._insights(dashboard_payload, risk_summary, profile_summary, actions) return { + "tenant_id": self.tenant_id, "report_type": report_type, "period": period.to_dict(), "generated_at": generated_at.isoformat(), @@ -135,6 +141,7 @@ class FinanceReportContextService: rows = list( self.db.scalars( select(RiskObservation).where( + RiskObservation.tenant_id == self.tenant_id, RiskObservation.created_at >= start_dt, RiskObservation.created_at < end_dt, ) @@ -168,6 +175,7 @@ class FinanceReportContextService: rows = list( self.db.scalars( select(EmployeeBehaviorProfileSnapshot).where( + EmployeeBehaviorProfileSnapshot.tenant_id == self.tenant_id, EmployeeBehaviorProfileSnapshot.calculated_at >= start_dt, EmployeeBehaviorProfileSnapshot.calculated_at < end_dt, ) @@ -203,6 +211,8 @@ class FinanceReportContextService: rows = list( self.db.scalars( select(AgentRun).where( + AgentRun.route_json["tenant_id"].as_string() == self.tenant_id, + AgentRun.ontology_json["tenant_id"].as_string() == self.tenant_id, AgentRun.agent == "hermes", AgentRun.started_at >= start_dt, AgentRun.started_at < end_dt, @@ -237,8 +247,7 @@ class FinanceReportContextService: "owner": str(item.get("role") or "财务运营组"), "priority": "high" if tone == "danger" else "medium", "suggestion": ( - f"请跟进{name}：" - f"{item.get('duration') or ''} {item.get('status') or ''}" + f"请跟进{name}：{item.get('duration') or ''} {item.get('status') or ''}" ).strip(), } ) diff --git a/server/src/app/services/finance_report_mailer.py b/server/src/app/services/finance_report_mailer.py index 7fc64e8..8456b1d 100644 --- a/server/src/app/services/finance_report_mailer.py +++ b/server/src/app/services/finance_report_mailer.py @@ -11,6 +11,10 @@ from sqlalchemy.orm import Session from app.core.secret_box import decrypt_secret from app.models.system_setting import SystemSetting from app.models.system_setting_secret import SystemSettingSecret +from app.services.finance_report_tenant import ( + TenantFinanceReportConfigService, + require_report_tenant_id, +) from app.services.settings import SettingsService @@ -36,11 +40,19 @@ class FinanceReportMailer: *, context: dict[str, Any], pdf_path: Path, + tenant_id: str, recipients: list[str] | None = None, dry_run: bool = False, ) -> FinanceReportDeliveryResult: settings_row, secrets_row = SettingsService(self.db).ensure_settings_ready() - resolved_recipients = self._resolve_recipients(settings_row, recipients) + tenant = require_report_tenant_id(tenant_id) + context_tenant = require_report_tenant_id(context.get("tenant_id")) + if tenant != context_tenant: + raise ValueError("报告上下文与投递租户不一致。") + resolved_recipients = TenantFinanceReportConfigService(self.db).configured_recipients( + tenant_id=tenant, + requested=recipients, + ) subject = self._subject(context) missing = self._missing_config(settings_row, secrets_row, resolved_recipients) @@ -136,24 +148,6 @@ class FinanceReportMailer: ) return message - @staticmethod - def _resolve_recipients( - settings_row: SystemSetting, - override_recipients: list[str] | None, - ) -> list[str]: - raw_values = override_recipients or [ - str(settings_row.default_receiver or ""), - str(settings_row.notice_email or ""), - str(settings_row.admin_email or ""), - ] - recipients: list[str] = [] - for raw in raw_values: - for item in str(raw or "").replace(";", ",").split(","): - email = item.strip() - if email and "@" in email and email not in recipients: - recipients.append(email) - return recipients - @staticmethod def _missing_config( settings_row: SystemSetting, @@ -172,7 +166,7 @@ class FinanceReportMailer: if not str(secrets_row.smtp_password_encrypted or "").strip(): missing.append("smtp_password") if not recipients: - missing.append("recipients") + missing.append("tenant_report_recipients") return missing @staticmethod diff --git a/server/src/app/services/finance_report_renderer.py b/server/src/app/services/finance_report_renderer.py index 61c4c74..b769aa7 100644 --- a/server/src/app/services/finance_report_renderer.py +++ b/server/src/app/services/finance_report_renderer.py @@ -1,5 +1,6 @@ from __future__ import annotations +import hashlib import html import re from dataclasses import dataclass @@ -7,6 +8,7 @@ from pathlib import Path from typing import Any from app.core.config import get_settings +from app.services.finance_report_tenant import require_report_tenant_id @dataclass(frozen=True, slots=True) @@ -45,6 +47,8 @@ class FinanceReportRenderer: top_claims = list(dashboard.get("top_claims") or []) actions = list(context.get("action_items") or []) insights = list(context.get("insights") or []) + reimbursement_count = f"{int(totals.get('reimbursementCount') or 0)} 单" + budget_usage_rate = f"{float(totals.get('budgetUsageRate') or 0):.1f}%" return f""" @@ -90,13 +94,13 @@ class FinanceReportRenderer:

管理摘要

- {''.join(f'
{_e(item)}
' for item in insights)} + {"".join(f'
{_e(item)}
' for item in insights)}

关键指标

{_metric_html("报销金额", _money(totals.get("reimbursementAmount")))} - {_metric_html("报销单数", f'{int(totals.get("reimbursementCount") or 0)} 单')} + {_metric_html("报销单数", reimbursement_count)} {_metric_html("待付款", _money(totals.get("pendingPaymentAmount")))} - {_metric_html("预算使用率", f'{float(totals.get("budgetUsageRate") or 0):.1f}%')} + {_metric_html("预算使用率", budget_usage_rate)}

每日报销趋势

{_trend_html(trend)} @@ -113,9 +117,18 @@ class FinanceReportRenderer: def _report_dir(self, context: dict[str, Any]) -> Path: settings = get_settings() period = context.get("period") or {} + tenant_id = require_report_tenant_id(context.get("tenant_id")) + tenant_scope = hashlib.sha256(tenant_id.encode()).hexdigest()[:24] report_type = str(context.get("report_type") or "weekly") label = re.sub(r"[^0-9A-Za-z\u4e00-\u9fff_-]+", "_", str(period.get("label") or "latest")) - return settings.resolved_storage_root_dir / "finance_reports" / report_type / label + return ( + settings.resolved_storage_root_dir + / "finance_reports" + / "tenants" + / tenant_scope + / report_type + / label + ) @staticmethod def _storage_key(pdf_path: Path) -> str: @@ -214,8 +227,7 @@ class SimpleFinancePdfWriter: { "type": "bullet", "text": ( - f"{item.get('title')} / {item.get('owner')}：" - f"{item.get('suggestion')}" + f"{item.get('title')} / {item.get('owner')}：{item.get('suggestion')}" ), } for item in actions @@ -263,8 +275,7 @@ class SimpleFinancePdfWriter: def _bars(self, commands: list[str], labels: list[Any], values: list[Any], y: int) -> int: pairs = [ - (str(label), float(value or 0)) - for label, value in zip(labels, values, strict=False) + (str(label), float(value or 0)) for label, value in zip(labels, values, strict=False) ] max_value = max([value for _label, value in pairs] or [1]) for label, value in pairs[:10]: @@ -309,8 +320,7 @@ class SimpleFinancePdfWriter: payload.extend(f"{offset:010d} 00000 n \n".encode("latin-1")) payload.extend( ( - f"trailer\n<< /Size {len(objects) + 1} /Root 1 0 R >>\n" - f"startxref\n{xref_at}\n%%EOF" + f"trailer\n<< /Size {len(objects) + 1} /Root 1 0 R >>\nstartxref\n{xref_at}\n%%EOF" ).encode("latin-1") ) path.write_bytes(bytes(payload)) @@ -344,7 +354,7 @@ def _bar_html(labels: list[Any], values: list[float]) -> str: '
' f'
{_e(label)}
' f'
' - f'
{_e(_money(value))}
' + f"
{_e(_money(value))}
" "
" ) return "".join(rows) or '
暂无数据
' @@ -372,7 +382,7 @@ def _actions_html(rows: list[dict[str, Any]]) -> str: return "".join( ( f'
{_e(item.get("title"))}' - f'｜{_e(item.get("owner"))}
{_e(item.get("suggestion"))}
' + f"｜{_e(item.get('owner'))}
{_e(item.get('suggestion'))}" ) for item in rows ) diff --git a/server/src/app/services/finance_report_scheduler.py b/server/src/app/services/finance_report_scheduler.py index 30c5e5a..88f2e9f 100644 --- a/server/src/app/services/finance_report_scheduler.py +++ b/server/src/app/services/finance_report_scheduler.py @@ -7,12 +7,11 @@ from zoneinfo import ZoneInfo from sqlalchemy import select -from app.core.agent_enums import AgentRunSource, AgentRunStatus +from app.core.agent_enums import AgentRunSource from app.core.logging import get_logger from app.db.session import get_session_factory -from app.models.agent_run import AgentRun +from app.models.tenant_finance_report import TenantFinanceReportConfig from app.services.digital_employee_finance_report_task import ( - FINANCE_REPORT_TASK_TYPE, DigitalEmployeeFinanceReportTaskService, ) @@ -75,52 +74,57 @@ class FinanceReportScheduler: due_types.append("quarterly") if now.day <= 7 and now.month == 1: due_types.append("annual") - for report_type in due_types: - self._run_report_once(report_type=report_type, now=now) - - def _run_report_once(self, *, report_type: str, now: datetime) -> None: db = get_session_factory()() try: - if self._already_generated(db, report_type=report_type, now=now): - return + tenant_ids = list( + db.scalars( + select(TenantFinanceReportConfig.tenant_id).where( + TenantFinanceReportConfig.status == "active", + TenantFinanceReportConfig.delivery_enabled.is_(True), + ) + ).all() + ) + finally: + db.close() + for tenant_id in tenant_ids: + for report_type in due_types: + self._run_report_once( + report_type=report_type, + now=now, + tenant_id=tenant_id, + ) + + def _run_report_once( + self, + *, + report_type: str, + now: datetime, + tenant_id: str, + ) -> None: + db = get_session_factory()() + try: result = DigitalEmployeeFinanceReportTaskService(db).generate_report( report_type=report_type, # type: ignore[arg-type] source=AgentRunSource.SCHEDULE.value, + tenant_id=tenant_id, ) db.commit() logger.info( - "Finance report generated type=%s status=%s", + "Finance report generated tenant=%s type=%s status=%s", + tenant_id, report_type, (result.get("delivery") or {}).get("status"), ) except Exception: db.rollback() - logger.exception("Scheduled finance report failed type=%s", report_type) + logger.exception( + "Scheduled finance report failed tenant=%s type=%s", + tenant_id, + report_type, + ) finally: db.close() - def _already_generated(self, db, *, report_type: str, now: datetime) -> bool: - day_start = datetime.combine( - now.date(), - time.min, - tzinfo=self._timezone, - ).astimezone(ZoneInfo("UTC")) - day_end = day_start + timedelta(days=1) - stmt = ( - select(AgentRun) - .where(AgentRun.started_at >= day_start) - .where(AgentRun.started_at < day_end) - .where(AgentRun.status == AgentRunStatus.SUCCEEDED.value) - ) - for run in db.scalars(stmt).all(): - route_json = run.route_json or {} - if ( - str(route_json.get("task_type") or "") == FINANCE_REPORT_TASK_TYPE - and str(route_json.get("report_type") or "") == report_type - ): - return True - return False - def _seconds_until_next_report_time(self) -> float: now = datetime.now(self._timezone) target = datetime.combine(now.date(), self._report_time, tzinfo=self._timezone) diff --git a/server/src/app/services/finance_report_tenant.py b/server/src/app/services/finance_report_tenant.py new file mode 100644 index 0000000..cb6ae07 --- /dev/null +++ b/server/src/app/services/finance_report_tenant.py @@ -0,0 +1,208 @@ +from __future__ import annotations + +import hashlib +import re +from datetime import UTC, date, datetime, timedelta +from typing import Any + +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.models.tenant import Tenant +from app.models.tenant_finance_report import ( + TenantFinanceReportConfig, + TenantFinanceReportRun, +) + +EMAIL_PATTERN = re.compile(r"^[^\s@]+@[^\s@]+\.[^\s@]+$") +RUN_STALE_AFTER = timedelta(hours=2) + + +def require_report_tenant_id(value: object) -> str: + tenant_id = str(value or "").strip() + if not tenant_id: + raise ValueError("tenant_id 不能为空。") + return tenant_id + + +def normalize_report_recipients(values: list[str] | None) -> list[str]: + recipients: list[str] = [] + for raw in list(values or []): + for item in str(raw or "").replace(";", ",").split(","): + email = item.strip().lower() + if email and EMAIL_PATTERN.fullmatch(email) and email not in recipients: + recipients.append(email) + return recipients + + +class TenantFinanceReportConfigService: + def __init__(self, db: Session) -> None: + self.db = db + + def get(self, *, tenant_id: str) -> TenantFinanceReportConfig | None: + tenant = require_report_tenant_id(tenant_id) + return self.db.scalar( + select(TenantFinanceReportConfig).where(TenantFinanceReportConfig.tenant_id == tenant) + ) + + def configured_recipients( + self, + *, + tenant_id: str, + requested: list[str] | None = None, + ) -> list[str]: + config = self.get(tenant_id=tenant_id) + if config is None or config.status != "active" or not config.delivery_enabled: + return [] + configured = normalize_report_recipients( + [str(item) for item in list(config.recipients_json or [])] + ) + if not configured: + return [] + if requested is None: + return configured + requested_values = normalize_report_recipients(requested) + # 手工触发也只能缩小到本租户已配置的收件人，不能把数据外发到任意地址。 + return [item for item in requested_values if item in configured] + + def upsert( + self, + *, + tenant_id: str, + recipients: list[str], + delivery_enabled: bool, + updated_by: str, + ) -> TenantFinanceReportConfig: + tenant = require_report_tenant_id(tenant_id) + if self.db.get(Tenant, tenant) is None: + raise LookupError("租户不存在。") + normalized = normalize_report_recipients(recipients) + if delivery_enabled and not normalized: + raise ValueError("启用财务报告投递前必须配置至少一个有效收件邮箱。") + row = self.get(tenant_id=tenant) + if row is None: + row = TenantFinanceReportConfig(tenant_id=tenant) + row.recipients_json = normalized + row.delivery_enabled = bool(delivery_enabled) + row.status = "active" if delivery_enabled else "disabled" + row.updated_by = str(updated_by or "").strip()[:100] + self.db.add(row) + self.db.commit() + self.db.refresh(row) + return row + + +class TenantFinanceReportRunService: + def __init__(self, db: Session) -> None: + self.db = db + + def reserve( + self, + *, + tenant_id: str, + report_type: str, + period_start: date, + period_end: date, + ) -> tuple[TenantFinanceReportRun, bool]: + tenant = require_report_tenant_id(tenant_id) + key = self.idempotency_key( + tenant_id=tenant, + report_type=report_type, + period_start=period_start, + period_end=period_end, + ) + existing = self._get(tenant, key, for_update=True) + if existing is not None: + if existing.status == "failed" or self._is_stale(existing): + existing.status = "running" + existing.error_message = None + existing.started_at = datetime.now(UTC) + existing.finished_at = None + self.db.commit() + self.db.refresh(existing) + return existing, True + return existing, False + + row = TenantFinanceReportRun( + tenant_id=tenant, + report_type=report_type, + period_start=period_start, + period_end=period_end, + idempotency_key=key, + status="running", + ) + self.db.add(row) + try: + self.db.commit() + self.db.refresh(row) + return row, True + except IntegrityError: + self.db.rollback() + winner = self._get(tenant, key, for_update=False) + if winner is None: + raise + return winner, False + + def attach_agent_run(self, row: TenantFinanceReportRun, run_id: str) -> None: + row.agent_run_id = str(run_id or "").strip() or None + self.db.add(row) + self.db.commit() + + def succeed(self, row: TenantFinanceReportRun, result: dict[str, Any]) -> None: + row.status = "succeeded" + row.storage_key = str((result.get("pdf") or {}).get("storage_key") or "") + row.result_json = dict(result) + row.error_message = None + row.finished_at = datetime.now(UTC) + self.db.add(row) + self.db.commit() + + def fail(self, row: TenantFinanceReportRun, exc: Exception) -> None: + row.status = "failed" + row.error_message = str(exc)[:2000] + row.finished_at = datetime.now(UTC) + self.db.add(row) + self.db.commit() + + @staticmethod + def idempotency_key( + *, + tenant_id: str, + report_type: str, + period_start: date, + period_end: date, + ) -> str: + canonical = ":".join( + ( + require_report_tenant_id(tenant_id), + str(report_type or "").strip(), + period_start.isoformat(), + period_end.isoformat(), + ) + ) + return "finance-report:v1:" + hashlib.sha256(canonical.encode()).hexdigest() + + def _get( + self, + tenant_id: str, + key: str, + *, + for_update: bool, + ) -> TenantFinanceReportRun | None: + stmt = select(TenantFinanceReportRun).where( + TenantFinanceReportRun.tenant_id == tenant_id, + TenantFinanceReportRun.idempotency_key == key, + ) + if for_update: + stmt = stmt.with_for_update() + return self.db.scalar(stmt) + + @staticmethod + def _is_stale(row: TenantFinanceReportRun) -> bool: + if row.status != "running" or row.started_at is None: + return False + started = row.started_at + if started.tzinfo is None: + started = started.replace(tzinfo=UTC) + return datetime.now(UTC) - started > RUN_STALE_AFTER diff --git a/server/src/app/services/financial_connector_actions.py b/server/src/app/services/financial_connector_actions.py new file mode 100644 index 0000000..04f7dd4 --- /dev/null +++ b/server/src/app/services/financial_connector_actions.py @@ -0,0 +1,156 @@ +from __future__ import annotations + +from decimal import Decimal, InvalidOperation +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.financial_record import ExpenseClaim +from app.services.expense_cases import ExpenseCaseService +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.expense_claims import ExpenseClaimService + + +class FinancialConnectorActionService: + """连接器对现有费用状态机的唯一写入口；不负责 commit。""" + + def __init__(self, db: Session) -> None: + self.db = db + self.claims = ExpenseClaimService(db) + self.expense_cases = ExpenseCaseService(db) + + def lock_claim(self, *, tenant_id: str, claim_id: str) -> ExpenseClaim | None: + statement = select(ExpenseClaim).where( + ExpenseClaim.id == str(claim_id).strip(), + ExpenseClaimTenantScopeMixin.build_claim_tenant_condition(tenant_id), + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + return self.db.scalar(statement.execution_options(populate_existing=True)) + + @staticmethod + def settlement_mismatch(claim: ExpenseClaim, payload: dict[str, Any]) -> str | None: + if str(claim.status or "").strip().lower() != "pending_payment": + return "claim_not_pending_payment" + return FinancialConnectorActionService.payment_payload_mismatch(claim, payload) + + @staticmethod + def payment_payload_mismatch( + claim: ExpenseClaim, + payload: dict[str, Any], + *, + require_external_reference: bool = True, + ) -> str | None: + if str(payload.get("claim_reference") or "").strip() != str( + claim.claim_no or "" + ).strip(): + return "claim_reference_mismatch" + try: + actual_amount = Decimal(str(payload.get("amount"))).quantize(Decimal("0.01")) + except (InvalidOperation, TypeError, ValueError): + return "amount_invalid" + expected_amount = Decimal(claim.amount or Decimal("0.00")).quantize(Decimal("0.01")) + if actual_amount != expected_amount: + return "amount_mismatch" + if str(payload.get("currency") or "").strip().upper() != str( + claim.currency or "CNY" + ).strip().upper(): + return "currency_mismatch" + if require_external_reference and not str( + payload.get("external_payment_reference") or "" + ).strip(): + return "external_reference_missing" + return None + + def settle_claim( + self, + claim: ExpenseClaim, + *, + tenant_id: str, + provider: str, + connector_event_id: str, + external_event_id: str, + content_hash: str, + verification_level: str, + evidence_classification: str, + external_reference_tail: str, + correlation_id: str, + ) -> ExpenseClaim: + return self.claims.mark_claim_paid_from_connector( + claim, + tenant_id=tenant_id, + provider=provider, + connector_event_id=connector_event_id, + external_event_id=external_event_id, + content_hash=content_hash, + verification_level=verification_level, + evidence_classification=evidence_classification, + external_reference_tail=external_reference_tail, + correlation_id=correlation_id, + ) + + def reopen_claim( + self, + claim: ExpenseClaim, + *, + tenant_id: str, + provider: str, + connector_event_id: str, + external_event_id: str, + origin_connector_event_id: str, + content_hash: str, + verification_level: str, + evidence_classification: str, + correlation_id: str, + ) -> ExpenseClaim: + return self.claims.reopen_claim_after_connector_reversal( + claim, + tenant_id=tenant_id, + provider=provider, + connector_event_id=connector_event_id, + external_event_id=external_event_id, + origin_connector_event_id=origin_connector_event_id, + content_hash=content_hash, + verification_level=verification_level, + evidence_classification=evidence_classification, + correlation_id=correlation_id, + ) + + def record_erp_event( + self, + claim: ExpenseClaim, + *, + tenant_id: str, + provider: str, + connector_event_id: str, + correlation_id: str, + event_type: str, + payload: dict[str, Any], + ) -> None: + expense_case = self.expense_cases.ensure_case_for_claim(claim, tenant_id=tenant_id) + self.expense_cases.record_claim_event( + claim, + event_type=event_type, + actor_id=f"financial-connector:{provider}", + tenant_id=tenant_id, + correlation_id=correlation_id, + idempotency_key=connector_event_id, + previous_status=str(claim.status or ""), + previous_approval_stage=str(claim.approval_stage or ""), + extra_payload={ + "connector_event_id": connector_event_id, + "origin_external_event_id": str( + payload.get("origin_external_event_id") or "" + ), + "erp_document_tail": _tail(payload.get("erp_document_number")), + "accounting_period": str(payload.get("accounting_period") or "")[:24], + }, + expense_case=expense_case, + update_case_state=False, + ) + + +def _tail(value: Any) -> str: + return str(value or "").strip()[-8:] diff --git a/server/src/app/services/financial_connector_auth.py b/server/src/app/services/financial_connector_auth.py new file mode 100644 index 0000000..535a057 --- /dev/null +++ b/server/src/app/services/financial_connector_auth.py @@ -0,0 +1,340 @@ +from __future__ import annotations + +import hashlib +import hmac +import json +import time +from collections.abc import Mapping +from dataclasses import dataclass +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.core.config import get_settings +from app.models.financial_connector import FinancialConnectorConfig +from app.schemas.financial_connector import FinancialEventEnvelope +from app.services.financial_connector_operational_events import ( + FinancialConnectorOperationalContext, + FinancialConnectorOperationalEventCandidate, +) + +FINANCIAL_EVENT_SIGNATURE_PATH = "/api/v1/integrations/financial-events" + + +class FinancialConnectorAuthError(PermissionError): + def __init__( + self, + message: str, + *, + code: str, + operational_context: FinancialConnectorOperationalContext | None = None, + ) -> None: + super().__init__(message) + self.code = code + self.operational_context = operational_context + self.operational_event: FinancialConnectorOperationalEventCandidate | None = None + + +@dataclass(frozen=True, slots=True) +class VerifiedConnectorRequest: + config: FinancialConnectorConfig + canonical_body: bytes + request_fingerprint: str + content_hash: str + verification_level: str + evidence_classification: str + operational_context: FinancialConnectorOperationalContext + + +class FinancialConnectorSecretResolver: + """从仅服务端可见配置解析密钥；数据库和响应都不持有密钥。""" + + def __init__(self, secrets: Mapping[str, str] | None = None) -> None: + self._secrets = { + str(key).strip(): str(value) + for key, value in dict(secrets or {}).items() + if str(key).strip() and str(value) + } + + @classmethod + def from_settings(cls) -> FinancialConnectorSecretResolver: + raw = str(get_settings().financial_connector_hmac_keys_json or "").strip() + if not raw: + return cls() + try: + parsed = json.loads(raw) + except (TypeError, ValueError) as error: + raise FinancialConnectorAuthError( + "服务端连接器密钥配置无法解析。", + code="secret_configuration_invalid", + ) from error + if not isinstance(parsed, dict): + raise FinancialConnectorAuthError( + "服务端连接器密钥配置格式无效。", + code="secret_configuration_invalid", + ) + return cls(parsed) + + def resolve(self, secret_ref: str) -> bytes: + value = self._secrets.get(str(secret_ref or "").strip()) + if not value: + raise FinancialConnectorAuthError( + "连接器密钥不可用。", + code="secret_unavailable", + ) + secret = value.encode("utf-8") + if len(secret) < 16 or not value.strip(): + raise FinancialConnectorAuthError( + "连接器密钥强度不足。", + code="secret_too_short", + ) + return secret + + +class FinancialConnectorAuthenticator: + def __init__( + self, + db: Session, + *, + secrets: FinancialConnectorSecretResolver | None = None, + now_epoch: int | None = None, + ) -> None: + self.db = db + self.secrets = secrets or FinancialConnectorSecretResolver.from_settings() + self.now_epoch = int(now_epoch) if now_epoch is not None else None + + def verify( + self, + envelope: FinancialEventEnvelope, + *, + tenant_header: str, + provider_header: str, + key_version_header: str, + timestamp_header: str, + signature_header: str, + ) -> VerifiedConnectorRequest: + tenant_id = str(tenant_header or "").strip() + provider = str(provider_header or "").strip().lower() + key_version = str(key_version_header or "").strip() + if not tenant_id or not provider or not key_version: + raise FinancialConnectorAuthError( + "连接器认证头不完整。", + code="authentication_headers_missing", + ) + if envelope.tenant_id != tenant_id: + raise FinancialConnectorAuthError( + "事件租户与签名租户不一致。", + code="tenant_mismatch", + ) + config = self.db.scalar( + select(FinancialConnectorConfig).where( + FinancialConnectorConfig.tenant_id == tenant_id, + FinancialConnectorConfig.provider == provider, + FinancialConnectorConfig.key_version == key_version, + ) + ) + if config is None or config.status != "active": + raise FinancialConnectorAuthError( + "连接器配置不存在或未激活。", + code="connector_inactive", + ) + secret = self.secrets.resolve(config.secret_ref) + operational_context = _operational_context( + envelope, + config=config, + secret=secret, + ) + try: + timestamp = int(str(timestamp_header or "").strip()) + except ValueError as error: + raise FinancialConnectorAuthError( + "连接器时间戳无效。", + code="timestamp_invalid", + operational_context=operational_context, + ) from error + now_epoch = self.now_epoch if self.now_epoch is not None else int(time.time()) + if abs(now_epoch - timestamp) > int(config.clock_skew_seconds): + raise FinancialConnectorAuthError( + "连接器事件超出允许时间窗口。", + code="timestamp_outside_window", + operational_context=operational_context, + ) + allowed = {str(item).strip() for item in list(config.allowed_event_types_json or [])} + if envelope.event_type not in allowed: + raise FinancialConnectorAuthError( + "连接器未获准发送该事件类型。", + code="event_type_not_allowed", + operational_context=operational_context, + ) + + canonical = canonical_financial_event(envelope) + signed_request = canonical_financial_request( + envelope, + tenant_id=tenant_id, + provider=provider, + key_version=key_version, + timestamp=timestamp, + ) + expected = hmac.new( + secret, + signed_request, + hashlib.sha256, + ).hexdigest() + supplied = str(signature_header or "").strip().lower() + if supplied.startswith("sha256="): + supplied = supplied.removeprefix("sha256=") + if len(supplied) != 64 or not hmac.compare_digest(expected, supplied): + raise FinancialConnectorAuthError( + "连接器签名验证失败。", + code="signature_invalid", + operational_context=operational_context, + ) + + content_digest = hashlib.sha256(canonical).hexdigest() + request_digest = hashlib.sha256( + canonical_financial_request( + envelope, + tenant_id=tenant_id, + provider=provider, + key_version=key_version, + timestamp=None, + ) + ).hexdigest() + verification, classification = _verification_for_environment(config.environment) + return VerifiedConnectorRequest( + config=config, + canonical_body=canonical, + request_fingerprint=f"sha256:{request_digest}", + content_hash=f"sha256:{content_digest}", + verification_level=verification, + evidence_classification=classification, + operational_context=operational_context, + ) + + +def canonical_financial_event(envelope: FinancialEventEnvelope | dict[str, Any]) -> bytes: + payload = ( + envelope.model_dump(mode="json") + if isinstance(envelope, FinancialEventEnvelope) + else dict(envelope) + ) + return json.dumps( + payload, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + + +def sign_financial_event( + envelope: FinancialEventEnvelope | dict[str, Any], + *, + timestamp: int, + secret: str, + tenant_id: str, + provider: str, + key_version: str, +) -> str: + canonical = canonical_financial_request( + envelope, + tenant_id=tenant_id, + provider=provider, + key_version=key_version, + timestamp=timestamp, + ) + digest = hmac.new( + secret.encode("utf-8"), + canonical, + hashlib.sha256, + ).hexdigest() + return f"sha256={digest}" + + +def canonical_financial_request( + envelope: FinancialEventEnvelope | dict[str, Any], + *, + tenant_id: str, + provider: str, + key_version: str, + timestamp: int | None, +) -> bytes: + """签名上下文绑定来源命名空间与固定 HTTP 入口，时间戳可从幂等指纹排除。""" + + body = ( + envelope.model_dump(mode="json") + if isinstance(envelope, FinancialEventEnvelope) + else dict(envelope) + ) + return json.dumps( + { + "version": "x-financial-event-v2", + "method": "POST", + "path": FINANCIAL_EVENT_SIGNATURE_PATH, + "tenant_id": str(tenant_id or "").strip(), + "provider": str(provider or "").strip().lower(), + "key_version": str(key_version or "").strip(), + "timestamp": timestamp, + "body": body, + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + + +def _verification_for_environment(environment: str) -> tuple[str, str]: + if environment == "production": + return "production_verified", "external_cash" + if environment == "staging": + return "staging_verified", "staging_connector" + return "simulated", "simulated_connector" + + +def _operational_context( + envelope: FinancialEventEnvelope, + *, + config: FinancialConnectorConfig, + secret: bytes, +) -> FinancialConnectorOperationalContext: + request_payload = canonical_financial_request( + envelope, + tenant_id=config.tenant_id, + provider=config.provider, + key_version=config.key_version, + timestamp=None, + ) + external_payload = json.dumps( + { + "version": "x-financial-operational-event-v1", + "path": FINANCIAL_EVENT_SIGNATURE_PATH, + "tenant_id": config.tenant_id, + "config_id": config.id, + "provider": config.provider, + "external_event_id": envelope.external_event_id, + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + return FinancialConnectorOperationalContext( + tenant_id=config.tenant_id, + config_id=config.id, + provider=config.provider, + environment=config.environment, + request_fingerprint=_hmac_fingerprint( + secret, + b"request:", + request_payload, + ), + external_event_fingerprint=_hmac_fingerprint( + secret, + b"external-event:", + external_payload, + ), + ) + + +def _hmac_fingerprint(secret: bytes, domain: bytes, payload: bytes) -> str: + digest = hmac.new(secret, domain + payload, hashlib.sha256).hexdigest() + return f"hmac-sha256:{digest}" diff --git a/server/src/app/services/financial_connector_commercial.py b/server/src/app/services/financial_connector_commercial.py new file mode 100644 index 0000000..1ba255a --- /dev/null +++ b/server/src/app/services/financial_connector_commercial.py @@ -0,0 +1,123 @@ +"""金融连接器已接受持久化事件的商业计量边界。""" + +from __future__ import annotations + +import hashlib +from dataclasses import dataclass +from datetime import UTC, datetime + +from sqlalchemy.orm import Session, sessionmaker + +from app.services.commercial_direct_operation import ( + CommercialDirectOperationBridge, + DirectOperationIdentity, + DirectOperationResult, +) +from app.services.commercial_transaction_callbacks import ( + bind_commercial_transaction_outcome, +) +from app.services.tenant_registry import required_tenant_id + + +class FinancialConnectorCommercialAccessDenied(RuntimeError): + pass + + +@dataclass(frozen=True, slots=True) +class FinancialConnectorCommercialAttempt: + identity: DirectOperationIdentity + + +class FinancialConnectorCommercialObserver: + """只在认证、去重之后预占，并在业务事件提交后结算一个 event。""" + + def __init__(self, bridge: CommercialDirectOperationBridge) -> None: + self.bridge = bridge + + def permit( + self, + *, + tenant_id: str, + config_id: str, + provider: str, + request_fingerprint: str, + received_at: datetime, + ) -> FinancialConnectorCommercialAttempt: + tenant = required_tenant_id(tenant_id) + fingerprint = _sha256_token(request_fingerprint) + config_token = _sha256_token(config_id) + identity = DirectOperationIdentity( + tenant_id=tenant, + operation_key=f"financial-ingest:{config_token}:{fingerprint}", + run_key=f"financial-ingest-run:{fingerprint}", + tool_type="connector", + tool_name="financial.ingest", + provider=str(provider or "").strip() or None, + started_at=_utc(received_at), + ) + permit = self.bridge.permit( + identity, + requested_quantity=1, + required_quantity_basis="events", + ) + if not permit.allowed: + raise FinancialConnectorCommercialAccessDenied(permit.reason) + return FinancialConnectorCommercialAttempt(identity=identity) + + def bind_to_transaction( + self, + db: Session, + attempt: FinancialConnectorCommercialAttempt, + ) -> None: + bind_commercial_transaction_outcome( + db, + on_commit=lambda: self.complete(attempt), + on_rollback=lambda: self.release(attempt), + operation_name="financial_connector_ingestion", + ) + + def complete( + self, + attempt: FinancialConnectorCommercialAttempt, + ) -> DirectOperationResult: + return self.bridge.complete( + attempt.identity, + outcome="succeeded", + authoritative_quantities={"events": 1}, + completed_at=datetime.now(UTC), + usage_source="persisted_financial_connector_event", + usage_availability="available", + ) + + def release( + self, + attempt: FinancialConnectorCommercialAttempt, + ) -> DirectOperationResult: + return self.bridge.complete( + attempt.identity, + outcome="not_sent", + authoritative_quantities={}, + completed_at=datetime.now(UTC), + usage_source="business_transaction_rolled_back", + usage_availability="unavailable", + ) + + +def build_financial_connector_commercial_observer( + db: Session, +) -> FinancialConnectorCommercialObserver: + factory = sessionmaker(bind=db.get_bind(), expire_on_commit=False) + return FinancialConnectorCommercialObserver( + CommercialDirectOperationBridge(factory, lookup_session=db) + ) + + +def _sha256_token(value: str) -> str: + normalized = str(value or "").strip() + if not normalized: + raise ValueError("连接器商业计量身份缺少可信摘要字段。") + return hashlib.sha256(normalized.encode("utf-8")).hexdigest() + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/financial_connector_config_audit.py b/server/src/app/services/financial_connector_config_audit.py new file mode 100644 index 0000000..883f7b6 --- /dev/null +++ b/server/src/app/services/financial_connector_config_audit.py @@ -0,0 +1,59 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy.orm import Session + +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorConfigEvent, +) + + +def connector_config_state(row: FinancialConnectorConfig) -> dict[str, Any]: + """返回可审计的配置状态，刻意排除 secret_ref。""" + + return { + "id": row.id, + "tenant_id": row.tenant_id, + "provider": row.provider, + "environment": row.environment, + "key_version": row.key_version, + "allowed_event_types": sorted( + str(item) for item in list(row.allowed_event_types_json or []) + ), + "clock_skew_seconds": int(row.clock_skew_seconds), + "status": row.status, + "version": int(row.version), + } + + +def append_connector_config_event( + db: Session, + row: FinancialConnectorConfig, + *, + action: str, + actor_id: str, + request_id: str, + reason: str, + expected_version: int | None, + before: dict[str, Any], + after: dict[str, Any] | None = None, +) -> FinancialConnectorConfigEvent: + event = FinancialConnectorConfigEvent( + id=str(uuid.uuid4()), + tenant_id=row.tenant_id, + config_id=row.id, + action=action, + actor_id=str(actor_id).strip(), + request_id=str(request_id).strip(), + reason=str(reason).strip(), + expected_version=expected_version, + before_json=dict(before), + after_json=dict(after if after is not None else connector_config_state(row)), + occurred_at=datetime.now(UTC), + ) + db.add(event) + return event diff --git a/server/src/app/services/financial_connector_config_lifecycle.py b/server/src/app/services/financial_connector_config_lifecycle.py new file mode 100644 index 0000000..8d88144 --- /dev/null +++ b/server/src/app/services/financial_connector_config_lifecycle.py @@ -0,0 +1,237 @@ +from __future__ import annotations + +import uuid +from dataclasses import dataclass + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorConfigEvent, +) +from app.schemas.financial_connector import ( + FinancialConnectorConfigLifecycleAction, + FinancialConnectorConfigRotateAction, +) +from app.services.financial_connector_auth import FinancialConnectorSecretResolver +from app.services.financial_connector_config_audit import ( + append_connector_config_event, + connector_config_state, +) +from app.services.financial_connector_configs import FinancialConnectorConfigError + + +class FinancialConnectorConfigConflictError(FinancialConnectorConfigError): + pass + + +@dataclass(frozen=True, slots=True) +class FinancialConnectorRotationResult: + previous: FinancialConnectorConfig + replacement: FinancialConnectorConfig + + +class FinancialConnectorConfigLifecycleService: + """以乐观版本和数据库行锁编排配置激活、停用与密钥轮换。""" + + def __init__( + self, + db: Session, + *, + secrets: FinancialConnectorSecretResolver | None = None, + ) -> None: + self.db = db + self.secrets = secrets or FinancialConnectorSecretResolver.from_settings() + + def activate( + self, + *, + tenant_id: str, + config_id: str, + payload: FinancialConnectorConfigLifecycleAction, + actor_id: str, + ) -> FinancialConnectorConfig: + row = self._lock_config(tenant_id, config_id) + self._require_version(row, payload.expected_version) + if row.status != "disabled": + raise FinancialConnectorConfigConflictError("只有停用配置可以激活。") + self._ensure_provider_has_no_live_config(row) + self.secrets.resolve(row.secret_ref) + + before = connector_config_state(row) + row.status = "active" + row.version += 1 + append_connector_config_event( + self.db, + row, + action="activated", + actor_id=actor_id, + request_id=payload.request_id, + reason=payload.reason, + expected_version=payload.expected_version, + before=before, + ) + self.db.flush() + return row + + def disable( + self, + *, + tenant_id: str, + config_id: str, + payload: FinancialConnectorConfigLifecycleAction, + actor_id: str, + ) -> FinancialConnectorConfig: + row = self._lock_config(tenant_id, config_id) + self._require_version(row, payload.expected_version) + if row.status not in {"active", "rotating"}: + raise FinancialConnectorConfigConflictError("只有激活或轮换中的配置可以停用。") + + before = connector_config_state(row) + row.status = "disabled" + row.version += 1 + append_connector_config_event( + self.db, + row, + action="disabled", + actor_id=actor_id, + request_id=payload.request_id, + reason=payload.reason, + expected_version=payload.expected_version, + before=before, + ) + self.db.flush() + return row + + def rotate( + self, + *, + tenant_id: str, + config_id: str, + payload: FinancialConnectorConfigRotateAction, + actor_id: str, + ) -> FinancialConnectorRotationResult: + row = self._lock_config(tenant_id, config_id) + self._require_version(row, payload.expected_version) + if row.status != "active": + raise FinancialConnectorConfigConflictError("只有激活配置可以发起密钥轮换。") + if payload.new_key_version == row.key_version: + raise FinancialConnectorConfigConflictError("新密钥版本必须不同于当前版本。") + existing = self.db.scalar( + select(FinancialConnectorConfig.id).where( + FinancialConnectorConfig.tenant_id == row.tenant_id, + FinancialConnectorConfig.provider == row.provider, + FinancialConnectorConfig.key_version == payload.new_key_version, + ) + ) + if existing is not None: + raise FinancialConnectorConfigConflictError("新密钥版本的配置已存在。") + self.secrets.resolve(payload.new_secret_ref) + + before = connector_config_state(row) + replacement = FinancialConnectorConfig( + id=str(uuid.uuid4()), + tenant_id=row.tenant_id, + provider=row.provider, + environment=row.environment, + key_version=payload.new_key_version, + secret_ref=payload.new_secret_ref, + allowed_event_types_json=list(row.allowed_event_types_json or []), + clock_skew_seconds=row.clock_skew_seconds, + status="active", + version=1, + created_by=str(actor_id).strip(), + ) + row.status = "rotating" + row.version += 1 + self.db.add(replacement) + self.db.flush([row, replacement]) + + previous_after = connector_config_state(row) + previous_after["replacement_config_id"] = replacement.id + append_connector_config_event( + self.db, + row, + action="rotation_started", + actor_id=actor_id, + request_id=payload.request_id, + reason=payload.reason, + expected_version=payload.expected_version, + before=before, + after=previous_after, + ) + replacement_after = connector_config_state(replacement) + replacement_after["rotated_from_config_id"] = row.id + append_connector_config_event( + self.db, + replacement, + action="rotation_replacement_created", + actor_id=actor_id, + request_id=payload.request_id, + reason=payload.reason, + expected_version=None, + before={}, + after=replacement_after, + ) + self.db.flush() + return FinancialConnectorRotationResult(previous=row, replacement=replacement) + + def list_events( + self, + *, + tenant_id: str, + config_id: str | None = None, + ) -> list[FinancialConnectorConfigEvent]: + conditions = [ + FinancialConnectorConfigEvent.tenant_id == str(tenant_id).strip() + ] + if str(config_id or "").strip(): + conditions.append( + FinancialConnectorConfigEvent.config_id == str(config_id).strip() + ) + return list( + self.db.scalars( + select(FinancialConnectorConfigEvent) + .where(*conditions) + .order_by( + FinancialConnectorConfigEvent.occurred_at, + FinancialConnectorConfigEvent.id, + ) + ).all() + ) + + def _lock_config(self, tenant_id: str, config_id: str) -> FinancialConnectorConfig: + statement = select(FinancialConnectorConfig).where( + FinancialConnectorConfig.tenant_id == str(tenant_id).strip(), + FinancialConnectorConfig.id == str(config_id).strip(), + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + row = self.db.scalar(statement.execution_options(populate_existing=True)) + if row is None: + raise LookupError("连接器配置不存在。") + return row + + @staticmethod + def _require_version(row: FinancialConnectorConfig, expected_version: int) -> None: + if row.version != expected_version: + raise FinancialConnectorConfigConflictError( + f"连接器配置版本已变化，当前版本为 {row.version}。" + ) + + def _ensure_provider_has_no_live_config(self, row: FinancialConnectorConfig) -> None: + statement = select(FinancialConnectorConfig.id).where( + FinancialConnectorConfig.tenant_id == row.tenant_id, + FinancialConnectorConfig.provider == row.provider, + FinancialConnectorConfig.id != row.id, + FinancialConnectorConfig.status.in_(("active", "rotating")), + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + if self.db.scalar(statement) is not None: + raise FinancialConnectorConfigConflictError( + "同一来源已有激活或轮换中的配置，请使用轮换动作切换密钥。" + ) diff --git a/server/src/app/services/financial_connector_configs.py b/server/src/app/services/financial_connector_configs.py new file mode 100644 index 0000000..dc89357 --- /dev/null +++ b/server/src/app/services/financial_connector_configs.py @@ -0,0 +1,83 @@ +from __future__ import annotations + +import uuid + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.financial_connector import FinancialConnectorConfig +from app.schemas.financial_connector import FinancialConnectorConfigCreate +from app.services.financial_connector_config_audit import ( + append_connector_config_event, + connector_config_state, +) + + +class FinancialConnectorConfigError(ValueError): + pass + + +class FinancialConnectorConfigService: + def __init__(self, db: Session) -> None: + self.db = db + + def create( + self, + *, + tenant_id: str, + payload: FinancialConnectorConfigCreate, + actor_id: str, + ) -> FinancialConnectorConfig: + normalized_tenant = str(tenant_id or "").strip() + if not normalized_tenant: + raise FinancialConnectorConfigError("连接器租户不能为空。") + provider = payload.provider.strip().lower() + existing = self.db.scalar( + select(FinancialConnectorConfig.id).where( + FinancialConnectorConfig.tenant_id == normalized_tenant, + FinancialConnectorConfig.provider == provider, + FinancialConnectorConfig.key_version == payload.key_version, + ) + ) + if existing is not None: + raise FinancialConnectorConfigError("该租户、来源和密钥版本的配置已存在。") + row = FinancialConnectorConfig( + id=str(uuid.uuid4()), + tenant_id=normalized_tenant, + provider=provider, + environment=payload.environment, + key_version=payload.key_version, + secret_ref=payload.secret_ref, + allowed_event_types_json=sorted(set(payload.allowed_event_types)), + clock_skew_seconds=payload.clock_skew_seconds, + status="disabled", + version=1, + created_by=str(actor_id or "platform-admin").strip() or "platform-admin", + ) + self.db.add(row) + self.db.flush() + append_connector_config_event( + self.db, + row, + action="created", + actor_id=row.created_by, + request_id=payload.request_id, + reason=payload.reason, + expected_version=None, + before={}, + after=connector_config_state(row), + ) + self.db.flush() + return row + + def list_for_tenant(self, tenant_id: str) -> list[FinancialConnectorConfig]: + return list( + self.db.scalars( + select(FinancialConnectorConfig) + .where(FinancialConnectorConfig.tenant_id == str(tenant_id).strip()) + .order_by( + FinancialConnectorConfig.provider, + FinancialConnectorConfig.key_version, + ) + ).all() + ) diff --git a/server/src/app/services/financial_connector_ingestion.py b/server/src/app/services/financial_connector_ingestion.py new file mode 100644 index 0000000..6866d7c --- /dev/null +++ b/server/src/app/services/financial_connector_ingestion.py @@ -0,0 +1,296 @@ +from __future__ import annotations + +import hashlib +import uuid +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import select, text +from sqlalchemy.orm import Session + +from app.models.financial_connector import FinancialConnectorEvent +from app.schemas.financial_connector import ( + FinancialEventEnvelope, + FinancialEventIngestionRead, +) +from app.services.financial_connector_auth import ( + FinancialConnectorAuthenticator, + FinancialConnectorAuthError, + FinancialConnectorSecretResolver, + VerifiedConnectorRequest, +) +from app.services.financial_connector_commercial import ( + FinancialConnectorCommercialObserver, + build_financial_connector_commercial_observer, +) +from app.services.financial_connector_operational_events import ( + FinancialConnectorOperationalEventCandidate, + FinancialConnectorOperationalEventService, + operational_event_candidate, +) +from app.services.financial_connector_simulation import FinancialConnectorSimulationService +from app.services.payment_reconciliation import PaymentReconciliationService + + +class FinancialConnectorConflictError(ValueError): + def __init__( + self, + message: str, + *, + operational_event: FinancialConnectorOperationalEventCandidate, + ) -> None: + super().__init__(message) + self.operational_event = operational_event + + +class FinancialConnectorIngestionService: + def __init__( + self, + db: Session, + *, + secrets: FinancialConnectorSecretResolver | None = None, + now_epoch: int | None = None, + commercial_observer: FinancialConnectorCommercialObserver | None = None, + ) -> None: + self.db = db + self.received_at = ( + datetime.fromtimestamp(now_epoch, UTC) if now_epoch is not None else datetime.now(UTC) + ) + self.authenticator = FinancialConnectorAuthenticator( + db, + secrets=secrets, + now_epoch=now_epoch, + ) + self.reconciliation = PaymentReconciliationService(db) + self.simulation = FinancialConnectorSimulationService(db) + self.operational_events = FinancialConnectorOperationalEventService(db) + self.commercial_observer = ( + commercial_observer or build_financial_connector_commercial_observer(db) + ) + + def ingest( + self, + envelope: FinancialEventEnvelope, + *, + tenant_header: str, + provider_header: str, + key_version_header: str, + timestamp_header: str, + signature_header: str, + ) -> FinancialEventIngestionRead: + try: + verified = self.authenticator.verify( + envelope, + tenant_header=tenant_header, + provider_header=provider_header, + key_version_header=key_version_header, + timestamp_header=timestamp_header, + signature_header=signature_header, + ) + except FinancialConnectorAuthError as error: + if error.operational_context is not None: + candidate = operational_event_candidate( + error.operational_context, + event_type="auth_failure", + reason_code=error.code, + occurred_at=self.received_at, + ) + self.operational_events.record(candidate) + error.operational_event = candidate + raise + provider = verified.config.provider + self._serialize_external_event( + tenant_id=envelope.tenant_id, + provider=provider, + external_event_id=envelope.external_event_id, + ) + existing = self.db.scalar( + select(FinancialConnectorEvent).where( + FinancialConnectorEvent.tenant_id == envelope.tenant_id, + FinancialConnectorEvent.provider == provider, + FinancialConnectorEvent.external_event_id == envelope.external_event_id, + ) + ) + if existing is not None: + if existing.request_fingerprint != verified.request_fingerprint: + candidate = operational_event_candidate( + verified.operational_context, + event_type="payload_conflict", + reason_code="external_event_payload_conflict", + occurred_at=self.received_at, + ) + self.operational_events.record(candidate) + raise FinancialConnectorConflictError( + "相同 external_event_id 已对应不同 payload，首次事实不会被覆盖。", + operational_event=candidate, + ) + self.operational_events.record( + operational_event_candidate( + verified.operational_context, + event_type="replay", + reason_code="duplicate_external_event", + occurred_at=self.received_at, + ) + ) + replay = FinancialEventIngestionRead.model_validate(existing.response_json) + return replay.model_copy(update={"replayed": True}) + + commercial_attempt = self.commercial_observer.permit( + tenant_id=verified.config.tenant_id, + config_id=verified.config.id, + provider=provider, + request_fingerprint=verified.request_fingerprint, + received_at=self.received_at, + ) + connector_event_id = str(uuid.uuid4()) + try: + with self.db.no_autoflush: + if verified.verification_level == "production_verified": + result = self.reconciliation.process( + envelope, + verified, + connector_event_id=connector_event_id, + ) + else: + result = self.simulation.project(envelope, verified) + # 对账过程会更新现有投影并让它引用本次事实。原始事件查询必须继续 + # 禁止自动 flush，确保新事实先入库，再更新带外键的投影。 + origin_event_id = self._origin_event_id( + envelope, + provider=provider, + verified=verified, + ) + response = FinancialEventIngestionRead( + accepted=True, + replayed=False, + event_id=connector_event_id, + external_event_id=envelope.external_event_id, + processing_status=result.processing_status, + reconciliation_case_id=result.case.id if result.case is not None else None, + reconciliation_status=result.case.status if result.case is not None else None, + claim_status=result.claim_status, + verification_level=verified.verification_level, + evidence_classification=verified.evidence_classification, + projection_scope=( + "canonical" + if verified.verification_level == "production_verified" + else "simulation_only" + ), + error_code=result.error_code, + ) + event = FinancialConnectorEvent( + id=connector_event_id, + tenant_id=envelope.tenant_id, + config_id=verified.config.id, + provider=provider, + environment=verified.config.environment, + direction="inbound", + external_event_id=envelope.external_event_id, + event_type=envelope.event_type, + occurred_at=envelope.occurred_at, + received_at=self.received_at, + key_version=verified.config.key_version, + verification_level=verified.verification_level, + request_fingerprint=verified.request_fingerprint, + content_hash=verified.content_hash, + processing_status=result.processing_status, + error_code=result.error_code, + claim_id=result.claim_id, + expense_case_id=result.expense_case_id, + origin_event_id=origin_event_id, + correlation_id=envelope.correlation_id, + external_reference_tail=_tail(envelope.payload.get("external_payment_reference")) + or None, + normalized_payload_json=_sanitized_payload(envelope, result.claim_id), + response_json=response.model_dump(mode="json"), + ) + self.db.add(event) + self.db.flush([event]) + if result.case is not None: + self.db.add(result.case) + self.db.flush([result.case]) + if result.audit_event is not None: + self.db.add(result.audit_event) + self.db.flush([result.audit_event]) + + now = self.received_at + if result.processing_status == "processed": + verified.config.last_success_at = now + verified.config.last_error_code = None + else: + verified.config.last_error_at = now + verified.config.last_error_code = result.error_code + self.commercial_observer.bind_to_transaction(self.db, commercial_attempt) + return response + except Exception: + self.commercial_observer.release(commercial_attempt) + raise + + def _origin_event_id( + self, + envelope: FinancialEventEnvelope, + *, + provider: str, + verified: VerifiedConnectorRequest, + ) -> str | None: + if envelope.event_type not in { + "payment_refunded", + "payment_reversed", + "erp_posted", + "erp_posting_failed", + }: + return None + return self.db.scalar( + select(FinancialConnectorEvent.id).where( + FinancialConnectorEvent.tenant_id == envelope.tenant_id, + FinancialConnectorEvent.provider == provider, + FinancialConnectorEvent.environment == verified.config.environment, + FinancialConnectorEvent.verification_level == verified.verification_level, + FinancialConnectorEvent.external_event_id + == str(envelope.payload.get("origin_external_event_id") or ""), + ) + ) + + def _serialize_external_event( + self, + *, + tenant_id: str, + provider: str, + external_event_id: str, + ) -> None: + bind = self.db.get_bind() + if bind is None or bind.dialect.name != "postgresql": + return + lock_key = f"financial-event:{tenant_id}:{provider}:{external_event_id}" + self.db.execute( + text("SELECT pg_advisory_xact_lock(hashtextextended(:lock_key, 0))"), + {"lock_key": lock_key}, + ) + + +def _sanitized_payload( + envelope: FinancialEventEnvelope, + trusted_claim_id: str | None, +) -> dict[str, Any]: + payload = envelope.payload + normalized: dict[str, Any] = { + "claim_id": trusted_claim_id, + "amount": str(payload.get("amount") or "")[:40], + "currency": str(payload.get("currency") or "").upper()[:3], + "external_reference_tail": _tail(payload.get("external_payment_reference")), + "origin_external_event_id": str(payload.get("origin_external_event_id") or "")[:160] + or None, + "failure_code": str(payload.get("failure_code") or "")[:80] or None, + "accounting_period": str(payload.get("accounting_period") or "")[:24] or None, + } + document = str(payload.get("erp_document_number") or "").strip() + if document: + normalized["erp_document_tail"] = _tail(document) + normalized["erp_document_hash"] = ( + f"sha256:{hashlib.sha256(document.encode('utf-8')).hexdigest()}" + ) + return normalized + + +def _tail(value: Any) -> str: + return str(value or "").strip()[-8:] diff --git a/server/src/app/services/financial_connector_mock_adapter.py b/server/src/app/services/financial_connector_mock_adapter.py new file mode 100644 index 0000000..89ff99a --- /dev/null +++ b/server/src/app/services/financial_connector_mock_adapter.py @@ -0,0 +1,326 @@ +from __future__ import annotations + +import hashlib +import json +from datetime import UTC, datetime +from decimal import Decimal + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.financial_connector import FinancialConnectorConfig, FinancialConnectorEvent +from app.models.financial_record import ExpenseClaim +from app.schemas.financial_connector import ( + FinancialConnectorSimulationCreate, + FinancialConnectorSimulationRead, + FinancialConnectorSimulationStepRead, + FinancialEventEnvelope, +) +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.financial_connector_auth import ( + FinancialConnectorSecretResolver, + sign_financial_event, +) +from app.services.financial_connector_ingestion import ( + FinancialConnectorConflictError, + FinancialConnectorIngestionService, +) + + +class FinancialConnectorMockAdapterError(ValueError): + pass + + +class FinancialConnectorMockAdapter: + """通过正式认证入口生成确定性的非生产连接器场景。""" + + def __init__( + self, + db: Session, + *, + secrets: FinancialConnectorSecretResolver | None = None, + now: datetime | None = None, + ) -> None: + self.db = db + self.secrets = secrets or FinancialConnectorSecretResolver.from_settings() + self.now = now or datetime.now(UTC) + + def run( + self, + *, + tenant_id: str, + config_id: str, + payload: FinancialConnectorSimulationCreate, + ) -> FinancialConnectorSimulationRead: + tenant = str(tenant_id or "").strip() + config = self.db.scalar( + select(FinancialConnectorConfig).where( + FinancialConnectorConfig.tenant_id == tenant, + FinancialConnectorConfig.id == str(config_id or "").strip(), + ) + ) + if config is None: + raise LookupError("连接器配置不存在。") + if config.status != "active": + raise FinancialConnectorMockAdapterError("只有已激活的连接器配置可以运行模拟。") + if config.environment == "production": + raise FinancialConnectorMockAdapterError("生产连接器禁止运行模拟场景。") + + claim = self.db.scalar( + select(ExpenseClaim).where( + ExpenseClaim.id == payload.claim_id, + ExpenseClaimTenantScopeMixin.build_claim_tenant_condition(tenant), + ) + ) + if claim is None: + raise LookupError("报销单不存在。") + + required_types = _SCENARIO_EVENT_TYPES[payload.scenario] + allowed_types = {str(item).strip() for item in config.allowed_event_types_json or []} + missing = sorted(required_types - allowed_types) + if missing: + raise FinancialConnectorMockAdapterError( + f"连接器未允许模拟所需事件类型：{', '.join(missing)}。" + ) + + request_fingerprint, digest = _simulation_fingerprint( + tenant_id=tenant, + config_id=config.id, + claim_id=claim.id, + scenario=payload.scenario, + request_id=payload.request_id, + ) + timestamp = int(self.now.timestamp()) + ingestion = FinancialConnectorIngestionService( + self.db, + secrets=self.secrets, + now_epoch=timestamp, + ) + steps = self._run_scenario( + ingestion=ingestion, + config=config, + claim=claim, + scenario=payload.scenario, + digest=digest, + timestamp=timestamp, + ) + return FinancialConnectorSimulationRead( + tenant_id=tenant, + config_id=config.id, + provider=config.provider, + environment=config.environment, + scenario=payload.scenario, + request_fingerprint=request_fingerprint, + evidence_classification=( + "staging_connector" if config.environment == "staging" else "simulated_connector" + ), + steps=steps, + ) + + def _run_scenario( + self, + *, + ingestion: FinancialConnectorIngestionService, + config: FinancialConnectorConfig, + claim: ExpenseClaim, + scenario: str, + digest: str, + timestamp: int, + ) -> list[FinancialConnectorSimulationStepRead]: + if scenario == "failure": + return [ + self._ingest_step( + ingestion, + config, + self._envelope(config, claim, scenario, digest, "failure", "payment_failed"), + timestamp, + ) + ] + if scenario == "out_of_order": + return [ + self._ingest_step( + ingestion, + config, + self._envelope( + config, + claim, + scenario, + digest, + "refund", + "payment_refunded", + origin_external_event_id=f"sim-missing-{digest[:24]}", + ), + timestamp, + ) + ] + + settlement = self._envelope( + config, + claim, + scenario, + digest, + "settlement", + "payment_settled", + ) + first = self._ingest_step(ingestion, config, settlement, timestamp) + if scenario == "success": + return [first] + if scenario == "duplicate": + return [first, self._ingest_step(ingestion, config, settlement, timestamp)] + if scenario == "conflict": + conflicting = self._envelope( + config, + claim, + scenario, + digest, + "settlement", + "payment_settled", + amount=Decimal(claim.amount) + Decimal("0.01"), + ) + try: + second = self._ingest_step(ingestion, config, conflicting, timestamp) + except FinancialConnectorConflictError: + second = FinancialConnectorSimulationStepRead( + name="settlement", + event_type="payment_settled", + outcome="conflict", + error_code="external_event_payload_conflict", + ) + return [first, second] + + follow_up_type = "payment_refunded" if scenario == "refund" else "erp_posted" + follow_up_name = "refund" if scenario == "refund" else "erp_receipt" + follow_up = self._envelope( + config, + claim, + scenario, + digest, + follow_up_name, + follow_up_type, + origin_external_event_id=settlement.external_event_id, + ) + return [first, self._ingest_step(ingestion, config, follow_up, timestamp)] + + def _envelope( + self, + config: FinancialConnectorConfig, + claim: ExpenseClaim, + scenario: str, + digest: str, + step: str, + event_type: str, + *, + origin_external_event_id: str | None = None, + amount: Decimal | None = None, + ) -> FinancialEventEnvelope: + external_event_id = f"sim-{scenario}-{digest[:24]}-{step}"[:160] + existing_time = self.db.scalar( + select(FinancialConnectorEvent.occurred_at).where( + FinancialConnectorEvent.tenant_id == config.tenant_id, + FinancialConnectorEvent.provider == config.provider, + FinancialConnectorEvent.external_event_id == external_event_id, + ) + ) + event_payload: dict[str, str] = { + "claim_id": claim.id, + "claim_reference": claim.claim_no, + "amount": str(amount if amount is not None else claim.amount), + "currency": str(claim.currency or "CNY").strip().upper(), + } + if event_type in {"payment_settled", "payment_failed", "payment_refunded"}: + event_payload["external_payment_reference"] = f"SIM-PAY-{digest[-16:]}" + if origin_external_event_id: + event_payload["origin_external_event_id"] = origin_external_event_id + if event_type == "payment_failed": + event_payload["failure_code"] = "simulated_provider_failure" + if event_type == "erp_posted": + event_payload["erp_document_number"] = f"SIM-ERP-{digest[-16:]}" + event_payload["accounting_period"] = self.now.strftime("%Y-%m") + return FinancialEventEnvelope( + tenant_id=config.tenant_id, + external_event_id=external_event_id, + event_type=event_type, + occurred_at=_aware(existing_time or self.now), + correlation_id=f"sim-{digest[:56]}", + payload=event_payload, + ) + + def _ingest_step( + self, + ingestion: FinancialConnectorIngestionService, + config: FinancialConnectorConfig, + envelope: FinancialEventEnvelope, + timestamp: int, + ) -> FinancialConnectorSimulationStepRead: + secret = self.secrets.resolve(config.secret_ref).decode("utf-8") + signature = sign_financial_event( + envelope, + timestamp=timestamp, + secret=secret, + tenant_id=config.tenant_id, + provider=config.provider, + key_version=config.key_version, + ) + result = ingestion.ingest( + envelope, + tenant_header=config.tenant_id, + provider_header=config.provider, + key_version_header=config.key_version, + timestamp_header=str(timestamp), + signature_header=signature, + ) + if result.replayed: + outcome = "replayed" + elif result.processing_status == "exception": + outcome = "expected_exception" + else: + outcome = "accepted" + return FinancialConnectorSimulationStepRead( + name=envelope.external_event_id.rsplit("-", 1)[-1], + event_type=envelope.event_type, + outcome=outcome, + event_id=result.event_id, + processing_status=result.processing_status, + error_code=result.error_code, + ) + + +_SCENARIO_EVENT_TYPES: dict[str, set[str]] = { + "success": {"payment_settled"}, + "failure": {"payment_failed"}, + "out_of_order": {"payment_refunded"}, + "duplicate": {"payment_settled"}, + "conflict": {"payment_settled"}, + "refund": {"payment_settled", "payment_refunded"}, + "erp_receipt": {"payment_settled", "erp_posted"}, +} + + +def _simulation_fingerprint( + *, + tenant_id: str, + config_id: str, + claim_id: str, + scenario: str, + request_id: str, +) -> tuple[str, str]: + raw = json.dumps( + { + "tenant_id": tenant_id, + "config_id": config_id, + "claim_id": claim_id, + "scenario": scenario, + "request_id": request_id, + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + digest = hashlib.sha256(raw).hexdigest() + return f"sha256:{digest}", digest + + +def _aware(value: datetime) -> datetime: + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value diff --git a/server/src/app/services/financial_connector_observability.py b/server/src/app/services/financial_connector_observability.py new file mode 100644 index 0000000..d789190 --- /dev/null +++ b/server/src/app/services/financial_connector_observability.py @@ -0,0 +1,349 @@ +from __future__ import annotations + +from collections.abc import Iterable +from dataclasses import dataclass +from datetime import UTC, datetime, timedelta + +from sqlalchemy import case, func, select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorEvent, + FinancialConnectorOperationalEvent, + PaymentReconciliationCase, +) +from app.schemas.financial_connector import ( + FinancialConnectorMetricAvailabilityRead, + FinancialConnectorObservabilityItemRead, + FinancialConnectorObservabilityRead, + FinancialConnectorObservabilitySummaryRead, +) +from app.services.financial_connector_operational_events import ( + FINANCIAL_CONNECTOR_OPERATIONAL_SOURCE_REVISION, +) + + +class FinancialConnectorObservabilityPermissionError(PermissionError): + pass + + +class FinancialConnectorObservabilityService: + """只从租户最小化事实聚合指标，不读取或返回原始财务 payload。""" + + def __init__(self, db: Session, *, now: datetime | None = None) -> None: + self.db = db + self.now = now or datetime.now(UTC) + + @staticmethod + def require_finance_or_admin(current_user: CurrentUserContext) -> None: + roles = {str(item).strip().lower() for item in current_user.role_codes} + if not current_user.is_admin and not roles & {"finance", "executive"}: + raise FinancialConnectorObservabilityPermissionError( + "只有财务业务角色或平台管理员可以查看连接器运行指标。" + ) + + def read_for_current_user( + self, + current_user: CurrentUserContext, + *, + window_hours: int, + ) -> FinancialConnectorObservabilityRead: + self.require_finance_or_admin(current_user) + return self.read_for_tenant(current_user.tenant_id, window_hours=window_hours) + + def read_for_tenant( + self, + tenant_id: str, + *, + window_hours: int, + ) -> FinancialConnectorObservabilityRead: + tenant = str(tenant_id or "").strip() + started_at = self.now - timedelta(hours=window_hours) + configs = list( + self.db.scalars( + select(FinancialConnectorConfig) + .where(FinancialConnectorConfig.tenant_id == tenant) + .order_by( + FinancialConnectorConfig.provider, + FinancialConnectorConfig.key_version, + ) + ).all() + ) + event_stats = self._event_stats(tenant, started_at, self.now) + operational_stats = self._operational_stats(tenant, started_at, self.now) + anomaly_stats = self._anomaly_stats(tenant, started_at, self.now) + anomaly_owners = _anomaly_owner_config_ids(configs) + items = [ + self._item( + config, + event_stats.get(config.id, _EMPTY_EVENT_STATS), + operational_stats.get(config.id, _OperationalStats()), + ( + anomaly_stats.get(config.provider, 0) + if anomaly_owners.get(config.provider) == config.id + else 0 + ), + ) + for config in configs + ] + event_count = sum(item.event_count for item in items) + failed_count = sum(item.failed_event_count for item in items) + retry_metric = _durable_metric("financial_connector_operational_events.replay") + auth_failure_metric = _durable_metric( + "financial_connector_operational_events.auth_failure" + ) + signature_metric = _durable_metric( + "financial_connector_operational_events.auth_failure:signature_invalid" + ) + conflict_metric = _durable_metric( + "financial_connector_operational_events.payload_conflict" + ) + return FinancialConnectorObservabilityRead( + tenant_id=tenant, + window_hours=window_hours, + window_started_at=started_at, + as_of=self.now, + generated_at=self.now, + source_revision=FINANCIAL_CONNECTOR_OPERATIONAL_SOURCE_REVISION, + summary=FinancialConnectorObservabilitySummaryRead( + connector_count=len(items), + active_connector_count=sum(item.status == "active" for item in items), + event_count=event_count, + processed_event_count=sum(item.processed_event_count for item in items), + failed_event_count=failed_count, + failure_rate=_rate(failed_count, event_count), + backlog_count=sum(item.backlog_count for item in items), + reconciliation_anomaly_count=sum(anomaly_stats.values()), + retry_count=sum(int(item.retry_count or 0) for item in items), + auth_failure_count=sum(item.auth_failure_count for item in items), + signature_failure_count=sum( + int(item.signature_failure_count or 0) for item in items + ), + payload_conflict_count=sum(item.payload_conflict_count for item in items), + latest_replay_at=_latest(item.latest_replay_at for item in items), + latest_auth_failure_at=_latest( + item.latest_auth_failure_at for item in items + ), + latest_signature_failure_at=_latest( + item.latest_signature_failure_at for item in items + ), + latest_payload_conflict_at=_latest( + item.latest_payload_conflict_at for item in items + ), + ), + retry_metric=retry_metric, + auth_failure_metric=auth_failure_metric, + signature_failure_metric=signature_metric, + payload_conflict_metric=conflict_metric, + items=items, + ) + + def _event_stats( + self, + tenant_id: str, + started_at: datetime, + as_of: datetime, + ) -> dict[str, tuple[int, int, int, int]]: + rows = self.db.execute( + select( + FinancialConnectorEvent.config_id, + func.count(FinancialConnectorEvent.id), + func.sum( + case((FinancialConnectorEvent.processing_status == "processed", 1), else_=0) + ), + func.sum( + case((FinancialConnectorEvent.processing_status == "exception", 1), else_=0) + ), + func.sum( + case((FinancialConnectorEvent.processing_status == "pending", 1), else_=0) + ), + ) + .where( + FinancialConnectorEvent.tenant_id == tenant_id, + FinancialConnectorEvent.received_at >= started_at, + FinancialConnectorEvent.received_at <= as_of, + ) + .group_by(FinancialConnectorEvent.config_id) + ).all() + return { + str(config_id): ( + int(total or 0), + int(processed or 0), + int(failed or 0), + int(pending or 0), + ) + for config_id, total, processed, failed, pending in rows + } + + def _operational_stats( + self, + tenant_id: str, + started_at: datetime, + as_of: datetime, + ) -> dict[str, _OperationalStats]: + rows = self.db.execute( + select( + FinancialConnectorOperationalEvent.config_id, + FinancialConnectorOperationalEvent.event_type, + FinancialConnectorOperationalEvent.reason_code, + func.count(FinancialConnectorOperationalEvent.id), + func.max(FinancialConnectorOperationalEvent.occurred_at), + ) + .where( + FinancialConnectorOperationalEvent.tenant_id == tenant_id, + FinancialConnectorOperationalEvent.occurred_at >= started_at, + FinancialConnectorOperationalEvent.occurred_at <= as_of, + ) + .group_by( + FinancialConnectorOperationalEvent.config_id, + FinancialConnectorOperationalEvent.event_type, + FinancialConnectorOperationalEvent.reason_code, + ) + ).all() + result: dict[str, _OperationalStats] = {} + for config_id, event_type, reason_code, total, latest in rows: + stats = result.setdefault(str(config_id), _OperationalStats()) + count = int(total or 0) + if event_type == "replay": + stats.replay_count += count + stats.latest_replay_at = _latest((stats.latest_replay_at, latest)) + elif event_type == "auth_failure": + stats.auth_failure_count += count + stats.latest_auth_failure_at = _latest( + (stats.latest_auth_failure_at, latest) + ) + if reason_code == "signature_invalid": + stats.signature_failure_count += count + stats.latest_signature_failure_at = _latest( + (stats.latest_signature_failure_at, latest) + ) + elif event_type == "payload_conflict": + stats.payload_conflict_count += count + stats.latest_payload_conflict_at = _latest( + (stats.latest_payload_conflict_at, latest) + ) + return result + + def _anomaly_stats( + self, + tenant_id: str, + started_at: datetime, + as_of: datetime, + ) -> dict[str, int]: + rows = self.db.execute( + select(PaymentReconciliationCase.provider, func.count(PaymentReconciliationCase.id)) + .where( + PaymentReconciliationCase.tenant_id == tenant_id, + PaymentReconciliationCase.updated_at >= started_at, + PaymentReconciliationCase.updated_at <= as_of, + PaymentReconciliationCase.status.in_({"exception", "reopened"}), + ) + .group_by(PaymentReconciliationCase.provider) + ).all() + return {str(provider): int(total or 0) for provider, total in rows} + + @staticmethod + def _item( + config: FinancialConnectorConfig, + stats: tuple[int, int, int, int], + operational: _OperationalStats, + anomaly_count: int, + ) -> FinancialConnectorObservabilityItemRead: + total, processed, failed, pending = stats + classification, label = _evidence_for_environment(config.environment) + return FinancialConnectorObservabilityItemRead( + config_id=config.id, + provider=config.provider, + environment=config.environment, + key_version=config.key_version, + status=config.status, + evidence_classification=classification, + evidence_label=label, + last_success_at=config.last_success_at, + last_error_at=config.last_error_at, + last_error_code=config.last_error_code, + event_count=total, + processed_event_count=processed, + failed_event_count=failed, + failure_rate=_rate(failed, total), + backlog_count=pending, + reconciliation_anomaly_count=anomaly_count, + retry_count=operational.replay_count, + auth_failure_count=operational.auth_failure_count, + signature_failure_count=operational.signature_failure_count, + payload_conflict_count=operational.payload_conflict_count, + latest_replay_at=operational.latest_replay_at, + latest_auth_failure_at=operational.latest_auth_failure_at, + latest_signature_failure_at=operational.latest_signature_failure_at, + latest_payload_conflict_at=operational.latest_payload_conflict_at, + ) + + +_EMPTY_EVENT_STATS = (0, 0, 0, 0) + + +@dataclass(slots=True) +class _OperationalStats: + replay_count: int = 0 + auth_failure_count: int = 0 + signature_failure_count: int = 0 + payload_conflict_count: int = 0 + latest_replay_at: datetime | None = None + latest_auth_failure_at: datetime | None = None + latest_signature_failure_at: datetime | None = None + latest_payload_conflict_at: datetime | None = None + + +def _rate(numerator: int, denominator: int) -> float: + if denominator <= 0: + return 0.0 + return round(numerator / denominator, 4) + + +def _evidence_for_environment(environment: str) -> tuple[str, str]: + if environment == "production": + return "external_cash", "生产外部现金回执" + if environment == "staging": + return "staging_connector", "预发布验证回执（不入核心账）" + return "simulated_connector", "模拟回执（不入核心账）" + + +def _durable_metric(source: str) -> FinancialConnectorMetricAvailabilityRead: + return FinancialConnectorMetricAvailabilityRead( + status="available", + source=source, + reason=None, + ) + + +def _latest(values: Iterable[datetime | None]) -> datetime | None: + timestamps = [ + item.replace(tzinfo=UTC) if item.tzinfo is None else item.astimezone(UTC) + for item in values + if isinstance(item, datetime) + ] + return max(timestamps) if timestamps else None + + +def _anomaly_owner_config_ids( + configs: list[FinancialConnectorConfig], +) -> dict[str, str]: + """对账投影只有 provider 维度，异常只归到一个配置版本以避免轮换期重复计数。""" + + owners: dict[str, FinancialConnectorConfig] = {} + priority = {"active": 3, "rotating": 2, "disabled": 1} + for config in configs: + current = owners.get(config.provider) + if current is None or ( + priority.get(config.status, 0), + int(config.version or 0), + config.key_version, + ) > ( + priority.get(current.status, 0), + int(current.version or 0), + current.key_version, + ): + owners[config.provider] = config + return {provider: config.id for provider, config in owners.items()} diff --git a/server/src/app/services/financial_connector_operational_events.py b/server/src/app/services/financial_connector_operational_events.py new file mode 100644 index 0000000..48f6d03 --- /dev/null +++ b/server/src/app/services/financial_connector_operational_events.py @@ -0,0 +1,129 @@ +from __future__ import annotations + +import hashlib +import json +import uuid +from dataclasses import dataclass +from datetime import UTC, datetime + +from sqlalchemy import select +from sqlalchemy.dialects.postgresql import insert as postgresql_insert +from sqlalchemy.dialects.sqlite import insert as sqlite_insert +from sqlalchemy.orm import Session + +from app.models.financial_connector import FinancialConnectorOperationalEvent + +FINANCIAL_CONNECTOR_OPERATIONAL_SOURCE_REVISION = "20260716_0022" + + +@dataclass(frozen=True, slots=True) +class FinancialConnectorOperationalContext: + """认证后可安全归属的最小上下文，只包含不可逆指纹。""" + + tenant_id: str + config_id: str + provider: str + environment: str + request_fingerprint: str + external_event_fingerprint: str + + +@dataclass(frozen=True, slots=True) +class FinancialConnectorOperationalEventCandidate: + context: FinancialConnectorOperationalContext + event_type: str + reason_code: str + occurred_at: datetime + idempotency_key: str + + +def operational_event_candidate( + context: FinancialConnectorOperationalContext, + *, + event_type: str, + reason_code: str, + occurred_at: datetime | None = None, +) -> FinancialConnectorOperationalEventCandidate: + event_kind = str(event_type or "").strip() + reason = str(reason_code or "").strip() + occurrence_time = _aware_utc(occurred_at or datetime.now(UTC)) + payload = json.dumps( + { + "version": FINANCIAL_CONNECTOR_OPERATIONAL_SOURCE_REVISION, + "tenant_id": context.tenant_id, + "config_id": context.config_id, + "event_type": event_kind, + "reason_code": reason, + "request_fingerprint": context.request_fingerprint, + "external_event_fingerprint": context.external_event_fingerprint, + # 同一个 candidate 的补偿重试必须幂等；不同 HTTP 尝试即使载荷 + # 完全相同也必须分别计数，因此把本次发生时间纳入幂等命名空间。 + "occurred_at": occurrence_time.isoformat(timespec="microseconds"), + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + return FinancialConnectorOperationalEventCandidate( + context=context, + event_type=event_kind, + reason_code=reason, + occurred_at=occurrence_time, + idempotency_key=f"sha256:{hashlib.sha256(payload).hexdigest()}", + ) + + +class FinancialConnectorOperationalEventService: + """以数据库幂等唯一约束写入最小化运营事实。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def record(self, candidate: FinancialConnectorOperationalEventCandidate) -> None: + values = { + "id": str(uuid.uuid4()), + "tenant_id": candidate.context.tenant_id, + "config_id": candidate.context.config_id, + "provider": candidate.context.provider, + "environment": candidate.context.environment, + "event_type": candidate.event_type, + "reason_code": candidate.reason_code, + "request_fingerprint": candidate.context.request_fingerprint, + "external_event_fingerprint": candidate.context.external_event_fingerprint, + "idempotency_key": candidate.idempotency_key, + "occurred_at": candidate.occurred_at, + } + bind = self.db.get_bind() + dialect_name = bind.dialect.name if bind is not None else "" + if dialect_name == "postgresql": + statement = postgresql_insert(FinancialConnectorOperationalEvent).values(**values) + self.db.execute( + statement.on_conflict_do_nothing( + index_elements=["tenant_id", "idempotency_key"] + ) + ) + return + if dialect_name == "sqlite": + statement = sqlite_insert(FinancialConnectorOperationalEvent).values(**values) + self.db.execute( + statement.on_conflict_do_nothing( + index_elements=["tenant_id", "idempotency_key"] + ) + ) + return + existing_id = self.db.scalar( + select(FinancialConnectorOperationalEvent.id).where( + FinancialConnectorOperationalEvent.tenant_id + == candidate.context.tenant_id, + FinancialConnectorOperationalEvent.idempotency_key + == candidate.idempotency_key, + ) + ) + if existing_id is None: + self.db.add(FinancialConnectorOperationalEvent(**values)) + + +def _aware_utc(value: datetime) -> datetime: + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value.astimezone(UTC) diff --git a/server/src/app/services/financial_connector_payment_evidence.py b/server/src/app/services/financial_connector_payment_evidence.py new file mode 100644 index 0000000..19d0aa3 --- /dev/null +++ b/server/src/app/services/financial_connector_payment_evidence.py @@ -0,0 +1,116 @@ +from __future__ import annotations + +from datetime import datetime +from typing import Any + +from app.models.financial_record import ExpenseClaim +from app.schemas.financial_connector import FinancialPaymentEvidenceRead + + +class FinancialConnectorPaymentEvidenceService: + """把付款状态映射为明确证据等级，不把内部状态冒充外部现金事实。""" + + @staticmethod + def read(claim: ExpenseClaim) -> FinancialPaymentEvidenceRead: + status = str(claim.status or "").strip().lower() + if status != "paid": + return FinancialPaymentEvidenceRead( + claim_id=claim.id, + claim_status=status, + payment_state="not_paid", + evidence_classification="none", + evidence_label="尚无付款证据", + trust_level="none", + source_type="none", + disclaimer="当前单据尚未形成付款完成事实。", + ) + + flag = _latest_payment_flag(claim.risk_flags_json) + if flag and str(flag.get("source") or "").strip() == "external_payment": + classification = str(flag.get("evidence_classification") or "").strip() + verification = str(flag.get("verification_level") or "").strip() or None + if classification == "external_cash" and verification == "production_verified": + return FinancialPaymentEvidenceRead( + claim_id=claim.id, + claim_status=status, + payment_state="paid", + evidence_classification="external_cash", + evidence_label="生产外部现金回执", + trust_level="high", + source_type="external_connector", + provider=_text(flag.get("provider")), + verification_level=verification, + external_reference_tail=_tail(flag.get("external_reference_tail")), + recorded_at=_datetime(flag.get("created_at")), + disclaimer="该状态由生产连接器签名回执和精确对账共同证明。", + ) + simulated_classification = ( + "staging_connector" + if classification == "staging_connector" + else "simulated_connector" + ) + return FinancialPaymentEvidenceRead( + claim_id=claim.id, + claim_status=status, + payment_state="paid", + evidence_classification=simulated_classification, + evidence_label=( + "预发布连接器回执" + if simulated_classification == "staging_connector" + else "模拟连接器回执" + ), + trust_level="low", + source_type="external_connector", + provider=_text(flag.get("provider")), + verification_level=verification, + external_reference_tail=_tail(flag.get("external_reference_tail")), + recorded_at=_datetime(flag.get("created_at")), + disclaimer="该回执不构成生产现金或入账证明，不应进入核心财务投影。", + ) + + return FinancialPaymentEvidenceRead( + claim_id=claim.id, + claim_status=status, + payment_state="paid", + evidence_classification="internal_manual_payment", + evidence_label="人工付款确认（内部状态）", + trust_level="low", + source_type="manual_confirmation", + recorded_at=_datetime(flag.get("created_at")) if flag else None, + disclaimer="该状态仅证明平台内有人确认已付款，不等同于银行、支付平台或 ERP 外部回执。", + ) + + +def _latest_payment_flag(value: Any) -> dict[str, Any] | None: + if not isinstance(value, list): + return None + for item in reversed(value): + if not isinstance(item, dict): + continue + event_type = str(item.get("event_type") or "").strip() + if event_type in { + "expense_claim_external_payment_settled", + "expense_claim_payment_completed", + }: + return item + return None + + +def _text(value: Any) -> str | None: + normalized = str(value or "").strip() + return normalized or None + + +def _tail(value: Any) -> str | None: + normalized = str(value or "").strip() + return normalized[-8:] or None + + +def _datetime(value: Any) -> datetime | None: + normalized = str(value or "").strip() + if not normalized: + return None + try: + return datetime.fromisoformat(normalized.replace("Z", "+00:00")) + except ValueError: + return None diff --git a/server/src/app/services/financial_connector_projection.py b/server/src/app/services/financial_connector_projection.py new file mode 100644 index 0000000..3dbb1a6 --- /dev/null +++ b/server/src/app/services/financial_connector_projection.py @@ -0,0 +1,214 @@ +from __future__ import annotations + +import hashlib +import json +import uuid +from datetime import UTC, datetime + +from sqlalchemy import func, select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.financial_connector import ( + PaymentReconciliationCase, + PaymentReconciliationEvent, +) +from app.models.financial_record import ExpenseClaim +from app.schemas.financial_connector import ( + PaymentReconciliationActionCreate, + PaymentReconciliationCaseDetailRead, + PaymentReconciliationCaseRead, + PaymentReconciliationEventRead, + PaymentReconciliationListRead, +) + + +class FinancialReconciliationPermissionError(PermissionError): + pass + + +class FinancialReconciliationConflictError(ValueError): + pass + + +class FinancialConnectorProjectionService: + def __init__(self, db: Session) -> None: + self.db = db + + @staticmethod + def require_finance(current_user: CurrentUserContext) -> None: + roles = {str(item).strip().lower() for item in current_user.role_codes} + if not roles & {"finance", "executive"}: + raise FinancialReconciliationPermissionError( + "只有财务业务角色可以访问或处置对账记录。" + ) + + def list_cases( + self, + current_user: CurrentUserContext, + *, + status_filter: str | None, + page: int, + page_size: int, + ) -> PaymentReconciliationListRead: + self.require_finance(current_user) + conditions = [PaymentReconciliationCase.tenant_id == current_user.tenant_id] + if str(status_filter or "").strip(): + conditions.append(PaymentReconciliationCase.status == str(status_filter).strip()) + total = int( + self.db.scalar( + select(func.count(PaymentReconciliationCase.id)).where(*conditions) + ) + or 0 + ) + items = list( + self.db.scalars( + select(PaymentReconciliationCase) + .where(*conditions) + .order_by(PaymentReconciliationCase.updated_at.desc()) + .offset((page - 1) * page_size) + .limit(page_size) + ).all() + ) + return PaymentReconciliationListRead( + items=[PaymentReconciliationCaseRead.model_validate(item) for item in items], + total=total, + page=page, + page_size=page_size, + ) + + def detail( + self, + case_id: str, + current_user: CurrentUserContext, + ) -> PaymentReconciliationCaseDetailRead | None: + self.require_finance(current_user) + row = self.db.scalar( + select(PaymentReconciliationCase).where( + PaymentReconciliationCase.tenant_id == current_user.tenant_id, + PaymentReconciliationCase.id == case_id, + ) + ) + if row is None: + return None + events = list( + self.db.scalars( + select(PaymentReconciliationEvent) + .where( + PaymentReconciliationEvent.tenant_id == current_user.tenant_id, + PaymentReconciliationEvent.reconciliation_case_id == row.id, + ) + .order_by(PaymentReconciliationEvent.occurred_at) + ).all() + ) + base = PaymentReconciliationCaseRead.model_validate(row).model_dump() + return PaymentReconciliationCaseDetailRead( + **base, + timeline=[PaymentReconciliationEventRead.model_validate(item) for item in events], + ) + + def resolve( + self, + case_id: str, + payload: PaymentReconciliationActionCreate, + current_user: CurrentUserContext, + *, + action: str, + ) -> PaymentReconciliationCaseDetailRead | None: + self.require_finance(current_user) + statement = select(PaymentReconciliationCase).where( + PaymentReconciliationCase.tenant_id == current_user.tenant_id, + PaymentReconciliationCase.id == case_id, + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + row = self.db.scalar(statement.execution_options(populate_existing=True)) + if row is None: + return None + if row.version != payload.expected_version: + raise FinancialReconciliationConflictError( + f"对账记录版本已变化，当前版本为 {row.version}。" + ) + if row.status not in {"exception", "reopened"}: + raise FinancialReconciliationConflictError("当前对账状态不允许人工处置。") + claim = self.db.get(ExpenseClaim, row.claim_id) + if claim is None: + raise FinancialReconciliationConflictError("关联报销单不存在。") + if self._is_claim_owner(claim, current_user): + raise FinancialReconciliationPermissionError("申请人不能确认或拒绝自己的对账异常。") + + before = _state(row) + row.status = "confirmed" if action == "confirmed" else "rejected" + row.assigned_to = current_user.username + row.version += 1 + row.updated_at = datetime.now(UTC) + fingerprint = _fingerprint( + case_id=row.id, + action=action, + expected_version=payload.expected_version, + actor_id=current_user.username, + reason=payload.reason, + ) + self.db.add( + PaymentReconciliationEvent( + id=str(uuid.uuid4()), + tenant_id=row.tenant_id, + reconciliation_case_id=row.id, + connector_event_id=row.last_connector_event_id, + action=action, + actor_type="user", + actor_id=current_user.username, + request_fingerprint=fingerprint, + before_json=before, + after_json=_state(row), + response_json={"status": row.status, "version": row.version}, + reason=payload.reason, + correlation_id=f"reconciliation:{row.id}"[:64], + occurred_at=datetime.now(UTC), + ) + ) + self.db.flush() + return self.detail(row.id, current_user) + + @staticmethod + def _is_claim_owner(claim: ExpenseClaim, current_user: CurrentUserContext) -> bool: + if current_user.employee_id and str(claim.employee_id or "") == current_user.employee_id: + return True + return bool( + current_user.name + and str(claim.employee_name or "").strip().casefold() + == str(current_user.name).strip().casefold() + ) + + +def _state(row: PaymentReconciliationCase) -> dict[str, object]: + return { + "status": row.status, + "exception_code": row.exception_code, + "erp_status": row.erp_status, + "version": row.version, + } + + +def _fingerprint( + *, + case_id: str, + action: str, + expected_version: int, + actor_id: str, + reason: str, +) -> str: + raw = json.dumps( + { + "case_id": case_id, + "action": action, + "expected_version": expected_version, + "actor_id": actor_id, + "reason": reason, + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ).encode("utf-8") + return f"sha256:{hashlib.sha256(raw).hexdigest()}" diff --git a/server/src/app/services/financial_connector_simulation.py b/server/src/app/services/financial_connector_simulation.py new file mode 100644 index 0000000..98e3589 --- /dev/null +++ b/server/src/app/services/financial_connector_simulation.py @@ -0,0 +1,104 @@ +from __future__ import annotations + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.financial_connector import FinancialConnectorEvent +from app.models.financial_record import ExpenseClaim +from app.schemas.financial_connector import FinancialEventEnvelope +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.financial_connector_actions import FinancialConnectorActionService +from app.services.financial_connector_auth import VerifiedConnectorRequest +from app.services.payment_reconciliation import ReconciliationResult + + +class FinancialConnectorSimulationService: + """只计算非生产连接器结果，不创建或修改任何核心财务投影。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def project( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + ) -> ReconciliationResult: + if envelope.event_type in {"payment_settled", "payment_failed"}: + return self._payment_projection(envelope) + return self._origin_projection(envelope, verified) + + def _payment_projection(self, envelope: FinancialEventEnvelope) -> ReconciliationResult: + claim = self._claim(envelope.tenant_id, str(envelope.payload.get("claim_id") or "")) + if claim is None: + return self._result(error_code="claim_not_found") + mismatch = FinancialConnectorActionService.payment_payload_mismatch( + claim, + envelope.payload, + ) + if mismatch is None and envelope.event_type == "payment_settled": + return self._result(claim=claim) + return self._result( + claim=claim, + error_code=mismatch or "external_payment_failed", + ) + + def _origin_projection( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + ) -> ReconciliationResult: + origin = self.db.scalar( + select(FinancialConnectorEvent).where( + FinancialConnectorEvent.tenant_id == envelope.tenant_id, + FinancialConnectorEvent.provider == verified.config.provider, + FinancialConnectorEvent.environment == verified.config.environment, + FinancialConnectorEvent.verification_level == verified.verification_level, + FinancialConnectorEvent.external_event_id + == str(envelope.payload.get("origin_external_event_id") or ""), + ) + ) + if origin is None or not origin.claim_id: + return self._result(error_code="origin_settlement_not_found") + if origin.event_type != "payment_settled" or origin.processing_status != "processed": + return self._result(error_code="origin_settlement_not_processed") + if str(envelope.payload.get("claim_id") or "") != str(origin.claim_id): + return self._result(error_code="origin_claim_mismatch") + claim = self._claim(envelope.tenant_id, origin.claim_id) + if claim is None: + return self._result(error_code="claim_not_found") + mismatch = FinancialConnectorActionService.payment_payload_mismatch( + claim, + envelope.payload, + require_external_reference=envelope.event_type + in {"payment_refunded", "payment_reversed"}, + ) + if mismatch: + prefix = "erp_" if envelope.event_type.startswith("erp_") else "" + return self._result(claim=claim, error_code=f"{prefix}{mismatch}") + if envelope.event_type == "erp_posting_failed": + return self._result(claim=claim, error_code="erp_posting_failed") + return self._result(claim=claim) + + def _claim(self, tenant_id: str, claim_id: str) -> ExpenseClaim | None: + return self.db.scalar( + select(ExpenseClaim).where( + ExpenseClaim.id == str(claim_id).strip(), + ExpenseClaimTenantScopeMixin.build_claim_tenant_condition(tenant_id), + ) + ) + + @staticmethod + def _result( + *, + claim: ExpenseClaim | None = None, + error_code: str | None = None, + ) -> ReconciliationResult: + return ReconciliationResult( + case=None, + audit_event=None, + processing_status="exception" if error_code else "processed", + error_code=error_code, + claim_id=claim.id if claim is not None else None, + expense_case_id=None, + claim_status=str(claim.status or "") if claim is not None else None, + ) diff --git a/server/src/app/services/hermes_employee_profile_scanner.py b/server/src/app/services/hermes_employee_profile_scanner.py index b0679e6..4bdc307 100644 --- a/server/src/app/services/hermes_employee_profile_scanner.py +++ b/server/src/app/services/hermes_employee_profile_scanner.py @@ -1,14 +1,14 @@ from __future__ import annotations from sqlalchemy import select -from sqlalchemy.orm import Session -from sqlalchemy.orm import selectinload +from sqlalchemy.orm import Session, selectinload -from app.core.logging import get_logger from app.algorithem.risk_graph.models import RiskGraphClaimSnapshot from app.algorithem.risk_graph.profile_baselines import ProfileBaselineUpdater +from app.core.logging import get_logger from app.models.financial_record import ExpenseClaim from app.services.employee_behavior_profile_service import EmployeeBehaviorProfileService +from app.services.finance_report_tenant import require_report_tenant_id logger = get_logger("app.services.hermes_employee_profile_scanner") @@ -17,10 +17,19 @@ class HermesEmployeeProfileScannerService: def __init__(self, db: Session) -> None: self.db = db - def scan_employee_profiles(self, log_id: str | None = None) -> dict: + def scan_employee_profiles( + self, + log_id: str | None = None, + *, + tenant_id: str = "default", + ) -> dict: + tenant = require_report_tenant_id(tenant_id) logger.info("Starting Hermes employee behavior profile scan...") - summary = EmployeeBehaviorProfileService(self.db).scan_profiles(log_id=log_id) - baseline_summary = self._build_baseline_summary() + summary = EmployeeBehaviorProfileService( + self.db, + tenant_id=tenant, + ).scan_profiles(log_id=log_id) + baseline_summary = self._build_baseline_summary(tenant_id=tenant) summary["baseline_summary"] = baseline_summary logger.info( "Hermes employee profile scan completed: %s", @@ -41,17 +50,15 @@ class HermesEmployeeProfileScannerService: "baseline_bucket_count": len(buckets) if isinstance(buckets, list) else 0, } - def _build_baseline_summary(self) -> dict: + def _build_baseline_summary(self, *, tenant_id: str) -> dict: stmt = ( select(ExpenseClaim) .options(selectinload(ExpenseClaim.items)) + .where(ExpenseClaim.tenant_id == tenant_id) .order_by(ExpenseClaim.occurred_at.desc()) .limit(500) ) - claims = [ - RiskGraphClaimSnapshot.from_orm(claim) - for claim in self.db.scalars(stmt).all() - ] + claims = [RiskGraphClaimSnapshot.from_orm(claim) for claim in self.db.scalars(stmt).all()] return ProfileBaselineUpdater().build_from_claims(claims).as_dict() @staticmethod diff --git a/server/src/app/services/hermes_expense_report.py b/server/src/app/services/hermes_expense_report.py index 1b2d0f3..616fbc6 100644 --- a/server/src/app/services/hermes_expense_report.py +++ b/server/src/app/services/hermes_expense_report.py @@ -1,104 +1,28 @@ from __future__ import annotations -import json -from datetime import datetime, timedelta, timezone from typing import Any -from sqlalchemy import func, select from sqlalchemy.orm import Session -from app.core.logging import get_logger -from app.models.financial_record import ExpenseClaim -from app.services.runtime_chat import RuntimeChatService - -logger = get_logger("app.services.hermes_expense_report") +from app.services.digital_employee_finance_report_task import ( + DigitalEmployeeFinanceReportTaskService, +) class HermesExpenseReportService: + """旧任务名的安全入口，统一复用租户化 PDF/邮件报告链。""" + def __init__(self, db: Session) -> None: self.db = db - self.chat_service = RuntimeChatService(db) - def generate_weekly_report(self, log_id: str | None = None) -> None: - logger.info("Starting Hermes weekly expense report generation...") - - # 1. 聚合数据 - aggregated_data = self._aggregate_recent_expenses(days=7) - if not aggregated_data.get("total_amount"): - logger.info("No expense data in the last 7 days. Skipping report.") - return - - # 2. 传入大模型分析 - report_markdown = self._generate_insights_with_llm(aggregated_data) - - if not report_markdown: - logger.warning("Failed to generate expense report from LLM.") - return - - # 3. 模拟发送报告 - self._deliver_report(report_markdown, log_id) - logger.info("Hermes weekly expense report generation completed.") - - def _aggregate_recent_expenses(self, days: int = 7) -> dict[str, Any]: - target_date = datetime.now(timezone.utc) - timedelta(days=days) - - # 基础过滤：最近N天且不是驳回状态的单据 - base_filter = [ - ExpenseClaim.occurred_at >= target_date, - ExpenseClaim.status != "rejected" - ] - - # 1. 按部门汇总 - dept_stmt = select( - ExpenseClaim.department_name, - func.sum(ExpenseClaim.amount).label("total") - ).where(*base_filter).group_by(ExpenseClaim.department_name) - - dept_results = self.db.execute(dept_stmt).all() - by_department = {row.department_name or "Unknown": float(row.total or 0) for row in dept_results} - - # 2. 按类目汇总 - type_stmt = select( - ExpenseClaim.expense_type, - func.sum(ExpenseClaim.amount).label("total") - ).where(*base_filter).group_by(ExpenseClaim.expense_type) - - type_results = self.db.execute(type_stmt).all() - by_expense_type = {row.expense_type or "Unknown": float(row.total or 0) for row in type_results} - - # 3. 总花费 - total_amount = sum(by_department.values()) - - return { - "period": f"Last {days} days", - "total_amount": total_amount, - "by_department": by_department, - "by_expense_type": by_expense_type - } - - def _generate_insights_with_llm(self, data: dict[str, Any]) -> str | None: - system_prompt = ( - "你是公司的财务分析专家。请根据提供的最近期业务开销数据，撰写一份简洁有力的【高管费控洞察周报】。\n" - "要求：\n" - "1. 不要机械地罗列数字，要像人一样指出异常（例如：哪个部门花钱最多？打车费是不是异常高？）。\n" - "2. 给出 1 条削减成本的实操建议。\n" - "3. 纯 Markdown 格式输出，不超过 300 字。" + def generate_weekly_report( + self, + log_id: str | None = None, + *, + tenant_id: str = "default", + ) -> dict[str, Any]: + del log_id + return DigitalEmployeeFinanceReportTaskService(self.db).generate_report( + report_type="weekly", + tenant_id=tenant_id, ) - - messages = [ - {"role": "system", "content": system_prompt}, - {"role": "user", "content": f"开销统计数据：\n{json.dumps(data, ensure_ascii=False, indent=2)}"} - ] - - response = self.chat_service.complete( - messages, - max_tokens=800, - temperature=0.4 - ) - return response - - def _deliver_report(self, report_markdown: str, log_id: str | None) -> None: - # TODO: 未来在这里接入企微/钉钉机器人或邮件发送接口 - logger.info(f"\n================ Hermes Weekly Report [LogID: {log_id}] ================\n" - f"{report_markdown}\n" - f"==========================================================================") diff --git a/server/src/app/services/hermes_risk_clue_collector.py b/server/src/app/services/hermes_risk_clue_collector.py index cfed88a..8b12ac6 100644 --- a/server/src/app/services/hermes_risk_clue_collector.py +++ b/server/src/app/services/hermes_risk_clue_collector.py @@ -11,6 +11,7 @@ from sqlalchemy.orm import Session, selectinload from app.models.financial_record import ExpenseClaim from app.models.risk_observation import RiskObservation, RiskObservationFeedback from app.services.document_numbering import is_application_claim_no +from app.services.finance_report_tenant import require_report_tenant_id from app.services.risk_observations import RiskObservationService @@ -25,12 +26,17 @@ class HermesRiskClueCollectorService: *, run_id: str | None = None, limit: int = 100, + tenant_id: str = "default", ) -> dict[str, Any]: + tenant = require_report_tenant_id(tenant_id) RiskObservationService(self.db).ensure_storage_ready() safe_limit = max(1, min(int(limit or 100), 200)) - claims = self._fetch_recent_claims(safe_limit) - observations = self._fetch_recent_observations(safe_limit * 2) - feedback_items = self._fetch_recent_feedback(safe_limit) + claims = self._fetch_recent_claims(safe_limit, tenant_id=tenant) + observations = self._fetch_recent_observations( + safe_limit * 2, + tenant_id=tenant, + ) + feedback_items = self._fetch_recent_feedback(safe_limit, tenant_id=tenant) facts = [self._claim_fact(claim) for claim in claims] claim_rule_hits = self._claim_rule_hits(claims) @@ -55,6 +61,7 @@ class HermesRiskClueCollectorService: "task_type": "risk_clue_collect", "output_format": "risk_clue_review_packet", "run_id": run_id, + "tenant_id": tenant, "fact_count": len(facts), "rule_hit_count": len(rule_hits), "risk_clue_count": len(risk_clues), @@ -80,23 +87,49 @@ class HermesRiskClueCollectorService: "generated_at": datetime.now(UTC).isoformat(), } - def _fetch_recent_claims(self, limit: int) -> list[ExpenseClaim]: - stmt = select(ExpenseClaim).order_by(ExpenseClaim.created_at.desc()).limit(limit) + def _fetch_recent_claims( + self, + limit: int, + *, + tenant_id: str, + ) -> list[ExpenseClaim]: + stmt = ( + select(ExpenseClaim) + .where(ExpenseClaim.tenant_id == tenant_id) + .order_by(ExpenseClaim.created_at.desc()) + .limit(limit) + ) return list(self.db.scalars(stmt).all()) - def _fetch_recent_observations(self, limit: int) -> list[RiskObservation]: + def _fetch_recent_observations( + self, + limit: int, + *, + tenant_id: str, + ) -> list[RiskObservation]: stmt = ( select(RiskObservation) .options(selectinload(RiskObservation.feedback_items)) + .where(RiskObservation.tenant_id == tenant_id) .order_by(RiskObservation.risk_score.desc(), RiskObservation.created_at.desc()) .limit(limit) ) return list(self.db.scalars(stmt).all()) - def _fetch_recent_feedback(self, limit: int) -> list[RiskObservationFeedback]: + def _fetch_recent_feedback( + self, + limit: int, + *, + tenant_id: str, + ) -> list[RiskObservationFeedback]: stmt = ( select(RiskObservationFeedback) + .join( + RiskObservation, + RiskObservation.id == RiskObservationFeedback.observation_id, + ) .options(selectinload(RiskObservationFeedback.observation)) + .where(RiskObservation.tenant_id == tenant_id) .order_by(RiskObservationFeedback.created_at.desc()) .limit(limit) ) @@ -108,7 +141,9 @@ class HermesRiskClueCollectorService: "source": "expense_claims", "claim_id": claim.id, "claim_no": claim.claim_no, - "claim_kind": "application" if is_application_claim_no(claim.claim_no) else "reimbursement", + "claim_kind": ( + "application" if is_application_claim_no(claim.claim_no) else "reimbursement" + ), "employee_name": claim.employee_name, "department_name": claim.department_name, "expense_type": claim.expense_type, @@ -146,7 +181,9 @@ class HermesRiskClueCollectorService: "claim_id": claim.id, "claim_no": claim.claim_no, "title": _text(flag.get("label") or flag.get("title")) or signal, - "message": _text(flag.get("message") or flag.get("summary") or flag.get("reason")), + "message": _text( + flag.get("message") or flag.get("summary") or flag.get("reason") + ), "severity": _text(flag.get("severity") or flag.get("risk_level")), "metadata": flag, } @@ -191,9 +228,18 @@ class HermesRiskClueCollectorService: continue refs.append( { - "evidence_id": f"evidence:observation:{observation.observation_key}:{index}", - "source": _text(evidence.get("source")) or observation.source or "risk_observation", - "title": _text(evidence.get("title") or evidence.get("code")) or observation.title, + "evidence_id": ( + f"evidence:observation:{observation.observation_key}:{index}" + ), + "source": ( + _text(evidence.get("source")) + or observation.source + or "risk_observation" + ), + "title": ( + _text(evidence.get("title") or evidence.get("code")) + or observation.title + ), "detail": _text( evidence.get("detail") or evidence.get("message") @@ -271,9 +317,7 @@ class HermesRiskClueCollectorService: or f"{observation.claim_no or observation.subject_label} 存在待复核线索。", "confidence_score": confidence, "evidence_refs": evidence_ids, - "rule_hits": [ - f"rule_hit:observation:{observation.observation_key}" - ] + "rule_hits": [f"rule_hit:observation:{observation.observation_key}"] if _is_rule_hit_observation(observation) else [], "fact_refs": [f"fact:claim:{observation.claim_id}"] if observation.claim_id else [], diff --git a/server/src/app/services/hermes_risk_scanner.py b/server/src/app/services/hermes_risk_scanner.py index 0c263a4..8186bb3 100644 --- a/server/src/app/services/hermes_risk_scanner.py +++ b/server/src/app/services/hermes_risk_scanner.py @@ -1,6 +1,6 @@ from __future__ import annotations -from datetime import datetime, timezone +from datetime import UTC, datetime from sqlalchemy import or_, select from sqlalchemy.orm import Session, selectinload @@ -14,7 +14,7 @@ from app.core.logging import get_logger from app.models.financial_record import ExpenseClaim from app.models.hermes_report import HermesRiskReport from app.services.expense_claim_risk_stage import with_risk_business_stage -from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.finance_report_tenant import require_report_tenant_id from app.services.risk_observations import RiskObservationService logger = get_logger("app.services.hermes_risk_scanner") @@ -28,10 +28,12 @@ class HermesRiskScannerService: self, log_id: str | None = None, run_id: str | None = None, + tenant_id: str = "default", ) -> dict[str, int]: + tenant = require_report_tenant_id(tenant_id) logger.info("Starting global risk scan for Hermes...") - claims = self._fetch_unscanned_claims() + claims = self._fetch_unscanned_claims(tenant_id=tenant) if not claims: logger.info("No unscanned claims found. Aborting scan.") return {"scanned_claim_count": 0, "risk_observation_count": 0} @@ -43,7 +45,7 @@ class HermesRiskScannerService: scanned_claim_count = 0 graph_node_count = 0 graph_edge_count = 0 - now = datetime.now(timezone.utc) + now = datetime.now(UTC) grouped_claims = self._group_claims_by_tenant(claims) for tenant_id in sorted(grouped_claims): tenant_claims = sorted(grouped_claims[tenant_id], key=lambda item: str(item.id)) @@ -93,6 +95,7 @@ class HermesRiskScannerService: if log_id: self.db.add( HermesRiskReport( + tenant_id=tenant_id, claim_id=observation.claim_id, execution_log_id=log_id, risk_level=observation.risk_level, @@ -119,24 +122,22 @@ class HermesRiskScannerService: "graph_edge_count": graph_edge_count, } + @staticmethod def _group_claims_by_tenant( - self, claims: list[ExpenseClaim], ) -> dict[str, list[ExpenseClaim]]: grouped: dict[str, list[ExpenseClaim]] = {} for claim in claims: - tenant_id = ExpenseClaimTenantScopeMixin.resolve_claim_tenant_id( - self.db, - claim.id, - ) + tenant_id = require_report_tenant_id(claim.tenant_id) grouped.setdefault(tenant_id, []).append(claim) return grouped - def _fetch_unscanned_claims(self) -> list[ExpenseClaim]: + def _fetch_unscanned_claims(self, *, tenant_id: str) -> list[ExpenseClaim]: stmt = ( select(ExpenseClaim) .options(selectinload(ExpenseClaim.items)) .where( + ExpenseClaim.tenant_id == tenant_id, ExpenseClaim.status.in_(["draft", "submitted", "review"]), or_( ExpenseClaim.hermes_scanned_at.is_(None), diff --git a/server/src/app/services/hermes_scheduler.py b/server/src/app/services/hermes_scheduler.py index b36e627..046c8cf 100644 --- a/server/src/app/services/hermes_scheduler.py +++ b/server/src/app/services/hermes_scheduler.py @@ -8,8 +8,10 @@ from sqlalchemy.orm import Session from app.core.logging import get_logger from app.db.session import get_session_factory from app.models.hermes_config import HermesTaskConfig, HermesTaskExecutionLog +from app.services.digital_employee_finance_report_task import ( + DigitalEmployeeFinanceReportTaskService, +) from app.services.hermes_employee_profile_scanner import HermesEmployeeProfileScannerService -from app.services.hermes_expense_report import HermesExpenseReportService from app.services.hermes_risk_clue_collector import HermesRiskClueCollectorService from app.services.hermes_risk_scanner import HermesRiskScannerService @@ -77,6 +79,7 @@ class HermesScheduler: select(HermesTaskExecutionLog) .where( HermesTaskExecutionLog.config_id == config.id, + HermesTaskExecutionLog.tenant_id == config.tenant_id, HermesTaskExecutionLog.status.in_(["success", "running"]), ) .order_by(HermesTaskExecutionLog.started_at.desc()) @@ -145,7 +148,11 @@ class HermesScheduler: logger.info(f"Triggering Hermes task: {config.task_type} (Config ID: {config.id})") # 创建执行日志，标记为 running - log_record = HermesTaskExecutionLog(config_id=config.id, status="running") + log_record = HermesTaskExecutionLog( + tenant_id=config.tenant_id, + config_id=config.id, + status="running", + ) db.add(log_record) db.commit() db.refresh(log_record) @@ -153,17 +160,29 @@ class HermesScheduler: try: if config.task_type == "global_risk_scan": scanner = HermesRiskScannerService(db) - summary = scanner.scan_global_risks(log_id=log_record.id) + summary = scanner.scan_global_risks( + log_id=log_record.id, + tenant_id=config.tenant_id, + ) log_record.result_summary = ( f"风险图谱巡检完成：扫描 {summary.get('scanned_claim_count', 0)} 张单据，" f"生成 {summary.get('risk_observation_count', 0)} 条风险观察。" ) elif config.task_type == "weekly_expense_report": - reporter = HermesExpenseReportService(db) - reporter.generate_weekly_report(log_id=log_record.id) + result = DigitalEmployeeFinanceReportTaskService(db).generate_report( + report_type="weekly", + tenant_id=config.tenant_id, + ) + log_record.result_summary = ( + f"财务经营周报完成：邮件状态 " + f"{(result.get('delivery') or {}).get('status', 'skipped')}。" + ) elif config.task_type == "employee_behavior_profile_scan": scanner = HermesEmployeeProfileScannerService(db) - summary = scanner.scan_employee_profiles(log_id=log_record.id) + summary = scanner.scan_employee_profiles( + log_id=log_record.id, + tenant_id=config.tenant_id, + ) log_record.result_summary = ( f"员工画像巡检完成：目标 {summary.get('target_employee_count', 0)} 人，" f"生成 {summary.get('snapshot_count', 0)} 条快照，" @@ -171,7 +190,10 @@ class HermesScheduler: ) elif config.task_type == "risk_clue_collect": collector = HermesRiskClueCollectorService(db) - summary = collector.collect_risk_clues(run_id=log_record.id) + summary = collector.collect_risk_clues( + run_id=log_record.id, + tenant_id=config.tenant_id, + ) log_record.result_summary = ( f"风险线索归集完成：读取 {summary.get('fact_count', 0)} 条事实，" f"整理 {summary.get('rule_hit_count', 0)} 条规则命中，" diff --git a/server/src/app/services/knowledge.py b/server/src/app/services/knowledge.py index d4dc797..57750ce 100644 --- a/server/src/app/services/knowledge.py +++ b/server/src/app/services/knowledge.py @@ -1,18 +1,15 @@ from __future__ import annotations import hashlib -import json import mimetypes from datetime import UTC, datetime from pathlib import Path from typing import Any -from urllib.request import Request, urlopen from uuid import uuid4 from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext -from app.core.agent_enums import AgentRunStatus from app.core.config import get_settings from app.core.logging import get_logger from app.schemas.knowledge import ( @@ -23,23 +20,23 @@ from app.schemas.knowledge import ( KnowledgeOnlyOfficeConfigRead, KnowledgePreviewPageRead, ) -from app.services.knowledge_rag import KnowledgeRagService - -logger = get_logger("app.services.knowledge") - from app.services.knowledge_constants import ( FIXED_KNOWLEDGE_FOLDERS, ICON_BY_TYPE, - KNOWLEDGE_INGEST_STATUS_FAILED, KNOWLEDGE_INGEST_STATUS_INGESTED, KNOWLEDGE_INGEST_STATUS_META, KNOWLEDGE_INGEST_STATUS_PUBLISHED, - KNOWLEDGE_INGEST_STATUS_SYNCING, KNOWLEDGE_SEARCH_RESULT_LIMIT, ) +from app.services.knowledge_constants import ( + KNOWLEDGE_INGEST_STATUS_FAILED as KNOWLEDGE_INGEST_STATUS_FAILED, +) +from app.services.knowledge_constants import ( + KNOWLEDGE_INGEST_STATUS_SYNCING as KNOWLEDGE_INGEST_STATUS_SYNCING, +) from app.services.knowledge_document_extractors import ( - _extract_docx_text, _extract_document_text_from_path, + _extract_docx_text, _extract_pdf_text, _extract_pptx_slides, _extract_text_with_ocr, @@ -58,37 +55,60 @@ from app.services.knowledge_file_utils import ( resolve_file_type, resolve_file_type_label, ) -from app.services.knowledge_onlyoffice import ( - OnlyOfficeCallbackPayload, - build_onlyoffice_config as build_onlyoffice_config_payload, - build_onlyoffice_access_token, - build_onlyoffice_document_key, - parse_onlyoffice_callback, - resolve_onlyoffice_document_type, - validate_onlyoffice_access_token, -) +from app.services.knowledge_index_state import KnowledgeIndexStateMixin from app.services.knowledge_ingest_status import ( - is_syncing_status_stale, normalize_ingest_status_code, - resolve_linked_ingest_run_status, - should_preserve_syncing_status, +) +from app.services.knowledge_onlyoffice import ( + build_onlyoffice_config as build_onlyoffice_config_payload, +) +from app.services.knowledge_onlyoffice import ( + build_onlyoffice_document_key, + resolve_onlyoffice_document_type, ) from app.services.knowledge_preview import build_preview +from app.services.knowledge_rag import KnowledgeRagService +from app.services.knowledge_tenant_scope import ( + PLATFORM_KNOWLEDGE_SCOPE, + TENANT_KNOWLEDGE_SCOPE, + KnowledgeStorageScope, + migrate_legacy_library_to_platform, + require_knowledge_tenant_id, +) + +logger = get_logger("app.services.knowledge") def prepare_knowledge_library() -> None: - KnowledgeService().ensure_library_ready() + KnowledgeService(scope=PLATFORM_KNOWLEDGE_SCOPE).ensure_library_ready() -class KnowledgeService: - def __init__(self, storage_root: Path | None = None, db: Session | None = None) -> None: +class KnowledgeService(KnowledgeIndexStateMixin): + def __init__( + self, + storage_root: Path | None = None, + db: Session | None = None, + *, + tenant_id: str | None = None, + scope: str | None = None, + ) -> None: settings = get_settings() self.db = db self.storage_root = Path(storage_root or settings.resolved_storage_root_dir) - self.library_root = self.storage_root / "knowledge" - self.index_path = self.library_root / ".index.json" + if scope == PLATFORM_KNOWLEDGE_SCOPE: + self.storage_scope = KnowledgeStorageScope.platform(self.storage_root) + elif scope in {None, TENANT_KNOWLEDGE_SCOPE} and tenant_id is not None: + self.storage_scope = KnowledgeStorageScope.tenant(self.storage_root, tenant_id) + else: + raise ValueError("知识库操作必须显式提供可信 tenant_id 或 platform scope。") + self.scope = self.storage_scope.scope + self.tenant_id = self.storage_scope.tenant_id + self.library_root = self.storage_scope.library_root + self.index_path = self.storage_scope.index_path def ensure_library_ready(self) -> None: + if self.scope == PLATFORM_KNOWLEDGE_SCOPE: + migrate_legacy_library_to_platform(self.storage_scope) self.library_root.mkdir(parents=True, exist_ok=True) for folder_name in FIXED_KNOWLEDGE_FOLDERS: (self.library_root / folder_name).mkdir(parents=True, exist_ok=True) @@ -102,6 +122,11 @@ class KnowledgeService: def list_library(self) -> KnowledgeLibraryRead: documents = self._load_documents() + if self.scope == TENANT_KNOWLEDGE_SCOPE: + platform_documents = self._platform_service()._load_documents() + tenant_ids = {item.id for item in documents} + documents.extend(item for item in platform_documents if item.id not in tenant_ids) + documents.sort(key=lambda item: item.time, reverse=True) folders = [ KnowledgeFolderRead( name=folder_name, @@ -113,6 +138,9 @@ class KnowledgeService: return KnowledgeLibraryRead(folders=folders, documents=documents) def get_document_detail(self, document_id: str) -> KnowledgeDocumentDetailRead: + owner = self._document_owner(document_id) + if owner is not self: + return owner.get_document_detail(document_id) self.ensure_library_ready() index = self._load_index() if self._reconcile_document_ingest_statuses(index, document_ids=[document_id]): @@ -133,6 +161,10 @@ class KnowledgeService: content: bytes, current_user: CurrentUserContext, ) -> KnowledgeDocumentDetailRead: + if self.storage_scope.read_only: + raise ValueError("平台知识文档为只读资源，不能上传或覆盖。") + if require_knowledge_tenant_id(current_user.tenant_id) != self.tenant_id: + raise ValueError("当前用户不能写入其他租户的知识库。") self.ensure_library_ready() normalized_folder = self._normalize_folder(folder) normalized_name = self._normalize_filename(filename) @@ -140,7 +172,12 @@ class KnowledgeService: if not content: raise ValueError("上传文件不能为空。") - rag_service = KnowledgeRagService(db=self.db, storage_root=self.storage_root) + rag_service = KnowledgeRagService( + db=self.db, + storage_root=self.storage_root, + tenant_id=self.tenant_id, + scope=self.scope, + ) index = self._load_index() existing_entry = next( ( @@ -235,6 +272,8 @@ class KnowledgeService: return self.get_document_detail(document_id) def delete_document(self, document_id: str) -> None: + if self.storage_scope.read_only: + raise ValueError("平台知识文档为只读资源，不能删除。") self.ensure_library_ready() index = self._load_index() entry = self._require_entry(index, document_id) @@ -244,12 +283,20 @@ class KnowledgeService: index["documents"] = [item for item in index["documents"] if item["id"] != document_id] self._save_index(index) - KnowledgeRagService(db=self.db, storage_root=self.storage_root).delete_document(document_id) + KnowledgeRagService( + db=self.db, + storage_root=self.storage_root, + tenant_id=self.tenant_id, + scope=self.scope, + ).delete_document(document_id) logger.info( "Knowledge document deleted id=%s filename=%s", document_id, entry["original_name"] ) def get_document_content(self, document_id: str) -> tuple[Path, str, str]: + owner = self._document_owner(document_id) + if owner is not self: + return owner.get_document_content(document_id) self.ensure_library_ready() index = self._load_index() entry = self._require_entry(index, document_id) @@ -349,11 +396,25 @@ class KnowledgeService: limit: int = KNOWLEDGE_SEARCH_RESULT_LIMIT, ) -> dict[str, Any]: self.ensure_library_ready() - return KnowledgeRagService(db=self.db, storage_root=self.storage_root).query_knowledge( + tenant_result = self._rag_service().query_knowledge( query, conversation_history=conversation_history, limit=limit, ) + if self.scope == PLATFORM_KNOWLEDGE_SCOPE: + return tenant_result + platform_service = self._platform_service() + platform_service.ensure_library_ready() + platform_result = platform_service._rag_service().query_knowledge( + query, + conversation_history=conversation_history, + limit=limit, + ) + return self._merge_scoped_search_results( + tenant_result, + platform_result, + limit=limit, + ) def extract_document_text(self, document_id: str) -> str: self.ensure_library_ready() @@ -371,39 +432,25 @@ class KnowledgeService: self, document_id: str, current_user: CurrentUserContext, + *, + editable: bool = False, ) -> KnowledgeOnlyOfficeConfigRead: - self.ensure_library_ready() - index = self._load_index() - entry = self._require_entry(index, document_id) + if self.db is None: + raise ValueError("ONLYOFFICE 安全会话需要数据库连接。") + owner = self._document_owner(document_id) + owner.ensure_library_ready() + index = owner._load_index() + entry = owner._require_entry(index, document_id) return build_onlyoffice_config_payload( + db=self.db, document_id=document_id, entry=entry, current_user=current_user, + resource_scope=owner.scope, + tenant_id=owner.tenant_id, + editable=editable, ) - def validate_onlyoffice_access_token(self, document_id: str, access_token: str) -> None: - validate_onlyoffice_access_token(document_id, access_token) - - def handle_onlyoffice_callback(self, document_id: str, payload: dict[str, Any]) -> None: - self.ensure_library_ready() - callback = self._parse_onlyoffice_callback(payload) - if callback.status not in {2, 6} or not callback.download_url: - return - - logger.info( - "ONLYOFFICE callback received id=%s status=%s users=%s", - document_id, - callback.status, - ",".join(callback.users) if callback.users else "-", - ) - - request = Request(callback.download_url, headers={"User-Agent": "x-financial-onlyoffice"}) - with urlopen(request, timeout=30) as response: # noqa: S310 - content = response.read() - - actor_name = callback.users[0] if callback.users else "ONLYOFFICE" - self._replace_document_content(document_id, content, actor_name=actor_name) - def _load_documents(self) -> list[KnowledgeDocumentRead]: self.ensure_library_ready() index = self._load_index() @@ -432,6 +479,8 @@ class KnowledgeService: return KnowledgeDocumentRead( id=entry["id"], + scope=self.scope, + readOnly=self.storage_scope.read_only, name=entry["original_name"], folder=entry["folder"], tag=f"{entry['folder']} / {extension.upper() or 'FILE'}", @@ -445,7 +494,10 @@ class KnowledgeService: icon=ICON_BY_TYPE.get(file_type, ICON_BY_TYPE["binary"]), fileType=file_type, fileTypeLabel=self._resolve_file_type_label(file_type), - summary=f"{entry['folder']} ? {extension.upper() or 'FILE'} ? {self._format_size(size_bytes)}", + summary=( + f"{entry['folder']} ? {extension.upper() or 'FILE'} ? " + f"{self._format_size(size_bytes)}" + ), mimeType=entry.get("mime_type") or "application/octet-stream", extension=extension, sizeBytes=size_bytes, @@ -458,281 +510,89 @@ class KnowledgeService: def _build_preview(self, entry: dict[str, Any]) -> tuple[str, list[KnowledgePreviewPageRead]]: return build_preview(entry, resolve_document_path=self._resolve_document_path) - def _load_index(self) -> dict[str, Any]: - try: - payload = json.loads(self.index_path.read_text(encoding="utf-8")) - except (FileNotFoundError, json.JSONDecodeError): - payload = {"version": 1, "documents": []} - payload.setdefault("documents", []) - return payload - - def _save_index(self, index: dict[str, Any]) -> None: - self.index_path.write_text( - json.dumps(index, ensure_ascii=False, indent=2), - encoding="utf-8", + def _rag_service(self) -> KnowledgeRagService: + return KnowledgeRagService( + db=self.db, + storage_root=self.storage_root, + tenant_id=self.tenant_id, + scope=self.scope, ) - def _reconcile_index(self, index: dict[str, Any]) -> bool: - changed = False - documents = index.setdefault("documents", []) - known_by_stored = { - (item["folder"], item["stored_name"]): item - for item in documents - if item.get("folder") and item.get("stored_name") - } + def _platform_service(self) -> KnowledgeService: + return KnowledgeService( + storage_root=self.storage_root, + db=self.db, + scope=PLATFORM_KNOWLEDGE_SCOPE, + ) - existing_items: list[dict[str, Any]] = [] - for item in documents: - file_path = self._resolve_document_path(item) - if file_path.exists(): - item["size_bytes"] = file_path.stat().st_size - item["extension"] = self._extract_extension(item["original_name"]) - item["mime_type"] = item.get("mime_type") or ( - mimetypes.guess_type(item["original_name"])[0] or "application/octet-stream" - ) - normalized_status = normalize_ingest_status_code(item.get("ingest_status")) - if item.get("ingest_status") != normalized_status: - item["ingest_status"] = normalized_status - changed = True - if "ingest_agent_run_id" not in item: - item["ingest_agent_run_id"] = "" - changed = True - if "ingest_status_updated_at" not in item: - item["ingest_status_updated_at"] = ( - item.get("updated_at") or item.get("created_at") or "" - ) - changed = True - if "ingest_completed_at" not in item: - item["ingest_completed_at"] = "" - changed = True - if "ingest_document_name" not in item: - item["ingest_document_name"] = "" - changed = True - if "ingest_document_updated_at" not in item: - item["ingest_document_updated_at"] = "" - changed = True - if "ingest_document_sha256" not in item: - item["ingest_document_sha256"] = "" - changed = True - existing_items.append(item) - else: - changed = True + def _document_owner(self, document_id: str) -> KnowledgeService: + self.ensure_library_ready() + index = self._load_index() + try: + self._require_entry(index, document_id) + return self + except FileNotFoundError: + if self.scope == PLATFORM_KNOWLEDGE_SCOPE: + raise + platform_service = self._platform_service() + platform_service.ensure_library_ready() + platform_service._require_entry(platform_service._load_index(), document_id) + return platform_service - for folder_name in FIXED_KNOWLEDGE_FOLDERS: - folder_path = self.library_root / folder_name - for file_path in folder_path.iterdir(): - if not file_path.is_file() or file_path.name.startswith("."): + @staticmethod + def _merge_scoped_search_results( + tenant_result: dict[str, Any], + platform_result: dict[str, Any], + *, + limit: int, + ) -> dict[str, Any]: + merged_hits: list[dict[str, Any]] = [] + seen_codes: set[str] = set() + for scope, payload in ( + (TENANT_KNOWLEDGE_SCOPE, tenant_result), + (PLATFORM_KNOWLEDGE_SCOPE, platform_result), + ): + for raw_hit in list(payload.get("hits") or []): + if not isinstance(raw_hit, dict): continue - - key = (folder_name, file_path.name) - if key in known_by_stored: + code = str(raw_hit.get("code") or "").strip() + if code and code in seen_codes: continue - - document_id, original_name = self._parse_stored_name(file_path.name) - stat = file_path.stat() - existing_items.append( + if code: + seen_codes.add(code) + merged_hits.append( { - "id": document_id, - "folder": folder_name, - "original_name": original_name, - "stored_name": file_path.name, - "mime_type": mimetypes.guess_type(original_name)[0] - or "application/octet-stream", - "extension": self._extract_extension(original_name), - "size_bytes": stat.st_size, - "sha256": "", - "created_at": datetime.fromtimestamp(stat.st_ctime, tz=UTC).isoformat(), - "updated_at": datetime.fromtimestamp(stat.st_mtime, tz=UTC).isoformat(), - "uploaded_by": "系统导入", - "version_number": 1, - "ingest_status": KNOWLEDGE_INGEST_STATUS_PUBLISHED, - "ingest_status_updated_at": datetime.now(UTC).isoformat(), - "ingest_completed_at": "", - "ingest_document_name": "", - "ingest_document_updated_at": "", - "ingest_document_sha256": "", - "ingest_agent_run_id": "", + **raw_hit, + "scope": scope, + "read_only": scope == PLATFORM_KNOWLEDGE_SCOPE, } ) - changed = True - - if changed or len(existing_items) != len(documents): - index["documents"] = existing_items - return True - return False - - def _reconcile_document_ingest_statuses( - self, - index: dict[str, Any], - *, - document_ids: list[str] | None = None, - preserve_syncing: bool = True, - ) -> bool: - changed = False - target_ids = {str(item).strip() for item in document_ids or [] if str(item).strip()} - status_map = KnowledgeRagService( - db=self.db, storage_root=self.storage_root - ).get_document_status_map( - list(target_ids) - if target_ids - else [ - str(item.get("id") or "").strip() - for item in index.get("documents", []) - if str(item.get("id") or "").strip() - ] - ) - - for entry in index.get("documents", []): - document_id = str(entry.get("id") or "").strip() - if target_ids and document_id not in target_ids: - continue - - current_status = normalize_ingest_status_code(entry.get("ingest_status")) - if entry.get("ingest_status") != current_status: - entry["ingest_status"] = current_status - changed = True - - if ( - current_status == KNOWLEDGE_INGEST_STATUS_SYNCING - and preserve_syncing - and should_preserve_syncing_status(entry, db=self.db) - ): - continue - - status_payload = status_map.get(document_id) or {} - rag_status = str(status_payload.get("status") or "").strip().lower() - linked_run_status = resolve_linked_ingest_run_status(entry, db=self.db) - if not status_payload: - if ( - current_status == KNOWLEDGE_INGEST_STATUS_SYNCING - and linked_run_status == AgentRunStatus.FAILED.value - ): - desired_status = KNOWLEDGE_INGEST_STATUS_FAILED - else: - continue - elif linked_run_status == AgentRunStatus.FAILED.value and rag_status in { - "pending", - "processing", - "preprocessed", - }: - desired_status = KNOWLEDGE_INGEST_STATUS_FAILED - elif bool(status_payload.get("query_ready")): - desired_status = KNOWLEDGE_INGEST_STATUS_INGESTED - elif rag_status in {"pending", "processing", "preprocessed"}: - desired_status = KNOWLEDGE_INGEST_STATUS_SYNCING - elif rag_status == "failed": - desired_status = KNOWLEDGE_INGEST_STATUS_FAILED - else: - desired_status = KNOWLEDGE_INGEST_STATUS_PUBLISHED - - if ( - current_status == KNOWLEDGE_INGEST_STATUS_FAILED - and desired_status == KNOWLEDGE_INGEST_STATUS_PUBLISHED - ): - continue - if current_status != desired_status: - entry["ingest_status"] = desired_status - entry["ingest_status_updated_at"] = ( - str(status_payload.get("updated_at") or "").strip() - or datetime.now(UTC).isoformat() - ) - if desired_status == KNOWLEDGE_INGEST_STATUS_INGESTED: - self._mark_entry_ingested( - entry, - completed_at=entry.get("ingest_status_updated_at") - or datetime.now(UTC).isoformat(), - ) - changed = True - elif desired_status == KNOWLEDGE_INGEST_STATUS_INGESTED: - changed = self._mark_entry_ingested(entry) or changed - - return changed - - def _apply_ingest_status_to_entry( - self, - entry: dict[str, Any], - *, - status_code: int, - updated_at: str, - agent_run_id: str | None, - ) -> bool: - changed = False - current_status = normalize_ingest_status_code(entry.get("ingest_status")) - if current_status != status_code: - entry["ingest_status"] = status_code - changed = True - - if str(entry.get("ingest_status_updated_at") or "").strip() != updated_at: - entry["ingest_status_updated_at"] = updated_at - changed = True - - if agent_run_id is not None and entry.get("ingest_agent_run_id") != agent_run_id: - entry["ingest_agent_run_id"] = agent_run_id - changed = True - - if status_code == KNOWLEDGE_INGEST_STATUS_INGESTED: - changed = self._mark_entry_ingested(entry, completed_at=updated_at) or changed - - return changed - - def _mark_entry_ingested( - self, - entry: dict[str, Any], - *, - completed_at: str | None = None, - ) -> bool: - completed_value = ( - str(completed_at or entry.get("ingest_completed_at") or "").strip() - or datetime.now(UTC).isoformat() - ) - expected_values = { - "ingest_completed_at": completed_value, - "ingest_document_name": str(entry.get("original_name") or "").strip(), - "ingest_document_updated_at": str(entry.get("updated_at") or "").strip(), - "ingest_document_sha256": str(entry.get("sha256") or "").strip(), + merged_hits.sort(key=lambda item: int(item.get("score") or 0), reverse=True) + merged_hits = merged_hits[: max(1, limit)] + references = [ + str(item.get("code") or "").strip() + for item in merged_hits + if str(item.get("code") or "").strip() + ] + return { + "result_type": "knowledge_search", + "query": str(tenant_result.get("query") or platform_result.get("query") or ""), + "record_count": len(merged_hits), + "hits": merged_hits, + "references": references, + "raw_references": list(tenant_result.get("raw_references") or []) + + list(platform_result.get("raw_references") or []), + "metadata": { + "retrieval_strategy": "tenant_platform_isolated_fusion", + "tenant_record_count": int(tenant_result.get("record_count") or 0), + "platform_record_count": int(platform_result.get("record_count") or 0), + }, + "message": ( + f"已从当前租户与平台只读知识库中联合检索到 {len(merged_hits)} 条相关内容。" + if merged_hits + else "当前租户与平台知识库中没有检索到直接匹配的内容。" + ), } - changed = False - for key, value in expected_values.items(): - if str(entry.get(key) or "").strip() != value: - entry[key] = value - changed = True - return changed - - def _should_index_document(self, entry: dict[str, Any]) -> bool: - status_code = normalize_ingest_status_code(entry.get("ingest_status")) - if status_code in { - KNOWLEDGE_INGEST_STATUS_PUBLISHED, - KNOWLEDGE_INGEST_STATUS_FAILED, - }: - return True - if status_code == KNOWLEDGE_INGEST_STATUS_SYNCING: - return is_syncing_status_stale(entry) - - return any( - [ - not str(entry.get("ingest_completed_at") or "").strip(), - str(entry.get("ingest_document_name") or "").strip() - != str(entry.get("original_name") or "").strip(), - str(entry.get("ingest_document_updated_at") or "").strip() - != str(entry.get("updated_at") or "").strip(), - str(entry.get("ingest_document_sha256") or "").strip() - != str(entry.get("sha256") or "").strip(), - ] - ) - - @staticmethod - def _load_json_file(path: Path, *, default: Any) -> Any: - try: - return json.loads(path.read_text(encoding="utf-8")) - except (FileNotFoundError, json.JSONDecodeError): - return default - - @staticmethod - def _load_text_file(path: Path) -> str: - try: - return path.read_text(encoding="utf-8").strip() - except FileNotFoundError: - return "" def _require_entry(self, index: dict[str, Any], document_id: str) -> dict[str, Any]: for entry in index["documents"]: @@ -741,18 +601,40 @@ class KnowledgeService: raise FileNotFoundError(document_id) def _resolve_document_path(self, entry: dict[str, Any]) -> Path: - return self.library_root / entry["folder"] / entry["stored_name"] + folder = self._normalize_folder(str(entry.get("folder") or "")) + stored_name = str(entry.get("stored_name") or "").strip() + if not stored_name or Path(stored_name).name != stored_name: + raise ValueError("知识文档存储路径不合法。") + folder_root = (self.library_root / folder).resolve() + candidate = (folder_root / stored_name).resolve() + if not candidate.is_relative_to(folder_root): + raise ValueError("知识文档存储路径越界。") + return candidate - def _replace_document_content( - self, document_id: str, content: bytes, actor_name: str + def replace_document_content_from_onlyoffice( + self, + document_id: str, + content: bytes, + *, + actor_name: str, + expected_document_key: str, + expected_version: int, ) -> KnowledgeDocumentDetailRead: + if self.storage_scope.read_only or self.tenant_id is None: + raise ValueError("平台知识文档为只读资源，不能回写。") index = self._load_index() entry = self._require_entry(index, document_id) + if ( + build_onlyoffice_document_key(entry) != expected_document_key + or int(entry.get("version_number") or 1) != expected_version + ): + raise ValueError("ONLYOFFICE 编辑基线已过期，请重新打开文档后再编辑。") current_user = CurrentUserContext( username="onlyoffice", name=actor_name or "ONLYOFFICE", role_codes=["manager"], is_admin=True, + tenant_id=self.tenant_id, ) return self.upload_document( folder=entry["folder"], @@ -761,12 +643,7 @@ class KnowledgeService: current_user=current_user, ) - @staticmethod - def _parse_onlyoffice_callback(payload: dict[str, Any]) -> OnlyOfficeCallbackPayload: - return parse_onlyoffice_callback(payload) - _build_onlyoffice_document_key = staticmethod(build_onlyoffice_document_key) - _build_onlyoffice_access_token = staticmethod(build_onlyoffice_access_token) _resolve_onlyoffice_document_type = staticmethod(resolve_onlyoffice_document_type) _normalize_filename = staticmethod(normalize_filename) diff --git a/server/src/app/services/knowledge_file_utils.py b/server/src/app/services/knowledge_file_utils.py index 7a87b03..f28b671 100644 --- a/server/src/app/services/knowledge_file_utils.py +++ b/server/src/app/services/knowledge_file_utils.py @@ -16,6 +16,7 @@ from app.services.knowledge_constants import ( WORD_EXTENSIONS, ) + def normalize_filename(filename: str) -> str: normalized = Path(str(filename or "").strip()).name.strip() normalized = normalized.replace("/", "_").replace("\\", "_") @@ -33,28 +34,6 @@ def extract_extension(filename: str) -> str: suffix = Path(filename).suffix.lower().lstrip(".") return suffix -def _build_onlyoffice_document_key(entry: dict[str, Any]) -> str: - version = int(entry.get("version_number", 1)) - checksum = str(entry.get("sha256") or "")[:12] - return f"{entry['id']}-v{version}-{checksum or 'nochecksum'}" - -def _build_onlyoffice_access_token(self, document_id: str) -> str: - onlyoffice_settings = resolve_onlyoffice_settings() - payload = { - "scope": "onlyoffice-content", - "document_id": document_id, - } - return jwt.encode(payload, onlyoffice_settings.jwt_secret, algorithm="HS256") - -def _resolve_onlyoffice_document_type(extension: str) -> str: - if extension in WORD_EXTENSIONS: - return "word" - if extension in EXCEL_EXTENSIONS: - return "cell" - if extension in PPT_EXTENSIONS: - return "slide" - raise ValueError("当前文件格式不支持 ONLYOFFICE 预览。") - def parse_stored_name(stored_name: str) -> tuple[str, str]: if "__" not in stored_name: return uuid4().hex, stored_name @@ -109,4 +88,3 @@ def resolve_file_type_label(file_type: str) -> str: def can_preview(extension: str) -> bool: return extension in INLINE_PREVIEW_EXTENSIONS or extension in STRUCTURED_PREVIEW_EXTENSIONS - diff --git a/server/src/app/services/knowledge_index_state.py b/server/src/app/services/knowledge_index_state.py new file mode 100644 index 0000000..92a7336 --- /dev/null +++ b/server/src/app/services/knowledge_index_state.py @@ -0,0 +1,266 @@ +from __future__ import annotations + +import json +import mimetypes +from datetime import UTC, datetime +from typing import Any + +from app.core.agent_enums import AgentRunStatus +from app.services.knowledge_constants import ( + FIXED_KNOWLEDGE_FOLDERS, + KNOWLEDGE_INGEST_STATUS_FAILED, + KNOWLEDGE_INGEST_STATUS_INGESTED, + KNOWLEDGE_INGEST_STATUS_PUBLISHED, + KNOWLEDGE_INGEST_STATUS_SYNCING, +) +from app.services.knowledge_ingest_status import ( + is_syncing_status_stale, + normalize_ingest_status_code, + resolve_linked_ingest_run_status, + should_preserve_syncing_status, +) + + +class KnowledgeIndexStateMixin: + def _load_index(self) -> dict[str, Any]: + try: + payload = json.loads(self.index_path.read_text(encoding="utf-8")) + except (FileNotFoundError, json.JSONDecodeError): + payload = {"version": 1, "documents": []} + payload.setdefault("documents", []) + return payload + + def _save_index(self, index: dict[str, Any]) -> None: + self.index_path.write_text( + json.dumps(index, ensure_ascii=False, indent=2), + encoding="utf-8", + ) + + def _reconcile_index(self, index: dict[str, Any]) -> bool: + changed = False + documents = index.setdefault("documents", []) + known_by_stored = { + (item["folder"], item["stored_name"]): item + for item in documents + if item.get("folder") and item.get("stored_name") + } + existing_items: list[dict[str, Any]] = [] + for item in documents: + file_path = self._resolve_document_path(item) + if not file_path.exists(): + changed = True + continue + item["size_bytes"] = file_path.stat().st_size + item["extension"] = self._extract_extension(item["original_name"]) + item["mime_type"] = item.get("mime_type") or ( + mimetypes.guess_type(item["original_name"])[0] or "application/octet-stream" + ) + normalized_status = normalize_ingest_status_code(item.get("ingest_status")) + if item.get("ingest_status") != normalized_status: + item["ingest_status"] = normalized_status + changed = True + defaults = { + "ingest_agent_run_id": "", + "ingest_status_updated_at": item.get("updated_at") + or item.get("created_at") + or "", + "ingest_completed_at": "", + "ingest_document_name": "", + "ingest_document_updated_at": "", + "ingest_document_sha256": "", + } + for key, value in defaults.items(): + if key not in item: + item[key] = value + changed = True + existing_items.append(item) + + for folder_name in FIXED_KNOWLEDGE_FOLDERS: + folder_path = self.library_root / folder_name + for file_path in folder_path.iterdir(): + if not file_path.is_file() or file_path.name.startswith("."): + continue + key = (folder_name, file_path.name) + if key in known_by_stored: + continue + document_id, original_name = self._parse_stored_name(file_path.name) + stat = file_path.stat() + existing_items.append( + { + "id": document_id, + "folder": folder_name, + "original_name": original_name, + "stored_name": file_path.name, + "mime_type": mimetypes.guess_type(original_name)[0] + or "application/octet-stream", + "extension": self._extract_extension(original_name), + "size_bytes": stat.st_size, + "sha256": "", + "created_at": datetime.fromtimestamp(stat.st_ctime, tz=UTC).isoformat(), + "updated_at": datetime.fromtimestamp(stat.st_mtime, tz=UTC).isoformat(), + "uploaded_by": "系统导入", + "version_number": 1, + "ingest_status": KNOWLEDGE_INGEST_STATUS_PUBLISHED, + "ingest_status_updated_at": datetime.now(UTC).isoformat(), + "ingest_completed_at": "", + "ingest_document_name": "", + "ingest_document_updated_at": "", + "ingest_document_sha256": "", + "ingest_agent_run_id": "", + } + ) + changed = True + + if changed or len(existing_items) != len(documents): + index["documents"] = existing_items + return True + return False + + def _reconcile_document_ingest_statuses( + self, + index: dict[str, Any], + *, + document_ids: list[str] | None = None, + preserve_syncing: bool = True, + ) -> bool: + changed = False + target_ids = {str(item).strip() for item in document_ids or [] if str(item).strip()} + status_map = self._rag_service().get_document_status_map( + list(target_ids) + if target_ids + else [ + str(item.get("id") or "").strip() + for item in index.get("documents", []) + if str(item.get("id") or "").strip() + ] + ) + for entry in index.get("documents", []): + document_id = str(entry.get("id") or "").strip() + if target_ids and document_id not in target_ids: + continue + current_status = normalize_ingest_status_code(entry.get("ingest_status")) + if entry.get("ingest_status") != current_status: + entry["ingest_status"] = current_status + changed = True + if ( + current_status == KNOWLEDGE_INGEST_STATUS_SYNCING + and preserve_syncing + and should_preserve_syncing_status(entry, db=self.db) + ): + continue + + status_payload = status_map.get(document_id) or {} + rag_status = str(status_payload.get("status") or "").strip().lower() + linked_run_status = resolve_linked_ingest_run_status(entry, db=self.db) + if not status_payload: + if ( + current_status == KNOWLEDGE_INGEST_STATUS_SYNCING + and linked_run_status == AgentRunStatus.FAILED.value + ): + desired_status = KNOWLEDGE_INGEST_STATUS_FAILED + else: + continue + elif linked_run_status == AgentRunStatus.FAILED.value and rag_status in { + "pending", + "processing", + "preprocessed", + }: + desired_status = KNOWLEDGE_INGEST_STATUS_FAILED + elif bool(status_payload.get("query_ready")): + desired_status = KNOWLEDGE_INGEST_STATUS_INGESTED + elif rag_status in {"pending", "processing", "preprocessed"}: + desired_status = KNOWLEDGE_INGEST_STATUS_SYNCING + elif rag_status == "failed": + desired_status = KNOWLEDGE_INGEST_STATUS_FAILED + else: + desired_status = KNOWLEDGE_INGEST_STATUS_PUBLISHED + + if ( + current_status == KNOWLEDGE_INGEST_STATUS_FAILED + and desired_status == KNOWLEDGE_INGEST_STATUS_PUBLISHED + ): + continue + if current_status != desired_status: + entry["ingest_status"] = desired_status + entry["ingest_status_updated_at"] = ( + str(status_payload.get("updated_at") or "").strip() + or datetime.now(UTC).isoformat() + ) + if desired_status == KNOWLEDGE_INGEST_STATUS_INGESTED: + self._mark_entry_ingested( + entry, + completed_at=entry.get("ingest_status_updated_at") + or datetime.now(UTC).isoformat(), + ) + changed = True + elif desired_status == KNOWLEDGE_INGEST_STATUS_INGESTED: + changed = self._mark_entry_ingested(entry) or changed + return changed + + def _apply_ingest_status_to_entry( + self, + entry: dict[str, Any], + *, + status_code: int, + updated_at: str, + agent_run_id: str | None, + ) -> bool: + changed = False + current_status = normalize_ingest_status_code(entry.get("ingest_status")) + if current_status != status_code: + entry["ingest_status"] = status_code + changed = True + if str(entry.get("ingest_status_updated_at") or "").strip() != updated_at: + entry["ingest_status_updated_at"] = updated_at + changed = True + if agent_run_id is not None and entry.get("ingest_agent_run_id") != agent_run_id: + entry["ingest_agent_run_id"] = agent_run_id + changed = True + if status_code == KNOWLEDGE_INGEST_STATUS_INGESTED: + changed = self._mark_entry_ingested(entry, completed_at=updated_at) or changed + return changed + + def _mark_entry_ingested( + self, + entry: dict[str, Any], + *, + completed_at: str | None = None, + ) -> bool: + completed_value = ( + str(completed_at or entry.get("ingest_completed_at") or "").strip() + or datetime.now(UTC).isoformat() + ) + expected_values = { + "ingest_completed_at": completed_value, + "ingest_document_name": str(entry.get("original_name") or "").strip(), + "ingest_document_updated_at": str(entry.get("updated_at") or "").strip(), + "ingest_document_sha256": str(entry.get("sha256") or "").strip(), + } + changed = False + for key, value in expected_values.items(): + if str(entry.get(key) or "").strip() != value: + entry[key] = value + changed = True + return changed + + @staticmethod + def _should_index_document(entry: dict[str, Any]) -> bool: + status_code = normalize_ingest_status_code(entry.get("ingest_status")) + if status_code in { + KNOWLEDGE_INGEST_STATUS_PUBLISHED, + KNOWLEDGE_INGEST_STATUS_FAILED, + }: + return True + if status_code == KNOWLEDGE_INGEST_STATUS_SYNCING: + return is_syncing_status_stale(entry) + return any( + [ + not str(entry.get("ingest_completed_at") or "").strip(), + str(entry.get("ingest_document_name") or "").strip() + != str(entry.get("original_name") or "").strip(), + str(entry.get("ingest_document_updated_at") or "").strip() + != str(entry.get("updated_at") or "").strip(), + str(entry.get("ingest_document_sha256") or "").strip() + != str(entry.get("sha256") or "").strip(), + ] + ) diff --git a/server/src/app/services/knowledge_index_tasks.py b/server/src/app/services/knowledge_index_tasks.py index 243887c..322a2dc 100644 --- a/server/src/app/services/knowledge_index_tasks.py +++ b/server/src/app/services/knowledge_index_tasks.py @@ -17,6 +17,8 @@ from app.services.knowledge import ( KnowledgeService, ) from app.services.knowledge_rag import KnowledgeRagService +from app.services.knowledge_run_scope import resolve_trusted_knowledge_run_tenant +from app.services.knowledge_tenant_scope import require_knowledge_tenant_id logger = get_logger("app.services.knowledge_index_tasks") HEARTBEAT_INTERVAL_SECONDS = 10 @@ -57,8 +59,16 @@ class KnowledgeIndexTaskManager: document_ids: list[str], force: bool, ) -> None: + requested_tenant_id = require_knowledge_tenant_id(current_user.tenant_id) session_factory = get_session_factory() db = session_factory() + try: + tenant_id = resolve_trusted_knowledge_run_tenant(db, agent_run_id) + if tenant_id != requested_tenant_id: + raise ValueError("知识索引任务 tenant_id 与 Agent Run 不一致。") + except Exception: + db.close() + raise started = perf_counter() heartbeat_stop = threading.Event() heartbeat_thread: threading.Thread | None = None @@ -66,6 +76,7 @@ class KnowledgeIndexTaskManager: knowledge_ingest: dict[str, Any] | None = None tool_request_json = { "agent": AgentName.HERMES.value, + "tenant_id": tenant_id, "folder": folder, "document_ids": document_ids, "force": force, @@ -73,8 +84,8 @@ class KnowledgeIndexTaskManager: try: run_service = AgentRunService(db) - knowledge_service = KnowledgeService(db=db) - rag_service = KnowledgeRagService(db=db) + knowledge_service = KnowledgeService(db=db, tenant_id=tenant_id) + rag_service = KnowledgeRagService(db=db, tenant_id=tenant_id) knowledge_ingest = _build_initial_knowledge_ingest_state( knowledge_service, document_ids=document_ids, @@ -368,7 +379,7 @@ class KnowledgeIndexTaskManager: duration_ms=int((perf_counter() - started) * 1000), error_message=str(exc), ) - KnowledgeService(db=db).set_document_ingest_statuses( + KnowledgeService(db=db, tenant_id=tenant_id).set_document_ingest_statuses( _resolve_failed_ingest_document_ids(knowledge_ingest, document_ids), KNOWLEDGE_INGEST_STATUS_FAILED, agent_run_id=agent_run_id, diff --git a/server/src/app/services/knowledge_ingest_log.py b/server/src/app/services/knowledge_ingest_log.py index 9f77ae0..64be33e 100644 --- a/server/src/app/services/knowledge_ingest_log.py +++ b/server/src/app/services/knowledge_ingest_log.py @@ -101,15 +101,21 @@ def build_document_graph_summary( *, workspace: str, document_id: str, + lightrag_root: Path | None = None, ) -> dict[str, Any]: - workspace_dir = ( - Path(storage_root) / "knowledge" / ".lightrag" / str(workspace).strip() + resolved_lightrag_root = Path( + lightrag_root or (Path(storage_root) / "knowledge" / ".lightrag") ).resolve() + workspace_dir = (resolved_lightrag_root / str(workspace).strip()).resolve() entities_payload = _load_json_file(workspace_dir / "kv_store_full_entities.json") relations_payload = _load_json_file(workspace_dir / "kv_store_full_relations.json") chunks_payload = _load_json_file(workspace_dir / "kv_store_text_chunks.json") entity_chunks_payload = _load_json_file(workspace_dir / "kv_store_entity_chunks.json") - graph_snapshot = _load_lightrag_graph_snapshot(storage_root, workspace=workspace) + graph_snapshot = _load_lightrag_graph_snapshot( + storage_root, + workspace=workspace, + lightrag_root=resolved_lightrag_root, + ) entities = _normalize_document_entities(entities_payload, document_id) relations = _normalize_document_relations(relations_payload, document_id) @@ -135,7 +141,10 @@ def _resolve_lightrag_workspace(route_json: dict[str, Any]) -> str: ).strip() if explicit_workspace: return explicit_workspace - return os.environ.get("LIGHTRAG_WORKSPACE", "x_financial_knowledge").strip() or "x_financial_knowledge" + return ( + os.environ.get("LIGHTRAG_WORKSPACE", "x_financial_knowledge").strip() + or "x_financial_knowledge" + ) def _enrich_graph_payload( @@ -207,11 +216,17 @@ def _enrich_relation_list( return enriched_relations -def _load_lightrag_graph_snapshot(storage_root: Path, *, workspace: str) -> dict[str, Any]: +def _load_lightrag_graph_snapshot( + storage_root: Path, + *, + workspace: str, + lightrag_root: Path | None = None, +) -> dict[str, Any]: + resolved_lightrag_root = Path( + lightrag_root or (Path(storage_root) / "knowledge" / ".lightrag") + ) graphml_path = ( - Path(storage_root) - / "knowledge" - / ".lightrag" + resolved_lightrag_root / str(workspace).strip() / "graph_chunk_entity_relation.graphml" ) diff --git a/server/src/app/services/knowledge_onlyoffice.py b/server/src/app/services/knowledge_onlyoffice.py index 8e7e57a..49ab7c2 100644 --- a/server/src/app/services/knowledge_onlyoffice.py +++ b/server/src/app/services/knowledge_onlyoffice.py @@ -2,8 +2,10 @@ from __future__ import annotations from dataclasses import dataclass from typing import Any +from urllib.parse import quote import jwt +from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext from app.core.config import get_settings @@ -16,6 +18,15 @@ from app.services.knowledge_constants import ( WORD_EXTENSIONS, ) from app.services.knowledge_file_utils import extract_extension +from app.services.knowledge_onlyoffice_security import ( + KnowledgeOnlyOfficeSessionService, + OnlyOfficeValidatedSession, +) +from app.services.knowledge_tenant_scope import ( + PLATFORM_KNOWLEDGE_SCOPE, + TENANT_KNOWLEDGE_SCOPE, + require_knowledge_tenant_id, +) from app.services.settings import resolve_onlyoffice_settings logger = get_logger("app.services.knowledge") @@ -41,26 +52,22 @@ def build_onlyoffice_document_key(entry: dict[str, Any]) -> str: return f"{entry['id']}-v{version}-{checksum or 'nochecksum'}" -def build_onlyoffice_access_token(document_id: str) -> str: - onlyoffice_settings = resolve_onlyoffice_settings() - payload = { - "scope": "onlyoffice-content", - "document_id": document_id, - } - return jwt.encode(payload, onlyoffice_settings.jwt_secret, algorithm="HS256") - - def build_onlyoffice_config( *, + db: Session, document_id: str, entry: dict[str, Any], current_user: CurrentUserContext, + resource_scope: str, + tenant_id: str | None, + editable: bool = False, ) -> KnowledgeOnlyOfficeConfigRead: settings = get_settings() - onlyoffice_settings = resolve_onlyoffice_settings() + onlyoffice_settings = resolve_onlyoffice_settings(db) if not onlyoffice_settings.enabled: logger.warning( - "ONLYOFFICE disabled in runtime config doc=%s enabled=%s public_url=%s backend_url=%s jwt_set=%s", + "ONLYOFFICE disabled in runtime config doc=%s enabled=%s " + "public_url=%s backend_url=%s jwt_set=%s", document_id, onlyoffice_settings.enabled, onlyoffice_settings.public_url, @@ -70,7 +77,8 @@ def build_onlyoffice_config( raise ValueError("ONLYOFFICE 预览未启用。") if not onlyoffice_settings.public_url or not onlyoffice_settings.backend_url: logger.warning( - "ONLYOFFICE config incomplete doc=%s enabled=%s public_url=%s backend_url=%s jwt_set=%s", + "ONLYOFFICE config incomplete doc=%s enabled=%s " + "public_url=%s backend_url=%s jwt_set=%s", document_id, onlyoffice_settings.enabled, onlyoffice_settings.public_url, @@ -92,16 +100,39 @@ def build_onlyoffice_config( extension = extract_extension(entry["original_name"]) if extension not in ONLYOFFICE_EDITABLE_EXTENSIONS: raise ValueError("当前文件格式不支持 ONLYOFFICE 预览。") + requester_tenant_id = require_knowledge_tenant_id(current_user.tenant_id) + if resource_scope == TENANT_KNOWLEDGE_SCOPE: + normalized_tenant_id = require_knowledge_tenant_id(tenant_id) + if requester_tenant_id != normalized_tenant_id: + raise ValueError("当前用户不能访问其他租户的知识文档。") + elif resource_scope == PLATFORM_KNOWLEDGE_SCOPE: + if editable: + raise ValueError("平台知识文档为只读资源，不能编辑。") + else: + raise ValueError("知识文档 scope 不合法。") + if editable and not ( + current_user.is_admin + or "manager" in {str(item).strip().lower() for item in current_user.role_codes} + ): + raise ValueError("只有租户管理员可以编辑知识文档。") backend_base_url = onlyoffice_settings.backend_url.rstrip("/") public_url = onlyoffice_settings.public_url.rstrip("/") - access_token = build_onlyoffice_access_token(document_id) + session_tokens = KnowledgeOnlyOfficeSessionService(db).issue( + resource_scope=resource_scope, + tenant_id=requester_tenant_id, + document_id=document_id, + entry=entry, + current_user=current_user, + editable=editable, + ) document_url = ( f"{backend_base_url}{settings.api_v1_prefix}/knowledge/documents/{document_id}/onlyoffice/content" - f"?access_token={access_token}" + f"?access_token={quote(session_tokens.content_token, safe='')}" ) callback_url = ( f"{backend_base_url}{settings.api_v1_prefix}/knowledge/documents/{document_id}/onlyoffice/callback" + f"?callback_token={quote(session_tokens.callback_token, safe='')}" ) config: dict[str, Any] = { @@ -113,13 +144,13 @@ def build_onlyoffice_config( "url": document_url, "permissions": { "download": True, - "edit": False, + "edit": editable, "print": True, "copy": True, }, }, "editorConfig": { - "mode": "view", + "mode": "edit" if editable else "view", "lang": "zh-CN", "callbackUrl": callback_url, "user": { @@ -130,8 +161,8 @@ def build_onlyoffice_config( "compactHeader": True, "compactToolbar": True, "toolbarNoTabs": False, - "autosave": False, - "forcesave": False, + "autosave": editable, + "forcesave": editable, }, }, "width": "100%", @@ -141,19 +172,15 @@ def build_onlyoffice_config( return KnowledgeOnlyOfficeConfigRead(documentServerUrl=public_url, config=config) -def validate_onlyoffice_access_token(document_id: str, access_token: str) -> None: - onlyoffice_settings = resolve_onlyoffice_settings() - try: - payload = jwt.decode( - access_token, - onlyoffice_settings.jwt_secret, - algorithms=["HS256"], - ) - except jwt.PyJWTError as exc: - raise ValueError("ONLYOFFICE 文件访问令牌无效。") from exc - - if payload.get("scope") != "onlyoffice-content" or payload.get("document_id") != document_id: - raise ValueError("ONLYOFFICE 文件访问令牌无效。") +def validate_onlyoffice_access_token( + db: Session, + document_id: str, + access_token: str, +) -> OnlyOfficeValidatedSession: + return KnowledgeOnlyOfficeSessionService(db).validate_content( + document_id=document_id, + token=access_token, + ) def resolve_onlyoffice_document_type(extension: str) -> str: diff --git a/server/src/app/services/knowledge_onlyoffice_callback.py b/server/src/app/services/knowledge_onlyoffice_callback.py new file mode 100644 index 0000000..b7c09be --- /dev/null +++ b/server/src/app/services/knowledge_onlyoffice_callback.py @@ -0,0 +1,125 @@ +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from sqlalchemy.orm import Session + +from app.services.knowledge import KnowledgeService +from app.services.knowledge_onlyoffice_security import ( + KnowledgeOnlyOfficeSessionService, + OnlyOfficeSecurityError, + OnlyOfficeValidatedSession, + download_onlyoffice_document, +) +from app.services.knowledge_tenant_scope import PLATFORM_KNOWLEDGE_SCOPE + +_WRITE_CALLBACK_STATUSES = {2, 6} + + +def resolve_onlyoffice_content( + *, + db: Session, + storage_root: Path | None, + document_id: str, + access_token: str, +) -> tuple[Path, str, str]: + validated = KnowledgeOnlyOfficeSessionService(db).validate_content( + document_id=document_id, + token=access_token, + ) + service = _service_for_session( + db=db, + storage_root=storage_root, + session=validated, + ) + entry = service.get_document_entry(document_id) + if ( + service._build_onlyoffice_document_key(entry) != validated.document_key + or int(entry.get("version_number") or 1) != validated.document_version + ): + raise OnlyOfficeSecurityError("ONLYOFFICE 会话绑定的文档版本已失效。") + return service.get_document_content(document_id) + + +def handle_onlyoffice_callback( + *, + db: Session, + storage_root: Path | None, + document_id: str, + callback_token: str, + payload: dict[str, Any], +) -> None: + session_service = KnowledgeOnlyOfficeSessionService(db) + session_service.validate_callback( + document_id=document_id, + token=callback_token, + ) + try: + callback_status = int(payload.get("status") or 0) + except (TypeError, ValueError) as exc: + raise ValueError("ONLYOFFICE 回调状态不合法。") from exc + if callback_status not in _WRITE_CALLBACK_STATUSES: + return + + download_url = str(payload.get("url") or "").strip() + if not download_url: + raise ValueError("ONLYOFFICE 回写回调缺少下载 URL。") + claimed = session_service.claim_callback( + document_id=document_id, + token=callback_token, + payload_document_key=str(payload.get("key") or "").strip(), + ) + try: + service = _service_for_session( + db=db, + storage_root=storage_root, + session=claimed, + ) + entry = service.get_document_entry(document_id) + if ( + service._build_onlyoffice_document_key(entry) != claimed.document_key + or int(entry.get("version_number") or 1) != claimed.document_version + ): + raise OnlyOfficeSecurityError("ONLYOFFICE 编辑基线已失效。") + content = download_onlyoffice_document( + download_url, + expected_filename=str(entry.get("original_name") or ""), + ) + users = [str(item).strip() for item in payload.get("users") or [] if str(item).strip()] + service.replace_document_content_from_onlyoffice( + document_id, + content, + actor_name=users[0] if users else "ONLYOFFICE", + expected_document_key=claimed.document_key, + expected_version=claimed.document_version, + ) + except Exception as exc: + session_service.finish_callback( + claimed.jti, + succeeded=False, + failure_reason=type(exc).__name__, + ) + raise + session_service.finish_callback(claimed.jti, succeeded=True) + + +def _service_for_session( + *, + db: Session, + storage_root: Path | None, + session: OnlyOfficeValidatedSession, +) -> KnowledgeService: + if session.resource_scope == PLATFORM_KNOWLEDGE_SCOPE: + return KnowledgeService( + storage_root=storage_root, + db=db, + scope=PLATFORM_KNOWLEDGE_SCOPE, + ) + if session.tenant_id is None: + raise ValueError("ONLYOFFICE 租户会话缺少 tenant_id。") + return KnowledgeService( + storage_root=storage_root, + db=db, + tenant_id=session.tenant_id, + ) diff --git a/server/src/app/services/knowledge_onlyoffice_security.py b/server/src/app/services/knowledge_onlyoffice_security.py new file mode 100644 index 0000000..7ef1e74 --- /dev/null +++ b/server/src/app/services/knowledge_onlyoffice_security.py @@ -0,0 +1,520 @@ +from __future__ import annotations + +import http.client +import ipaddress +import os +import socket +import ssl +import zipfile +from dataclasses import dataclass +from datetime import UTC, datetime, timedelta +from io import BytesIO +from typing import Any +from urllib.parse import SplitResult, urlsplit +from uuid import uuid4 + +import jwt +from sqlalchemy import update +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.knowledge_security import KnowledgeOnlyOfficeSession +from app.services.knowledge_tenant_scope import ( + PLATFORM_KNOWLEDGE_SCOPE, + TENANT_KNOWLEDGE_SCOPE, + require_knowledge_tenant_id, +) +from app.services.settings import resolve_onlyoffice_settings + +ONLYOFFICE_TOKEN_AUDIENCE = "onlyoffice-document-server" +ONLYOFFICE_TOKEN_ISSUER = "x-financial" +ONLYOFFICE_CONTENT_SCOPE = "onlyoffice-content" +ONLYOFFICE_CALLBACK_SCOPE = "onlyoffice-callback" +DEFAULT_CONTENT_TOKEN_TTL_SECONDS = 300 +DEFAULT_CALLBACK_SESSION_TTL_SECONDS = 4 * 60 * 60 +MAX_CALLBACK_SESSION_TTL_SECONDS = 12 * 60 * 60 +DEFAULT_MAX_DOWNLOAD_BYTES = 25 * 1024 * 1024 +MAX_DOWNLOAD_BYTES_CEILING = 100 * 1024 * 1024 +_OOXML_REQUIRED_ENTRY = { + "docx": "word/", + "xlsx": "xl/", + "pptx": "ppt/", +} +_OOXML_MIME_TYPES = { + "docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document", + "xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", + "pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation", +} +_GENERIC_BINARY_MIME_TYPES = {"application/octet-stream", "application/zip"} + + +class OnlyOfficeSecurityError(ValueError): + pass + + +class OnlyOfficeReplayError(OnlyOfficeSecurityError): + pass + + +@dataclass(frozen=True, slots=True) +class OnlyOfficeSessionTokens: + jti: str + content_token: str + callback_token: str + expires_at: datetime + + +@dataclass(frozen=True, slots=True) +class OnlyOfficeValidatedSession: + jti: str + tenant_id: str | None + resource_scope: str + document_id: str + document_key: str + document_version: int + editable: bool + status: str + + +class KnowledgeOnlyOfficeSessionService: + def __init__(self, db: Session) -> None: + self.db = db + + def issue( + self, + *, + resource_scope: str, + tenant_id: str | None, + document_id: str, + entry: dict[str, Any], + current_user: CurrentUserContext, + editable: bool, + ) -> OnlyOfficeSessionTokens: + normalized_tenant_id = require_knowledge_tenant_id(tenant_id) + if resource_scope not in {TENANT_KNOWLEDGE_SCOPE, PLATFORM_KNOWLEDGE_SCOPE}: + raise OnlyOfficeSecurityError("ONLYOFFICE 资源 scope 不合法。") + if resource_scope == PLATFORM_KNOWLEDGE_SCOPE and editable: + raise OnlyOfficeSecurityError("平台知识文档为只读资源，不能进入编辑会话。") + + settings = resolve_onlyoffice_settings(self.db) + if not settings.jwt_secret: + raise OnlyOfficeSecurityError("ONLYOFFICE JWT 密钥未配置。") + now = datetime.now(UTC) + content_expires_at = now + timedelta(seconds=_content_token_ttl_seconds()) + expires_at = now + timedelta(seconds=_callback_session_ttl_seconds()) + jti = str(uuid4()) + document_key = _build_document_key(entry) + document_version = int(entry.get("version_number") or 1) + row = KnowledgeOnlyOfficeSession( + jti=jti, + tenant_id=normalized_tenant_id, + resource_scope=resource_scope, + document_id=document_id, + document_key=document_key, + document_version=document_version, + audience=ONLYOFFICE_TOKEN_AUDIENCE, + editable=bool(editable), + status="active", + created_by=current_user.username, + expires_at=expires_at, + failure_reason="", + ) + self.db.add(row) + self.db.commit() + + common_claims = { + "iss": ONLYOFFICE_TOKEN_ISSUER, + "aud": ONLYOFFICE_TOKEN_AUDIENCE, + "sub": document_id, + "jti": jti, + "iat": int(now.timestamp()), + "nbf": int(now.timestamp()), + "tenant_id": normalized_tenant_id, + "resource_scope": resource_scope, + "document_id": document_id, + "document_key": document_key, + "document_version": document_version, + "editable": bool(editable), + } + return OnlyOfficeSessionTokens( + jti=jti, + content_token=jwt.encode( + { + **common_claims, + "scope": ONLYOFFICE_CONTENT_SCOPE, + "exp": int(content_expires_at.timestamp()), + }, + settings.jwt_secret, + algorithm="HS256", + ), + callback_token=jwt.encode( + { + **common_claims, + "scope": ONLYOFFICE_CALLBACK_SCOPE, + "exp": int(expires_at.timestamp()), + }, + settings.jwt_secret, + algorithm="HS256", + ), + expires_at=expires_at, + ) + + def validate_content( + self, + *, + document_id: str, + token: str, + ) -> OnlyOfficeValidatedSession: + return self._validate( + document_id=document_id, + token=token, + expected_scope=ONLYOFFICE_CONTENT_SCOPE, + allowed_statuses={"active"}, + ) + + def validate_callback( + self, + *, + document_id: str, + token: str, + ) -> OnlyOfficeValidatedSession: + return self._validate( + document_id=document_id, + token=token, + expected_scope=ONLYOFFICE_CALLBACK_SCOPE, + allowed_statuses={"active"}, + ) + + def claim_callback( + self, + *, + document_id: str, + token: str, + payload_document_key: str, + ) -> OnlyOfficeValidatedSession: + validated = self.validate_callback(document_id=document_id, token=token) + if not validated.editable or validated.resource_scope != TENANT_KNOWLEDGE_SCOPE: + raise OnlyOfficeSecurityError("只读 ONLYOFFICE 会话禁止回写文档。") + if not payload_document_key or payload_document_key != validated.document_key: + raise OnlyOfficeSecurityError("ONLYOFFICE 回调文档 key 与编辑会话不一致。") + + now = datetime.now(UTC) + result = self.db.execute( + update(KnowledgeOnlyOfficeSession) + .where( + KnowledgeOnlyOfficeSession.jti == validated.jti, + KnowledgeOnlyOfficeSession.status == "active", + KnowledgeOnlyOfficeSession.expires_at > now, + ) + .values(status="processing", claimed_at=now) + ) + if result.rowcount != 1: + self.db.rollback() + raise OnlyOfficeReplayError("ONLYOFFICE 回调会话已被使用或已过期。") + self.db.commit() + return OnlyOfficeValidatedSession( + jti=validated.jti, + tenant_id=validated.tenant_id, + resource_scope=validated.resource_scope, + document_id=validated.document_id, + document_key=validated.document_key, + document_version=validated.document_version, + editable=validated.editable, + status="processing", + ) + + def finish_callback(self, jti: str, *, succeeded: bool, failure_reason: str = "") -> None: + row = self.db.get(KnowledgeOnlyOfficeSession, jti) + if row is None or row.status != "processing": + raise OnlyOfficeReplayError("ONLYOFFICE 回调会话状态不可更新。") + if succeeded: + row.status = "consumed" + row.consumed_at = datetime.now(UTC) + row.failure_reason = "" + else: + row.status = "failed" + row.failure_reason = str(failure_reason or "callback_failed")[:1000] + self.db.commit() + + def _validate( + self, + *, + document_id: str, + token: str, + expected_scope: str, + allowed_statuses: set[str], + ) -> OnlyOfficeValidatedSession: + settings = resolve_onlyoffice_settings(self.db) + if not settings.jwt_secret: + raise OnlyOfficeSecurityError("ONLYOFFICE JWT 密钥未配置。") + try: + claims = jwt.decode( + token, + settings.jwt_secret, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + issuer=ONLYOFFICE_TOKEN_ISSUER, + options={ + "require": [ + "iss", + "aud", + "sub", + "jti", + "iat", + "nbf", + "exp", + "scope", + "resource_scope", + "document_id", + "document_key", + "document_version", + "editable", + ] + }, + ) + except jwt.PyJWTError as exc: + raise OnlyOfficeSecurityError("ONLYOFFICE 会话令牌无效或已过期。") from exc + + jti = str(claims.get("jti") or "").strip() + row = self.db.get(KnowledgeOnlyOfficeSession, jti) + if row is None: + raise OnlyOfficeSecurityError("ONLYOFFICE 会话不存在。") + now = datetime.now(UTC) + expires_at = _as_utc(row.expires_at) + claim_tenant_id = claims.get("tenant_id") + if claim_tenant_id is not None: + claim_tenant_id = require_knowledge_tenant_id(claim_tenant_id) + comparisons = ( + claims.get("scope") == expected_scope, + claims.get("sub") == document_id, + claims.get("document_id") == document_id == row.document_id, + claims.get("resource_scope") == row.resource_scope, + claim_tenant_id == row.tenant_id, + claims.get("document_key") == row.document_key, + int(claims.get("document_version") or 0) == row.document_version, + bool(claims.get("editable")) == row.editable, + row.audience == ONLYOFFICE_TOKEN_AUDIENCE, + row.status in allowed_statuses, + expires_at > now, + ) + if not all(comparisons): + if row.status not in allowed_statuses: + raise OnlyOfficeReplayError("ONLYOFFICE 会话已被使用或已撤销。") + raise OnlyOfficeSecurityError("ONLYOFFICE 会话与目标文档不匹配。") + return OnlyOfficeValidatedSession( + jti=row.jti, + tenant_id=row.tenant_id, + resource_scope=row.resource_scope, + document_id=row.document_id, + document_key=row.document_key, + document_version=row.document_version, + editable=row.editable, + status=row.status, + ) + + +def download_onlyoffice_document(download_url: str, *, expected_filename: str) -> bytes: + parsed, resolved_ip = _validate_download_target(download_url) + connection = _open_pinned_connection(parsed, resolved_ip) + try: + target = parsed.path or "/" + if parsed.query: + target = f"{target}?{parsed.query}" + connection.request( + "GET", + target, + headers={ + "Host": _host_header(parsed), + "User-Agent": "x-financial-onlyoffice", + "Accept": "application/octet-stream, application/zip, " + "application/vnd.openxmlformats-officedocument.*", + }, + ) + response = connection.getresponse() + if response.status != 200: + raise OnlyOfficeSecurityError( + f"ONLYOFFICE 文档下载失败，状态码 {response.status}。" + ) + max_bytes = _max_download_bytes() + declared_size = _safe_int(response.getheader("Content-Length")) + if declared_size is not None and declared_size > max_bytes: + raise OnlyOfficeSecurityError("ONLYOFFICE 文档超过允许的大小。") + content_type = str(response.getheader("Content-Type") or "application/octet-stream") + content_type = content_type.partition(";")[0].strip().lower() + _validate_download_mime(content_type, expected_filename) + content = response.read(max_bytes + 1) + if len(content) > max_bytes: + raise OnlyOfficeSecurityError("ONLYOFFICE 文档超过允许的大小。") + if not content: + raise OnlyOfficeSecurityError("ONLYOFFICE 返回了空文档。") + _validate_ooxml_content(content, expected_filename, max_bytes=max_bytes) + return content + except (OSError, http.client.HTTPException) as exc: + raise OnlyOfficeSecurityError("ONLYOFFICE 文档下载连接失败。") from exc + finally: + connection.close() + + +def _validate_download_target(download_url: str) -> tuple[SplitResult, str]: + parsed = urlsplit(str(download_url or "").strip()) + if parsed.scheme not in {"http", "https"} or not parsed.hostname: + raise OnlyOfficeSecurityError("ONLYOFFICE 下载 URL 协议或主机不合法。") + if parsed.username or parsed.password or parsed.fragment: + raise OnlyOfficeSecurityError("ONLYOFFICE 下载 URL 不允许包含用户信息或片段。") + origin = _normalized_origin(parsed) + if origin not in _allowed_download_origins(): + raise OnlyOfficeSecurityError("ONLYOFFICE 下载 URL 不属于已配置的文档服务。") + + try: + addresses = { + str(item[4][0]) + for item in socket.getaddrinfo( + parsed.hostname, + parsed.port or _default_port(parsed.scheme), + type=socket.SOCK_STREAM, + ) + if item[4] + } + except OSError as exc: + raise OnlyOfficeSecurityError("ONLYOFFICE 下载主机无法解析。") from exc + if not addresses: + raise OnlyOfficeSecurityError("ONLYOFFICE 下载主机没有可用地址。") + if any(not ipaddress.ip_address(address).is_global for address in addresses): + raise OnlyOfficeSecurityError("ONLYOFFICE 下载主机解析到了非公网地址。") + return parsed, sorted(addresses)[0] + + +class _PinnedHTTPSConnection(http.client.HTTPSConnection): + def __init__(self, hostname: str, resolved_ip: str, port: int, timeout: int) -> None: + super().__init__(hostname, port=port, timeout=timeout, context=ssl.create_default_context()) + self._resolved_ip = resolved_ip + + def connect(self) -> None: + raw_socket = socket.create_connection( + (self._resolved_ip, self.port), + timeout=self.timeout, + source_address=self.source_address, + ) + self.sock = self._context.wrap_socket(raw_socket, server_hostname=self.host) + + +def _open_pinned_connection( + parsed: SplitResult, + resolved_ip: str, +) -> http.client.HTTPConnection: + port = parsed.port or _default_port(parsed.scheme) + if parsed.scheme == "https": + return _PinnedHTTPSConnection(parsed.hostname or "", resolved_ip, port, timeout=20) + return http.client.HTTPConnection(resolved_ip, port=port, timeout=20) + + +def _allowed_download_origins() -> set[str]: + configured = str(os.environ.get("ONLYOFFICE_DOWNLOAD_ALLOWED_ORIGINS") or "").strip() + raw_origins = [item.strip() for item in configured.split(",") if item.strip()] + if not raw_origins: + public_url = str(resolve_onlyoffice_settings().public_url or "").strip() + raw_origins = [public_url] if public_url else [] + origins: set[str] = set() + for raw_origin in raw_origins: + parsed = urlsplit(raw_origin) + if parsed.scheme in {"http", "https"} and parsed.hostname: + origins.add(_normalized_origin(parsed)) + if not origins: + raise OnlyOfficeSecurityError("未配置 ONLYOFFICE 安全下载来源。") + return origins + + +def _normalized_origin(parsed: SplitResult) -> str: + hostname = str(parsed.hostname or "").encode("idna").decode("ascii").lower() + port = parsed.port or _default_port(parsed.scheme) + return f"{parsed.scheme.lower()}://{hostname}:{port}" + + +def _host_header(parsed: SplitResult) -> str: + hostname = str(parsed.hostname or "") + port = parsed.port or _default_port(parsed.scheme) + if port == _default_port(parsed.scheme): + return hostname + return f"{hostname}:{port}" + + +def _validate_download_mime(content_type: str, expected_filename: str) -> None: + extension = expected_filename.rsplit(".", maxsplit=1)[-1].lower() + allowed = {_OOXML_MIME_TYPES.get(extension, ""), *_GENERIC_BINARY_MIME_TYPES} + if content_type not in allowed: + raise OnlyOfficeSecurityError("ONLYOFFICE 返回的文档 MIME 类型不合法。") + + +def _validate_ooxml_content(content: bytes, expected_filename: str, *, max_bytes: int) -> None: + extension = expected_filename.rsplit(".", maxsplit=1)[-1].lower() + required_prefix = _OOXML_REQUIRED_ENTRY.get(extension) + if required_prefix is None: + raise OnlyOfficeSecurityError("ONLYOFFICE 回写文件格式不受支持。") + try: + with zipfile.ZipFile(BytesIO(content)) as archive: + members = archive.infolist() + if len(members) > 5000: + raise OnlyOfficeSecurityError("ONLYOFFICE 文档包含过多压缩条目。") + total_uncompressed = sum(int(member.file_size or 0) for member in members) + if total_uncompressed > min(max_bytes * 20, 300 * 1024 * 1024): + raise OnlyOfficeSecurityError("ONLYOFFICE 文档解压体积超过限制。") + names = [member.filename.replace("\\", "/") for member in members] + if any( + name.startswith("/") or ".." in name.split("/") or member.flag_bits & 0x1 + for name, member in zip(names, members, strict=True) + ): + raise OnlyOfficeSecurityError("ONLYOFFICE 文档压缩结构不安全。") + if "[Content_Types].xml" not in names or not any( + name.startswith(required_prefix) for name in names + ): + raise OnlyOfficeSecurityError("ONLYOFFICE 文档内容与扩展名不匹配。") + except (zipfile.BadZipFile, RuntimeError) as exc: + raise OnlyOfficeSecurityError("ONLYOFFICE 返回的文档不是有效 OOXML 文件。") from exc + + +def _content_token_ttl_seconds() -> int: + configured = _safe_int(os.environ.get("ONLYOFFICE_CONTENT_TOKEN_TTL_SECONDS")) + return min( + 900, + max(60, configured or DEFAULT_CONTENT_TOKEN_TTL_SECONDS), + ) + + +def _callback_session_ttl_seconds() -> int: + configured = _safe_int( + os.environ.get("ONLYOFFICE_CALLBACK_SESSION_TTL_SECONDS") + or os.environ.get("ONLYOFFICE_SESSION_TTL_SECONDS") + ) + return min( + MAX_CALLBACK_SESSION_TTL_SECONDS, + max(15 * 60, configured or DEFAULT_CALLBACK_SESSION_TTL_SECONDS), + ) + + +def _max_download_bytes() -> int: + configured = _safe_int(os.environ.get("ONLYOFFICE_MAX_DOWNLOAD_BYTES")) + return min( + MAX_DOWNLOAD_BYTES_CEILING, + max(1024, configured or DEFAULT_MAX_DOWNLOAD_BYTES), + ) + + +def _safe_int(value: object) -> int | None: + try: + return int(str(value).strip()) + except (TypeError, ValueError): + return None + + +def _default_port(scheme: str) -> int: + return 443 if scheme == "https" else 80 + + +def _as_utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) + + +def _build_document_key(entry: dict[str, Any]) -> str: + version = int(entry.get("version_number") or 1) + checksum = str(entry.get("sha256") or "")[:12] + return f"{entry['id']}-v{version}-{checksum or 'nochecksum'}" diff --git a/server/src/app/services/knowledge_rag.py b/server/src/app/services/knowledge_rag.py index 2d36fd9..e6eae14 100644 --- a/server/src/app/services/knowledge_rag.py +++ b/server/src/app/services/knowledge_rag.py @@ -1,12 +1,12 @@ from __future__ import annotations import os -import re import socket import threading +from collections.abc import Callable from concurrent.futures import ThreadPoolExecutor from pathlib import Path -from typing import Any, Callable +from typing import Any from sqlalchemy.orm import Session @@ -24,82 +24,62 @@ from app.services.knowledge_rag_runtime import ( RuntimeModelConfig, _LightRagRuntime, ) +from app.services.knowledge_rag_scoring import ( + MAX_KNOWLEDGE_HIT_CONTENT_LENGTH, + MAX_KNOWLEDGE_HIT_EXCERPT_LENGTH, + TABLE_OR_STANDARD_QUERY_HINTS, +) +from app.services.knowledge_rag_scoring import ( + build_query_focused_excerpt as _build_query_focused_excerpt, +) +from app.services.knowledge_rag_scoring import ( + extract_query_terms as _extract_query_terms, +) +from app.services.knowledge_rag_scoring import ( + parse_document_identity as _parse_document_identity, +) +from app.services.knowledge_rag_scoring import ( + score_knowledge_hit as _score_knowledge_hit, +) +from app.services.knowledge_rag_scoring import ( + truncate_text as _truncate_text, +) +from app.services.knowledge_tenant_scope import ( + PLATFORM_KNOWLEDGE_SCOPE, + TENANT_KNOWLEDGE_SCOPE, + KnowledgeStorageScope, +) from app.services.settings import SettingsService logger = get_logger("app.services.knowledge_rag") DEFAULT_QDRANT_URL = "http://127.0.0.1:6333" CONTAINER_QDRANT_URL = "http://qdrant:6333" -DEFAULT_LIGHTRAG_WORKSPACE = "x_financial_knowledge" -MAX_KNOWLEDGE_HIT_CONTENT_LENGTH = 2200 -MAX_KNOWLEDGE_HIT_EXCERPT_LENGTH = 220 -MAX_QUERY_TERMS = 12 -QUERY_TERM_STOPWORDS = { - "什么", - "多少", - "哪些", - "怎么", - "如何", - "请问", - "一下", - "关于", - "规定", - "标准", - "可以", - "是否", - "一个", - "哪些人", -} -TABLE_OR_STANDARD_QUERY_HINTS = ( - "表", - "表格", - "清单", - "明细", - "目录", - "科目", - "标准", - "金额", - "限额", - "补贴", - "住宿", - "餐费", - "交通", - "报销", - "档位", - "额度", -) -QUERY_ANCHOR_TERMS = ( - "财务基础知识手册", - "基础知识手册", - "会计科目", - "常用会计科目", - "财务报表", - "主要税种", - "税种", - "标准", - "清单", - "明细", - "流程", -) -GENERIC_TITLE_TERMS = {"远光软件", "股份有限", "有限公司"} -STRUCTURED_APPENDIX_LEADING_MARKERS = ( - "# 章节导航", - "# 重点章节摘录", - "# 问答线索补充", - "# 结构化表格补充", -) -STRUCTURED_APPENDIX_LEADING_WINDOW = 220 _runtime_lock = threading.RLock() _runtime_executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="knowledge-rag-runtime") _runtime_instances: dict[str, _LightRagRuntime] = {} _runtime_signatures: dict[str, tuple[Any, ...]] = {} -_RUNTIME_CACHE_KEY = "lightrag" class KnowledgeRagService: - def __init__(self, db: Session | None = None, storage_root: Path | None = None) -> None: + def __init__( + self, + db: Session | None = None, + storage_root: Path | None = None, + *, + tenant_id: str | None = None, + scope: str | None = None, + ) -> None: self.db = db self.storage_root = Path(storage_root or get_settings().resolved_storage_root_dir) + if scope == PLATFORM_KNOWLEDGE_SCOPE: + self.storage_scope = KnowledgeStorageScope.platform(self.storage_root) + elif scope in {None, TENANT_KNOWLEDGE_SCOPE} and tenant_id is not None: + self.storage_scope = KnowledgeStorageScope.tenant(self.storage_root, tenant_id) + else: + raise ValueError("知识检索必须显式提供可信 tenant_id 或 platform scope。") + self.scope = self.storage_scope.scope + self.tenant_id = self.storage_scope.tenant_id def query_knowledge( self, @@ -123,13 +103,9 @@ class KnowledgeRagService: if conversation_history: rewritten_query = self._rewrite_query(normalized_query, conversation_history) - workspace = ( - os.environ.get("LIGHTRAG_WORKSPACE", DEFAULT_LIGHTRAG_WORKSPACE).strip() - or DEFAULT_LIGHTRAG_WORKSPACE - ) local_result = query_local_text_chunks( - lightrag_root=(self.storage_root / "knowledge" / ".lightrag").resolve(), - workspace=workspace, + lightrag_root=self.storage_scope.lightrag_root, + workspace=self.storage_scope.workspace, query=rewritten_query, limit=limit, ) @@ -147,7 +123,9 @@ class KnowledgeRagService: data = raw.get("data") if isinstance(raw, dict) else {} chunks = list(data.get("chunks") or []) if isinstance(data, dict) else [] entities = list(data.get("entities") or []) if isinstance(data, dict) else [] - runtime_references = list(data.get("references") or []) if isinstance(data, dict) else [] + runtime_references = ( + list(data.get("references") or []) if isinstance(data, dict) else [] + ) runtime_hits = self._build_hits_from_query_data( query=rewritten_query, chunks=chunks, @@ -165,14 +143,21 @@ class KnowledgeRagService: for hit in runtime_hits: code = hit["code"] if code in all_hits: - all_hits[code]["score"] = max(all_hits[code]["score"], int(hit.get("score") or 0) + 20) + all_hits[code]["score"] = max( + all_hits[code]["score"], + int(hit.get("score") or 0) + 20, + ) if not all_hits[code].get("tags") and hit.get("tags"): all_hits[code]["tags"] = hit["tags"] else: hit["score"] = int(hit.get("score") or 0) all_hits[code] = hit - merged_hits = sorted(all_hits.values(), key=lambda x: int(x.get("score") or 0), reverse=True)[:max(1, limit)] + merged_hits = sorted( + all_hits.values(), + key=lambda item: int(item.get("score") or 0), + reverse=True, + )[: max(1, limit)] if not merged_hits: return { @@ -211,9 +196,23 @@ class KnowledgeRagService: from app.services.runtime_chat import RuntimeChatService try: chat_service = RuntimeChatService(self.db) - messages: list[dict[str, Any]] = [{"role": "system", "content": "你是一个查询重写助手。你的任务是根据用户的多轮对话历史，将用户的最后一次提问重写为一句独立、完整的查询语句，以便于在知识库中进行向量检索。只输出重写后的句子，不要任何解释。"}] + messages: list[dict[str, Any]] = [ + { + "role": "system", + "content": ( + "你是一个查询重写助手。请根据用户的多轮对话历史，将最后一次提问" + "重写为一句独立、完整、可用于知识库向量检索的查询语句。" + "只输出重写后的句子，不要解释。" + ), + } + ] for msg in conversation_history[-6:]: - messages.append({"role": msg.get("role", "user"), "content": msg.get("content", "")}) + messages.append( + { + "role": msg.get("role", "user"), + "content": msg.get("content", ""), + } + ) messages.append({"role": "user", "content": f"当前提问：{query}\n\n请重写当前提问。"}) rewritten = chat_service.complete( @@ -244,7 +243,12 @@ class KnowledgeRagService: from app.services.knowledge import KnowledgeService from app.services.knowledge_normalizer import KnowledgeNormalizationService - knowledge_service = KnowledgeService(storage_root=self.storage_root, db=self.db) + knowledge_service = KnowledgeService( + storage_root=self.storage_root, + db=self.db, + tenant_id=self.tenant_id, + scope=self.scope, + ) normalization_service = ( KnowledgeNormalizationService(self.db) if self.db is not None else None ) @@ -311,14 +315,11 @@ class KnowledgeRagService: status_obj = statuses.get(document_id) status_text = self._status_value(status_obj) status_payload = self._serialize_status(status_obj) - workspace = ( - os.environ.get("LIGHTRAG_WORKSPACE", DEFAULT_LIGHTRAG_WORKSPACE).strip() - or DEFAULT_LIGHTRAG_WORKSPACE - ) graph_summary = build_document_graph_summary( self.storage_root, - workspace=workspace, + workspace=self.storage_scope.workspace, document_id=document_id, + lightrag_root=self.storage_scope.lightrag_root, ) if document_id in summary_by_id: summary_by_id[document_id].update( @@ -407,8 +408,9 @@ class KnowledgeRagService: if signature is None or runtime_kwargs is None: signature, runtime_kwargs = self._build_runtime_signature() with _runtime_lock: - runtime = _runtime_instances.get(_RUNTIME_CACHE_KEY) - if runtime is not None and _runtime_signatures.get(_RUNTIME_CACHE_KEY) == signature: + cache_key = self.storage_scope.runtime_cache_key + runtime = _runtime_instances.get(cache_key) + if runtime is not None and _runtime_signatures.get(cache_key) == signature: return runtime if runtime is not None: @@ -418,18 +420,15 @@ class KnowledgeRagService: logger.warning("Finalize previous LightRAG runtime failed: %s", exc) runtime = _LightRagRuntime(**runtime_kwargs) - _runtime_instances[_RUNTIME_CACHE_KEY] = runtime - _runtime_signatures[_RUNTIME_CACHE_KEY] = signature + _runtime_instances[cache_key] = runtime + _runtime_signatures[cache_key] = signature return runtime def _build_runtime_signature(self) -> tuple[tuple[Any, ...], dict[str, Any]]: configs = self._load_runtime_configs() settings = get_settings() - working_dir = (self.storage_root / "knowledge" / ".lightrag").resolve() - workspace = ( - os.environ.get("LIGHTRAG_WORKSPACE", DEFAULT_LIGHTRAG_WORKSPACE).strip() - or DEFAULT_LIGHTRAG_WORKSPACE - ) + working_dir = self.storage_scope.lightrag_root.resolve() + workspace = self.storage_scope.workspace qdrant_url = os.environ.get("QDRANT_URL", "").strip() or _resolve_default_qdrant_url() qdrant_api_key = os.environ.get("QDRANT_API_KEY", "").strip() @@ -688,56 +687,6 @@ def _shutdown_runtime_instances() -> None: _runtime_signatures.clear() -def _parse_document_identity(file_path: str) -> tuple[str, str]: - path = Path(str(file_path or "").strip()) - name = path.name - if "__" not in name: - return "", name - document_id, document_name = name.split("__", maxsplit=1) - return document_id.strip(), document_name.strip() - - -def _build_excerpt(text: str, *, max_length: int = 180) -> str: - normalized = " ".join(str(text or "").split()).strip() - if len(normalized) <= max_length: - return normalized - return f"{normalized[: max_length - 3].rstrip()}..." - - -def _build_query_focused_excerpt( - text: str, - *, - query_terms: list[str], - max_length: int = 180, -) -> str: - normalized = " ".join(str(text or "").split()).strip() - if not normalized: - return "" - - lowered = normalized.lower() - match_positions = [ - lowered.find(term) for term in query_terms if term and lowered.find(term) >= 0 - ] - if not match_positions: - return _build_excerpt(normalized, max_length=max_length) - - start = max(0, min(match_positions) - max_length // 3) - end = min(len(normalized), start + max_length) - snippet = normalized[start:end].strip() - if start > 0: - snippet = f"...{snippet.lstrip()}" - if end < len(normalized): - snippet = f"{snippet.rstrip()}..." - return snippet - - -def _truncate_text(text: str, *, max_length: int) -> str: - normalized = str(text or "").strip() - if len(normalized) <= max_length: - return normalized - return f"{normalized[: max_length - 3].rstrip()}..." - - def _resolve_default_qdrant_url() -> str: if _hostname_resolves("qdrant"): return CONTAINER_QDRANT_URL @@ -750,128 +699,3 @@ def _hostname_resolves(hostname: str) -> bool: except OSError: return False return True - - -def _extract_query_terms(query: str) -> list[str]: - normalized_query = str(query or "").strip().lower() - if not normalized_query: - return [] - - terms: list[str] = [] - seen: set[str] = set() - - def remember(term: str) -> None: - normalized_term = str(term or "").strip().lower() - if ( - not normalized_term - or normalized_term in seen - or normalized_term in QUERY_TERM_STOPWORDS - or len(normalized_term) < 2 - ): - return - seen.add(normalized_term) - terms.append(normalized_term) - - for item in re.findall(r"[a-z0-9][a-z0-9_\-]{1,}", normalized_query): - remember(item) - - for block in re.findall(r"[\u4e00-\u9fff]{2,20}", normalized_query): - for marker in ("标准", "金额", "限额", "额度"): - marker_index = block.find(marker) - if marker_index <= 0: - continue - subject = block[:marker_index] - for width in (6, 4, 3, 2): - remember(subject[-width:]) - for anchor in QUERY_ANCHOR_TERMS: - if anchor in block: - remember(anchor) - tail = block[-14:] - for size in (8, 7, 6, 5, 4): - for start in range(0, len(tail) - size + 1): - piece = tail[start : start + size] - if any(anchor in piece for anchor in QUERY_ANCHOR_TERMS): - remember(piece) - if len(terms) >= MAX_QUERY_TERMS: - return terms - if len(block) <= 4: - remember(block) - continue - for size in (4, 3, 2): - for start in range(0, len(block) - size + 1): - remember(block[start : start + size]) - if len(terms) >= MAX_QUERY_TERMS: - return terms - - return terms[:MAX_QUERY_TERMS] - - -def _score_knowledge_hit( - item: dict[str, Any], - *, - query_terms: list[str], - prefers_tabular_evidence: bool, -) -> int: - rank = max(1, int(item.get("_rank") or 1)) - title = str(item.get("title") or item.get("document_name") or "").lower() - content = str(item.get("content") or "").lower() - excerpt = str(item.get("excerpt") or "").lower() - tags = " ".join(str(value).lower() for value in list(item.get("tags") or [])[:5]) - haystack = "\n".join([title, excerpt, tags, content[:1200]]) - - score = max(1, 120 - rank * 4) - matched_terms = [term for term in query_terms if term in haystack] - score += len(matched_terms) * 8 - score += sum(1 for term in matched_terms if term in title) * 6 - score += sum( - (len(term) - 3) * 12 - for term in matched_terms - if len(term) >= 4 and term in title and term not in GENERIC_TITLE_TERMS - ) - - leading_appendix_marker = _leading_structured_appendix_marker(content) - if leading_appendix_marker == "# 章节导航": - score -= 24 - elif leading_appendix_marker == "# 重点章节摘录": - score += 4 if matched_terms else -12 - elif leading_appendix_marker == "# 问答线索补充": - score += ( - 8 if matched_terms and not prefers_tabular_evidence else 2 if matched_terms else -20 - ) - elif leading_appendix_marker == "# 结构化表格补充": - if prefers_tabular_evidence and matched_terms: - score += 16 - elif matched_terms: - score += 6 - else: - score -= 18 - - if prefers_tabular_evidence and matched_terms and ("|" in content or "表" in content): - score += 10 - if matched_terms and any(marker in content for marker in ("：", ":")): - score += 10 - if matched_terms and "\n" in content: - score += 4 - if matched_terms and any(marker in content for marker in ("附表", "第", "条")): - score += 4 - if ( - not prefers_tabular_evidence - and matched_terms - and any(marker in content for marker in ("第", "条", "：", "-", "•")) - ): - score += 4 - if title and any(term in title for term in query_terms): - score += 6 - if re.search(r"没有.{0,8}(信息|规定|说明|依据)", content): - score -= 12 - - return score - - -def _leading_structured_appendix_marker(content: str) -> str: - normalized = str(content or "").lstrip() - for marker in STRUCTURED_APPENDIX_LEADING_MARKERS: - index = normalized.find(marker) - if 0 <= index <= STRUCTURED_APPENDIX_LEADING_WINDOW: - return marker - return "" diff --git a/server/src/app/services/knowledge_rag_scoring.py b/server/src/app/services/knowledge_rag_scoring.py new file mode 100644 index 0000000..5c5eb16 --- /dev/null +++ b/server/src/app/services/knowledge_rag_scoring.py @@ -0,0 +1,230 @@ +from __future__ import annotations + +import re +from pathlib import Path +from typing import Any + +MAX_KNOWLEDGE_HIT_CONTENT_LENGTH = 2200 +MAX_KNOWLEDGE_HIT_EXCERPT_LENGTH = 220 +MAX_QUERY_TERMS = 12 +QUERY_TERM_STOPWORDS = { + "什么", + "多少", + "哪些", + "怎么", + "如何", + "请问", + "一下", + "关于", + "规定", + "标准", + "可以", + "是否", + "一个", + "哪些人", +} +TABLE_OR_STANDARD_QUERY_HINTS = ( + "表", + "表格", + "清单", + "明细", + "目录", + "科目", + "标准", + "金额", + "限额", + "补贴", + "住宿", + "餐费", + "交通", + "报销", + "档位", + "额度", +) +QUERY_ANCHOR_TERMS = ( + "财务基础知识手册", + "基础知识手册", + "会计科目", + "常用会计科目", + "财务报表", + "主要税种", + "税种", + "标准", + "清单", + "明细", + "流程", +) +GENERIC_TITLE_TERMS = {"远光软件", "股份有限", "有限公司"} +STRUCTURED_APPENDIX_LEADING_MARKERS = ( + "# 章节导航", + "# 重点章节摘录", + "# 问答线索补充", + "# 结构化表格补充", +) +STRUCTURED_APPENDIX_LEADING_WINDOW = 220 + + +def parse_document_identity(file_path: str) -> tuple[str, str]: + path = Path(str(file_path or "").strip()) + name = path.name + if "__" not in name: + return "", name + document_id, document_name = name.split("__", maxsplit=1) + return document_id.strip(), document_name.strip() + + +def build_query_focused_excerpt( + text: str, + *, + query_terms: list[str], + max_length: int = 180, +) -> str: + normalized = " ".join(str(text or "").split()).strip() + if not normalized: + return "" + lowered = normalized.lower() + match_positions = [ + lowered.find(term) for term in query_terms if term and lowered.find(term) >= 0 + ] + if not match_positions: + return build_excerpt(normalized, max_length=max_length) + start = max(0, min(match_positions) - max_length // 3) + end = min(len(normalized), start + max_length) + snippet = normalized[start:end].strip() + if start > 0: + snippet = f"...{snippet.lstrip()}" + if end < len(normalized): + snippet = f"{snippet.rstrip()}..." + return snippet + + +def build_excerpt(text: str, *, max_length: int = 180) -> str: + normalized = " ".join(str(text or "").split()).strip() + if len(normalized) <= max_length: + return normalized + return f"{normalized[: max_length - 3].rstrip()}..." + + +def truncate_text(text: str, *, max_length: int) -> str: + normalized = str(text or "").strip() + if len(normalized) <= max_length: + return normalized + return f"{normalized[: max_length - 3].rstrip()}..." + + +def extract_query_terms(query: str) -> list[str]: + normalized_query = str(query or "").strip().lower() + if not normalized_query: + return [] + terms: list[str] = [] + seen: set[str] = set() + + def remember(term: str) -> None: + normalized_term = str(term or "").strip().lower() + if ( + not normalized_term + or normalized_term in seen + or normalized_term in QUERY_TERM_STOPWORDS + or len(normalized_term) < 2 + ): + return + seen.add(normalized_term) + terms.append(normalized_term) + + for item in re.findall(r"[a-z0-9][a-z0-9_\-]{1,}", normalized_query): + remember(item) + for block in re.findall(r"[\u4e00-\u9fff]{2,20}", normalized_query): + for marker in ("标准", "金额", "限额", "额度"): + marker_index = block.find(marker) + if marker_index <= 0: + continue + subject = block[:marker_index] + for width in (6, 4, 3, 2): + remember(subject[-width:]) + for anchor in QUERY_ANCHOR_TERMS: + if anchor in block: + remember(anchor) + tail = block[-14:] + for size in (8, 7, 6, 5, 4): + for start in range(0, len(tail) - size + 1): + piece = tail[start : start + size] + if any(anchor in piece for anchor in QUERY_ANCHOR_TERMS): + remember(piece) + if len(terms) >= MAX_QUERY_TERMS: + return terms + if len(block) <= 4: + remember(block) + continue + for size in (4, 3, 2): + for start in range(0, len(block) - size + 1): + remember(block[start : start + size]) + if len(terms) >= MAX_QUERY_TERMS: + return terms + return terms[:MAX_QUERY_TERMS] + + +def score_knowledge_hit( + item: dict[str, Any], + *, + query_terms: list[str], + prefers_tabular_evidence: bool, +) -> int: + rank = max(1, int(item.get("_rank") or 1)) + title = str(item.get("title") or item.get("document_name") or "").lower() + content = str(item.get("content") or "").lower() + excerpt = str(item.get("excerpt") or "").lower() + tags = " ".join(str(value).lower() for value in list(item.get("tags") or [])[:5]) + haystack = "\n".join([title, excerpt, tags, content[:1200]]) + + score = max(1, 120 - rank * 4) + matched_terms = [term for term in query_terms if term in haystack] + score += len(matched_terms) * 8 + score += sum(1 for term in matched_terms if term in title) * 6 + score += sum( + (len(term) - 3) * 12 + for term in matched_terms + if len(term) >= 4 and term in title and term not in GENERIC_TITLE_TERMS + ) + leading_marker = _leading_structured_appendix_marker(content) + if leading_marker == "# 章节导航": + score -= 24 + elif leading_marker == "# 重点章节摘录": + score += 4 if matched_terms else -12 + elif leading_marker == "# 问答线索补充": + if matched_terms: + score += 8 if not prefers_tabular_evidence else 2 + else: + score -= 20 + elif leading_marker == "# 结构化表格补充": + if matched_terms: + score += 16 if prefers_tabular_evidence else 6 + else: + score -= 18 + if prefers_tabular_evidence and matched_terms and ("|" in content or "表" in content): + score += 10 + if matched_terms and any(marker in content for marker in ("：", ":")): + score += 10 + if matched_terms and "\n" in content: + score += 4 + if matched_terms and any(marker in content for marker in ("附表", "第", "条")): + score += 4 + if ( + not prefers_tabular_evidence + and matched_terms + and any(marker in content for marker in ("第", "条", "：", "-", "•")) + ): + score += 4 + if title and any(term in title for term in query_terms): + score += 6 + if re.search(r"没有.{0,8}(信息|规定|说明|依据)", content): + score -= 12 + return score + + +def _leading_structured_appendix_marker(content: str) -> str: + normalized = str(content or "").lstrip() + for marker in STRUCTURED_APPENDIX_LEADING_MARKERS: + index = normalized.find(marker) + if 0 <= index <= STRUCTURED_APPENDIX_LEADING_WINDOW: + return marker + return "" diff --git a/server/src/app/services/knowledge_run_scope.py b/server/src/app/services/knowledge_run_scope.py new file mode 100644 index 0000000..3fbf9c5 --- /dev/null +++ b/server/src/app/services/knowledge_run_scope.py @@ -0,0 +1,23 @@ +from __future__ import annotations + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.agent_run import AgentRun +from app.services.knowledge_tenant_scope import require_knowledge_tenant_id + + +def resolve_trusted_knowledge_run_tenant(db: Session, agent_run_id: str) -> str: + """只从持久化 Agent Run 还原后台知识任务的可信租户。""" + run = db.scalar(select(AgentRun).where(AgentRun.run_id == agent_run_id)) + if run is None: + raise LookupError("知识索引任务关联的 Agent Run 不存在。") + route_tenant_id = require_knowledge_tenant_id( + (run.route_json or {}).get("tenant_id") + ) + ontology_tenant_id = require_knowledge_tenant_id( + (run.ontology_json or {}).get("tenant_id") + ) + if route_tenant_id != ontology_tenant_id: + raise ValueError("知识索引任务的 Agent Run 租户上下文冲突。") + return route_tenant_id diff --git a/server/src/app/services/knowledge_scheduler.py b/server/src/app/services/knowledge_scheduler.py index 423c07a..b22a8c7 100644 --- a/server/src/app/services/knowledge_scheduler.py +++ b/server/src/app/services/knowledge_scheduler.py @@ -5,10 +5,13 @@ import threading from datetime import datetime, time, timedelta from zoneinfo import ZoneInfo +from sqlalchemy import select + from app.api.deps import CurrentUserContext from app.core.agent_enums import AgentRunSource from app.core.logging import get_logger from app.db.session import get_session_factory +from app.models.tenant import Tenant from app.services.knowledge_sync import KnowledgeSyncDispatchService logger = get_logger("app.services.knowledge_scheduler") @@ -16,7 +19,10 @@ logger = get_logger("app.services.knowledge_scheduler") class KnowledgeIndexScheduler: def __init__(self) -> None: - timezone_name = str(os.environ.get("X_FINANCIAL_SCHEDULER_TZ") or "Asia/Shanghai").strip() or "Asia/Shanghai" + timezone_name = ( + str(os.environ.get("X_FINANCIAL_SCHEDULER_TZ") or "Asia/Shanghai").strip() + or "Asia/Shanghai" + ) self._timezone = ZoneInfo(timezone_name) self._stop_event = threading.Event() self._thread: threading.Thread | None = None @@ -33,7 +39,10 @@ class KnowledgeIndexScheduler: daemon=True, ) self._thread.start() - logger.info("Knowledge index scheduler started timezone=%s trigger=00:00", self._timezone.key) + logger.info( + "Knowledge index scheduler started timezone=%s trigger=00:00", + self._timezone.key, + ) def shutdown(self) -> None: with self._lock: @@ -59,27 +68,50 @@ class KnowledgeIndexScheduler: def _run_incremental_sync(self) -> None: db = get_session_factory()() try: - current_user = CurrentUserContext( - username="hermes", - name="Hermes", - role_codes=["manager"], - is_admin=True, - ) - result = KnowledgeSyncDispatchService(db).queue_sync( - current_user=current_user, - folder=None, - document_ids=None, - source=AgentRunSource.SCHEDULE.value, - force=False, - changed_only=True, - ) - logger.info( - "Scheduled knowledge index sync result run_id=%s docs=%s reused=%s summary=%s", - result.agent_run_id, - len(result.document_ids), - result.reused, - result.summary, + tenants = list( + db.scalars( + select(Tenant) + .where(Tenant.status == "active") + .order_by(Tenant.tenant_id.asc()) + ) ) + if not tenants: + logger.warning( + "Scheduled knowledge index sync skipped: no active tenant registry rows" + ) + return + for tenant in tenants: + current_user = CurrentUserContext( + username="hermes", + name="Hermes", + role_codes=["manager"], + is_admin=True, + tenant_id=tenant.tenant_id, + ) + try: + result = KnowledgeSyncDispatchService(db).queue_sync( + current_user=current_user, + folder=None, + document_ids=None, + source=AgentRunSource.SCHEDULE.value, + force=False, + changed_only=True, + ) + logger.info( + "Scheduled knowledge index sync result tenant=%s run_id=%s " + "docs=%s reused=%s summary=%s", + tenant.tenant_id, + result.agent_run_id, + len(result.document_ids), + result.reused, + result.summary, + ) + except Exception: + db.rollback() + logger.exception( + "Scheduled knowledge index sync failed tenant=%s", + tenant.tenant_id, + ) finally: db.close() diff --git a/server/src/app/services/knowledge_sync.py b/server/src/app/services/knowledge_sync.py index 2b1b7ee..94e94f0 100644 --- a/server/src/app/services/knowledge_sync.py +++ b/server/src/app/services/knowledge_sync.py @@ -3,12 +3,12 @@ from __future__ import annotations from dataclasses import dataclass, field from datetime import UTC, datetime, timedelta -from sqlalchemy import select from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext from app.core.agent_enums import AgentName, AgentPermissionLevel, AgentRunSource, AgentRunStatus from app.models.agent_asset import AgentAsset +from app.services.agent_asset_access import platform_asset_statement from app.services.agent_foundation_constants import DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE from app.services.agent_runs import AgentRunService from app.services.knowledge import ( @@ -17,6 +17,7 @@ from app.services.knowledge import ( KnowledgeService, ) from app.services.knowledge_index_tasks import knowledge_index_task_manager +from app.services.knowledge_tenant_scope import require_knowledge_tenant_id ALL_KNOWLEDGE_FOLDERS_LABEL = "全部知识库" @@ -37,7 +38,7 @@ class KnowledgeSyncDispatchService: def __init__(self, db: Session) -> None: self.db = db self.run_service = AgentRunService(db) - self.knowledge_service = KnowledgeService(db=db) + self.knowledge_service: KnowledgeService | None = None def queue_sync( self, @@ -49,6 +50,13 @@ class KnowledgeSyncDispatchService: force: bool = False, changed_only: bool = True, ) -> KnowledgeSyncDispatchResult: + tenant_id = require_knowledge_tenant_id(current_user.tenant_id) + if ( + self.knowledge_service is None + or self.knowledge_service.tenant_id != tenant_id + ): + self.knowledge_service = KnowledgeService(db=self.db, tenant_id=tenant_id) + knowledge_service = self.knowledge_service normalized_folder = str(folder or "").strip() or None folder_label = normalized_folder or ALL_KNOWLEDGE_FOLDERS_LABEL normalized_requested_ids = [ @@ -57,12 +65,12 @@ class KnowledgeSyncDispatchService: if str(item).strip() ] - all_documents = self.knowledge_service.list_documents_for_ingest( + all_documents = knowledge_service.list_documents_for_ingest( folder=normalized_folder, document_ids=normalized_requested_ids, changed_only=False, ) - target_documents = self.knowledge_service.list_documents_for_ingest( + target_documents = knowledge_service.list_documents_for_ingest( folder=normalized_folder, document_ids=normalized_requested_ids, changed_only=(False if force else changed_only), @@ -90,13 +98,17 @@ class KnowledgeSyncDispatchService: ) active_run = self._find_active_run( + tenant_id=tenant_id, folder=folder_label, requested_document_ids=target_document_ids, ) if active_run is not None: active_document_ids = [ str(item).strip() - for item in list(active_run.route_json.get("requested_document_ids") or target_document_ids) + for item in list( + active_run.route_json.get("requested_document_ids") + or target_document_ids + ) if str(item).strip() ] return KnowledgeSyncDispatchResult( @@ -110,11 +122,14 @@ class KnowledgeSyncDispatchService: ) task_asset = self.db.scalar( - select(AgentAsset).where(AgentAsset.code == DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE) + platform_asset_statement().where( + AgentAsset.code == DIGITAL_EMPLOYEE_FINANCE_POLICY_TASK_CODE + ) ) run = self.run_service.create_run( agent=AgentName.HERMES.value, source=source, + tenant_id=tenant_id, user_id=current_user.username, task_id=task_asset.id if task_asset is not None else None, permission_level=AgentPermissionLevel.READ.value, @@ -140,7 +155,7 @@ class KnowledgeSyncDispatchService: ) try: - self.knowledge_service.set_document_ingest_statuses( + knowledge_service.set_document_ingest_statuses( target_document_ids, status_code=KNOWLEDGE_INGEST_STATUS_SYNCING, agent_run_id=run.run_id, @@ -168,7 +183,7 @@ class KnowledgeSyncDispatchService: result_summary=str(exc), finished_at=datetime.now(UTC), ) - self.knowledge_service.set_document_ingest_statuses( + knowledge_service.set_document_ingest_statuses( target_document_ids, status_code=KNOWLEDGE_INGEST_STATUS_FAILED, agent_run_id=run.run_id, @@ -178,12 +193,17 @@ class KnowledgeSyncDispatchService: def _find_active_run( self, *, + tenant_id: str, folder: str, requested_document_ids: list[str], ): + knowledge_service = self.knowledge_service + if knowledge_service is None: + raise RuntimeError("知识同步服务尚未绑定租户 scope。") requested_set = {str(item).strip() for item in requested_document_ids if str(item).strip()} - for item in self.run_service.list_runs( + for item in self.run_service.list_runs_for_tenant( + tenant_id=tenant_id, agent=AgentName.HERMES.value, status=AgentRunStatus.RUNNING.value, limit=100, @@ -210,7 +230,7 @@ class KnowledgeSyncDispatchService: if str(document_id).strip() ] if stale_document_ids: - self.knowledge_service.set_document_ingest_statuses( + knowledge_service.set_document_ingest_statuses( stale_document_ids, status_code=KNOWLEDGE_INGEST_STATUS_FAILED, agent_run_id=item.run_id, diff --git a/server/src/app/services/knowledge_tenant_scope.py b/server/src/app/services/knowledge_tenant_scope.py new file mode 100644 index 0000000..8fdc9af --- /dev/null +++ b/server/src/app/services/knowledge_tenant_scope.py @@ -0,0 +1,124 @@ +from __future__ import annotations + +import hashlib +import os +import re +import shutil +from dataclasses import dataclass +from pathlib import Path + +from app.services.knowledge_constants import FIXED_KNOWLEDGE_FOLDERS + +PLATFORM_KNOWLEDGE_SCOPE = "platform" +TENANT_KNOWLEDGE_SCOPE = "tenant" +_TENANT_ID_PATTERN = re.compile(r"^[A-Za-z0-9](?:[A-Za-z0-9_.-]{0,62}[A-Za-z0-9])?$") + + +def require_knowledge_tenant_id(value: object) -> str: + tenant_id = str(value or "").strip() + if not tenant_id: + raise ValueError("知识库 tenant_id 不能为空。") + if not _TENANT_ID_PATTERN.fullmatch(tenant_id) or tenant_id in {".", ".."}: + raise ValueError("知识库 tenant_id 格式不合法。") + return tenant_id + + +def resolve_lightrag_workspace(*, tenant_id: str | None, scope: str) -> str: + base = str(os.environ.get("LIGHTRAG_WORKSPACE") or "x_financial_knowledge").strip() + base = re.sub(r"[^A-Za-z0-9_-]+", "_", base).strip("_") or "x_financial_knowledge" + if scope == PLATFORM_KNOWLEDGE_SCOPE: + suffix = "platform" + else: + normalized_tenant_id = require_knowledge_tenant_id(tenant_id) + suffix = f"tenant_{hashlib.sha256(normalized_tenant_id.encode('utf-8')).hexdigest()[:20]}" + return f"{base[:80]}__{suffix}" + + +@dataclass(frozen=True, slots=True) +class KnowledgeStorageScope: + storage_root: Path + scope: str + tenant_id: str | None + library_root: Path + index_path: Path + lightrag_root: Path + workspace: str + runtime_cache_key: str + read_only: bool + + @classmethod + def tenant(cls, storage_root: Path, tenant_id: object) -> KnowledgeStorageScope: + normalized_tenant_id = require_knowledge_tenant_id(tenant_id) + root = Path(storage_root).resolve() + library_root = root / "knowledge" / "tenants" / normalized_tenant_id + workspace = resolve_lightrag_workspace( + tenant_id=normalized_tenant_id, + scope=TENANT_KNOWLEDGE_SCOPE, + ) + return cls( + storage_root=root, + scope=TENANT_KNOWLEDGE_SCOPE, + tenant_id=normalized_tenant_id, + library_root=library_root, + index_path=library_root / ".index.json", + lightrag_root=library_root / ".lightrag", + workspace=workspace, + runtime_cache_key=f"tenant:{hashlib.sha256(normalized_tenant_id.encode('utf-8')).hexdigest()}", + read_only=False, + ) + + @classmethod + def platform(cls, storage_root: Path) -> KnowledgeStorageScope: + root = Path(storage_root).resolve() + library_root = root / "knowledge" / "platform" + workspace = resolve_lightrag_workspace( + tenant_id=None, + scope=PLATFORM_KNOWLEDGE_SCOPE, + ) + return cls( + storage_root=root, + scope=PLATFORM_KNOWLEDGE_SCOPE, + tenant_id=None, + library_root=library_root, + index_path=library_root / ".index.json", + lightrag_root=library_root / ".lightrag", + workspace=workspace, + runtime_cache_key=PLATFORM_KNOWLEDGE_SCOPE, + read_only=True, + ) + + +def migrate_legacy_library_to_platform(scope: KnowledgeStorageScope) -> bool: + """保留旧文件并复制到显式 platform 空间，避免升级时丢失现有制度资料。""" + if scope.scope != PLATFORM_KNOWLEDGE_SCOPE or scope.index_path.exists(): + return False + + legacy_root = scope.storage_root / "knowledge" + legacy_index = legacy_root / ".index.json" + has_legacy_documents = any( + (legacy_root / folder_name).is_dir() for folder_name in FIXED_KNOWLEDGE_FOLDERS + ) + if not legacy_index.exists() and not has_legacy_documents: + return False + + scope.library_root.mkdir(parents=True, exist_ok=True) + for folder_name in FIXED_KNOWLEDGE_FOLDERS: + source = legacy_root / folder_name + target = scope.library_root / folder_name + if source.is_dir(): + shutil.copytree(source, target, dirs_exist_ok=True) + + legacy_lightrag_root = legacy_root / ".lightrag" + if legacy_lightrag_root.is_dir(): + shutil.copytree( + legacy_lightrag_root, + scope.lightrag_root, + dirs_exist_ok=True, + ) + legacy_workspace = legacy_lightrag_root / scope.workspace.split("__", maxsplit=1)[0] + platform_workspace = scope.lightrag_root / scope.workspace + if legacy_workspace.is_dir() and not platform_workspace.exists(): + shutil.copytree(legacy_workspace, platform_workspace) + if legacy_index.exists(): + shutil.copy2(legacy_index, scope.index_path) + return True diff --git a/server/src/app/services/linked_reimbursement_draft_jobs.py b/server/src/app/services/linked_reimbursement_draft_jobs.py index 420533d..3d33c1e 100644 --- a/server/src/app/services/linked_reimbursement_draft_jobs.py +++ b/server/src/app/services/linked_reimbursement_draft_jobs.py @@ -128,7 +128,8 @@ def run_linked_reimbursement_draft_job( conversation_id=None, message=state.message, context_json=dict(state.context_json), - ) + ), + current_user=current_user, ) run_id = response.run_id result = response.result if isinstance(response.result, dict) else {} diff --git a/server/src/app/services/ocr.py b/server/src/app/services/ocr.py index 1367972..b44576c 100644 --- a/server/src/app/services/ocr.py +++ b/server/src/app/services/ocr.py @@ -5,7 +5,6 @@ import hashlib import json import re import shutil -import subprocess import threading from collections import OrderedDict from dataclasses import dataclass, field @@ -15,14 +14,28 @@ from uuid import uuid4 from sqlalchemy.orm import Session from app.core.config import SERVER_DIR, get_settings -from app.schemas.ocr import OcrRecognizeBatchRead, OcrRecognizeDocumentRead, OcrRecognizeFieldRead, OcrRecognizeLineRead -from app.services.document_preview import DocumentPreviewAssets +from app.schemas.ocr import ( + OcrRecognizeBatchRead, + OcrRecognizeDocumentRead, + OcrRecognizeFieldRead, + OcrRecognizeLineRead, +) from app.services.document_intelligence import DocumentIntelligenceService +from app.services.document_preview import DocumentPreviewAssets +from app.services.ocr_commercial import ( + OcrCommercialObserver, + OcrOperationContext, + build_ocr_commercial_observer, +) +from app.services.ocr_pdf_runtime import convert_pdf_to_images, extract_pdf_text_layer +from app.services.ocr_worker_runtime import invoke_ocr_worker WORKER_JSON_PREFIX = "__OCR_JSON__=" SUPPORTED_SUFFIXES = {".png", ".jpg", ".jpeg", ".webp", ".bmp", ".pdf"} OCR_RESULT_CACHE_LIMIT = 32 -OCR_RESULT_CACHE_PIPELINE_VERSION = f"pdf-image-ocr:{DocumentPreviewAssets.PDF_RENDERER_ID}:no-pdf-direct-v2" +OCR_RESULT_CACHE_PIPELINE_VERSION = ( + f"pdf-image-ocr:{DocumentPreviewAssets.PDF_RENDERER_ID}:no-pdf-direct-v2" +) @dataclass(slots=True) @@ -62,9 +75,23 @@ class OcrService: _worker_semaphore: threading.Semaphore | None = None _worker_semaphore_limit = 0 - def __init__(self, db: Session | None = None) -> None: + def __init__( + self, + db: Session | None = None, + *, + operation_context: OcrOperationContext | None = None, + commercial_observer: OcrCommercialObserver | None = None, + ) -> None: self.settings = get_settings() self.document_intelligence_service = DocumentIntelligenceService(db) + self.operation_context = operation_context + self.commercial_observer = ( + commercial_observer + if commercial_observer is not None + else build_ocr_commercial_observer(db) + if db is not None and operation_context is not None + else None + ) def recognize_files( self, @@ -203,11 +230,7 @@ class OcrService: cache_keys_by_source=cache_keys_by_source, ) - success_count = sum( - 1 - for item in documents - if item.line_count > 0 or not item.warnings - ) + success_count = sum(1 for item in documents if item.line_count > 0 or not item.warnings) engine = ( str(worker_payload.get("engine", "paddleocr_mobile")) if prepared_inputs @@ -361,39 +384,17 @@ class OcrService: worker_path: str, input_paths: list[Path], ) -> dict: - command = [ - python_bin, - worker_path, - "--lang", - self.settings.ocr_language, - "--text-detection-model", - self.settings.ocr_text_detection_model, - "--text-recognition-model", - self.settings.ocr_text_recognition_model, - ] - configured_device = str(self.settings.ocr_device or "").strip() - if configured_device: - command.extend(["--device", configured_device]) - for path in input_paths: - command.extend(["--input", str(path)]) - semaphore = self._resolve_worker_semaphore(self.settings.ocr_max_concurrent_workers) - with semaphore: - completed = subprocess.run( - command, - capture_output=True, - text=True, - timeout=self.settings.ocr_timeout_seconds, - check=False, - ) - if completed.returncode != 0: - detail = (completed.stderr or completed.stdout or "").strip() - raise RuntimeError(f"OCR 执行失败：{detail or 'worker 返回非 0 状态码。'}") - - payload = self._parse_worker_stdout(completed.stdout) - if payload is None: - raise RuntimeError("OCR worker 未返回可解析的 JSON 结果。") - return payload + return invoke_ocr_worker( + settings=self.settings, + python_bin=python_bin, + worker_path=worker_path, + input_paths=input_paths, + semaphore=semaphore, + parse_stdout=self._parse_worker_stdout, + commercial_observer=self.commercial_observer, + operation_context=self.operation_context, + ) @staticmethod def _parse_worker_stdout(stdout: str) -> dict | None: @@ -416,7 +417,9 @@ class OcrService: output_dir.mkdir(parents=True, exist_ok=True) cleanup_paths.append(output_dir) - image_paths, preview_usable = self._convert_pdf_to_images(pdf_path=pdf_path, output_dir=output_dir) + image_paths, preview_usable = self._convert_pdf_to_images( + pdf_path=pdf_path, output_dir=output_dir + ) if not image_paths: raise RuntimeError("PDF 转图片后未生成可识别页面。") @@ -473,38 +476,20 @@ class OcrService: return self._finalize_document(aggregated) def _extract_pdf_text_layer(self, pdf_path: Path) -> str: - try: - completed = subprocess.run( - [ - "pdftotext", - "-layout", - str(pdf_path), - "-", - ], - capture_output=True, - text=True, - timeout=self.settings.ocr_timeout_seconds, - check=False, - ) - except (OSError, subprocess.SubprocessError, UnicodeError): - return "" + return extract_pdf_text_layer( + pdf_path, + timeout_seconds=self.settings.ocr_timeout_seconds, + normalize=self._normalize_extracted_text, + ) - if completed.returncode != 0: - return "" - - return self._normalize_extracted_text(completed.stdout) - - def _convert_pdf_to_images(self, *, pdf_path: Path, output_dir: Path) -> tuple[list[Path], bool]: - try: - pages = DocumentPreviewAssets.render_pdf_pages( - pdf_path=pdf_path, - output_dir=output_dir, - timeout_seconds=self.settings.ocr_timeout_seconds, - ) - except RuntimeError as exc: - raise RuntimeError(f"PDF 转图片失败：{exc}") from exc - - return pages, True + def _convert_pdf_to_images( + self, *, pdf_path: Path, output_dir: Path + ) -> tuple[list[Path], bool]: + return convert_pdf_to_images( + pdf_path, + output_dir=output_dir, + timeout_seconds=self.settings.ocr_timeout_seconds, + ) @staticmethod def _extract_pdf_page_sort_key(path: Path) -> tuple[int, str]: @@ -564,7 +549,10 @@ class OcrService: aggregated.preview_kind = descriptor.preview_kind if descriptor.preview_data_url and not aggregated.preview_data_url: aggregated.preview_data_url = descriptor.preview_data_url - if descriptor.text_layer and descriptor.text_layer not in aggregated.text_layer_fragments: + if ( + descriptor.text_layer + and descriptor.text_layer not in aggregated.text_layer_fragments + ): aggregated.text_layer_fragments.append(descriptor.text_layer) page_summary = str(payload.get("summary", "") or "").strip() @@ -688,8 +676,12 @@ class OcrService: def _finalize_document(self, aggregated: AggregatedOcrDocument) -> OcrRecognizeDocumentRead: ocr_text = "\n".join(fragment for fragment in aggregated.text_fragments if fragment).strip() - text_layer = "\n".join(fragment for fragment in aggregated.text_layer_fragments if fragment).strip() - full_text, used_text_layer = self._choose_document_text(ocr_text=ocr_text, text_layer=text_layer) + text_layer = "\n".join( + fragment for fragment in aggregated.text_layer_fragments if fragment + ).strip() + full_text, used_text_layer = self._choose_document_text( + ocr_text=ocr_text, text_layer=text_layer + ) summary = self._truncate_summary(aggregated.summary_fragments or aggregated.text_fragments) if used_text_layer or self._placeholder_ratio(summary) >= 0.12: summary = self._summarize_text(full_text) @@ -752,15 +744,24 @@ class OcrService: return normalized_ocr_text, False if not normalized_ocr_text: return normalized_text_layer, True - if cls._placeholder_ratio(normalized_ocr_text) >= 0.12 and cls._meaningful_char_count(normalized_text_layer) >= 8: + if ( + cls._placeholder_ratio(normalized_ocr_text) >= 0.12 + and cls._meaningful_char_count(normalized_text_layer) >= 8 + ): return normalized_text_layer, True - if cls._meaningful_char_count(normalized_text_layer) > cls._meaningful_char_count(normalized_ocr_text) * 1.3: + if ( + cls._meaningful_char_count(normalized_text_layer) + > cls._meaningful_char_count(normalized_ocr_text) * 1.3 + ): return normalized_text_layer, True return normalized_ocr_text, False @staticmethod def _normalize_extracted_text(value: str) -> str: - lines = [re.sub(r"[ \t]+", " ", line).strip() for line in str(value or "").replace("\r", "\n").split("\n")] + lines = [ + re.sub(r"[ \t]+", " ", line).strip() + for line in str(value or "").replace("\r", "\n").split("\n") + ] return "\n".join(line for line in lines if line).strip() @staticmethod diff --git a/server/src/app/services/ocr_commercial.py b/server/src/app/services/ocr_commercial.py new file mode 100644 index 0000000..aa86ad4 --- /dev/null +++ b/server/src/app/services/ocr_commercial.py @@ -0,0 +1,154 @@ +"""PaddleOCR 真实 worker 的独立事务商业计量适配器。""" + +from __future__ import annotations + +import hashlib +from dataclasses import dataclass +from datetime import UTC, datetime + +from sqlalchemy.orm import Session, sessionmaker + +from app.api.deps import CurrentUserContext +from app.services.commercial_direct_operation import ( + CommercialDirectOperationBridge, + DirectOperationIdentity, + DirectOperationOutcome, + DirectOperationResult, +) +from app.services.tenant_registry import required_tenant_id + + +class OcrCommercialAccessDenied(RuntimeError): + pass + + +@dataclass(frozen=True, slots=True) +class OcrOperationContext: + tenant_id: str + operation_id: str + run_id: str + invocation_seq: int = 1 + + def __post_init__(self) -> None: + object.__setattr__(self, "tenant_id", required_tenant_id(self.tenant_id)) + for field_name in ("operation_id", "run_id"): + value = str(getattr(self, field_name) or "").strip() + if not value: + raise ValueError(f"OcrOperationContext.{field_name} 不能为空。") + object.__setattr__(self, field_name, value) + if self.invocation_seq < 1: + raise ValueError("OcrOperationContext.invocation_seq 必须大于 0。") + + +@dataclass(frozen=True, slots=True) +class OcrCommercialAttempt: + identity: DirectOperationIdentity + page_count: int + + +class OcrCommercialObserver: + def __init__(self, bridge: CommercialDirectOperationBridge) -> None: + self.bridge = bridge + + def permit( + self, + context: OcrOperationContext, + *, + page_count: int, + provider: str, + model_name: str, + started_at: datetime, + ) -> OcrCommercialAttempt: + if isinstance(page_count, bool) or page_count < 1: + raise ValueError("OCR worker page_count 必须是权威正整数。") + identity = DirectOperationIdentity( + tenant_id=context.tenant_id, + operation_key=(f"{context.operation_id}:ocr-worker:{context.invocation_seq}"), + run_key=context.run_id, + tool_type="ocr", + tool_name="paddle.worker", + provider=provider, + model_name=model_name, + started_at=_utc(started_at), + ) + permit = self.bridge.permit( + identity, + requested_quantity=page_count, + required_quantity_basis="pages", + ) + if not permit.allowed: + raise OcrCommercialAccessDenied(permit.reason) + return OcrCommercialAttempt(identity=identity, page_count=page_count) + + def complete( + self, + attempt: OcrCommercialAttempt, + *, + outcome: DirectOperationOutcome, + completed_at: datetime, + ) -> DirectOperationResult: + return self.bridge.complete( + attempt.identity, + outcome=outcome, + authoritative_quantities={"pages": attempt.page_count}, + completed_at=_utc(completed_at), + usage_source="prepared_ocr_worker_pages", + usage_availability="available", + ) + + +def build_ocr_commercial_observer(db: Session) -> OcrCommercialObserver: + factory = sessionmaker(bind=db.get_bind(), expire_on_commit=False) + return OcrCommercialObserver( + CommercialDirectOperationBridge(factory, lookup_session=db) + ) + + +def trusted_ocr_operation_context( + current_user: CurrentUserContext, + *, + operation_scope: str, + content_digests: list[str], + request_id: str = "", + run_id: str = "", +) -> OcrOperationContext: + """身份只取认证会话；文件仅使用 SHA256，不把名称或正文写入商业事实。""" + + normalized_scope = str(operation_scope or "").strip() + if not normalized_scope: + raise ValueError("operation_scope 不能为空。") + normalized_digests = [ + item.strip().lower() for item in content_digests if len(str(item or "").strip()) == 64 + ] + if not normalized_digests: + raise ValueError("OCR operation 至少需要一个内容摘要。") + request_component = str(request_id or "").strip() + if not request_component: + request_component = hashlib.sha256( + "\x1f".join(normalized_digests).encode("utf-8") + ).hexdigest() + session_component = str(current_user.auth_session_id or "").strip() + operation_digest = hashlib.sha256( + "\x1f".join( + ( + required_tenant_id(current_user.tenant_id), + session_component, + normalized_scope, + request_component, + ) + ).encode("utf-8") + ).hexdigest() + resolved_run_id = str(run_id or "").strip() or (f"ocr-{operation_digest[:32]}") + return OcrOperationContext( + tenant_id=current_user.tenant_id, + operation_id=f"ocr-operation:{operation_digest}", + run_id=resolved_run_id, + ) + + +def content_digest(content: bytes) -> str: + return hashlib.sha256(content).hexdigest() + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/ocr_pdf_runtime.py b/server/src/app/services/ocr_pdf_runtime.py new file mode 100644 index 0000000..968ddbf --- /dev/null +++ b/server/src/app/services/ocr_pdf_runtime.py @@ -0,0 +1,47 @@ +"""OCR 使用的 PDF 文本层提取与页面渲染边界。""" + +from __future__ import annotations + +import subprocess +from collections.abc import Callable +from pathlib import Path + +from app.services.document_preview import DocumentPreviewAssets + + +def extract_pdf_text_layer( + pdf_path: Path, + *, + timeout_seconds: int, + normalize: Callable[[str], str], +) -> str: + try: + completed = subprocess.run( + ["pdftotext", "-layout", str(pdf_path), "-"], + capture_output=True, + text=True, + timeout=timeout_seconds, + check=False, + ) + except (OSError, subprocess.SubprocessError, UnicodeError): + return "" + if completed.returncode != 0: + return "" + return normalize(completed.stdout) + + +def convert_pdf_to_images( + pdf_path: Path, + *, + output_dir: Path, + timeout_seconds: int, +) -> tuple[list[Path], bool]: + try: + pages = DocumentPreviewAssets.render_pdf_pages( + pdf_path=pdf_path, + output_dir=output_dir, + timeout_seconds=timeout_seconds, + ) + except RuntimeError as exc: + raise RuntimeError(f"PDF 转图片失败：{exc}") from exc + return pages, True diff --git a/server/src/app/services/ocr_worker_runtime.py b/server/src/app/services/ocr_worker_runtime.py new file mode 100644 index 0000000..83d10cb --- /dev/null +++ b/server/src/app/services/ocr_worker_runtime.py @@ -0,0 +1,159 @@ +"""OCR 子进程边界及其真实发生结果分类。""" + +from __future__ import annotations + +import subprocess +from collections.abc import Callable +from datetime import UTC, datetime +from pathlib import Path +from threading import Semaphore +from typing import Any + +from app.core.logging import get_logger +from app.services.commercial_direct_operation import DirectOperationOutcome +from app.services.ocr_commercial import ( + OcrCommercialAttempt, + OcrCommercialObserver, + OcrOperationContext, +) + +logger = get_logger("app.services.ocr_worker_runtime") + + +def invoke_ocr_worker( + *, + settings: Any, + python_bin: str, + worker_path: str, + input_paths: list[Path], + semaphore: Semaphore, + parse_stdout: Callable[[str], dict | None], + commercial_observer: OcrCommercialObserver | None, + operation_context: OcrOperationContext | None, +) -> dict: + started_at = datetime.now(UTC) + command = _worker_command( + settings=settings, + python_bin=python_bin, + worker_path=worker_path, + input_paths=input_paths, + ) + attempt = _permit_commercial( + observer=commercial_observer, + context=operation_context, + page_count=len(input_paths), + provider="PaddleOCR", + model_name=str(settings.ocr_text_recognition_model or "PP-OCRv5_mobile"), + started_at=started_at, + ) + try: + with semaphore: + completed = subprocess.run( + command, + capture_output=True, + text=True, + timeout=settings.ocr_timeout_seconds, + check=False, + ) + except Exception: + _complete_commercial( + commercial_observer, + attempt, + outcome="outcome_unknown", + ) + raise + if completed.returncode != 0: + _complete_commercial( + commercial_observer, + attempt, + outcome="provider_rejected", + ) + detail = (completed.stderr or completed.stdout or "").strip() + raise RuntimeError(f"OCR 执行失败：{detail or 'worker 返回非 0 状态码。'}") + try: + payload = parse_stdout(completed.stdout) + except Exception: + _complete_commercial( + commercial_observer, + attempt, + outcome="postprocess_failed", + ) + raise + if payload is None: + _complete_commercial( + commercial_observer, + attempt, + outcome="postprocess_failed", + ) + raise RuntimeError("OCR worker 未返回可解析的 JSON 结果。") + _complete_commercial(commercial_observer, attempt, outcome="succeeded") + return payload + + +def _worker_command( + *, + settings: Any, + python_bin: str, + worker_path: str, + input_paths: list[Path], +) -> list[str]: + command = [ + python_bin, + worker_path, + "--lang", + settings.ocr_language, + "--text-detection-model", + settings.ocr_text_detection_model, + "--text-recognition-model", + settings.ocr_text_recognition_model, + ] + configured_device = str(settings.ocr_device or "").strip() + if configured_device: + command.extend(["--device", configured_device]) + for path in input_paths: + command.extend(["--input", str(path)]) + return command + + +def _permit_commercial( + *, + observer: OcrCommercialObserver | None, + context: OcrOperationContext | None, + page_count: int, + provider: str, + model_name: str, + started_at: datetime, +) -> OcrCommercialAttempt | None: + if observer is None or context is None: + return None + return observer.permit( + context, + page_count=page_count, + provider=provider, + model_name=model_name, + started_at=started_at, + ) + + +def _complete_commercial( + observer: OcrCommercialObserver | None, + attempt: OcrCommercialAttempt | None, + *, + outcome: DirectOperationOutcome, +) -> None: + if observer is None or attempt is None: + return + try: + result = observer.complete( + attempt, + outcome=outcome, + completed_at=datetime.now(UTC), + ) + if result.requires_reconciliation: + logger.warning( + "OCR commercial reconciliation required operation_call_id=%s reason=%s", + result.operation_call_id, + result.reason_code, + ) + except Exception: + logger.exception("OCR commercial completion observer failed; worker result is preserved") diff --git a/server/src/app/services/ontology.py b/server/src/app/services/ontology.py index ef28f4d..6f1da9d 100644 --- a/server/src/app/services/ontology.py +++ b/server/src/app/services/ontology.py @@ -1,6 +1,7 @@ from __future__ import annotations import re +from datetime import UTC, datetime from typing import Any from sqlalchemy import select @@ -33,15 +34,20 @@ from app.services.ontology_field_registry import normalize_ontology_context_json from app.services.ontology_rules import ( CONTEXTUAL_SCENARIOS, EXPENSE_REVIEW_ACTIONS, - LlmOntologyEntityHint, LlmOntologyParseResult, ReferenceCatalog, ) from app.services.ontology_validation import OntologyValidationMixin from app.services.runtime_chat import RuntimeChatService +from app.services.runtime_chat_attempts import RuntimeChatOperationContext +from app.services.runtime_chat_commercial import ( + build_runtime_chat_commercial_observer, + trusted_runtime_chat_operation_context, +) logger = get_logger("app.services.ontology") + class SemanticOntologyService( OntologyDetectionMixin, OntologyExtractionMixin, @@ -50,55 +56,112 @@ class SemanticOntologyService( def __init__(self, db: Session) -> None: self.db = db self.run_service = AgentRunService(db) - self.runtime_chat_service = RuntimeChatService(db) + self.runtime_chat_service = RuntimeChatService( + db, + attempt_observer=build_runtime_chat_commercial_observer(db), + ) - def parse(self, payload: OntologyParseRequest) -> OntologyParseResult: - analyzed = self._analyze(payload) + def parse( + self, + payload: OntologyParseRequest, + *, + tenant_id: str = "default", + ) -> OntologyParseResult: + tenant = self._require_tenant_id(tenant_id) run = self.run_service.create_run( agent=AgentName.ORCHESTRATOR.value, source=AgentRunSource.USER_MESSAGE.value, + tenant_id=tenant, user_id=payload.user_id, - ontology_json=self._build_ontology_json(analyzed), + ontology_json={"scenario": "unknown", "intent": "query"}, route_json={ "stage": "semantic_parse", - "model_invocation_summary": self._build_model_invocation_summary(analyzed), - "clarification_required": analyzed["clarification_required"], - "field_error_count": len(analyzed["field_errors"]), + "phase": "pending_model_analysis", }, - permission_level=analyzed["permission"].level, - status=( + permission_level=AgentPermissionLevel.READ.value, + status=AgentRunStatus.RUNNING.value, + result_summary="语义解析请求已接收。", + ) + operation_context = trusted_runtime_chat_operation_context( + self.db, + run_id=run.run_id, + attempt_scope="semantic-ontology-parse", + ) + if operation_context is None: + self._mark_failed(run.run_id, "可信租户运行上下文缺失。") + raise ValueError("可信租户运行上下文缺失。") + try: + analyzed = self._analyze( + payload, + tenant_id=tenant, + operation_context=operation_context, + ) + self._record_semantic_parse( + run_id=run.run_id, + payload=payload, + analyzed=analyzed, + ) + self._record_model_invocations(run_id=run.run_id, analyzed=analyzed) + final_status = ( AgentRunStatus.BLOCKED.value if analyzed["clarification_required"] or analyzed["permission"].level == AgentPermissionLevel.FORBIDDEN.value else AgentRunStatus.SUCCEEDED.value - ), - result_summary=self._build_result_summary( - analyzed["scenario"], - analyzed["intent"], - analyzed["permission"].level, - analyzed["confidence"], - ), - error_message=( - analyzed["permission"].reason - if analyzed["permission"].level == AgentPermissionLevel.FORBIDDEN.value - else None - ), - ) - self._record_semantic_parse( - run_id=run.run_id, - payload=payload, - analyzed=analyzed, - ) - self._record_model_invocations(run_id=run.run_id, analyzed=analyzed) - return self._build_result(analyzed, run.run_id) + ) + self.run_service.update_run( + run.run_id, + ontology_json=self._build_ontology_json(analyzed), + route_json={ + "stage": "semantic_parse", + "phase": "completed", + "model_invocation_summary": self._build_model_invocation_summary(analyzed), + "clarification_required": analyzed["clarification_required"], + "field_error_count": len(analyzed["field_errors"]), + }, + permission_level=analyzed["permission"].level, + status=final_status, + result_summary=self._build_result_summary( + analyzed["scenario"], + analyzed["intent"], + analyzed["permission"].level, + analyzed["confidence"], + ), + error_message=( + analyzed["permission"].reason + if analyzed["permission"].level == AgentPermissionLevel.FORBIDDEN.value + else None + ), + finished_at=datetime.now(UTC), + ) + return self._build_result(analyzed, run.run_id) + except Exception as exc: + self._mark_failed(run.run_id, str(exc)) + raise def parse_for_run(self, payload: OntologyParseRequest, *, run_id: str) -> OntologyParseResult: - analyzed = self._analyze(payload) + operation_context = trusted_runtime_chat_operation_context( + self.db, + run_id=run_id, + attempt_scope="orchestrator-semantic-ontology", + ) + if operation_context is None: + raise ValueError("语义解析运行缺少可信租户上下文。") + analyzed = self._analyze( + payload, + tenant_id=operation_context.tenant_id, + operation_context=operation_context, + ) self._record_semantic_parse(run_id=run_id, payload=payload, analyzed=analyzed) self._record_model_invocations(run_id=run_id, analyzed=analyzed) return self._build_result(analyzed, run_id) - def _analyze(self, payload: OntologyParseRequest) -> dict[str, object]: + def _analyze( + self, + payload: OntologyParseRequest, + *, + tenant_id: str, + operation_context: RuntimeChatOperationContext | None, + ) -> dict[str, object]: query = payload.query.strip() if not query: raise ValueError("query 不能为空。") @@ -110,8 +173,13 @@ class SemanticOntologyService( raise ValueError("当前系统仅支持财务业务相关问题。") AgentFoundationService(self.db).ensure_foundation_ready() - reference = self._load_reference_catalog() - entities = self._extract_entities(query, compact_query, reference, context_json=context_json) + reference = self._load_reference_catalog(tenant_id=tenant_id) + entities = self._extract_entities( + query, + compact_query, + reference, + context_json=context_json, + ) rule_scenario, scenario_score = self._detect_scenario(compact_query) time_range, _time_score = self._extract_time_range( query, @@ -182,6 +250,7 @@ class SemanticOntologyService( time_range=time_range, metrics=metrics, constraints=constraints, + operation_context=operation_context, ) model_guardrail_reason = ( self._resolve_model_guardrail_reason( @@ -268,8 +337,7 @@ class SemanticOntologyService( intent=intent, ), model_clarification_required=bool( - accepted_model_parse is not None - and accepted_model_parse.clarification_required + accepted_model_parse is not None and accepted_model_parse.clarification_required ), model_clarification_question=( accepted_model_parse.clarification_question @@ -294,9 +362,7 @@ class SemanticOntologyService( ) confidence = self._resolve_confidence( model_confidence=( - accepted_model_parse.confidence - if accepted_model_parse is not None - else None + accepted_model_parse.confidence if accepted_model_parse is not None else None ), fallback_confidence=fallback_confidence, clarification_required=clarification_required, @@ -398,11 +464,7 @@ class SemanticOntologyService( ) ) if has_transport and not has_entertainment and not explicit_entertainment: - return [ - item - for item in missing_slots - if item not in {"customer_name", "participants"} - ] + return [item for item in missing_slots if item not in {"customer_name", "participants"}] return missing_slots def _record_semantic_parse( @@ -441,9 +503,7 @@ class SemanticOntologyService( analyzed: dict[str, object], ) -> None: invocations = [ - item - for item in list(analyzed.get("model_invocations") or []) - if isinstance(item, dict) + item for item in list(analyzed.get("model_invocations") or []) if isinstance(item, dict) ] if not invocations: return @@ -492,9 +552,7 @@ class SemanticOntologyService( @staticmethod def _build_model_invocation_summary(analyzed: dict[str, object]) -> dict[str, object]: invocations = [ - item - for item in list(analyzed.get("model_invocations") or []) - if isinstance(item, dict) + item for item in list(analyzed.get("model_invocations") or []) if isinstance(item, dict) ] statuses = [str(item.get("status") or "unknown") for item in invocations] return { @@ -548,13 +606,30 @@ class SemanticOntologyService( field_errors=analyzed["field_errors"], ) - def _load_reference_catalog(self) -> ReferenceCatalog: - employees = self._read_distinct_values(select(Employee.name)) - departments = self._read_distinct_values(select(OrganizationUnit.name)) - departments += self._read_distinct_values(select(ExpenseClaim.department_name)) - customers = self._read_distinct_values(select(AccountsReceivableRecord.customer_name)) - vendors = self._read_distinct_values(select(AccountsPayableRecord.vendor_name)) - projects = self._read_distinct_values(select(ExpenseClaim.project_code)) + def _load_reference_catalog(self, *, tenant_id: str) -> ReferenceCatalog: + tenant = self._require_tenant_id(tenant_id) + employees = self._read_distinct_values( + select(Employee.name).where(Employee.tenant_id == tenant) + ) + departments = self._read_distinct_values( + select(OrganizationUnit.name).where(OrganizationUnit.tenant_id == tenant) + ) + departments += self._read_distinct_values( + select(ExpenseClaim.department_name).where(ExpenseClaim.tenant_id == tenant) + ) + customers = self._read_distinct_values( + select(AccountsReceivableRecord.customer_name).where( + AccountsReceivableRecord.tenant_id == tenant + ) + ) + vendors = self._read_distinct_values( + select(AccountsPayableRecord.vendor_name).where( + AccountsPayableRecord.tenant_id == tenant + ) + ) + projects = self._read_distinct_values( + select(ExpenseClaim.project_code).where(ExpenseClaim.tenant_id == tenant) + ) return ReferenceCatalog( employees=self._dedupe_and_sort(employees), @@ -568,6 +643,29 @@ class SemanticOntologyService( values = self.db.scalars(stmt.distinct()).all() return [str(item).strip() for item in values if item] + def _mark_failed(self, run_id: str, message: str) -> None: + try: + self.run_service.merge_route_json( + run_id, + { + "stage": "semantic_parse", + "phase": "failed", + "heartbeat_at": datetime.now(UTC).isoformat(), + }, + status=AgentRunStatus.FAILED.value, + error_message=str(message or "语义解析失败。")[:2000], + finished_at=datetime.now(UTC), + ) + except Exception: + logger.exception("Failed to persist ontology run failure run_id=%s", run_id) + + @staticmethod + def _require_tenant_id(value: object) -> str: + tenant_id = str(value or "").strip() + if not tenant_id: + raise ValueError("tenant_id 不能为空。") + return tenant_id + @staticmethod def _dedupe_and_sort(values: list[str]) -> list[str]: items = {str(item).strip() for item in values if str(item).strip()} diff --git a/server/src/app/services/ontology_budget.py b/server/src/app/services/ontology_budget.py index 81ef08a..e87b728 100644 --- a/server/src/app/services/ontology_budget.py +++ b/server/src/app/services/ontology_budget.py @@ -31,7 +31,25 @@ class BudgetOntologyMixin: @staticmethod def _has_budget_signal(compact_query: str) -> bool: - return any(keyword in compact_query for keyword in BUDGET_KEYWORDS) + if any(keyword in compact_query for keyword in BUDGET_KEYWORDS): + return True + has_subject = any(keyword in compact_query for keyword in BUDGET_SUBJECT_KEYWORDS) + has_metric = any( + keyword in compact_query + for keyword in ( + "已占用", + "已预占", + "占用金额", + "可用余额", + "剩余可用", + "已发生", + "已消耗", + "已使用", + "执行率", + "使用率", + ) + ) + return has_subject and has_metric @staticmethod def _infer_budget_missing_slots( @@ -75,13 +93,11 @@ class BudgetOntologyMixin: if any(keyword in compact_query for keyword in ("预算金额", "预算总额", "预算额度")): metrics.append(OntologyMetric(name="budget_amount", aggregation="sum", unit="CNY")) if any( - keyword in compact_query - for keyword in ("可用预算", "剩余预算", "可用余额", "剩余可用") + keyword in compact_query for keyword in ("可用预算", "剩余预算", "可用余额", "剩余可用") ): metrics.append(OntologyMetric(name="available_amount", aggregation="sum", unit="CNY")) if any( - keyword in compact_query - for keyword in ("已占用", "已预占", "预算占用", "占用金额") + keyword in compact_query for keyword in ("已占用", "已预占", "预算占用", "占用金额") ): metrics.append(OntologyMetric(name="reserved_amount", aggregation="sum", unit="CNY")) if any(keyword in compact_query for keyword in ("已发生", "已核销", "已消耗", "已使用")): diff --git a/server/src/app/services/ontology_detection.py b/server/src/app/services/ontology_detection.py index 901b0d9..160a0d9 100644 --- a/server/src/app/services/ontology_detection.py +++ b/server/src/app/services/ontology_detection.py @@ -14,11 +14,13 @@ from app.schemas.ontology import ( OntologyParseRequest, OntologyTimeRange, ) +from app.services.document_numbering import DOCUMENT_NUMBER_EXTRACT_PATTERN from app.services.ontology_rules import ( AP_CORE_KEYWORDS, AR_CORE_KEYWORDS, BUDGET_DRAFT_KEYWORDS, BUDGET_OPERATE_KEYWORDS, + BUDGET_SUBJECT_KEYWORDS, COMPARE_KEYWORDS, DRAFT_FOLLOW_UP_KEYWORDS, DRAFT_KEYWORDS, @@ -28,7 +30,6 @@ from app.services.ontology_rules import ( EXPLAIN_KEYWORDS, GENERIC_EXPENSE_PROMPTS, KNOWLEDGE_INTENTS, - looks_like_expense_application_signal, OPERATE_KEYWORDS, QUERY_KEYWORDS, RISK_KEYWORDS, @@ -36,7 +37,9 @@ from app.services.ontology_rules import ( STATUS_KEYWORDS, LlmOntologyEntityHint, LlmOntologyParseResult, + looks_like_expense_application_signal, ) +from app.services.runtime_chat_attempts import RuntimeChatOperationContext logger = get_logger("app.services.ontology") @@ -97,25 +100,51 @@ class OntologyDetectionMixin: def _looks_like_expense_application(compact_query: str) -> bool: return looks_like_expense_application_signal(compact_query) - def _has_supported_business_signal(self, compact_query: str, context_json: dict[str, Any]) -> bool: + def _has_supported_business_signal( + self, + compact_query: str, + context_json: dict[str, Any], + ) -> bool: has_business_context = ( self._is_expense_application_context(context_json) or self._resolve_session_type_scenario(context_json) == "knowledge" or self._resolve_context_scenario(context_json) is not None ) + if has_business_context: + return True if self._looks_like_expense_application(compact_query): return True + if DOCUMENT_NUMBER_EXTRACT_PATTERN.search(compact_query): + return True if any(keyword in compact_query for keyword in ENGLISH_FINANCE_BUSINESS_KEYWORDS): return True domain_keywords = [ - keyword - for keywords in SCENARIO_KEYWORDS.values() - for keyword, _weight in keywords + keyword for keywords in SCENARIO_KEYWORDS.values() for keyword, _weight in keywords ] if any(keyword in compact_query for keyword in domain_keywords): return True + if any(keyword in compact_query for keyword in BUDGET_SUBJECT_KEYWORDS) and any( + keyword in compact_query + for keyword in ( + "已占用", + "已预占", + "占用金额", + "可用余额", + "剩余可用", + "已发生", + "已消耗", + "已使用", + "执行率", + "使用率", + ) + ): + return True + if "单据" in compact_query and ( + "状态" in compact_query or any(keyword in compact_query for keyword in STATUS_KEYWORDS) + ): + return True if any(keyword in compact_query for keyword in EXPENSE_NARRATIVE_KEYWORDS): return True knowledge_keywords = ( @@ -156,9 +185,6 @@ class OntologyDetectionMixin: ) if any(keyword in compact_query for keyword in approval_keywords): return True - if has_business_context and self._looks_like_contextual_business_follow_up(compact_query): - return True - return False @staticmethod @@ -219,7 +245,6 @@ class OntologyDetectionMixin: return best_scenario, round(min(best_score, 0.34), 2) - def _detect_intent( self, compact_query: str, @@ -247,44 +272,53 @@ class OntologyDetectionMixin: keyword in compact_query for keyword in ("报销的单据", "报销单据", "报销过的单据", "报销记录") ) - if scenario == "expense" and any( - keyword in compact_query - for keyword in ( - "报销了吗", - "报销了么", - "报销了没", - "报销了没有", - "报销没", - "单据状态", - "审批状态", - "报销进度", - "到哪了", - "到了哪", - "有没有报销", - "是否报销", - "进行中的单据", - "草稿单据", - "草稿的单据", - "待补充单据", - "审批中的单据", - "已提交单据", - "已入账单据", + if ( + scenario == "expense" + and any( + keyword in compact_query + for keyword in ( + "报销了吗", + "报销了么", + "报销了没", + "报销了没有", + "报销没", + "单据状态", + "审批状态", + "报销进度", + "到哪了", + "到了哪", + "有没有报销", + "是否报销", + "进行中的单据", + "草稿单据", + "草稿的单据", + "待补充单据", + "审批中的单据", + "已提交单据", + "已入账单据", + ) ) - ) or (scenario == "expense" and (status_document_query or historical_document_query)): + or (scenario == "expense" and (status_document_query or historical_document_query)) + ): return "query", 0.24 if any(keyword in compact_query for keyword in DRAFT_KEYWORDS): return "draft", 0.26 - if scenario == "expense" and "报销" in compact_query and any( - item.type == "expense_type" - and str(item.normalized_value or item.value or "").strip() - for item in entities - ) and not any( - keyword in compact_query - for keyword in ( - *QUERY_KEYWORDS, - *COMPARE_KEYWORDS, - *EXPLAIN_KEYWORDS, - *RISK_KEYWORDS, + if ( + scenario == "expense" + and "报销" in compact_query + and any( + item.type == "expense_type" + and str(item.normalized_value or item.value or "").strip() + for item in entities + ) + and not any( + keyword in compact_query + for keyword in ( + *QUERY_KEYWORDS, + *COMPARE_KEYWORDS, + *EXPLAIN_KEYWORDS, + *RISK_KEYWORDS, + ) ) ): return "draft", 0.25 @@ -391,9 +425,10 @@ class OntologyDetectionMixin: return False entity_types = {item.type for item in entities} - has_expense_signal = any( - keyword in compact_query for keyword in EXPENSE_NARRATIVE_KEYWORDS - ) or "expense_type" in entity_types + has_expense_signal = ( + any(keyword in compact_query for keyword in EXPENSE_NARRATIVE_KEYWORDS) + or "expense_type" in entity_types + ) has_context_signal = ( bool(time_range.start_date) or "amount" in entity_types @@ -414,6 +449,7 @@ class OntologyDetectionMixin: time_range: OntologyTimeRange, metrics: list[OntologyMetric], constraints: list[OntologyConstraint], + operation_context: RuntimeChatOperationContext | None = None, ) -> tuple[LlmOntologyParseResult | None, list[dict[str, Any]], str | None]: messages = self._build_model_messages( payload=payload, @@ -430,6 +466,7 @@ class OntologyDetectionMixin: messages, max_tokens=600, temperature=0.0, + operation_context=operation_context, ) response_text = chat_result.text traces = chat_result.calls_as_dicts() @@ -644,10 +681,7 @@ class OntologyDetectionMixin: items = [ item for item in items - if not ( - item.type == "expense_type" - and item.normalized_value == "entertainment" - ) + if not (item.type == "expense_type" and item.normalized_value == "entertainment") ] return items diff --git a/server/src/app/services/orchestrator.py b/server/src/app/services/orchestrator.py index 238f86e..680f049 100644 --- a/server/src/app/services/orchestrator.py +++ b/server/src/app/services/orchestrator.py @@ -4,6 +4,7 @@ import re from datetime import UTC, datetime from typing import Any +from sqlalchemy import select from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext @@ -16,6 +17,7 @@ from app.core.agent_enums import ( AgentRunStatus, ) from app.core.logging import get_logger +from app.models.tenant import Tenant from app.schemas.agent_asset import AgentAssetListItem, AgentAssetRead from app.schemas.ontology import OntologyParseRequest, OntologyParseResult from app.schemas.orchestrator import ( @@ -31,7 +33,6 @@ from app.services.agent_runs import AgentRunService from app.services.agent_traces import AgentTraceService from app.services.auth import AuthService from app.services.expense_claims import ExpenseClaimService -from app.services.knowledge import KnowledgeService from app.services.ontology import SemanticOntologyService from app.services.orchestrator_execution import ExecutionOutcome, OrchestratorExecutionEngine from app.services.orchestrator_expense_application_workflow import ( @@ -62,7 +63,7 @@ class OrchestratorService: self.asset_service = AgentAssetService(db) self.conversation_service = AgentConversationService(db) self.expense_claim_service = ExpenseClaimService(db) - self.knowledge_service = KnowledgeService(db=db) + self.knowledge_service = None self.run_service = AgentRunService(db) self.trace_service = AgentTraceService(db) self.ontology_service = SemanticOntologyService(db) @@ -84,9 +85,23 @@ class OrchestratorService: payload: OrchestratorRequest, *, current_user: CurrentUserContext | None = None, + trusted_tenant_id: str | None = None, ) -> OrchestratorResponse: + tenant_id = self._resolve_trusted_tenant_id( + current_user=current_user, + trusted_tenant_id=trusted_tenant_id, + ) if current_user is not None: payload = self._build_authenticated_user_payload(payload, current_user) + else: + payload = payload.model_copy( + update={ + "context_json": { + **dict(payload.context_json or {}), + "tenant_id": tenant_id, + } + } + ) AgentFoundationService(self.db).ensure_foundation_ready() context_json = self._hydrate_user_context( user_id=payload.user_id, @@ -117,9 +132,11 @@ class OrchestratorService: } if conversation_id: route_json["conversation_id"] = conversation_id + route_json["tenant_id"] = tenant_id run = self.run_service.create_run( agent=AgentName.ORCHESTRATOR.value, source=payload.source, + tenant_id=tenant_id, user_id=payload.user_id, task_id=payload.task_id, ontology_json={}, @@ -201,7 +218,9 @@ class OrchestratorService: ontology=ontology, task_asset=task_asset, ) - selected_capability_codes = self.execution_engine._flatten_capability_codes(capabilities) + selected_capability_codes = self.execution_engine._flatten_capability_codes( + capabilities + ) is_expense_review_action = self.execution_engine._is_expense_review_action(context_json) is_expense_application_context = self._is_expense_application_context(context_json) requires_confirmation = ( @@ -237,6 +256,7 @@ class OrchestratorService: "selected_capability_codes": selected_capability_codes, "ontology_run_id": ontology.run_id, } + route_json["tenant_id"] = tenant_id if task_asset is not None: task_config = task_asset.config_json or {} route_json["job_type"] = str(task_config.get("task_type") or "").strip() @@ -249,16 +269,14 @@ class OrchestratorService: and current_user is not None and ontology.permission.level != AgentPermissionLevel.FORBIDDEN.value ): - authenticated_application_outcome = ( - self.expense_application_workflow.execute( - payload=payload, - current_user=current_user, - run_id=run.run_id, - conversation_id=conversation_id, - ontology=ontology, - context_json=context_json, - selected_capability_codes=selected_capability_codes, - ) + authenticated_application_outcome = self.expense_application_workflow.execute( + payload=payload, + current_user=current_user, + run_id=run.run_id, + conversation_id=conversation_id, + ontology=ontology, + context_json=context_json, + selected_capability_codes=selected_capability_codes, ) if ontology.permission.level == AgentPermissionLevel.FORBIDDEN.value: @@ -372,19 +390,16 @@ class OrchestratorService: and ( ( requires_confirmation - and ontology.permission.level == AgentPermissionLevel.APPROVAL_REQUIRED.value - ) - or ( - is_expense_application_context - and result_requires_confirmation + and ontology.permission.level + == AgentPermissionLevel.APPROVAL_REQUIRED.value ) + or (is_expense_application_context and result_requires_confirmation) ) else outcome.status ) response_status = self._normalize_response_status(final_status) result_message = ( - str(outcome.result.get("message", "")).strip() - or "Orchestrator 执行完成。" + str(outcome.result.get("message", "")).strip() or "Orchestrator 执行完成。" ) trace_summary = OrchestratorTraceSummary( scenario=ontology.scenario, @@ -586,15 +601,51 @@ class OrchestratorService: } ) + def _resolve_trusted_tenant_id( + self, + *, + current_user: CurrentUserContext | None, + trusted_tenant_id: str | None, + ) -> str: + user_tenant = str(current_user.tenant_id or "").strip() if current_user is not None else "" + internal_tenant = str(trusted_tenant_id or "").strip() + if user_tenant and internal_tenant and user_tenant != internal_tenant: + raise ValueError("登录用户租户与内部任务租户不一致。") + if user_tenant: + return user_tenant + if not internal_tenant: + raise ValueError("Orchestrator 缺少可信租户上下文。") + registered = self.db.scalar( + select(Tenant.tenant_id).where( + Tenant.tenant_id == internal_tenant, + Tenant.status == "active", + ) + ) + if registered is None: + raise ValueError("Orchestrator 内部任务租户不存在或未启用。") + return internal_tenant + def _record_trace_event(self, **kwargs: Any) -> None: self.trace_service.record_event_safe(**kwargs) - def _hydrate_user_context(self, user_id: str | None, context_json: dict[str, Any]) -> dict[str, Any]: - identifier = str(user_id or context_json.get("username") or context_json.get("email") or "").strip() + def _hydrate_user_context( + self, + user_id: str | None, + context_json: dict[str, Any], + ) -> dict[str, Any]: + identifier = str( + user_id or context_json.get("username") or context_json.get("email") or "" + ).strip() if not identifier: return context_json - snapshot = AuthService(self.db).get_user_snapshot(identifier) + tenant_id = str(context_json.get("tenant_id") or "").strip() + if not tenant_id: + return context_json + snapshot = AuthService(self.db).get_user_snapshot( + identifier, + tenant_id=tenant_id, + ) if snapshot is None: return context_json @@ -771,5 +822,7 @@ class OrchestratorService: isinstance(draft_payload, dict) and str(draft_payload.get("draft_type") or "").strip() == "expense_application" and str(draft_payload.get("status") or "").strip() == "submitted" - and bool(str(draft_payload.get("claim_no") or draft_payload.get("claim_id") or "").strip()) + and bool( + str(draft_payload.get("claim_no") or draft_payload.get("claim_id") or "").strip() + ) ) diff --git a/server/src/app/services/orchestrator_execution.py b/server/src/app/services/orchestrator_execution.py index 3967107..b18dca1 100644 --- a/server/src/app/services/orchestrator_execution.py +++ b/server/src/app/services/orchestrator_execution.py @@ -1,7 +1,6 @@ from __future__ import annotations from dataclasses import asdict, dataclass -from time import perf_counter from typing import Any from sqlalchemy.orm import Session @@ -18,7 +17,10 @@ from app.services.digital_employee_finance_report_task import ( from app.services.hermes_employee_profile_scanner import HermesEmployeeProfileScannerService from app.services.hermes_risk_clue_collector import HermesRiskClueCollectorService from app.services.hermes_risk_scanner import HermesRiskScannerService +from app.services.knowledge import KnowledgeService from app.services.knowledge_sync import KnowledgeSyncDispatchService +from app.services.knowledge_tenant_scope import require_knowledge_tenant_id +from app.services.orchestrator_tool_execution import OrchestratorToolExecutionMixin @dataclass(slots=True) @@ -120,34 +122,58 @@ class OrchestratorExecutionTaskMixin: context_json: dict[str, Any], ) -> ExecutionOutcome | None: task_type = self._resolve_task_type(task_asset) + tenant_id = self._trusted_run_tenant_id(run_id) if task_type == "global_risk_scan": - return self._execute_risk_graph_scan(run_id=run_id, context_json=context_json) + return self._execute_risk_graph_scan( + run_id=run_id, + context_json=context_json, + tenant_id=tenant_id, + ) if task_type == "employee_behavior_profile_scan": - return self._execute_employee_profile_scan(run_id=run_id, context_json=context_json) + return self._execute_employee_profile_scan( + run_id=run_id, + context_json=context_json, + tenant_id=tenant_id, + ) if task_type == "finance_policy_knowledge_organize": return self._execute_finance_policy_knowledge_sync( payload=payload, run_id=run_id, task_asset=task_asset, context_json=context_json, + tenant_id=tenant_id, ) if task_type == "risk_clue_collect": - return self._execute_risk_clue_collect(run_id=run_id, context_json=context_json) + return self._execute_risk_clue_collect( + run_id=run_id, + context_json=context_json, + tenant_id=tenant_id, + ) if task_type == "finance_report_orchestration": return self._execute_finance_report_orchestration( run_id=run_id, context_json=context_json, + tenant_id=tenant_id, ) return None - def _execute_risk_graph_scan(self, *, run_id: str, context_json: dict[str, Any]) -> ExecutionOutcome: + def _execute_risk_graph_scan( + self, + *, + run_id: str, + context_json: dict[str, Any], + tenant_id: str, + ) -> ExecutionOutcome: summary, degraded = self._invoke_tool( run_id=run_id, tool_type=AgentToolType.RULE_ENGINE.value, tool_name="digital_employee.financial_risk_graph.scan", request_json={"task_type": "global_risk_scan"}, context_json=context_json, - executor=lambda: HermesRiskScannerService(self.db).scan_global_risks(run_id=run_id), + executor=lambda: HermesRiskScannerService(self.db).scan_global_risks( + run_id=run_id, + tenant_id=tenant_id, + ), fallback_factory=lambda exc: { "message": f"财务风险图谱巡检失败，已转人工检查：{exc}", "degraded": True, @@ -161,13 +187,24 @@ class OrchestratorExecutionTaskMixin: ) return ExecutionOutcome( status=AgentRunStatus.SUCCEEDED.value, - result={"message": message, "report_type": "global_risk_scan", "summary": summary, "degraded": degraded}, + result={ + "message": message, + "report_type": "global_risk_scan", + "summary": summary, + "degraded": degraded, + }, degraded=degraded, tool_count=1, failed_tool_count=1 if degraded else 0, ) - def _execute_employee_profile_scan(self, *, run_id: str, context_json: dict[str, Any]) -> ExecutionOutcome: + def _execute_employee_profile_scan( + self, + *, + run_id: str, + context_json: dict[str, Any], + tenant_id: str, + ) -> ExecutionOutcome: summary, degraded = self._invoke_tool( run_id=run_id, tool_type=AgentToolType.DATABASE.value, @@ -175,7 +212,8 @@ class OrchestratorExecutionTaskMixin: request_json={"task_type": "employee_behavior_profile_scan"}, context_json=context_json, executor=lambda: HermesEmployeeProfileScannerService(self.db).scan_employee_profiles( - log_id=run_id + log_id=run_id, + tenant_id=tenant_id, ), fallback_factory=lambda exc: { "message": f"员工行为画像巡检失败，已保留失败记录：{exc}", @@ -191,7 +229,12 @@ class OrchestratorExecutionTaskMixin: ) return ExecutionOutcome( status=AgentRunStatus.SUCCEEDED.value, - result={"message": message, "report_type": "employee_behavior_profile_scan", "summary": summary, "degraded": degraded}, + result={ + "message": message, + "report_type": "employee_behavior_profile_scan", + "summary": summary, + "degraded": degraded, + }, degraded=degraded, tool_count=1, failed_tool_count=1 if degraded else 0, @@ -204,6 +247,7 @@ class OrchestratorExecutionTaskMixin: run_id: str, task_asset: AgentAssetRead | None, context_json: dict[str, Any], + tenant_id: str, ) -> ExecutionOutcome: config = task_asset.config_json if task_asset is not None else {} username = str( @@ -234,6 +278,7 @@ class OrchestratorExecutionTaskMixin: name=display_name or username or "数字员工", role_codes=["admin"], is_admin=True, + tenant_id=tenant_id, ), folder=str(config.get("folder") or "").strip() or None, source=AgentRunSource.SCHEDULE.value, @@ -251,7 +296,12 @@ class OrchestratorExecutionTaskMixin: message = f"{message} 日志编号：{dispatch['agent_run_id']}" return ExecutionOutcome( status=AgentRunStatus.SUCCEEDED.value, - result={"message": message, "report_type": "finance_policy_knowledge_organize", "summary": dispatch, "degraded": degraded}, + result={ + "message": message, + "report_type": "finance_policy_knowledge_organize", + "summary": dispatch, + "degraded": degraded, + }, degraded=degraded, tool_count=1, failed_tool_count=1 if degraded else 0, @@ -262,6 +312,7 @@ class OrchestratorExecutionTaskMixin: *, run_id: str, context_json: dict[str, Any], + tenant_id: str, ) -> ExecutionOutcome: summary, degraded = self._invoke_tool( run_id=run_id, @@ -270,7 +321,8 @@ class OrchestratorExecutionTaskMixin: request_json={"task_type": "risk_clue_collect"}, context_json=context_json, executor=lambda: HermesRiskClueCollectorService(self.db).collect_risk_clues( - run_id=run_id + run_id=run_id, + tenant_id=tenant_id, ), fallback_factory=lambda exc: { "message": f"风险线索归集失败，已保留失败记录：{exc}", @@ -286,7 +338,12 @@ class OrchestratorExecutionTaskMixin: ) return ExecutionOutcome( status=AgentRunStatus.SUCCEEDED.value, - result={"message": message, "report_type": "risk_clue_collect", "summary": summary, "degraded": degraded}, + result={ + "message": message, + "report_type": "risk_clue_collect", + "summary": summary, + "degraded": degraded, + }, degraded=degraded, tool_count=1, failed_tool_count=1 if degraded else 0, @@ -297,6 +354,7 @@ class OrchestratorExecutionTaskMixin: *, run_id: str, context_json: dict[str, Any], + tenant_id: str, ) -> ExecutionOutcome: report_type = str(context_json.get("report_type") or "weekly").strip().lower() if report_type not in {"weekly", "quarterly", "annual"}: @@ -317,12 +375,14 @@ class OrchestratorExecutionTaskMixin: source=AgentRunSource.SCHEDULE.value, run_id=run_id, record_tool_call=False, + tenant_id=tenant_id, ), fallback_factory=lambda exc: { "message": f"财务报告生成失败，已保留失败记录：{exc}", "degraded": True, }, ) + message = ( str(summary.get("message") or "").strip() or "财务报告编排完成：" @@ -344,151 +404,6 @@ class OrchestratorExecutionTaskMixin: class OrchestratorExecutionHelperMixin: - @staticmethod - def _resolve_task_type(task_asset: AgentAssetRead | None) -> str: - if task_asset is None: - return "" - config = task_asset.config_json or {} - task_type = str(config.get("task_type") or "").strip() - if task_type: - return task_type.replace("-", "_").replace(".", "_") - return str(task_asset.code or "").removeprefix("task.hermes.").replace(".", "_") - - @staticmethod - def _resolve_next_step( - ontology: OntologyParseResult, - source: str, - *, - context_json: dict[str, Any] | None = None, - ) -> str: - if OrchestratorExecutionEngine._is_expense_review_action(context_json or {}): - return "create_draft" - if ontology.clarification_required: - return "ask_clarification" - if ontology.intent == "draft": - return "create_draft" - if ontology.scenario == "knowledge" or ontology.intent == "explain": - return "search_knowledge" - if ontology.intent == "risk_check" or source == AgentRunSource.SCHEDULE.value: - return "run_rule" - if ontology.intent in {"query", "compare"}: - return "query_database" - return "create_draft" - - @staticmethod - def _is_expense_review_action(context_json: dict[str, Any]) -> bool: - review_action = str((context_json or {}).get("review_action") or "").strip() - return review_action in { - "save_draft", - "next_step", - "edit_review", - "link_to_existing_draft", - "create_new_claim_from_documents", - } - - @staticmethod - def _is_expense_persistence_action(context_json: dict[str, Any]) -> bool: - review_action = str((context_json or {}).get("review_action") or "").strip() - return review_action in { - "save_draft", - "next_step", - "link_to_existing_draft", - "create_new_claim_from_documents", - } - - @staticmethod - def _flatten_capability_codes( - capabilities: dict[str, list[AgentAssetListItem | AgentAssetRead]], - ) -> list[str]: - codes: list[str] = [] - for items in capabilities.values(): - for item in items[:2]: - if item.code not in codes: - codes.append(item.code) - return codes - - def _rank_assets( - self, - items: list[AgentAssetListItem], - ontology: OntologyParseResult, - ) -> list[AgentAssetListItem]: - def score(item: AgentAssetListItem) -> tuple[int, str]: - item_tags = {str(value) for value in item.scenario_json or []} - weight = 0 - if ontology.scenario in item_tags: - weight += 3 - if ontology.intent in item_tags: - weight += 2 - for risk_flag in ontology.risk_flags: - if risk_flag in item_tags: - weight += 4 - return weight, item.code - - ranked = sorted(items, key=score, reverse=True) - if not ranked: - return [] - scored = [item for item in ranked if score(item)[0] > 0] - return scored or ranked[:1] - - def _invoke_tool( - self, - *, - run_id: str, - tool_type: str, - tool_name: str, - request_json: dict[str, Any], - context_json: dict[str, Any], - executor, - fallback_factory, - ) -> tuple[dict[str, Any], bool]: - started = perf_counter() - try: - self._maybe_raise_simulated_failure(tool_type, context_json) - response = executor() - duration_ms = int((perf_counter() - started) * 1000) - self.run_service.record_tool_call( - run_id=run_id, - tool_type=tool_type, - tool_name=tool_name, - request_json=request_json, - response_json=response, - status="succeeded", - duration_ms=duration_ms, - ) - if self.trace_service: - self.trace_service.record_tool_event_safe( - run_id, tool_type, tool_name, request_json, response, - "succeeded", duration_ms, context_json, - ) - return response, False - except Exception as exc: - duration_ms = int((perf_counter() - started) * 1000) - response = fallback_factory(exc) - self.run_service.record_tool_call( - run_id=run_id, - tool_type=tool_type, - tool_name=tool_name, - request_json=request_json, - response_json=response, - status="failed", - duration_ms=duration_ms, - error_message=str(exc), - ) - if self.trace_service: - self.trace_service.record_tool_event_safe( - run_id, tool_type, tool_name, request_json, response, - "failed", duration_ms, context_json, str(exc), - ) - return response, True - - @staticmethod - def _maybe_raise_simulated_failure(tool_type: str, context_json: dict[str, Any]) -> None: - expected = str(context_json.get("simulate_tool_failure") or "").strip().lower() - if not expected: - return - if expected == tool_type.lower(): - raise RuntimeError(f"simulated {tool_type} failure") - @staticmethod def _build_user_query_result( ontology: OntologyParseResult, @@ -544,6 +459,7 @@ class OrchestratorExecutionHelperMixin: def _build_knowledge_answer( self, *, + run_id: str, message: str, ontology: OntologyParseResult, capabilities: dict[str, list[AgentAssetListItem | AgentAssetRead]], @@ -553,12 +469,19 @@ class OrchestratorExecutionHelperMixin: conversation_history = context_json.get("conversation_history") if not isinstance(conversation_history, list): conversation_history = None - payload = self.knowledge_service.search_knowledge( + payload = KnowledgeService( + db=self.db, + tenant_id=self._trusted_run_tenant_id(run_id), + ).search_knowledge( message, conversation_history=conversation_history, limit=8, ) - references = [str(item).strip() for item in list(payload.get("references") or []) if str(item).strip()] + references = [ + str(item).strip() + for item in list(payload.get("references") or []) + if str(item).strip() + ] if references: payload["references"] = references return payload @@ -643,7 +566,11 @@ class OrchestratorExecutionHelperMixin: } -class OrchestratorExecutionEngine(OrchestratorExecutionTaskMixin, OrchestratorExecutionHelperMixin): +class OrchestratorExecutionEngine( + OrchestratorExecutionTaskMixin, + OrchestratorToolExecutionMixin, + OrchestratorExecutionHelperMixin, +): def __init__( self, *, @@ -663,6 +590,20 @@ class OrchestratorExecutionEngine(OrchestratorExecutionTaskMixin, OrchestratorEx self.database_query_builder = database_query_builder self.trace_service = trace_service + def _trusted_run_tenant_id(self, run_id: str) -> str: + run = self.run_service.get_run(run_id) + if run is None: + raise ValueError("知识操作关联的 Agent Run 不存在。") + route_tenant_id = require_knowledge_tenant_id( + (run.route_json or {}).get("tenant_id") + ) + ontology_tenant_raw = (run.ontology_json or {}).get("tenant_id") + if ontology_tenant_raw is not None: + ontology_tenant_id = require_knowledge_tenant_id(ontology_tenant_raw) + if ontology_tenant_id != route_tenant_id: + raise ValueError("Agent Run 租户上下文不一致，知识操作已拒绝。") + return route_tenant_id + def _execute_user_agent( self, *, @@ -756,6 +697,7 @@ class OrchestratorExecutionEngine(OrchestratorExecutionTaskMixin, OrchestratorEx request_json=self._build_ontology_json(ontology), context_json=context_json, executor=lambda: self._build_knowledge_answer( + run_id=run_id, message=payload.message or "", ontology=ontology, capabilities=capabilities, @@ -901,4 +843,3 @@ class OrchestratorExecutionEngine(OrchestratorExecutionTaskMixin, OrchestratorEx tool_count=1, failed_tool_count=1 if degraded else 0, ) - diff --git a/server/src/app/services/orchestrator_tool_execution.py b/server/src/app/services/orchestrator_tool_execution.py new file mode 100644 index 0000000..3c9c57a --- /dev/null +++ b/server/src/app/services/orchestrator_tool_execution.py @@ -0,0 +1,208 @@ +"""Orchestrator 的工具选择、商业预检与调用记录职责。""" + +from __future__ import annotations + +import uuid +from time import perf_counter +from typing import Any + +from app.core.agent_enums import AgentRunSource +from app.schemas.agent_asset import AgentAssetListItem, AgentAssetRead +from app.schemas.ontology import OntologyParseResult +from app.services.commercial_access_policy import CommercialConflictError +from app.services.commercial_runtime_bridge import CommercialRuntimeBridge + + +class OrchestratorToolExecutionMixin: + @staticmethod + def _resolve_task_type(task_asset: AgentAssetRead | None) -> str: + if task_asset is None: + return "" + config = task_asset.config_json or {} + task_type = str(config.get("task_type") or "").strip() + if task_type: + return task_type.replace("-", "_").replace(".", "_") + return str(task_asset.code or "").removeprefix("task.hermes.").replace(".", "_") + + @staticmethod + def _resolve_next_step( + ontology: OntologyParseResult, + source: str, + *, + context_json: dict[str, Any] | None = None, + ) -> str: + if OrchestratorToolExecutionMixin._is_expense_review_action(context_json or {}): + return "create_draft" + if ontology.clarification_required: + return "ask_clarification" + if ontology.intent == "draft": + return "create_draft" + if ontology.scenario == "knowledge" or ontology.intent == "explain": + return "search_knowledge" + if ontology.intent == "risk_check" or source == AgentRunSource.SCHEDULE.value: + return "run_rule" + if ontology.intent in {"query", "compare"}: + return "query_database" + return "create_draft" + + @staticmethod + def _is_expense_review_action(context_json: dict[str, Any]) -> bool: + review_action = str((context_json or {}).get("review_action") or "").strip() + return review_action in { + "save_draft", + "next_step", + "edit_review", + "link_to_existing_draft", + "create_new_claim_from_documents", + } + + @staticmethod + def _is_expense_persistence_action(context_json: dict[str, Any]) -> bool: + review_action = str((context_json or {}).get("review_action") or "").strip() + return review_action in { + "save_draft", + "next_step", + "link_to_existing_draft", + "create_new_claim_from_documents", + } + + @staticmethod + def _flatten_capability_codes( + capabilities: dict[str, list[AgentAssetListItem | AgentAssetRead]], + ) -> list[str]: + codes: list[str] = [] + for items in capabilities.values(): + for item in items[:2]: + if item.code not in codes: + codes.append(item.code) + return codes + + def _rank_assets( + self, + items: list[AgentAssetListItem], + ontology: OntologyParseResult, + ) -> list[AgentAssetListItem]: + def score(item: AgentAssetListItem) -> tuple[int, str]: + item_tags = {str(value) for value in item.scenario_json or []} + weight = 0 + if ontology.scenario in item_tags: + weight += 3 + if ontology.intent in item_tags: + weight += 2 + for risk_flag in ontology.risk_flags: + if risk_flag in item_tags: + weight += 4 + return weight, item.code + + ranked = sorted(items, key=score, reverse=True) + if not ranked: + return [] + scored = [item for item in ranked if score(item)[0] > 0] + return scored or ranked[:1] + + def _invoke_tool( + self, + *, + run_id: str, + tool_type: str, + tool_name: str, + request_json: dict[str, Any], + context_json: dict[str, Any], + executor, + fallback_factory, + ) -> tuple[dict[str, Any], bool]: + tool_call_id = str(uuid.uuid4()) + permit = CommercialRuntimeBridge(self.db).reserve_tool( + run_id, + tool_call_id=tool_call_id, + tool_type=tool_type, + tool_name=tool_name, + ) + gate = permit.gate + if not gate.allowed: + error = CommercialConflictError(gate.reason) + response = fallback_factory(error) + self.run_service.record_tool_call( + run_id=run_id, + tool_call_id=tool_call_id, + tool_type=tool_type, + tool_name=tool_name, + request_json=request_json, + response_json=response, + status="blocked", + duration_ms=0, + error_message=str(error), + ) + if self.trace_service: + self.trace_service.record_tool_event_safe( + run_id, + tool_type, + tool_name, + request_json, + response, + "blocked", + 0, + context_json, + str(error), + ) + return response, True + started = perf_counter() + try: + self._maybe_raise_simulated_failure(tool_type, context_json) + response = executor() + duration_ms = int((perf_counter() - started) * 1000) + self.run_service.record_tool_call( + run_id=run_id, + tool_call_id=tool_call_id, + tool_type=tool_type, + tool_name=tool_name, + request_json=request_json, + response_json=response, + status="succeeded", + duration_ms=duration_ms, + ) + if self.trace_service: + self.trace_service.record_tool_event_safe( + run_id, + tool_type, + tool_name, + request_json, + response, + "succeeded", + duration_ms, + context_json, + ) + return response, False + except Exception as exc: + duration_ms = int((perf_counter() - started) * 1000) + response = fallback_factory(exc) + self.run_service.record_tool_call( + run_id=run_id, + tool_call_id=tool_call_id, + tool_type=tool_type, + tool_name=tool_name, + request_json=request_json, + response_json=response, + status="failed", + duration_ms=duration_ms, + error_message=str(exc), + ) + if self.trace_service: + self.trace_service.record_tool_event_safe( + run_id, + tool_type, + tool_name, + request_json, + response, + "failed", + duration_ms, + context_json, + str(exc), + ) + return response, True + + @staticmethod + def _maybe_raise_simulated_failure(tool_type: str, context_json: dict[str, Any]) -> None: + expected = str(context_json.get("simulate_tool_failure") or "").strip().lower() + if expected and expected == tool_type.lower(): + raise RuntimeError(f"simulated {tool_type} failure") diff --git a/server/src/app/services/payment_reconciliation.py b/server/src/app/services/payment_reconciliation.py new file mode 100644 index 0000000..511178e --- /dev/null +++ b/server/src/app/services/payment_reconciliation.py @@ -0,0 +1,509 @@ +from __future__ import annotations + +import hashlib +import uuid +from dataclasses import dataclass +from datetime import UTC, datetime +from decimal import Decimal, InvalidOperation +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.financial_connector import ( + FinancialConnectorEvent, + PaymentReconciliationCase, + PaymentReconciliationEvent, +) +from app.schemas.financial_connector import FinancialEventEnvelope +from app.services.expense_cases import ExpenseCaseService +from app.services.financial_connector_actions import FinancialConnectorActionService +from app.services.financial_connector_auth import VerifiedConnectorRequest + + +@dataclass(slots=True) +class ReconciliationResult: + case: PaymentReconciliationCase | None + audit_event: PaymentReconciliationEvent | None + processing_status: str + error_code: str | None + claim_id: str | None + expense_case_id: str | None + claim_status: str | None + + +class PaymentReconciliationService: + """把已认证事件精确匹配到现有付款状态机；不负责 commit。""" + + def __init__(self, db: Session) -> None: + self.db = db + self.actions = FinancialConnectorActionService(db) + self.expense_cases = ExpenseCaseService(db) + + def process( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + *, + connector_event_id: str, + ) -> ReconciliationResult: + if envelope.event_type == "payment_settled": + return self._payment_settled(envelope, verified, connector_event_id) + if envelope.event_type == "payment_failed": + return self._payment_failed(envelope, verified, connector_event_id) + if envelope.event_type in {"erp_posted", "erp_posting_failed"}: + return self._erp_event(envelope, verified, connector_event_id) + return self._payment_reversal(envelope, verified, connector_event_id) + + def _payment_settled( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + connector_event_id: str, + ) -> ReconciliationResult: + tenant_id = envelope.tenant_id + provider = verified.config.provider + claim = self.actions.lock_claim( + tenant_id=tenant_id, + claim_id=str(envelope.payload.get("claim_id") or ""), + ) + if claim is None: + return self._without_case("claim_not_found") + + existing = self._lock_case(tenant_id, provider, claim.id) + mismatch = self.actions.settlement_mismatch(claim, envelope.payload) + amount = _amount(envelope.payload.get("amount")) + if amount is None or amount < 0: + mismatch = "amount_invalid" + amount = Decimal("0.00") + expense_case = self.expense_cases.ensure_case_for_claim(claim, tenant_id=tenant_id) + status = "exception" if mismatch else "matched" + before = _case_state(existing) + if mismatch is None: + self.actions.settle_claim( + claim, + tenant_id=tenant_id, + provider=provider, + connector_event_id=connector_event_id, + external_event_id=envelope.external_event_id, + content_hash=verified.content_hash, + verification_level=verified.verification_level, + evidence_classification=verified.evidence_classification, + external_reference_tail=_tail(envelope.payload.get("external_payment_reference")), + correlation_id=envelope.correlation_id, + ) + case = self._build_or_update_case( + existing, + tenant_id=tenant_id, + provider=provider, + claim=claim, + expense_case_id=expense_case.id, + actual_amount=amount, + actual_currency=str(envelope.payload.get("currency") or "").upper(), + external_reference=_tail(envelope.payload.get("external_payment_reference")), + status=status, + exception_code=mismatch, + connector_event_id=connector_event_id, + reset_erp=True, + ) + action = "exception_created" if mismatch else "auto_matched" + audit = self._audit_event( + case, + connector_event_id=connector_event_id, + action=action, + fingerprint=verified.request_fingerprint, + before=before, + reason=mismatch, + correlation_id=envelope.correlation_id, + ) + return ReconciliationResult( + case=case, + audit_event=audit, + processing_status="exception" if mismatch else "processed", + error_code=mismatch, + claim_id=claim.id, + expense_case_id=expense_case.id, + claim_status=str(claim.status or ""), + ) + + def _payment_failed( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + connector_event_id: str, + ) -> ReconciliationResult: + tenant_id = envelope.tenant_id + provider = verified.config.provider + claim = self.actions.lock_claim( + tenant_id=tenant_id, + claim_id=str(envelope.payload.get("claim_id") or ""), + ) + if claim is None: + return self._without_case("claim_not_found") + expense_case = self.expense_cases.ensure_case_for_claim(claim, tenant_id=tenant_id) + existing = self._lock_case(tenant_id, provider, claim.id) + before = _case_state(existing) + amount = _amount(envelope.payload.get("amount")) or Decimal("0.00") + association_error = self.actions.settlement_mismatch(claim, envelope.payload) + error_code = association_error or "external_payment_failed" + case = self._build_or_update_case( + existing, + tenant_id=tenant_id, + provider=provider, + claim=claim, + expense_case_id=expense_case.id, + actual_amount=max(amount, Decimal("0.00")), + actual_currency=str(envelope.payload.get("currency") or "").upper(), + external_reference=_tail(envelope.payload.get("external_payment_reference")), + status="exception", + exception_code=error_code, + connector_event_id=connector_event_id, + ) + audit = self._audit_event( + case, + connector_event_id=connector_event_id, + action="exception_created", + fingerprint=verified.request_fingerprint, + before=before, + reason=error_code, + correlation_id=envelope.correlation_id, + ) + return ReconciliationResult( + case=case, + audit_event=audit, + processing_status="exception", + error_code=error_code, + claim_id=claim.id, + expense_case_id=expense_case.id, + claim_status=str(claim.status or ""), + ) + + def _erp_event( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + connector_event_id: str, + ) -> ReconciliationResult: + origin, claim, case = self._origin_graph(envelope, verified) + if origin is None or claim is None or case is None: + return self._without_case("origin_settlement_not_found") + if origin.event_type != "payment_settled" or origin.processing_status != "processed": + return self._without_case("origin_settlement_not_processed") + if str(envelope.payload.get("claim_id") or "") != str(origin.claim_id or ""): + return self._without_case("origin_claim_mismatch") + + # ERP 回执通过 origin_external_event_id 关联已认证的支付事实，协议本身 + # 不要求重复携带支付参考号，因此这里只校验申请、金额与币种。 + mismatch = self.actions.payment_payload_mismatch( + claim, + envelope.payload, + require_external_reference=False, + ) + if str(claim.status or "").lower() != "paid": + mismatch = "claim_not_paid" + if mismatch: + return self._existing_case_exception( + case, + claim_status=str(claim.status or ""), + connector_event_id=connector_event_id, + fingerprint=verified.request_fingerprint, + correlation_id=envelope.correlation_id, + error_code=f"erp_{mismatch}", + ) + + before = _case_state(case) + posted = envelope.event_type == "erp_posted" + self.actions.record_erp_event( + claim, + tenant_id=envelope.tenant_id, + provider=verified.config.provider, + connector_event_id=connector_event_id, + correlation_id=envelope.correlation_id, + event_type=envelope.event_type, + payload=envelope.payload, + ) + case.erp_status = "posted" if posted else "posting_failed" + case.last_connector_event_id = connector_event_id + if posted and str(case.exception_code or "").startswith("erp_"): + case.status = "matched" + case.exception_code = None if posted else "erp_posting_failed" + case.version += 1 + case.updated_at = datetime.now(UTC) + if posted: + document_number = str(envelope.payload.get("erp_document_number") or "").strip() + case.erp_document_tail = _tail(document_number) + case.erp_document_hash = ( + f"sha256:{hashlib.sha256(document_number.encode()).hexdigest()}" + ) + action = "erp_posted" if posted else "erp_posting_failed" + audit = self._audit_event( + case, + connector_event_id=connector_event_id, + action=action, + fingerprint=verified.request_fingerprint, + before=before, + reason=None if posted else "erp_posting_failed", + correlation_id=envelope.correlation_id, + ) + return ReconciliationResult( + case=case, + audit_event=audit, + processing_status="processed" if posted else "exception", + error_code=None if posted else "erp_posting_failed", + claim_id=claim.id, + expense_case_id=case.expense_case_id, + claim_status=str(claim.status or ""), + ) + + def _payment_reversal( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + connector_event_id: str, + ) -> ReconciliationResult: + origin, claim, case = self._origin_graph(envelope, verified) + if origin is None or claim is None or case is None: + return self._without_case("origin_settlement_not_found") + mismatch = self.actions.payment_payload_mismatch(claim, envelope.payload) + if str(claim.status or "").lower() != "paid": + mismatch = "claim_not_paid" + if str(envelope.payload.get("claim_id") or "") != str(origin.claim_id or ""): + mismatch = "origin_claim_mismatch" + if origin.event_type != "payment_settled" or origin.processing_status != "processed": + mismatch = "origin_settlement_not_processed" + amount = _amount(envelope.payload.get("amount")) + if amount is None or amount != Decimal(claim.amount).quantize(Decimal("0.01")): + mismatch = "reversal_amount_mismatch" + if mismatch: + return self._existing_case_exception( + case, + claim_status=str(claim.status or ""), + connector_event_id=connector_event_id, + fingerprint=verified.request_fingerprint, + correlation_id=envelope.correlation_id, + error_code=mismatch, + ) + + before = _case_state(case) + self.actions.reopen_claim( + claim, + tenant_id=envelope.tenant_id, + provider=verified.config.provider, + connector_event_id=connector_event_id, + external_event_id=envelope.external_event_id, + origin_connector_event_id=origin.id, + content_hash=verified.content_hash, + verification_level=verified.verification_level, + evidence_classification=verified.evidence_classification, + correlation_id=envelope.correlation_id, + ) + case.status = "reopened" + case.exception_code = "payment_reversed" + case.last_connector_event_id = connector_event_id + case.version += 1 + case.updated_at = datetime.now(UTC) + audit = self._audit_event( + case, + connector_event_id=connector_event_id, + action="reopened", + fingerprint=verified.request_fingerprint, + before=before, + reason="payment_reversed", + correlation_id=envelope.correlation_id, + ) + return ReconciliationResult( + case=case, + audit_event=audit, + processing_status="processed", + error_code=None, + claim_id=claim.id, + expense_case_id=case.expense_case_id, + claim_status=str(claim.status or ""), + ) + + def _origin_graph( + self, + envelope: FinancialEventEnvelope, + verified: VerifiedConnectorRequest, + ) -> tuple[ + FinancialConnectorEvent | None, + Any | None, + PaymentReconciliationCase | None, + ]: + origin = self.db.scalar( + select(FinancialConnectorEvent).where( + FinancialConnectorEvent.tenant_id == envelope.tenant_id, + FinancialConnectorEvent.provider == verified.config.provider, + FinancialConnectorEvent.environment == "production", + FinancialConnectorEvent.verification_level == "production_verified", + FinancialConnectorEvent.external_event_id + == str(envelope.payload.get("origin_external_event_id") or ""), + ) + ) + if origin is None or not origin.claim_id: + return None, None, None + claim = self.actions.lock_claim( + tenant_id=envelope.tenant_id, + claim_id=origin.claim_id, + ) + case = self._lock_case(envelope.tenant_id, verified.config.provider, origin.claim_id) + return origin, claim, case + + def _lock_case( + self, + tenant_id: str, + provider: str, + claim_id: str, + ) -> PaymentReconciliationCase | None: + statement = select(PaymentReconciliationCase).where( + PaymentReconciliationCase.tenant_id == tenant_id, + PaymentReconciliationCase.provider == provider, + PaymentReconciliationCase.claim_id == claim_id, + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + return self.db.scalar(statement.execution_options(populate_existing=True)) + + @staticmethod + def _build_or_update_case( + existing: PaymentReconciliationCase | None, + *, + tenant_id: str, + provider: str, + claim: Any, + expense_case_id: str, + actual_amount: Decimal, + actual_currency: str, + external_reference: str, + status: str, + exception_code: str | None, + connector_event_id: str, + reset_erp: bool = False, + ) -> PaymentReconciliationCase: + expected = Decimal(claim.amount or Decimal("0.00")).quantize(Decimal("0.01")) + now = datetime.now(UTC) + row = existing or PaymentReconciliationCase( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + provider=provider, + claim_id=claim.id, + expected_reference=str(claim.claim_no or ""), + erp_status="pending_posting", + version=0, + created_at=now, + ) + row.expense_case_id = expense_case_id + row.expected_amount = expected + row.actual_amount = actual_amount + row.amount_difference = actual_amount - expected + row.expected_currency = str(claim.currency or "CNY").upper() + row.actual_currency = (actual_currency or row.expected_currency).upper() + row.external_reference_tail = external_reference or None + row.status = status + row.exception_code = exception_code + row.last_connector_event_id = connector_event_id + if reset_erp: + row.erp_status = "pending_posting" + row.erp_document_tail = None + row.erp_document_hash = None + row.version += 1 + row.updated_at = now + return row + + @staticmethod + def _audit_event( + case: PaymentReconciliationCase, + *, + connector_event_id: str, + action: str, + fingerprint: str, + before: dict[str, Any], + reason: str | None, + correlation_id: str, + ) -> PaymentReconciliationEvent: + return PaymentReconciliationEvent( + id=str(uuid.uuid4()), + tenant_id=case.tenant_id, + reconciliation_case_id=case.id, + connector_event_id=connector_event_id, + action=action, + actor_type="connector", + actor_id=f"financial-connector:{case.provider}", + request_fingerprint=fingerprint, + before_json=before, + after_json=_case_state(case), + response_json={"status": case.status, "version": case.version}, + reason=reason, + correlation_id=correlation_id, + occurred_at=datetime.now(UTC), + ) + + def _existing_case_exception( + self, + case: PaymentReconciliationCase, + *, + claim_status: str, + connector_event_id: str, + fingerprint: str, + correlation_id: str, + error_code: str, + ) -> ReconciliationResult: + before = _case_state(case) + case.status = "exception" + case.exception_code = error_code + case.last_connector_event_id = connector_event_id + case.version += 1 + case.updated_at = datetime.now(UTC) + return ReconciliationResult( + case=case, + audit_event=self._audit_event( + case, + connector_event_id=connector_event_id, + action="exception_created", + fingerprint=fingerprint, + before=before, + reason=error_code, + correlation_id=correlation_id, + ), + processing_status="exception", + error_code=error_code, + claim_id=case.claim_id, + expense_case_id=case.expense_case_id, + claim_status=claim_status, + ) + + @staticmethod + def _without_case(error_code: str) -> ReconciliationResult: + return ReconciliationResult( + case=None, + audit_event=None, + processing_status="exception", + error_code=error_code, + claim_id=None, + expense_case_id=None, + claim_status=None, + ) + + +def _amount(value: Any) -> Decimal | None: + try: + return Decimal(str(value)).quantize(Decimal("0.01")) + except (InvalidOperation, TypeError, ValueError): + return None + + +def _tail(value: Any) -> str: + return str(value or "").strip()[-8:] + + +def _case_state(case: PaymentReconciliationCase | None) -> dict[str, Any]: + if case is None: + return {} + return { + "status": str(case.status or ""), + "exception_code": case.exception_code, + "erp_status": str(case.erp_status or ""), + "version": int(case.version or 0), + "last_connector_event_id": str(case.last_connector_event_id or ""), + } diff --git a/server/src/app/services/risk_disposition_learning.py b/server/src/app/services/risk_disposition_learning.py new file mode 100644 index 0000000..9a21ca2 --- /dev/null +++ b/server/src/app/services/risk_disposition_learning.py @@ -0,0 +1,44 @@ +from __future__ import annotations + +from sqlalchemy.orm import Session + +from app.models.financial_record import ExpenseClaim +from app.models.risk_disposition import RiskDispositionEvent +from app.models.risk_observation import RiskObservation +from app.services.expense_cases import ExpenseCaseService + + +class RiskDispositionLearningBridge: + """把类型化风险处置结论桥接为同事务费用事件。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def record( + self, + *, + claim: ExpenseClaim | None, + observation: RiskObservation, + event: RiskDispositionEvent, + action: str, + actor_id: str, + ) -> None: + # 评论、原因等自由文本刻意不进入事件，避免成为高置信学习证据。 + if claim is None or action not in {"confirm", "false_positive", "resolve"}: + return + ExpenseCaseService(self.db).record_claim_event( + claim, + event_type="audit_conclusion_recorded", + actor_id=actor_id, + tenant_id=observation.tenant_id, + correlation_id=event.request_id, + idempotency_key=f"risk-disposition:{event.id}", + update_case_state=False, + extra_payload={ + "audit_decision": action, + "risk_observation_id": observation.id, + "risk_type": str(observation.risk_type or "").strip()[:80], + "risk_signal": str(observation.risk_signal or "").strip()[:100], + "risk_level": str(observation.risk_level or "").strip()[:20], + }, + ) diff --git a/server/src/app/services/risk_disposition_release_sync.py b/server/src/app/services/risk_disposition_release_sync.py new file mode 100644 index 0000000..495924f --- /dev/null +++ b/server/src/app/services/risk_disposition_release_sync.py @@ -0,0 +1,77 @@ +"""风险人工结论提交后的发布标签与门禁同步。""" + +from __future__ import annotations + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.core.logging import get_logger +from app.models.agent_asset import AgentAsset +from app.services.agent_asset_release_disposition_labels import ( + AgentAssetReleaseDispositionLabelService, +) +from app.services.agent_asset_release_monitor import AgentAssetReleaseMonitor + +logger = get_logger("app.services.risk_disposition_release_sync") + + +def sync_risk_disposition_release( + db: Session, + *, + tenant_id: str, + disposition_event_id: str, +) -> None: + """追加可信标签并用最新真实样本触发租户级发布门禁。""" + + try: + labels = AgentAssetReleaseDispositionLabelService(db).record_current_labels( + tenant_id=tenant_id, + disposition_event_id=disposition_event_id, + ) + if not labels: + return + db.commit() + except Exception: + db.rollback() + # 发布学习链路故障不能撤销已成功的人工风险处置。 + logger.exception( + "release telemetry labeling failed for disposition_event_id=%s", + disposition_event_id, + ) + return + + for asset_id in sorted({label.asset_id for label in labels}): + asset = db.scalar( + select(AgentAsset).where( + AgentAsset.id == asset_id, + AgentAsset.scope == "tenant", + AgentAsset.tenant_id == tenant_id, + ) + ) + config = ( + asset.config_json + if asset is not None and isinstance(asset.config_json, dict) + else {} + ) + # 全局资产的门禁必须由平台聚合任务处理，不能由单租户样本单独回滚。 + if ( + asset is None + or asset.scope != "tenant" + or asset.tenant_id != tenant_id + or str(config.get("tenant_id") or "").strip() not in {"", tenant_id} + ): + continue + try: + AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id=tenant_id, + asset_id=asset_id, + actor="release-telemetry-monitor", + ) + except Exception: + db.rollback() + # 真实标签已经提交，周期任务可据此安全重试门禁评测。 + logger.exception( + "release monitor failed after disposition_event_id=%s asset_id=%s", + disposition_event_id, + asset_id, + ) diff --git a/server/src/app/services/risk_dispositions.py b/server/src/app/services/risk_dispositions.py index 518a37a..47bd177 100644 --- a/server/src/app/services/risk_dispositions.py +++ b/server/src/app/services/risk_dispositions.py @@ -28,6 +28,8 @@ from app.services.approval_task_projection_refresh import ( ApprovalTaskProjectionRefreshService, ) from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.risk_disposition_learning import RiskDispositionLearningBridge +from app.services.risk_disposition_release_sync import sync_risk_disposition_release from app.services.risk_observation_access_policy import RiskObservationAccessPolicy from app.services.risk_waiver_decision_policy import ( RISK_WAIVER_DECISION_ACTIONS, @@ -131,6 +133,11 @@ class RiskDispositionService: fingerprint=fingerprint, ) if replay is not None: + sync_risk_disposition_release( + self.db, + tenant_id=normalized_tenant, + disposition_event_id=replay.event.id, + ) return replay try: @@ -251,6 +258,13 @@ class RiskDispositionService: # 风险事实与审批任务队列投影必须原子可见；autoflush 关闭时先显式 # 落下 observation/disposition，再按最新风险状态重算开放任务。 self.db.flush() + RiskDispositionLearningBridge(self.db).record( + claim=locked_claim, + observation=observation, + event=event, + action=payload.action, + actor_id=waiver_actor_id, + ) ApprovalTaskProjectionRefreshService(self.db).refresh_claim( tenant_id=normalized_tenant, claim_id=str(observation.claim_id or "").strip(), @@ -259,6 +273,11 @@ class RiskDispositionService: self.db.commit() self.db.refresh(disposition) self.db.refresh(event) + sync_risk_disposition_release( + self.db, + tenant_id=normalized_tenant, + disposition_event_id=event.id, + ) if legacy_feedback is not None: self.db.refresh(legacy_feedback) self._ingest_feedback_sample(observation, legacy_feedback) @@ -282,6 +301,11 @@ class RiskDispositionService: fingerprint=fingerprint, ) if replay is not None: + sync_risk_disposition_release( + self.db, + tenant_id=normalized_tenant, + disposition_event_id=replay.event.id, + ) return replay raise RiskDispositionConflictError( "Risk disposition was changed concurrently; reload and retry." diff --git a/server/src/app/services/risk_rule_generation.py b/server/src/app/services/risk_rule_generation.py index 16eee60..d188f48 100644 --- a/server/src/app/services/risk_rule_generation.py +++ b/server/src/app/services/risk_rule_generation.py @@ -1,30 +1,28 @@ from __future__ import annotations import json -import re from datetime import UTC, datetime from typing import Any from sqlalchemy.orm import Session -from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.core.agent_enums import AgentAssetStatus, AgentAssetType from app.models.agent_asset import AgentAsset, AgentAssetVersion from app.schemas.agent_asset import AgentAssetRiskRuleGenerateRequest +from app.services.agent_asset_access import platform_resource_identity, tenant_resource_identity from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.audit import AuditLogService from app.services.expense_claim_risk_stage import infer_risk_domain from app.services.risk_rule_dsl_validator import validate_risk_rule_draft from app.services.risk_rule_explainability import build_risk_rule_explainability_artifacts +from app.services.risk_rule_generation_fields import RiskRuleGenerationFieldMixin from app.services.risk_rule_generation_interpreter import COMPOSITE_RULE_TEMPLATE_KEY from app.services.risk_rule_generation_markdown import build_risk_rule_version_markdown from app.services.risk_rule_generation_ontology import ( BUSINESS_DOMAIN_LABELS, - DOMAIN_FIELD_PREFIXES, EXPENSE_BUSINESS_STAGE_LABELS, - EXPENSE_RISK_CATEGORY_ALIASES, EXPENSE_RISK_CATEGORY_LABELS, - FIELD_ONTOLOGY, RISK_LEVEL_LABELS, RiskRuleField, ) @@ -32,15 +30,13 @@ from app.services.risk_rule_generation_prompt import build_risk_rule_compiler_me from app.services.risk_rule_generation_semantic_plan import unwrap_semantic_plan_payload from app.services.risk_rule_generation_semantics import ( CITY_CONSISTENCY_SEMANTIC_TYPE, - CITY_CONSISTENCY_SEMANTIC_TYPES, - build_city_consistency_draft, build_city_consistency_params, ) from app.services.risk_rule_scoring import apply_risk_score_to_draft, calculate_risk_rule_score from app.services.runtime_chat import RuntimeChatService -class RiskRuleGenerationService: +class RiskRuleGenerationService(RiskRuleGenerationFieldMixin): def __init__( self, db: Session, @@ -133,6 +129,12 @@ class RiskRuleGenerationService: ) rule_code = str(payload["rule_code"]) file_name = f"{rule_code}.json" + normalized_tenant = str(tenant_id or "").strip() + resource_tenant_id, resource_scope = ( + tenant_resource_identity(normalized_tenant) + if normalized_tenant + else platform_resource_identity() + ) self.rule_library_manager.write_rule_library_json( library=RISK_RULES_LIBRARY, @@ -141,6 +143,8 @@ class RiskRuleGenerationService: ) asset = AgentAsset( + tenant_id=resource_tenant_id, + scope=resource_scope, asset_type=AgentAssetType.RULE.value, code=rule_code, name=str(payload["name"]), @@ -189,6 +193,8 @@ class RiskRuleGenerationService: self.db.flush() self.db.add( AgentAssetVersion( + tenant_id=asset.tenant_id, + scope=asset.scope, asset_id=asset.id, version="v0.1.0", content=build_risk_rule_version_markdown(payload), @@ -611,249 +617,3 @@ class RiskRuleGenerationService: } ) return payload - - @staticmethod - def _normalize_expense_category(value: str | None, domain: str) -> str | None: - if domain != AgentAssetDomain.EXPENSE.value: - return None - - normalized = str(value or "").strip().lower() - if not normalized: - return None - - normalized = EXPENSE_RISK_CATEGORY_ALIASES.get(normalized, normalized) - if normalized not in EXPENSE_RISK_CATEGORY_LABELS: - allowed = "、".join(EXPENSE_RISK_CATEGORY_LABELS.values()) - raise ValueError(f"费用领域仅支持：{allowed}。") - return normalized - - @staticmethod - def _normalize_business_stage(value: str | None, domain: str) -> str: - if domain != AgentAssetDomain.EXPENSE.value: - return "reimbursement" - - normalized = str(value or "reimbursement").strip().lower() - if not normalized: - normalized = "reimbursement" - if normalized not in EXPENSE_BUSINESS_STAGE_LABELS: - allowed = "、".join(EXPENSE_BUSINESS_STAGE_LABELS.values()) - raise ValueError(f"业务环节仅支持：{allowed}。") - return normalized - - def _resolve_fields(self, text: str, *, domain: str) -> list[RiskRuleField]: - prefixes = DOMAIN_FIELD_PREFIXES.get(domain, ()) - candidates = [field for field in FIELD_ONTOLOGY if field.key.startswith(prefixes)] - normalized = text.lower() - matched: list[tuple[int, RiskRuleField]] = [] - for field in candidates: - score = self._score_field_match(field, text, normalized) - if score > 0: - matched.append((score, field)) - - if domain == AgentAssetDomain.EXPENSE.value: - if any(keyword in text for keyword in ("住宿", "酒店", "行程", "城市", "出差")): - matched.extend( - (10, field) - for field in candidates - if field.key - in { - "claim.reason", - "claim.location", - "item.item_date", - "item.item_reason", - "item.item_location", - "attachment.hotel_city", - "attachment.route_cities", - "attachment.issue_date", - "attachment.stay_start_date", - "attachment.stay_end_date", - } - ) - if any(keyword in text for keyword in ("发票", "票据", "品名", "抬头", "开票")): - matched.extend( - (6, field) - for field in candidates - if field.key - in { - "attachment.invoice_no", - "attachment.buyer_name", - "attachment.goods_name", - "attachment.ocr_text", - } - ) - - matched.sort(key=lambda item: item[0], reverse=True) - deduped: list[RiskRuleField] = [] - seen: set[str] = set() - for _, field in matched: - if field.key in seen: - continue - seen.add(field.key) - deduped.append(field) - if deduped: - return deduped[:10] - return candidates[:4] - - @staticmethod - def _score_field_match(field: RiskRuleField, text: str, normalized: str) -> int: - score = 0 - if field.label in text: - score += 8 - for alias in field.aliases: - if alias.lower() in normalized: - score += 4 + min(len(alias), 6) - - if field.key == "attachment.hotel_city" and any(term in text for term in ("酒店", "住宿")): - score += 12 - if field.key == "attachment.route_cities" and any( - term in text for term in ("行程", "交通票", "路线", "途经") - ): - score += 10 - if field.key in { - "claim.trip_start_date", - "claim.trip_end_date", - "item.item_date", - "attachment.stay_start_date", - "attachment.stay_end_date", - } and any(term in text for term in ("日期", "时间", "出差开始", "出差结束", "入住", "离店")): - score += 10 - if field.key == "claim.location" and any( - term in text for term in ("申报目的地", "申报地点", "目的地", "出差地") - ): - score += 10 - if field.key.startswith("attachment.") and any(term in text for term in ("发票", "票据")): - score += 2 - return score - - def _align_draft_fields( - self, - draft: dict[str, Any], - *, - natural_language: str, - risk_level: str, - fields: list[RiskRuleField], - ) -> dict[str, Any]: - if str(draft.get("semantic_type") or "").strip() in CITY_CONSISTENCY_SEMANTIC_TYPES: - return build_city_consistency_draft( - draft, - natural_language=natural_language, - fields=fields, - risk_level=risk_level, - ) - - field_by_key = {field.key: field for field in fields} - original_keys = [ - str(item or "").strip() - for item in list(draft.get("field_keys") or []) - if str(item or "").strip() in field_by_key - ] - if draft.get("template_key") == COMPOSITE_RULE_TEMPLATE_KEY: - return {**draft, "field_keys": original_keys or [field.key for field in fields[:8]]} - - preferred_keys: list[str] = [] - - def add_preferred(key: str, *terms: str) -> None: - if key in field_by_key and any(term in natural_language for term in terms): - preferred_keys.append(key) - - add_preferred("attachment.hotel_city", "酒店", "住宿") - add_preferred("claim.location", "申报目的地", "申报地点", "目的地", "出差地") - add_preferred("attachment.route_cities", "行程", "交通票", "路线", "途经") - - merged_keys: list[str] = [] - for key in [*preferred_keys, *original_keys, *[field.key for field in fields]]: - if key in field_by_key and key not in merged_keys: - merged_keys.append(key) - if len(merged_keys) >= 4: - break - - if draft.get("template_key") == "field_compare_v1" and len(merged_keys) < 2: - for field in fields: - if field.key not in merged_keys: - merged_keys.append(field.key) - if len(merged_keys) >= 2: - break - - aligned = {**draft, "field_keys": merged_keys} - selected_fields = [field_by_key[key] for key in merged_keys if key in field_by_key] - if selected_fields: - aligned["condition_summary"] = self._build_condition_summary( - natural_language, - template_key=str(aligned.get("template_key") or "field_required_v1"), - fields=selected_fields, - ) - flow = aligned.get("flow") if isinstance(aligned.get("flow"), dict) else {} - aligned["flow"] = { - **flow, - "evidence": "读取" + "、".join(field.label for field in selected_fields[:3]), - "decision": aligned["condition_summary"], - } - return aligned - - @staticmethod - def _build_compare_conditions(field_keys: list[str]) -> list[dict[str, str]]: - if len(field_keys) >= 2: - return [{"left": field_keys[0], "operator": "overlap", "right": field_keys[1]}] - if field_keys: - return [{"left": field_keys[0], "operator": "is_empty", "right": ""}] - return [] - - @staticmethod - def _infer_template_key(text: str) -> str: - if any(keyword in text for keyword in ("超过", "超出", "超预算", "预算", "阈值", "早于", "晚于", "范围")): - return COMPOSITE_RULE_TEMPLATE_KEY - if any( - keyword in text - for keyword in ("一致", "匹配", "相同", "不一致", "不符", "对应", "出现在") - ): - return "field_compare_v1" - if any( - keyword in text - for keyword in ("关键词", "包含", "出现", "品名", "摘要", "服务费", "咨询费") - ): - return "keyword_match_v1" - return "field_required_v1" - - @staticmethod - def _infer_keywords(text: str) -> list[str]: - quoted = re.findall(r"[“\"']([^“”\"']{2,20})[”\"']", text) - keywords = [item.strip() for item in quoted if item.strip()] - for candidate in ("咨询费", "服务费", "其他", "办公用品", "招待", "红冲", "作废"): - if candidate in text and candidate not in keywords: - keywords.append(candidate) - return keywords[:8] - - @staticmethod - def _infer_rule_name(text: str) -> str: - normalized = re.sub(r"\s+", "", str(text or "")) - normalized = re.sub(r"[，。；;：:、,.!?！？]", "", normalized) - if not normalized: - return "自然语言风险规则" - return f"{normalized[:18]}风险规则" - - @staticmethod - def _build_condition_summary( - natural_language: str, - *, - template_key: str, - fields: list[RiskRuleField], - ) -> str: - field_text = "、".join(item.label for item in fields[:3]) or "业务字段" - if template_key == "field_compare_v1": - return f"对比{field_text}之间是否一致或存在交集" - if template_key == "keyword_match_v1": - return f"检查{field_text}是否出现规则描述中的风险关键词" - return f"检查{field_text}是否满足必填和完整性要求" - - @staticmethod - def _clean_text(value: Any) -> str: - return re.sub(r"\s+", " ", str(value or "")).strip() - - @staticmethod - def _extract_json_object(text: str) -> str: - normalized = re.sub(r"^```(?:json)?|```$", "", str(text or "").strip(), flags=re.IGNORECASE) - start = normalized.find("{") - end = normalized.rfind("}") - if start < 0 or end <= start: - raise ValueError("JSON object not found.") - return normalized[start : end + 1] diff --git a/server/src/app/services/risk_rule_generation_fields.py b/server/src/app/services/risk_rule_generation_fields.py new file mode 100644 index 0000000..96ade5d --- /dev/null +++ b/server/src/app/services/risk_rule_generation_fields.py @@ -0,0 +1,277 @@ +from __future__ import annotations + +import re +from typing import Any + +from app.core.agent_enums import AgentAssetDomain +from app.services.risk_rule_generation_interpreter import COMPOSITE_RULE_TEMPLATE_KEY +from app.services.risk_rule_generation_ontology import ( + DOMAIN_FIELD_PREFIXES, + EXPENSE_BUSINESS_STAGE_LABELS, + EXPENSE_RISK_CATEGORY_ALIASES, + EXPENSE_RISK_CATEGORY_LABELS, + FIELD_ONTOLOGY, + RiskRuleField, +) +from app.services.risk_rule_generation_semantics import ( + CITY_CONSISTENCY_SEMANTIC_TYPES, + build_city_consistency_draft, +) + + +class RiskRuleGenerationFieldMixin: + @staticmethod + def _normalize_expense_category(value: str | None, domain: str) -> str | None: + if domain != AgentAssetDomain.EXPENSE.value: + return None + + normalized = str(value or "").strip().lower() + if not normalized: + return None + + normalized = EXPENSE_RISK_CATEGORY_ALIASES.get(normalized, normalized) + if normalized not in EXPENSE_RISK_CATEGORY_LABELS: + allowed = "、".join(EXPENSE_RISK_CATEGORY_LABELS.values()) + raise ValueError(f"费用领域仅支持：{allowed}。") + return normalized + + @staticmethod + def _normalize_business_stage(value: str | None, domain: str) -> str: + if domain != AgentAssetDomain.EXPENSE.value: + return "reimbursement" + + normalized = str(value or "reimbursement").strip().lower() + if not normalized: + normalized = "reimbursement" + if normalized not in EXPENSE_BUSINESS_STAGE_LABELS: + allowed = "、".join(EXPENSE_BUSINESS_STAGE_LABELS.values()) + raise ValueError(f"业务环节仅支持：{allowed}。") + return normalized + + def _resolve_fields(self, text: str, *, domain: str) -> list[RiskRuleField]: + prefixes = DOMAIN_FIELD_PREFIXES.get(domain, ()) + candidates = [field for field in FIELD_ONTOLOGY if field.key.startswith(prefixes)] + normalized = text.lower() + matched: list[tuple[int, RiskRuleField]] = [] + for field in candidates: + score = self._score_field_match(field, text, normalized) + if score > 0: + matched.append((score, field)) + + if domain == AgentAssetDomain.EXPENSE.value: + if any(keyword in text for keyword in ("住宿", "酒店", "行程", "城市", "出差")): + matched.extend( + (10, field) + for field in candidates + if field.key + in { + "claim.reason", + "claim.location", + "item.item_date", + "item.item_reason", + "item.item_location", + "attachment.hotel_city", + "attachment.route_cities", + "attachment.issue_date", + "attachment.stay_start_date", + "attachment.stay_end_date", + } + ) + if any(keyword in text for keyword in ("发票", "票据", "品名", "抬头", "开票")): + matched.extend( + (6, field) + for field in candidates + if field.key + in { + "attachment.invoice_no", + "attachment.buyer_name", + "attachment.goods_name", + "attachment.ocr_text", + } + ) + + matched.sort(key=lambda item: item[0], reverse=True) + deduped: list[RiskRuleField] = [] + seen: set[str] = set() + for _, field in matched: + if field.key in seen: + continue + seen.add(field.key) + deduped.append(field) + if deduped: + return deduped[:10] + return candidates[:4] + + @staticmethod + def _score_field_match(field: RiskRuleField, text: str, normalized: str) -> int: + score = 0 + if field.label in text: + score += 8 + for alias in field.aliases: + if alias.lower() in normalized: + score += 4 + min(len(alias), 6) + + if field.key == "attachment.hotel_city" and any(term in text for term in ("酒店", "住宿")): + score += 12 + if field.key == "attachment.route_cities" and any( + term in text for term in ("行程", "交通票", "路线", "途经") + ): + score += 10 + if field.key in { + "claim.trip_start_date", + "claim.trip_end_date", + "item.item_date", + "attachment.stay_start_date", + "attachment.stay_end_date", + } and any( + term in text for term in ("日期", "时间", "出差开始", "出差结束", "入住", "离店") + ): + score += 10 + if field.key == "claim.location" and any( + term in text for term in ("申报目的地", "申报地点", "目的地", "出差地") + ): + score += 10 + if field.key.startswith("attachment.") and any(term in text for term in ("发票", "票据")): + score += 2 + return score + + def _align_draft_fields( + self, + draft: dict[str, Any], + *, + natural_language: str, + risk_level: str, + fields: list[RiskRuleField], + ) -> dict[str, Any]: + if str(draft.get("semantic_type") or "").strip() in CITY_CONSISTENCY_SEMANTIC_TYPES: + return build_city_consistency_draft( + draft, + natural_language=natural_language, + fields=fields, + risk_level=risk_level, + ) + + field_by_key = {field.key: field for field in fields} + original_keys = [ + str(item or "").strip() + for item in list(draft.get("field_keys") or []) + if str(item or "").strip() in field_by_key + ] + if draft.get("template_key") == COMPOSITE_RULE_TEMPLATE_KEY: + return {**draft, "field_keys": original_keys or [field.key for field in fields[:8]]} + + preferred_keys: list[str] = [] + + def add_preferred(key: str, *terms: str) -> None: + if key in field_by_key and any(term in natural_language for term in terms): + preferred_keys.append(key) + + add_preferred("attachment.hotel_city", "酒店", "住宿") + add_preferred("claim.location", "申报目的地", "申报地点", "目的地", "出差地") + add_preferred("attachment.route_cities", "行程", "交通票", "路线", "途经") + + merged_keys: list[str] = [] + for key in [*preferred_keys, *original_keys, *[field.key for field in fields]]: + if key in field_by_key and key not in merged_keys: + merged_keys.append(key) + if len(merged_keys) >= 4: + break + + if draft.get("template_key") == "field_compare_v1" and len(merged_keys) < 2: + for field in fields: + if field.key not in merged_keys: + merged_keys.append(field.key) + if len(merged_keys) >= 2: + break + + aligned = {**draft, "field_keys": merged_keys} + selected_fields = [field_by_key[key] for key in merged_keys if key in field_by_key] + if selected_fields: + aligned["condition_summary"] = self._build_condition_summary( + natural_language, + template_key=str(aligned.get("template_key") or "field_required_v1"), + fields=selected_fields, + ) + flow = aligned.get("flow") if isinstance(aligned.get("flow"), dict) else {} + aligned["flow"] = { + **flow, + "evidence": "读取" + "、".join(field.label for field in selected_fields[:3]), + "decision": aligned["condition_summary"], + } + return aligned + + @staticmethod + def _build_compare_conditions(field_keys: list[str]) -> list[dict[str, str]]: + if len(field_keys) >= 2: + return [{"left": field_keys[0], "operator": "overlap", "right": field_keys[1]}] + if field_keys: + return [{"left": field_keys[0], "operator": "is_empty", "right": ""}] + return [] + + @staticmethod + def _infer_template_key(text: str) -> str: + if any( + keyword in text + for keyword in ("超过", "超出", "超预算", "预算", "阈值", "早于", "晚于", "范围") + ): + return COMPOSITE_RULE_TEMPLATE_KEY + if any( + keyword in text + for keyword in ("一致", "匹配", "相同", "不一致", "不符", "对应", "出现在") + ): + return "field_compare_v1" + if any( + keyword in text + for keyword in ("关键词", "包含", "出现", "品名", "摘要", "服务费", "咨询费") + ): + return "keyword_match_v1" + return "field_required_v1" + + @staticmethod + def _infer_keywords(text: str) -> list[str]: + quoted = re.findall(r"[“\"']([^“”\"']{2,20})[”\"']", text) + keywords = [item.strip() for item in quoted if item.strip()] + for candidate in ("咨询费", "服务费", "其他", "办公用品", "招待", "红冲", "作废"): + if candidate in text and candidate not in keywords: + keywords.append(candidate) + return keywords[:8] + + @staticmethod + def _infer_rule_name(text: str) -> str: + normalized = re.sub(r"\s+", "", str(text or "")) + normalized = re.sub(r"[，。；;：:、,.!?！？]", "", normalized) + if not normalized: + return "自然语言风险规则" + return f"{normalized[:18]}风险规则" + + @staticmethod + def _build_condition_summary( + natural_language: str, + *, + template_key: str, + fields: list[RiskRuleField], + ) -> str: + field_text = "、".join(item.label for item in fields[:3]) or "业务字段" + if template_key == "field_compare_v1": + return f"对比{field_text}之间是否一致或存在交集" + if template_key == "keyword_match_v1": + return f"检查{field_text}是否出现规则描述中的风险关键词" + return f"检查{field_text}是否满足必填和完整性要求" + + @staticmethod + def _clean_text(value: Any) -> str: + return re.sub(r"\s+", " ", str(value or "")).strip() + + @staticmethod + def _extract_json_object(text: str) -> str: + normalized = re.sub( + r"^```(?:json)?|```$", + "", + str(text or "").strip(), + flags=re.IGNORECASE, + ) + start = normalized.find("{") + end = normalized.rfind("}") + if start < 0 or end <= start: + raise ValueError("JSON object not found.") + return normalized[start : end + 1] diff --git a/server/src/app/services/risk_rule_generation_jobs.py b/server/src/app/services/risk_rule_generation_jobs.py index 900dff6..5ddd5f4 100644 --- a/server/src/app/services/risk_rule_generation_jobs.py +++ b/server/src/app/services/risk_rule_generation_jobs.py @@ -8,6 +8,7 @@ from sqlalchemy.orm import Session from app.core.agent_enums import AgentAssetStatus, AgentAssetType from app.models.agent_asset import AgentAsset, AgentAssetVersion from app.schemas.agent_asset import AgentAssetRiskRuleGenerateRequest +from app.services.agent_asset_access import platform_resource_identity, tenant_resource_identity from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.audit import AuditLogService from app.services.risk_rule_generation import ( @@ -65,8 +66,16 @@ class RiskRuleGenerationJobService: category_label = expense_category_label or BUSINESS_DOMAIN_LABELS[domain] display_name = rule_title or self.generator._infer_rule_name(natural_language) file_name = f"{rule_code}.json" + normalized_tenant = str(tenant_id or "").strip() + resource_tenant_id, resource_scope = ( + tenant_resource_identity(normalized_tenant) + if normalized_tenant + else platform_resource_identity() + ) asset = AgentAsset( + tenant_id=resource_tenant_id, + scope=resource_scope, asset_type=AgentAssetType.RULE.value, code=rule_code, name=display_name, @@ -138,7 +147,11 @@ class RiskRuleGenerationJobService: ) -> None: try: asset = self.db.get(AgentAsset, asset_id) - if asset is None or asset.status != AgentAssetStatus.GENERATING.value: + if ( + asset is None + or not self._matches_tenant(asset, tenant_id) + or asset.status != AgentAssetStatus.GENERATING.value + ): return self._complete_rule_asset( asset, @@ -152,6 +165,7 @@ class RiskRuleGenerationJobService: asset_id, error_message=str(exc) or exc.__class__.__name__, actor=actor, + tenant_id=tenant_id, request_id=request_id, ) @@ -161,10 +175,11 @@ class RiskRuleGenerationJobService: *, error_message: str, actor: str, + tenant_id: str | None = None, request_id: str | None = None, ) -> None: asset = self.db.get(AgentAsset, asset_id) - if asset is None: + if asset is None or not self._matches_tenant(asset, tenant_id): return config_json = dict(asset.config_json or {}) @@ -317,6 +332,8 @@ class RiskRuleGenerationJobService: self.db.add(asset) self.db.add( AgentAssetVersion( + tenant_id=asset.tenant_id, + scope=asset.scope, asset_id=asset.id, version="v0.1.0", content=build_risk_rule_version_markdown(payload), @@ -350,6 +367,13 @@ class RiskRuleGenerationJobService: raise ValueError("当前仅支持报销、应收、应付业务域的新建风险规则。") return domain + @staticmethod + def _matches_tenant(asset: AgentAsset, tenant_id: str | None) -> bool: + normalized = str(tenant_id or "").strip() + if not normalized: + return asset.scope == "platform" and asset.tenant_id == "platform" + return asset.scope == "tenant" and asset.tenant_id == normalized + def _validate_natural_language(self, body: AgentAssetRiskRuleGenerateRequest) -> str: natural_language = self.generator._clean_text(body.natural_language) if len(natural_language) < 8: diff --git a/server/src/app/services/risk_rule_golden_evaluator.py b/server/src/app/services/risk_rule_golden_evaluator.py index 9abebab..d24b81f 100644 --- a/server/src/app/services/risk_rule_golden_evaluator.py +++ b/server/src/app/services/risk_rule_golden_evaluator.py @@ -24,7 +24,6 @@ from typing import Any from sqlalchemy import select from sqlalchemy.orm import Session -from app.core.agent_enums import AgentAssetType from app.core.logging import get_logger from app.models.agent_asset import AgentAsset, AgentAssetTestRun from app.models.employee import Employee @@ -60,6 +59,8 @@ class GoldenEvalReport: precision: float = 0.0 recall: float = 0.0 all_passed: bool = True + gate_status: str = "evaluated" + failure_reason: str = "" results: list[GoldenCaseResult] = field(default_factory=list) def to_dict(self) -> dict[str, Any]: @@ -71,6 +72,8 @@ class GoldenEvalReport: "precision": round(self.precision, 4), "recall": round(self.recall, 4), "all_passed": self.all_passed, + "gate_status": self.gate_status, + "failure_reason": self.failure_reason, "results": [ { "case_id": r.case_id, @@ -93,6 +96,7 @@ def _gate_enabled() -> bool: # ---- synthetic claim 构建（与 AgentAssetRiskRuleTestingMixin._build_synthetic_claim 一致）---- + def _extract_manifest_fields(manifest: dict[str, Any]) -> list[dict[str, str]]: inputs = manifest.get("inputs") if isinstance(manifest.get("inputs"), dict) else {} fields = inputs.get("fields") if isinstance(inputs.get("fields"), list) else [] @@ -156,8 +160,8 @@ def _build_synthetic_claim( attachment_fields: list[dict[str, Any]] = [] document_info: dict[str, Any] = {"fields": attachment_fields} - for field in _extract_manifest_fields(manifest): - key = field["key"] + for manifest_field in _extract_manifest_fields(manifest): + key = manifest_field["key"] if key not in values: continue value = _coerce_sample_value(key, values.get(key)) @@ -168,7 +172,9 @@ def _build_synthetic_claim( elif key.startswith("attachment."): short_key = key.removeprefix("attachment.") document_info[short_key] = value - attachment_fields.append({"key": short_key, "label": field["label"], "value": value}) + attachment_fields.append( + {"key": short_key, "label": manifest_field["label"], "value": value} + ) return claim, [{"document_info": document_info, "ocr_text": document_info.get("ocr_text", "")}] @@ -179,7 +185,9 @@ def _run_single_case( expected_severity: str, ) -> GoldenCaseResult: claim, contexts = _build_synthetic_claim(values, manifest) - execution = RiskRuleTemplateExecutor().evaluate_with_trace(manifest, claim=claim, contexts=contexts) + execution = RiskRuleTemplateExecutor().evaluate_with_trace( + manifest, claim=claim, contexts=contexts + ) result = execution["result"] actual_hit = result is not None actual_severity = ( @@ -211,8 +219,8 @@ def _aggregate(results: list[GoldenCaseResult]) -> GoldenEvalReport: return GoldenEvalReport(total=0, all_passed=True) passed_count = sum(1 for r in results if r.passed) tp = sum(1 for r in results if r.expected_hit and r.actual_hit) - fp = sum(1 for r in results if r.expected_hit and not r.actual_hit) # 应命中未命中 - fn = sum(1 for r in results if not r.expected_hit and r.actual_hit) # 不应命中却命中 + fp = sum(1 for r in results if not r.expected_hit and r.actual_hit) # 不应命中却命中 + fn = sum(1 for r in results if r.expected_hit and not r.actual_hit) # 应命中未命中 precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0 recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0 return GoldenEvalReport( @@ -271,25 +279,61 @@ class RiskRuleGoldenEvaluator: rule_code: str, *, actor: str, + precondition_error: str = "", ) -> GoldenEvalReport: """发布门禁入口：跑 golden set，未 100% 通过抛 PermissionError。 - golden set 为空或门禁关闭时放行； evaluator 异常时降级放行（记日志）。 - 无论放行与否，都写一条 ``AgentAssetTestRun(test_type='golden')`` 记录。 + 只有显式关闭 feature flag 才跳过门禁；缺失用例、配置错误或评测异常 + 均 fail-closed。无论放行、拦截或显式跳过，都会留下测试记录。 """ if not _gate_enabled(): - return GoldenEvalReport(total=0, all_passed=True) + report = GoldenEvalReport( + total=0, + all_passed=True, + gate_status="skipped", + failure_reason="gate_explicitly_disabled", + ) + self._record_test_run(db, asset, version, report, rule_code=rule_code, actor=actor) + return report + if precondition_error: + report = GoldenEvalReport( + total=0, + all_passed=False, + gate_status="failed", + failure_reason=precondition_error, + ) + self._record_test_run(db, asset, version, report, rule_code=rule_code, actor=actor) + raise PermissionError(f"golden set 发布门禁前置条件失败：{precondition_error}") try: report = self.evaluate_for_rule(db, manifest, rule_code) - except Exception: - logger.exception("golden set 评测异常，降级放行 asset_id=%s", asset.id) - report = GoldenEvalReport(total=0, all_passed=True) + except Exception as exc: + logger.exception("golden set 评测异常，拦截发布 asset_id=%s", asset.id) + report = GoldenEvalReport( + total=0, + all_passed=False, + gate_status="failed", + failure_reason=f"evaluation_error:{type(exc).__name__}", + ) + self._record_test_run(db, asset, version, report, rule_code=rule_code, actor=actor) + raise PermissionError("golden set 评测异常，发布已按 fail-closed 策略拦截。") from exc - self._record_test_run(db, asset, version, report, actor=actor) + if report.total == 0: + report.all_passed = False + report.gate_status = "failed" + report.failure_reason = "missing_active_golden_cases" + elif report.all_passed: + report.gate_status = "passed" + else: + report.gate_status = "failed" + report.failure_reason = "golden_case_regression" - if report.total > 0 and not report.all_passed: + self._record_test_run(db, asset, version, report, rule_code=rule_code, actor=actor) + + if not report.all_passed: failures = report.to_dict()["results"] + if report.total == 0: + raise PermissionError("当前规则缺少必要的 active golden case，发布被拦截。") raise PermissionError( f"golden set 回归未通过（{report.passed_count}/{report.total}），" f"发布被拦截。失败用例：{failures}" @@ -303,27 +347,25 @@ class RiskRuleGoldenEvaluator: version: str, report: GoldenEvalReport, *, + rule_code: str, actor: str, ) -> None: - try: - run = AgentAssetTestRun( - id=str(uuid.uuid4()), - asset_id=asset.id, - version=version, - test_type="golden", - status="completed", - passed=report.all_passed, - summary=( - f"golden set {report.passed_count}/{report.total} passed" - if report.total > 0 - else "golden set empty, gate skipped" - ), - input_json={"rule_code": getattr(asset, "rule_code", "") or ""}, - result_json=report.to_dict(), - created_by=actor, - ) - db.add(run) - db.commit() - except Exception: - logger.warning("golden test run 记录失败 asset_id=%s", asset.id, exc_info=True) - db.rollback() + summary = f"golden set {report.passed_count}/{report.total} passed" + if report.gate_status == "skipped": + summary = "golden set gate explicitly disabled" + elif report.failure_reason: + summary = f"golden set gate failed: {report.failure_reason}" + run = AgentAssetTestRun( + id=str(uuid.uuid4()), + asset_id=asset.id, + version=version, + test_type="golden", + status=report.gate_status, + passed=report.all_passed, + summary=summary, + input_json={"rule_code": rule_code}, + result_json=report.to_dict(), + created_by=actor, + ) + db.add(run) + db.commit() diff --git a/server/src/app/services/runtime_chat.py b/server/src/app/services/runtime_chat.py index 0fdeb83..bc3721e 100644 --- a/server/src/app/services/runtime_chat.py +++ b/server/src/app/services/runtime_chat.py @@ -1,22 +1,31 @@ from __future__ import annotations -import json -from dataclasses import dataclass -from http import HTTPStatus from time import monotonic, sleep from typing import Any from sqlalchemy.orm import Session from app.core.logging import get_logger -from app.services.model_connectivity import ( - AZURE_API_VERSION, - ConnectivityCheckError, - _build_azure_deployment_base, - _build_headers, - _ensure_path, - _normalize_endpoint, - _send_json_request, +from app.services.model_connectivity import _send_json_request +from app.services.runtime_chat_attempts import ( + RuntimeChatAttemptObserver, + RuntimeChatAttemptOutcome, + RuntimeChatAttemptTracker, + RuntimeChatAuthoritativeUsage, + RuntimeChatCallTrace, + RuntimeChatOperationContext, + RuntimeChatProviderAttemptError, + RuntimeChatProviderResponse, + RuntimeChatResult, + RuntimeChatToolCall, + RuntimeToolCallResult, +) +from app.services.runtime_chat_provider import ( + request_azure_openai_completion, + request_azure_openai_tool_call, + request_ollama_completion, + request_openai_compatible_completion, + request_openai_compatible_tool_call, ) from app.services.settings import SettingsService @@ -35,60 +44,16 @@ def clear_runtime_chat_failure_cache() -> int: return cleared_count -@dataclass(slots=True) -class RuntimeChatCallTrace: - slot: str - provider: str - model: str - attempt: int - status: str - duration_ms: int = 0 - error_message: str | None = None - skipped_reason: str | None = None - - def model_dump(self) -> dict[str, Any]: - return { - "slot": self.slot, - "provider": self.provider, - "model": self.model, - "attempt": self.attempt, - "status": self.status, - "duration_ms": self.duration_ms, - "error_message": self.error_message, - "skipped_reason": self.skipped_reason, - } - - -@dataclass(slots=True) -class RuntimeChatResult: - text: str | None - calls: list[RuntimeChatCallTrace] - - def calls_as_dicts(self) -> list[dict[str, Any]]: - return [item.model_dump() for item in self.calls] - - -@dataclass(slots=True) -class RuntimeChatToolCall: - name: str - arguments: dict[str, Any] - call_id: str | None = None - raw_arguments: str = "" - - -@dataclass(slots=True) -class RuntimeToolCallResult: - tool_call: RuntimeChatToolCall | None - calls: list[RuntimeChatCallTrace] - - def calls_as_dicts(self) -> list[dict[str, Any]]: - return [item.model_dump() for item in self.calls] - - class RuntimeChatService: - def __init__(self, db: Session) -> None: + def __init__( + self, + db: Session, + *, + attempt_observer: RuntimeChatAttemptObserver | None = None, + ) -> None: self.db = db self.settings_service = SettingsService(db) + self.attempt_observer = attempt_observer def complete( self, @@ -100,6 +65,7 @@ class RuntimeChatService: timeout_seconds: int | None = None, slot_timeouts: dict[str, int] | None = None, max_attempts: int | None = None, + operation_context: RuntimeChatOperationContext | None = None, ) -> str | None: return self.complete_with_trace( messages, @@ -109,6 +75,7 @@ class RuntimeChatService: timeout_seconds=timeout_seconds, slot_timeouts=slot_timeouts, max_attempts=max_attempts, + operation_context=operation_context, ).text def complete_with_trace( @@ -121,6 +88,7 @@ class RuntimeChatService: timeout_seconds: int | None = None, slot_timeouts: dict[str, int] | None = None, max_attempts: int | None = None, + operation_context: RuntimeChatOperationContext | None = None, ) -> RuntimeChatResult: configs: list[dict[str, str]] = [] calls: list[RuntimeChatCallTrace] = [] @@ -167,40 +135,70 @@ class RuntimeChatService: ) continue started = monotonic() + tracker = RuntimeChatAttemptTracker( + observer=self.attempt_observer, + operation_context=operation_context, + slot=config["slot"], + provider=config["provider"], + model=config["model"], + attempt=attempt, + ) + permit = tracker.notify_permit() + if not permit.allowed: + permit_error = RuntimeError( + permit.reason or "Runtime chat attempt permit denied." + ) + calls.append( + self._complete_attempt_trace( + tracker=tracker, + config=config, + attempt=attempt, + status="blocked", + outcome="not_sent", + duration_ms=int((monotonic() - started) * 1000), + error=permit_error, + ) + ) + continue try: - response_text = self._request_chat_completion( - config, - messages, - max_tokens=max_tokens, - temperature=temperature, - timeout_seconds=resolved_slot_timeouts.get( - config["slot"], - resolved_timeout_seconds, - ), + provider_response = self._coerce_provider_response( + self._request_chat_completion( + config, + messages, + max_tokens=max_tokens, + temperature=temperature, + timeout_seconds=resolved_slot_timeouts.get( + config["slot"], + resolved_timeout_seconds, + ), + ) ) duration_ms = int((monotonic() - started) * 1000) + response_text = str(provider_response.output or "").strip() if response_text: _slot_failure_until.pop(cache_key, None) calls.append( - RuntimeChatCallTrace( - slot=config["slot"], - provider=config["provider"], - model=config["model"], + self._complete_attempt_trace( + tracker=tracker, + config=config, attempt=attempt, status="succeeded", + outcome="succeeded", duration_ms=duration_ms, + provider_response=provider_response, ) ) - return RuntimeChatResult(response_text.strip(), calls) + return RuntimeChatResult(response_text, calls) calls.append( - RuntimeChatCallTrace( - slot=config["slot"], - provider=config["provider"], - model=config["model"], + self._complete_attempt_trace( + tracker=tracker, + config=config, attempt=attempt, status="empty", + outcome="empty", duration_ms=duration_ms, - error_message="模型返回空内容。", + provider_response=provider_response, + error=ValueError("模型返回空内容。"), ) ) except Exception as exc: @@ -208,15 +206,26 @@ class RuntimeChatService: _slot_failure_until[cache_key] = ( monotonic() + DEFAULT_RUNTIME_CHAT_FAILURE_COOLDOWN_SECONDS ) + ( + outcome, + response_id, + response_model, + provider_status_code, + usage, + ) = self._error_attempt_metadata(exc) calls.append( - RuntimeChatCallTrace( - slot=config["slot"], - provider=config["provider"], - model=config["model"], + self._complete_attempt_trace( + tracker=tracker, + config=config, attempt=attempt, status="failed", + outcome=outcome, duration_ms=duration_ms, - error_message=str(exc), + response_id=response_id, + response_model=response_model, + provider_status_code=provider_status_code, + usage=usage, + error=exc, ) ) logger.warning( @@ -245,6 +254,7 @@ class RuntimeChatService: slot_timeouts: dict[str, int] | None = None, max_attempts: int | None = None, use_failure_cooldown: bool = True, + operation_context: RuntimeChatOperationContext | None = None, ) -> RuntimeToolCallResult: configs: list[dict[str, str]] = [] calls: list[RuntimeChatCallTrace] = [] @@ -291,42 +301,72 @@ class RuntimeChatService: ) continue started = monotonic() + tracker = RuntimeChatAttemptTracker( + observer=self.attempt_observer, + operation_context=operation_context, + slot=config["slot"], + provider=config["provider"], + model=config["model"], + attempt=attempt, + ) + permit = tracker.notify_permit() + if not permit.allowed: + permit_error = RuntimeError( + permit.reason or "Runtime chat attempt permit denied." + ) + calls.append( + self._complete_attempt_trace( + tracker=tracker, + config=config, + attempt=attempt, + status="blocked", + outcome="not_sent", + duration_ms=int((monotonic() - started) * 1000), + error=permit_error, + ) + ) + continue try: - tool_call = self._request_chat_tool_call( - config, - messages, - tools=tools, - tool_choice=tool_choice, - max_tokens=max_tokens, - temperature=temperature, - timeout_seconds=resolved_slot_timeouts.get( - config["slot"], - resolved_timeout_seconds, - ), + provider_response = self._coerce_provider_response( + self._request_chat_tool_call( + config, + messages, + tools=tools, + tool_choice=tool_choice, + max_tokens=max_tokens, + temperature=temperature, + timeout_seconds=resolved_slot_timeouts.get( + config["slot"], + resolved_timeout_seconds, + ), + ) ) duration_ms = int((monotonic() - started) * 1000) + tool_call = provider_response.output if tool_call is not None: _slot_failure_until.pop(cache_key, None) calls.append( - RuntimeChatCallTrace( - slot=config["slot"], - provider=config["provider"], - model=config["model"], + self._complete_attempt_trace( + tracker=tracker, + config=config, attempt=attempt, status="succeeded", + outcome="succeeded", duration_ms=duration_ms, + provider_response=provider_response, ) ) return RuntimeToolCallResult(tool_call, calls) calls.append( - RuntimeChatCallTrace( - slot=config["slot"], - provider=config["provider"], - model=config["model"], + self._complete_attempt_trace( + tracker=tracker, + config=config, attempt=attempt, status="empty", + outcome="empty", duration_ms=duration_ms, - error_message="模型未返回工具调用。", + provider_response=provider_response, + error=ValueError("模型未返回工具调用。"), ) ) except Exception as exc: @@ -335,15 +375,26 @@ class RuntimeChatService: _slot_failure_until[cache_key] = ( monotonic() + DEFAULT_RUNTIME_CHAT_FAILURE_COOLDOWN_SECONDS ) + ( + outcome, + response_id, + response_model, + provider_status_code, + usage, + ) = self._error_attempt_metadata(exc) calls.append( - RuntimeChatCallTrace( - slot=config["slot"], - provider=config["provider"], - model=config["model"], + self._complete_attempt_trace( + tracker=tracker, + config=config, attempt=attempt, status="failed", + outcome=outcome, duration_ms=duration_ms, - error_message=str(exc), + response_id=response_id, + response_model=response_model, + provider_status_code=provider_status_code, + usage=usage, + error=exc, ) ) logger.warning( @@ -359,6 +410,89 @@ class RuntimeChatService: return RuntimeToolCallResult(None, calls) + def _complete_attempt_trace( + self, + *, + tracker: RuntimeChatAttemptTracker, + config: dict[str, str], + attempt: int, + status: str, + outcome: RuntimeChatAttemptOutcome, + duration_ms: int, + provider_response: RuntimeChatProviderResponse[Any] | None = None, + response_id: str | None = None, + response_model: str | None = None, + provider_status_code: int | None = None, + usage: RuntimeChatAuthoritativeUsage | None = None, + error: Exception | None = None, + ) -> RuntimeChatCallTrace: + resolved_response_id = ( + provider_response.response_id if provider_response is not None else response_id + ) + resolved_response_model = ( + provider_response.response_model + if provider_response is not None + else response_model + ) + resolved_usage = ( + provider_response.usage + if provider_response is not None + else usage or RuntimeChatAuthoritativeUsage() + ) + notification = tracker.notify_completed( + outcome=outcome, + response_id=resolved_response_id, + response_model=resolved_response_model, + provider_status_code=provider_status_code, + usage=resolved_usage, + error=error, + ) + return RuntimeChatCallTrace( + slot=config["slot"], + provider=config["provider"], + model=config["model"], + attempt=attempt, + status=status, + duration_ms=duration_ms, + error_message=str(error) if error is not None else None, + outcome=outcome, + started_at=notification.started_at, + completed_at=notification.completed_at, + response_id=resolved_response_id, + response_model=resolved_response_model, + provider_status_code=provider_status_code, + usage=resolved_usage, + observer_status=notification.observer_status, + observer_failures=list(notification.observer_failures), + ) + + @staticmethod + def _coerce_provider_response(value: Any) -> RuntimeChatProviderResponse[Any]: + if isinstance(value, RuntimeChatProviderResponse): + return value + return RuntimeChatProviderResponse(output=value) + + @staticmethod + def _error_attempt_metadata( + exc: Exception, + ) -> tuple[ + RuntimeChatAttemptOutcome, + str | None, + str | None, + int | None, + RuntimeChatAuthoritativeUsage, + ]: + if isinstance(exc, RuntimeChatProviderAttemptError): + return ( + exc.outcome, + exc.response_id, + exc.response_model, + exc.status_code, + exc.usage, + ) + # 未经 provider adapter 分类的异常无法证明请求未发送，保守进入补偿。 + return "outcome_unknown", None, None, None, RuntimeChatAuthoritativeUsage() + @staticmethod def _build_slot_cache_key(config: dict[str, str]) -> str: return "|".join( @@ -407,14 +541,37 @@ class RuntimeChatService: max_tokens: int, temperature: float, timeout_seconds: int, - ) -> str: + ) -> RuntimeChatProviderResponse[str]: provider = config["provider"] endpoint = config["endpoint"] model = config["model"] api_key = config["apiKey"] - if provider == "Azure OpenAI": - return self._request_azure_openai( + try: + if provider == "Azure OpenAI": + return self._request_azure_openai( + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + max_tokens=max_tokens, + temperature=temperature, + timeout_seconds=timeout_seconds, + ) + + if provider == "Ollama": + return self._request_ollama( + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + max_tokens=max_tokens, + temperature=temperature, + timeout_seconds=timeout_seconds, + ) + + return self._request_openai_compatible( + provider=provider, endpoint=endpoint, model=model, api_key=api_key, @@ -423,28 +580,13 @@ class RuntimeChatService: temperature=temperature, timeout_seconds=timeout_seconds, ) - - if provider == "Ollama": - return self._request_ollama( - endpoint=endpoint, - model=model, - api_key=api_key, - messages=messages, - max_tokens=max_tokens, - temperature=temperature, - timeout_seconds=timeout_seconds, - ) - - return self._request_openai_compatible( - provider=provider, - endpoint=endpoint, - model=model, - api_key=api_key, - messages=messages, - max_tokens=max_tokens, - temperature=temperature, - timeout_seconds=timeout_seconds, - ) + except RuntimeChatProviderAttemptError: + raise + except Exception as exc: + raise RuntimeChatProviderAttemptError( + str(exc) or "模型请求发送前失败。", + outcome="not_sent", + ) from exc def _request_chat_tool_call( self, @@ -456,14 +598,34 @@ class RuntimeChatService: max_tokens: int, temperature: float, timeout_seconds: int, - ) -> RuntimeChatToolCall | None: + ) -> RuntimeChatProviderResponse[RuntimeChatToolCall | None]: provider = config["provider"] endpoint = config["endpoint"] model = config["model"] api_key = config["apiKey"] - if provider == "Azure OpenAI": - return self._request_azure_openai_tool_call( + try: + if provider == "Azure OpenAI": + return self._request_azure_openai_tool_call( + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + tools=tools, + tool_choice=tool_choice, + max_tokens=max_tokens, + temperature=temperature, + timeout_seconds=timeout_seconds, + ) + + if provider == "Ollama": + raise RuntimeChatProviderAttemptError( + "Ollama 暂不支持小财管家 function calling。", + outcome="not_sent", + ) + + return self._request_openai_compatible_tool_call( + provider=provider, endpoint=endpoint, model=model, api_key=api_key, @@ -474,22 +636,13 @@ class RuntimeChatService: temperature=temperature, timeout_seconds=timeout_seconds, ) - - if provider == "Ollama": - raise ConnectivityCheckError("Ollama 暂不支持小财管家 function calling。") - - return self._request_openai_compatible_tool_call( - provider=provider, - endpoint=endpoint, - model=model, - api_key=api_key, - messages=messages, - tools=tools, - tool_choice=tool_choice, - max_tokens=max_tokens, - temperature=temperature, - timeout_seconds=timeout_seconds, - ) + except RuntimeChatProviderAttemptError: + raise + except Exception as exc: + raise RuntimeChatProviderAttemptError( + str(exc) or "模型工具请求发送前失败。", + outcome="not_sent", + ) from exc def _request_openai_compatible( self, @@ -502,30 +655,18 @@ class RuntimeChatService: max_tokens: int, temperature: float, timeout_seconds: int, - ) -> str: - url = _ensure_path(_normalize_endpoint(endpoint), "chat/completions") - request_payload: dict[str, Any] = { - "model": model, - "messages": messages, - "max_tokens": max_tokens, - "temperature": temperature, - } - if provider == "GLM": - request_payload["thinking"] = {"type": "disabled"} - - status_code, payload = _send_json_request( - "POST", - url, - headers=_build_headers(api_key=api_key, use_bearer=True), - payload=request_payload, + ) -> RuntimeChatProviderResponse[str]: + return request_openai_compatible_completion( + send_json_request=_send_json_request, + provider=provider, + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + max_tokens=max_tokens, + temperature=temperature, timeout_seconds=timeout_seconds, ) - if status_code >= HTTPStatus.BAD_REQUEST: - raise ConnectivityCheckError( - f"模型接口返回异常状态 {status_code}。", - status_code=status_code, - ) - return self._extract_openai_text(payload) def _request_openai_compatible_tool_call( self, @@ -540,34 +681,20 @@ class RuntimeChatService: max_tokens: int, temperature: float, timeout_seconds: int, - ) -> RuntimeChatToolCall | None: - url = _ensure_path(_normalize_endpoint(endpoint), "chat/completions") - request_payload: dict[str, Any] = { - "model": model, - "messages": messages, - "tools": tools, - "tool_choice": tool_choice or "auto", - "max_tokens": max_tokens, - "temperature": temperature, - } - # function calling 需要确定性结构化输出,thinking mode 与强制 tool_choice 冲突 - # (如 Ali 通义在 thinking 模式下拒绝 tool_choice=object),这里统一禁用。 - if provider in {"GLM", "Ali"}: - request_payload["thinking"] = {"type": "disabled"} - - status_code, payload = _send_json_request( - "POST", - url, - headers=_build_headers(api_key=api_key, use_bearer=True), - payload=request_payload, + ) -> RuntimeChatProviderResponse[RuntimeChatToolCall | None]: + return request_openai_compatible_tool_call( + send_json_request=_send_json_request, + provider=provider, + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + tools=tools, + tool_choice=tool_choice, + max_tokens=max_tokens, + temperature=temperature, timeout_seconds=timeout_seconds, ) - if status_code >= HTTPStatus.BAD_REQUEST: - raise ConnectivityCheckError( - f"模型接口返回异常状态 {status_code}。", - status_code=status_code, - ) - return self._extract_openai_tool_call(payload) def _request_ollama( self, @@ -579,29 +706,17 @@ class RuntimeChatService: max_tokens: int, temperature: float, timeout_seconds: int, - ) -> str: - url = _ensure_path(_normalize_endpoint(endpoint), "api/chat") - status_code, payload = _send_json_request( - "POST", - url, - headers=_build_headers(api_key=api_key, use_bearer=False), - payload={ - "model": model, - "messages": messages, - "stream": False, - "options": { - "num_predict": max_tokens, - "temperature": temperature, - }, - }, + ) -> RuntimeChatProviderResponse[str]: + return request_ollama_completion( + send_json_request=_send_json_request, + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + max_tokens=max_tokens, + temperature=temperature, timeout_seconds=timeout_seconds, ) - if status_code >= HTTPStatus.BAD_REQUEST: - raise ConnectivityCheckError( - f"Ollama 返回异常状态 {status_code}。", - status_code=status_code, - ) - return str((payload or {}).get("message", {}).get("content", "")).strip() def _request_azure_openai( self, @@ -613,26 +728,17 @@ class RuntimeChatService: max_tokens: int, temperature: float, timeout_seconds: int, - ) -> str: - deployment_base = _build_azure_deployment_base(endpoint, model) - url = f"{deployment_base}/chat/completions?api-version={AZURE_API_VERSION}" - status_code, payload = _send_json_request( - "POST", - url, - headers=_build_headers(api_key=api_key, use_bearer=False, use_api_key=True), - payload={ - "messages": messages, - "max_tokens": max_tokens, - "temperature": temperature, - }, + ) -> RuntimeChatProviderResponse[str]: + return request_azure_openai_completion( + send_json_request=_send_json_request, + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + max_tokens=max_tokens, + temperature=temperature, timeout_seconds=timeout_seconds, ) - if status_code >= HTTPStatus.BAD_REQUEST: - raise ConnectivityCheckError( - f"Azure OpenAI 返回异常状态 {status_code}。", - status_code=status_code, - ) - return self._extract_openai_text(payload) def _request_azure_openai_tool_call( self, @@ -646,127 +752,16 @@ class RuntimeChatService: max_tokens: int, temperature: float, timeout_seconds: int, - ) -> RuntimeChatToolCall | None: - deployment_base = _build_azure_deployment_base(endpoint, model) - url = f"{deployment_base}/chat/completions?api-version={AZURE_API_VERSION}" - status_code, payload = _send_json_request( - "POST", - url, - headers=_build_headers(api_key=api_key, use_bearer=False, use_api_key=True), - payload={ - "messages": messages, - "tools": tools, - "tool_choice": tool_choice or "auto", - "max_tokens": max_tokens, - "temperature": temperature, - }, + ) -> RuntimeChatProviderResponse[RuntimeChatToolCall | None]: + return request_azure_openai_tool_call( + send_json_request=_send_json_request, + endpoint=endpoint, + model=model, + api_key=api_key, + messages=messages, + tools=tools, + tool_choice=tool_choice, + max_tokens=max_tokens, + temperature=temperature, timeout_seconds=timeout_seconds, ) - if status_code >= HTTPStatus.BAD_REQUEST: - raise ConnectivityCheckError( - f"Azure OpenAI 返回异常状态 {status_code}。", - status_code=status_code, - ) - return self._extract_openai_tool_call(payload) - - @staticmethod - def _extract_openai_text(payload: Any) -> str: - if not isinstance(payload, dict): - return "" - - choices = payload.get("choices") - if not isinstance(choices, list) or not choices: - return "" - - first_choice = choices[0] - if not isinstance(first_choice, dict): - return "" - - message = first_choice.get("message") - if isinstance(message, dict): - content = message.get("content", "") - if isinstance(content, str): - return content.strip() - if isinstance(content, list): - parts: list[str] = [] - for item in content: - if isinstance(item, dict) and item.get("type") == "text": - parts.append(str(item.get("text", ""))) - return "\n".join(part.strip() for part in parts if part.strip()).strip() - - text = first_choice.get("text") - if isinstance(text, str): - return text.strip() - - return "" - - @staticmethod - def _extract_openai_tool_call(payload: Any) -> RuntimeChatToolCall | None: - if not isinstance(payload, dict): - return None - - choices = payload.get("choices") - if not isinstance(choices, list) or not choices: - return None - - first_choice = choices[0] - if not isinstance(first_choice, dict): - return None - - message = first_choice.get("message") - if not isinstance(message, dict): - return None - - tool_calls = message.get("tool_calls") - if isinstance(tool_calls, list) and tool_calls: - first_tool = tool_calls[0] - if isinstance(first_tool, dict): - function_payload = first_tool.get("function") - if isinstance(function_payload, dict): - return RuntimeChatService._build_runtime_tool_call( - name=function_payload.get("name"), - arguments=function_payload.get("arguments"), - call_id=first_tool.get("id"), - ) - - function_call = message.get("function_call") - if isinstance(function_call, dict): - return RuntimeChatService._build_runtime_tool_call( - name=function_call.get("name"), - arguments=function_call.get("arguments"), - call_id=None, - ) - - return None - - @staticmethod - def _build_runtime_tool_call( - *, - name: Any, - arguments: Any, - call_id: Any, - ) -> RuntimeChatToolCall | None: - tool_name = str(name or "").strip() - if not tool_name: - return None - - raw_arguments = "" - if isinstance(arguments, dict): - parsed_arguments = arguments - raw_arguments = json.dumps(arguments, ensure_ascii=False) - else: - raw_arguments = str(arguments or "").strip() - if not raw_arguments: - parsed_arguments = {} - else: - parsed = json.loads(raw_arguments) - if not isinstance(parsed, dict): - raise ValueError("工具调用参数必须是 JSON object。") - parsed_arguments = parsed - - return RuntimeChatToolCall( - name=tool_name, - arguments=parsed_arguments, - call_id=str(call_id).strip() if call_id else None, - raw_arguments=raw_arguments, - ) diff --git a/server/src/app/services/runtime_chat_attempts.py b/server/src/app/services/runtime_chat_attempts.py new file mode 100644 index 0000000..ad936db --- /dev/null +++ b/server/src/app/services/runtime_chat_attempts.py @@ -0,0 +1,462 @@ +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass, field +from datetime import UTC, datetime +from typing import Any, Generic, Literal, Protocol, TypeVar, runtime_checkable + +from app.core.logging import get_logger + +logger = get_logger("app.services.runtime_chat_attempts") + +RuntimeChatAttemptOutcome = Literal[ + "succeeded", + "empty", + "provider_rejected", + "postprocess_failed", + "not_sent", + "outcome_unknown", +] +RuntimeChatObserverStatus = Literal[ + "not_applicable", + "not_configured", + "permit_notified", + "completed_notified", + "reconciliation_required", +] +RuntimeChatUsageAvailability = Literal["available", "partial", "unavailable"] +RuntimeChatObserverPhase = Literal["permit", "completion", "failure_callback"] + + +@dataclass(frozen=True, slots=True) +class RuntimeChatOperationContext: + """由可信业务入口构造；不得从 prompt 或客户端 payload 推导。""" + + tenant_id: str + operation_id: str + invocation_seq: int + attempt_scope: str + run_id: str | None = None + + def __post_init__(self) -> None: + tenant_id = str(self.tenant_id or "").strip() + operation_id = str(self.operation_id or "").strip() + attempt_scope = str(self.attempt_scope or "").strip() + run_id = str(self.run_id or "").strip() or None + if not tenant_id: + raise ValueError("RuntimeChatOperationContext.tenant_id 不能为空。") + if not operation_id: + raise ValueError("RuntimeChatOperationContext.operation_id 不能为空。") + if ( + isinstance(self.invocation_seq, bool) + or not isinstance(self.invocation_seq, int) + or self.invocation_seq < 1 + ): + raise ValueError("RuntimeChatOperationContext.invocation_seq 必须大于 0。") + if not attempt_scope: + raise ValueError("RuntimeChatOperationContext.attempt_scope 不能为空。") + object.__setattr__(self, "tenant_id", tenant_id) + object.__setattr__(self, "operation_id", operation_id) + object.__setattr__(self, "attempt_scope", attempt_scope) + object.__setattr__(self, "run_id", run_id) + + def build_attempt_identity( + self, + *, + slot: str, + provider: str, + model: str, + attempt: int, + ) -> RuntimeChatAttemptIdentity: + return RuntimeChatAttemptIdentity( + tenant_id=self.tenant_id, + operation_id=self.operation_id, + run_id=self.run_id, + invocation_seq=self.invocation_seq, + attempt_scope=self.attempt_scope, + slot=str(slot or "").strip(), + provider=str(provider or "").strip(), + model=str(model or "").strip(), + attempt=int(attempt), + ) + + +@dataclass(frozen=True, slots=True) +class RuntimeChatAttemptIdentity: + tenant_id: str + operation_id: str + run_id: str | None + invocation_seq: int + attempt_scope: str + slot: str + provider: str + model: str + attempt: int + + @property + def attempt_key(self) -> str: + canonical_identity = json.dumps( + [ + self.tenant_id, + self.operation_id, + self.run_id or "-", + self.invocation_seq, + self.attempt_scope, + self.slot, + self.provider, + self.model, + self.attempt, + ], + ensure_ascii=False, + separators=(",", ":"), + ) + digest = hashlib.sha256(canonical_identity.encode("utf-8")).hexdigest() + return f"runtime-chat-attempt:v1:{digest}" + + +@dataclass(frozen=True, slots=True) +class RuntimeChatAuthoritativeUsage: + source: str = "unavailable" + availability: RuntimeChatUsageAvailability = "unavailable" + prompt_tokens: int | None = None + completion_tokens: int | None = None + total_tokens: int | None = None + prompt_eval_count: int | None = None + eval_count: int | None = None + + def model_dump(self) -> dict[str, Any]: + return { + "source": self.source, + "availability": self.availability, + "prompt_tokens": self.prompt_tokens, + "completion_tokens": self.completion_tokens, + "total_tokens": self.total_tokens, + "prompt_eval_count": self.prompt_eval_count, + "eval_count": self.eval_count, + } + + +OutputT = TypeVar("OutputT") + + +@dataclass(slots=True) +class RuntimeChatProviderResponse(Generic[OutputT]): + output: OutputT + response_id: str | None = None + response_model: str | None = None + usage: RuntimeChatAuthoritativeUsage = field( + default_factory=RuntimeChatAuthoritativeUsage + ) + + +class RuntimeChatProviderAttemptError(RuntimeError): + def __init__( + self, + message: str, + *, + outcome: RuntimeChatAttemptOutcome, + response_id: str | None = None, + response_model: str | None = None, + usage: RuntimeChatAuthoritativeUsage | None = None, + status_code: int | None = None, + ) -> None: + super().__init__(message) + self.outcome = outcome + self.response_id = response_id + self.response_model = response_model + self.usage = usage or RuntimeChatAuthoritativeUsage() + self.status_code = status_code + + +@dataclass(frozen=True, slots=True) +class RuntimeChatAttemptPermit: + allowed: bool = True + reason: str = "" + + +@dataclass(frozen=True, slots=True) +class RuntimeChatAttemptPermitEvent: + identity: RuntimeChatAttemptIdentity + started_at: datetime + phase: Literal["permit"] = "permit" + + +@dataclass(frozen=True, slots=True) +class RuntimeChatAttemptCompletedEvent: + identity: RuntimeChatAttemptIdentity + started_at: datetime + completed_at: datetime + outcome: RuntimeChatAttemptOutcome + response_id: str | None + response_model: str | None + provider_status_code: int | None + usage: RuntimeChatAuthoritativeUsage + error_type: str | None = None + error_message: str | None = None + + @property + def request_may_have_been_sent(self) -> bool: + return self.outcome != "not_sent" + + @property + def requires_reconciliation(self) -> bool: + """发送结果未知时，后续只能补账核对，不能按零用量释放。""" + + return self.outcome == "outcome_unknown" + + +@dataclass(frozen=True, slots=True) +class RuntimeChatObserverFailure: + phase: RuntimeChatObserverPhase + identity: RuntimeChatAttemptIdentity + occurred_at: datetime + error_type: str + error_message: str + + def model_dump(self) -> dict[str, str]: + return { + "phase": self.phase, + "attempt_key": self.identity.attempt_key, + "occurred_at": self.occurred_at.isoformat(), + "error_type": self.error_type, + "error_message": self.error_message, + } + + +@runtime_checkable +class RuntimeChatAttemptObserver(Protocol): + def on_permit( + self, + event: RuntimeChatAttemptPermitEvent, + ) -> RuntimeChatAttemptPermit | None: ... + + def on_completed(self, event: RuntimeChatAttemptCompletedEvent) -> None: ... + + def on_observer_failure(self, failure: RuntimeChatObserverFailure) -> None: ... + + +@dataclass(frozen=True, slots=True) +class RuntimeChatAttemptNotification: + started_at: datetime + completed_at: datetime + observer_status: RuntimeChatObserverStatus + observer_failures: tuple[RuntimeChatObserverFailure, ...] + + +class RuntimeChatAttemptTracker: + def __init__( + self, + *, + observer: RuntimeChatAttemptObserver | None, + operation_context: RuntimeChatOperationContext | None, + slot: str, + provider: str, + model: str, + attempt: int, + ) -> None: + self.observer = observer + self.started_at = datetime.now(UTC) + self.identity = ( + operation_context.build_attempt_identity( + slot=slot, + provider=provider, + model=model, + attempt=attempt, + ) + if operation_context is not None + else None + ) + self.observer_failures: list[RuntimeChatObserverFailure] = [] + if self.identity is None: + self.observer_status: RuntimeChatObserverStatus = "not_applicable" + elif observer is None: + self.observer_status = "not_configured" + else: + self.observer_status = "permit_notified" + + def notify_permit(self) -> RuntimeChatAttemptPermit: + if self.identity is None or self.observer is None: + return RuntimeChatAttemptPermit() + try: + decision = self.observer.on_permit( + RuntimeChatAttemptPermitEvent( + identity=self.identity, + started_at=self.started_at, + ) + ) + if decision is None: + return RuntimeChatAttemptPermit() + if not isinstance(decision, RuntimeChatAttemptPermit): + raise TypeError( + "attempt observer on_permit 必须返回 " + "RuntimeChatAttemptPermit 或 None。" + ) + return decision + except Exception as exc: + self._capture_observer_failure("permit", exc) + # permit 可能承担额度预留或策略授权。状态未知时不得继续发起 + # provider 请求；completion(not_sent) 会给观察器释放或补偿机会。 + return RuntimeChatAttemptPermit( + allowed=False, + reason="Runtime chat attempt permit observer failed.", + ) + + def notify_completed( + self, + *, + outcome: RuntimeChatAttemptOutcome, + response_id: str | None, + response_model: str | None, + provider_status_code: int | None, + usage: RuntimeChatAuthoritativeUsage, + error: Exception | None = None, + ) -> RuntimeChatAttemptNotification: + completed_at = datetime.now(UTC) + if self.identity is not None and self.observer is not None: + try: + self.observer.on_completed( + RuntimeChatAttemptCompletedEvent( + identity=self.identity, + started_at=self.started_at, + completed_at=completed_at, + outcome=outcome, + response_id=response_id, + response_model=response_model, + provider_status_code=provider_status_code, + usage=usage, + error_type=type(error).__name__ if error is not None else None, + error_message=str(error) if error is not None else None, + ) + ) + if not self.observer_failures: + self.observer_status = "completed_notified" + except Exception as exc: + self._capture_observer_failure("completion", exc) + return RuntimeChatAttemptNotification( + started_at=self.started_at, + completed_at=completed_at, + observer_status=self.observer_status, + observer_failures=tuple(self.observer_failures), + ) + + def _capture_observer_failure( + self, + phase: Literal["permit", "completion"], + exc: Exception, + ) -> None: + if self.identity is None: + return + failure = RuntimeChatObserverFailure( + phase=phase, + identity=self.identity, + occurred_at=datetime.now(UTC), + error_type=type(exc).__name__, + error_message=str(exc), + ) + self.observer_failures.append(failure) + self.observer_status = "reconciliation_required" + logger.warning( + "Runtime chat attempt observer failed phase=%s attempt_key=%s error=%s", + phase, + self.identity.attempt_key, + exc, + ) + callback = getattr(self.observer, "on_observer_failure", None) + if not callable(callback): + return + try: + callback(failure) + except Exception as callback_exc: + callback_failure = RuntimeChatObserverFailure( + phase="failure_callback", + identity=self.identity, + occurred_at=datetime.now(UTC), + error_type=type(callback_exc).__name__, + error_message=str(callback_exc), + ) + self.observer_failures.append(callback_failure) + logger.warning( + "Runtime chat observer failure callback failed attempt_key=%s error=%s", + self.identity.attempt_key, + callback_exc, + ) + + +@dataclass(slots=True) +class RuntimeChatCallTrace: + slot: str + provider: str + model: str + attempt: int + status: str + duration_ms: int = 0 + error_message: str | None = None + skipped_reason: str | None = None + outcome: RuntimeChatAttemptOutcome | None = None + started_at: datetime | None = None + completed_at: datetime | None = None + response_id: str | None = None + response_model: str | None = None + provider_status_code: int | None = None + usage: RuntimeChatAuthoritativeUsage = field( + default_factory=RuntimeChatAuthoritativeUsage + ) + observer_status: RuntimeChatObserverStatus = "not_applicable" + observer_failures: list[RuntimeChatObserverFailure] = field(default_factory=list) + + @property + def requires_reconciliation(self) -> bool: + return ( + self.outcome == "outcome_unknown" + or self.observer_status + in {"not_configured", "reconciliation_required"} + ) + + def model_dump(self) -> dict[str, Any]: + return { + "slot": self.slot, + "provider": self.provider, + "model": self.model, + "attempt": self.attempt, + "status": self.status, + "duration_ms": self.duration_ms, + "error_message": self.error_message, + "skipped_reason": self.skipped_reason, + "outcome": self.outcome, + "started_at": self.started_at.isoformat() if self.started_at else None, + "completed_at": self.completed_at.isoformat() if self.completed_at else None, + "response_id": self.response_id, + "response_model": self.response_model, + "provider_status_code": self.provider_status_code, + "usage": self.usage.model_dump(), + "observer_status": self.observer_status, + "requires_reconciliation": self.requires_reconciliation, + "observer_failures": [item.model_dump() for item in self.observer_failures], + } + + +@dataclass(slots=True) +class RuntimeChatResult: + text: str | None + calls: list[RuntimeChatCallTrace] + + def calls_as_dicts(self) -> list[dict[str, Any]]: + return [item.model_dump() for item in self.calls] + + +@dataclass(slots=True) +class RuntimeChatToolCall: + name: str + arguments: dict[str, Any] + call_id: str | None = None + raw_arguments: str = "" + + +@dataclass(slots=True) +class RuntimeToolCallResult: + tool_call: RuntimeChatToolCall | None + calls: list[RuntimeChatCallTrace] + + def calls_as_dicts(self) -> list[dict[str, Any]]: + return [item.model_dump() for item in self.calls] diff --git a/server/src/app/services/runtime_chat_commercial.py b/server/src/app/services/runtime_chat_commercial.py new file mode 100644 index 0000000..54015cb --- /dev/null +++ b/server/src/app/services/runtime_chat_commercial.py @@ -0,0 +1,158 @@ +"""把 RuntimeChat provider attempt 接到独立事务商业计量。""" + +from __future__ import annotations + +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session, sessionmaker + +from app.core.logging import get_logger +from app.models.agent_run import AgentRun +from app.services.commercial_direct_operation import ( + CommercialDirectOperationBridge, + DirectOperationIdentity, +) +from app.services.commercial_runtime_policy import tenant_from_agent_route +from app.services.runtime_chat_attempts import ( + RuntimeChatAttemptCompletedEvent, + RuntimeChatAttemptObserver, + RuntimeChatAttemptPermit, + RuntimeChatAttemptPermitEvent, + RuntimeChatObserverFailure, + RuntimeChatOperationContext, +) + +logger = get_logger("app.services.runtime_chat_commercial") + + +class CommercialDirectReconciliationRequired(RuntimeError): + """补偿状态已持久化；由 RuntimeChat trace 显式标记而不吞掉回答。""" + + +class CommercialRuntimeChatAttemptObserver(RuntimeChatAttemptObserver): + def __init__(self, bridge: CommercialDirectOperationBridge) -> None: + self.bridge = bridge + + def on_permit( + self, + event: RuntimeChatAttemptPermitEvent, + ) -> RuntimeChatAttemptPermit: + permit = self.bridge.permit(_identity(event.identity, event.started_at)) + return RuntimeChatAttemptPermit( + allowed=permit.allowed, + reason="" if permit.allowed else permit.reason, + ) + + def on_completed(self, event: RuntimeChatAttemptCompletedEvent) -> None: + result = self.bridge.complete( + _identity(event.identity, event.started_at), + outcome=event.outcome, + authoritative_quantities=_quantities(event), + completed_at=event.completed_at, + usage_source=event.usage.source, + usage_availability=event.usage.availability, + ) + if result.requires_reconciliation: + raise CommercialDirectReconciliationRequired( + f"{result.reason_code}: {result.reason}" + ) + + def on_observer_failure(self, failure: RuntimeChatObserverFailure) -> None: + if failure.phase != "completion": + return + if failure.error_type == CommercialDirectReconciliationRequired.__name__: + return + result = self.bridge.complete( + _identity(failure.identity, failure.occurred_at), + outcome="outcome_unknown", + authoritative_quantities={}, + completed_at=failure.occurred_at, + usage_source="observer_failure", + usage_availability="unavailable", + ) + if result.requires_reconciliation: + logger.warning( + "RuntimeChat commercial observer persisted compensation " + "operation_call_id=%s reason_code=%s", + result.operation_call_id, + result.reason_code, + ) + + +def build_runtime_chat_commercial_observer( + db: Session, +) -> CommercialRuntimeChatAttemptObserver: + """只复用 Engine；permit/completion 从不提交传入的业务 Session。""" + + factory = sessionmaker(bind=db.get_bind(), expire_on_commit=False) + return CommercialRuntimeChatAttemptObserver( + CommercialDirectOperationBridge(factory, lookup_session=db), + ) + + +def trusted_runtime_chat_operation_context( + db: Session, + *, + run_id: str, + attempt_scope: str, + invocation_seq: int = 1, +) -> RuntimeChatOperationContext | None: + """只从 AgentRun.route_json 解析租户，不相信 prompt/context_json 身份。""" + + normalized_run_id = str(run_id or "").strip() + if not normalized_run_id: + return None + run = db.scalar(select(AgentRun).where(AgentRun.run_id == normalized_run_id)) + if run is None: + return None + tenant_id = tenant_from_agent_route(run.route_json) + if tenant_id is None: + return None + return RuntimeChatOperationContext( + tenant_id=tenant_id, + operation_id=f"agent-run:{run.run_id}", + invocation_seq=invocation_seq, + attempt_scope=str(attempt_scope or "").strip(), + run_id=run.run_id, + ) + + +def _identity(raw: Any, started_at: datetime) -> DirectOperationIdentity: + return DirectOperationIdentity( + tenant_id=raw.tenant_id, + operation_key=raw.attempt_key, + run_key=raw.run_id or raw.operation_id, + tool_type="llm", + tool_name="chat.completions", + provider=raw.provider, + model_name=raw.model, + started_at=_utc(started_at), + ) + + +def _quantities(event: RuntimeChatAttemptCompletedEvent) -> dict[str, int | None]: + usage = event.usage + input_tokens = ( + usage.prompt_tokens + if usage.prompt_tokens is not None + else usage.prompt_eval_count + ) + output_tokens = ( + usage.completion_tokens + if usage.completion_tokens is not None + else usage.eval_count + ) + total_tokens = usage.total_tokens + if total_tokens is None and input_tokens is not None and output_tokens is not None: + total_tokens = input_tokens + output_tokens + return { + "input_tokens": input_tokens, + "output_tokens": output_tokens, + "total_tokens": total_tokens, + } + + +def _utc(value: datetime) -> datetime: + return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC) diff --git a/server/src/app/services/runtime_chat_provider.py b/server/src/app/services/runtime_chat_provider.py new file mode 100644 index 0000000..7a859fd --- /dev/null +++ b/server/src/app/services/runtime_chat_provider.py @@ -0,0 +1,490 @@ +from __future__ import annotations + +import json +from collections.abc import Callable +from http import HTTPStatus +from typing import Any + +from app.services.model_connectivity import ( + AZURE_API_VERSION, + _build_azure_deployment_base, + _build_headers, + _ensure_path, + _normalize_endpoint, +) +from app.services.runtime_chat_attempts import ( + RuntimeChatAuthoritativeUsage, + RuntimeChatProviderAttemptError, + RuntimeChatProviderResponse, + RuntimeChatToolCall, +) + +SendJsonRequest = Callable[..., tuple[int, Any]] + + +def request_openai_compatible_completion( + *, + send_json_request: SendJsonRequest, + provider: str, + endpoint: str, + model: str, + api_key: str, + messages: list[dict[str, Any]], + max_tokens: int, + temperature: float, + timeout_seconds: int, +) -> RuntimeChatProviderResponse[str]: + url = _ensure_path(_normalize_endpoint(endpoint), "chat/completions") + request_payload: dict[str, Any] = { + "model": model, + "messages": messages, + "max_tokens": max_tokens, + "temperature": temperature, + } + if provider == "GLM": + request_payload["thinking"] = {"type": "disabled"} + payload = _send_and_require_success( + send_json_request=send_json_request, + provider=provider, + method="POST", + url=url, + headers=_build_headers(api_key=api_key, use_bearer=True), + request_payload=request_payload, + timeout_seconds=timeout_seconds, + error_prefix="模型接口", + ) + response_id, response_model, usage = extract_response_metadata(provider, payload) + try: + output = extract_openai_text(payload) + except Exception as exc: + raise _postprocess_error( + exc, + response_id=response_id, + response_model=response_model, + usage=usage, + ) from exc + return RuntimeChatProviderResponse( + output=output, + response_id=response_id, + response_model=response_model, + usage=usage, + ) + + +def request_openai_compatible_tool_call( + *, + send_json_request: SendJsonRequest, + provider: str, + endpoint: str, + model: str, + api_key: str, + messages: list[dict[str, Any]], + tools: list[dict[str, Any]], + tool_choice: dict[str, Any] | str | None, + max_tokens: int, + temperature: float, + timeout_seconds: int, +) -> RuntimeChatProviderResponse[RuntimeChatToolCall | None]: + url = _ensure_path(_normalize_endpoint(endpoint), "chat/completions") + request_payload: dict[str, Any] = { + "model": model, + "messages": messages, + "tools": tools, + "tool_choice": tool_choice or "auto", + "max_tokens": max_tokens, + "temperature": temperature, + } + # function calling 的强制 tool_choice 与部分供应商 thinking mode 冲突。 + if provider in {"GLM", "Ali"}: + request_payload["thinking"] = {"type": "disabled"} + payload = _send_and_require_success( + send_json_request=send_json_request, + provider=provider, + method="POST", + url=url, + headers=_build_headers(api_key=api_key, use_bearer=True), + request_payload=request_payload, + timeout_seconds=timeout_seconds, + error_prefix="模型接口", + ) + return _build_tool_response(provider, payload) + + +def request_ollama_completion( + *, + send_json_request: SendJsonRequest, + endpoint: str, + model: str, + api_key: str, + messages: list[dict[str, Any]], + max_tokens: int, + temperature: float, + timeout_seconds: int, +) -> RuntimeChatProviderResponse[str]: + provider = "Ollama" + url = _ensure_path(_normalize_endpoint(endpoint), "api/chat") + payload = _send_and_require_success( + send_json_request=send_json_request, + provider=provider, + method="POST", + url=url, + headers=_build_headers(api_key=api_key, use_bearer=False), + request_payload={ + "model": model, + "messages": messages, + "stream": False, + "options": { + "num_predict": max_tokens, + "temperature": temperature, + }, + }, + timeout_seconds=timeout_seconds, + error_prefix="Ollama", + ) + response_id, response_model, usage = extract_response_metadata(provider, payload) + try: + message = payload.get("message") if isinstance(payload, dict) else None + output = str(message.get("content") or "").strip() if isinstance(message, dict) else "" + except Exception as exc: + raise _postprocess_error( + exc, + response_id=response_id, + response_model=response_model, + usage=usage, + ) from exc + return RuntimeChatProviderResponse( + output=output, + response_id=response_id, + response_model=response_model, + usage=usage, + ) + + +def request_azure_openai_completion( + *, + send_json_request: SendJsonRequest, + endpoint: str, + model: str, + api_key: str, + messages: list[dict[str, Any]], + max_tokens: int, + temperature: float, + timeout_seconds: int, +) -> RuntimeChatProviderResponse[str]: + provider = "Azure OpenAI" + deployment_base = _build_azure_deployment_base(endpoint, model) + url = f"{deployment_base}/chat/completions?api-version={AZURE_API_VERSION}" + payload = _send_and_require_success( + send_json_request=send_json_request, + provider=provider, + method="POST", + url=url, + headers=_build_headers(api_key=api_key, use_bearer=False, use_api_key=True), + request_payload={ + "messages": messages, + "max_tokens": max_tokens, + "temperature": temperature, + }, + timeout_seconds=timeout_seconds, + error_prefix=provider, + ) + response_id, response_model, usage = extract_response_metadata(provider, payload) + try: + output = extract_openai_text(payload) + except Exception as exc: + raise _postprocess_error( + exc, + response_id=response_id, + response_model=response_model, + usage=usage, + ) from exc + return RuntimeChatProviderResponse( + output=output, + response_id=response_id, + response_model=response_model, + usage=usage, + ) + + +def request_azure_openai_tool_call( + *, + send_json_request: SendJsonRequest, + endpoint: str, + model: str, + api_key: str, + messages: list[dict[str, Any]], + tools: list[dict[str, Any]], + tool_choice: dict[str, Any] | str | None, + max_tokens: int, + temperature: float, + timeout_seconds: int, +) -> RuntimeChatProviderResponse[RuntimeChatToolCall | None]: + provider = "Azure OpenAI" + deployment_base = _build_azure_deployment_base(endpoint, model) + url = f"{deployment_base}/chat/completions?api-version={AZURE_API_VERSION}" + payload = _send_and_require_success( + send_json_request=send_json_request, + provider=provider, + method="POST", + url=url, + headers=_build_headers(api_key=api_key, use_bearer=False, use_api_key=True), + request_payload={ + "messages": messages, + "tools": tools, + "tool_choice": tool_choice or "auto", + "max_tokens": max_tokens, + "temperature": temperature, + }, + timeout_seconds=timeout_seconds, + error_prefix=provider, + ) + return _build_tool_response(provider, payload) + + +def extract_response_metadata( + provider: str, + payload: Any, +) -> tuple[str | None, str | None, RuntimeChatAuthoritativeUsage]: + if not isinstance(payload, dict): + return None, None, RuntimeChatAuthoritativeUsage() + response_id = _non_empty_string(payload.get("id")) + response_model = _non_empty_string(payload.get("model")) + return response_id, response_model, extract_authoritative_usage(provider, payload) + + +def extract_authoritative_usage( + provider: str, + payload: Any, +) -> RuntimeChatAuthoritativeUsage: + if not isinstance(payload, dict): + return RuntimeChatAuthoritativeUsage() + if provider == "Ollama": + prompt_eval_count = _authoritative_non_negative_int( + payload.get("prompt_eval_count") + ) + eval_count = _authoritative_non_negative_int(payload.get("eval_count")) + has_usage = prompt_eval_count is not None or eval_count is not None + return RuntimeChatAuthoritativeUsage( + source="ollama_counts" if has_usage else "unavailable", + availability=_availability((prompt_eval_count, eval_count)), + prompt_eval_count=prompt_eval_count, + eval_count=eval_count, + ) + + usage = payload.get("usage") + if not isinstance(usage, dict): + return RuntimeChatAuthoritativeUsage() + prompt_tokens = _authoritative_non_negative_int(usage.get("prompt_tokens")) + completion_tokens = _authoritative_non_negative_int( + usage.get("completion_tokens") + ) + total_tokens = _authoritative_non_negative_int(usage.get("total_tokens")) + values = (prompt_tokens, completion_tokens, total_tokens) + return RuntimeChatAuthoritativeUsage( + source="openai_usage" if any(value is not None for value in values) else "unavailable", + availability=_availability(values), + prompt_tokens=prompt_tokens, + completion_tokens=completion_tokens, + total_tokens=total_tokens, + ) + + +def extract_openai_text(payload: Any) -> str: + if not isinstance(payload, dict): + return "" + choices = payload.get("choices") + if not isinstance(choices, list) or not choices: + return "" + first_choice = choices[0] + if not isinstance(first_choice, dict): + return "" + message = first_choice.get("message") + if isinstance(message, dict): + content = message.get("content", "") + if isinstance(content, str): + return content.strip() + if isinstance(content, list): + parts = [ + str(item.get("text") or "").strip() + for item in content + if isinstance(item, dict) and item.get("type") == "text" + ] + return "\n".join(part for part in parts if part).strip() + text = first_choice.get("text") + return text.strip() if isinstance(text, str) else "" + + +def extract_openai_tool_call(payload: Any) -> RuntimeChatToolCall | None: + if not isinstance(payload, dict): + return None + choices = payload.get("choices") + if not isinstance(choices, list) or not choices: + return None + first_choice = choices[0] + if not isinstance(first_choice, dict): + return None + message = first_choice.get("message") + if not isinstance(message, dict): + return None + tool_calls = message.get("tool_calls") + if isinstance(tool_calls, list) and tool_calls: + first_tool = tool_calls[0] + if isinstance(first_tool, dict): + function_payload = first_tool.get("function") + if isinstance(function_payload, dict): + return build_runtime_tool_call( + name=function_payload.get("name"), + arguments=function_payload.get("arguments"), + call_id=first_tool.get("id"), + ) + function_call = message.get("function_call") + if isinstance(function_call, dict): + return build_runtime_tool_call( + name=function_call.get("name"), + arguments=function_call.get("arguments"), + call_id=None, + ) + return None + + +def build_runtime_tool_call( + *, + name: Any, + arguments: Any, + call_id: Any, +) -> RuntimeChatToolCall | None: + tool_name = str(name or "").strip() + if not tool_name: + return None + if isinstance(arguments, dict): + parsed_arguments = arguments + raw_arguments = json.dumps(arguments, ensure_ascii=False) + else: + raw_arguments = str(arguments or "").strip() + if not raw_arguments: + parsed_arguments = {} + else: + parsed = json.loads(raw_arguments) + if not isinstance(parsed, dict): + raise ValueError("工具调用参数必须是 JSON object。") + parsed_arguments = parsed + return RuntimeChatToolCall( + name=tool_name, + arguments=parsed_arguments, + call_id=str(call_id).strip() if call_id else None, + raw_arguments=raw_arguments, + ) + + +def _send_and_require_success( + *, + send_json_request: SendJsonRequest, + provider: str, + method: str, + url: str, + headers: dict[str, str], + request_payload: dict[str, Any], + timeout_seconds: int, + error_prefix: str, +) -> Any: + try: + status_code, payload = send_json_request( + method, + url, + headers=headers, + payload=request_payload, + timeout_seconds=timeout_seconds, + ) + except Exception as exc: + status_code = _provider_status_code(exc) + if status_code is not None and status_code >= HTTPStatus.BAD_REQUEST: + raise RuntimeChatProviderAttemptError( + str(exc) or f"{error_prefix}返回异常状态 {status_code}。", + outcome="provider_rejected", + status_code=status_code, + ) from exc + raise RuntimeChatProviderAttemptError( + str(exc) or f"{error_prefix}请求结果未知。", + outcome="outcome_unknown", + ) from exc + if isinstance(status_code, bool) or not isinstance(status_code, int): + raise RuntimeChatProviderAttemptError( + f"{error_prefix}返回了无效 HTTP 状态码。", + outcome="outcome_unknown", + ) + if status_code < HTTPStatus.BAD_REQUEST: + return payload + response_id, response_model, usage = extract_response_metadata(provider, payload) + raise RuntimeChatProviderAttemptError( + f"{error_prefix}返回异常状态 {status_code}。", + outcome="provider_rejected", + response_id=response_id, + response_model=response_model, + usage=usage, + status_code=status_code, + ) + + +def _build_tool_response( + provider: str, + payload: Any, +) -> RuntimeChatProviderResponse[RuntimeChatToolCall | None]: + response_id, response_model, usage = extract_response_metadata(provider, payload) + try: + output = extract_openai_tool_call(payload) + except Exception as exc: + raise _postprocess_error( + exc, + response_id=response_id, + response_model=response_model, + usage=usage, + ) from exc + return RuntimeChatProviderResponse( + output=output, + response_id=response_id, + response_model=response_model, + usage=usage, + ) + + +def _postprocess_error( + exc: Exception, + *, + response_id: str | None, + response_model: str | None, + usage: RuntimeChatAuthoritativeUsage, +) -> RuntimeChatProviderAttemptError: + return RuntimeChatProviderAttemptError( + str(exc) or "模型响应后处理失败。", + outcome="postprocess_failed", + response_id=response_id, + response_model=response_model, + usage=usage, + ) + + +def _authoritative_non_negative_int(value: Any) -> int | None: + if isinstance(value, bool) or not isinstance(value, int) or value < 0: + return None + return value + + +def _availability(values: tuple[int | None, ...]) -> str: + available_count = sum(value is not None for value in values) + if available_count == 0: + return "unavailable" + if available_count == len(values): + return "available" + return "partial" + + +def _non_empty_string(value: Any) -> str | None: + normalized = str(value or "").strip() + return normalized or None + + +def _provider_status_code(exc: Exception) -> int | None: + value = getattr(exc, "status_code", None) + if isinstance(value, bool) or not isinstance(value, int): + return None + return value diff --git a/server/src/app/services/savings_access_policy.py b/server/src/app/services/savings_access_policy.py new file mode 100644 index 0000000..9673696 --- /dev/null +++ b/server/src/app/services/savings_access_policy.py @@ -0,0 +1,233 @@ +from __future__ import annotations + +from dataclasses import dataclass + +from sqlalchemy import Select, or_ + +from app.api.deps import CurrentUserContext +from app.models.savings import SavingsOpportunity, SavingsRealization +from app.services.expense_claim_access_policy import ExpenseClaimAccessPolicy + +SAVINGS_READ_ROLE_CODES = frozenset({"finance", "executive"}) +SAVINGS_MANAGE_ROLE_CODES = frozenset({"finance", "executive"}) +SAVINGS_SCOPED_READ_ROLE_CODES = frozenset({"budget_monitor"}) + + +class SavingsPermissionError(PermissionError): + pass + + +@dataclass(frozen=True, slots=True) +class SavingsPermission: + can_read: bool + available_actions: tuple[str, ...] = () + read_only_reason: str = "" + + +class SavingsAccessPolicy: + """统一 Savings 列表、详情和动作的服务端业务权限。""" + + @staticmethod + def role_codes(current_user: CurrentUserContext) -> set[str]: + return ExpenseClaimAccessPolicy.normalize_role_codes(current_user) + + @classmethod + def can_read_tenant_value(cls, current_user: CurrentUserContext) -> bool: + return bool(current_user.is_admin or cls.role_codes(current_user) & SAVINGS_READ_ROLE_CODES) + + @classmethod + def can_read_scoped_value(cls, current_user: CurrentUserContext) -> bool: + return bool( + cls.role_codes(current_user) & SAVINGS_SCOPED_READ_ROLE_CODES + and cls._scope_values(current_user) + ) + + @classmethod + def can_manage_savings(cls, current_user: CurrentUserContext) -> bool: + # 平台管理员只有在同时持有财务业务角色时才可改变节省事实。 + return bool(cls.role_codes(current_user) & SAVINGS_MANAGE_ROLE_CODES) + + @classmethod + def require_tenant_value_read(cls, current_user: CurrentUserContext) -> None: + if not ( + cls.can_read_tenant_value(current_user) + or cls.can_read_scoped_value(current_user) + ): + raise SavingsPermissionError("当前用户无权访问企业经营价值数据。") + + @classmethod + def apply_opportunity_read_scope( + cls, + statement: Select, + current_user: CurrentUserContext, + *, + include_owner: bool, + ) -> Select: + if cls.can_read_tenant_value(current_user): + return statement + conditions = [] + if cls.can_read_scoped_value(current_user): + for key, value in cls._scope_values(current_user): + conditions.append( + SavingsOpportunity.dimension_json[key].as_string() == value + ) + if include_owner: + conditions.extend( + SavingsOpportunity.owner_id.ilike(actor_id) + for actor_id in cls._actor_ids(current_user) + ) + return statement.where(or_(*conditions)) if conditions else statement.where(False) + + @classmethod + def opportunity_permission( + cls, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + ) -> SavingsPermission: + if str(opportunity.tenant_id) != str(current_user.tenant_id): + return SavingsPermission(False, (), "该节省机会不在当前租户范围内。") + + is_owner = cls._matches_actor(opportunity.owner_id, current_user) + can_read = ( + cls.can_read_tenant_value(current_user) + or cls._matches_data_scope(opportunity, current_user) + or is_owner + ) + if not can_read: + return SavingsPermission(False, (), "当前用户不是该节省机会的负责人。") + + can_manage = cls.can_manage_savings(current_user) or is_owner + actions: list[str] = [] + status = str(opportunity.status or "").strip() + if can_manage: + if status == "identified": + actions.extend(("accept", "reject")) + elif status == "accepted": + actions.extend(("start", "reject")) + elif status == "in_progress": + actions.append("record_realization") + if cls.can_manage_savings(current_user) and status in { + "identified", + "accepted", + "in_progress", + }: + actions.append("expire") + + if actions: + return SavingsPermission(True, tuple(dict.fromkeys(actions)), "") + if current_user.is_admin and not cls.can_manage_savings(current_user): + reason = "管理员可查看经营价值，但未持有财务业务角色，不能改变节省事实。" + elif status in {"realized", "verified", "reversed", "rejected", "expired"}: + reason = "当前机会状态仅允许查看审计记录。" + else: + reason = "当前用户没有可执行的节省机会动作。" + return SavingsPermission(True, (), reason) + + @classmethod + def realization_permission( + cls, + realization: SavingsRealization, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + ) -> SavingsPermission: + opportunity_permission = cls.opportunity_permission(opportunity, current_user) + if not opportunity_permission.can_read: + return opportunity_permission + + if not cls.can_manage_savings(current_user): + return SavingsPermission( + True, + (), + "只有持有财务或管理层业务角色的人员可以确认节省结果。", + ) + + actor_ids = cls._actor_ids(current_user) + excluded_ids = { + str(opportunity.owner_id or "").strip().casefold(), + str(realization.recorded_by_id or "").strip().casefold(), + } + excluded_ids.discard("") + status = str(realization.status or "").strip() + + if status == "pending_confirmation": + if actor_ids & excluded_ids: + return SavingsPermission( + True, + (), + "机会负责人或结果填报人不能确认自己的节省结果。", + ) + return SavingsPermission(True, ("confirm", "reject"), "") + if ( + status == "finance_confirmed" + and realization.realization_type == "actual" + and realization.reversed_at is None + ): + return SavingsPermission(True, ("reverse",), "") + if realization.realization_type == "reversal": + return SavingsPermission(True, (), "冲回是只追加的财务事实，不能再次冲回。") + if realization.reversed_at is not None: + return SavingsPermission(True, (), "该确认结果已经通过负向事实完成冲回。") + return SavingsPermission(True, (), "该实际结果已经结束，当前仅可查看审计记录。") + + @classmethod + def require_opportunity_action( + cls, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + action: str, + ) -> None: + permission = cls.opportunity_permission(opportunity, current_user) + if action not in permission.available_actions: + raise SavingsPermissionError(permission.read_only_reason or "当前用户无权执行该动作。") + + @classmethod + def require_realization_action( + cls, + realization: SavingsRealization, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + action: str, + ) -> None: + permission = cls.realization_permission(realization, opportunity, current_user) + if action not in permission.available_actions: + raise SavingsPermissionError(permission.read_only_reason or "当前用户无权执行该动作。") + + @classmethod + def _matches_actor(cls, owner_id: str | None, current_user: CurrentUserContext) -> bool: + normalized_owner = str(owner_id or "").strip().casefold() + return bool(normalized_owner and normalized_owner in cls._actor_ids(current_user)) + + @classmethod + def _matches_data_scope( + cls, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + ) -> bool: + if not cls.can_read_scoped_value(current_user): + return False + dimensions = dict(opportunity.dimension_json or {}) + return any( + str(dimensions.get(key) or "").strip() == value + for key, value in cls._scope_values(current_user) + ) + + @staticmethod + def _scope_values(current_user: CurrentUserContext) -> tuple[tuple[str, str], ...]: + candidates = ( + ("department_id", str(current_user.department_id or "").strip()), + ("department_name", str(current_user.department_name or "").strip()), + ("cost_center", str(current_user.cost_center or "").strip()), + ) + return tuple((key, value) for key, value in candidates if value) + + @staticmethod + def _actor_ids(current_user: CurrentUserContext) -> set[str]: + return { + value.casefold() + for value in ( + str(current_user.employee_id or "").strip(), + str(current_user.username or "").strip(), + str(current_user.employee_no or "").strip(), + ) + if value + } diff --git a/server/src/app/services/savings_actions.py b/server/src/app/services/savings_actions.py new file mode 100644 index 0000000..1a6ba60 --- /dev/null +++ b/server/src/app/services/savings_actions.py @@ -0,0 +1,294 @@ +from __future__ import annotations + +import uuid +from dataclasses import dataclass +from datetime import UTC, datetime +from typing import Any + +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.expense_case import ExpenseCase +from app.models.savings import SavingsEvent, SavingsOpportunity +from app.schemas.savings import ( + SavingsOpportunityActionCreate, + SavingsOpportunityMutationRead, +) +from app.services.expense_cases import ExpenseCaseService +from app.services.savings_access_policy import SavingsAccessPolicy, SavingsPermissionError +from app.services.savings_protocol import ( + SavingsIdempotencyConflictError, + SavingsRequestProtocol, + SavingsVersionConflictError, + savings_actor, + savings_payload_fingerprint, +) +from app.services.savings_read_projection import SavingsReadProjection + + +class SavingsTransitionError(ValueError): + pass + + +@dataclass(slots=True) +class SavingsOpportunityMutation: + response: SavingsOpportunityMutationRead + + +class SavingsActionService: + """机会状态动作；一次调用只更新一个聚合并追加一个不可变事件。""" + + _TRANSITIONS = { + ("identified", "accept"): "accepted", + ("accepted", "start"): "in_progress", + ("identified", "reject"): "rejected", + ("accepted", "reject"): "rejected", + ("in_progress", "reject"): "rejected", + ("identified", "expire"): "expired", + ("accepted", "expire"): "expired", + ("in_progress", "expire"): "expired", + } + + def __init__(self, db: Session) -> None: + self.db = db + self.protocol = SavingsRequestProtocol(db) + self.projection = SavingsReadProjection() + self.expense_cases = ExpenseCaseService(db) + + def execute( + self, + opportunity_id: str, + payload: SavingsOpportunityActionCreate, + current_user: CurrentUserContext, + ) -> SavingsOpportunityMutation: + tenant_id = self._tenant(current_user) + actor_id, actor_name = savings_actor(current_user) + fingerprint = savings_payload_fingerprint( + action=payload.action, + aggregate_type="opportunity", + aggregate_id=opportunity_id, + actor_id=actor_id, + payload=payload.model_dump(mode="json"), + ) + try: + with self.protocol.serialize( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + ): + replay = self._find_replay( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + fingerprint=fingerprint, + ) + if replay is not None: + return replay + + opportunity = self._lock_opportunity(opportunity_id, tenant_id=tenant_id) + if opportunity is None: + raise LookupError("节省机会不存在。") + SavingsAccessPolicy.require_opportunity_action( + opportunity, + current_user, + payload.action, + ) + if opportunity.version != payload.expected_version: + raise SavingsVersionConflictError(opportunity.version) + + target = self._TRANSITIONS.get((opportunity.status, payload.action)) + if target is None: + raise SavingsTransitionError( + f"机会状态 {opportunity.status} 不允许执行 {payload.action}。" + ) + now = datetime.now(UTC) + before = self._state(opportunity) + self._apply_transition(opportunity, target=target, now=now) + opportunity.version += 1 + opportunity.updated_at = now + event = self._new_event( + opportunity=opportunity, + payload=payload, + actor_id=actor_id, + actor_name=actor_name, + fingerprint=fingerprint, + before=before, + now=now, + ) + response = SavingsOpportunityMutationRead( + opportunity=self.projection.opportunity_snapshot( + opportunity, + current_user, + ), + event=self.projection.event_read(event), + replayed=False, + ) + event.response_json = response.model_dump(mode="json") + self.db.add(event) + self._append_business_event( + opportunity, + event, + current_user=current_user, + comment=payload.comment, + ) + self.db.commit() + return SavingsOpportunityMutation(response=response) + except ( + LookupError, + SavingsPermissionError, + SavingsTransitionError, + SavingsVersionConflictError, + SavingsIdempotencyConflictError, + ): + self.db.rollback() + raise + except IntegrityError as error: + self.db.rollback() + replay = self._find_replay( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + fingerprint=fingerprint, + ) + if replay is not None: + return replay + raise SavingsTransitionError("节省机会已被并发更新，请刷新后重试。") from error + + def _lock_opportunity( + self, + opportunity_id: str, + *, + tenant_id: str, + ) -> SavingsOpportunity | None: + statement = select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == tenant_id, + SavingsOpportunity.id == opportunity_id, + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + return self.db.scalar(statement.execution_options(populate_existing=True)) + + def _find_replay( + self, + *, + tenant_id: str, + actor_id: str, + request_id: str, + fingerprint: str, + ) -> SavingsOpportunityMutation | None: + event = self.db.scalar( + select(SavingsEvent).where( + SavingsEvent.tenant_id == tenant_id, + SavingsEvent.actor_id == actor_id, + SavingsEvent.request_id == request_id, + ) + ) + if event is None: + return None + if event.payload_fingerprint != fingerprint: + raise SavingsIdempotencyConflictError("request_id 已被不同的节省机会动作使用。") + response = SavingsOpportunityMutationRead.model_validate(event.response_json) + return SavingsOpportunityMutation(response=response.model_copy(update={"replayed": True})) + + @classmethod + def _apply_transition( + cls, + opportunity: SavingsOpportunity, + *, + target: str, + now: datetime, + ) -> None: + opportunity.status = target + if target == "accepted": + opportunity.accepted_at = now + elif target == "in_progress": + opportunity.started_at = now + elif target in {"rejected", "expired"}: + opportunity.closed_at = now + + def _new_event( + self, + *, + opportunity: SavingsOpportunity, + payload: SavingsOpportunityActionCreate, + actor_id: str, + actor_name: str, + fingerprint: str, + before: dict[str, Any], + now: datetime, + ) -> SavingsEvent: + return SavingsEvent( + id=str(uuid.uuid4()), + tenant_id=opportunity.tenant_id, + aggregate_type="opportunity", + aggregate_id=opportunity.id, + opportunity_id=opportunity.id, + action=payload.action, + actor_id=actor_id, + actor_name=actor_name, + actor_type="user", + request_id=payload.request_id, + expected_version=payload.expected_version, + result_version=opportunity.version, + payload_fingerprint=fingerprint, + payload_json=payload.model_dump(mode="json"), + before_json=before, + after_json=self._state(opportunity), + response_json={}, + correlation_id=str(uuid.uuid4()), + occurred_at=now, + ) + + def _append_business_event( + self, + opportunity: SavingsOpportunity, + event: SavingsEvent, + *, + current_user: CurrentUserContext, + comment: str, + ) -> None: + expense_case = self.db.get(ExpenseCase, opportunity.expense_case_id) + if expense_case is None or expense_case.tenant_id != opportunity.tenant_id: + raise RuntimeError("节省机会关联的费用事件不存在或租户不一致。") + self.expense_cases.link_resource( + expense_case, + resource_type="savings_opportunity", + resource_id=opportunity.id, + relation_type="savings", + tenant_id=opportunity.tenant_id, + ) + self.expense_cases.record_resource_event( + expense_case, + aggregate_type="savings_opportunity", + aggregate_id=opportunity.id, + event_type=f"saving_opportunity_{event.action}", + actor_id=current_user.username, + idempotency_key=event.request_id, + tenant_id=opportunity.tenant_id, + correlation_id=event.correlation_id, + payload={ + "status": opportunity.status, + "version": opportunity.version, + "comment": comment, + }, + ) + + @staticmethod + def _state(opportunity: SavingsOpportunity) -> dict[str, Any]: + return { + "id": opportunity.id, + "status": opportunity.status, + "version": opportunity.version, + "accepted_at": opportunity.accepted_at.isoformat() if opportunity.accepted_at else None, + "started_at": opportunity.started_at.isoformat() if opportunity.started_at else None, + "realized_at": opportunity.realized_at.isoformat() if opportunity.realized_at else None, + "verified_at": opportunity.verified_at.isoformat() if opportunity.verified_at else None, + "closed_at": opportunity.closed_at.isoformat() if opportunity.closed_at else None, + } + + @staticmethod + def _tenant(current_user: CurrentUserContext) -> str: + return str(current_user.tenant_id or "default").strip() or "default" diff --git a/server/src/app/services/savings_baseline_generation.py b/server/src/app/services/savings_baseline_generation.py new file mode 100644 index 0000000..085d722 --- /dev/null +++ b/server/src/app/services/savings_baseline_generation.py @@ -0,0 +1,675 @@ +from __future__ import annotations + +import uuid +from collections import defaultdict +from datetime import UTC, datetime +from decimal import Decimal +from statistics import median + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsEvidenceLink, +) +from app.schemas.savings import ProfileBaselineSnapshotRead +from app.schemas.savings_insights import ( + SavingsAnalysisQualityIssue, + SavingsBaselineGenerateRequest, + SavingsBaselineGenerationRead, +) +from app.services.savings_fact_scope import ( + ExpenseSavingsFact, + ExpenseWorkflowCycleFact, + SavingsFactScopeReader, + stable_digest, +) +from app.services.savings_protocol import ( + SavingsRequestProtocol, + savings_actor, + savings_payload_fingerprint, +) + + +class SavingsBaselineGenerationService: + """从租户隔离的已归档费用事实冻结可重放历史基线。""" + + ALGORITHM_VERSION = "archived-expense-median-v1" + WORKFLOW_ALGORITHM_VERSION = "completed-workflow-elapsed-median-v1" + + def __init__(self, db: Session) -> None: + self.db = db + self.protocol = SavingsRequestProtocol(db) + self.fact_reader = SavingsFactScopeReader(db) + + def generate( + self, + payload: SavingsBaselineGenerateRequest, + current_user: CurrentUserContext, + ) -> SavingsBaselineGenerationRead: + tenant_id = str(current_user.tenant_id or "default").strip() or "default" + actor_id, actor_name = savings_actor(current_user) + request_fingerprint = self._request_fingerprint(payload, actor_id) + now = datetime.now(UTC) + try: + with self.protocol.serialize( + tenant_id=tenant_id, + actor_id="baseline-generator", + request_id=request_fingerprint, + ): + fact_set = self.fact_reader.load_archived_facts( + window_start=payload.window_start, + window_end=payload.window_end, + as_of=payload.as_of, + current_user=current_user, + include_workflow_cycles="workflow" in payload.dimensions, + ) + snapshots, issues, existing_count = self._freeze_dimensions( + payload=payload, + tenant_id=tenant_id, + actor_id=actor_id, + actor_name=actor_name, + request_fingerprint=request_fingerprint, + facts=fact_set.facts, + workflow_facts=fact_set.workflow_cycle_facts, + data_scope=fact_set.data_scope, + now=now, + ) + self.db.commit() + except Exception: + self.db.rollback() + raise + + return SavingsBaselineGenerationRead( + request_id=payload.request_id, + request_fingerprint=request_fingerprint, + tenant_id=tenant_id, + data_scope=fact_set.data_scope, + snapshots=[ProfileBaselineSnapshotRead.model_validate(row) for row in snapshots], + quality_issues=[*fact_set.quality_issues, *issues], + source_claim_count=fact_set.claim_count, + source_item_count=fact_set.item_count, + source_workflow_cycle_count=len(fact_set.workflow_cycle_facts), + replayed=bool(snapshots) and existing_count == len(snapshots), + generated_at=now, + ) + + def _freeze_dimensions( + self, + *, + payload: SavingsBaselineGenerateRequest, + tenant_id: str, + actor_id: str, + actor_name: str, + request_fingerprint: str, + facts: list[ExpenseSavingsFact], + workflow_facts: list[ExpenseWorkflowCycleFact], + data_scope: str, + now: datetime, + ) -> tuple[ + list[ProfileBaselineSnapshot], + list[SavingsAnalysisQualityIssue], + int, + ]: + snapshots: list[ProfileBaselineSnapshot] = [] + issues: list[SavingsAnalysisQualityIssue] = [] + existing_count = 0 + # PostgreSQL advisory transaction lock 持续到 commit，统一顺序避免反序死锁。 + for dimension_type in sorted(payload.dimensions): + if dimension_type == "supplier": + issues.append(self._supplier_coverage_issue()) + continue + if dimension_type == "workflow": + workflow_snapshots, workflow_existing = self._freeze_workflow_dimension( + payload=payload, + tenant_id=tenant_id, + actor_id=actor_id, + actor_name=actor_name, + request_fingerprint=request_fingerprint, + facts=workflow_facts, + data_scope=data_scope, + now=now, + ) + snapshots.extend(workflow_snapshots) + existing_count += workflow_existing + continue + grouped: dict[tuple[str, str, str], list[ExpenseSavingsFact]] = defaultdict(list) + missing_count = 0 + for fact in facts: + dimension = fact.dimension(dimension_type) + if dimension is None: + missing_count += 1 + continue + dimension_id, dimension_label = dimension + grouped[(dimension_id, dimension_label, fact.currency)].append(fact) + if missing_count: + issues.append( + SavingsAnalysisQualityIssue( + code="baseline_dimension_value_missing", + message=f"部分费用事实缺少 {dimension_type} 维度，未纳入该维度基线。", + dimension_type=dimension_type, + metadata={"fact_count": missing_count}, + ) + ) + for (dimension_id, dimension_label, currency), group in sorted(grouped.items()): + baseline, existed = self._freeze_group( + payload=payload, + tenant_id=tenant_id, + actor_id=actor_id, + actor_name=actor_name, + request_fingerprint=request_fingerprint, + dimension_type=dimension_type, + dimension_id=dimension_id, + dimension_label=dimension_label, + currency=currency, + facts=group, + data_scope=data_scope, + now=now, + ) + snapshots.append(baseline) + existing_count += int(existed) + snapshots.sort( + key=lambda row: (row.dimension_type, row.dimension_id, row.original_currency or "") + ) + return snapshots, issues, existing_count + + def _freeze_workflow_dimension( + self, + *, + payload: SavingsBaselineGenerateRequest, + tenant_id: str, + actor_id: str, + actor_name: str, + request_fingerprint: str, + facts: list[ExpenseWorkflowCycleFact], + data_scope: str, + now: datetime, + ) -> tuple[list[ProfileBaselineSnapshot], int]: + grouped: dict[tuple[str, str], list[ExpenseWorkflowCycleFact]] = defaultdict(list) + for fact in facts: + grouped[(fact.workflow_key, fact.workflow_label)].append(fact) + + snapshots: list[ProfileBaselineSnapshot] = [] + existing_count = 0 + for (workflow_key, workflow_label), group in sorted(grouped.items()): + baseline, existed = self._freeze_workflow_group( + payload=payload, + tenant_id=tenant_id, + actor_id=actor_id, + actor_name=actor_name, + request_fingerprint=request_fingerprint, + workflow_key=workflow_key, + workflow_label=workflow_label, + facts=group, + data_scope=data_scope, + now=now, + ) + snapshots.append(baseline) + existing_count += int(existed) + return snapshots, existing_count + + def _freeze_workflow_group( + self, + *, + payload: SavingsBaselineGenerateRequest, + tenant_id: str, + actor_id: str, + actor_name: str, + request_fingerprint: str, + workflow_key: str, + workflow_label: str, + facts: list[ExpenseWorkflowCycleFact], + data_scope: str, + now: datetime, + ) -> tuple[ProfileBaselineSnapshot, bool]: + query_fingerprint = stable_digest( + { + "algorithm_version": self.WORKFLOW_ALGORITHM_VERSION, + "tenant_id": tenant_id, + "data_scope": data_scope, + "dimension_type": "workflow", + "dimension_id": workflow_key, + "metric_key": "median_submission_to_payment_elapsed_minutes", + "window_start": payload.window_start, + "window_end": payload.window_end, + "as_of": payload.as_of, + "minimum_complete_samples": payload.minimum_complete_samples, + "source_hashes": sorted(fact.content_hash for fact in facts), + } + ) + baseline_key = f"historical:workflow:{query_fingerprint}" + with self.protocol.serialize( + tenant_id=tenant_id, + actor_id="baseline-workflow-group", + request_id=baseline_key, + ): + existing = self.db.scalar( + select(ProfileBaselineSnapshot).where( + ProfileBaselineSnapshot.tenant_id == tenant_id, + ProfileBaselineSnapshot.baseline_key == baseline_key, + ) + ) + if existing is not None: + return existing, True + + quality_status, quality_score, quality_issues = self._workflow_quality( + facts, + payload.minimum_complete_samples, + ) + baseline = ProfileBaselineSnapshot( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + baseline_key=baseline_key, + baseline_type="historical_cohort", + dimension_type="workflow", + dimension_id=workflow_key, + metric_key="median_submission_to_payment_elapsed_minutes", + unit="minutes", + original_currency=None, + baseline_value=Decimal( + median([fact.elapsed_minutes for fact in facts]) + ).quantize(Decimal("0.0001")), + window_start=payload.window_start, + window_end=payload.window_end, + sample_count=len(facts), + method=f"median_completed_workflow_elapsed_{data_scope}_scope", + query_fingerprint=query_fingerprint, + data_quality_status=quality_status, + data_quality_score=quality_score, + quality_issues_json=[ + issue.model_dump(mode="json") for issue in quality_issues + ], + algorithm_version=self.WORKFLOW_ALGORITHM_VERSION, + frozen_at=now, + frozen_by=actor_id, + version=1, + created_at=now, + ) + self.db.add(baseline) + self.db.flush() + self._record_workflow_evidence( + baseline, + facts=facts, + workflow_label=workflow_label, + now=now, + ) + self._record_event( + baseline, + payload=payload, + actor_id=actor_id, + actor_name=actor_name, + request_fingerprint=request_fingerprint, + now=now, + ) + return baseline, False + + def _freeze_group( + self, + *, + payload: SavingsBaselineGenerateRequest, + tenant_id: str, + actor_id: str, + actor_name: str, + request_fingerprint: str, + dimension_type: str, + dimension_id: str, + dimension_label: str, + currency: str, + facts: list[ExpenseSavingsFact], + data_scope: str, + now: datetime, + ) -> tuple[ProfileBaselineSnapshot, bool]: + query_fingerprint = stable_digest( + { + "algorithm_version": self.ALGORITHM_VERSION, + "tenant_id": tenant_id, + "data_scope": data_scope, + "dimension_type": dimension_type, + "dimension_id": dimension_id, + "currency": currency, + "window_start": payload.window_start, + "window_end": payload.window_end, + "as_of": payload.as_of, + "minimum_complete_samples": payload.minimum_complete_samples, + "source_hashes": sorted(fact.content_hash for fact in facts), + } + ) + baseline_key = f"historical:{dimension_type}:{query_fingerprint}" + with self.protocol.serialize( + tenant_id=tenant_id, + actor_id="baseline-group", + request_id=baseline_key, + ): + existing = self.db.scalar( + select(ProfileBaselineSnapshot).where( + ProfileBaselineSnapshot.tenant_id == tenant_id, + ProfileBaselineSnapshot.baseline_key == baseline_key, + ) + ) + if existing is not None: + return existing, True + + sample_count = len(facts) + quality_status, quality_score, quality_issues = self._quality( + facts, + payload.minimum_complete_samples, + ) + baseline = ProfileBaselineSnapshot( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + baseline_key=baseline_key, + baseline_type="historical_cohort", + dimension_type=dimension_type, + dimension_id=dimension_id, + metric_key="median_expense_fact_amount", + unit="currency", + original_currency=currency, + baseline_value=Decimal(median([fact.amount for fact in facts])).quantize( + Decimal("0.0001") + ), + window_start=payload.window_start, + window_end=payload.window_end, + sample_count=sample_count, + method=f"median_archived_expense_facts_{data_scope}_scope", + query_fingerprint=query_fingerprint, + data_quality_status=quality_status, + data_quality_score=quality_score, + quality_issues_json=[ + issue.model_dump(mode="json") for issue in quality_issues + ], + algorithm_version=self.ALGORITHM_VERSION, + frozen_at=now, + frozen_by=actor_id, + version=1, + created_at=now, + ) + self.db.add(baseline) + self.db.flush() + self._record_evidence( + baseline, + facts=facts, + dimension_label=dimension_label, + now=now, + ) + self._record_event( + baseline, + payload=payload, + actor_id=actor_id, + actor_name=actor_name, + request_fingerprint=request_fingerprint, + now=now, + ) + return baseline, False + + def _record_evidence( + self, + baseline: ProfileBaselineSnapshot, + *, + facts: list[ExpenseSavingsFact], + dimension_label: str, + now: datetime, + ) -> None: + source_ids = sorted({fact.source_id for fact in facts}) + self.db.add( + SavingsEvidenceLink( + id=str(uuid.uuid4()), + tenant_id=baseline.tenant_id, + evidence_key=f"baseline-cohort:{baseline.query_fingerprint}", + entity_type="baseline", + entity_id=baseline.id, + baseline_snapshot_id=baseline.id, + evidence_role="archived_expense_cohort", + resource_type="expense_claim_fact_set", + resource_id=baseline.query_fingerprint, + source_system="x-financial", + external_event_id=None, + content_hash=baseline.query_fingerprint, + occurred_at=baseline.window_end or now, + collected_at=now, + verification_status="verified", + verified_by="savings-baseline-generator", + verified_at=now, + metadata_json={ + "dimension_label": dimension_label, + "source_count": len(source_ids), + "source_ids": source_ids[:100], + "source_ids_truncated": len(source_ids) > 100, + }, + created_at=now, + ) + ) + + def _record_workflow_evidence( + self, + baseline: ProfileBaselineSnapshot, + *, + facts: list[ExpenseWorkflowCycleFact], + workflow_label: str, + now: datetime, + ) -> None: + source_ids = sorted({fact.source_id for fact in facts}) + self.db.add( + SavingsEvidenceLink( + id=str(uuid.uuid4()), + tenant_id=baseline.tenant_id, + evidence_key=f"baseline-workflow:{baseline.query_fingerprint}", + entity_type="baseline", + entity_id=baseline.id, + baseline_snapshot_id=baseline.id, + evidence_role="completed_workflow_cycle_cohort", + resource_type="business_event_cycle_fact_set", + resource_id=baseline.query_fingerprint, + source_system="x-financial", + external_event_id=None, + content_hash=baseline.query_fingerprint, + occurred_at=baseline.window_end or now, + collected_at=now, + verification_status="verified", + verified_by="savings-baseline-generator", + verified_at=now, + metadata_json={ + "workflow_label": workflow_label, + "metric_semantics": "elapsed_cycle_not_active_labor", + "active_labor_available": False, + "source_count": len(source_ids), + "source_ids": source_ids[:100], + "source_ids_truncated": len(source_ids) > 100, + }, + created_at=now, + ) + ) + + def _record_event( + self, + baseline: ProfileBaselineSnapshot, + *, + payload: SavingsBaselineGenerateRequest, + actor_id: str, + actor_name: str, + request_fingerprint: str, + now: datetime, + ) -> None: + event_request_id = f"{payload.request_id[:96]}:{baseline.query_fingerprint[:16]}" + state = { + "id": baseline.id, + "baseline_key": baseline.baseline_key, + "dimension_type": baseline.dimension_type, + "dimension_id": baseline.dimension_id, + "baseline_value": str(baseline.baseline_value), + "currency": baseline.original_currency, + "sample_count": baseline.sample_count, + "data_quality_status": baseline.data_quality_status, + "query_fingerprint": baseline.query_fingerprint, + } + self.db.add( + SavingsEvent( + id=str(uuid.uuid4()), + tenant_id=baseline.tenant_id, + aggregate_type="baseline", + aggregate_id=baseline.id, + baseline_snapshot_id=baseline.id, + action="historical_baseline_frozen", + actor_id=actor_id, + actor_name=actor_name, + actor_type="user", + request_id=event_request_id, + expected_version=0, + result_version=1, + payload_fingerprint=savings_payload_fingerprint( + action="historical_baseline_frozen", + aggregate_type="baseline", + aggregate_id=baseline.id, + actor_id=actor_id, + payload={ + "request_fingerprint": request_fingerprint, + "query_fingerprint": baseline.query_fingerprint, + }, + ), + payload_json={ + "request_fingerprint": request_fingerprint, + "window_start": payload.window_start.isoformat(), + "window_end": payload.window_end.isoformat(), + "as_of": payload.as_of.isoformat(), + }, + before_json={}, + after_json=state, + response_json=state, + correlation_id=request_fingerprint, + occurred_at=now, + ) + ) + + @staticmethod + def _quality( + facts: list[ExpenseSavingsFact], + minimum_complete_samples: int, + ) -> tuple[str, Decimal, list[SavingsAnalysisQualityIssue]]: + sample_count = len(facts) + distinct_claim_count = len({fact.claim_id for fact in facts}) + issues: list[SavingsAnalysisQualityIssue] = [] + if ( + sample_count >= minimum_complete_samples + and distinct_claim_count >= minimum_complete_samples + ): + status = "complete" + score = Decimal("1.0000") + elif sample_count >= 2 and distinct_claim_count >= 2: + status = "partial" + score = Decimal("0.7000") + issues.append( + SavingsAnalysisQualityIssue( + code="baseline_sample_partial", + message="样本量低于完整基线阈值，只能用于提示性分析。", + metadata={ + "sample_count": sample_count, + "distinct_claim_count": distinct_claim_count, + "minimum_complete_samples": minimum_complete_samples, + }, + ) + ) + else: + status = "insufficient" + score = Decimal("0.3000") + issues.append( + SavingsAnalysisQualityIssue( + code="baseline_sample_insufficient", + message="基线缺少至少两个独立单据样本，禁止据此计算货币化节省。", + severity="error", + metadata={ + "sample_count": sample_count, + "distinct_claim_count": distinct_claim_count, + }, + ) + ) + fallback_count = sum(fact.source_level == "claim_fallback" for fact in facts) + if fallback_count: + status = "partial" if status == "complete" else status + score = min(score, Decimal("0.8000")) + issues.append( + SavingsAnalysisQualityIssue( + code="baseline_contains_claim_fallback", + message="基线包含缺少明细的单据总额，不能与标准费用明细直接比较。", + metadata={"fallback_count": fallback_count}, + ) + ) + return status, score, issues + + @staticmethod + def _workflow_quality( + facts: list[ExpenseWorkflowCycleFact], + minimum_complete_samples: int, + ) -> tuple[str, Decimal, list[SavingsAnalysisQualityIssue]]: + sample_count = len(facts) + distinct_claim_count = len({fact.claim_id for fact in facts}) + issues: list[SavingsAnalysisQualityIssue] = [ + SavingsAnalysisQualityIssue( + code="workflow_elapsed_not_active_labor", + message="该基线只表示端到端经过时间，不能换算人工活跃分钟或工时价值。", + severity="info", + dimension_type="workflow", + ) + ] + if ( + sample_count >= minimum_complete_samples + and distinct_claim_count >= minimum_complete_samples + ): + return "complete", Decimal("1.0000"), issues + if sample_count >= 2 and distinct_claim_count >= 2: + issues.append( + SavingsAnalysisQualityIssue( + code="workflow_baseline_sample_partial", + message="完成流程样本低于完整基线阈值，只能用于周期提示。", + dimension_type="workflow", + metadata={ + "sample_count": sample_count, + "distinct_claim_count": distinct_claim_count, + "minimum_complete_samples": minimum_complete_samples, + }, + ) + ) + return "partial", Decimal("0.7000"), issues + issues.append( + SavingsAnalysisQualityIssue( + code="workflow_baseline_sample_insufficient", + message="流程周期缺少至少两个独立完成单据，不用于经营判断。", + severity="error", + dimension_type="workflow", + metadata={ + "sample_count": sample_count, + "distinct_claim_count": distinct_claim_count, + }, + ) + ) + return "insufficient", Decimal("0.3000"), issues + + @staticmethod + def _supplier_coverage_issue() -> SavingsAnalysisQualityIssue: + return SavingsAnalysisQualityIssue( + code="supplier_dimension_unavailable", + message="当前费用明细没有经过核验的供应商主数据，未生成供应商基线或价格漂移金额。", + severity="error", + dimension_type="supplier", + metadata={ + "required_sources": [ + "verified_supplier_id", + "invoice_line_quantity", + "invoice_line_unit_price", + ] + }, + ) + + @staticmethod + def _request_fingerprint( + payload: SavingsBaselineGenerateRequest, + actor_id: str, + ) -> str: + return savings_payload_fingerprint( + action="generate_historical_baselines", + aggregate_type="baseline_batch", + aggregate_id=payload.request_id, + actor_id=actor_id, + payload=payload.model_dump(mode="json", exclude={"request_id"}), + ) diff --git a/server/src/app/services/savings_discovery.py b/server/src/app/services/savings_discovery.py new file mode 100644 index 0000000..e5f8808 --- /dev/null +++ b/server/src/app/services/savings_discovery.py @@ -0,0 +1,508 @@ +from __future__ import annotations + +import hashlib +import json +import uuid +from datetime import UTC, datetime, timedelta +from decimal import Decimal, InvalidOperation +from typing import Any + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.expense_case import ExpenseCase +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, +) +from app.services.expense_cases import ExpenseCaseService +from app.services.savings_protocol import savings_payload_fingerprint + + +class SavingsDiscoveryService: + """只从服务端已核验业务事实发现机会；不负责提交事务。""" + + ALGORITHM_VERSION = "standard-adjustment-v1" + + def __init__(self, db: Session) -> None: + self.db = db + self.expense_cases = ExpenseCaseService(db) + + def discover_standard_adjustments( + self, + *, + claim: ExpenseClaim, + items_by_id: dict[str, ExpenseClaimItem], + adjustment_flags: list[dict[str, Any]], + current_user: CurrentUserContext, + request_id: str | None, + ) -> list[SavingsOpportunity]: + tenant_id = self._tenant(current_user) + expense_case = self.expense_cases.ensure_case_for_claim( + claim, + tenant_id=tenant_id, + relation_type="reimbursement", + ) + if expense_case.tenant_id != tenant_id: + raise PermissionError("报销单费用事件与当前租户不一致。") + + discovered: list[SavingsOpportunity] = [] + now = datetime.now(UTC) + for flag in adjustment_flags: + item_id = str(flag.get("item_id") or "").strip() + item = items_by_id.get(item_id) + if item is None: + raise ValueError("节省机会对应的费用明细已不存在。") + original = self._money(flag.get("original_amount")) + target = self._money(flag.get("reimbursable_amount")) + saving = (original - target).quantize(Decimal("0.0001")) + if saving <= Decimal("0"): + continue + currency = str(claim.currency or "CNY").strip().upper() + if len(currency) != 3: + continue + + calculation_fingerprint = str(flag.get("calculation_fingerprint") or "").strip() + policy_version = str(flag.get("policy_rule_version") or "").strip() + if not calculation_fingerprint or not policy_version: + continue + stable_material = { + "tenant_id": tenant_id, + "claim_id": claim.id, + "item_id": item.id, + "policy_version": policy_version, + "calculation_fingerprint": calculation_fingerprint, + } + stable_digest = self._digest(stable_material) + baseline_key = f"standard-adjustment:{stable_digest}" + opportunity_key = f"standard-adjustment:{stable_digest}" + existing = self.db.scalar( + select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == tenant_id, + SavingsOpportunity.opportunity_key == opportunity_key, + ) + ) + if existing is not None: + discovered.append(existing) + continue + + discovery_event = self._record_adjustment_event( + expense_case, + claim=claim, + item=item, + flag=flag, + current_user=current_user, + stable_digest=stable_digest, + request_id=request_id, + ) + baseline = self._ensure_baseline( + tenant_id=tenant_id, + baseline_key=baseline_key, + claim=claim, + item=item, + flag=flag, + original=original, + policy_version=policy_version, + calculation_fingerprint=calculation_fingerprint, + actor_id=self._actor_id(current_user), + now=now, + ) + baseline_snapshot = self._baseline_snapshot(baseline) + confidence = ( + Decimal("1.0000") + if baseline.data_quality_status == "complete" + else Decimal("0.9500") + ) + opportunity = SavingsOpportunity( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + opportunity_key=opportunity_key, + benefit_key=f"claim-item:{claim.id}:{item.id}:policy-difference", + expense_case_id=expense_case.id, + claim_id=claim.id, + claim_no_snapshot=str(claim.claim_no or claim.id), + claim_item_id=item.id, + discovery_business_event_id=discovery_event.id, + source_type="standard_adjustment", + source_id=item.id, + category="policy_compliance", + value_kind="cash", + title="住宿标准重算节省机会", + description=str(flag.get("message") or "按职级住宿标准降低实际报销金额。"), + exposure_amount=original, + baseline_snapshot_id=baseline.id, + baseline_amount=original, + target_amount=target, + estimated_gross=saving, + estimated_cost=Decimal("0.0000"), + estimated_net=saving, + estimated_low=saving, + estimated_high=saving, + confidence=confidence, + currency=currency, + reporting_currency=currency, + attribution_method="server_policy_counterfactual", + suggested_action="按锁定职级住宿标准完成审批和付款，并由独立财务复核。", + owner_id="finance", + owner_name=str(claim.finance_owner_name or "财务运营"), + owner_role="finance", + due_at=now + timedelta(days=30), + status="in_progress", + version=1, + dimension_json=self._dimensions(claim, item, flag), + baseline_snapshot_json=baseline_snapshot, + evidence_json=[ + { + "role": "server_policy_calculation", + "resource_type": "expense_claim_item", + "resource_id": item.id, + "content_hash": calculation_fingerprint, + "business_event_id": discovery_event.id, + } + ], + accepted_at=now, + started_at=now, + created_at=now, + updated_at=now, + ) + self.db.add(opportunity) + self.db.flush() + self._add_evidence_links( + baseline, + opportunity, + item=item, + discovery_event_id=discovery_event.id, + content_hash=calculation_fingerprint, + occurred_at=self._item_occurred_at(item, now), + now=now, + ) + self._add_opportunity_event( + opportunity, + current_user=current_user, + stable_digest=stable_digest, + causation_id=discovery_event.id, + now=now, + ) + self.expense_cases.link_resource( + expense_case, + resource_type="savings_opportunity", + resource_id=opportunity.id, + relation_type="savings", + tenant_id=tenant_id, + ) + self.expense_cases.record_resource_event( + expense_case, + aggregate_type="savings_opportunity", + aggregate_id=opportunity.id, + event_type="saving_opportunity_created", + actor_id=current_user.username, + idempotency_key=opportunity_key[:120], + tenant_id=tenant_id, + correlation_id=discovery_event.correlation_id, + causation_id=discovery_event.id, + payload={ + "claim_id": claim.id, + "claim_item_id": item.id, + "estimated_net": str(saving), + "currency": currency, + "status": opportunity.status, + }, + ) + discovered.append(opportunity) + self.db.flush() + return discovered + + def _ensure_baseline( + self, + *, + tenant_id: str, + baseline_key: str, + claim: ExpenseClaim, + item: ExpenseClaimItem, + flag: dict[str, Any], + original: Decimal, + policy_version: str, + calculation_fingerprint: str, + actor_id: str, + now: datetime, + ) -> ProfileBaselineSnapshot: + baseline = self.db.scalar( + select(ProfileBaselineSnapshot).where( + ProfileBaselineSnapshot.tenant_id == tenant_id, + ProfileBaselineSnapshot.baseline_key == baseline_key, + ) + ) + if baseline is not None: + return baseline + version_source = str(flag.get("policy_rule_version_source") or "").strip() + has_published_version = version_source == "published" + quality_issues = [] if has_published_version else [ + { + "code": ( + "policy_version_uses_content_fingerprint" + if version_source + else "policy_version_source_missing" + ), + "message": ( + "政策版本来自内容指纹，具备可重放性，但不是正式发布版本。" + if version_source + else "政策版本来源未记录，基线按部分可信处理。" + ), + "source": version_source or "unknown", + } + ] + baseline = ProfileBaselineSnapshot( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + baseline_key=baseline_key, + baseline_type="policy_counterfactual", + dimension_type="expense_claim_item", + dimension_id=item.id, + metric_key="pre_adjustment_reimbursable_amount", + unit="currency", + original_currency=str(claim.currency or "CNY").strip().upper(), + baseline_value=original, + sample_count=1, + method="locked_claim_item_before_policy_acceptance", + query_fingerprint=calculation_fingerprint, + data_quality_status="complete" if has_published_version else "partial", + data_quality_score=( + Decimal("1.0000") if has_published_version else Decimal("0.9500") + ), + quality_issues_json=quality_issues, + algorithm_version=self.ALGORITHM_VERSION, + policy_version=policy_version, + policy_effective_from=item.item_date, + target_resource_type="expense_claim_item", + target_resource_id=item.id, + frozen_at=now, + frozen_by=actor_id, + version=1, + created_at=now, + ) + self.db.add(baseline) + self.db.flush() + event = SavingsEvent( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + aggregate_type="baseline", + aggregate_id=baseline.id, + baseline_snapshot_id=baseline.id, + action="baseline_frozen", + actor_id="savings-ledger", + actor_name="Savings Ledger", + actor_type="service", + request_id=f"baseline:{baseline_key}"[:120], + expected_version=0, + result_version=1, + payload_fingerprint=savings_payload_fingerprint( + action="baseline_frozen", + aggregate_type="baseline", + aggregate_id=baseline.id, + actor_id="savings-ledger", + payload={"query_fingerprint": calculation_fingerprint}, + ), + payload_json={"policy_version": policy_version}, + before_json={}, + after_json=self._baseline_snapshot(baseline), + response_json=self._baseline_snapshot(baseline), + correlation_id=str(uuid.uuid4()), + occurred_at=now, + ) + self.db.add(event) + return baseline + + def _record_adjustment_event( + self, + expense_case: ExpenseCase, + *, + claim: ExpenseClaim, + item: ExpenseClaimItem, + flag: dict[str, Any], + current_user: CurrentUserContext, + stable_digest: str, + request_id: str | None, + ): + return self.expense_cases.record_resource_event( + expense_case, + aggregate_type="expense_claim", + aggregate_id=claim.id, + event_type="standard_adjustment_accepted", + actor_id=current_user.username, + idempotency_key=f"standard-adjustment:{stable_digest}"[:120], + tenant_id=expense_case.tenant_id, + correlation_id=request_id or stable_digest, + payload={ + "claim_item_id": item.id, + "original_amount": str(flag.get("original_amount") or ""), + "reimbursable_amount": str(flag.get("reimbursable_amount") or ""), + "employee_absorbed_amount": str(flag.get("employee_absorbed_amount") or ""), + "policy_rule_version": str(flag.get("policy_rule_version") or ""), + "calculation_fingerprint": str(flag.get("calculation_fingerprint") or ""), + }, + ) + + def _add_evidence_links( + self, + baseline: ProfileBaselineSnapshot, + opportunity: SavingsOpportunity, + *, + item: ExpenseClaimItem, + discovery_event_id: str, + content_hash: str, + occurred_at: datetime, + now: datetime, + ) -> None: + common = { + "tenant_id": opportunity.tenant_id, + "evidence_role": "server_policy_calculation", + "resource_type": "expense_claim_item", + "resource_id": item.id, + "source_system": "x-financial", + "external_event_id": discovery_event_id, + "content_hash": content_hash, + "occurred_at": occurred_at, + "collected_at": now, + "verification_status": "verified", + "verified_by": "savings-ledger", + "verified_at": now, + "metadata_json": {"business_event_id": discovery_event_id}, + "created_at": now, + } + self.db.add_all( + [ + SavingsEvidenceLink( + id=str(uuid.uuid4()), + evidence_key=f"baseline:{baseline.baseline_key}"[:180], + entity_type="baseline", + entity_id=baseline.id, + baseline_snapshot_id=baseline.id, + **common, + ), + SavingsEvidenceLink( + id=str(uuid.uuid4()), + evidence_key=f"opportunity:{opportunity.opportunity_key}"[:180], + entity_type="opportunity", + entity_id=opportunity.id, + opportunity_id=opportunity.id, + **common, + ), + ] + ) + + def _add_opportunity_event( + self, + opportunity: SavingsOpportunity, + *, + current_user: CurrentUserContext, + stable_digest: str, + causation_id: str, + now: datetime, + ) -> None: + state = { + "id": opportunity.id, + "status": opportunity.status, + "version": opportunity.version, + "estimated_net": str(opportunity.estimated_net), + "currency": opportunity.currency, + } + self.db.add( + SavingsEvent( + id=str(uuid.uuid4()), + tenant_id=opportunity.tenant_id, + aggregate_type="opportunity", + aggregate_id=opportunity.id, + opportunity_id=opportunity.id, + action="discovered_and_started", + actor_id=self._actor_id(current_user), + actor_name=str(current_user.name or current_user.username), + actor_type="user", + request_id=f"discover:{stable_digest}"[:120], + expected_version=0, + result_version=1, + payload_fingerprint=savings_payload_fingerprint( + action="discovered_and_started", + aggregate_type="opportunity", + aggregate_id=opportunity.id, + actor_id=self._actor_id(current_user), + payload={"causation_id": causation_id}, + ), + payload_json={"causation_id": causation_id}, + before_json={}, + after_json=state, + response_json=state, + correlation_id=str(uuid.uuid4()), + causation_id=causation_id, + occurred_at=now, + ) + ) + + @staticmethod + def _dimensions( + claim: ExpenseClaim, + item: ExpenseClaimItem, + flag: dict[str, Any], + ) -> dict[str, Any]: + return { + "department_id": str(claim.department_id or ""), + "department_name": str(claim.department_name or ""), + "project_code": str(claim.project_code or ""), + "expense_type": str(claim.expense_type or item.item_type or ""), + "supplier_id": "", + "supplier_name": "", + "city": str(flag.get("policy_matched_city") or item.item_location or ""), + "employee_id": str(claim.employee_id or ""), + "employee_grade": str(flag.get("policy_grade") or ""), + } + + @staticmethod + def _baseline_snapshot(baseline: ProfileBaselineSnapshot) -> dict[str, Any]: + return { + "id": baseline.id, + "baseline_key": baseline.baseline_key, + "baseline_value": str(baseline.baseline_value), + "method": baseline.method, + "query_fingerprint": baseline.query_fingerprint, + "policy_version": baseline.policy_version, + "policy_effective_from": baseline.policy_effective_from.isoformat() + if baseline.policy_effective_from + else None, + "data_quality_status": baseline.data_quality_status, + "algorithm_version": baseline.algorithm_version, + "frozen_at": baseline.frozen_at.isoformat(), + } + + @staticmethod + def _digest(material: dict[str, Any]) -> str: + canonical = json.dumps( + material, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() + + @staticmethod + def _money(value: Any) -> Decimal: + try: + return Decimal(str(value or "0")).quantize(Decimal("0.0001")) + except (InvalidOperation, ValueError) as error: + raise ValueError("标准调整金额证据无效，不能创建节省机会。") from error + + @staticmethod + def _item_occurred_at(item: ExpenseClaimItem, fallback: datetime) -> datetime: + if item.item_date: + return datetime.combine(item.item_date, datetime.min.time(), tzinfo=UTC) + return fallback + + @staticmethod + def _tenant(current_user: CurrentUserContext) -> str: + return str(current_user.tenant_id or "default").strip() or "default" + + @staticmethod + def _actor_id(current_user: CurrentUserContext) -> str: + return str(current_user.employee_id or current_user.username).strip() diff --git a/server/src/app/services/savings_fact_scope.py b/server/src/app/services/savings_fact_scope.py new file mode 100644 index 0000000..eb492bb --- /dev/null +++ b/server/src/app/services/savings_fact_scope.py @@ -0,0 +1,587 @@ +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass, field +from datetime import UTC, datetime +from decimal import Decimal, InvalidOperation +from typing import Any + +from sqlalchemy import Select, false, func, or_, select +from sqlalchemy.orm import Session, selectinload + +from app.api.deps import CurrentUserContext +from app.models.expense_case import BusinessEvent, ExpenseCaseLink +from app.models.financial_record import ExpenseClaim +from app.schemas.savings_insights import SavingsAnalysisQualityIssue +from app.services.expense_claim_access_policy import ExpenseClaimAccessPolicy +from app.services.expense_claim_tenant_scope import ExpenseClaimTenantScopeMixin +from app.services.savings_access_policy import SavingsAccessPolicy + + +@dataclass(frozen=True, slots=True) +class ExpenseSavingsFact: + claim_id: str + claim_no: str + item_id: str | None + employee_id: str + employee_name: str + department_id: str + department_name: str + project_code: str + expense_type: str + city: str + amount: Decimal + currency: str + occurred_at: datetime + source_updated_at: datetime | None + source_level: str + + def dimension(self, dimension_type: str) -> tuple[str, str] | None: + values = { + "employee": (self.employee_id, self.employee_name), + "department": (self.department_id, self.department_name), + "expense_type": (self.expense_type, self.expense_type), + "city": (self.city, self.city), + "project": (self.project_code, self.project_code), + } + identity, label = values.get(dimension_type, ("", "")) + normalized_identity = str(identity or "").strip() + normalized_label = str(label or "").strip() + if not normalized_identity and not normalized_label: + return None + if not normalized_identity: + normalized_identity = f"name:{normalized_label.casefold()}" + return normalized_identity, normalized_label or normalized_identity + + @property + def source_id(self) -> str: + return self.item_id or self.claim_id + + @property + def content_hash(self) -> str: + return stable_digest( + { + "claim_id": self.claim_id, + "item_id": self.item_id, + "amount": str(self.amount), + "currency": self.currency, + "occurred_at": self.occurred_at, + "updated_at": self.source_updated_at, + } + ) + + +@dataclass(frozen=True, slots=True) +class ExpenseWorkflowCycleFact: + """首个付款完成事件形成的端到端流程历时，不代表人工活跃工时。""" + + claim_id: str + claim_no: str + workflow_key: str + workflow_label: str + started_at: datetime + completed_at: datetime + completion_event_id: str + completion_event_fingerprint: str + + @property + def elapsed_minutes(self) -> Decimal: + seconds = Decimal(str((self.completed_at - self.started_at).total_seconds())) + return (seconds / Decimal("60")).quantize(Decimal("0.0001")) + + @property + def source_id(self) -> str: + return self.completion_event_id + + @property + def content_hash(self) -> str: + return stable_digest( + { + "claim_id": self.claim_id, + "workflow_key": self.workflow_key, + "started_at": self.started_at, + "completed_at": self.completed_at, + "completion_event_id": self.completion_event_id, + "completion_event_fingerprint": self.completion_event_fingerprint, + } + ) + + +@dataclass(slots=True) +class ExpenseSavingsFactSet: + facts: list[ExpenseSavingsFact] = field(default_factory=list) + workflow_cycle_facts: list[ExpenseWorkflowCycleFact] = field(default_factory=list) + claim_count: int = 0 + item_count: int = 0 + data_scope: str = "tenant" + quality_issues: list[SavingsAnalysisQualityIssue] = field(default_factory=list) + + +class SavingsFactScopeReader: + """按 Expense Case tenant link 与用户数据范围读取可审计费用事实。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def load_archived_facts( + self, + *, + window_start: datetime, + window_end: datetime, + as_of: datetime, + current_user: CurrentUserContext, + include_workflow_cycles: bool = False, + ) -> ExpenseSavingsFactSet: + SavingsAccessPolicy.require_tenant_value_read(current_user) + statement = ( + select(ExpenseClaim) + .options(selectinload(ExpenseClaim.items)) + .where( + ExpenseClaimTenantScopeMixin.build_claim_tenant_condition( + current_user.tenant_id + ), + ExpenseClaim.occurred_at >= window_start, + ExpenseClaim.occurred_at <= window_end, + ExpenseClaim.created_at <= as_of, + ExpenseClaimAccessPolicy.build_archived_claim_condition(), + self._reimbursement_claim_condition(), + ) + .order_by(ExpenseClaim.occurred_at.asc(), ExpenseClaim.id.asc()) + ) + statement, data_scope, scope_issue = self._apply_data_scope( + statement, + current_user, + ) + claims = list(self.db.scalars(statement).unique().all()) + result = ExpenseSavingsFactSet( + claim_count=len(claims), + data_scope=data_scope, + ) + if scope_issue is not None: + result.quality_issues.append(scope_issue) + fallback_count = 0 + invalid_count = 0 + outside_window_count = 0 + modified_after_as_of_count = 0 + for claim in claims: + currency = str(claim.currency or "").strip().upper() + if len(currency) != 3: + invalid_count += 1 + continue + items = list(claim.items or []) + if not items: + fact = self._claim_fallback_fact(claim, currency) + if fact is not None: + if not self._fact_in_window( + fact, + window_start=window_start, + window_end=window_end, + as_of=as_of, + ): + outside_window_count += 1 + elif self._datetime_after(fact.source_updated_at, as_of): + modified_after_as_of_count += 1 + else: + result.facts.append(fact) + fallback_count += 1 + else: + invalid_count += 1 + continue + for item in items: + fact = self._item_fact(claim, item, currency) + if fact is None: + invalid_count += 1 + continue + if not self._fact_in_window( + fact, + window_start=window_start, + window_end=window_end, + as_of=as_of, + ): + outside_window_count += 1 + continue + if self._datetime_after(fact.source_updated_at, as_of): + modified_after_as_of_count += 1 + continue + result.facts.append(fact) + result.item_count += 1 + + if fallback_count: + result.quality_issues.append( + SavingsAnalysisQualityIssue( + code="claim_amount_used_without_items", + message="部分已归档单据没有费用明细，基线使用单据总额并降低质量等级。", + metadata={"claim_count": fallback_count}, + ) + ) + if invalid_count: + result.quality_issues.append( + SavingsAnalysisQualityIssue( + code="invalid_expense_fact_excluded", + message="金额非正数、币种无效或日期缺失的费用事实已从分析中排除。", + metadata={"fact_count": invalid_count}, + ) + ) + if outside_window_count: + result.quality_issues.append( + SavingsAnalysisQualityIssue( + code="expense_fact_outside_window_excluded", + message="单据内日期不在分析窗口的费用明细已排除。", + metadata={"fact_count": outside_window_count}, + ) + ) + if modified_after_as_of_count: + result.quality_issues.append( + SavingsAnalysisQualityIssue( + code="expense_fact_modified_after_as_of_excluded", + message=( + "当前表没有双时态历史，截止时间之后被修改的事实无法安全回放，已排除。" + ), + severity="error", + metadata={"fact_count": modified_after_as_of_count}, + ) + ) + if not result.facts: + result.quality_issues.append( + SavingsAnalysisQualityIssue( + code="archived_expense_facts_unavailable", + message="当前租户和数据范围在指定窗口内没有可用于分析的已归档费用事实。", + severity="error", + ) + ) + if include_workflow_cycles: + workflow_facts, workflow_issues = self._load_workflow_cycle_facts( + claims=claims, + window_start=window_start, + window_end=window_end, + as_of=as_of, + current_user=current_user, + ) + result.workflow_cycle_facts = workflow_facts + result.quality_issues.extend(workflow_issues) + return result + + def _load_workflow_cycle_facts( + self, + *, + claims: list[ExpenseClaim], + window_start: datetime, + window_end: datetime, + as_of: datetime, + current_user: CurrentUserContext, + ) -> tuple[list[ExpenseWorkflowCycleFact], list[SavingsAnalysisQualityIssue]]: + issues = [ + SavingsAnalysisQualityIssue( + code="workflow_active_labor_unavailable", + message=( + "流程基线只统计提交至首个付款完成事件的经过时间；当前没有人工活跃计时，" + "不得据此推算人工工时或工时价值。" + ), + severity="info", + dimension_type="workflow", + metadata={ + "metric_key": "median_submission_to_payment_elapsed_minutes", + "unit": "minutes", + "active_labor_available": False, + }, + ) + ] + claim_by_id = {str(claim.id): claim for claim in claims} + if not claim_by_id: + issues.append(self._workflow_unavailable_issue()) + return [], issues + + tenant_id = ExpenseClaimTenantScopeMixin.normalize_tenant_id( + current_user.tenant_id + ) + cutoff = min(self._utc_datetime(window_end), self._utc_datetime(as_of)) + event_matches_claim_case = ( + select(ExpenseCaseLink.id) + .where( + ExpenseCaseLink.tenant_id == tenant_id, + ExpenseCaseLink.expense_case_id == BusinessEvent.expense_case_id, + ExpenseCaseLink.resource_type == "expense_claim", + ExpenseCaseLink.resource_id == BusinessEvent.aggregate_id, + ) + .exists() + ) + events = list( + self.db.scalars( + select(BusinessEvent) + .where( + BusinessEvent.tenant_id == tenant_id, + BusinessEvent.aggregate_type == "expense_claim", + BusinessEvent.aggregate_id.in_(tuple(claim_by_id)), + BusinessEvent.event_type == "payment_completed", + BusinessEvent.occurred_at <= cutoff, + event_matches_claim_case, + ) + .order_by( + BusinessEvent.aggregate_id.asc(), + BusinessEvent.occurred_at.asc(), + BusinessEvent.id.asc(), + ) + ).all() + ) + first_completion_by_claim: dict[str, BusinessEvent] = {} + for event in events: + first_completion_by_claim.setdefault(str(event.aggregate_id), event) + + facts: list[ExpenseWorkflowCycleFact] = [] + invalid_count = 0 + modified_after_as_of_count = 0 + for claim_id, event in first_completion_by_claim.items(): + claim = claim_by_id.get(claim_id) + if claim is None: + continue + completed_at = self._utc_datetime(event.occurred_at) + if not ( + self._utc_datetime(window_start) + <= completed_at + <= self._utc_datetime(window_end) + ): + continue + if self._datetime_after(claim.updated_at, as_of): + modified_after_as_of_count += 1 + continue + if claim.submitted_at is None: + invalid_count += 1 + continue + started_at = self._utc_datetime(claim.submitted_at) + if started_at > completed_at: + invalid_count += 1 + continue + event_fingerprint = stable_digest( + { + "event_id": event.id, + "event_type": event.event_type, + "idempotency_key": event.idempotency_key, + "correlation_id": event.correlation_id, + "occurred_at": completed_at, + } + ) + facts.append( + ExpenseWorkflowCycleFact( + claim_id=claim_id, + claim_no=str(claim.claim_no or claim_id), + workflow_key="reimbursement_submission_to_payment", + workflow_label="报销提交至首个付款完成", + started_at=started_at, + completed_at=completed_at, + completion_event_id=str(event.id), + completion_event_fingerprint=event_fingerprint, + ) + ) + + if invalid_count: + issues.append( + SavingsAnalysisQualityIssue( + code="workflow_cycle_timestamp_invalid", + message="缺少提交时间或完成早于提交的流程事实已排除。", + dimension_type="workflow", + metadata={"fact_count": invalid_count}, + ) + ) + if modified_after_as_of_count: + issues.append( + SavingsAnalysisQualityIssue( + code="workflow_cycle_modified_after_as_of_excluded", + message=( + "当前单据表没有双时态历史，截止时间后被修改的流程起点无法安全回放，已排除。" + ), + severity="error", + dimension_type="workflow", + metadata={"fact_count": modified_after_as_of_count}, + ) + ) + if not facts: + issues.append(self._workflow_unavailable_issue()) + facts.sort(key=lambda fact: (fact.completed_at, fact.claim_id)) + return facts, issues + + @staticmethod + def _workflow_unavailable_issue() -> SavingsAnalysisQualityIssue: + return SavingsAnalysisQualityIssue( + code="workflow_completion_event_unavailable", + message="当前窗口没有同租户可核验的首个付款完成事件，未生成流程周期基线。", + dimension_type="workflow", + ) + + @classmethod + def _fact_in_window( + cls, + fact: ExpenseSavingsFact, + *, + window_start: datetime, + window_end: datetime, + as_of: datetime, + ) -> bool: + occurred_at = cls._utc_datetime(fact.occurred_at) + return ( + cls._utc_datetime(window_start) + <= occurred_at + <= cls._utc_datetime(window_end) + and occurred_at <= cls._utc_datetime(as_of) + ) + + @classmethod + def _datetime_after(cls, value: datetime | None, boundary: datetime) -> bool: + return bool(value and cls._utc_datetime(value) > cls._utc_datetime(boundary)) + + @staticmethod + def _utc_datetime(value: datetime) -> datetime: + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value.astimezone(UTC) + + @staticmethod + def _reimbursement_claim_condition() -> Any: + normalized_type = func.lower(func.coalesce(ExpenseClaim.expense_type, "")) + claim_no = func.upper(func.coalesce(ExpenseClaim.claim_no, "")) + return ~or_( + claim_no.like("AP-%"), + claim_no.like("APP-%"), + claim_no.like("A________"), + normalized_type == "application", + normalized_type.like("%\\_application", escape="\\"), + ) + + @staticmethod + def _apply_data_scope( + statement: Select[tuple[ExpenseClaim]], + current_user: CurrentUserContext, + ) -> tuple[ + Select[tuple[ExpenseClaim]], + str, + SavingsAnalysisQualityIssue | None, + ]: + if SavingsAccessPolicy.can_read_tenant_value(current_user): + return statement, "tenant", None + department_id = str(current_user.department_id or "").strip() + department_name = str(current_user.department_name or "").strip() + if department_id: + return ( + statement.where(ExpenseClaim.department_id == department_id), + "department", + None, + ) + if department_name: + return ( + statement.where(ExpenseClaim.department_name == department_name), + "department", + None, + ) + return ( + statement.where(false()), + "unavailable", + SavingsAnalysisQualityIssue( + code="claim_cost_center_scope_unavailable", + message="费用单据没有独立成本中心字段，不能把仅成本中心权限安全映射到单据事实。", + severity="error", + ), + ) + + @classmethod + def _item_fact( + cls, + claim: ExpenseClaim, + item: Any, + currency: str, + ) -> ExpenseSavingsFact | None: + amount = cls._positive_money(item.item_amount) + if amount is None or item.item_date is None: + return None + occurred_at = datetime.combine(item.item_date, datetime.min.time(), tzinfo=UTC) + return cls._fact( + claim, + item_id=str(item.id), + amount=amount, + currency=currency, + occurred_at=occurred_at, + expense_type=str(item.item_type or claim.expense_type or "").strip(), + city=str(item.item_location or claim.location or "").strip(), + source_updated_at=cls._latest_datetime(item.updated_at, claim.updated_at), + source_level="claim_item", + ) + + @classmethod + def _claim_fallback_fact( + cls, + claim: ExpenseClaim, + currency: str, + ) -> ExpenseSavingsFact | None: + amount = cls._positive_money(claim.amount) + if amount is None or claim.occurred_at is None: + return None + return cls._fact( + claim, + item_id=None, + amount=amount, + currency=currency, + occurred_at=claim.occurred_at, + expense_type=str(claim.expense_type or "").strip(), + city=str(claim.location or "").strip(), + source_updated_at=claim.updated_at, + source_level="claim_fallback", + ) + + @staticmethod + def _fact( + claim: ExpenseClaim, + *, + item_id: str | None, + amount: Decimal, + currency: str, + occurred_at: datetime, + expense_type: str, + city: str, + source_updated_at: datetime | None, + source_level: str, + ) -> ExpenseSavingsFact: + return ExpenseSavingsFact( + claim_id=str(claim.id), + claim_no=str(claim.claim_no or claim.id), + item_id=item_id, + employee_id=str(claim.employee_id or "").strip(), + employee_name=str(claim.employee_name or "").strip(), + department_id=str(claim.department_id or "").strip(), + department_name=str(claim.department_name or "").strip(), + project_code=str(claim.project_code or "").strip(), + expense_type=expense_type, + city=city, + amount=amount, + currency=currency, + occurred_at=occurred_at, + source_updated_at=source_updated_at, + source_level=source_level, + ) + + @staticmethod + def _positive_money(value: Any) -> Decimal | None: + try: + amount = Decimal(str(value or "0")).quantize(Decimal("0.0001")) + except (InvalidOperation, ValueError): + return None + return amount if amount > 0 else None + + @classmethod + def _latest_datetime( + cls, + *values: datetime | None, + ) -> datetime | None: + available = [value for value in values if value is not None] + if not available: + return None + return max(available, key=cls._utc_datetime) + + +def stable_digest(value: Any) -> str: + canonical = json.dumps( + value, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + default=lambda item: item.isoformat() if hasattr(item, "isoformat") else str(item), + ) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() diff --git a/server/src/app/services/savings_insight_analysis.py b/server/src/app/services/savings_insight_analysis.py new file mode 100644 index 0000000..1acd2e9 --- /dev/null +++ b/server/src/app/services/savings_insight_analysis.py @@ -0,0 +1,408 @@ +from __future__ import annotations + +from collections import defaultdict +from datetime import UTC, datetime +from decimal import Decimal + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.savings import ProfileBaselineSnapshot +from app.schemas.savings_insights import ( + SavingsAnalysisQualityIssue, + SavingsInsightAnalysisRead, + SavingsInsightAnalyzeRequest, + SavingsInsightCandidateRead, + SavingsInsightEvidenceRead, +) +from app.services.savings_access_policy import SavingsAccessPolicy +from app.services.savings_fact_scope import ( + ExpenseSavingsFact, + SavingsFactScopeReader, + stable_digest, +) +from app.services.savings_insight_attribution import ( + SavingsAnomalyAttributionAnalyzer, +) +from app.services.savings_insight_budget import SavingsBudgetForecastAnalyzer +from app.services.savings_protocol import ( + SavingsRequestProtocol, + savings_actor, + savings_payload_fingerprint, +) + + +class SavingsInsightAnalysisService: + """发现可行动费用信号；无可信反事实时明确拒绝货币化。""" + + MAX_CANDIDATES = 200 + + def __init__(self, db: Session) -> None: + self.db = db + self.protocol = SavingsRequestProtocol(db) + self.fact_reader = SavingsFactScopeReader(db) + self.budget_analyzer = SavingsBudgetForecastAnalyzer(db) + self.attribution_analyzer = SavingsAnomalyAttributionAnalyzer() + + def analyze( + self, + payload: SavingsInsightAnalyzeRequest, + current_user: CurrentUserContext, + ) -> SavingsInsightAnalysisRead: + tenant_id = str(current_user.tenant_id or "default").strip() or "default" + actor_id, _ = savings_actor(current_user) + request_fingerprint = savings_payload_fingerprint( + action="analyze_savings_insights", + aggregate_type="insight_batch", + aggregate_id=payload.request_id, + actor_id=actor_id, + payload=payload.model_dump(mode="json", exclude={"request_id"}), + ) + now = datetime.now(UTC) + with self.protocol.serialize( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + ): + fact_set = self.fact_reader.load_archived_facts( + window_start=payload.window_start, + window_end=payload.window_end, + as_of=payload.as_of, + current_user=current_user, + ) + budget_candidates, budget_issues = self.budget_analyzer.analyze( + window_start=payload.window_start, + window_end=payload.window_end, + as_of=payload.as_of, + current_user=current_user, + ) + repeated_candidates = self._repeated_small_expenses( + fact_set.facts, + payload, + ) + price_candidates, baseline_issues = self._historical_price_deviations( + tenant_id=tenant_id, + facts=fact_set.facts, + payload=payload, + current_user=current_user, + ) + attribution_candidates = self.attribution_analyzer.analyze( + price_candidates, + minimum_group_size=payload.minimum_repeat_count, + ) + + candidates = [ + *budget_candidates, + *repeated_candidates, + *price_candidates, + *attribution_candidates, + ] + candidates.sort(key=lambda item: (item.insight_type, item.candidate_key)) + issues = [ + *fact_set.quality_issues, + *budget_issues, + *baseline_issues, + self._supplier_coverage_issue(), + ] + if len(candidates) > self.MAX_CANDIDATES: + issues.append( + SavingsAnalysisQualityIssue( + code="candidate_result_truncated", + message="候选结果超过单次返回上限，已按稳定键截断；建议缩小分析窗口。", + metadata={ + "candidate_count": len(candidates), + "returned_count": self.MAX_CANDIDATES, + }, + ) + ) + candidates = candidates[: self.MAX_CANDIDATES] + return SavingsInsightAnalysisRead( + request_id=payload.request_id, + request_fingerprint=request_fingerprint, + tenant_id=tenant_id, + data_scope=fact_set.data_scope, + candidates=candidates, + quality_issues=issues, + source_claim_count=fact_set.claim_count, + source_item_count=fact_set.item_count, + created_opportunity_ids=[], + monetized_opportunity_count=0, + generated_at=now, + ) + + def _repeated_small_expenses( + self, + facts: list[ExpenseSavingsFact], + payload: SavingsInsightAnalyzeRequest, + ) -> list[SavingsInsightCandidateRead]: + grouped: dict[tuple[str, str, str, str, Decimal, str], list[ExpenseSavingsFact]] = ( + defaultdict(list) + ) + for fact in facts: + if fact.amount > payload.small_amount_threshold: + continue + employee = fact.dimension("employee") + if employee is None or not fact.expense_type: + continue + employee_id, employee_label = employee + grouped[ + ( + employee_id, + employee_label, + fact.expense_type, + fact.city, + fact.amount.quantize(Decimal("0.01")), + fact.currency, + ) + ].append(fact) + + candidates: list[SavingsInsightCandidateRead] = [] + for key, group in sorted(grouped.items(), key=lambda row: str(row[0])): + distinct_claim_ids = sorted({fact.claim_id for fact in group}) + if len(distinct_claim_ids) < payload.minimum_repeat_count: + continue + employee_id, employee_label, expense_type, city, amount, currency = key + material = { + "employee_id": employee_id, + "expense_type": expense_type, + "city": city, + "amount": str(amount), + "currency": currency, + "source_hashes": sorted(fact.content_hash for fact in group), + } + candidate_key = f"repeated-small:{stable_digest(material)}" + observed_spend = sum((fact.amount for fact in group), Decimal("0.0000")) + evidence = [self._fact_evidence(fact, "repeated_expense_fact") for fact in group[:100]] + quality_issues = [ + SavingsAnalysisQualityIssue( + code="procurement_counterfactual_missing", + message=( + "重复发生只能证明支出模式，不能证明浪费；缺少协议价、合并采购报价或政策上限，" + "因此不生成节省金额。" + ), + metadata={"distinct_claim_count": len(distinct_claim_ids)}, + ) + ] + if len(group) > 100: + quality_issues.append( + SavingsAnalysisQualityIssue( + code="candidate_evidence_truncated", + message="候选证据列表只返回前 100 条，完整来源仍包含在稳定指纹中。", + metadata={"source_count": len(group)}, + ) + ) + candidates.append( + SavingsInsightCandidateRead( + candidate_key=candidate_key, + insight_type="repeated_small_expense_pattern", + title=f"{employee_label} 的重复小额 {expense_type} 支出", + description=( + f"窗口内有 {len(distinct_claim_ids)} 张不同单据出现相同金额 {amount} " + f"{currency} 的 {expense_type} 支出，建议核查是否可合并采购或使用协议渠道。" + ), + dimension_json={ + "employee_id": employee_id, + "employee_name": employee_label, + "expense_type": expense_type, + "city": city, + "amount": str(amount), + "distinct_claim_count": len(distinct_claim_ids), + }, + evidence=evidence, + evidence_sufficient_for_signal=True, + data_quality_status=( + "complete" + if all(fact.source_level == "claim_item" for fact in group) + else "partial" + ), + quality_issues=quality_issues, + exposure_amount=observed_spend.quantize(Decimal("0.0001")), + exposure_meaning="observed_repeated_spend_not_savings", + currency=currency, + estimated_savings=None, + monetization_status="withheld_no_counterfactual", + ) + ) + return candidates + + def _historical_price_deviations( + self, + *, + tenant_id: str, + facts: list[ExpenseSavingsFact], + payload: SavingsInsightAnalyzeRequest, + current_user: CurrentUserContext, + ) -> tuple[list[SavingsInsightCandidateRead], list[SavingsAnalysisQualityIssue]]: + if not SavingsAccessPolicy.can_read_tenant_value(current_user): + return [], [ + SavingsAnalysisQualityIssue( + code="historical_baseline_scope_unverifiable", + message=( + "历史费用类型/城市基线没有记录部门范围，不能向仅部门权限用户安全披露。" + ), + severity="error", + ) + ] + baselines = list( + self.db.scalars( + select(ProfileBaselineSnapshot) + .where( + ProfileBaselineSnapshot.tenant_id == tenant_id, + ProfileBaselineSnapshot.baseline_type == "historical_cohort", + ProfileBaselineSnapshot.metric_key == "median_expense_fact_amount", + ProfileBaselineSnapshot.method + == "median_archived_expense_facts_tenant_scope", + ProfileBaselineSnapshot.dimension_type.in_(("expense_type", "city")), + ProfileBaselineSnapshot.data_quality_status.in_(("complete", "partial")), + ProfileBaselineSnapshot.window_end < payload.window_start, + ProfileBaselineSnapshot.frozen_at <= payload.as_of, + ) + .order_by( + ProfileBaselineSnapshot.window_end.desc(), + ProfileBaselineSnapshot.frozen_at.desc(), + ProfileBaselineSnapshot.id.desc(), + ) + ).all() + ) + baselines = [ + baseline + for baseline in baselines + if not any( + isinstance(issue, dict) + and issue.get("code") == "baseline_contains_claim_fallback" + for issue in list(baseline.quality_issues_json or []) + ) + ] + if not baselines: + return [], [ + SavingsAnalysisQualityIssue( + code="historical_baseline_unavailable", + message="观察窗口之前没有质量合格的费用类型或城市历史基线。", + ) + ] + by_dimension: dict[tuple[str, str, str], ProfileBaselineSnapshot] = {} + for baseline in baselines: + key = ( + str(baseline.dimension_type), + str(baseline.dimension_id), + str(baseline.original_currency or ""), + ) + by_dimension.setdefault(key, baseline) + + candidates: list[SavingsInsightCandidateRead] = [] + for fact in facts: + matches: list[ProfileBaselineSnapshot] = [] + for dimension_type in ("expense_type", "city"): + dimension = fact.dimension(dimension_type) + if dimension is None: + continue + baseline = by_dimension.get( + (dimension_type, dimension[0], fact.currency) + ) + if baseline is not None: + matches.append(baseline) + if not matches: + continue + baseline = max(matches, key=lambda row: int(row.sample_count or 0)) + baseline_value = Decimal(baseline.baseline_value or 0) + if baseline_value <= 0: + continue + ratio = fact.amount / baseline_value + if ratio < payload.price_deviation_ratio: + continue + delta = (fact.amount - baseline_value).quantize(Decimal("0.0001")) + material = { + "baseline_id": baseline.id, + "fact_hash": fact.content_hash, + "ratio": str(payload.price_deviation_ratio), + } + candidates.append( + SavingsInsightCandidateRead( + candidate_key=f"historical-price:{stable_digest(material)}", + insight_type="historical_price_deviation", + title=f"单据 {fact.claim_no} 的费用金额高于历史中位数", + description=( + f"当前金额 {fact.amount} {fact.currency}，历史中位数为 " + f"{baseline_value} {fact.currency}。差额是偏差信号，" + "不是已证明的可节省金额。" + ), + dimension_json={ + "claim_id": fact.claim_id, + "claim_item_id": fact.item_id or "", + "expense_type": fact.expense_type, + "city": fact.city, + "department_id": fact.department_id, + "department_name": fact.department_name, + "project_code": fact.project_code, + "baseline_dimension_type": baseline.dimension_type, + "baseline_dimension_id": baseline.dimension_id, + "deviation_ratio": str(ratio.quantize(Decimal("0.0001"))), + }, + evidence=[ + self._fact_evidence(fact, "observed_expense_fact"), + SavingsInsightEvidenceRead( + evidence_role="historical_baseline", + resource_type="profile_baseline_snapshot", + resource_id=baseline.id, + content_hash=baseline.query_fingerprint, + occurred_at=baseline.frozen_at, + metadata={ + "sample_count": baseline.sample_count, + "quality_status": baseline.data_quality_status, + }, + ), + ], + evidence_sufficient_for_signal=True, + data_quality_status="partial", + quality_issues=[ + SavingsAnalysisQualityIssue( + code="unit_supplier_comparability_missing", + message=( + "当前模型缺少数量、单位和核验供应商，不能证明与历史样本同质，" + "因此不生成节省金额。" + ), + dimension_type=str(baseline.dimension_type), + dimension_id=str(baseline.dimension_id), + ) + ], + exposure_amount=delta, + exposure_meaning="observed_amount_above_historical_median", + currency=fact.currency, + estimated_savings=None, + monetization_status="withheld_no_counterfactual", + baseline_snapshot_id=baseline.id, + ) + ) + return candidates, [] + + @staticmethod + def _fact_evidence( + fact: ExpenseSavingsFact, + role: str, + ) -> SavingsInsightEvidenceRead: + return SavingsInsightEvidenceRead( + evidence_role=role, + resource_type=("expense_claim_item" if fact.item_id else "expense_claim"), + resource_id=fact.source_id, + content_hash=fact.content_hash, + occurred_at=fact.occurred_at, + metadata={ + "claim_id": fact.claim_id, + "claim_no": fact.claim_no, + "amount": str(fact.amount), + "currency": fact.currency, + }, + ) + + @staticmethod + def _supplier_coverage_issue() -> SavingsAnalysisQualityIssue: + return SavingsAnalysisQualityIssue( + code="supplier_price_drift_unavailable", + message=( + "费用事实没有核验供应商、商品数量和单位价格，供应商价格漂移分析保持不可用，" + "不会推算节省金额。" + ), + severity="error", + dimension_type="supplier", + ) diff --git a/server/src/app/services/savings_insight_attribution.py b/server/src/app/services/savings_insight_attribution.py new file mode 100644 index 0000000..90929f0 --- /dev/null +++ b/server/src/app/services/savings_insight_attribution.py @@ -0,0 +1,230 @@ +from __future__ import annotations + +from collections import defaultdict +from decimal import Decimal + +from app.schemas.savings_insights import ( + SavingsAnalysisQualityIssue, + SavingsInsightCandidateRead, + SavingsInsightEvidenceRead, +) +from app.services.savings_fact_scope import stable_digest + + +class SavingsAnomalyAttributionAnalyzer: + """把异常集中度变成描述性归因与政策模拟准备项，不声称因果或节省。""" + + DRIVER_FIELDS = ( + ("department", "department_id", "department_name"), + ("expense_type", "expense_type", "expense_type"), + ("city", "city", "city"), + ("project", "project_code", "project_code"), + ) + + def analyze( + self, + price_deviations: list[SavingsInsightCandidateRead], + *, + minimum_group_size: int, + ) -> list[SavingsInsightCandidateRead]: + attributions = self._attributions( + price_deviations, + minimum_group_size=minimum_group_size, + ) + simulations = [ + self._policy_simulation_candidate(candidate) + for candidate in attributions + if candidate.dimension_json.get("driver_type") in {"expense_type", "city"} + ] + return [*attributions, *simulations] + + def _attributions( + self, + candidates: list[SavingsInsightCandidateRead], + *, + minimum_group_size: int, + ) -> list[SavingsInsightCandidateRead]: + grouped: dict[ + tuple[str, str, str, str], + list[SavingsInsightCandidateRead], + ] = defaultdict(list) + for candidate in candidates: + dimensions = candidate.dimension_json + currency = str(candidate.currency or "").strip().upper() + claim_id = str(dimensions.get("claim_id") or "").strip() + if not currency or not claim_id: + continue + for driver_type, identity_key, label_key in self.DRIVER_FIELDS: + identity = str(dimensions.get(identity_key) or "").strip() + label = str(dimensions.get(label_key) or identity).strip() + if not identity and not label: + continue + if not identity: + identity = f"name:{label.casefold()}" + grouped[(driver_type, identity, label or identity, currency)].append( + candidate + ) + + results: list[SavingsInsightCandidateRead] = [] + for (driver_type, driver_id, driver_label, currency), group in sorted( + grouped.items(), + key=lambda item: item[0], + ): + claim_ids = sorted( + { + str(candidate.dimension_json.get("claim_id") or "").strip() + for candidate in group + if str(candidate.dimension_json.get("claim_id") or "").strip() + } + ) + if len(claim_ids) < minimum_group_size: + continue + source_keys = sorted(candidate.candidate_key for candidate in group) + evidence = self._deduplicated_evidence(group) + exposure = sum( + ( + Decimal(candidate.exposure_amount) + for candidate in group + if candidate.exposure_amount is not None + ), + Decimal("0.0000"), + ).quantize(Decimal("0.0001")) + quality_issues = [ + SavingsAnalysisQualityIssue( + code="descriptive_attribution_not_causal", + message=( + "该归因只证明异常在此维度集中，不证明该维度导致支出，也不能直接作为节省金额。" + ), + dimension_type=driver_type, + dimension_id=driver_id, + metadata={"distinct_claim_count": len(claim_ids)}, + ) + ] + if len(evidence) == 100 and sum(len(item.evidence) for item in group) > 100: + quality_issues.append( + SavingsAnalysisQualityIssue( + code="candidate_evidence_truncated", + message="归因候选只返回前 100 条去重证据，完整来源保留在稳定指纹中。", + metadata={"source_candidate_count": len(group)}, + ) + ) + material = { + "driver_type": driver_type, + "driver_id": driver_id, + "currency": currency, + "source_candidate_keys": source_keys, + } + results.append( + SavingsInsightCandidateRead( + candidate_key=f"anomaly-attribution:{stable_digest(material)}", + insight_type="anomaly_driver_attribution", + title=f"{driver_label} 的费用异常集中度需要复核", + description=( + f"{len(claim_ids)} 张不同单据在 {driver_type}={driver_label} 维度出现" + f"高于历史中位数的金额，合计偏差暴露 {exposure} {currency}。" + "这是描述性集中信号，不是因果结论或可确认节省。" + ), + dimension_json={ + "driver_type": driver_type, + "driver_id": driver_id, + "driver_label": driver_label, + "distinct_claim_count": len(claim_ids), + "claim_ids": claim_ids[:100], + "source_candidate_count": len(source_keys), + "source_candidate_keys": source_keys[:100], + "source_candidate_keys_truncated": len(source_keys) > 100, + "attribution_kind": "descriptive_concentration_not_causal", + "recommended_action": ( + "由费用治理负责人核验业务必要性、同质性和适用政策，再决定是否运行政策模拟。" + ), + }, + evidence=evidence, + evidence_sufficient_for_signal=True, + data_quality_status="partial", + quality_issues=quality_issues, + exposure_amount=exposure, + exposure_meaning="aggregated_historical_deviation_not_savings", + currency=currency, + estimated_savings=None, + monetization_status="withheld_no_counterfactual", + ) + ) + return results + + @staticmethod + def _deduplicated_evidence( + candidates: list[SavingsInsightCandidateRead], + ) -> list[SavingsInsightEvidenceRead]: + unique: dict[tuple[str, str, str, str], SavingsInsightEvidenceRead] = {} + for candidate in candidates: + for evidence in candidate.evidence: + key = ( + evidence.evidence_role, + evidence.resource_type, + evidence.resource_id, + evidence.content_hash, + ) + unique.setdefault(key, evidence) + return [unique[key] for key in sorted(unique)[:100]] + + @staticmethod + def _policy_simulation_candidate( + attribution: SavingsInsightCandidateRead, + ) -> SavingsInsightCandidateRead: + dimensions = dict(attribution.dimension_json) + driver_type = str(dimensions.get("driver_type") or "") + driver_id = str(dimensions.get("driver_id") or "") + driver_label = str(dimensions.get("driver_label") or driver_id) + material = { + "attribution_candidate_key": attribution.candidate_key, + "policy_scope": {"dimension_type": driver_type, "dimension_id": driver_id}, + "required_counterfactual": "versioned_policy_limit", + } + return SavingsInsightCandidateRead( + candidate_key=f"policy-simulation:{stable_digest(material)}", + insight_type="policy_simulation_candidate", + title=f"为 {driver_label} 准备版本化政策模拟", + description=( + "已有异常集中证据，可先创建带生效期、适用范围和审批人的政策草案并运行只读模拟；" + "当前没有核验政策反事实，因此不估算节省、不自动创建机会。" + ), + dimension_json={ + "driver_type": driver_type, + "driver_id": driver_id, + "driver_label": driver_label, + "source_attribution_candidate_key": attribution.candidate_key, + "simulation_action": "run_versioned_policy_counterfactual", + "required_inputs": [ + "published_policy_version", + "effective_period", + "eligible_expense_scope", + "approved_limit_or_rate", + "exception_and_appeal_rules", + ], + "write_mode": "read_only_no_opportunity_creation", + }, + evidence=list(attribution.evidence), + evidence_sufficient_for_signal=True, + data_quality_status="partial", + quality_issues=[ + SavingsAnalysisQualityIssue( + code="versioned_policy_counterfactual_required", + message=( + "历史中位数不是政策反事实；只有正式版本化政策、适用期和目标口径齐全后" + "才能计算模拟结果。" + ), + dimension_type=driver_type, + dimension_id=driver_id, + ), + SavingsAnalysisQualityIssue( + code="policy_simulation_does_not_create_opportunity", + message="政策模拟候选保持只读，需业务审批和反事实核验后另行发现机会。", + severity="info", + ), + ], + exposure_amount=attribution.exposure_amount, + exposure_meaning="observed_anomaly_exposure_for_simulation_not_savings", + currency=attribution.currency, + estimated_savings=None, + monetization_status="withheld_no_counterfactual", + ) diff --git a/server/src/app/services/savings_insight_budget.py b/server/src/app/services/savings_insight_budget.py new file mode 100644 index 0000000..b522983 --- /dev/null +++ b/server/src/app/services/savings_insight_budget.py @@ -0,0 +1,318 @@ +from __future__ import annotations + +import calendar +import re +from datetime import UTC, datetime +from decimal import Decimal + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.budget import BudgetAllocation, BudgetTransaction +from app.schemas.savings_insights import ( + SavingsAnalysisQualityIssue, + SavingsInsightCandidateRead, + SavingsInsightEvidenceRead, +) +from app.services.finance_dashboard_scope import finance_dashboard_includes_legacy_budget +from app.services.savings_access_policy import SavingsAccessPolicy +from app.services.savings_fact_scope import stable_digest + + +class SavingsBudgetForecastAnalyzer: + """只把预算偏差作为暴露信号，不把预测超支伪装成节省。""" + + def __init__(self, db: Session) -> None: + self.db = db + + def analyze( + self, + *, + window_start: datetime, + window_end: datetime, + as_of: datetime, + current_user: CurrentUserContext, + ) -> tuple[list[SavingsInsightCandidateRead], list[SavingsAnalysisQualityIssue]]: + if not finance_dashboard_includes_legacy_budget(current_user.tenant_id): + return [], [ + SavingsAnalysisQualityIssue( + code="tenant_budget_scope_unavailable", + message="旧预算表没有 tenant_id，非 default 租户禁止读取，未生成预算预测洞察。", + severity="error", + ) + ] + + analysis_cutoff = min(as_of, window_end) + statement = select(BudgetAllocation).where( + BudgetAllocation.status == "active", + BudgetAllocation.created_at <= analysis_cutoff, + BudgetAllocation.updated_at <= analysis_cutoff, + ) + statement = self._apply_scope(statement, current_user) + allocations = list(self.db.scalars(statement).all()) + candidates: list[SavingsInsightCandidateRead] = [] + issues: list[SavingsAnalysisQualityIssue] = [] + unsupported_periods = 0 + insufficient_samples = 0 + for allocation in allocations: + bounds = self._period_bounds(allocation) + if bounds is None: + unsupported_periods += 1 + continue + period_start, period_end = bounds + if period_end < window_start or period_start > window_end: + continue + rows = list( + self.db.scalars( + select(BudgetTransaction) + .where( + BudgetTransaction.allocation_id == allocation.id, + BudgetTransaction.created_at >= period_start, + BudgetTransaction.created_at + <= min(analysis_cutoff, period_end), + BudgetTransaction.transaction_type.in_(("consume", "rollback")), + ) + .order_by(BudgetTransaction.created_at.asc(), BudgetTransaction.id.asc()) + ).all() + ) + distinct_sources = { + (str(row.source_type or ""), str(row.source_id or "")) for row in rows + } + if len(distinct_sources) < 2: + insufficient_samples += 1 + continue + consumed = sum( + ( + -Decimal(row.amount) + if str(row.transaction_type).strip().lower() == "rollback" + else Decimal(row.amount) + for row in rows + ), + Decimal("0.0000"), + ) + consumed = max(consumed, Decimal("0.0000")) + total_budget = ( + Decimal(allocation.original_amount or 0) + + Decimal(allocation.adjusted_amount or 0) + ).quantize(Decimal("0.0001")) + if total_budget <= 0: + continue + elapsed_ratio = self._elapsed_ratio( + period_start, + period_end, + min(analysis_cutoff, period_end), + ) + if elapsed_ratio < Decimal("0.2000"): + insufficient_samples += 1 + continue + projected = (consumed / elapsed_ratio).quantize(Decimal("0.0001")) + if projected <= total_budget * Decimal("1.0500"): + continue + exposure = (projected - total_budget).quantize(Decimal("0.0001")) + candidates.append( + self._candidate( + allocation=allocation, + rows=rows, + consumed=consumed, + projected=projected, + total_budget=total_budget, + exposure=exposure, + elapsed_ratio=elapsed_ratio, + as_of=analysis_cutoff, + ) + ) + + if unsupported_periods: + issues.append( + SavingsAnalysisQualityIssue( + code="budget_period_format_unsupported", + message="部分预算期间无法解析，已从预测中排除。", + metadata={"allocation_count": unsupported_periods}, + ) + ) + if allocations and insufficient_samples and not candidates: + issues.append( + SavingsAnalysisQualityIssue( + code="budget_forecast_sample_insufficient", + message="预算核销来源或期间进度不足，未生成预测偏差信号。", + metadata={"allocation_count": insufficient_samples}, + ) + ) + if not allocations: + issues.append( + SavingsAnalysisQualityIssue( + code="budget_allocation_unavailable", + message="当前数据范围没有活动预算额度，未生成预算预测洞察。", + ) + ) + return candidates, issues + + @staticmethod + def _apply_scope(statement, current_user: CurrentUserContext): + if SavingsAccessPolicy.can_read_tenant_value(current_user): + return statement + department_id = str(current_user.department_id or "").strip() + if department_id: + return statement.where(BudgetAllocation.department_id == department_id) + department_name = str(current_user.department_name or "").strip() + if department_name: + return statement.where(BudgetAllocation.department_name == department_name) + cost_center = str(current_user.cost_center or "").strip() + if cost_center: + return statement.where(BudgetAllocation.cost_center == cost_center) + return statement.where(False) + + @staticmethod + def _period_bounds( + allocation: BudgetAllocation, + ) -> tuple[datetime, datetime] | None: + key = str(allocation.period_key or "").strip().upper() + quarter = re.fullmatch(r"(\d{4})Q([1-4])", key) + if quarter: + year, number = int(quarter.group(1)), int(quarter.group(2)) + start_month = (number - 1) * 3 + 1 + end_month = start_month + 2 + return ( + datetime(year, start_month, 1, tzinfo=UTC), + datetime( + year, + end_month, + calendar.monthrange(year, end_month)[1], + 23, + 59, + 59, + 999999, + tzinfo=UTC, + ), + ) + month = re.fullmatch(r"(\d{4})[-M](\d{1,2})", key) + if month: + year, number = int(month.group(1)), int(month.group(2)) + if not 1 <= number <= 12: + return None + return ( + datetime(year, number, 1, tzinfo=UTC), + datetime( + year, + number, + calendar.monthrange(year, number)[1], + 23, + 59, + 59, + 999999, + tzinfo=UTC, + ), + ) + if re.fullmatch(r"\d{4}", key): + year = int(key) + return ( + datetime(year, 1, 1, tzinfo=UTC), + datetime(year, 12, 31, 23, 59, 59, 999999, tzinfo=UTC), + ) + return None + + @staticmethod + def _elapsed_ratio(start: datetime, end: datetime, as_of: datetime) -> Decimal: + total_seconds = Decimal(str((end - start).total_seconds())) + elapsed_seconds = Decimal(str((min(as_of, end) - start).total_seconds())) + if total_seconds <= 0: + return Decimal("1.0000") + return max( + Decimal("0.0000"), + min(Decimal("1.0000"), elapsed_seconds / total_seconds), + ).quantize(Decimal("0.0001")) + + @staticmethod + def _candidate( + *, + allocation: BudgetAllocation, + rows: list[BudgetTransaction], + consumed: Decimal, + projected: Decimal, + total_budget: Decimal, + exposure: Decimal, + elapsed_ratio: Decimal, + as_of: datetime, + ) -> SavingsInsightCandidateRead: + material = { + "allocation_id": allocation.id, + "as_of": as_of, + "transaction_ids": [row.id for row in rows], + "projected": str(projected), + } + candidate_key = f"budget-forecast:{stable_digest(material)}" + evidence = [ + SavingsInsightEvidenceRead( + evidence_role="budget_allocation", + resource_type="budget_allocation", + resource_id=allocation.id, + content_hash=stable_digest( + { + "original_amount": allocation.original_amount, + "adjusted_amount": allocation.adjusted_amount, + "period_key": allocation.period_key, + } + ), + occurred_at=allocation.updated_at or allocation.created_at or as_of, + metadata={"budget_no": allocation.budget_no}, + ) + ] + evidence.extend( + SavingsInsightEvidenceRead( + evidence_role="budget_consumption", + resource_type="budget_transaction", + resource_id=row.id, + content_hash=stable_digest( + { + "type": row.transaction_type, + "amount": row.amount, + "source_type": row.source_type, + "source_id": row.source_id, + } + ), + occurred_at=row.created_at or as_of, + metadata={"source_id": row.source_id, "source_no": row.source_no}, + ) + for row in rows[:100] + ) + return SavingsInsightCandidateRead( + candidate_key=candidate_key, + insight_type="budget_forecast_variance", + title=f"{allocation.department_name} {allocation.subject_name} 预算预测偏差", + description=( + f"按当前期间进度和已核销金额推算，期末支出约 {projected}，" + f"预算额度为 {total_budget}。该差额仅表示超支暴露，不代表可节省金额。" + ), + dimension_json={ + "department_id": allocation.department_id or "", + "department_name": allocation.department_name, + "cost_center": allocation.cost_center or "", + "project_code": allocation.project_code or "", + "subject_code": allocation.subject_code, + "period_key": allocation.period_key, + "consumed_amount": str(consumed), + "projected_amount": str(projected), + "total_budget": str(total_budget), + "elapsed_ratio": str(elapsed_ratio), + }, + evidence=evidence, + evidence_sufficient_for_signal=True, + data_quality_status="partial", + quality_issues=[ + SavingsAnalysisQualityIssue( + code="forecast_exposure_not_savings", + message="没有可执行的费用削减反事实，预测超支暴露不得计入节省台账。", + metadata={"transaction_count": len(rows)}, + ), + SavingsAnalysisQualityIssue( + code="budget_currency_unavailable", + message="旧预算事实没有币种字段，暴露金额不标注或推断币种。", + ), + ], + exposure_amount=exposure, + exposure_meaning="projected_spend_above_budget", + currency=None, + estimated_savings=None, + monetization_status="withheld_no_counterfactual", + ) diff --git a/server/src/app/services/savings_payment_reversal.py b/server/src/app/services/savings_payment_reversal.py new file mode 100644 index 0000000..c6d3026 --- /dev/null +++ b/server/src/app/services/savings_payment_reversal.py @@ -0,0 +1,199 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.expense_case import BusinessEvent +from app.models.financial_record import ExpenseClaim +from app.models.savings import SavingsOpportunity, SavingsRealization +from app.schemas.savings import SavingsEvidenceCreate +from app.services.savings_protocol import savings_payload_fingerprint +from app.services.savings_realization import SavingsRealizationService + + +class SavingsPaymentReversalService: + """把可信付款冲回同步成追加式 Savings 纠正事实；不负责 commit。""" + + def __init__(self, db: Session) -> None: + self.db = db + self.core = SavingsRealizationService(db) + + def reconcile( + self, + claim: ExpenseClaim, + payment_reversal_event: BusinessEvent, + current_user: CurrentUserContext, + ) -> list[SavingsRealization]: + tenant_id = str(current_user.tenant_id or "default") + if payment_reversal_event.tenant_id != tenant_id: + raise PermissionError("付款冲回事件与 Savings 租户不一致。") + statement = select(SavingsRealization).where( + SavingsRealization.tenant_id == tenant_id, + SavingsRealization.claim_id == claim.id, + SavingsRealization.realization_type == "actual", + SavingsRealization.status.in_(("pending_confirmation", "finance_confirmed")), + SavingsRealization.reversed_at.is_(None), + SavingsRealization.realization_key.like("payment:%"), + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + realizations = list(self.db.scalars(statement).all()) + corrected: list[SavingsRealization] = [] + for realization in realizations: + opportunity = self.core._lock_opportunity( + realization.opportunity_id, + tenant_id=tenant_id, + ) + if opportunity is None: + raise RuntimeError("付款冲回关联的 Savings 机会不存在。") + corrected.append( + self._correct_one( + opportunity, + realization, + payment_reversal_event=payment_reversal_event, + current_user=current_user, + ) + ) + self.db.flush() + return corrected + + def _correct_one( + self, + opportunity: SavingsOpportunity, + realization: SavingsRealization, + *, + payment_reversal_event: BusinessEvent, + current_user: CurrentUserContext, + ) -> SavingsRealization: + now = payment_reversal_event.occurred_at or datetime.now(UTC) + actor_id = current_user.username + actor_name = current_user.name or current_user.username + prior_realization_version = realization.version + prior_opportunity_version = opportunity.version + before = self.core._realization_state(realization) + comment = "外部付款已退款或冲回，原现金节省结果同步失效。" + if realization.status == "finance_confirmed": + result = self.core._reverse( + opportunity, + realization, + actor_id=actor_id, + actor_name=actor_name, + amount=Decimal(realization.actual_net), + comment=comment, + now=now, + ) + action = "reverse" + business_event_type = "saving_reversed" + else: + realization.status = "rejected" + realization.dedupe_status = "excluded" + realization.rejected_by_id = actor_id + realization.rejected_by_name = actor_name + realization.rejected_at = now + realization.rejection_reason = comment + # 尚未进入财务确认口径的结果没有“已核实”事实，直接拒绝而不是伪造 reversed。 + opportunity.status = "rejected" + opportunity.closed_at = now + opportunity.version += 1 + opportunity.updated_at = now + result = realization + action = "reject" + business_event_type = "saving_result_rejected" + + evidence = self._reversal_evidence(result, payment_reversal_event, now=now) + self.core._add_evidence( + result, + [evidence], + actor_id=actor_id, + verified=evidence.metadata_json.get("evidence_level") == "external_cash", + now=now, + ) + realization.version += 1 + realization.updated_at = now + request_id = f"payment-reversal:{payment_reversal_event.id}:{realization.id}"[:120] + fingerprint = savings_payload_fingerprint( + action=action, + aggregate_type="realization", + aggregate_id=realization.id, + actor_id=actor_id, + payload={"payment_reversal_event_id": payment_reversal_event.id}, + ) + event = self.core._new_realization_event( + realization, + action=action, + request_id=request_id, + expected_version=prior_realization_version, + actor_id=actor_id, + actor_name=actor_name, + fingerprint=fingerprint, + payload_json={"payment_reversal_event_id": payment_reversal_event.id}, + before_json=before, + now=now, + causation_id=payment_reversal_event.id, + ) + response = self.core._mutation_response(result, opportunity, event, current_user) + event.response_json = response.model_dump(mode="json") + self.db.add(event) + self.core._add_opportunity_projection_event( + opportunity, + action="payment_reversed", + request_id=request_id, + actor_id=actor_id, + prior_version=prior_opportunity_version, + causation_id=event.id, + now=now, + ) + if result.id != realization.id: + self.core._add_realization_projection_event( + result, + request_id=request_id, + causation_id=event.id, + now=now, + ) + self.core._append_business_event( + opportunity, + result, + event_type=business_event_type, + actor_id=actor_id, + idempotency_key=request_id, + correlation_id=payment_reversal_event.correlation_id, + causation_id=payment_reversal_event.id, + payload={"source": "external_payment_reversal"}, + ) + return result + + @staticmethod + def _reversal_evidence( + realization: SavingsRealization, + event: BusinessEvent, + *, + now: datetime, + ) -> SavingsEvidenceCreate: + payload = dict(event.payload_json or {}) + connector_event_id = str(payload.get("connector_event_id") or "").strip() + classification = str( + payload.get("evidence_classification") or "external_cash" + ).strip() + return SavingsEvidenceCreate( + evidence_key=f"payment-reversal:{connector_event_id}:{realization.id}", + evidence_role="external_payment_reversal", + resource_type="financial_connector_event", + resource_id=connector_event_id, + source_system=str(payload.get("provider") or "financial-connector"), + external_event_id=str(payload.get("external_event_id") or "") or None, + content_hash=str(payload.get("content_hash") or ""), + occurred_at=now, + verification_status=( + "unavailable" if classification == "simulated_connector" else "unverified" + ), + metadata_json={ + "evidence_level": classification, + "verification_level": payload.get("verification_level"), + "origin_connector_event_id": payload.get("origin_connector_event_id"), + }, + ) diff --git a/server/src/app/services/savings_protocol.py b/server/src/app/services/savings_protocol.py new file mode 100644 index 0000000..ceb2c33 --- /dev/null +++ b/server/src/app/services/savings_protocol.py @@ -0,0 +1,129 @@ +from __future__ import annotations + +import hashlib +import json +import threading +from collections.abc import Iterator, Mapping +from contextlib import contextmanager +from decimal import Decimal +from typing import Any + +from sqlalchemy import text +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext + + +class SavingsConflictError(RuntimeError): + pass + + +class SavingsVersionConflictError(SavingsConflictError): + def __init__(self, current_version: int) -> None: + self.current_version = current_version + super().__init__(f"节省事实版本已更新，当前版本为 {current_version}。") + + +class SavingsIdempotencyConflictError(SavingsConflictError): + pass + + +class SavingsRequestProtocol: + """为账本动作提供 PostgreSQL advisory lock 和测试环境进程锁。""" + + _registry_guard = threading.Lock() + _locks: dict[str, tuple[threading.RLock, int]] = {} + + def __init__(self, db: Session) -> None: + self.db = db + + @contextmanager + def serialize( + self, + *, + tenant_id: str, + actor_id: str, + request_id: str, + ) -> Iterator[None]: + name = f"savings:{tenant_id}:{actor_id}:{request_id}" + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + self.db.execute( + text("SELECT pg_advisory_xact_lock(:lock_id)"), + {"lock_id": self._signed_lock_id(name)}, + ) + yield + return + with self._fallback_lock(name): + yield + + @classmethod + @contextmanager + def _fallback_lock(cls, name: str) -> Iterator[None]: + with cls._registry_guard: + lock, references = cls._locks.get(name, (threading.RLock(), 0)) + cls._locks[name] = (lock, references + 1) + try: + with lock: + yield + finally: + with cls._registry_guard: + current = cls._locks.get(name) + if current is None or current[0] is not lock: + pass + elif current[1] <= 1: + cls._locks.pop(name, None) + else: + cls._locks[name] = (lock, current[1] - 1) + + @staticmethod + def _signed_lock_id(value: str) -> int: + unsigned = int.from_bytes( + hashlib.sha256(value.encode("utf-8")).digest()[:8], + byteorder="big", + signed=False, + ) + return unsigned - (1 << 64) if unsigned >= (1 << 63) else unsigned + + +def savings_payload_fingerprint( + *, + action: str, + aggregate_type: str, + aggregate_id: str, + actor_id: str, + payload: Mapping[str, Any], +) -> str: + canonical = json.dumps( + { + "action": str(action or "").strip(), + "aggregate_type": str(aggregate_type or "").strip(), + "aggregate_id": str(aggregate_id or "").strip(), + "actor_id": str(actor_id or "").strip().casefold(), + "payload": _json_safe(dict(payload)), + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() + + +def savings_actor(current_user: CurrentUserContext) -> tuple[str, str]: + actor_id = str(current_user.employee_id or current_user.username or "").strip() + if not actor_id: + raise ValueError("当前用户缺少可审计的账号标识。") + actor_name = str(current_user.name or current_user.username or actor_id).strip() + return actor_id, actor_name + + +def _json_safe(value: Any) -> Any: + if isinstance(value, Decimal): + return f"{value:f}" + if hasattr(value, "isoformat"): + return value.isoformat() + if isinstance(value, Mapping): + return {str(key): _json_safe(item) for key, item in value.items()} + if isinstance(value, (list, tuple, set, frozenset)): + return [_json_safe(item) for item in value] + return value diff --git a/server/src/app/services/savings_query.py b/server/src/app/services/savings_query.py new file mode 100644 index 0000000..a33c0f6 --- /dev/null +++ b/server/src/app/services/savings_query.py @@ -0,0 +1,192 @@ +from __future__ import annotations + +import math +from datetime import UTC, datetime + +from sqlalchemy import Select, func, select +from sqlalchemy.orm import Session, selectinload + +from app.api.deps import CurrentUserContext +from app.models.savings import SavingsOpportunity +from app.schemas.savings import SavingsOpportunityListRead, SavingsOpportunityRead +from app.services.savings_access_policy import SavingsAccessPolicy +from app.services.savings_read_projection import SavingsReadProjection + + +class SavingsQueryService: + """Savings Ledger 的租户安全查询边界。""" + + def __init__(self, db: Session) -> None: + self.db = db + self.projection = SavingsReadProjection() + + def list_opportunities( + self, + current_user: CurrentUserContext, + *, + page: int = 1, + page_size: int = 20, + status: str | None = None, + source_type: str | None = None, + value_kind: str | None = None, + department_id: str | None = None, + project_code: str | None = None, + expense_type: str | None = None, + supplier_id: str | None = None, + city: str | None = None, + owner_id: str | None = None, + claim_id: str | None = None, + created_from: datetime | None = None, + created_to: datetime | None = None, + sort: str = "created_desc", + ) -> SavingsOpportunityListRead: + statement = select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == self._tenant(current_user) + ) + statement = self._apply_user_scope(statement, current_user) + statement = self._apply_filters( + statement, + status=status, + source_type=source_type, + value_kind=value_kind, + department_id=department_id, + project_code=project_code, + expense_type=expense_type, + supplier_id=supplier_id, + city=city, + owner_id=owner_id, + claim_id=claim_id, + created_from=created_from, + created_to=created_to, + ) + + count_statement = select(func.count()).select_from(statement.order_by(None).subquery()) + total = int(self.db.scalar(count_statement) or 0) + statement = ( + self._apply_sort(statement, sort).offset((page - 1) * page_size).limit(page_size) + ) + opportunities = list(self.db.scalars(statement).all()) + items = [ + self.projection.opportunity_read(item, current_user, include_details=False) + for item in opportunities + ] + return SavingsOpportunityListRead( + items=items, + total=total, + page=page, + page_size=page_size, + total_pages=math.ceil(total / page_size) if total else 0, + generated_at=datetime.now(UTC), + ) + + def get_opportunity( + self, + opportunity_id: str, + current_user: CurrentUserContext, + ) -> SavingsOpportunityRead | None: + statement = ( + select(SavingsOpportunity) + .options( + selectinload(SavingsOpportunity.baseline_snapshot), + selectinload(SavingsOpportunity.realizations), + selectinload(SavingsOpportunity.evidence_links), + selectinload(SavingsOpportunity.events), + ) + .where( + SavingsOpportunity.tenant_id == self._tenant(current_user), + SavingsOpportunity.id == opportunity_id, + ) + ) + opportunity = self.db.scalar(statement) + if opportunity is None: + return None + permission = SavingsAccessPolicy.opportunity_permission(opportunity, current_user) + if not permission.can_read: + return None + return self.projection.opportunity_read(opportunity, current_user) + + @classmethod + def _apply_user_scope( + cls, + statement: Select[tuple[SavingsOpportunity]], + current_user: CurrentUserContext, + ) -> Select[tuple[SavingsOpportunity]]: + return SavingsAccessPolicy.apply_opportunity_read_scope( + statement, + current_user, + include_owner=True, + ) + + @staticmethod + def _apply_filters( + statement: Select[tuple[SavingsOpportunity]], + *, + status: str | None, + source_type: str | None, + value_kind: str | None, + department_id: str | None, + project_code: str | None, + expense_type: str | None, + supplier_id: str | None, + city: str | None, + owner_id: str | None, + claim_id: str | None, + created_from: datetime | None, + created_to: datetime | None, + ) -> Select[tuple[SavingsOpportunity]]: + scalar_filters = ( + (SavingsOpportunity.status, status), + (SavingsOpportunity.source_type, source_type), + (SavingsOpportunity.value_kind, value_kind), + (SavingsOpportunity.owner_id, owner_id), + (SavingsOpportunity.claim_id, claim_id), + ) + for column, value in scalar_filters: + normalized = str(value or "").strip() + if normalized: + statement = statement.where(column == normalized) + + dimension_filters = ( + ("department_id", department_id), + ("project_code", project_code), + ("expense_type", expense_type), + ("supplier_id", supplier_id), + ("city", city), + ) + for key, value in dimension_filters: + normalized = str(value or "").strip() + if normalized: + statement = statement.where( + SavingsOpportunity.dimension_json[key].as_string() == normalized + ) + if created_from is not None: + statement = statement.where(SavingsOpportunity.created_at >= created_from) + if created_to is not None: + statement = statement.where(SavingsOpportunity.created_at <= created_to) + return statement + + @staticmethod + def _apply_sort( + statement: Select[tuple[SavingsOpportunity]], + sort: str, + ) -> Select[tuple[SavingsOpportunity]]: + sort_map = { + "created_asc": (SavingsOpportunity.created_at.asc(), SavingsOpportunity.id.asc()), + "due_asc": ( + SavingsOpportunity.due_at.asc().nullslast(), + SavingsOpportunity.created_at.desc(), + ), + "estimated_desc": ( + SavingsOpportunity.estimated_net.desc(), + SavingsOpportunity.created_at.desc(), + ), + "created_desc": ( + SavingsOpportunity.created_at.desc(), + SavingsOpportunity.id.desc(), + ), + } + return statement.order_by(*sort_map.get(sort, sort_map["created_desc"])) + + @staticmethod + def _tenant(current_user: CurrentUserContext) -> str: + return str(current_user.tenant_id or "default").strip() or "default" diff --git a/server/src/app/services/savings_read_projection.py b/server/src/app/services/savings_read_projection.py new file mode 100644 index 0000000..0d16003 --- /dev/null +++ b/server/src/app/services/savings_read_projection.py @@ -0,0 +1,124 @@ +from __future__ import annotations + +from collections.abc import Iterable +from typing import Any + +from app.api.deps import CurrentUserContext +from app.models.savings import ( + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) +from app.schemas.savings import ( + SavingsEventRead, + SavingsEvidenceLinkRead, + SavingsOpportunityRead, + SavingsRealizationRead, +) +from app.services.savings_access_policy import SavingsAccessPolicy + + +class SavingsReadProjection: + """把账本实体投影为字段白名单 DTO，并在最后一步附加可用动作。""" + + def opportunity_read( + self, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + *, + include_details: bool = True, + ) -> SavingsOpportunityRead: + permission = SavingsAccessPolicy.opportunity_permission(opportunity, current_user) + if not permission.can_read: + raise PermissionError(permission.read_only_reason or "当前用户无权读取该节省机会。") + + data = self._column_values(opportunity, SavingsOpportunityRead) + data.update( + available_actions=list(permission.available_actions), + read_only_reason=permission.read_only_reason, + ) + if include_details: + data.update( + baseline=getattr(opportunity, "baseline_snapshot", None), + realizations=self.realization_list_read( + list(getattr(opportunity, "realizations", ()) or ()), + opportunity, + current_user, + ), + evidence=self.evidence_list_read( + list(getattr(opportunity, "evidence_links", ()) or ()) + ), + events=self.event_list_read(list(getattr(opportunity, "events", ()) or ())), + ) + else: + data.update(baseline=None, realizations=[], evidence=[], events=[]) + return SavingsOpportunityRead.model_validate(data) + + def opportunity_snapshot( + self, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + ) -> SavingsOpportunityRead: + return self.opportunity_read(opportunity, current_user, include_details=False) + + def realization_read( + self, + realization: SavingsRealization, + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + ) -> SavingsRealizationRead: + permission = SavingsAccessPolicy.realization_permission( + realization, + opportunity, + current_user, + ) + if not permission.can_read: + raise PermissionError(permission.read_only_reason or "当前用户无权读取该实际结果。") + data = self._column_values(realization, SavingsRealizationRead) + data.update( + available_actions=list(permission.available_actions), + read_only_reason=permission.read_only_reason, + ) + return SavingsRealizationRead.model_validate(data) + + def realization_list_read( + self, + realizations: Iterable[SavingsRealization], + opportunity: SavingsOpportunity, + current_user: CurrentUserContext, + ) -> list[SavingsRealizationRead]: + return [ + self.realization_read(realization, opportunity, current_user) + for realization in realizations + ] + + @staticmethod + def evidence_list_read( + evidence: Iterable[SavingsEvidenceLink], + ) -> list[SavingsEvidenceLinkRead]: + return [SavingsEvidenceLinkRead.model_validate(item) for item in evidence] + + @staticmethod + def event_list_read(events: Iterable[SavingsEvent]) -> list[SavingsEventRead]: + return [SavingsEventRead.model_validate(item) for item in events] + + @staticmethod + def event_read(event: SavingsEvent) -> SavingsEventRead: + return SavingsEventRead.model_validate(event) + + @staticmethod + def _column_values(instance: Any, schema_type: type[Any]) -> dict[str, Any]: + excluded = { + "available_actions", + "read_only_reason", + "baseline", + "realizations", + "evidence", + "events", + } + return { + field_name: getattr(instance, field_name) + for field_name in schema_type.model_fields + if field_name not in excluded and hasattr(instance, field_name) + } diff --git a/server/src/app/services/savings_realization.py b/server/src/app/services/savings_realization.py new file mode 100644 index 0000000..9094338 --- /dev/null +++ b/server/src/app/services/savings_realization.py @@ -0,0 +1,643 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, datetime +from decimal import Decimal + +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext +from app.models.expense_case import BusinessEvent +from app.models.financial_record import ExpenseClaim +from app.models.savings import ( + SavingsOpportunity, + SavingsRealization, +) +from app.schemas.savings import ( + SavingsEvidenceCreate, + SavingsRealizationActionCreate, + SavingsRealizationCreate, +) +from app.services.expense_cases import ExpenseCaseService +from app.services.savings_access_policy import SavingsAccessPolicy, SavingsPermissionError +from app.services.savings_protocol import ( + SavingsIdempotencyConflictError, + SavingsRequestProtocol, + SavingsVersionConflictError, + savings_actor, + savings_payload_fingerprint, +) +from app.services.savings_read_projection import SavingsReadProjection +from app.services.savings_realization_support import ( + SavingsRealizationMutation, + SavingsRealizationSupport, +) + + +class SavingsRealizationError(ValueError): + pass + + +class SavingsRealizationService(SavingsRealizationSupport): + """记录实际结果，并执行独立财务确认、拒绝和追加冲回。""" + + def __init__(self, db: Session) -> None: + self.db = db + self.protocol = SavingsRequestProtocol(db) + self.projection = SavingsReadProjection() + self.expense_cases = ExpenseCaseService(db) + + def record( + self, + opportunity_id: str, + payload: SavingsRealizationCreate, + current_user: CurrentUserContext, + ) -> SavingsRealizationMutation: + tenant_id = self._tenant(current_user) + actor_id, actor_name = savings_actor(current_user) + fingerprint = savings_payload_fingerprint( + action="record_realization", + aggregate_type="opportunity", + aggregate_id=opportunity_id, + actor_id=actor_id, + payload=payload.model_dump(mode="json"), + ) + try: + with self.protocol.serialize( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + ): + replay = self._find_replay( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + fingerprint=fingerprint, + ) + if replay is not None: + return replay + opportunity = self._lock_opportunity(opportunity_id, tenant_id=tenant_id) + if opportunity is None: + raise LookupError("节省机会不存在。") + SavingsAccessPolicy.require_opportunity_action( + opportunity, + current_user, + "record_realization", + ) + if opportunity.version != payload.expected_version: + raise SavingsVersionConflictError(opportunity.version) + if opportunity.status != "in_progress": + raise SavingsRealizationError("只有执行中的机会可以记录实际结果。") + if payload.currency != opportunity.reporting_currency: + raise SavingsRealizationError( + "当前未提供可锁定汇率，实际结果只能使用机会报告币种。" + ) + + now = datetime.now(UTC) + realization = self._new_actual_realization( + opportunity, + actor_id=actor_id, + actor_name=actor_name, + actual_gross=payload.actual_gross, + incremental_cost=payload.incremental_cost, + currency=payload.currency, + realized_at=payload.realized_at, + attribution_method=payload.attribution_method, + attribution_ratio=payload.attribution_ratio, + evidence_level=payload.evidence_level, + realization_key=f"manual:{opportunity.id}:{payload.request_id}", + now=now, + ) + self.db.add(realization) + self.db.flush() + self._add_evidence( + realization, + payload.evidence, + actor_id=actor_id, + verified=False, + now=now, + ) + prior_opportunity_version = opportunity.version + self._mark_opportunity_realized(opportunity, realized_at=payload.realized_at) + main_event = self._new_realization_event( + realization, + action="record_realization", + request_id=payload.request_id, + expected_version=0, + actor_id=actor_id, + actor_name=actor_name, + fingerprint=fingerprint, + payload_json=payload.model_dump(mode="json"), + before_json={}, + now=now, + ) + response = self._mutation_response( + realization, + opportunity, + main_event, + current_user, + ) + main_event.response_json = response.model_dump(mode="json") + self.db.add(main_event) + self._add_opportunity_projection_event( + opportunity, + action="realization_recorded", + request_id=payload.request_id, + actor_id=actor_id, + prior_version=prior_opportunity_version, + causation_id=main_event.id, + now=now, + ) + self._append_business_event( + opportunity, + realization, + event_type="saving_action_completed", + actor_id=current_user.username, + idempotency_key=payload.request_id, + correlation_id=main_event.correlation_id, + payload={"evidence_level": payload.evidence_level}, + ) + self.db.commit() + return SavingsRealizationMutation(response=response) + except ( + LookupError, + SavingsPermissionError, + SavingsRealizationError, + SavingsVersionConflictError, + SavingsIdempotencyConflictError, + ): + self.db.rollback() + raise + except IntegrityError as error: + self.db.rollback() + replay = self._find_replay( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + fingerprint=fingerprint, + ) + if replay is not None: + return replay + raise SavingsRealizationError("实际结果已被并发记录，请刷新后重试。") from error + + def execute_action( + self, + realization_id: str, + payload: SavingsRealizationActionCreate, + current_user: CurrentUserContext, + ) -> SavingsRealizationMutation: + tenant_id = self._tenant(current_user) + actor_id, actor_name = savings_actor(current_user) + fingerprint = savings_payload_fingerprint( + action=payload.action, + aggregate_type="realization", + aggregate_id=realization_id, + actor_id=actor_id, + payload=payload.model_dump(mode="json"), + ) + try: + with self.protocol.serialize( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + ): + replay = self._find_replay( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + fingerprint=fingerprint, + ) + if replay is not None: + return replay + opportunity, realization = self._lock_realization_graph( + realization_id, + tenant_id=tenant_id, + ) + if opportunity is None or realization is None: + raise LookupError("实际节省结果不存在。") + SavingsAccessPolicy.require_realization_action( + realization, + opportunity, + current_user, + payload.action, + ) + if realization.version != payload.expected_version: + raise SavingsVersionConflictError(realization.version) + + now = datetime.now(UTC) + before = self._realization_state(realization) + prior_opportunity_version = opportunity.version + if payload.action == "confirm": + self._add_evidence( + realization, + payload.evidence, + actor_id=actor_id, + verified=True, + now=now, + ) + self._confirm( + opportunity, + realization, + actor_id=actor_id, + actor_name=actor_name, + comment=payload.comment, + now=now, + ) + result = realization + business_event_type = "saving_confirmed" + elif payload.action == "reject": + self._reject( + opportunity, + realization, + actor_id=actor_id, + actor_name=actor_name, + comment=payload.comment, + now=now, + ) + result = realization + business_event_type = "saving_result_rejected" + elif payload.action == "reverse": + result = self._reverse( + opportunity, + realization, + actor_id=actor_id, + actor_name=actor_name, + amount=payload.reversal_amount, + comment=payload.comment, + now=now, + ) + business_event_type = "saving_reversed" + else: # pragma: no cover - Pydantic 已限制枚举 + raise SavingsRealizationError("未知的实际结果动作。") + + if payload.action != "confirm": + self._add_evidence( + result, + payload.evidence, + actor_id=actor_id, + verified=False, + now=now, + ) + realization.version += 1 + realization.updated_at = now + fingerprint_payload = payload.model_dump(mode="json") + main_event = self._new_realization_event( + realization, + action=payload.action, + request_id=payload.request_id, + expected_version=payload.expected_version, + actor_id=actor_id, + actor_name=actor_name, + fingerprint=fingerprint, + payload_json=fingerprint_payload, + before_json=before, + now=now, + ) + response = self._mutation_response( + result, + opportunity, + main_event, + current_user, + ) + main_event.response_json = response.model_dump(mode="json") + self.db.add(main_event) + self._add_opportunity_projection_event( + opportunity, + action=f"realization_{payload.action}", + request_id=payload.request_id, + actor_id=actor_id, + prior_version=prior_opportunity_version, + causation_id=main_event.id, + now=now, + ) + if result.id != realization.id: + self._add_realization_projection_event( + result, + request_id=payload.request_id, + causation_id=main_event.id, + now=now, + ) + self._append_business_event( + opportunity, + result, + event_type=business_event_type, + actor_id=current_user.username, + idempotency_key=payload.request_id, + correlation_id=main_event.correlation_id, + payload={"comment": payload.comment, "action": payload.action}, + ) + self.db.commit() + return SavingsRealizationMutation(response=response) + except ( + LookupError, + SavingsPermissionError, + SavingsRealizationError, + SavingsVersionConflictError, + SavingsIdempotencyConflictError, + ): + self.db.rollback() + raise + except IntegrityError as error: + self.db.rollback() + replay = self._find_replay( + tenant_id=tenant_id, + actor_id=actor_id, + request_id=payload.request_id, + fingerprint=fingerprint, + ) + if replay is not None: + return replay + raise SavingsRealizationError("实际结果已被并发处理，请刷新后重试。") from error + + def realize_paid_claim( + self, + claim: ExpenseClaim, + payment_event: BusinessEvent, + current_user: CurrentUserContext, + ) -> list[SavingsRealization]: + """付款事务内生成待确认 actual；调用方负责最终 commit。""" + tenant_id = self._tenant(current_user) + if payment_event.tenant_id != tenant_id: + raise PermissionError("付款事件与当前租户不一致。") + opportunity_statement = select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == tenant_id, + SavingsOpportunity.claim_id == claim.id, + SavingsOpportunity.status == "in_progress", + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + opportunity_statement = opportunity_statement.with_for_update() + opportunities = list(self.db.scalars(opportunity_statement).all()) + created: list[SavingsRealization] = [] + actor_id, actor_name = savings_actor(current_user) + for opportunity in opportunities: + key = f"payment:{payment_event.id}:{opportunity.id}" + existing = self.db.scalar( + select(SavingsRealization).where( + SavingsRealization.tenant_id == tenant_id, + SavingsRealization.realization_key == key, + ) + ) + if existing is not None: + continue + if opportunity.currency != opportunity.reporting_currency: + continue + realized_at = payment_event.occurred_at or datetime.now(UTC) + external_evidence = dict( + (payment_event.payload_json or {}).get("external_payment_evidence") or {} + ) + evidence_classification = str( + external_evidence.get("evidence_classification") or "business_state" + ).strip() + realization = self._new_actual_realization( + opportunity, + actor_id=actor_id, + actor_name=actor_name, + actual_gross=opportunity.estimated_gross, + incremental_cost=opportunity.estimated_cost, + currency=opportunity.currency, + realized_at=realized_at, + attribution_method=opportunity.attribution_method, + attribution_ratio=Decimal("1.000000"), + evidence_level=evidence_classification, + realization_key=key, + now=realized_at, + business_event_id=payment_event.id, + ) + self.db.add(realization) + self.db.flush() + connector_event_id = str( + external_evidence.get("connector_event_id") or "" + ).strip() + if connector_event_id: + evidence = SavingsEvidenceCreate( + evidence_key=f"connector:{connector_event_id}:{realization.id}", + evidence_role="external_payment_settlement", + resource_type="financial_connector_event", + resource_id=connector_event_id, + source_system=str(external_evidence.get("provider") or "financial-connector"), + external_event_id=str(external_evidence.get("external_event_id") or "") or None, + content_hash=str(external_evidence.get("content_hash") or ""), + occurred_at=realized_at, + verification_status=( + "unavailable" + if evidence_classification == "simulated_connector" + else "unverified" + ), + metadata_json={ + "evidence_level": evidence_classification, + "verification_level": external_evidence.get("verification_level"), + "external_reference_tail": external_evidence.get( + "external_reference_tail" + ), + }, + ) + else: + evidence = SavingsEvidenceCreate( + evidence_key=f"payment:{payment_event.id}:{realization.id}", + evidence_role="payment_business_state", + resource_type="business_event", + resource_id=payment_event.id, + source_system="x-financial", + external_event_id=payment_event.id, + content_hash=self._event_content_hash(payment_event), + occurred_at=realized_at, + verification_status="unverified", + metadata_json={"evidence_level": "business_state"}, + ) + self._add_evidence( + realization, + [evidence], + actor_id=actor_id, + verified=False, + now=realized_at, + ) + prior_opportunity_version = opportunity.version + self._mark_opportunity_realized(opportunity, realized_at=realized_at) + request_id = key[:120] + fingerprint = savings_payload_fingerprint( + action="record_realization", + aggregate_type="realization", + aggregate_id=realization.id, + actor_id=actor_id, + payload={"payment_event_id": payment_event.id}, + ) + event = self._new_realization_event( + realization, + action="record_realization", + request_id=request_id, + expected_version=0, + actor_id=actor_id, + actor_name=actor_name, + fingerprint=fingerprint, + payload_json={"payment_event_id": payment_event.id}, + before_json={}, + now=realized_at, + causation_id=payment_event.id, + ) + response = self._mutation_response( + realization, + opportunity, + event, + current_user, + ) + event.response_json = response.model_dump(mode="json") + self.db.add(event) + self._add_opportunity_projection_event( + opportunity, + action="payment_realized", + request_id=request_id, + actor_id=actor_id, + prior_version=prior_opportunity_version, + causation_id=event.id, + now=realized_at, + ) + self._append_business_event( + opportunity, + realization, + event_type="saving_action_completed", + actor_id=current_user.username, + idempotency_key=request_id, + correlation_id=event.correlation_id, + causation_id=payment_event.id, + payload={"evidence_level": "business_state"}, + ) + created.append(realization) + self.db.flush() + return created + + def _confirm( + self, + opportunity: SavingsOpportunity, + realization: SavingsRealization, + *, + actor_id: str, + actor_name: str, + comment: str, + now: datetime, + ) -> None: + if self._verify_realization_evidence( + realization, + actor_id=actor_id, + now=now, + ) == 0: + raise SavingsRealizationError("实际节省结果缺少可复核证据，不能进入财务确认口径。") + duplicate = self.db.scalar( + select(SavingsRealization.id).where( + SavingsRealization.tenant_id == realization.tenant_id, + SavingsRealization.benefit_key == realization.benefit_key, + SavingsRealization.realization_type == "actual", + SavingsRealization.dedupe_status == "canonical", + SavingsRealization.id != realization.id, + ) + ) + if duplicate is not None: + raise SavingsRealizationError("同一经济收益已经存在 canonical 实际结果。") + realization.status = "finance_confirmed" + realization.dedupe_status = "canonical" + realization.finance_confirmer_id = actor_id + realization.finance_confirmer_name = actor_name + realization.confirmed_at = now + realization.confirmation_note = comment + opportunity.status = "verified" + opportunity.verified_at = now + opportunity.closed_at = now + opportunity.version += 1 + opportunity.updated_at = now + + @staticmethod + def _reject( + opportunity: SavingsOpportunity, + realization: SavingsRealization, + *, + actor_id: str, + actor_name: str, + comment: str, + now: datetime, + ) -> None: + realization.status = "rejected" + realization.dedupe_status = "excluded" + realization.rejected_by_id = actor_id + realization.rejected_by_name = actor_name + realization.rejected_at = now + realization.rejection_reason = comment + opportunity.status = "rejected" + opportunity.closed_at = now + opportunity.version += 1 + opportunity.updated_at = now + + def _reverse( + self, + opportunity: SavingsOpportunity, + realization: SavingsRealization, + *, + actor_id: str, + actor_name: str, + amount: Decimal | None, + comment: str, + now: datetime, + ) -> SavingsRealization: + normalized = Decimal(amount or Decimal("0.00")).quantize(Decimal("0.0001")) + if normalized != Decimal(realization.actual_net).quantize(Decimal("0.0001")): + raise SavingsRealizationError("首期仅支持对已确认净收益执行全额冲回。") + existing_reversal = self.db.scalar( + select(SavingsRealization.id).where( + SavingsRealization.tenant_id == realization.tenant_id, + SavingsRealization.reversal_of_realization_id == realization.id, + ) + ) + if existing_reversal is not None: + raise SavingsRealizationError("该已确认结果已经完成冲回，不能重复冲回。") + reversal = SavingsRealization( + id=str(uuid.uuid4()), + tenant_id=realization.tenant_id, + realization_key=f"reversal:{realization.id}:{uuid.uuid4().hex[:12]}", + opportunity_id=opportunity.id, + expense_case_id=opportunity.expense_case_id, + claim_id=opportunity.claim_id, + claim_item_id=opportunity.claim_item_id, + realization_type="reversal", + reversal_of_realization_id=realization.id, + realized_at=now, + recorded_by_id=actor_id, + recorded_by_name=actor_name, + actual_gross=-normalized, + incremental_cost=Decimal("0.0000"), + actual_net=-normalized, + original_currency=realization.original_currency, + reporting_amount=-Decimal(realization.reporting_amount), + reporting_currency=realization.reporting_currency, + fx_rate=realization.fx_rate, + fx_source=realization.fx_source, + fx_date=realization.fx_date, + fx_version=realization.fx_version, + attribution_method="full_reversal", + attribution_ratio=realization.attribution_ratio, + benefit_key=realization.benefit_key, + dedupe_status="canonical", + status="finance_confirmed", + finance_confirmer_id=actor_id, + finance_confirmer_name=actor_name, + confirmed_at=now, + confirmation_note=comment, + baseline_snapshot_json=dict(realization.baseline_snapshot_json or {}), + final_snapshot_json={"reversal_reason": comment, "reversal_of": realization.id}, + evidence_json=[], + version=1, + created_at=now, + updated_at=now, + ) + self.db.add(reversal) + self.db.flush() + # 原正向确认事实保持 finance_confirmed；冲回由独立负向事实表达。 + realization.reversed_by_id = actor_id + realization.reversed_by_name = actor_name + realization.reversed_at = now + realization.reversal_reason = comment + opportunity.status = "reversed" + opportunity.closed_at = now + opportunity.version += 1 + opportunity.updated_at = now + return reversal diff --git a/server/src/app/services/savings_realization_support.py b/server/src/app/services/savings_realization_support.py new file mode 100644 index 0000000..66c8b19 --- /dev/null +++ b/server/src/app/services/savings_realization_support.py @@ -0,0 +1,459 @@ +from __future__ import annotations + +import uuid +from dataclasses import dataclass +from datetime import datetime +from decimal import Decimal +from typing import Any + +from sqlalchemy import select + +from app.api.deps import CurrentUserContext +from app.models.expense_case import BusinessEvent, ExpenseCase +from app.models.savings import ( + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) +from app.schemas.savings import ( + SavingsEvidenceCreate, + SavingsRealizationMutationRead, +) +from app.services.savings_protocol import ( + SavingsIdempotencyConflictError, + savings_payload_fingerprint, +) + + +@dataclass(slots=True) +class SavingsRealizationMutation: + response: SavingsRealizationMutationRead + + +class SavingsRealizationSupport: + """实际结果服务的锁、证据、事件和响应构建辅助职责。""" + + @staticmethod + def _new_actual_realization( + opportunity: SavingsOpportunity, + *, + actor_id: str, + actor_name: str, + actual_gross: Decimal, + incremental_cost: Decimal, + currency: str, + realized_at: datetime, + attribution_method: str, + attribution_ratio: Decimal, + evidence_level: str, + realization_key: str, + now: datetime, + business_event_id: str | None = None, + ) -> SavingsRealization: + gross = Decimal(actual_gross).quantize(Decimal("0.0001")) + cost = Decimal(incremental_cost).quantize(Decimal("0.0001")) + net = gross - cost + return SavingsRealization( + id=str(uuid.uuid4()), + tenant_id=opportunity.tenant_id, + realization_key=realization_key[:180], + opportunity_id=opportunity.id, + expense_case_id=opportunity.expense_case_id, + claim_id=opportunity.claim_id, + claim_item_id=opportunity.claim_item_id, + business_event_id=business_event_id, + realization_type="actual", + realized_at=realized_at, + recorded_by_id=actor_id, + recorded_by_name=actor_name, + actual_gross=gross, + incremental_cost=cost, + actual_net=net, + original_currency=currency, + reporting_amount=net, + reporting_currency=opportunity.reporting_currency, + fx_rate=Decimal("1.00000000"), + fx_source="same_currency", + fx_date=realized_at.date(), + fx_version="identity-v1", + attribution_method=attribution_method[:60], + attribution_ratio=attribution_ratio, + benefit_key=opportunity.benefit_key, + dedupe_status="pending_review", + status="pending_confirmation", + baseline_snapshot_json=dict(opportunity.baseline_snapshot_json or {}), + final_snapshot_json={ + "evidence_level": evidence_level, + "estimated_net": str(opportunity.estimated_net), + }, + evidence_json=[], + version=1, + created_at=now, + updated_at=now, + ) + + def _lock_realization_graph( + self, + realization_id: str, + *, + tenant_id: str, + ) -> tuple[SavingsOpportunity | None, SavingsRealization | None]: + opportunity_id = self.db.scalar( + select(SavingsRealization.opportunity_id).where( + SavingsRealization.tenant_id == tenant_id, + SavingsRealization.id == realization_id, + ) + ) + if opportunity_id is None: + return None, None + opportunity = self._lock_opportunity(opportunity_id, tenant_id=tenant_id) + statement = select(SavingsRealization).where( + SavingsRealization.tenant_id == tenant_id, + SavingsRealization.id == realization_id, + SavingsRealization.opportunity_id == opportunity_id, + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + realization = self.db.scalar(statement.execution_options(populate_existing=True)) + return opportunity, realization + + def _lock_opportunity( + self, + opportunity_id: str, + *, + tenant_id: str, + ) -> SavingsOpportunity | None: + statement = select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == tenant_id, + SavingsOpportunity.id == opportunity_id, + ) + bind = self.db.get_bind() + if bind is not None and bind.dialect.name == "postgresql": + statement = statement.with_for_update() + return self.db.scalar(statement.execution_options(populate_existing=True)) + + @staticmethod + def _mark_opportunity_realized( + opportunity: SavingsOpportunity, + *, + realized_at: datetime, + ) -> None: + opportunity.status = "realized" + opportunity.realized_at = realized_at + opportunity.version += 1 + opportunity.updated_at = realized_at + + def _new_realization_event( + self, + realization: SavingsRealization, + *, + action: str, + request_id: str, + expected_version: int, + actor_id: str, + actor_name: str, + fingerprint: str, + payload_json: dict[str, Any], + before_json: dict[str, Any], + now: datetime, + causation_id: str | None = None, + ) -> SavingsEvent: + return SavingsEvent( + id=str(uuid.uuid4()), + tenant_id=realization.tenant_id, + aggregate_type="realization", + aggregate_id=realization.id, + realization_id=realization.id, + action=action, + actor_id=actor_id, + actor_name=actor_name, + actor_type="user", + request_id=request_id[:120], + expected_version=expected_version, + result_version=realization.version, + payload_fingerprint=fingerprint, + payload_json=payload_json, + before_json=before_json, + after_json=self._realization_state(realization), + response_json={}, + correlation_id=str(uuid.uuid4()), + causation_id=causation_id, + occurred_at=now, + ) + + def _add_opportunity_projection_event( + self, + opportunity: SavingsOpportunity, + *, + action: str, + request_id: str, + actor_id: str, + prior_version: int, + causation_id: str, + now: datetime, + ) -> None: + fingerprint = savings_payload_fingerprint( + action=action, + aggregate_type="opportunity", + aggregate_id=opportunity.id, + actor_id="savings-ledger", + payload={"causation_id": causation_id, "request_id": request_id}, + ) + event = SavingsEvent( + id=str(uuid.uuid4()), + tenant_id=opportunity.tenant_id, + aggregate_type="opportunity", + aggregate_id=opportunity.id, + opportunity_id=opportunity.id, + action=action, + actor_id="savings-ledger", + actor_name="Savings Ledger", + actor_type="service", + request_id=f"projection:{opportunity.id}:{fingerprint[-64:]}", + expected_version=prior_version, + result_version=opportunity.version, + payload_fingerprint=fingerprint, + payload_json={"source_actor_id": actor_id}, + before_json={"version": prior_version}, + after_json={"status": opportunity.status, "version": opportunity.version}, + response_json={"status": opportunity.status, "version": opportunity.version}, + correlation_id=str(uuid.uuid4()), + causation_id=causation_id, + occurred_at=now, + ) + self.db.add(event) + + def _add_realization_projection_event( + self, + realization: SavingsRealization, + *, + request_id: str, + causation_id: str, + now: datetime, + ) -> None: + fingerprint = savings_payload_fingerprint( + action="reversal_created", + aggregate_type="realization", + aggregate_id=realization.id, + actor_id="savings-ledger", + payload={"causation_id": causation_id, "request_id": request_id}, + ) + event = SavingsEvent( + id=str(uuid.uuid4()), + tenant_id=realization.tenant_id, + aggregate_type="realization", + aggregate_id=realization.id, + realization_id=realization.id, + action="reversal_created", + actor_id="savings-ledger", + actor_name="Savings Ledger", + actor_type="service", + request_id=f"reversal:{realization.id}:{fingerprint[-64:]}", + expected_version=0, + result_version=1, + payload_fingerprint=fingerprint, + payload_json={"causation_id": causation_id}, + before_json={}, + after_json=self._realization_state(realization), + response_json=self._realization_state(realization), + correlation_id=str(uuid.uuid4()), + causation_id=causation_id, + occurred_at=now, + ) + self.db.add(event) + + def _add_evidence( + self, + realization: SavingsRealization, + evidence: list[SavingsEvidenceCreate], + *, + actor_id: str, + verified: bool, + now: datetime, + ) -> None: + public_evidence = list(realization.evidence_json or []) + for item in evidence: + verification_status = ( + "verified" + if verified + else ("unavailable" if item.verification_status == "unavailable" else "unverified") + ) + self.db.add( + SavingsEvidenceLink( + id=str(uuid.uuid4()), + tenant_id=realization.tenant_id, + evidence_key=item.evidence_key, + entity_type="realization", + entity_id=realization.id, + realization_id=realization.id, + evidence_role=item.evidence_role, + resource_type=item.resource_type, + resource_id=item.resource_id, + source_system=item.source_system, + external_event_id=item.external_event_id, + content_hash=item.content_hash, + occurred_at=item.occurred_at, + collected_at=now, + verification_status=verification_status, + verified_by=actor_id if verified else None, + verified_at=now if verified else None, + metadata_json=dict(item.metadata_json or {}), + created_at=now, + ) + ) + public_evidence.append( + { + "evidence_key": item.evidence_key, + "role": item.evidence_role, + "resource_type": item.resource_type, + "verification_status": verification_status, + } + ) + realization.evidence_json = public_evidence + + def _verify_realization_evidence( + self, + realization: SavingsRealization, + *, + actor_id: str, + now: datetime, + ) -> int: + """财务确认时固化证据复核结果，避免“无证据可核验”进入 CFO 口径。""" + links = list( + self.db.scalars( + select(SavingsEvidenceLink).where( + SavingsEvidenceLink.tenant_id == realization.tenant_id, + SavingsEvidenceLink.entity_type == "realization", + SavingsEvidenceLink.entity_id == realization.id, + ) + ).all() + ) + if not links: + return 0 + verified_keys: set[str] = set() + for link in links: + link.verification_status = "verified" + link.verified_by = actor_id + link.verified_at = now + verified_keys.add(link.evidence_key) + realization.evidence_json = [ + { + **item, + "verification_status": "verified", + } + if item.get("evidence_key") in verified_keys + else item + for item in list(realization.evidence_json or []) + ] + return len(links) + + def _append_business_event( + self, + opportunity: SavingsOpportunity, + realization: SavingsRealization, + *, + event_type: str, + actor_id: str, + idempotency_key: str, + correlation_id: str | None, + payload: dict[str, Any], + causation_id: str | None = None, + ) -> None: + expense_case = self.db.get(ExpenseCase, opportunity.expense_case_id) + if expense_case is None or expense_case.tenant_id != opportunity.tenant_id: + raise RuntimeError("实际结果关联的费用事件不存在或租户不一致。") + self.expense_cases.link_resource( + expense_case, + resource_type="savings_realization", + resource_id=realization.id, + relation_type="savings", + tenant_id=opportunity.tenant_id, + ) + self.expense_cases.record_resource_event( + expense_case, + aggregate_type="savings_realization", + aggregate_id=realization.id, + event_type=event_type, + actor_id=actor_id, + idempotency_key=idempotency_key[:120], + tenant_id=opportunity.tenant_id, + correlation_id=correlation_id, + causation_id=causation_id, + payload={ + **payload, + "opportunity_id": opportunity.id, + "realization_id": realization.id, + "status": realization.status, + }, + ) + + def _mutation_response( + self, + realization: SavingsRealization, + opportunity: SavingsOpportunity, + event: SavingsEvent, + current_user: CurrentUserContext, + ) -> SavingsRealizationMutationRead: + return SavingsRealizationMutationRead( + realization=self.projection.realization_read( + realization, + opportunity, + current_user, + ), + opportunity=self.projection.opportunity_snapshot(opportunity, current_user), + event=self.projection.event_read(event), + replayed=False, + ) + + def _find_replay( + self, + *, + tenant_id: str, + actor_id: str, + request_id: str, + fingerprint: str, + ) -> SavingsRealizationMutation | None: + event = self.db.scalar( + select(SavingsEvent).where( + SavingsEvent.tenant_id == tenant_id, + SavingsEvent.actor_id == actor_id, + SavingsEvent.request_id == request_id, + ) + ) + if event is None: + return None + if event.payload_fingerprint != fingerprint: + raise SavingsIdempotencyConflictError("request_id 已被不同的实际节省动作使用。") + response = SavingsRealizationMutationRead.model_validate(event.response_json) + return SavingsRealizationMutation(response=response.model_copy(update={"replayed": True})) + + @staticmethod + def _realization_state(realization: SavingsRealization) -> dict[str, Any]: + return { + "id": realization.id, + "status": realization.status, + "dedupe_status": realization.dedupe_status, + "actual_net": str(realization.actual_net), + "version": realization.version, + "confirmed_at": realization.confirmed_at.isoformat() + if realization.confirmed_at + else None, + "rejected_at": realization.rejected_at.isoformat() if realization.rejected_at else None, + "reversed_at": realization.reversed_at.isoformat() if realization.reversed_at else None, + } + + @staticmethod + def _event_content_hash(event: BusinessEvent) -> str: + return savings_payload_fingerprint( + action=event.event_type, + aggregate_type=event.aggregate_type, + aggregate_id=event.aggregate_id, + actor_id=event.actor_id, + payload=event.payload_json or {}, + ) + + @staticmethod + def _tenant(current_user: CurrentUserContext) -> str: + return str(current_user.tenant_id or "default").strip() or "default" diff --git a/server/src/app/services/tenant_registry.py b/server/src/app/services/tenant_registry.py new file mode 100644 index 0000000..b006cfd --- /dev/null +++ b/server/src/app/services/tenant_registry.py @@ -0,0 +1,82 @@ +from __future__ import annotations + +import re + +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.models.employee import Employee +from app.models.tenant import Tenant, TenantMembership + +DEFAULT_TENANT_ID = "default" +PLATFORM_TENANT_ID = "platform" +_TENANT_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,63}$") + + +def required_tenant_id(value: object) -> str: + tenant_id = str(value or "").strip() + if not _TENANT_PATTERN.fullmatch(tenant_id): + raise ValueError("tenant_id 缺失或格式不合法。") + return tenant_id + + +class TenantRegistryService: + def __init__(self, db: Session) -> None: + self.db = db + + def ensure_builtin(self) -> None: + builtins = ( + (DEFAULT_TENANT_ID, DEFAULT_TENANT_ID, "默认企业"), + (PLATFORM_TENANT_ID, PLATFORM_TENANT_ID, "平台管理域"), + ) + for tenant_id, tenant_code, name in builtins: + if self.db.get(Tenant, tenant_id) is None: + self.db.add( + Tenant( + tenant_id=tenant_id, + tenant_code=tenant_code, + name=name, + status="active", + ) + ) + self.db.flush() + + def require_active(self, tenant_id: object) -> Tenant: + normalized = required_tenant_id(tenant_id) + tenant = self.db.scalar( + select(Tenant).where( + Tenant.tenant_id == normalized, + Tenant.status == "active", + ) + ) + if tenant is None: + raise LookupError("租户不存在或当前不可用。") + return tenant + + def active_tenant_ids(self, *, include_platform: bool = False) -> tuple[str, ...]: + statement = select(Tenant.tenant_id).where(Tenant.status == "active") + if not include_platform: + statement = statement.where(Tenant.tenant_id != PLATFORM_TENANT_ID) + return tuple(self.db.scalars(statement.order_by(Tenant.tenant_id)).all()) + + def ensure_employee_membership(self, employee: Employee) -> TenantMembership: + tenant_id = required_tenant_id(employee.tenant_id) + self.require_active(tenant_id) + membership = self.db.scalar( + select(TenantMembership).where( + TenantMembership.tenant_id == tenant_id, + TenantMembership.employee_id == employee.id, + ) + ) + if membership is None: + membership = TenantMembership( + tenant_id=tenant_id, + employee_id=employee.id, + status="active", + is_primary=True, + ) + self.db.add(membership) + elif membership.status != "active": + membership.status = "active" + self.db.flush() + return membership diff --git a/server/src/app/services/travel_reimbursement_calculator.py b/server/src/app/services/travel_reimbursement_calculator.py index edeca3b..bb2f0f7 100644 --- a/server/src/app/services/travel_reimbursement_calculator.py +++ b/server/src/app/services/travel_reimbursement_calculator.py @@ -8,16 +8,15 @@ from sqlalchemy import func, or_, select from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext -from app.core.agent_enums import AgentAssetType from app.models.employee import Employee from app.schemas.reimbursement import ( TravelReimbursementCalculatorRequest, TravelReimbursementCalculatorResponse, ) -from app.services.agent_assets import AgentAssetService from app.services.application_location_semantics import validate_application_location_text from app.services.expense_claims import ExpenseClaimService from app.services.expense_rule_runtime import ExpenseRuleRuntimeService, RuntimeTravelPolicy +from app.services.tenant_registry import required_tenant_id from app.services.travel_policy_grades import travel_policy_grade_key_candidates from app.services.travel_reimbursement_regions import ( AMBIGUOUS_PROVINCE_CITY_NAMES, @@ -45,7 +44,7 @@ class TravelReimbursementCalculatorService: if location_error: raise ValueError(f"{location_error}请填写真实出差地点后再计算。") - policy = self._load_travel_policy() + policy = self._load_travel_policy(current_user) grade = self._resolve_grade(payload.grade, current_user) if not grade: raise ValueError("未识别到当前员工职级，请在个人信息中维护职级后再计算。") @@ -57,7 +56,9 @@ class TravelReimbursementCalculatorService: matched_city = self._resolve_city(location, policy) matched_other_region = "" if matched_city else self._resolve_other_region(location) if not matched_city and not matched_other_region: - raise ValueError(f"出差地点“{location}”未识别为有效出差地区，请按真实省市或规则表地点重新填写。") + raise ValueError( + f"出差地点“{location}”未识别为有效出差地区，请按真实省市或规则表地点重新填写。" + ) city_tier = policy.city_tiers.get(matched_city, "tier_3") if matched_city else "tier_3" hotel_rate = self._resolve_hotel_rate( policy, @@ -66,10 +67,18 @@ class TravelReimbursementCalculatorService: city_tier, payload.travel_date, ) - allowance_region = self._resolve_allowance_region(location, matched_city or matched_other_region) + allowance_region = self._resolve_allowance_region( + location, + matched_city or matched_other_region, + ) meal_rate = self._resolve_allowance_rate(policy, "meal", allowance_region) basic_rate = self._resolve_allowance_rate(policy, "basic", allowance_region) - total_allowance_rate = self._resolve_total_allowance_rate(policy, allowance_region, meal_rate, basic_rate) + total_allowance_rate = self._resolve_total_allowance_rate( + policy, + allowance_region, + meal_rate, + basic_rate, + ) origin_city = self._resolve_origin_city(payload, current_user, policy) transport_mode = self._normalize_transport_mode(payload.transport_mode) transport_estimate = self._resolve_transport_estimate( @@ -139,8 +148,12 @@ class TravelReimbursementCalculatorService: basic_allowance_rate=basic_rate, total_allowance_rate=total_allowance_rate, allowance_amount=allowance_amount, - transport_mode=transport_mode or str(transport_estimate.get("transport_mode") or "").strip(), - transport_origin=str(transport_estimate.get("origin_city") or origin_city or "").strip(), + transport_mode=( + transport_mode or str(transport_estimate.get("transport_mode") or "").strip() + ), + transport_origin=str( + transport_estimate.get("origin_city") or origin_city or "" + ).strip(), transport_destination=str( transport_estimate.get("destination_city") or display_city or location ).strip(), @@ -150,7 +163,9 @@ class TravelReimbursementCalculatorService: transport_estimate_source=str(transport_estimate.get("source") or "").strip(), transport_estimate_rule_code=str(policy.transport_estimate_rule_code or "").strip(), transport_estimate_rule_name=str(policy.transport_estimate_rule_name or "").strip(), - transport_estimate_rule_version=str(policy.transport_estimate_rule_version or "").strip(), + transport_estimate_rule_version=str( + policy.transport_estimate_rule_version or "" + ).strip(), travel_date=payload.travel_date, total_amount=total_amount, rule_name=rule_name, @@ -159,9 +174,16 @@ class TravelReimbursementCalculatorService: summary_text=summary_text, ) - def _load_travel_policy(self) -> RuntimeTravelPolicy: - AgentAssetService(self.db).list_assets(asset_type=AgentAssetType.RULE.value) - policy = ExpenseRuleRuntimeService(self.db).load_catalog().travel_policy + def _load_travel_policy(self, current_user: CurrentUserContext) -> RuntimeTravelPolicy: + # 规则资产由启动阶段初始化；计算器必须保持只读，不能在报销事务中触发规则同步提交。 + policy = ( + ExpenseRuleRuntimeService( + self.db, + tenant_id=current_user.tenant_id, + ) + .load_catalog() + .travel_policy + ) if policy is None: raise ValueError("规则中心暂未配置差旅报销规则。") return policy @@ -209,14 +231,27 @@ class TravelReimbursementCalculatorService: normalized = str(region or "").strip() if not normalized: return "" - if normalized in {"国外", "香港", "澳门", "台湾", "港澳台", "西藏", "拉萨", "新疆", "乌鲁木齐"}: + if normalized in { + "国外", + "香港", + "澳门", + "台湾", + "港澳台", + "西藏", + "拉萨", + "新疆", + "乌鲁木齐", + }: return normalized return f"{normalized}（其他地区）" def _resolve_current_employee(self, current_user: CurrentUserContext) -> Employee | None: + tenant_id = required_tenant_id(current_user.tenant_id) candidates = [ + str(current_user.employee_id or "").strip(), str(current_user.username or "").strip(), str(current_user.name or "").strip(), + str(current_user.employee_no or "").strip(), ] normalized_candidates = [ item @@ -230,10 +265,12 @@ class TravelReimbursementCalculatorService: employee = self.db.scalar( select(Employee) .where( + Employee.tenant_id == tenant_id, or_( + Employee.id == candidate, func.lower(Employee.email) == candidate.lower(), func.lower(Employee.employee_no) == candidate.lower(), - ) + ), ) .limit(1) ) @@ -244,7 +281,10 @@ class TravelReimbursementCalculatorService: matches = list( self.db.scalars( select(Employee) - .where(Employee.name == candidate) + .where( + Employee.tenant_id == tenant_id, + Employee.name == candidate, + ) .limit(2) ).all() ) @@ -277,7 +317,9 @@ class TravelReimbursementCalculatorService: return "飞机" if any(keyword in normalized for keyword in ("火车", "高铁", "动车", "铁路", "列车")): return "火车" - if any(keyword in normalized for keyword in ("轮船", "船票", "客轮", "渡轮", "邮轮", "坐船")): + if any( + keyword in normalized for keyword in ("轮船", "船票", "客轮", "渡轮", "邮轮", "坐船") + ): return "轮船" return normalized if normalized in {"飞机", "火车", "轮船"} else "" @@ -293,9 +335,7 @@ class TravelReimbursementCalculatorService: if self._normalize_city_key(origin_city) == self._normalize_city_key(destination_city): return {} - location_band = self._resolve_transport_location_band( - destination_city or destination_text - ) + location_band = self._resolve_transport_location_band(destination_city or destination_text) candidate_modes = [transport_mode] if transport_mode else ["火车", "飞机", "轮船"] matched = None matched_mode = "" @@ -355,7 +395,9 @@ class TravelReimbursementCalculatorService: if not normalized or normalized in {"*", "默认", "通用"}: return 10 origin_key = self._normalize_city_key(origin_city) - return 30 if normalized == origin_key or normalized in origin_key or origin_key in normalized else 0 + if normalized == origin_key or normalized in origin_key or origin_key in normalized: + return 30 + return 0 def _transport_destination_match_score( self, @@ -381,9 +423,41 @@ class TravelReimbursementCalculatorService: @staticmethod def _resolve_transport_location_band(location: str) -> str: text = str(location or "").strip() - if any(keyword in text for keyword in ("新疆", "西藏", "青海", "甘肃", "宁夏", "内蒙古", "海南", "三亚", "海口", "香港", "澳门", "台湾", "海外", "国外")): + if any( + keyword in text + for keyword in ( + "新疆", + "西藏", + "青海", + "甘肃", + "宁夏", + "内蒙古", + "海南", + "三亚", + "海口", + "香港", + "澳门", + "台湾", + "海外", + "国外", + ) + ): return "remote" - if any(keyword in text for keyword in ("北京", "上海", "广州", "深圳", "杭州", "南京", "苏州", "成都", "重庆", "天津")): + if any( + keyword in text + for keyword in ( + "北京", + "上海", + "广州", + "深圳", + "杭州", + "南京", + "苏州", + "成都", + "重庆", + "天津", + ) + ): return "premium" if any(keyword in text for keyword in ("厦门", "福州", "青岛", "大连", "宁波", "舟山")): return "coastal" @@ -406,13 +480,21 @@ class TravelReimbursementCalculatorService: city_names = set(policy.city_tiers.keys()) city_names.update(policy.hotel_city_limits.keys()) for city in sorted(city_names, key=lambda item: len(item), reverse=True): - if city in AMBIGUOUS_PROVINCE_CITY_NAMES and normalized != city and f"{city}市" not in normalized: + if ( + city in AMBIGUOUS_PROVINCE_CITY_NAMES + and normalized != city + and f"{city}市" not in normalized + ): continue if city and city in normalized: return city compact = re.sub(r"(省|市|区|县|自治州|特别行政区)$", "", normalized) for city in sorted(city_names, key=lambda item: len(item), reverse=True): - if city in AMBIGUOUS_PROVINCE_CITY_NAMES and compact != city and f"{city}市" not in normalized: + if ( + city in AMBIGUOUS_PROVINCE_CITY_NAMES + and compact != city + and f"{city}市" not in normalized + ): continue if city and city in compact: return city @@ -458,7 +540,10 @@ class TravelReimbursementCalculatorService: if not matched_city or travel_date is None: return Decimal("0") period = (getattr(policy, "hotel_peak_periods", {}) or {}).get(matched_city, "") - if not period or not TravelReimbursementCalculatorService._month_in_peak_period(travel_date.month, period): + if not period or not TravelReimbursementCalculatorService._month_in_peak_period( + travel_date.month, + period, + ): return Decimal("0") peak_rate = (getattr(policy, "hotel_peak_city_limits", {}) or {}).get(matched_city) return Decimal(peak_rate or Decimal("0")) @@ -499,12 +584,19 @@ class TravelReimbursementCalculatorService: return "新疆-其他" if "西藏" in text or "拉萨" in text: return "西藏" - if any(keyword in text for keyword in ("北京", "上海", "天津", "重庆", "深圳", "珠海", "汕头", "厦门")): + if any( + keyword in text + for keyword in ("北京", "上海", "天津", "重庆", "深圳", "珠海", "汕头", "厦门") + ): return "直辖市/特区" return "其他地区" @staticmethod - def _resolve_allowance_rate(policy: RuntimeTravelPolicy, allowance_key: str, region: str) -> Decimal: + def _resolve_allowance_rate( + policy: RuntimeTravelPolicy, + allowance_key: str, + region: str, + ) -> Decimal: limits = policy.allowance_limits.get(allowance_key, {}) if limits.get(region) is not None: return Decimal(limits[region]) diff --git a/server/src/app/services/user_agent.py b/server/src/app/services/user_agent.py index 7630458..fc866fe 100644 --- a/server/src/app/services/user_agent.py +++ b/server/src/app/services/user_agent.py @@ -39,6 +39,7 @@ from app.services.expense_claims import ExpenseClaimService from app.services.expense_rule_runtime import ExpenseRuleRuntimeService, RuntimeTravelPolicy, resolve_document_type_label from app.services.risk_ontology_bridge import resolve_rule_codes_for_risk_check from app.services.runtime_chat import RuntimeChatService +from app.services.runtime_chat_commercial import build_runtime_chat_commercial_observer from app.services.travel_reimbursement_calculator import TravelReimbursementCalculatorService from app.services.user_agent_documents import UserAgentDocumentService from app.services.user_agent_knowledge import UserAgentKnowledgeMixin @@ -68,7 +69,10 @@ class UserAgentService( def __init__(self, db: Session) -> None: self.db = db self.asset_service = AgentAssetService(db) - self.runtime_chat_service = RuntimeChatService(db) + self.runtime_chat_service = RuntimeChatService( + db, + attempt_observer=build_runtime_chat_commercial_observer(db), + ) self._document_service = UserAgentDocumentService(group_scene_labels=GROUP_SCENE_LABELS) def respond(self, payload: UserAgentRequest) -> UserAgentResponse: diff --git a/server/src/app/services/user_agent_application.py b/server/src/app/services/user_agent_application.py index 2aa331f..f76da6f 100644 --- a/server/src/app/services/user_agent_application.py +++ b/server/src/app/services/user_agent_application.py @@ -845,6 +845,7 @@ class UserAgentApplicationPersistenceMixin: department_name = str(employee.organization_unit.name).strip() claim = ExpenseClaim( + tenant_id=current_user.tenant_id, claim_no=self._build_application_claim_no(payload, facts), employee_id=employee_id, employee_name=employee_name, diff --git a/server/src/app/services/user_agent_response.py b/server/src/app/services/user_agent_response.py index 78e2985..947ef65 100644 --- a/server/src/app/services/user_agent_response.py +++ b/server/src/app/services/user_agent_response.py @@ -36,6 +36,7 @@ from app.services.agent_assets import AgentAssetService from app.services.expense_claims import ExpenseClaimService from app.services.expense_rule_runtime import ExpenseRuleRuntimeService, RuntimeTravelPolicy, resolve_document_type_label from app.services.risk_ontology_bridge import resolve_rule_codes_for_risk_check +from app.services.runtime_chat_commercial import trusted_runtime_chat_operation_context from app.services.travel_reimbursement_calculator import TravelReimbursementCalculatorService from app.services.user_agent_constants import * @@ -164,6 +165,11 @@ class UserAgentResponseMixin: else None ), max_attempts=1 if payload.ontology.scenario == "knowledge" else None, + operation_context=trusted_runtime_chat_operation_context( + self.db, + run_id=payload.run_id, + attempt_scope="user-agent-response", + ), ) ) return self._reject_unsupported_location_inference(payload, answer) @@ -718,4 +724,3 @@ class UserAgentResponseMixin: description="补充业务对象、时间或单据范围，提升回答准确度。", ), ] - diff --git a/server/tests/commercial_migration_assertions.py b/server/tests/commercial_migration_assertions.py new file mode 100644 index 0000000..c9eb4b3 --- /dev/null +++ b/server/tests/commercial_migration_assertions.py @@ -0,0 +1,477 @@ +from __future__ import annotations + +from typing import Any + +import pytest +from sqlalchemy import inspect, text +from sqlalchemy.engine import Engine +from sqlalchemy.exc import DBAPIError, IntegrityError + +from app.models.commercial import ( + CommercialCostEvent, + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) +from app.models.commercial_billing import CommercialAdminEvent, CommercialBillingPeriod +from app.models.commercial_runtime import CommercialRuntimeReservation + +COMMERCIAL_MODELS = ( + TenantCommercialPlan, + TenantSubscription, + CommercialEntitlement, + UsageMeterEvent, + CommercialCostEvent, + CommercialRuntimeReservation, + CommercialBillingPeriod, + CommercialAdminEvent, +) + + +def _assert_commercial_head_schema(engine: Engine) -> None: + inspector = inspect(engine) + for model in COMMERCIAL_MODELS: + table = model.__table__ + live_columns = { + str(column["name"]): bool(column["nullable"]) + for column in inspector.get_columns(table.name, schema="public") + } + declared_columns = {column.name: bool(column.nullable) for column in table.columns} + assert live_columns == declared_columns + + live_constraint_names = { + str(item["name"]) + for loader in ( + inspector.get_unique_constraints, + inspector.get_check_constraints, + inspector.get_foreign_keys, + ) + for item in loader(table.name, schema="public") + if item.get("name") + } + declared_constraint_names = { + str(constraint.name) for constraint in table.constraints if constraint.name is not None + } + assert live_constraint_names == declared_constraint_names + + live_index_names = { + str(item["name"]) + for item in inspector.get_indexes(table.name, schema="public") + if not item.get("duplicates_constraint") + } + declared_index_names = {str(index.name) for index in table.indexes} + assert live_index_names == declared_index_names + + savings_tables = { + "profile_baseline_snapshots", + "savings_opportunities", + "savings_realizations", + "savings_evidence_links", + "savings_events", + } + for table_name in ("usage_meter_events", "commercial_cost_events"): + target_tables = { + str(foreign_key["referred_table"]) + for foreign_key in inspector.get_foreign_keys(table_name, schema="public") + } + assert target_tables.isdisjoint(savings_tables) + + with engine.connect() as connection: + active_plan_index = str( + connection.scalar( + text( + "SELECT indexdef FROM pg_indexes WHERE schemaname = 'public' " + "AND tablename = 'tenant_commercial_plans' " + "AND indexname = 'uq_tenant_commercial_plans_active_code'" + ) + ) + or "" + ).lower() + current_subscription_index = str( + connection.scalar( + text( + "SELECT indexdef FROM pg_indexes WHERE schemaname = 'public' " + "AND tablename = 'tenant_subscriptions' " + "AND indexname = 'uq_tenant_subscriptions_current'" + ) + ) + or "" + ).lower() + trigger_counts = { + table_name: int( + connection.scalar( + text( + "SELECT COUNT(*) FROM pg_trigger trigger " + "JOIN pg_class relation ON relation.oid = trigger.tgrelid " + "WHERE relation.relname = :table_name " + "AND trigger.tgname = :trigger_name " + "AND NOT trigger.tgisinternal" + ), + { + "table_name": table_name, + "trigger_name": f"trg_{table_name}_append_only", + }, + ) + or 0 + ) + for table_name in ( + "usage_meter_events", + "commercial_cost_events", + "commercial_billing_periods", + "commercial_admin_events", + ) + } + period_overlap_trigger_count = int( + connection.scalar( + text( + "SELECT COUNT(*) FROM pg_trigger trigger " + "JOIN pg_class relation ON relation.oid = trigger.tgrelid " + "WHERE relation.relname = 'commercial_billing_periods' " + "AND trigger.tgname = " + "'trg_commercial_billing_periods_no_overlap' " + "AND NOT trigger.tgisinternal" + ) + ) + or 0 + ) + assert "unique index" in active_plan_index and "status" in active_plan_index + assert "active" in active_plan_index + assert "unique index" in current_subscription_index + assert all(status in current_subscription_index for status in ("active", "suspended")) + assert trigger_counts == { + "usage_meter_events": 1, + "commercial_cost_events": 1, + "commercial_billing_periods": 1, + "commercial_admin_events": 1, + } + assert period_overlap_trigger_count == 1 + + +def _assert_commercial_runtime_invariants(engine: Engine) -> None: + def execute_rejected( + connection: Any, + statement: Any, + parameters: dict[str, Any], + error_type: type[DBAPIError] = IntegrityError, + ) -> None: + savepoint = connection.begin_nested() + try: + with pytest.raises(error_type): + connection.execute(statement, parameters) + finally: + if savepoint.is_active: + savepoint.rollback() + + subscription_insert = text( + """ + INSERT INTO tenant_subscriptions ( + id, tenant_id, subscription_key, plan_id, status, starts_at, + current_period_start, current_period_end, seats, base_fee_snapshot, + currency, billing_interval, created_by + ) VALUES ( + :id, :tenant_id, :subscription_key, :plan_id, 'active', now(), + now(), now() + interval '1 month', 10, 1000, 'CNY', 'monthly', 'probe' + ) + """ + ) + usage_insert = text( + """ + INSERT INTO usage_meter_events ( + id, tenant_id, subscription_id, entitlement_id, billing_period_id, + event_type, metric_key, quantity, unit, period_key, quota_period_key, + occurred_at, source_system, + idempotency_key, request_fingerprint, actor_type, actor_id + ) VALUES ( + :id, :tenant_id, :subscription_id, :entitlement_id, 'period-a', 'usage', + 'ai.review', 1, 'request', 'bp-probe', '2026-07', now(), 'runtime-probe', + :idempotency_key, :request_fingerprint, 'system', 'migration-probe' + ) + """ + ) + cost_insert = text( + """ + INSERT INTO commercial_cost_events ( + id, tenant_id, subscription_id, billing_period_id, usage_event_id, event_type, + cost_category, quantity, unit, unit_cost, cost_amount, + original_currency, reporting_amount, reporting_currency, fx_rate, + allocation_key, occurred_at, source_system, idempotency_key, + request_fingerprint + ) VALUES ( + :id, :tenant_id, :subscription_id, 'period-a', :usage_event_id, 'incurred', + 'ai_inference', 10, 'token', 0.01, 0.10, + 'CNY', 0.10, 'CNY', 1, 'ai.review', now(), 'runtime-probe', + :idempotency_key, :request_fingerprint + ) + """ + ) + + with engine.connect() as connection: + transaction = connection.begin() + try: + connection.execute( + text( + """ + INSERT INTO tenant_commercial_plans ( + id, tenant_id, plan_code, name, pricing_model, + billing_interval, currency, base_fee, included_seats, + status, effective_from, created_by + ) VALUES ( + 'plan-a', 'tenant-a', 'enterprise', '企业版', 'hybrid', + 'monthly', 'CNY', 1000, 10, 'active', now(), 'probe' + ) + """ + ) + ) + connection.execute( + subscription_insert, + { + "id": "subscription-a", + "tenant_id": "tenant-a", + "subscription_key": "subscription-a", + "plan_id": "plan-a", + }, + ) + connection.execute( + text( + """ + INSERT INTO commercial_billing_periods ( + id, tenant_id, subscription_id, plan_id, period_sequence, + period_key, status, period_start, period_end, + subscription_status_snapshot, plan_code_snapshot, + plan_version_snapshot, pricing_model_snapshot, + billing_interval, currency, base_fee_snapshot, + seats_snapshot, source, idempotency_key, created_by + ) VALUES ( + 'period-a', 'tenant-a', 'subscription-a', 'plan-a', 1, + 'bp-probe', 'issued', now(), now() + interval '1 month', + 'active', 'enterprise', 1, 'hybrid', 'monthly', 'CNY', + 1000, 10, 'subscription_created', 'period-request-a', 'probe' + ) + """ + ) + ) + execute_rejected( + connection, + text( + """ + INSERT INTO commercial_billing_periods ( + id, tenant_id, subscription_id, plan_id, period_sequence, + period_key, status, period_start, period_end, + subscription_status_snapshot, plan_code_snapshot, + plan_version_snapshot, pricing_model_snapshot, + billing_interval, currency, base_fee_snapshot, + seats_snapshot, source, idempotency_key, created_by + ) VALUES ( + 'period-overlap', 'tenant-a', 'subscription-a', 'plan-a', 2, + 'bp-overlap', 'issued', now() + interval '15 days', + now() + interval '45 days', 'active', 'enterprise', 1, + 'hybrid', 'monthly', 'CNY', 1000, 10, 'auto_renew', + 'period-request-overlap', 'probe' + ) + """ + ), + {}, + DBAPIError, + ) + connection.execute( + text( + """ + INSERT INTO commercial_admin_events ( + id, tenant_id, actor_type, actor_id, request_id, reason, + action, resource_type, resource_id, resource_version, + before_json, after_json + ) VALUES ( + 'admin-event-a', 'tenant-a', 'user', 'probe', + 'admin-request-a', '迁移不变量探针', 'billing_period_created', + 'billing_period', 'period-a', 1, '{}'::json, '{}'::json + ) + """ + ) + ) + execute_rejected( + connection, + subscription_insert, + { + "id": "cross-tenant-subscription", + "tenant_id": "tenant-b", + "subscription_key": "cross-tenant-subscription", + "plan_id": "plan-a", + }, + ) + connection.execute( + text( + """ + INSERT INTO commercial_entitlements ( + id, tenant_id, subscription_id, entitlement_key, + metric_key, entitlement_type, unit, included_quantity, + hard_limit_quantity, reset_interval, overage_policy, + status, effective_from + ) VALUES ( + 'entitlement-a', 'tenant-a', 'subscription-a', 'ai-review', + 'ai.review', 'metered', 'request', 100, 120, 'monthly', + 'block', 'active', now() + ) + """ + ) + ) + connection.execute( + text( + """ + INSERT INTO commercial_runtime_reservations ( + id, tenant_id, subscription_id, entitlement_id, + billing_period_id, run_id, + tool_call_id, tool_type, tool_name, quantity_basis, + reserved_quantity, period_key, quota_period_key, status, + request_fingerprint, + meter_config_json, expires_at + ) VALUES ( + 'reservation-a', 'tenant-a', 'subscription-a', + 'entitlement-a', 'period-a', 'run-a', 'tool-call-a', 'llm', + 'chat.completions', 'call', 1, 'bp-probe', '2026-07', 'reserved', + 'reservation-fingerprint-a', '{}'::json, now() + interval '15 minutes' + ) + """ + ) + ) + execute_rejected( + connection, + text( + """ + INSERT INTO commercial_runtime_reservations ( + id, tenant_id, subscription_id, entitlement_id, + billing_period_id, run_id, + tool_call_id, tool_type, tool_name, quantity_basis, + reserved_quantity, period_key, quota_period_key, status, + request_fingerprint, + meter_config_json, expires_at + ) VALUES ( + 'reservation-duplicate', 'tenant-a', 'subscription-a', + 'entitlement-a', 'period-a', 'run-b', 'tool-call-a', 'llm', + 'chat.completions', 'call', 1, 'bp-probe', '2026-07', 'reserved', + 'reservation-fingerprint-b', '{}'::json, now() + interval '15 minutes' + ) + """ + ), + {}, + ) + execute_rejected( + connection, + text( + """ + INSERT INTO commercial_runtime_reservations ( + id, tenant_id, subscription_id, entitlement_id, + billing_period_id, run_id, + tool_call_id, tool_type, tool_name, quantity_basis, + reserved_quantity, actual_quantity, period_key, + quota_period_key, status, + request_fingerprint, meter_config_json, expires_at, settled_at + ) VALUES ( + 'reservation-invalid', 'tenant-a', 'subscription-a', + 'entitlement-a', 'period-a', 'run-c', 'tool-call-c', 'llm', + 'chat.completions', 'call', 1, 2, 'bp-probe', '2026-07', 'committed', + 'reservation-fingerprint-c', '{}'::json, + now() + interval '15 minutes', now() + ) + """ + ), + {}, + ) + connection.execute( + usage_insert, + { + "id": "usage-a", + "tenant_id": "tenant-a", + "subscription_id": "subscription-a", + "entitlement_id": "entitlement-a", + "idempotency_key": "usage-request-a", + "request_fingerprint": "sha256:usage-a", + }, + ) + execute_rejected( + connection, + usage_insert, + { + "id": "usage-duplicate", + "tenant_id": "tenant-a", + "subscription_id": "subscription-a", + "entitlement_id": "entitlement-a", + "idempotency_key": "usage-request-a", + "request_fingerprint": "sha256:different-payload", + }, + ) + execute_rejected( + connection, + usage_insert, + { + "id": "usage-cross-tenant", + "tenant_id": "tenant-b", + "subscription_id": "subscription-a", + "entitlement_id": "entitlement-a", + "idempotency_key": "usage-cross-tenant", + "request_fingerprint": "sha256:cross-tenant", + }, + ) + connection.execute( + cost_insert, + { + "id": "cost-a", + "tenant_id": "tenant-a", + "subscription_id": "subscription-a", + "usage_event_id": "usage-a", + "idempotency_key": "cost-request-a", + "request_fingerprint": "sha256:cost-a", + }, + ) + execute_rejected( + connection, + cost_insert, + { + "id": "cost-duplicate", + "tenant_id": "tenant-a", + "subscription_id": "subscription-a", + "usage_event_id": "usage-a", + "idempotency_key": "cost-request-a", + "request_fingerprint": "sha256:different-cost", + }, + ) + execute_rejected( + connection, + cost_insert, + { + "id": "cost-cross-tenant", + "tenant_id": "tenant-b", + "subscription_id": "subscription-a", + "usage_event_id": "usage-a", + "idempotency_key": "cost-cross-tenant", + "request_fingerprint": "sha256:cross-cost", + }, + ) + execute_rejected( + connection, + text("UPDATE usage_meter_events SET quantity = 2 WHERE id = 'usage-a'"), + {}, + DBAPIError, + ) + execute_rejected( + connection, + text("DELETE FROM commercial_cost_events WHERE id = 'cost-a'"), + {}, + DBAPIError, + ) + execute_rejected( + connection, + text( + "UPDATE commercial_billing_periods SET currency = 'USD' WHERE id = 'period-a'" + ), + {}, + DBAPIError, + ) + execute_rejected( + connection, + text("DELETE FROM commercial_admin_events WHERE id = 'admin-event-a'"), + {}, + DBAPIError, + ) + finally: + transaction.rollback() diff --git a/server/tests/commercial_runtime_testkit.py b/server/tests/commercial_runtime_testkit.py new file mode 100644 index 0000000..69b6f26 --- /dev/null +++ b/server/tests/commercial_runtime_testkit.py @@ -0,0 +1,153 @@ +from __future__ import annotations + +import uuid +from datetime import datetime, timedelta +from decimal import Decimal +from typing import Any + +from sqlalchemy.orm import Session + +from app.models.agent_run import AgentRun, AgentToolCall +from app.schemas.commercial import ( + CommercialEntitlementUpsert, + CommercialPlanCreate, + CommercialSubscriptionCreate, +) +from app.services.commercial_admin import CommercialAdminService + + +def seed_meter( + db: Session, + tenant_id: str, + now: datetime, + *, + basis: str, + tool_type: str = "llm", + tool_name: str = "chat.completions", + hard_limit: Decimal = Decimal("1000"), + internal_cost: dict[str, Any] | None = None, + subscription_status: str = "active", + preflight_quantity: Decimal | None = None, +): + admin = CommercialAdminService(db) + plan = admin.create_plan( + tenant_id, + CommercialPlanCreate( + plan_code="runtime", + name="运行时计量测试套餐", + pricing_model="hybrid", + billing_interval="monthly", + currency="CNY", + base_fee=Decimal("100"), + included_seats=10, + effective_from=now - timedelta(days=30), + ), + actor_id="platform-admin", + ) + admin.activate_plan(tenant_id, plan.id, expected_version=plan.version) + subscription = admin.create_subscription( + tenant_id, + CommercialSubscriptionCreate( + subscription_key=f"{tenant_id}-runtime", + plan_id=plan.id, + status=subscription_status, + starts_at=now - timedelta(days=10), + current_period_start=now - timedelta(days=1), + current_period_end=now + timedelta(days=29), + seats=5, + ), + actor_id="platform-admin", + ) + runtime_meter: dict[str, Any] = { + "enabled": True, + "tool_type": tool_type, + "tool_name": tool_name, + "quantity_basis": basis, + } + if preflight_quantity is not None: + runtime_meter["preflight_quantity"] = str(preflight_quantity) + if internal_cost is not None: + runtime_meter["internal_cost"] = internal_cost + entitlement = admin.upsert_entitlement( + tenant_id, + CommercialEntitlementUpsert( + subscription_id=subscription.id, + entitlement_key=f"runtime_{basis}", + metric_key=f"runtime_{basis}", + entitlement_type="metered", + unit="call" if basis == "call" else basis, + included_quantity=hard_limit, + hard_limit_quantity=hard_limit, + reset_interval="monthly", + overage_policy="block", + status="active", + effective_from=now - timedelta(days=10), + config_json={"runtime_meter": runtime_meter}, + ), + ) + db.flush() + return subscription, entitlement + + +def seed_tool_call( + db: Session, + occurred_at: datetime, + *, + route_json: dict[str, Any], + request_json: dict[str, Any] | None = None, + response_json: dict[str, Any] | None = None, + duration_ms: int = 0, + tool_type: str = "llm", + tool_name: str = "chat.completions", + user_id: str = "sensitive-user-id", + run: AgentRun | None = None, + status: str = "succeeded", +) -> tuple[AgentRun, AgentToolCall]: + if run is None: + run = AgentRun( + run_id=f"run-{uuid.uuid4().hex}", + agent="user_agent", + source="chat", + user_id=user_id, + route_json=route_json, + permission_level="write", + status="succeeded", + started_at=occurred_at, + ) + db.add(run) + tool_call = AgentToolCall( + id=str(uuid.uuid4()), + run_id=run.run_id, + tool_type=tool_type, + tool_name=tool_name, + request_json=request_json or {}, + response_json=response_json or {}, + status=status, + duration_ms=duration_ms, + created_at=occurred_at, + ) + db.add(tool_call) + db.flush() + return run, tool_call + + +def seed_run( + db: Session, + started_at: datetime, + *, + route_json: dict[str, Any], + status: str = "running", +) -> AgentRun: + run = AgentRun( + run_id=f"run-{uuid.uuid4().hex}", + agent="user_agent", + source="chat", + user_id="runtime-user", + route_json=route_json, + permission_level="write", + status=status, + started_at=started_at, + ) + db.add(run) + db.commit() + return run diff --git a/server/tests/financial_connector_migration_assertions.py b/server/tests/financial_connector_migration_assertions.py new file mode 100644 index 0000000..6f08593 --- /dev/null +++ b/server/tests/financial_connector_migration_assertions.py @@ -0,0 +1,339 @@ +from __future__ import annotations + +from typing import Any + +import pytest +from sqlalchemy import inspect, text +from sqlalchemy.engine import Engine +from sqlalchemy.exc import DBAPIError, IntegrityError + +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorConfigEvent, + FinancialConnectorEvent, + FinancialConnectorOperationalEvent, + PaymentReconciliationCase, + PaymentReconciliationEvent, +) + +FINANCIAL_CONNECTOR_MODELS = ( + FinancialConnectorConfig, + FinancialConnectorConfigEvent, + FinancialConnectorEvent, + FinancialConnectorOperationalEvent, + PaymentReconciliationCase, + PaymentReconciliationEvent, +) + + +def _assert_financial_connector_head_schema(engine: Engine) -> None: + inspector = inspect(engine) + for model in FINANCIAL_CONNECTOR_MODELS: + table = model.__table__ + live_columns = { + str(column["name"]): bool(column["nullable"]) + for column in inspector.get_columns(table.name, schema="public") + } + declared_columns = {column.name: bool(column.nullable) for column in table.columns} + assert live_columns == declared_columns + + live_constraint_names = { + str(item["name"]) + for loader in ( + inspector.get_unique_constraints, + inspector.get_check_constraints, + inspector.get_foreign_keys, + ) + for item in loader(table.name, schema="public") + if item.get("name") + } + declared_constraint_names = { + str(constraint.name) + for constraint in table.constraints + if constraint.name is not None + } + assert live_constraint_names == declared_constraint_names + + live_index_names = { + str(item["name"]) + for item in inspector.get_indexes(table.name, schema="public") + if not item.get("duplicates_constraint") + } + declared_index_names = {str(index.name) for index in table.indexes} + assert live_index_names == declared_index_names + + for table_name in ( + "financial_connector_config_events", + "financial_connector_events", + "financial_connector_operational_events", + "payment_reconciliation_events", + ): + with engine.connect() as connection: + trigger_count = int( + connection.scalar( + text( + "SELECT COUNT(*) FROM pg_trigger trigger " + "JOIN pg_class relation ON relation.oid = trigger.tgrelid " + "WHERE relation.relname = :table_name " + "AND trigger.tgname = :trigger_name " + "AND NOT trigger.tgisinternal" + ), + { + "table_name": table_name, + "trigger_name": f"trg_{table_name}_append_only", + }, + ) + or 0 + ) + assert trigger_count == 1 + + +def _assert_financial_connector_runtime_invariants(engine: Engine) -> None: + def rejected( + connection: Any, + statement: Any, + parameters: dict[str, Any], + error_type: type[DBAPIError] = IntegrityError, + ) -> None: + savepoint = connection.begin_nested() + try: + with pytest.raises(error_type): + connection.execute(statement, parameters) + finally: + if savepoint.is_active: + savepoint.rollback() + + config_insert = text( + """ + INSERT INTO financial_connector_configs ( + id, tenant_id, provider, environment, key_version, secret_ref, + allowed_event_types_json, clock_skew_seconds, status, version, created_by + ) VALUES ( + :id, :tenant_id, 'probe-bank', 'production', 'v1', 'server/probe', + '["payment_settled"]', 300, 'active', 1, 'migration-probe' + ) + """ + ) + event_insert = text( + """ + INSERT INTO financial_connector_events ( + id, tenant_id, config_id, provider, environment, direction, + external_event_id, event_type, occurred_at, key_version, + verification_level, request_fingerprint, content_hash, + processing_status, claim_id, expense_case_id, correlation_id, + normalized_payload_json, response_json + ) VALUES ( + :id, :tenant_id, :config_id, 'probe-bank', 'production', 'inbound', + :external_event_id, 'payment_settled', now(), 'v1', + 'production_verified', :fingerprint, :content_hash, + 'processed', 'claim-soft-ref', :expense_case_id, 'probe-correlation', + '{}', '{}' + ) + """ + ) + with engine.connect() as connection: + transaction = connection.begin() + try: + connection.execute( + text( + """ + INSERT INTO expense_cases ( + id, tenant_id, case_no, scene_code, title, current_stage, status + ) VALUES ( + 'connector-case-a', 'tenant-a', 'CASE-CONNECTOR-A', + 'travel', '连接器迁移探针', 'paying', 'active' + ) + """ + ) + ) + connection.execute( + config_insert, + {"id": "connector-config-a", "tenant_id": "tenant-a"}, + ) + connection.execute( + text( + """ + INSERT INTO financial_connector_operational_events ( + id, tenant_id, config_id, provider, environment, + event_type, reason_code, request_fingerprint, + external_event_fingerprint, idempotency_key + ) VALUES ( + 'connector-operational-event-a', 'tenant-a', + 'connector-config-a', 'probe-bank', 'production', + 'replay', 'duplicate_external_event', + :request_fingerprint, :external_event_fingerprint, + :idempotency_key + ) + """ + ), + { + "request_fingerprint": "hmac-sha256:" + "a" * 64, + "external_event_fingerprint": "hmac-sha256:" + "b" * 64, + "idempotency_key": "sha256:" + "c" * 64, + }, + ) + rejected( + connection, + text( + """ + INSERT INTO financial_connector_operational_events ( + id, tenant_id, config_id, provider, environment, + event_type, reason_code, request_fingerprint, + external_event_fingerprint, idempotency_key + ) VALUES ( + 'connector-operational-event-cross', 'tenant-b', + 'connector-config-a', 'probe-bank', 'production', + 'replay', 'duplicate_external_event', + :request_fingerprint, :external_event_fingerprint, + :idempotency_key + ) + """ + ), + { + "request_fingerprint": "hmac-sha256:" + "d" * 64, + "external_event_fingerprint": "hmac-sha256:" + "e" * 64, + "idempotency_key": "sha256:" + "f" * 64, + }, + ) + rejected( + connection, + text( + """ + INSERT INTO financial_connector_operational_events ( + id, tenant_id, config_id, provider, environment, + event_type, reason_code, request_fingerprint, + external_event_fingerprint, idempotency_key + ) VALUES ( + 'connector-operational-weak', 'tenant-a', + 'connector-config-a', 'probe-bank', 'production', + 'auth_failure', 'signature_invalid', + :request_fingerprint, :external_event_fingerprint, + :idempotency_key + ) + """ + ), + { + "request_fingerprint": "sha256:" + "a" * 64, + "external_event_fingerprint": "hmac-sha256:" + "b" * 64, + "idempotency_key": "sha256:" + "d" * 64, + }, + ) + connection.execute( + text( + """ + INSERT INTO financial_connector_config_events ( + id, tenant_id, config_id, action, actor_id, request_id, + reason, expected_version, before_json, after_json + ) VALUES ( + 'connector-config-event-a', 'tenant-a', 'connector-config-a', + 'activated', 'migration-probe', 'migration-request-001', + '迁移运行时约束探针', 1, '{}', + '{"status": "active", "version": 2}' + ) + """ + ) + ) + rejected( + connection, + text( + "UPDATE financial_connector_configs SET version = 0 " + "WHERE id = 'connector-config-a'" + ), + {}, + ) + connection.execute( + event_insert, + { + "id": "connector-event-a", + "tenant_id": "tenant-a", + "config_id": "connector-config-a", + "external_event_id": "external-a", + "fingerprint": "sha256:" + "a" * 64, + "content_hash": "sha256:" + "b" * 64, + "expense_case_id": "connector-case-a", + }, + ) + rejected( + connection, + event_insert, + { + "id": "connector-event-cross", + "tenant_id": "tenant-b", + "config_id": "connector-config-a", + "external_event_id": "external-cross", + "fingerprint": "sha256:" + "c" * 64, + "content_hash": "sha256:" + "d" * 64, + "expense_case_id": "connector-case-a", + }, + ) + connection.execute( + text( + """ + INSERT INTO payment_reconciliation_cases ( + id, tenant_id, provider, claim_id, expense_case_id, + expected_amount, actual_amount, amount_difference, + expected_currency, actual_currency, expected_reference, + status, erp_status, last_connector_event_id, version + ) VALUES ( + 'reconciliation-a', 'tenant-a', 'probe-bank', 'claim-soft-ref', + 'connector-case-a', 10, 10, 0, 'CNY', 'CNY', 'BX-PROBE', + 'matched', 'pending_posting', 'connector-event-a', 1 + ) + """ + ) + ) + connection.execute( + text( + """ + INSERT INTO payment_reconciliation_events ( + id, tenant_id, reconciliation_case_id, connector_event_id, + action, actor_type, actor_id, request_fingerprint, + before_json, after_json, response_json, correlation_id + ) VALUES ( + 'reconciliation-event-a', 'tenant-a', 'reconciliation-a', + 'connector-event-a', 'auto_matched', 'connector', 'probe-bank', + :fingerprint, '{}', '{}', '{}', 'probe-correlation' + ) + """ + ), + {"fingerprint": "sha256:" + "e" * 64}, + ) + rejected( + connection, + text( + "UPDATE financial_connector_config_events SET reason = 'tampered' " + "WHERE id = 'connector-config-event-a'" + ), + {}, + DBAPIError, + ) + rejected( + connection, + text( + "UPDATE financial_connector_events SET error_code = 'tampered' " + "WHERE id = 'connector-event-a'" + ), + {}, + DBAPIError, + ) + rejected( + connection, + text( + "UPDATE financial_connector_operational_events " + "SET reason_code = 'tampered' " + "WHERE id = 'connector-operational-event-a'" + ), + {}, + DBAPIError, + ) + rejected( + connection, + text( + "DELETE FROM payment_reconciliation_events " + "WHERE id = 'reconciliation-event-a'" + ), + {}, + DBAPIError, + ) + finally: + transaction.rollback() diff --git a/server/tests/release_telemetry_migration_assertions.py b/server/tests/release_telemetry_migration_assertions.py new file mode 100644 index 0000000..b6191fb --- /dev/null +++ b/server/tests/release_telemetry_migration_assertions.py @@ -0,0 +1,324 @@ +from __future__ import annotations + +from typing import Any + +import pytest +from sqlalchemy import inspect, text +from sqlalchemy.engine import Engine +from sqlalchemy.exc import DBAPIError, IntegrityError + +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) + +RELEASE_TELEMETRY_MODELS = ( + AgentAssetReleaseObservation, + AgentAssetReleaseLabel, + AgentAssetReleaseAuditSample, +) + + +def _assert_release_telemetry_head_schema(engine: Engine) -> None: + inspector = inspect(engine) + for model in RELEASE_TELEMETRY_MODELS: + table = model.__table__ + live_columns = { + str(column["name"]): bool(column["nullable"]) + for column in inspector.get_columns(table.name, schema="public") + } + declared_columns = { + column.name: bool(column.nullable) for column in table.columns + } + assert live_columns == declared_columns + + live_constraint_names = { + str(item["name"]) + for loader in ( + inspector.get_unique_constraints, + inspector.get_check_constraints, + inspector.get_foreign_keys, + ) + for item in loader(table.name, schema="public") + if item.get("name") + } + declared_constraint_names = { + str(constraint.name) + for constraint in table.constraints + if constraint.name is not None + } + assert live_constraint_names == declared_constraint_names + + live_index_names = { + str(item["name"]) + for item in inspector.get_indexes(table.name, schema="public") + if not item.get("duplicates_constraint") + } + declared_index_names = {str(index.name) for index in table.indexes} + assert live_index_names == declared_index_names + + label_foreign_keys = inspector.get_foreign_keys( + "agent_asset_release_labels", + schema="public", + ) + assert len(label_foreign_keys) == 1 + foreign_key = label_foreign_keys[0] + assert tuple(foreign_key["constrained_columns"]) == ( + "tenant_id", + "observation_id", + "asset_id", + "release_id", + "stage", + "version", + ) + assert foreign_key["referred_table"] == "agent_asset_release_observations" + assert str(foreign_key.get("options", {}).get("ondelete", "")).upper() == "RESTRICT" + + audit_sample_foreign_keys = inspector.get_foreign_keys( + "agent_asset_release_audit_samples", + schema="public", + ) + assert len(audit_sample_foreign_keys) == 1 + audit_sample_foreign_key = audit_sample_foreign_keys[0] + assert tuple(audit_sample_foreign_key["constrained_columns"]) == ( + "tenant_id", + "observation_id", + "asset_id", + "release_id", + "stage", + "version", + ) + assert audit_sample_foreign_key["referred_table"] == ( + "agent_asset_release_observations" + ) + assert ( + str(audit_sample_foreign_key.get("options", {}).get("ondelete", "")).upper() + == "RESTRICT" + ) + + with engine.connect() as connection: + trigger_counts = { + table_name: int( + connection.scalar( + text( + "SELECT COUNT(*) FROM pg_trigger trigger " + "JOIN pg_class relation ON relation.oid = trigger.tgrelid " + "WHERE relation.relname = :table_name " + "AND trigger.tgname = :trigger_name " + "AND NOT trigger.tgisinternal" + ), + { + "table_name": table_name, + "trigger_name": f"trg_{table_name}_append_only", + }, + ) + or 0 + ) + for table_name in ( + "agent_asset_release_observations", + "agent_asset_release_labels", + "agent_asset_release_audit_samples", + ) + } + assert trigger_counts == { + "agent_asset_release_observations": 1, + "agent_asset_release_labels": 1, + "agent_asset_release_audit_samples": 1, + } + + +def _assert_release_telemetry_runtime_invariants(engine: Engine) -> None: + def rejected( + connection: Any, + statement: Any, + parameters: dict[str, Any], + error_type: type[DBAPIError] = IntegrityError, + ) -> None: + savepoint = connection.begin_nested() + try: + with pytest.raises(error_type): + connection.execute(statement, parameters) + finally: + if savepoint.is_active: + savepoint.rollback() + + observation_insert = text( + """ + INSERT INTO agent_asset_release_observations ( + id, tenant_id, asset_id, release_id, stage, version, rule_code, + business_stage, source_fingerprint, candidate_hit, baseline_hit, + runtime_status, idempotency_key, payload_fingerprint + ) VALUES ( + :id, :tenant_id, 'asset-a', 'release-a', 'canary', 'v1', 'TRAVEL-001', + 'pre_submit', :source_fingerprint, true, false, 'completed', + :idempotency_key, :payload_fingerprint + ) + """ + ) + label_insert = text( + """ + INSERT INTO agent_asset_release_labels ( + id, tenant_id, observation_id, asset_id, release_id, stage, version, + label, verification_source, source_event_fingerprint, + actor_fingerprint, idempotency_key, payload_fingerprint + ) VALUES ( + :id, :tenant_id, 'observation-a', 'asset-a', 'release-a', 'canary', + 'v1', 'confirmed', 'typed_risk_disposition', :event_fingerprint, + :actor_fingerprint, :idempotency_key, :payload_fingerprint + ) + """ + ) + audit_sample_insert = text( + """ + INSERT INTO agent_asset_release_audit_samples ( + id, tenant_id, observation_id, asset_id, release_id, stage, version, + stratum, sampling_probability_ppm, selection_score_ppm, + source_reference_encrypted, idempotency_key, payload_fingerprint + ) VALUES ( + :id, :tenant_id, 'observation-a', 'asset-a', 'release-a', 'canary', + 'v1', :stratum, :sampling_probability_ppm, :selection_score_ppm, + :source_reference_encrypted, :idempotency_key, :payload_fingerprint + ) + """ + ) + + with engine.connect() as connection: + transaction = connection.begin() + try: + observation_parameters = { + "id": "observation-a", + "tenant_id": "tenant-a", + "source_fingerprint": "a" * 64, + "idempotency_key": "observation-request-a", + "payload_fingerprint": "b" * 64, + } + connection.execute(observation_insert, observation_parameters) + rejected( + connection, + observation_insert, + { + **observation_parameters, + "id": "observation-duplicate", + "payload_fingerprint": "c" * 64, + }, + ) + audit_sample_parameters = { + "id": "audit-sample-a", + "tenant_id": "tenant-a", + "stratum": "candidate_negative_random", + "sampling_probability_ppm": 200_000, + "selection_score_ppm": 12_345, + "source_reference_encrypted": "encrypted:opaque", + "idempotency_key": "audit-sample-request-a", + "payload_fingerprint": "1" * 64, + } + connection.execute(audit_sample_insert, audit_sample_parameters) + rejected( + connection, + audit_sample_insert, + { + **audit_sample_parameters, + "id": "audit-sample-duplicate-observation", + "idempotency_key": "audit-sample-request-duplicate", + "payload_fingerprint": "2" * 64, + }, + ) + rejected( + connection, + audit_sample_insert, + { + **audit_sample_parameters, + "id": "audit-sample-cross-tenant", + "tenant_id": "tenant-b", + "idempotency_key": "audit-sample-request-cross-tenant", + }, + ) + label_parameters = { + "id": "label-a", + "tenant_id": "tenant-a", + "event_fingerprint": "d" * 64, + "actor_fingerprint": "e" * 64, + "idempotency_key": "label-request-a", + "payload_fingerprint": "f" * 64, + } + connection.execute(label_insert, label_parameters) + rejected( + connection, + label_insert, + { + **label_parameters, + "id": "label-cross-tenant", + "tenant_id": "tenant-b", + "idempotency_key": "label-request-cross-tenant", + }, + ) + rejected( + connection, + text( + """ + INSERT INTO agent_asset_release_labels ( + id, tenant_id, observation_id, asset_id, release_id, stage, + version, label, verification_source, source_event_fingerprint, + actor_fingerprint, idempotency_key, payload_fingerprint + ) VALUES ( + 'label-invalid-semantics', 'tenant-a', 'observation-a', + 'asset-a', 'release-a', 'canary', 'v1', 'risk_present', + 'release_review', :event_fingerprint, :actor_fingerprint, + 'label-invalid-semantics', :payload_fingerprint + ) + """ + ), + { + "event_fingerprint": "3" * 64, + "actor_fingerprint": "4" * 64, + "payload_fingerprint": "5" * 64, + }, + ) + connection.execute( + text( + """ + INSERT INTO agent_asset_release_labels ( + id, tenant_id, observation_id, asset_id, release_id, stage, + version, label, verification_source, source_event_fingerprint, + actor_fingerprint, idempotency_key, payload_fingerprint + ) VALUES ( + 'label-blind-a', 'tenant-a', 'observation-a', 'asset-a', + 'release-a', 'canary', 'v1', 'risk_present', + 'blind_release_review', :event_fingerprint, :actor_fingerprint, + 'label-blind-a', :payload_fingerprint + ) + """ + ), + { + "event_fingerprint": "6" * 64, + "actor_fingerprint": "7" * 64, + "payload_fingerprint": "8" * 64, + }, + ) + rejected( + connection, + text( + "UPDATE agent_asset_release_observations " + "SET failure_code = 'tampered' WHERE id = 'observation-a'" + ), + {}, + DBAPIError, + ) + rejected( + connection, + text("DELETE FROM agent_asset_release_labels WHERE id = 'label-a'"), + {}, + DBAPIError, + ) + rejected( + connection, + text( + "UPDATE agent_asset_release_audit_samples " + "SET selection_score_ppm = 999999 WHERE id = 'audit-sample-a'" + ), + {}, + DBAPIError, + ) + finally: + transaction.rollback() diff --git a/server/tests/runtime_chat_testkit.py b/server/tests/runtime_chat_testkit.py new file mode 100644 index 0000000..a2f1e0f --- /dev/null +++ b/server/tests/runtime_chat_testkit.py @@ -0,0 +1,79 @@ +from __future__ import annotations + +from app.services.runtime_chat import RuntimeChatOperationContext, RuntimeChatService +from app.services.runtime_chat_attempts import RuntimeChatAttemptPermit + + +class RecordingAttemptObserver: + def __init__(self, timeline: list[str] | None = None) -> None: + self.permits = [] + self.completed = [] + self.failures = [] + self.timeline = timeline + + def on_permit(self, event): + self.permits.append(event) + if self.timeline is not None: + self.timeline.append(f"permit:{event.identity.attempt}") + return None + + def on_completed(self, event) -> None: + self.completed.append(event) + if self.timeline is not None: + self.timeline.append( + f"completed:{event.identity.attempt}:{event.outcome}" + ) + + def on_observer_failure(self, failure) -> None: + self.failures.append(failure) + + +class FailingCompletionObserver(RecordingAttemptObserver): + def on_completed(self, event) -> None: + super().on_completed(event) + raise RuntimeError("completion observer unavailable") + + +class FailingPermitObserver(RecordingAttemptObserver): + def on_permit(self, event): + super().on_permit(event) + raise RuntimeError("permit observer unavailable") + + +class DenyingAttemptObserver(RecordingAttemptObserver): + def on_permit(self, event): + super().on_permit(event) + return RuntimeChatAttemptPermit( + allowed=False, + reason="tenant quota exhausted", + ) + + +def build_operation_context() -> RuntimeChatOperationContext: + return RuntimeChatOperationContext( + tenant_id="tenant-runtime-chat", + operation_id="operation-runtime-chat-001", + run_id="run-runtime-chat-001", + invocation_seq=7, + attempt_scope="unit-test-completion", + ) + + +def patch_single_slot( + monkeypatch, + service: RuntimeChatService, + *, + provider: str, + model: str, +) -> None: + monkeypatch.setattr( + service, + "_load_chat_slot", + lambda slot: { + "slot": slot, + "provider": provider, + "endpoint": "https://example.com/v1", + "model": model, + "apiKey": "secret", + }, + ) diff --git a/server/tests/savings_migration_assertions.py b/server/tests/savings_migration_assertions.py new file mode 100644 index 0000000..a9a2253 --- /dev/null +++ b/server/tests/savings_migration_assertions.py @@ -0,0 +1,386 @@ +from __future__ import annotations + +from typing import Any + +import pytest +from sqlalchemy import inspect, text +from sqlalchemy.engine import Engine +from sqlalchemy.exc import DBAPIError, IntegrityError + +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) + + +def _assert_savings_head_schema(engine: Engine) -> None: + inspector = inspect(engine) + for model in ( + ProfileBaselineSnapshot, + SavingsOpportunity, + SavingsRealization, + SavingsEvidenceLink, + SavingsEvent, + ): + table = model.__table__ + live_columns = { + str(column["name"]): bool(column["nullable"]) + for column in inspector.get_columns(table.name, schema="public") + } + declared_columns = { + column.name: bool(column.nullable) for column in table.columns + } + assert live_columns == declared_columns + + live_constraint_names = { + str(item["name"]) + for loader in ( + inspector.get_unique_constraints, + inspector.get_check_constraints, + inspector.get_foreign_keys, + ) + for item in loader(table.name, schema="public") + if item.get("name") + } + declared_constraint_names = { + str(constraint.name) + for constraint in table.constraints + if constraint.name is not None + } + assert live_constraint_names == declared_constraint_names + + live_index_names = { + str(item["name"]) + for item in inspector.get_indexes(table.name, schema="public") + if not item.get("duplicates_constraint") + } + declared_index_names = {str(index.name) for index in table.indexes} + assert live_index_names == declared_index_names + + for table_name in ("savings_opportunities", "savings_realizations"): + constrained_columns = { + str(column) + for foreign_key in inspector.get_foreign_keys(table_name, schema="public") + for column in foreign_key["constrained_columns"] + } + assert "claim_id" not in constrained_columns + assert "claim_item_id" not in constrained_columns + + with engine.connect() as connection: + canonical_index = str( + connection.scalar( + text( + "SELECT indexdef FROM pg_indexes " + "WHERE schemaname = 'public' " + "AND tablename = 'savings_realizations' " + "AND indexname = " + "'uq_savings_realizations_actual_canonical_benefit'" + ) + ) + or "" + ).lower() + trigger_count = int( + connection.scalar( + text( + "SELECT COUNT(*) FROM pg_trigger trigger " + "JOIN pg_class relation ON relation.oid = trigger.tgrelid " + "WHERE relation.relname = 'savings_events' " + "AND trigger.tgname = 'trg_savings_events_append_only' " + "AND NOT trigger.tgisinternal" + ) + ) + or 0 + ) + assert "unique index" in canonical_index + assert "realization_type" in canonical_index + assert "actual" in canonical_index + assert "dedupe_status" in canonical_index + assert "canonical" in canonical_index + assert trigger_count == 1 + + +def _assert_savings_runtime_invariants(engine: Engine) -> None: + opportunity_insert = text( + """ + INSERT INTO savings_opportunities ( + id, tenant_id, opportunity_key, benefit_key, expense_case_id, + claim_id, claim_no_snapshot, source_type, source_id, category, + value_kind, title, description, exposure_amount, + baseline_snapshot_id, baseline_amount, target_amount, + estimated_gross, estimated_cost, estimated_net, + estimated_low, estimated_high, confidence, currency, + reporting_currency, attribution_method, suggested_action, + owner_id, owner_name, owner_role + ) VALUES ( + :id, :tenant_id, :opportunity_key, :benefit_key, :expense_case_id, + 'claim-soft-ref', 'CLM-SNAPSHOT', 'policy_adjustment', :id, 'lodging', + 'cash', '住宿标准优化', '服务器政策反事实', 20, + :baseline_snapshot_id, 100, 80, 20, 0, 20, 15, 25, 1, + 'CNY', 'CNY', 'policy_counterfactual', '按职级标准重算', + 'finance-owner', '财务负责人', 'finance' + ) + """ + ) + realization_insert = text( + """ + INSERT INTO savings_realizations ( + id, tenant_id, realization_key, opportunity_id, expense_case_id, + claim_id, realization_type, reversal_of_realization_id, realized_at, + recorded_by_id, recorded_by_name, actual_gross, incremental_cost, + actual_net, original_currency, reporting_amount, reporting_currency, + fx_rate, fx_source, fx_date, fx_version, attribution_method, + attribution_ratio, benefit_key, dedupe_status, + canonical_realization_id, status + ) VALUES ( + :id, 'tenant-a', :id, :opportunity_id, 'case-a', 'claim-soft-ref', + :realization_type, :reversal_of_realization_id, now(), + 'recorder-a', '填报人', :actual_gross, 0, :actual_net, + 'CNY', :reporting_amount, 'CNY', 1, 'system-fixed', current_date, + 'fx-cny-v1', 'direct', 1, :benefit_key, :dedupe_status, + :canonical_realization_id, 'pending_confirmation' + ) + """ + ) + + def execute_rejected( + connection: Any, + statement: Any, + parameters: dict[str, Any], + error_type: type[DBAPIError] = IntegrityError, + ) -> None: + savepoint = connection.begin_nested() + try: + with pytest.raises(error_type): + connection.execute(statement, parameters) + finally: + if savepoint.is_active: + savepoint.rollback() + + with engine.connect() as connection: + transaction = connection.begin() + try: + connection.execute( + text( + """ + INSERT INTO expense_cases ( + id, tenant_id, case_no, scene_code, title, current_stage, status + ) VALUES ( + 'case-a', 'tenant-a', 'CASE-A', 'expense_reimbursement', + '迁移约束探针', 'payment', 'active' + ) + """ + ) + ) + connection.execute( + text( + """ + INSERT INTO profile_baseline_snapshots ( + id, tenant_id, baseline_key, baseline_type, dimension_type, + dimension_id, metric_key, unit, original_currency, + baseline_value, sample_count, method, query_fingerprint, + data_quality_status, data_quality_score, algorithm_version, + policy_version, policy_effective_from, target_resource_type, + target_resource_id, frozen_at, frozen_by + ) VALUES ( + 'baseline-a', 'tenant-a', 'baseline-a', + 'policy_counterfactual', 'claim_item', 'item-soft-ref', + 'approved_amount', 'currency', 'CNY', 100, 0, + 'server_policy', 'query-fingerprint-a', 'complete', 1, + 'policy-calculator-v1', 'policy-v1', current_date, + 'claim_item', 'item-soft-ref', now(), 'system' + ) + """ + ) + ) + connection.execute( + opportunity_insert, + { + "id": "opportunity-a", + "tenant_id": "tenant-a", + "opportunity_key": "opportunity-key-a", + "benefit_key": "benefit-a", + "expense_case_id": "case-a", + "baseline_snapshot_id": "baseline-a", + }, + ) + connection.execute( + opportunity_insert, + { + "id": "opportunity-b", + "tenant_id": "tenant-a", + "opportunity_key": "opportunity-key-b", + "benefit_key": "benefit-b", + "expense_case_id": "case-a", + "baseline_snapshot_id": "baseline-a", + }, + ) + execute_rejected( + connection, + opportunity_insert, + { + "id": "cross-tenant-opportunity", + "tenant_id": "tenant-b", + "opportunity_key": "cross-tenant-opportunity", + "benefit_key": "cross-tenant-benefit", + "expense_case_id": "case-a", + "baseline_snapshot_id": "baseline-a", + }, + ) + + common_actual = { + "opportunity_id": "opportunity-a", + "realization_type": "actual", + "reversal_of_realization_id": None, + "actual_gross": 20, + "actual_net": 20, + "reporting_amount": 20, + "canonical_realization_id": None, + } + connection.execute( + realization_insert, + { + **common_actual, + "id": "realization-a", + "benefit_key": "benefit-a", + "dedupe_status": "pending_review", + }, + ) + connection.execute( + realization_insert, + { + **common_actual, + "id": "realization-canonical", + "benefit_key": "benefit-canonical", + "dedupe_status": "canonical", + }, + ) + execute_rejected( + connection, + realization_insert, + { + **common_actual, + "id": "realization-canonical-duplicate", + "benefit_key": "benefit-canonical", + "dedupe_status": "canonical", + }, + ) + execute_rejected( + connection, + realization_insert, + { + **common_actual, + "id": "realization-wrong-canonical", + "benefit_key": "other-benefit", + "dedupe_status": "duplicate", + "canonical_realization_id": "realization-a", + }, + ) + execute_rejected( + connection, + realization_insert, + { + "id": "cross-opportunity-reversal", + "opportunity_id": "opportunity-b", + "realization_type": "reversal", + "reversal_of_realization_id": "realization-a", + "actual_gross": -5, + "actual_net": -5, + "reporting_amount": -5, + "benefit_key": "benefit-a", + "dedupe_status": "pending_review", + "canonical_realization_id": None, + }, + ) + execute_rejected( + connection, + text( + """ + UPDATE savings_realizations + SET status = 'finance_confirmed', + finance_confirmer_id = recorded_by_id, + finance_confirmer_name = '同一填报人', + confirmed_at = now(), confirmation_note = '不应通过' + WHERE id = 'realization-canonical' + """ + ), + {}, + ) + connection.execute( + text( + """ + UPDATE savings_realizations + SET status = 'finance_confirmed', + finance_confirmer_id = 'finance-b', + finance_confirmer_name = '独立财务', + confirmed_at = now(), confirmation_note = '证据完整' + WHERE id = 'realization-canonical' + """ + ) + ) + assert connection.scalar( + text( + "SELECT status FROM savings_realizations " + "WHERE id = 'realization-canonical'" + ) + ) == "finance_confirmed" + connection.execute( + realization_insert, + { + "id": "valid-reversal", + "opportunity_id": "opportunity-a", + "realization_type": "reversal", + "reversal_of_realization_id": "realization-a", + "actual_gross": -5, + "actual_net": -5, + "reporting_amount": -5, + "benefit_key": "benefit-a", + "dedupe_status": "canonical", + "canonical_realization_id": None, + }, + ) + connection.execute( + text( + """ + UPDATE savings_realizations + SET status = 'finance_confirmed', + finance_confirmer_id = recorded_by_id, + finance_confirmer_name = '冲回财务', + confirmed_at = now(), confirmation_note = '全额冲回' + WHERE id = 'valid-reversal' + """ + ) + ) + + connection.execute( + text( + """ + INSERT INTO savings_events ( + id, tenant_id, aggregate_type, aggregate_id, + opportunity_id, action, actor_id, actor_name, actor_type, + request_id, expected_version, result_version, + payload_fingerprint + ) VALUES ( + 'event-a', 'tenant-a', 'opportunity', 'opportunity-a', + 'opportunity-a', 'created', 'system', '系统', 'system', + 'request-a', 0, 1, 'fingerprint-a' + ) + """ + ) + ) + execute_rejected( + connection, + text("UPDATE savings_events SET action = 'tampered' WHERE id = 'event-a'"), + {}, + DBAPIError, + ) + execute_rejected( + connection, + text("DELETE FROM savings_events WHERE id = 'event-a'"), + {}, + DBAPIError, + ) + finally: + transaction.rollback() diff --git a/server/tests/savings_postgres_testkit.py b/server/tests/savings_postgres_testkit.py new file mode 100644 index 0000000..3242ac0 --- /dev/null +++ b/server/tests/savings_postgres_testkit.py @@ -0,0 +1,379 @@ +from __future__ import annotations + +import os +import re +import uuid +from collections.abc import Iterator +from dataclasses import dataclass +from datetime import UTC, date, datetime +from decimal import Decimal +from pathlib import Path + +import pytest +from alembic.config import Config +from sqlalchemy import create_engine +from sqlalchemy.engine import make_url +from sqlalchemy.orm import Session, sessionmaker + +import app.models # noqa: F401 - 注册迁移对应的完整 metadata +from alembic import command +from app.api.deps import CurrentUserContext +from app.core.config import get_settings +from app.db.schema_ownership import create_legacy_schema +from app.models.expense_case import BusinessEvent, ExpenseCase +from app.models.financial_record import ExpenseClaim +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) +from app.models.tenant import Tenant + +SERVER_DIR = Path(__file__).resolve().parents[1] +ALEMBIC_INI_PATH = SERVER_DIR / "alembic.ini" + + +@pytest.fixture(scope="module", name="pg_factory") +def _pg_factory_fixture() -> Iterator[sessionmaker[Session]]: + database_url = _require_disposable_database_url() + previous_database_url = os.environ.get("DATABASE_URL") + os.environ["DATABASE_URL"] = database_url + get_settings.cache_clear() + config = Config(str(ALEMBIC_INI_PATH)) + config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%")) + command.upgrade(config, "head") + + engine = create_engine(database_url, pool_pre_ping=True) + create_legacy_schema(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + try: + yield factory + finally: + engine.dispose() + if previous_database_url is None: + os.environ.pop("DATABASE_URL", None) + else: + os.environ["DATABASE_URL"] = previous_database_url + get_settings.cache_clear() + + +@dataclass(frozen=True, slots=True) +class _SeededOpportunity: + suffix: str + tenant_id: str + opportunity_id: str + expense_case_id: str + claim_id: str + + +@dataclass(frozen=True, slots=True) +class _SeededRealization(_SeededOpportunity): + realization_id: str + + +@dataclass(frozen=True, slots=True) +class _SeededPayment(_SeededOpportunity): + payment_event_id: str + + +def _seed_opportunity( + factory: sessionmaker[Session], + *, + tenant_id: str = "tenant-savings-concurrency", + status: str = "identified", + owner_id: str = "finance-owner", + benefit_key: str | None = None, + claim_id: str | None = None, +) -> _SeededOpportunity: + suffix = uuid.uuid4().hex[:12] + now = datetime.now(UTC) + case_id = str(uuid.uuid4()) + baseline_id = str(uuid.uuid4()) + opportunity_id = str(uuid.uuid4()) + claim_id = claim_id or str(uuid.uuid4()) + with factory.begin() as db: + db.add_all( + [ + ExpenseCase( + id=case_id, + tenant_id=tenant_id, + case_no=f"CASE-SAV-{suffix}", + scene_code="travel", + title="Savings PostgreSQL 并发验证", + current_stage="claiming", + status="active", + created_at=now, + updated_at=now, + ), + ProfileBaselineSnapshot( + id=baseline_id, + tenant_id=tenant_id, + baseline_key=f"baseline-{suffix}", + baseline_type="policy_counterfactual", + dimension_type="expense_claim_item", + dimension_id=f"item-{suffix}", + metric_key="pre_adjustment_reimbursable_amount", + unit="currency", + original_currency="CNY", + baseline_value=Decimal("100.0000"), + sample_count=1, + method="postgres_concurrency_probe", + query_fingerprint=f"sha256:{uuid.uuid4().hex}", + data_quality_status="complete", + data_quality_score=Decimal("1.0000"), + quality_issues_json=[], + algorithm_version="test-v1", + policy_version="policy-test-v1", + policy_effective_from=date(2026, 1, 1), + target_resource_type="expense_claim_item", + target_resource_id=f"item-{suffix}", + frozen_at=now, + frozen_by="postgres-test", + version=1, + created_at=now, + ), + ] + ) + db.flush() + db.add( + SavingsOpportunity( + id=opportunity_id, + tenant_id=tenant_id, + opportunity_key=f"opportunity-{suffix}", + benefit_key=benefit_key or f"benefit-{suffix}", + expense_case_id=case_id, + claim_id=claim_id, + claim_no_snapshot=f"BX-{suffix}", + source_type="standard_adjustment", + source_id=f"source-{suffix}", + category="policy_compliance", + value_kind="cash", + title="住宿标准重算", + description="真实 PostgreSQL 并发验证机会", + exposure_amount=Decimal("100.0000"), + baseline_snapshot_id=baseline_id, + baseline_amount=Decimal("100.0000"), + target_amount=Decimal("0.0000"), + estimated_gross=Decimal("100.0000"), + estimated_cost=Decimal("0.0000"), + estimated_net=Decimal("100.0000"), + estimated_low=Decimal("100.0000"), + estimated_high=Decimal("100.0000"), + confidence=Decimal("1.0000"), + currency="CNY", + reporting_currency="CNY", + attribution_method="server_policy_counterfactual", + suggested_action="付款后确认", + owner_id=owner_id, + owner_name=owner_id, + owner_role="finance", + status=status, + version=2 if status == "realized" else 1, + dimension_json={}, + baseline_snapshot_json={"baseline_value": "100.0000"}, + evidence_json=[], + accepted_at=now if status in {"accepted", "in_progress", "realized"} else None, + started_at=now if status in {"in_progress", "realized"} else None, + realized_at=now if status == "realized" else None, + created_at=now, + updated_at=now, + ) + ) + return _SeededOpportunity(suffix, tenant_id, opportunity_id, case_id, claim_id) + + +def _seed_pending_realization( + factory: sessionmaker[Session], + *, + tenant_id: str = "tenant-savings-concurrency", + benefit_key: str | None = None, + owner_id: str, + recorder_id: str, +) -> _SeededRealization: + seed = _seed_opportunity( + factory, + tenant_id=tenant_id, + status="realized", + owner_id=owner_id, + benefit_key=benefit_key, + ) + realization_id = str(uuid.uuid4()) + now = datetime.now(UTC) + with factory.begin() as db: + opportunity = db.get(SavingsOpportunity, seed.opportunity_id) + assert opportunity is not None + evidence_key = f"evidence-{seed.suffix}" + realization = SavingsRealization( + id=realization_id, + tenant_id=tenant_id, + realization_key=f"actual-{seed.suffix}", + opportunity_id=seed.opportunity_id, + expense_case_id=seed.expense_case_id, + claim_id=seed.claim_id, + realization_type="actual", + realized_at=now, + recorded_by_id=recorder_id, + recorded_by_name=recorder_id, + actual_gross=Decimal("100.0000"), + incremental_cost=Decimal("10.0000"), + actual_net=Decimal("90.0000"), + original_currency="CNY", + reporting_amount=Decimal("90.0000"), + reporting_currency="CNY", + fx_rate=Decimal("1.00000000"), + fx_source="same_currency", + fx_date=now.date(), + fx_version="identity-v1", + attribution_method="server_policy_counterfactual", + attribution_ratio=Decimal("1.000000"), + benefit_key=opportunity.benefit_key, + dedupe_status="pending_review", + status="pending_confirmation", + baseline_snapshot_json={"baseline_value": "100.0000"}, + final_snapshot_json={"evidence_level": "business_state"}, + evidence_json=[ + { + "evidence_key": evidence_key, + "role": "payment_business_state", + "resource_type": "business_event", + "verification_status": "unverified", + } + ], + version=1, + created_at=now, + updated_at=now, + ) + db.add_all( + [ + realization, + SavingsEvidenceLink( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + evidence_key=evidence_key, + entity_type="realization", + entity_id=realization_id, + realization_id=realization_id, + evidence_role="payment_business_state", + resource_type="business_event", + resource_id=f"payment-evidence-{seed.suffix}", + source_system="postgres-test", + external_event_id=f"payment-evidence-{seed.suffix}", + content_hash=f"sha256:{uuid.uuid4().hex}", + occurred_at=now, + collected_at=now, + verification_status="unverified", + metadata_json={"evidence_level": "business_state"}, + created_at=now, + ), + ] + ) + return _SeededRealization( + seed.suffix, + seed.tenant_id, + seed.opportunity_id, + seed.expense_case_id, + seed.claim_id, + realization_id, + ) + + +def _seed_payment_case(factory: sessionmaker[Session]) -> _SeededPayment: + claim_id = str(uuid.uuid4()) + seed = _seed_opportunity( + factory, + status="in_progress", + owner_id="finance-payment-owner", + claim_id=claim_id, + ) + now = datetime.now(UTC) + event_id = str(uuid.uuid4()) + with factory.begin() as db: + if db.get(Tenant, seed.tenant_id) is None: + db.add( + Tenant( + tenant_id=seed.tenant_id, + tenant_code=seed.tenant_id, + name="节省并发探针租户", + status="active", + ) + ) + db.flush() + db.add_all( + [ + ExpenseClaim( + id=claim_id, + tenant_id=seed.tenant_id, + claim_no=f"BX-PAY-{seed.suffix}", + employee_name="付款并发测试员工", + department_name="财务部", + expense_type="travel", + reason="付款并发验证", + location="上海", + amount=Decimal("100.00"), + currency="CNY", + invoice_count=1, + occurred_at=now, + status="paid", + risk_flags_json=[], + ), + BusinessEvent( + id=event_id, + tenant_id=seed.tenant_id, + expense_case_id=seed.expense_case_id, + aggregate_type="expense_claim", + aggregate_id=claim_id, + event_type="payment_completed", + event_version=1, + idempotency_key=f"payment-{seed.suffix}", + correlation_id=f"payment-{seed.suffix}", + actor_id="payment-actor", + actor_type="user", + payload_json={}, + delivery_status="pending", + occurred_at=now, + ), + ] + ) + return _SeededPayment( + seed.suffix, + seed.tenant_id, + seed.opportunity_id, + seed.expense_case_id, + seed.claim_id, + event_id, + ) + + +def _user( + username: str, + tenant_id: str, + *, + employee_id: str = "", + roles: list[str] | None = None, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=False, + tenant_id=tenant_id, + employee_id=employee_id, + ) + + +def _require_disposable_database_url() -> str: + database_url = os.environ.get("MIGRATION_TEST_DATABASE_URL", "").strip() + if not database_url: + pytest.skip("仅在显式配置 MIGRATION_TEST_DATABASE_URL 时运行 PostgreSQL 并发测试") + parsed = make_url(database_url) + host = re.sub(r"[^a-z0-9]+", "-", str(parsed.host or "").lower()).strip("-") + database = re.sub(r"[^a-z0-9]+", "-", str(parsed.database or "").lower()).strip("-") + if parsed.get_backend_name() != "postgresql": + raise RuntimeError("Savings 并发测试只允许连接 PostgreSQL 一次性数据库") + allowed_hosts = host.startswith( + ("migration-probe", "disposable-probe", "x-financial-disposable-probe") + ) + if not allowed_hosts or not database.startswith(("migration-probe", "disposable-probe")): + raise RuntimeError("Savings 并发测试数据库主机和库名必须使用 disposable-probe 前缀") + return database_url diff --git a/server/tests/test_agent_asset_release_guard.py b/server/tests/test_agent_asset_release_guard.py new file mode 100644 index 0000000..2430546 --- /dev/null +++ b/server/tests/test_agent_asset_release_guard.py @@ -0,0 +1,245 @@ +from __future__ import annotations + +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.db.base import Base +from app.models.agent_asset import AgentAsset, AgentAssetTestRun, AgentAssetVersion +from app.services.agent_asset_release_guard import ( + AgentAssetReleaseGuardService, + ReleaseEvaluationInput, + ReleaseGuardPolicy, +) + + +def _session() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False)() + + +def _seed_asset(db: Session) -> AgentAsset: + asset = AgentAsset( + id="asset-release", + asset_type="task", + code="task.release.guard", + name="发布门禁任务", + domain="expense", + owner="tester", + status="active", + current_version="v1", + working_version="v2", + published_version="v1", + config_json={"business_config": "preserved"}, + ) + db.add(asset) + db.add_all( + [ + AgentAssetVersion( + asset_id=asset.id, + version=version, + content="{}", + content_type="json", + created_by="tester", + ) + for version in ("v1", "v2") + ] + ) + db.commit() + return asset + + +def _policy() -> ReleaseGuardPolicy: + return ReleaseGuardPolicy( + shadow_min_samples=2, + canary_min_samples=3, + max_error_rate=0.1, + min_precision=0.9, + max_precision_drop=0.05, + canary_traffic_percent=10, + recall_gate_enabled=False, + ) + + +def test_release_moves_shadow_canary_active_and_preserves_previous_version() -> None: + with _session() as db: + asset = _seed_asset(db) + service = AgentAssetReleaseGuardService(db) + + shadow = service.start_shadow(asset.id, "v2", actor="manager", policy=_policy()) + assert shadow["stage"] == "shadow" + assert shadow["previous_version"] == "v1" + assert db.get(AgentAsset, asset.id).config_json["business_config"] == "preserved" + assert service.get_serving_plan(asset.id) == { + "stage": "shadow", + "primary_version": "v1", + "candidate_version": "v2", + "candidate_traffic_percent": 0, + "shadow_evaluation": True, + } + + service.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=2, failure_count=0, precision=0.99), + actor="evaluator", + ) + canary = service.promote(asset.id, actor="manager") + assert canary["stage"] == "canary" + assert canary["previous_version"] == "v1" + assert service.get_serving_plan(asset.id)["candidate_traffic_percent"] == 10 + + service.record_evaluation( + asset.id, + ReleaseEvaluationInput( + total=3, + failure_count=0, + precision=0.98, + baseline_precision=0.99, + ), + actor="evaluator", + ) + active = service.promote(asset.id, actor="manager") + + refreshed = db.get(AgentAsset, asset.id) + assert active["stage"] == "active" + assert active["previous_version"] == "v1" + assert refreshed.published_version == "v2" + assert service.get_serving_plan(asset.id)["primary_version"] == "v2" + + +def test_release_cannot_promote_while_samples_are_collecting() -> None: + with _session() as db: + asset = _seed_asset(db) + service = AgentAssetReleaseGuardService(db) + service.start_shadow(asset.id, "v2", actor="manager", policy=_policy()) + + result = service.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=1, failure_count=0, precision=0.99), + actor="evaluator", + ) + + assert result["status"] == "collecting" + try: + service.promote(asset.id, actor="manager") + except PermissionError as exc: + assert "尚无通过" in str(exc) + else: + raise AssertionError("collecting 状态不应允许晋级") + + +def test_active_quality_regression_automatically_restores_previous_version() -> None: + with _session() as db: + asset = _seed_asset(db) + service = AgentAssetReleaseGuardService(db) + service.start_shadow(asset.id, "v2", actor="manager", policy=_policy()) + service.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=2, failure_count=0, precision=0.99), + actor="evaluator", + ) + service.promote(asset.id, actor="manager") + service.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=3, failure_count=0, precision=0.99), + actor="evaluator", + ) + service.promote(asset.id, actor="manager") + + result = service.record_evaluation( + asset.id, + ReleaseEvaluationInput( + total=10, + failure_count=2, + precision=0.8, + baseline_precision=0.99, + ), + actor="monitor", + ) + + refreshed = db.get(AgentAsset, asset.id) + assert result["status"] == "failed" + assert result["release_stage"] == "rolled_back" + assert refreshed.published_version == "v1" + assert refreshed.config_json["release_guard"]["previous_version"] == "v1" + assert refreshed.config_json["release_guard"]["rollback"]["automatic"] is True + assert service.get_serving_plan(asset.id)["primary_version"] == "v1" + + +def test_invalid_or_missing_precision_fails_closed_and_writes_test_run() -> None: + with _session() as db: + asset = _seed_asset(db) + service = AgentAssetReleaseGuardService(db) + service.start_shadow(asset.id, "v2", actor="manager", policy=_policy()) + + result = service.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=2, failure_count=0, precision=None), + actor="evaluator", + ) + + run = db.query(AgentAssetTestRun).filter_by(asset_id=asset.id).one() + assert result["status"] == "failed" + assert result["release_stage"] == "rolled_back" + assert run.passed is False + assert "precision_metric_missing" in run.result_json["reasons"] + + +def test_malformed_policy_and_metric_counts_fail_closed() -> None: + with _session() as db: + asset = _seed_asset(db) + service = AgentAssetReleaseGuardService(db) + service.start_shadow( + asset.id, + "v2", + actor="manager", + policy=ReleaseGuardPolicy( + shadow_min_samples="invalid", # type: ignore[arg-type] + min_precision="invalid", # type: ignore[arg-type] + reviewer_quorum=99, + ), + ) + + state = service.get_state(asset.id) + assert state["policy"]["shadow_min_samples"] == 20 + assert state["policy"]["min_precision"] == 0.98 + assert state["policy"]["reviewer_quorum"] == 2 + assert ReleaseGuardPolicy(reviewer_quorum=0).to_dict()["reviewer_quorum"] == 1 + + result = service.record_evaluation( + asset.id, + ReleaseEvaluationInput( + total="invalid", # type: ignore[arg-type] + failure_count=0, + precision=0.99, + ), + actor="evaluator", + ) + + assert result["status"] == "failed" + assert result["release_stage"] == "rolled_back" + assert "invalid_evaluation_metrics" in result["reasons"] + + +def test_restarted_release_cannot_reuse_previous_attempt_quality_gate() -> None: + with _session() as db: + asset = _seed_asset(db) + service = AgentAssetReleaseGuardService(db) + first = service.start_shadow(asset.id, "v2", actor="manager", policy=_policy()) + service.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=2, failure_count=0, precision=0.99), + actor="evaluator", + ) + service.rollback(asset.id, actor="manager", reason="restart") + second = service.start_shadow(asset.id, "v2", actor="manager", policy=_policy()) + + assert first["release_id"] != second["release_id"] + with pytest.raises(PermissionError, match="尚无通过"): + service.promote(asset.id, actor="manager") diff --git a/server/tests/test_agent_asset_release_monitor.py b/server/tests/test_agent_asset_release_monitor.py new file mode 100644 index 0000000..8381699 --- /dev/null +++ b/server/tests/test_agent_asset_release_monitor.py @@ -0,0 +1,526 @@ +from __future__ import annotations + +import inspect +from collections.abc import Generator +from datetime import UTC, datetime, timedelta + +import pytest +from sqlalchemy import create_engine, select, update +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.db.base import Base +from app.models.agent_asset import AgentAsset, AgentAssetTestRun +from app.models.agent_asset_release_telemetry import AgentAssetReleaseObservation +from app.services.agent_asset_release_monitor import AgentAssetReleaseMonitor +from app.services.agent_asset_release_review import AgentAssetReleaseReviewService +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseObservationInput, +) + + +@pytest.fixture +def db() -> Generator[Session, None, None]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as session: + yield session + engine.dispose() + + +def _policy( + *, + min_precision: float = 0.8, + max_error_rate: float = 0.25, +) -> dict: + return { + "shadow_min_samples": 1, + "canary_min_samples": 1, + "max_error_rate": max_error_rate, + "min_precision": min_precision, + "max_precision_drop": 0.2, + "canary_traffic_percent": 10, + } + + +def _seed_asset( + db: Session, + *, + asset_id: str = "release-monitor-asset", + tenant_id: str | None = "tenant-a", + stage: str = "shadow", + candidate_version: str = "v2", + min_precision: float = 0.8, + max_error_rate: float = 0.25, + detail_mode: str = "json_risk", +) -> AgentAsset: + previous_config = { + "detail_mode": detail_mode, + "enabled": True, + "stable_marker": f"stable:{asset_id}", + } + if tenant_id is not None: + previous_config["tenant_id"] = tenant_id + state = { + "release_id": f"release-{asset_id}", + "stage": stage, + "candidate_version": candidate_version, + "previous_version": "v1", + "policy": _policy( + min_precision=min_precision, + max_error_rate=max_error_rate, + ), + "previous_config": dict(previous_config), + "history": [], + } + asset = AgentAsset( + id=asset_id, + tenant_id=tenant_id or "platform", + scope="tenant" if tenant_id is not None else "platform", + asset_type=AgentAssetType.RULE.value, + code=f"risk.{asset_id}", + name="发布监控规则", + description="", + domain=AgentAssetDomain.EXPENSE.value, + scenario_json=["travel"], + owner="finance", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={ + **previous_config, + "release_guard": state, + # 外部或客户端放入配置的数字不是发布评测输入。 + "untrusted_external_metrics": { + "total": 999999, + "precision": 1.0, + }, + }, + ) + db.add(asset) + db.commit() + return asset + + +def _record( + db: Session, + asset: AgentAsset, + *, + source: str, + candidate_hit: bool, + baseline_hit: bool | None = True, + label: str | None = None, + failed: bool = False, +) -> str: + state = dict((asset.config_json or {})["release_guard"]) + service = AgentAssetReleaseTelemetryService(db) + observation = service.record_observation( + ReleaseObservationInput( + tenant_id=str((asset.config_json or {}).get("tenant_id") or "tenant-a"), + asset_id=asset.id, + release_id=str(state["release_id"]), + stage=str(state["stage"]), # type: ignore[arg-type] + version=str(state["candidate_version"]), + rule_code=asset.code, + source_key=source, + candidate_hit=candidate_hit, + baseline_hit=baseline_hit, + runtime_status="failed" if failed else "completed", + failure_code="evaluator_error" if failed else "none", + ) + ) + if label is not None: + service.record_review_label( + tenant_id=observation.tenant_id, + observation_id=observation.id, + label=label, # type: ignore[arg-type] + request_id=f"review:{source}", + actor_id="trusted-auditor", + ) + db.commit() + return observation.id + + +class _NeverCalledGuard: + def __init__(self) -> None: + self.calls = 0 + + def record_evaluation(self, *_args, **_kwargs): + self.calls += 1 + raise AssertionError("collecting telemetry must never call Release Guard") + + +def test_collecting_never_calls_guard_or_rolls_back(db: Session) -> None: + asset = _seed_asset(db) + _record( + db, + asset, + source="claim-unlabeled", + candidate_hit=True, + label=None, + ) + guard = _NeverCalledGuard() + monitor = AgentAssetReleaseMonitor(db, guard_service=guard) # type: ignore[arg-type] + + result = monitor.evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + + refreshed = db.get(AgentAsset, asset.id) + assert result["status"] == "collecting" + assert result["telemetry_status"] == "collecting" + assert result["evaluation_submitted"] is False + assert result["metrics"]["candidate_pending_label_count"] == 1 + assert result["metrics"]["precision"] is None + assert result["rolled_back"] is False + assert guard.calls == 0 + assert refreshed.config_json["release_guard"]["stage"] == "shadow" + assert db.scalar(select(AgentAssetTestRun)) is None + + +def test_ready_metrics_are_evaluated_and_passed(db: Session) -> None: + asset = _seed_asset(db) + _record( + db, + asset, + source="claim-confirmed", + candidate_hit=True, + label="confirmed", + ) + + result = AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + run = db.scalar(select(AgentAssetTestRun).where(AgentAssetTestRun.asset_id == asset.id)) + + assert result["telemetry_status"] == "ready" + assert result["status"] == "passed" + assert result["evaluation_submitted"] is True + assert result["rolled_back"] is False + assert result["metrics"]["observed_count"] == 1 + assert result["metrics"]["precision"] == 1.0 + assert result["metrics"]["baseline_precision"] == 1.0 + assert result["metrics"]["recall"] == 1.0 + assert result["metrics"]["recall_lower_bound"] == 1.0 + assert run is not None + assert run.input_json["total"] == 1 + assert run.input_json["failure_count"] == 0 + assert run.input_json["precision"] == 1.0 + assert run.result_json["details"]["metric_source"] == "release_runtime_telemetry" + + +def test_same_ready_snapshot_reuses_test_run(db: Session) -> None: + asset = _seed_asset(db) + _record( + db, + asset, + source="claim-idempotent", + candidate_hit=True, + label="confirmed", + ) + monitor = AgentAssetReleaseMonitor(db) + + first = monitor.evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + second = monitor.evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor-retry", + ) + runs = list( + db.scalars( + select(AgentAssetTestRun).where(AgentAssetTestRun.asset_id == asset.id) + ).all() + ) + + assert second == first + assert len(runs) == 1 + + +def test_real_low_precision_automatically_rolls_back(db: Session) -> None: + asset = _seed_asset(db, min_precision=0.9, max_error_rate=1.0) + _record(db, asset, source="claim-true", candidate_hit=True, label="confirmed") + _record( + db, + asset, + source="claim-false-positive", + candidate_hit=True, + label="false_positive", + ) + + result = AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + refreshed = db.get(AgentAsset, asset.id) + + assert result["status"] == "failed" + assert result["rolled_back"] is True + assert result["release_stage"] == "rolled_back" + assert result["metrics"]["precision"] == 0.5 + assert "precision_below_threshold" in result["reasons"] + assert refreshed.published_version == "v1" + assert refreshed.config_json["stable_marker"] == f"stable:{asset.id}" + assert refreshed.config_json["release_guard"]["rollback"]["automatic"] is True + + +def test_real_runtime_failure_automatically_rolls_back(db: Session) -> None: + asset = _seed_asset(db, min_precision=0.5, max_error_rate=0.1) + _record(db, asset, source="claim-good", candidate_hit=True, label="confirmed") + _record( + db, + asset, + source="claim-evaluator-failed", + candidate_hit=False, + baseline_hit=None, + failed=True, + ) + + result = AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + + assert result["status"] == "failed" + assert result["rolled_back"] is True + assert result["metrics"]["observed_count"] == 2 + assert result["metrics"]["runtime_failure_count"] == 1 + assert result["metrics"]["runtime_failure_rate"] == 0.5 + assert "error_rate_exceeded" in result["reasons"] + assert {item["code"] for item in result["alerts"]} >= { + "runtime_failures_detected", + "release_auto_rolled_back", + } + + +def test_all_runtime_failures_without_hits_automatically_roll_back(db: Session) -> None: + asset = _seed_asset(db, min_precision=0.5, max_error_rate=0.1) + _record( + db, + asset, + source="claim-only-evaluator-failed", + candidate_hit=False, + baseline_hit=None, + failed=True, + ) + + result = AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + + assert result["telemetry_status"] == "ready" + assert result["status"] == "failed" + assert result["rolled_back"] is True + assert result["metrics"]["candidate_hit_count"] == 0 + assert result["metrics"]["precision"] is None + assert result["metrics"]["runtime_failure_count"] == 1 + assert "error_rate_exceeded" in result["reasons"] + + +def test_incomplete_labels_with_tolerated_failure_cannot_pass(db: Session) -> None: + asset = _seed_asset(db, min_precision=0.5, max_error_rate=1.0) + _record( + db, + asset, + source="claim-unlabeled-hit", + candidate_hit=True, + label=None, + ) + _record( + db, + asset, + source="claim-tolerated-evaluator-failed", + candidate_hit=False, + baseline_hit=None, + failed=True, + ) + + result = AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + + assert result["telemetry_status"] == "ready" + assert result["status"] == "collecting" + assert result["rolled_back"] is False + assert result["metrics"]["candidate_pending_label_count"] == 1 + assert "precision_metric_missing" in result["reasons"] + assert "release_labels_pending" in {item["code"] for item in result["alerts"]} + + +def test_cross_tenant_and_global_asset_scope_are_enforced(db: Session) -> None: + tenant_asset = _seed_asset(db) + _record(db, tenant_asset, source="tenant-a-claim", candidate_hit=True, label="confirmed") + monitor = AgentAssetReleaseMonitor(db) + + with pytest.raises(LookupError): + monitor.evaluate_current( + tenant_id="tenant-b", + asset_id=tenant_asset.id, + actor="foreign-monitor", + ) + assert monitor.batch_evaluate(tenant_id="tenant-b", actor="foreign-monitor")["scanned"] == 0 + + global_asset = _seed_asset( + db, + asset_id="global-release-monitor", + tenant_id=None, + ) + with pytest.raises(LookupError): + _record(db, global_asset, source="global-claim", candidate_hit=True, label="confirmed") + with pytest.raises(LookupError): + monitor.evaluate_current( + tenant_id="tenant-a", + asset_id=global_asset.id, + actor="tenant-monitor", + ) + with pytest.raises(ValueError, match="cross-tenant aggregate"): + monitor.evaluate_current( + tenant_id="tenant-a", + asset_id=global_asset.id, + actor="platform-monitor", + allow_global_management=True, + ) + assert global_asset.config_json["release_guard"]["stage"] == "shadow" + + +def test_batch_scan_is_bounded_tenant_safe_and_failure_isolated(db: Session) -> None: + collecting_asset = _seed_asset(db, asset_id="batch-collecting") + _record(db, collecting_asset, source="collecting", candidate_hit=True) + + broken_asset = _seed_asset( + db, + asset_id="batch-error", + candidate_version="", + ) + passed_asset = _seed_asset(db, asset_id="batch-passed") + _record(db, passed_asset, source="passed", candidate_hit=True, label="confirmed") + rolled_asset = _seed_asset( + db, + asset_id="batch-rolled-back", + min_precision=0.9, + max_error_rate=1.0, + ) + _record(db, rolled_asset, source="roll-true", candidate_hit=True, label="confirmed") + _record( + db, + rolled_asset, + source="roll-false", + candidate_hit=True, + label="false_positive", + ) + foreign_asset = _seed_asset(db, asset_id="foreign-ready", tenant_id="tenant-b") + _record( + db, + foreign_asset, + source="foreign", + candidate_hit=True, + label="confirmed", + ) + _seed_asset(db, asset_id="not-json", detail_mode="markdown") + _seed_asset(db, asset_id="not-current", stage="rolled_back") + monitor = AgentAssetReleaseMonitor(db) + + result = monitor.batch_evaluate( + tenant_id="tenant-a", + actor="batch-monitor", + limit=10, + ) + + assert result["scanned"] == 4 + assert result["evaluated"] == 2 + assert result["collecting"] == 1 + assert result["rolled_back"] == 1 + assert len(result["errors"]) == 1 + assert result["errors"][0]["asset_id"] == broken_asset.id + assert result["errors"][0]["alerts"][0]["code"] == "release_aggregation_failed" + assert {item["asset_id"] for item in result["results"]} == { + collecting_asset.id, + passed_asset.id, + rolled_asset.id, + } + assert foreign_asset.id not in str(result) + assert db.get(AgentAsset, passed_asset.id).config_json["release_guard"]["stage"] == "shadow" + assert db.get(AgentAsset, rolled_asset.id).config_json["release_guard"]["stage"] == ( + "rolled_back" + ) + with pytest.raises(ValueError, match="between 1 and 500"): + monitor.batch_evaluate(tenant_id="tenant-a", actor="batch-monitor", limit=501) + + +def test_overdue_review_sample_emits_actionable_alert(db: Session) -> None: + asset = _seed_asset(db) + observation_id = _record( + db, + asset, + source="claim-overdue-review", + candidate_hit=True, + ) + # SQLite 测试库没有 PostgreSQL append-only 触发器；用 Core 语句只调整 + # fixture 时间，避免把“运营超时”测试误写成 ORM 可更新契约。 + db.execute( + update(AgentAssetReleaseObservation) + .where(AgentAssetReleaseObservation.id == observation_id) + .values(created_at=datetime.now(UTC) - timedelta(hours=25)) + ) + db.commit() + + result = AgentAssetReleaseReviewService(db).list_pending( + tenant_id="tenant-a", + asset_id=asset.id, + ) + + assert result["telemetry_status"] == "collecting" + assert result["metrics"]["candidate_oldest_pending_age_seconds"] >= 86_400 + assert "release_labels_overdue" in {item["code"] for item in result["alerts"]} + + +def test_monitor_contract_cannot_accept_fabricated_quality_metrics(db: Session) -> None: + asset = _seed_asset(db, min_precision=0.9, max_error_rate=1.0) + _record( + db, + asset, + source="actual-false-positive", + candidate_hit=True, + label="false_positive", + ) + signature = inspect.signature(AgentAssetReleaseMonitor.evaluate_current) + + assert {"total", "failure_count", "precision", "baseline_precision"}.isdisjoint( + signature.parameters + ) + with pytest.raises(TypeError): + AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="malicious-monitor", + precision=1.0, # type: ignore[call-arg] + ) + + result = AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="release-monitor", + ) + assert result["metrics"]["precision"] == 0.0 + assert result["rolled_back"] is True diff --git a/server/tests/test_agent_asset_release_recall.py b/server/tests/test_agent_asset_release_recall.py new file mode 100644 index 0000000..8dce1f4 --- /dev/null +++ b/server/tests/test_agent_asset_release_recall.py @@ -0,0 +1,155 @@ +from __future__ import annotations + +import pytest + +from app.services.agent_asset_release_guard import ReleaseEvaluationInput +from app.services.agent_asset_release_policy import ( + ReleaseGuardPolicy, + evaluate_release, +) +from app.services.agent_asset_release_recall import estimate_release_recall + + +def test_recall_uses_random_stratum_to_estimate_full_negative_population() -> None: + result = estimate_release_recall( + true_positive_count=90, + disagreement_false_negative_count=2, + random_negative_population_count=800, + random_reviewed_count=80, + random_false_negative_count=1, + ) + + assert result.estimated_false_negative_count == 12.0 + assert result.recall == pytest.approx(90 / 102, abs=1e-6) + assert result.recall_lower_bound is not None + assert result.recall_lower_bound < result.recall + assert result.false_negative_upper_bound is not None + assert result.false_negative_upper_bound > result.estimated_false_negative_count + assert result.method == "stratified_random_audit_wilson_upper_bound" + + +def test_recall_is_exact_when_no_random_negative_stratum_exists() -> None: + result = estimate_release_recall( + true_positive_count=18, + disagreement_false_negative_count=2, + random_negative_population_count=0, + random_reviewed_count=0, + random_false_negative_count=0, + ) + + assert result.recall == 0.9 + assert result.recall_lower_bound == 0.9 + assert result.estimated_false_negative_count == 2.0 + + +def test_recall_remains_unavailable_before_random_audit_evidence_exists() -> None: + result = estimate_release_recall( + true_positive_count=10, + disagreement_false_negative_count=0, + random_negative_population_count=25, + random_reviewed_count=0, + random_false_negative_count=0, + ) + + assert result.recall is None + assert result.recall_lower_bound is None + assert result.estimated_false_negative_count is None + + +def test_zero_true_positives_with_confirmed_miss_has_zero_recall() -> None: + result = estimate_release_recall( + true_positive_count=0, + disagreement_false_negative_count=1, + random_negative_population_count=0, + random_reviewed_count=0, + random_false_negative_count=0, + ) + + assert result.recall == 0.0 + assert result.recall_lower_bound == 0.0 + + +@pytest.mark.parametrize( + "kwargs", + [ + {"random_reviewed_count": 3, "random_negative_population_count": 2}, + {"random_false_negative_count": 2, "random_reviewed_count": 1}, + {"true_positive_count": -1}, + {"confidence_level": 0.92}, + ], +) +def test_recall_rejects_inconsistent_evidence(kwargs: dict[str, int | float]) -> None: + values: dict[str, int | float] = { + "true_positive_count": 1, + "disagreement_false_negative_count": 0, + "random_negative_population_count": 2, + "random_reviewed_count": 1, + "random_false_negative_count": 0, + "confidence_level": 0.95, + } + values.update(kwargs) + + with pytest.raises(ValueError): + estimate_release_recall(**values) # type: ignore[arg-type] + + +def test_release_gate_collects_without_blind_negative_ground_truth() -> None: + result = evaluate_release( + "shadow", + ReleaseEvaluationInput( + total=20, + failure_count=0, + precision=1.0, + details={ + "metric_source": "release_runtime_telemetry", + "negative_ground_truth_status": "insufficient_random_negative_reviews", + "recall_lower_bound": None, + }, + ), + ReleaseGuardPolicy(shadow_min_samples=1).to_dict(), + ) + + assert result["status"] == "collecting" + assert result["reasons"] == ["insufficient_random_negative_reviews"] + + +def test_release_gate_uses_recall_confidence_lower_bound_not_point_estimate() -> None: + policy = ReleaseGuardPolicy(shadow_min_samples=1, min_recall=0.95).to_dict() + result = evaluate_release( + "shadow", + ReleaseEvaluationInput( + total=20, + failure_count=0, + precision=1.0, + details={ + "metric_source": "release_runtime_telemetry", + "negative_ground_truth_status": "available_stratified_random_audit", + "recall": 0.98, + "recall_lower_bound": 0.9, + }, + ), + policy, + ) + + assert result["status"] == "failed" + assert "recall_lower_bound_below_threshold" in result["reasons"] + + +def test_release_gate_passes_when_precision_recall_and_samples_are_sufficient() -> None: + result = evaluate_release( + "shadow", + ReleaseEvaluationInput( + total=20, + failure_count=0, + precision=0.99, + details={ + "metric_source": "release_runtime_telemetry", + "negative_ground_truth_status": "available_stratified_random_audit", + "recall": 0.98, + "recall_lower_bound": 0.96, + }, + ), + ReleaseGuardPolicy(shadow_min_samples=20, min_recall=0.95).to_dict(), + ) + + assert result["status"] == "passed" diff --git a/server/tests/test_agent_asset_release_runtime.py b/server/tests/test_agent_asset_release_runtime.py new file mode 100644 index 0000000..bda3fe3 --- /dev/null +++ b/server/tests/test_agent_asset_release_runtime.py @@ -0,0 +1,846 @@ +from __future__ import annotations + +import time +from collections.abc import Generator +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from auth_helpers import install_legacy_header_auth_override +from fastapi.testclient import TestClient +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.api.deps import get_db +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.db.base import Base +from app.main import create_app +from app.models.agent_asset import AgentAsset, AgentAssetTestRun, AgentAssetVersion +from app.models.agent_asset_release_telemetry import AgentAssetReleaseObservation +from app.models.audit_log import AuditLog +from app.models.financial_record import ExpenseClaim +from app.models.golden_case import GoldenCase +from app.services.agent_asset_release_guard import ( + AgentAssetReleaseGuardService, + ReleaseEvaluationInput, + ReleaseGuardPolicy, +) +from app.services.agent_asset_release_monitor import AgentAssetReleaseMonitor +from app.services.agent_asset_release_monitor_auth import build_release_monitor_signature +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseObservationInput, +) +from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager +from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY +from app.services.agent_assets import AgentAssetService +from app.services.expense_claim_risk_rule_loader import _is_candidate_route +from app.services.expense_claims import ExpenseClaimService + + +def _session_factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False) + + +def _manifest(*, name: str, enabled: bool) -> dict: + return { + "schema_version": "2.0", + "rule_code": "risk.release.runtime", + "name": name, + "description": name, + "evaluator": "template_rule", + "enabled": enabled, + "applies_to": { + "domains": ["expense"], + "business_stages": ["reimbursement"], + "expense_categories": ["travel"], + }, + "template_key": "composite_rule_v1", + "params": { + "template_key": "composite_rule_v1", + "field_keys": ["claim.reason"], + "conditions": [ + { + "id": "missing_exception_reason", + "operator": "not_contains_any", + "fields": ["claim.reason"], + "keywords": ["专项审批"], + } + ], + "hit_logic": {"all": ["missing_exception_reason"]}, + "message_template": name, + }, + "outcomes": { + "pass": {"action": "continue"}, + "fail": {"severity": "high", "action": "manual_review"}, + }, + } + + +def _seed_risk_release( + db: Session, + manager: AgentAssetRuleLibraryManager, + *, + with_golden: bool = True, + with_report: bool = True, +) -> AgentAsset: + base_file = "risk.release.runtime.json" + candidate_file = "risk.release.runtime.v2.json" + manager.write_rule_library_json( + library=RISK_RULES_LIBRARY, + file_name=base_file, + payload=_manifest(name="基线规则", enabled=True), + ) + manager.write_rule_library_json( + library=RISK_RULES_LIBRARY, + file_name=candidate_file, + payload=_manifest(name="候选规则", enabled=False), + ) + asset = AgentAsset( + id="risk-release-runtime", + tenant_id="tenant-a", + scope="tenant", + asset_type=AgentAssetType.RULE.value, + code="risk.release.runtime", + name="基线规则", + description="基线规则", + domain=AgentAssetDomain.EXPENSE.value, + scenario_json=["差旅费"], + owner="finance", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={ + "tenant_id": "tenant-a", + "detail_mode": "json_risk", + "enabled": True, + "rule_library": RISK_RULES_LIBRARY, + "rule_document": {"file_name": base_file}, + "revision_draft": { + "version": "v2", + "base_version": "v1", + "generation_status": "completed", + "rule_document": {"file_name": candidate_file}, + }, + }, + ) + db.add(asset) + db.add_all( + [ + AgentAssetVersion( + asset_id=asset.id, + tenant_id="tenant-a", + scope="tenant", + version=version, + content=f"# {version}", + content_type="markdown", + created_by="pytest", + ) + for version in ("v1", "v2") + ] + ) + if with_report: + db.add( + AgentAssetTestRun( + asset_id=asset.id, + tenant_id="tenant-a", + scope="tenant", + version="v2", + test_type="report", + status="passed", + passed=True, + summary="candidate report passed", + created_by="pytest", + ) + ) + if with_golden: + db.add( + GoldenCase( + case_key="risk-release-runtime-hit", + rule_code=asset.code, + name="普通报销应命中", + values_json={"claim.reason": "普通差旅报销"}, + expected_hit=True, + expected_severity="high", + status="active", + ) + ) + db.commit() + return asset + + +def _claim(claim_id: str) -> ExpenseClaim: + return ExpenseClaim( + id=claim_id, + claim_no=f"RE-{claim_id}", + employee_name="张三", + department_name="研发部", + expense_type="travel", + reason="普通差旅报销", + location="上海", + amount=Decimal("1200"), + currency="CNY", + invoice_count=0, + occurred_at=datetime(2026, 7, 16, tzinfo=UTC), + status="draft", + approval_stage="待提交", + risk_flags_json=[], + ) + + +def _policy() -> ReleaseGuardPolicy: + return ReleaseGuardPolicy( + shadow_min_samples=1, + canary_min_samples=1, + max_error_rate=0.1, + min_precision=0.9, + max_precision_drop=0.05, + canary_traffic_percent=50, + recall_gate_enabled=False, + ) + + +def test_risk_rule_runtime_consumes_shadow_canary_active_and_rollback( + tmp_path, + monkeypatch, +) -> None: + factory = _session_factory() + manager = AgentAssetRuleLibraryManager(rule_root=tmp_path / "rules") + with factory() as db: + asset = _seed_risk_release(db, manager) + from app.services import expense_claim_platform_risk + + monkeypatch.setattr( + expense_claim_platform_risk, + "AgentAssetRuleLibraryManager", + lambda: manager, + ) + guard = AgentAssetReleaseGuardService(db, rule_library_manager=manager) + guard.start_shadow( + asset.id, + "v2", + actor="manager", + tenant_id="tenant-a", + policy=_policy(), + ) + + # 启动后改写候选文件，运行时仍使用启动时冻结的可信快照。 + manager.write_rule_library_json( + library=RISK_RULES_LIBRARY, + file_name="risk.release.runtime.v2.json", + payload=_manifest(name="被篡改但不应生效", enabled=True), + ) + shadow = ExpenseClaimService(db).evaluate_platform_risk_rules( + _claim("shadow-claim"), + tenant_id="tenant-a", + business_stage="reimbursement", + ) + assert [item["label"] for item in shadow["flags"]] == ["基线规则"] + assert shadow["shadow_evaluations"] == [ + { + "asset_id": asset.id, + "rule_code": asset.code, + "rule_version": "v2", + "release_stage": "shadow", + "hit": True, + "severity": "high", + } + ] + shadow_sample = db.query(AgentAssetReleaseObservation).filter_by( + tenant_id="tenant-a", + asset_id=asset.id, + stage="shadow", + version="v2", + ).one() + assert shadow_sample.candidate_hit is True + assert shadow_sample.baseline_hit is True + + guard.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=1, failure_count=0, precision=0.99), + actor="monitor", + tenant_id="tenant-a", + ) + guard.promote(asset.id, actor="manager", tenant_id="tenant-a") + canary_key = next( + f"canary-{index}" + for index in range(1000) + if _is_candidate_route(asset.id, f"canary-{index}", 50) + ) + canary = ExpenseClaimService(db).evaluate_platform_risk_rules( + _claim(canary_key), + tenant_id="tenant-a", + business_stage="reimbursement", + ) + assert canary["flags"][0]["label"] == "候选规则" + assert canary["flags"][0]["rule_version"] == "v2" + assert canary["flags"][0]["release_stage"] == "canary" + canary_sample = db.query(AgentAssetReleaseObservation).filter_by( + tenant_id="tenant-a", + asset_id=asset.id, + stage="canary", + version="v2", + ).one() + assert canary_sample.candidate_hit is True + + guard.record_evaluation( + asset.id, + ReleaseEvaluationInput(total=1, failure_count=0, precision=0.99), + actor="monitor", + tenant_id="tenant-a", + ) + guard.promote(asset.id, actor="manager", tenant_id="tenant-a") + active = ExpenseClaimService(db).evaluate_platform_risk_rules( + _claim("active-claim"), + tenant_id="tenant-a", + business_stage="reimbursement", + ) + assert active["flags"][0]["label"] == "候选规则" + active_sample = db.query(AgentAssetReleaseObservation).filter_by( + tenant_id="tenant-a", + asset_id=asset.id, + stage="active", + version="v2", + ).one() + assert active_sample.candidate_hit is True + assert db.get(AgentAsset, asset.id).published_version == "v2" + + result = guard.record_evaluation( + asset.id, + ReleaseEvaluationInput( + total=10, + failure_count=2, + precision=0.7, + baseline_precision=0.99, + ), + actor="monitor", + tenant_id="tenant-a", + ) + rolled_back = ExpenseClaimService(db).evaluate_platform_risk_rules( + _claim("rolled-back-claim"), + tenant_id="tenant-a", + business_stage="reimbursement", + ) + refreshed = db.get(AgentAsset, asset.id) + assert result["release_stage"] == "rolled_back" + assert rolled_back["flags"][0]["label"] == "基线规则" + assert refreshed.published_version == "v1" + assert refreshed.config_json["rule_document"]["file_name"] == ("risk.release.runtime.json") + + +def test_risk_rule_runtime_blocks_when_candidate_and_stable_snapshots_are_corrupt( + tmp_path, + monkeypatch, +) -> None: + factory = _session_factory() + manager = AgentAssetRuleLibraryManager(rule_root=tmp_path / "rules") + with factory() as db: + asset = _seed_risk_release(db, manager) + from app.services import expense_claim_platform_risk + + monkeypatch.setattr( + expense_claim_platform_risk, + "AgentAssetRuleLibraryManager", + lambda: manager, + ) + guard = AgentAssetReleaseGuardService(db, rule_library_manager=manager) + guard.start_shadow( + asset.id, + "v2", + actor="manager", + tenant_id="tenant-a", + policy=_policy(), + ) + + refreshed = db.get(AgentAsset, asset.id) + config = dict(refreshed.config_json or {}) + state = dict(config["release_guard"]) + state["artifacts"] = { + version: {**dict(artifact), "sha256": "0" * 64} + for version, artifact in dict(state["artifacts"]).items() + } + config["release_guard"] = state + refreshed.config_json = config + db.add(refreshed) + db.commit() + + review = ExpenseClaimService(db).evaluate_platform_risk_rules( + _claim("corrupt-release"), + tenant_id="tenant-a", + business_stage="reimbursement", + ) + + assert len(review["flags"]) == 1 + assert review["flags"][0]["rule_code"] == asset.code + assert review["flags"][0]["severity"] == "critical" + assert review["flags"][0]["action"] == "block" + assert review["flags"][0]["evidence"]["failed_version"] == "v1" + assert review["blocking_reasons"] + + result = AgentAssetReleaseMonitor(db).evaluate_current( + tenant_id="tenant-a", + asset_id=asset.id, + actor="artifact-integrity-monitor", + ) + assert result["status"] == "failed" + assert result["rolled_back"] is True + assert result["metrics"]["runtime_failure_count"] == 1 + + +def test_risk_release_fails_closed_without_report_or_golden(tmp_path) -> None: + manager = AgentAssetRuleLibraryManager(rule_root=tmp_path / "rules") + with _session_factory()() as db: + asset = _seed_risk_release(db, manager, with_report=False) + with pytest.raises(PermissionError, match="测试报告"): + AgentAssetReleaseGuardService(db, rule_library_manager=manager).start_shadow( + asset.id, + "v2", + actor="manager", + tenant_id="tenant-a", + ) + + manager = AgentAssetRuleLibraryManager(rule_root=tmp_path / "rules-2") + with _session_factory()() as db: + asset = _seed_risk_release(db, manager, with_golden=False) + with pytest.raises(PermissionError, match="golden case"): + AgentAssetReleaseGuardService(db, rule_library_manager=manager).start_shadow( + asset.id, + "v2", + actor="manager", + tenant_id="tenant-a", + ) + failed = db.query(AgentAssetTestRun).filter_by(test_type="golden").one() + assert failed.passed is False + + +def _build_http_client() -> tuple[TestClient, sessionmaker[Session]]: + factory = _session_factory() + app = create_app() + install_legacy_header_auth_override(app) + + def override_db() -> Generator[Session, None, None]: + with factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + return TestClient(app), factory + + +def _seed_task_release(factory: sessionmaker[Session]) -> None: + with factory() as db: + asset = AgentAsset( + id="task-release-api", + tenant_id="tenant-a", + scope="tenant", + asset_type=AgentAssetType.TASK.value, + code="task.release.api", + name="发布 API 测试", + domain=AgentAssetDomain.EXPENSE.value, + owner="manager", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={"tenant_id": "tenant-a"}, + ) + db.add(asset) + db.add_all( + [ + AgentAssetVersion( + asset_id=asset.id, + tenant_id="tenant-a", + scope="tenant", + version=version, + content="{}", + content_type="json", + created_by="pytest", + ) + for version in ("v1", "v2") + ] + ) + db.commit() + + +def _seed_global_task_release(factory: sessionmaker[Session]) -> None: + with factory() as db: + asset = AgentAsset( + id="global-task-release-api", + tenant_id="platform", + scope="platform", + asset_type=AgentAssetType.TASK.value, + code="task.release.global", + name="平台共享发布 API 测试", + domain=AgentAssetDomain.EXPENSE.value, + owner="platform", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={}, + ) + db.add(asset) + db.add_all( + [ + AgentAssetVersion( + asset_id=asset.id, + tenant_id="platform", + scope="platform", + version=version, + content="{}", + content_type="json", + created_by="pytest", + ) + for version in ("v1", "v2") + ] + ) + db.commit() + + +def _seed_monitor_risk_release( + factory: sessionmaker[Session], + *, + asset_id: str = "risk-monitor-api", + tenant_id: str = "tenant-a", + min_precision: float = 0.9, +) -> None: + with factory() as db: + previous_config = { + "tenant_id": tenant_id, + "detail_mode": "json_risk", + "enabled": True, + } + db.add( + AgentAsset( + id=asset_id, + tenant_id=tenant_id, + scope="tenant", + asset_type=AgentAssetType.RULE.value, + code=f"risk.{asset_id}", + name="真实发布监控 API 测试", + domain=AgentAssetDomain.EXPENSE.value, + owner="manager", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={ + **previous_config, + "release_guard": { + "release_id": f"release-{asset_id}", + "stage": "shadow", + "candidate_version": "v2", + "previous_version": "v1", + "started_by": "username:manager", + "previous_config": previous_config, + "policy": { + "shadow_min_samples": 1, + "canary_min_samples": 1, + "max_error_rate": 1.0, + "min_precision": min_precision, + "max_precision_drop": 1.0, + "canary_traffic_percent": 10, + }, + "history": [], + }, + }, + ) + ) + db.commit() + + +def _append_release_review_samples( + factory: sessionmaker[Session], + *, + asset_id: str, + labels: list[str], + tenant_id: str = "tenant-a", +) -> dict: + with factory() as db: + asset = db.get(AgentAsset, asset_id) + assert asset is not None + state = dict((asset.config_json or {})["release_guard"]) + telemetry = AgentAssetReleaseTelemetryService(db) + for index, label in enumerate(labels): + observation = telemetry.record_observation( + ReleaseObservationInput( + tenant_id=tenant_id, + asset_id=asset.id, + release_id=str(state["release_id"]), + stage=str(state["stage"]), # type: ignore[arg-type] + version=str(state["candidate_version"]), + rule_code=asset.code, + source_key=f"{state['stage']}-sample-{index}", + candidate_hit=True, + baseline_hit=True, + ) + ) + telemetry.record_review_label( + tenant_id=tenant_id, + observation_id=observation.id, + label=label, # type: ignore[arg-type] + request_id=f"{state['release_id']}:{state['stage']}:review:{index}", + actor_id="trusted-release-reviewer", + ) + db.commit() + return state + + +def _headers(*, role: str = "manager", tenant: str = "tenant-a") -> dict[str, str]: + return { + "x-auth-username": role, + "x-auth-name": role, + "x-auth-role-codes": role, + "x-auth-tenant-id": tenant, + } + + +_TEST_MONITOR_SECRET = "release-monitor-test-secret-32-bytes-minimum" + + +def _monitor_headers( + *, + payload: dict, + state: dict, + role: str = "manager", + tenant: str = "tenant-a", + asset_id: str = "task-release-api", +) -> dict[str, str]: + timestamp = str(int(time.time())) + return { + **_headers(role=role, tenant=tenant), + "x-release-monitor-timestamp": timestamp, + "x-release-monitor-signature": build_release_monitor_signature( + timestamp=timestamp, + tenant_id=tenant, + asset_id=asset_id, + release_id=str(state.get("release_id") or ""), + stage=str(state.get("stage") or ""), + payload=payload, + secret=_TEST_MONITOR_SECRET, + ), + } + + +def test_release_management_http_permissions_tenant_and_auto_rollback(monkeypatch) -> None: + monkeypatch.setenv("AGENT_RELEASE_MONITOR_SECRET", _TEST_MONITOR_SECRET) + client, factory = _build_http_client() + _seed_task_release(factory) + path = "/api/v1/agent-assets/task-release-api/release" + + assert client.get(path, headers=_headers(role="finance")).status_code == 403 + assert client.get(path, headers=_headers(tenant="tenant-b")).status_code == 404 + + started = client.post( + f"{path}/shadow", + headers=_headers(), + json={ + "candidate_version": "v2", + "policy": { + "shadow_min_samples": 1, + "canary_min_samples": 1, + "max_error_rate": 0.1, + "min_precision": 0.9, + "max_precision_drop": 0.05, + "canary_traffic_percent": 10, + }, + }, + ) + assert started.status_code == 200 + assert started.json()["stage"] == "shadow" + release_state = started.json() + + unsigned = client.post( + f"{path}/evaluations", + headers=_headers(), + json={}, + ) + assert unsigned.status_code == 401 + fabricated = {"total": 1, "failure_count": 0, "precision": 1.0} + assert ( + client.post( + f"{path}/evaluations", + headers=_monitor_headers(payload=fabricated, state=release_state), + json=fabricated, + ).status_code + == 422 + ) + + _seed_monitor_risk_release(factory) + risk_path = "/api/v1/agent-assets/risk-monitor-api/release" + risk_state = _append_release_review_samples( + factory, + asset_id="risk-monitor-api", + labels=["confirmed", "false_positive"], + ) + trigger: dict = {} + failed = client.post( + f"{risk_path}/evaluations", + headers=_monitor_headers( + payload=trigger, + state=risk_state, + asset_id="risk-monitor-api", + ), + json=trigger, + ) + assert failed.status_code == 200 + assert failed.json()["status"] == "failed" + assert failed.json()["release_stage"] == "rolled_back" + assert failed.json()["metrics"]["precision"] == 0.5 + plan = client.get(f"{risk_path}/serving-plan", headers=_headers()) + assert plan.json()["primary_version"] == "v1" + + with factory() as db: + assert db.query(AuditLog).filter_by(resource_id="risk-monitor-api").count() >= 1 + + +def test_release_review_queue_is_tenant_safe_and_requires_independent_reviewer() -> None: + client, factory = _build_http_client() + _seed_monitor_risk_release(factory, asset_id="risk-review-api", min_precision=0.5) + with factory() as db: + asset = db.get(AgentAsset, "risk-review-api") + assert asset is not None + state = dict((asset.config_json or {})["release_guard"]) + observation = AgentAssetReleaseTelemetryService(db).record_observation( + ReleaseObservationInput( + tenant_id="tenant-a", + asset_id=asset.id, + release_id=str(state["release_id"]), + stage="shadow", + version="v2", + rule_code=asset.code, + source_key="claim-review-api", + candidate_hit=True, + baseline_hit=True, + ) + ) + db.commit() + observation_id = observation.id + + path = "/api/v1/agent-assets/risk-review-api/release/review-queue" + assert client.get(path, headers=_headers(tenant="tenant-b")).status_code == 404 + queued = client.get(path, headers=_headers()) + assert queued.status_code == 200 + assert queued.json()["pending_total"] == 1 + assert "source_fingerprint" not in queued.text + item = queued.json()["items"][0] + assert item["source_document_id"] == "claim-review-api" + assert item["prediction_blinded"] is True + assert "candidate_hit" not in item + assert "baseline_hit" not in item + + self_review = client.post( + f"{path}/{observation_id}/labels", + headers={**_headers(), "x-request-id": "self-review"}, + json={"label": "confirmed"}, + ) + assert self_review.status_code == 400 + reviewed = client.post( + f"{path}/{observation_id}/labels", + headers={**_headers(role="admin"), "x-request-id": "independent-review"}, + json={"label": "confirmed"}, + ) + assert reviewed.status_code == 200 + assert reviewed.json()["monitor"]["status"] == "passed" + assert client.get(path, headers=_headers(role="admin")).json()["pending_total"] == 0 + + +def test_only_platform_admin_can_manage_global_release_asset() -> None: + client, factory = _build_http_client() + _seed_global_task_release(factory) + path = "/api/v1/agent-assets/global-task-release-api/release" + + assert client.get(path, headers=_headers(role="manager")).status_code == 404 + assert client.get(path, headers=_headers(role="admin")).status_code == 200 + started = client.post( + f"{path}/shadow", + headers=_headers(role="admin"), + json={"candidate_version": "v2"}, + ) + assert started.status_code == 200 + assert started.json()["stage"] == "shadow" + + +def test_existing_publish_http_cannot_bypass_shadow_and_canary( + tmp_path, + monkeypatch, +) -> None: + monkeypatch.setenv("AGENT_RELEASE_MONITOR_SECRET", _TEST_MONITOR_SECRET) + client, factory = _build_http_client() + manager = AgentAssetRuleLibraryManager(rule_root=tmp_path / "rules") + with factory() as db: + _seed_risk_release(db, manager) + + original_init = AgentAssetService.__init__ + + def patched_init(self, db, *, current_user=None) -> None: + original_init(self, db, current_user=current_user) + self.rule_library_manager = manager + + monkeypatch.setattr(AgentAssetService, "__init__", patched_init) + asset_path = "/api/v1/agent-assets/risk-release-runtime" + assert ( + client.post( + f"{asset_path}/publish", + headers=_headers(tenant="tenant-b"), + ).status_code + == 404 + ) + published = client.post(f"{asset_path}/publish", headers=_headers()) + + assert published.status_code == 200 + assert published.json()["published_version"] == "v1" + assert published.json()["current_version"] == "v1" + assert published.json()["config_json"]["release_guard"]["stage"] == "shadow" + assert client.post(f"{asset_path}/activate", headers=_headers()).status_code == 400 + assert ( + client.post( + f"{asset_path}/risk-rule-enabled", + headers=_headers(), + json={"enabled": True}, + ).status_code + == 400 + ) + assert ( + client.patch( + asset_path, + headers=_headers(), + json={"published_version": "v2"}, + ).status_code + == 400 + ) + + release_path = f"{asset_path}/release" + for total, expected_stage in ((20, "canary"), (100, "active")): + release_state = client.get(release_path, headers=_headers()).json() + _append_release_review_samples( + factory, + asset_id="risk-release-runtime", + labels=["confirmed"] * total, + ) + payload: dict = {} + evaluated = client.post( + f"{release_path}/evaluations", + headers=_monitor_headers( + payload=payload, + state=release_state, + asset_id="risk-release-runtime", + ), + json=payload, + ) + assert evaluated.status_code == 200 + promoted = client.post(f"{release_path}/promote", headers=_headers()) + assert promoted.status_code == 200 + assert promoted.json()["stage"] == expected_stage + + with factory() as db: + active = db.get(AgentAsset, "risk-release-runtime") + assert active is not None + assert active.published_version == "v2" diff --git a/server/tests/test_agent_asset_release_scheduler.py b/server/tests/test_agent_asset_release_scheduler.py new file mode 100644 index 0000000..1c00a34 --- /dev/null +++ b/server/tests/test_agent_asset_release_scheduler.py @@ -0,0 +1,172 @@ +from __future__ import annotations + +import pytest +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.db.base import Base +from app.models.agent_asset import AgentAsset, AgentAssetTestRun +from app.services.agent_asset_release_scheduler import AgentAssetReleaseScheduler +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseObservationInput, +) + + +def _asset(asset_id: str, tenant_id: str | None) -> AgentAsset: + previous_config: dict[str, object] = { + "detail_mode": "json_risk", + "enabled": True, + "stable_marker": f"stable:{asset_id}", + } + if tenant_id is not None: + previous_config["tenant_id"] = tenant_id + return AgentAsset( + id=asset_id, + tenant_id=tenant_id or "platform", + scope="tenant" if tenant_id is not None else "platform", + asset_type=AgentAssetType.RULE.value, + code=f"risk.{asset_id}", + name="周期发布监控规则", + description="", + domain=AgentAssetDomain.EXPENSE.value, + scenario_json=["travel"], + owner="finance", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={ + **previous_config, + "release_guard": { + "release_id": f"release-{asset_id}", + "stage": "shadow", + "candidate_version": "v2", + "previous_version": "v1", + "previous_config": previous_config, + "policy": { + "shadow_min_samples": 1, + "canary_min_samples": 1, + "max_error_rate": 1.0, + "min_precision": 0.9, + "max_precision_drop": 1.0, + "canary_traffic_percent": 10, + }, + }, + }, + ) + + +def _false_positive_sample(db: Session, asset: AgentAsset, tenant_id: str) -> None: + telemetry = AgentAssetReleaseTelemetryService(db) + observation = telemetry.record_observation( + ReleaseObservationInput( + tenant_id=tenant_id, + asset_id=asset.id, + release_id=f"release-{asset.id}", + stage="shadow", + version="v2", + rule_code=asset.code, + source_key=f"claim-{asset.id}", + candidate_hit=True, + baseline_hit=True, + ) + ) + telemetry.record_review_label( + tenant_id=tenant_id, + observation_id=observation.id, + label="false_positive", + request_id=f"review-{asset.id}", + actor_id="trusted-reviewer", + ) + + +def test_scheduler_groups_tenants_rolls_back_and_skips_global_assets() -> None: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as db: + tenant_asset = _asset("tenant-release-scheduler", "tenant-a") + global_asset = _asset("global-release-scheduler", None) + db.add_all([tenant_asset, global_asset]) + db.commit() + _false_positive_sample(db, tenant_asset, "tenant-a") + with pytest.raises(LookupError, match="Agent asset not found"): + _false_positive_sample(db, global_asset, "tenant-a") + db.commit() + + result = AgentAssetReleaseScheduler(session_factory=factory)._run_once() + + with factory() as db: + tenant_asset = db.get(AgentAsset, "tenant-release-scheduler") + global_asset = db.get(AgentAsset, "global-release-scheduler") + assert tenant_asset is not None + assert global_asset is not None + assert tenant_asset.config_json["release_guard"]["stage"] == "rolled_back" + assert global_asset.config_json["release_guard"]["stage"] == "shadow" + engine.dispose() + + assert result == { + "tenants": 1, + "scanned": 1, + "evaluated": 1, + "collecting": 0, + "rolled_back": 1, + "errors": 0, + "global_skipped": 1, + } + + +def test_scheduler_cursor_rotates_beyond_fixed_batch_limit() -> None: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as db: + first = _asset("cursor-a", "tenant-a") + second = _asset("cursor-b", "tenant-a") + db.add_all([first, second]) + db.commit() + for asset in (first, second): + telemetry = AgentAssetReleaseTelemetryService(db) + observation = telemetry.record_observation( + ReleaseObservationInput( + tenant_id="tenant-a", + asset_id=asset.id, + release_id=f"release-{asset.id}", + stage="shadow", + version="v2", + rule_code=asset.code, + source_key=f"claim-{asset.id}", + candidate_hit=True, + baseline_hit=True, + ) + ) + telemetry.record_review_label( + tenant_id="tenant-a", + observation_id=observation.id, + label="confirmed", + request_id=f"review-{asset.id}", + actor_id="trusted-reviewer", + ) + db.commit() + + scheduler = AgentAssetReleaseScheduler(session_factory=factory) + scheduler._batch_size = 1 + scheduler._run_once() + scheduler._run_once() + + with factory() as db: + evaluated = set(db.scalars(select(AgentAssetTestRun.asset_id)).all()) + engine.dispose() + + assert evaluated == {"cursor-a", "cursor-b"} diff --git a/server/tests/test_agent_asset_release_telemetry.py b/server/tests/test_agent_asset_release_telemetry.py new file mode 100644 index 0000000..3dfe2ad --- /dev/null +++ b/server/tests/test_agent_asset_release_telemetry.py @@ -0,0 +1,871 @@ +from __future__ import annotations + +from collections.abc import Generator + +import pytest +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.db.base import Base +from app.models.agent_asset import AgentAsset +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) +from app.models.risk_disposition import RiskDisposition, RiskDispositionEvent +from app.models.risk_observation import RiskObservation +from app.services.agent_asset_release_disposition_labels import ( + AgentAssetReleaseDispositionLabelService, +) +from app.services.agent_asset_release_review import AgentAssetReleaseReviewService +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseObservationInput, + ReleaseTelemetryCollecting, + ReleaseTelemetryIdempotencyConflict, + ReleaseTelemetryStaleRelease, +) + + +@pytest.fixture +def db() -> Generator[Session, None, None]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all( + engine, + tables=[ + AgentAsset.__table__, + RiskObservation.__table__, + RiskDisposition.__table__, + RiskDispositionEvent.__table__, + AgentAssetReleaseObservation.__table__, + AgentAssetReleaseLabel.__table__, + AgentAssetReleaseAuditSample.__table__, + ], + ) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as session: + yield session + engine.dispose() + + +def _seed_asset( + db: Session, + *, + stage: str = "shadow", + tenant_id: str = "tenant-a", + release_id: str = "release-1", + reviewer_quorum: int = 1, + recall_gate_enabled: bool = False, + negative_sample_percent: int = 20, + negative_min_reviewed: int = 5, + min_recall: float = 0.95, +) -> AgentAsset: + asset = AgentAsset( + id="release-asset", + asset_type=AgentAssetType.RULE.value, + code="risk.release.telemetry", + name="发布遥测规则", + description="", + domain=AgentAssetDomain.EXPENSE.value, + scenario_json=["travel"], + owner="finance", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={ + "tenant_id": tenant_id, + "detail_mode": "json_risk", + "release_guard": { + "release_id": release_id, + "stage": stage, + "candidate_version": "v2", + "previous_version": "v1", + "started_by": "publisher", + "policy": { + "reviewer_quorum": reviewer_quorum, + "recall_gate_enabled": recall_gate_enabled, + "negative_sample_percent": negative_sample_percent, + "negative_min_reviewed": negative_min_reviewed, + "min_recall": min_recall, + "recall_confidence_level": 0.95, + }, + }, + }, + ) + db.add(asset) + db.commit() + return asset + + +def _shadow_result(*, candidate_hit: bool, baseline_hit: bool) -> dict: + flags = [] + if baseline_hit: + flags.append( + { + "rule_code": "risk.release.telemetry", + "rule_version": "v1", + "release_stage": "shadow", + "release_mode": "enforced", + } + ) + return { + "flags": flags, + "shadow_evaluations": [ + { + "asset_id": "release-asset", + "rule_code": "risk.release.telemetry", + "rule_version": "v2", + "release_stage": "shadow", + "hit": candidate_hit, + "severity": "high" if candidate_hit else "none", + } + ], + } + + +def _record_shadow( + db: Session, + *, + claim_id: str, + candidate_hit: bool = True, + baseline_hit: bool = True, +) -> AgentAssetReleaseObservation: + return AgentAssetReleaseTelemetryService(db).record_expense_risk_result( + tenant_id="tenant-a", + claim_id=claim_id, + result=_shadow_result( + candidate_hit=candidate_hit, + baseline_hit=baseline_hit, + ), + )[0] + + +def test_shadow_runtime_samples_and_labels_build_conservative_release_input( + db: Session, +) -> None: + _seed_asset(db) + service = AgentAssetReleaseTelemetryService(db) + + candidate = _record_shadow(db, claim_id="claim-secret-001") + baseline_only = _record_shadow( + db, + claim_id="claim-secret-002", + candidate_hit=False, + baseline_hit=True, + ) + replay = _record_shadow(db, claim_id="claim-secret-001") + db.commit() + + assert replay.id == candidate.id + assert candidate.candidate_hit is True + assert candidate.baseline_hit is True + assert baseline_only.candidate_hit is False + stored_text = str( + { + column.name: getattr(candidate, column.name) + for column in AgentAssetReleaseObservation.__table__.columns + } + ) + assert "claim-secret" not in stored_text + + collecting = service.aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + ) + assert collecting.status == "collecting" + assert collecting.precision is None + assert collecting.recall is None + assert collecting.false_negative_count is None + assert collecting.negative_ground_truth_status == "collecting_candidate_labels" + with pytest.raises(ReleaseTelemetryCollecting): + collecting.to_release_evaluation_input() + + candidate_label = service.record_review_label( + tenant_id="tenant-a", + observation_id=candidate.id, + label="confirmed", + request_id="review-001", + actor_id="auditor-secret-account", + ) + service.record_review_label( + tenant_id="tenant-a", + observation_id=baseline_only.id, + label="false_positive", + request_id="review-002", + actor_id="auditor-secret-account", + ) + service.record_review_label( + tenant_id="tenant-a", + observation_id=baseline_only.id, + label="false_positive", + request_id="review-002-second", + actor_id="second-auditor-secret-account", + ) + db.commit() + + assert candidate_label.actor_fingerprint != "auditor-secret-account" + assert "auditor-secret-account" not in str(candidate_label.__dict__) + ready = service.aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + ) + assert ready.status == "ready" + assert ready.observed_count == 2 + assert ready.candidate_hit_count == 1 + assert ready.candidate_labeled_count == 1 + assert ready.precision == 1.0 + assert ready.baseline_hit_count == 2 + assert ready.baseline_labeled_count == 2 + assert ready.baseline_precision == 0.5 + + evaluation = ready.to_release_evaluation_input() + assert evaluation.total == 2 + assert evaluation.failure_count == 0 + assert evaluation.precision == 1.0 + assert evaluation.baseline_precision == 0.5 + assert evaluation.details["metric_source"] == "release_runtime_telemetry" + assert evaluation.details["recall"] == 1.0 + assert evaluation.details["recall_lower_bound"] == 1.0 + + +def test_unlabeled_candidate_hit_never_counts_as_release_success(db: Session) -> None: + _seed_asset(db) + service = AgentAssetReleaseTelemetryService(db) + first = _record_shadow(db, claim_id="claim-1") + _record_shadow(db, claim_id="claim-2") + service.record_review_label( + tenant_id="tenant-a", + observation_id=first.id, + label="confirmed", + request_id="review-first", + actor_id="auditor", + ) + db.commit() + + aggregate = service.aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + ) + + assert aggregate.status == "collecting" + assert aggregate.candidate_hit_count == 2 + assert aggregate.candidate_labeled_count == 1 + assert aggregate.candidate_pending_label_count == 1 + assert aggregate.baseline_precision is None + assert "candidate_labels_pending" in aggregate.reasons + with pytest.raises(ReleaseTelemetryCollecting): + aggregate.to_release_evaluation_input() + + +def test_observation_and_label_idempotency_reject_changed_payload(db: Session) -> None: + _seed_asset(db) + service = AgentAssetReleaseTelemetryService(db) + observation = _record_shadow(db, claim_id="claim-1") + + with pytest.raises(ReleaseTelemetryIdempotencyConflict): + service.record_observation( + ReleaseObservationInput( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + rule_code="risk.release.telemetry", + source_key="claim-1", + candidate_hit=False, + baseline_hit=True, + ) + ) + + service.record_review_label( + tenant_id="tenant-a", + observation_id=observation.id, + label="confirmed", + request_id="same-review-request", + actor_id="auditor", + ) + with pytest.raises(ReleaseTelemetryIdempotencyConflict): + service.record_review_label( + tenant_id="tenant-a", + observation_id=observation.id, + label="false_positive", + request_id="same-review-request", + actor_id="auditor", + ) + + +def test_release_label_values_cannot_cross_verification_sources(db: Session) -> None: + _seed_asset(db) + service = AgentAssetReleaseTelemetryService(db) + observation = _record_shadow(db, claim_id="claim-label-semantics") + + with pytest.raises(ValueError, match="release reviews require confirmed"): + service.record_review_label( + tenant_id="tenant-a", + observation_id=observation.id, + label="risk_present", # type: ignore[arg-type] + request_id="wrong-release-review-semantics", + actor_id="auditor", + ) + with pytest.raises(ValueError, match="Blind release review requires"): + service.record_blind_review_label( + tenant_id="tenant-a", + observation_id=observation.id, + ground_truth="confirmed", # type: ignore[arg-type] + request_id="wrong-blind-review-semantics", + actor_id="auditor", + ) + + +def test_observation_rolls_back_when_blind_sample_cannot_be_protected( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + _seed_asset(db) + + def fail_encryption(_value: str) -> str: + raise ValueError("sample encryption unavailable") + + monkeypatch.setattr( + "app.services.agent_asset_release_sampling.encrypt_secret", + fail_encryption, + ) + + with pytest.raises(ValueError, match="sample encryption unavailable"): + _record_shadow(db, claim_id="claim-must-not-leak") + + assert db.query(AgentAssetReleaseObservation).count() == 0 + assert db.query(AgentAssetReleaseAuditSample).count() == 0 + + +def test_cross_tenant_and_stale_release_labels_are_rejected(db: Session) -> None: + asset = _seed_asset(db) + service = AgentAssetReleaseTelemetryService(db) + observation = _record_shadow(db, claim_id="claim-1") + db.commit() + + with pytest.raises(LookupError): + service.record_review_label( + tenant_id="tenant-b", + observation_id=observation.id, + label="confirmed", + request_id="foreign-review", + actor_id="foreign-auditor", + ) + + config = dict(asset.config_json or {}) + state = dict(config["release_guard"]) + state["release_id"] = "release-2" + config["release_guard"] = state + asset.config_json = config + db.add(asset) + db.commit() + + with pytest.raises(ReleaseTelemetryStaleRelease): + service.record_review_label( + tenant_id="tenant-a", + observation_id=observation.id, + label="confirmed", + request_id="stale-review", + actor_id="auditor", + ) + + +def test_typed_risk_disposition_event_is_a_trusted_release_label(db: Session) -> None: + _seed_asset(db) + service = AgentAssetReleaseTelemetryService(db) + telemetry = _record_shadow(db, claim_id="claim-typed") + risk_observation = RiskObservation( + id="risk-observation-1", + tenant_id="tenant-a", + observation_key="risk:claim-typed:platform:risk.release.telemetry", + subject_type="expense_claim", + subject_key="claim:claim-typed", + claim_id="claim-typed", + claim_no="REDACTED", + risk_type="policy", + risk_signal="risk.release.telemetry", + risk_score=82, + risk_level="high", + algorithm_version="v1", + decision_trace_json={"rule_code": "risk.release.telemetry"}, + ) + disposition = RiskDisposition( + id="disposition-1", + tenant_id="tenant-a", + observation_id=risk_observation.id, + adjudication="confirmed", + lifecycle_status="open", + version=1, + ) + event = RiskDispositionEvent( + id="disposition-event-1", + tenant_id="tenant-a", + disposition_id=disposition.id, + observation_id=risk_observation.id, + version=1, + action="confirm", + actor_id="real-auditor-secret", + actor_name="真实审计员", + request_id="typed-action-1", + payload_fingerprint="f" * 64, + before_json={"adjudication": "unreviewed"}, + after_json={"adjudication": "confirmed"}, + ) + db.add_all([risk_observation, disposition, event]) + db.commit() + + label = service.record_risk_disposition_label( + tenant_id="tenant-a", + observation_id=telemetry.id, + disposition_event_id=event.id, + ) + db.commit() + + assert label.label == "confirmed" + assert label.verification_source == "typed_risk_disposition" + assert "real-auditor-secret" not in str(label.__dict__) + aggregate = service.aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + ) + assert aggregate.status == "ready" + assert aggregate.precision == 1.0 + assert aggregate.baseline_precision == 1.0 + + +def test_typed_disposition_resolves_current_release_sample_without_client_observation_id( + db: Session, +) -> None: + _seed_asset(db) + telemetry = _record_shadow(db, claim_id="claim-current-label") + risk_observation = RiskObservation( + id="risk-observation-current", + tenant_id="tenant-a", + observation_key="risk:claim-current-label:platform:risk.release.telemetry", + subject_type="expense_claim", + subject_key="claim:claim-current-label", + claim_id="claim-current-label", + claim_no="REDACTED", + risk_type="policy", + risk_signal="risk.release.telemetry", + risk_score=82, + risk_level="high", + control_stage="reimbursement", + algorithm_version="v1", + decision_trace_json={"rule_code": "risk.release.telemetry"}, + ) + disposition = RiskDisposition( + id="disposition-current", + tenant_id="tenant-a", + observation_id=risk_observation.id, + adjudication="false_positive", + lifecycle_status="open", + version=1, + ) + event = RiskDispositionEvent( + id="disposition-event-current", + tenant_id="tenant-a", + disposition_id=disposition.id, + observation_id=risk_observation.id, + version=1, + action="false_positive", + actor_id="auditor-secret", + actor_name="审计员", + request_id="typed-current-action", + payload_fingerprint="e" * 64, + before_json={"adjudication": "unreviewed"}, + after_json={"adjudication": "false_positive"}, + ) + db.add_all([risk_observation, disposition, event]) + db.commit() + + labels = AgentAssetReleaseDispositionLabelService(db).record_current_labels( + tenant_id="tenant-a", + disposition_event_id=event.id, + ) + db.commit() + + assert [item.observation_id for item in labels] == [telemetry.id] + assert labels[0].label == "false_positive" + assert "auditor-secret" not in str(labels[0].__dict__) + + +def test_current_disposition_does_not_label_stale_release_sample(db: Session) -> None: + asset = _seed_asset(db) + _record_shadow(db, claim_id="claim-stale-label") + risk_observation = RiskObservation( + id="risk-observation-stale-current", + tenant_id="tenant-a", + observation_key="risk:claim-stale-label:platform:risk.release.telemetry", + subject_type="expense_claim", + subject_key="claim:claim-stale-label", + claim_id="claim-stale-label", + claim_no="REDACTED", + risk_type="policy", + risk_signal="risk.release.telemetry", + risk_score=82, + risk_level="high", + control_stage="reimbursement", + algorithm_version="v1", + decision_trace_json={"rule_code": "risk.release.telemetry"}, + ) + disposition = RiskDisposition( + id="disposition-stale-current", + tenant_id="tenant-a", + observation_id=risk_observation.id, + adjudication="confirmed", + lifecycle_status="open", + version=1, + ) + event = RiskDispositionEvent( + id="disposition-event-stale-current", + tenant_id="tenant-a", + disposition_id=disposition.id, + observation_id=risk_observation.id, + version=1, + action="confirm", + actor_id="auditor", + actor_name="审计员", + request_id="typed-stale-current-action", + payload_fingerprint="d" * 64, + before_json={"adjudication": "unreviewed"}, + after_json={"adjudication": "confirmed"}, + ) + db.add_all([risk_observation, disposition, event]) + db.commit() + config = dict(asset.config_json or {}) + state = dict(config["release_guard"]) + state["release_id"] = "release-2" + config["release_guard"] = state + asset.config_json = config + db.add(asset) + db.commit() + + labels = AgentAssetReleaseDispositionLabelService(db).record_current_labels( + tenant_id="tenant-a", + disposition_event_id=event.id, + ) + + assert labels == [] + + +def test_canary_hits_and_structured_runtime_failures_are_real_samples(db: Session) -> None: + _seed_asset(db, stage="canary") + service = AgentAssetReleaseTelemetryService(db) + records = service.record_expense_risk_result( + tenant_id="tenant-a", + claim_id="canary-claim-1", + result={ + "shadow_evaluations": [], + "flags": [ + { + "rule_code": "risk.release.telemetry", + "rule_version": "v2", + "release_stage": "canary", + "release_mode": "enforced", + } + ], + }, + ) + miss = service.record_manifest_evaluation( + tenant_id="tenant-a", + claim_id="canary-claim-miss", + manifest={ + "_rule_asset_id": "release-asset", + "_release_stage": "canary", + "_release_mode": "enforced", + "_rule_version": "v2", + "rule_code": "risk.release.telemetry", + }, + hit=False, + ) + failure = service.record_observation( + ReleaseObservationInput( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="canary", + version="v2", + rule_code="risk.release.telemetry", + source_key="canary-claim-2", + candidate_hit=False, + baseline_hit=None, + runtime_status="failed", + failure_code="evaluator_error", + ) + ) + service.record_review_label( + tenant_id="tenant-a", + observation_id=records[0].id, + label="false_positive", + request_id="canary-review", + actor_id="auditor", + ) + with pytest.raises(ValueError, match="independent ground truth"): + service.record_review_label( + tenant_id="tenant-a", + observation_id=miss.id, + label="confirmed", + request_id="unsupported-false-negative-label", + actor_id="auditor", + ) + db.commit() + + aggregate = service.aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="canary", + version="v2", + ) + evaluation = aggregate.to_release_evaluation_input() + assert records[0].candidate_hit is True + assert miss.candidate_hit is False + assert failure.runtime_status == "failed" + assert aggregate.status == "ready" + assert aggregate.runtime_failure_count == 1 + assert aggregate.precision == 0.0 + assert aggregate.baseline_precision is None + assert evaluation.total == 3 + assert evaluation.failure_count == 1 + + +def test_dedicated_review_queue_is_tenant_safe_and_separates_publisher( + db: Session, +) -> None: + _seed_asset(db) + observation = _record_shadow(db, claim_id="claim-review-queue") + service = AgentAssetReleaseReviewService(db) + + with pytest.raises(LookupError): + service.list_pending(tenant_id="tenant-b", asset_id="release-asset") + queue = service.list_pending(tenant_id="tenant-a", asset_id="release-asset") + assert queue["pending_total"] == 1 + assert queue["items"][0]["observation_id"] == observation.id + assert "source_fingerprint" not in queue["items"][0] + + with pytest.raises(PermissionError, match="发布发起人"): + service.record_label( + tenant_id="tenant-a", + asset_id="release-asset", + observation_id=observation.id, + label="confirmed", + actor_id="publisher", + request_id="publisher-self-review", + ) + label = service.record_label( + tenant_id="tenant-a", + asset_id="release-asset", + observation_id=observation.id, + label="confirmed", + actor_id="independent-reviewer", + request_id="independent-review", + ) + db.commit() + + assert label.label == "risk_present" + assert service.list_pending(tenant_id="tenant-a", asset_id="release-asset")[ + "pending_total" + ] == 0 + + +def test_release_review_quorum_requires_distinct_reviewers(db: Session) -> None: + _seed_asset(db, reviewer_quorum=2) + observation = _record_shadow(db, claim_id="claim-two-reviewers") + service = AgentAssetReleaseReviewService(db) + + service.record_label( + tenant_id="tenant-a", + asset_id="release-asset", + observation_id=observation.id, + label="confirmed", + actor_id="reviewer-one", + request_id="review-one", + ) + db.commit() + first_queue = service.list_pending(tenant_id="tenant-a", asset_id="release-asset") + first_aggregate = AgentAssetReleaseTelemetryService(db).aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + ) + assert first_queue["pending_total"] == 1 + assert first_queue["items"][0]["reviewer_count"] == 1 + assert first_queue["items"][0]["required_reviewers"] == 2 + assert first_aggregate.status == "collecting" + + service.record_label( + tenant_id="tenant-a", + asset_id="release-asset", + observation_id=observation.id, + label="confirmed", + actor_id="reviewer-two", + request_id="review-two", + ) + db.commit() + second_aggregate = AgentAssetReleaseTelemetryService(db).aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + ) + assert service.list_pending(tenant_id="tenant-a", asset_id="release-asset")[ + "pending_total" + ] == 0 + assert second_aggregate.status == "ready" + assert second_aggregate.precision == 1.0 + + +def test_blind_negative_audit_builds_real_false_negative_and_recall_evidence( + db: Session, +) -> None: + _seed_asset( + db, + recall_gate_enabled=True, + negative_sample_percent=100, + negative_min_reviewed=2, + min_recall=0.3, + ) + positive = _record_shadow( + db, + claim_id="claim-blind-positive", + candidate_hit=True, + baseline_hit=False, + ) + false_negative = _record_shadow( + db, + claim_id="claim-blind-missed-risk", + candidate_hit=False, + baseline_hit=False, + ) + true_negative = _record_shadow( + db, + claim_id="claim-blind-no-risk", + candidate_hit=False, + baseline_hit=False, + ) + db.commit() + review = AgentAssetReleaseReviewService(db) + + queue = review.list_pending(tenant_id="tenant-a", asset_id="release-asset") + + assert queue["pending_total"] == 3 + assert {item["source_document_id"] for item in queue["items"]} == { + "claim-blind-positive", + "claim-blind-missed-risk", + "claim-blind-no-risk", + } + assert all(item["prediction_blinded"] is True for item in queue["items"]) + assert all("candidate_hit" not in item for item in queue["items"]) + assert all("baseline_hit" not in item for item in queue["items"]) + quorum_by_source = { + item["source_document_id"]: item["required_reviewers"] + for item in queue["items"] + } + assert quorum_by_source["claim-blind-positive"] == 1 + assert quorum_by_source["claim-blind-missed-risk"] == 2 + assert quorum_by_source["claim-blind-no-risk"] == 2 + stored_samples = list(db.scalars(select(AgentAssetReleaseAuditSample)).all()) + assert len(stored_samples) == 3 + assert "claim-blind" not in str( + [item.source_reference_encrypted for item in stored_samples] + ) + + for observation, label in ( + (positive, "confirmed"), + (false_negative, "confirmed"), + (true_negative, "false_positive"), + ): + review.record_label( + tenant_id="tenant-a", + asset_id="release-asset", + observation_id=observation.id, + label=label, + actor_id="independent-auditor", + request_id=f"blind-review:{observation.id}", + ) + if not observation.candidate_hit: + review.record_label( + tenant_id="tenant-a", + asset_id="release-asset", + observation_id=observation.id, + label=label, + actor_id="second-independent-auditor", + request_id=f"blind-review-second:{observation.id}", + ) + db.commit() + + aggregate = AgentAssetReleaseTelemetryService(db).aggregate( + tenant_id="tenant-a", + asset_id="release-asset", + release_id="release-1", + stage="shadow", + version="v2", + ) + evaluation = aggregate.to_release_evaluation_input() + + assert aggregate.status == "ready" + assert aggregate.false_negative_count == 1 + assert aggregate.random_negative_population_count == 2 + assert aggregate.random_negative_labeled_count == 2 + assert aggregate.recall == 0.5 + assert aggregate.recall_lower_bound is not None + assert aggregate.recall_lower_bound < aggregate.recall + assert ( + aggregate.negative_ground_truth_status + == "available_stratified_random_audit" + ) + assert evaluation.details is not None + assert evaluation.details["recall"] == 0.5 + assert evaluation.details["false_negative_count"] == 1 + + +def test_telemetry_rows_are_append_only(db: Session) -> None: + _seed_asset(db) + service = AgentAssetReleaseTelemetryService(db) + observation = _record_shadow(db, claim_id="claim-immutable") + label = service.record_review_label( + tenant_id="tenant-a", + observation_id=observation.id, + label="confirmed", + request_id="immutable-review", + actor_id="auditor", + ) + db.commit() + + observation.candidate_hit = False + with pytest.raises(ValueError, match="append-only"): + db.flush() + db.rollback() + + persisted_label = db.scalar( + select(AgentAssetReleaseLabel).where(AgentAssetReleaseLabel.id == label.id) + ) + assert persisted_label is not None + db.delete(persisted_label) + with pytest.raises(ValueError, match="append-only"): + db.flush() diff --git a/server/tests/test_agent_asset_release_telemetry_concurrency_postgres.py b/server/tests/test_agent_asset_release_telemetry_concurrency_postgres.py new file mode 100644 index 0000000..d518566 --- /dev/null +++ b/server/tests/test_agent_asset_release_telemetry_concurrency_postgres.py @@ -0,0 +1,437 @@ +from __future__ import annotations + +import threading +import uuid +from concurrent.futures import ThreadPoolExecutor +from dataclasses import replace + +from savings_postgres_testkit import ( # noqa: F401 - 注册 pg_factory fixture + _pg_factory_fixture, +) +from sqlalchemy import func, select +from sqlalchemy.orm import Session, sessionmaker + +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType +from app.models.agent_asset import AgentAsset +from app.models.agent_asset_release_telemetry import ( + AgentAssetReleaseAuditSample, + AgentAssetReleaseLabel, + AgentAssetReleaseObservation, +) +from app.models.financial_record import ExpenseClaim +from app.services.agent_asset_release_review import AgentAssetReleaseReviewService +from app.services.agent_asset_release_scheduler import AgentAssetReleaseScheduler +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseObservationInput, + ReleaseTelemetryIdempotencyConflict, + ReleaseTelemetryStaleRelease, +) +from app.services.expense_claim_release_telemetry import ( + ExpenseClaimReleaseTelemetryRecorder, +) + + +def test_concurrent_observation_and_label_replay_create_one_fact( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, asset_id, rule_code = _seed_asset(pg_factory) + payload = _payload(tenant_id, asset_id, rule_code, source="claim-same") + ready = threading.Barrier(2) + + def record_observation() -> str: + with pg_factory() as db: + ready.wait(timeout=5) + row = AgentAssetReleaseTelemetryService(db).record_observation(payload) + db.commit() + return row.id + + with ThreadPoolExecutor(max_workers=2) as pool: + observation_ids = [ + future.result(timeout=10) + for future in ( + pool.submit(record_observation), + pool.submit(record_observation), + ) + ] + assert len(set(observation_ids)) == 1 + observation_id = observation_ids[0] + + label_ready = threading.Barrier(2) + + def record_label() -> str: + with pg_factory() as db: + label_ready.wait(timeout=5) + row = AgentAssetReleaseTelemetryService(db).record_review_label( + tenant_id=tenant_id, + observation_id=observation_id, + label="confirmed", + request_id="review-concurrent-same", + actor_id="trusted-reviewer", + ) + db.commit() + return row.id + + with ThreadPoolExecutor(max_workers=2) as pool: + label_ids = [ + future.result(timeout=10) + for future in (pool.submit(record_label), pool.submit(record_label)) + ] + assert len(set(label_ids)) == 1 + + with pg_factory() as db: + assert db.scalar( + select(func.count()) + .select_from(AgentAssetReleaseObservation) + .where( + AgentAssetReleaseObservation.tenant_id == tenant_id, + AgentAssetReleaseObservation.id == observation_id, + ) + ) == 1 + assert db.scalar( + select(func.count()) + .select_from(AgentAssetReleaseAuditSample) + .where( + AgentAssetReleaseAuditSample.tenant_id == tenant_id, + AgentAssetReleaseAuditSample.observation_id == observation_id, + ) + ) == 1 + assert db.scalar( + select(func.count()) + .select_from(AgentAssetReleaseLabel) + .where( + AgentAssetReleaseLabel.tenant_id == tenant_id, + AgentAssetReleaseLabel.observation_id == observation_id, + ) + ) == 1 + + +def test_stage_transition_wins_before_label_and_rejects_stale_fact( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, asset_id, rule_code = _seed_asset(pg_factory) + with pg_factory() as db: + observation = AgentAssetReleaseTelemetryService(db).record_observation( + _payload(tenant_id, asset_id, rule_code, source="claim-stage-race") + ) + db.commit() + observation_id = observation.id + + label_started = threading.Event() + + def record_stale_label() -> str: + with pg_factory() as db: + label_started.set() + try: + AgentAssetReleaseTelemetryService(db).record_review_label( + tenant_id=tenant_id, + observation_id=observation_id, + label="confirmed", + request_id="review-after-stage-transition", + actor_id="trusted-reviewer", + ) + db.commit() + return "created" + except ReleaseTelemetryStaleRelease: + db.rollback() + return "stale" + + with pg_factory() as transition_db: + asset = transition_db.scalar( + select(AgentAsset).where(AgentAsset.id == asset_id).with_for_update() + ) + assert asset is not None + config = dict(asset.config_json or {}) + state = dict(config["release_guard"]) + state["stage"] = "canary" + config["release_guard"] = state + asset.config_json = config + transition_db.flush() + with ThreadPoolExecutor(max_workers=1) as pool: + future = pool.submit(record_stale_label) + assert label_started.wait(timeout=5) + transition_db.commit() + outcome = future.result(timeout=10) + + assert outcome == "stale" + with pg_factory() as db: + assert db.scalar( + select(func.count()) + .select_from(AgentAssetReleaseLabel) + .where( + AgentAssetReleaseLabel.tenant_id == tenant_id, + AgentAssetReleaseLabel.observation_id == observation_id, + ) + ) == 0 + + +def test_concurrent_conflicting_observation_payload_has_one_winner( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, asset_id, rule_code = _seed_asset(pg_factory) + first = _payload(tenant_id, asset_id, rule_code, source="claim-conflict") + second = replace(first, candidate_hit=False) + ready = threading.Barrier(2) + + def record_once(payload: ReleaseObservationInput) -> str: + with pg_factory() as db: + ready.wait(timeout=5) + try: + AgentAssetReleaseTelemetryService(db).record_observation(payload) + db.commit() + return "created" + except ReleaseTelemetryIdempotencyConflict: + db.rollback() + return "conflict" + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in ( + pool.submit(record_once, first), + pool.submit(record_once, second), + ) + ] + assert sorted(outcomes) == ["conflict", "created"] + with pg_factory() as db: + assert db.scalar( + select(func.count()) + .select_from(AgentAssetReleaseObservation) + .where( + AgentAssetReleaseObservation.tenant_id == tenant_id, + AgentAssetReleaseObservation.asset_id == asset_id, + ) + ) == 1 + + +def test_concurrent_conflicting_label_payload_has_one_winner( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, asset_id, rule_code = _seed_asset(pg_factory) + with pg_factory() as db: + observation = AgentAssetReleaseTelemetryService(db).record_observation( + _payload(tenant_id, asset_id, rule_code, source="claim-label-conflict") + ) + db.commit() + observation_id = observation.id + ready = threading.Barrier(2) + + def record_once(label: str) -> str: + with pg_factory() as db: + ready.wait(timeout=5) + try: + AgentAssetReleaseTelemetryService(db).record_review_label( + tenant_id=tenant_id, + observation_id=observation_id, + label=label, # type: ignore[arg-type] + request_id="review-conflicting-same-request", + actor_id="trusted-reviewer", + ) + db.commit() + return "created" + except ReleaseTelemetryIdempotencyConflict: + db.rollback() + return "conflict" + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in ( + pool.submit(record_once, "confirmed"), + pool.submit(record_once, "false_positive"), + ) + ] + assert sorted(outcomes) == ["conflict", "created"] + with pg_factory() as db: + assert db.scalar( + select(func.count()) + .select_from(AgentAssetReleaseLabel) + .where( + AgentAssetReleaseLabel.tenant_id == tenant_id, + AgentAssetReleaseLabel.observation_id == observation_id, + ) + ) == 1 + + +def test_negative_blind_review_requires_two_distinct_actors_under_concurrency( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, asset_id, rule_code = _seed_asset(pg_factory) + with pg_factory() as db: + observation = AgentAssetReleaseTelemetryService(db).record_observation( + replace( + _payload( + tenant_id, + asset_id, + rule_code, + source="claim-negative-double-review", + ), + candidate_hit=False, + baseline_hit=True, + ) + ) + db.commit() + observation_id = observation.id + + ready = threading.Barrier(2) + + def record_same_actor(request_id: str) -> str: + with pg_factory() as db: + ready.wait(timeout=5) + row = AgentAssetReleaseReviewService(db).record_label( + tenant_id=tenant_id, + asset_id=asset_id, + observation_id=observation_id, + label="risk_present", + actor_id="same-independent-reviewer", + request_id=request_id, + ) + db.commit() + return row.id + + with ThreadPoolExecutor(max_workers=2) as pool: + same_actor_label_ids = [ + future.result(timeout=10) + for future in ( + pool.submit(record_same_actor, "same-actor-review-a"), + pool.submit(record_same_actor, "same-actor-review-b"), + ) + ] + assert len(set(same_actor_label_ids)) == 2 + + with pg_factory() as db: + queue = AgentAssetReleaseReviewService(db).list_pending( + tenant_id=tenant_id, + asset_id=asset_id, + ) + assert queue["pending_total"] == 1 + assert queue["items"][0]["reviewer_count"] == 1 + assert queue["items"][0]["required_reviewers"] == 2 + + AgentAssetReleaseReviewService(db).record_label( + tenant_id=tenant_id, + asset_id=asset_id, + observation_id=observation_id, + label="risk_present", + actor_id="second-independent-reviewer", + request_id="second-actor-review", + ) + db.commit() + + resolved = AgentAssetReleaseReviewService(db).list_pending( + tenant_id=tenant_id, + asset_id=asset_id, + ) + assert resolved["pending_total"] == 0 + + +def test_failed_observation_survives_business_transaction_rollback( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, asset_id, rule_code = _seed_asset(pg_factory) + claim = ExpenseClaim(id="claim-durable-failure", claim_no="BX-DURABLE-FAILURE") + manifest = { + "_rule_asset_id": asset_id, + "_release_stage": "shadow", + "_release_mode": "shadow", + "_rule_version": "v2", + "rule_code": rule_code, + } + with pg_factory() as business_db: + asset = business_db.get(AgentAsset, asset_id) + assert asset is not None + asset.description = "this business mutation must roll back" + business_db.flush() + + ExpenseClaimReleaseTelemetryRecorder(business_db).record_failure_durably( + tenant_id=tenant_id, + claim=claim, + manifest=manifest, + failure_code="evaluator_error", + business_stage="reimbursement", + ) + business_db.rollback() + + with pg_factory() as db: + asset = db.get(AgentAsset, asset_id) + observation = db.scalar( + select(AgentAssetReleaseObservation).where( + AgentAssetReleaseObservation.tenant_id == tenant_id, + AgentAssetReleaseObservation.asset_id == asset_id, + AgentAssetReleaseObservation.runtime_status == "failed", + ) + ) + assert asset is not None and asset.description == "" + assert observation is not None + assert observation.failure_code == "evaluator_error" + + +def test_release_scheduler_advisory_lease_has_single_owner_and_transfers( + pg_factory: sessionmaker[Session], +) -> None: + scheduler = AgentAssetReleaseScheduler(session_factory=pg_factory) + with pg_factory() as first, pg_factory() as second: + assert scheduler._try_acquire_lease(first) is True + assert scheduler._try_acquire_lease(second) is False + + scheduler._release_lease(first) + + assert scheduler._try_acquire_lease(second) is True + scheduler._release_lease(second) + + +def _seed_asset(factory: sessionmaker[Session]) -> tuple[str, str, str]: + suffix = uuid.uuid4().hex + tenant_id = f"tenant-release-{suffix}" + asset_id = str(uuid.uuid4()) + rule_code = f"risk.release.{suffix}" + with factory() as db: + db.add( + AgentAsset( + id=asset_id, + asset_type=AgentAssetType.RULE.value, + code=rule_code, + name="发布遥测并发规则", + description="", + domain=AgentAssetDomain.EXPENSE.value, + scenario_json=["travel"], + owner="postgres-test", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={ + "tenant_id": tenant_id, + "detail_mode": "json_risk", + "enabled": True, + "release_guard": { + "release_id": f"release-{suffix}", + "stage": "shadow", + "candidate_version": "v2", + "previous_version": "v1", + }, + }, + ) + ) + db.commit() + return tenant_id, asset_id, rule_code + + +def _payload( + tenant_id: str, + asset_id: str, + rule_code: str, + *, + source: str, +) -> ReleaseObservationInput: + return ReleaseObservationInput( + tenant_id=tenant_id, + asset_id=asset_id, + release_id=f"release-{rule_code.removeprefix('risk.release.')}", + stage="shadow", + version="v2", + rule_code=rule_code, + source_key=source, + candidate_hit=True, + baseline_hit=True, + ) diff --git a/server/tests/test_agent_asset_service.py b/server/tests/test_agent_asset_service.py index 2730b70..03ae0f2 100644 --- a/server/tests/test_agent_asset_service.py +++ b/server/tests/test_agent_asset_service.py @@ -4,6 +4,7 @@ import shutil import uuid from io import BytesIO from pathlib import Path +from urllib.parse import parse_qs, urlsplit import pytest from openpyxl import Workbook, load_workbook @@ -26,6 +27,7 @@ from app.core.config import SERVER_DIR from app.db.base import Base from app.models.agent_asset import AgentAsset from app.models.employee import Employee +from app.models.tenant import Tenant from app.schemas.agent_asset import ( AgentAssetCreate, AgentAssetReviewCreate, @@ -33,23 +35,28 @@ from app.schemas.agent_asset import ( ) from app.schemas.reimbursement import TravelReimbursementCalculatorRequest from app.services import agent_foundation as agent_foundation_module +from app.services.agent_asset_onlyoffice_security import ( + AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + AgentAssetOnlyOfficeReplayError, + AgentAssetOnlyOfficeSecurityError, +) from app.services.agent_asset_spreadsheet import ( COMPANY_COMMUNICATION_EXPENSE_RULE_CODE, COMPANY_COMMUNICATION_EXPENSE_RULE_FILENAME, COMPANY_PREAPPROVAL_RULE_CODE, COMPANY_PREAPPROVAL_RULE_FILENAME, - COMPANY_TRAVEL_GRADE_MAPPING_RULE_CODE, - COMPANY_TRAVEL_GRADE_MAPPING_RULE_FILENAME, COMPANY_TRAVEL_EXPENSE_RULE_CODE, COMPANY_TRAVEL_EXPENSE_RULE_FILENAME, + COMPANY_TRAVEL_GRADE_MAPPING_RULE_CODE, + COMPANY_TRAVEL_GRADE_MAPPING_RULE_FILENAME, COMPANY_TRAVEL_SEASON_MAPPING_RULE_CODE, COMPANY_TRAVEL_SEASON_MAPPING_RULE_FILENAME, COMPANY_TRAVEL_TRANSPORT_ESTIMATE_RULE_CODE, COMPANY_TRAVEL_TRANSPORT_ESTIMATE_RULE_FILENAME, FINANCE_RULES_LIBRARY, ) -from app.services.agent_foundation_constants import COMPANY_PREAPPROVAL_RULE_SCENARIO_JSON from app.services.agent_assets import AgentAssetService +from app.services.agent_foundation_constants import COMPANY_PREAPPROVAL_RULE_SCENARIO_JSON from app.services.agent_runs import AgentRunService from app.services.audit import AuditLogService from app.services.expense_rule_runtime import ExpenseRuleRuntimeService @@ -194,7 +201,9 @@ def test_finance_rules_use_risk_rule_scenario_categories() -> None: communication_rule = next( item for item in rules if item.code == COMPANY_COMMUNICATION_EXPENSE_RULE_CODE ) - preapproval_rule = next(item for item in rules if item.code == COMPANY_PREAPPROVAL_RULE_CODE) + preapproval_rule = next( + item for item in rules if item.code == COMPANY_PREAPPROVAL_RULE_CODE + ) travel_config = travel_rule.config_json or {} communication_config = communication_rule.config_json or {} preapproval_config = preapproval_rule.config_json or {} @@ -288,7 +297,9 @@ def test_existing_budget_risk_assets_are_hidden_from_rule_lists() -> None: config_json={ "detail_mode": "json_risk", "finance_rule_code": "budget.execution.policy", - "rule_document": {"file_name": "risk.budget.available_balance_insufficient.json"}, + "rule_document": { + "file_name": "risk.budget.available_balance_insufficient.json" + }, }, ) ) @@ -422,7 +433,7 @@ def test_pending_review_can_name_new_working_version_before_submission() -> None assert detail.working_version == "v1.2.0" assert detail.published_version == "v1.1.0" assert detail.latest_review is not None - assert detail.latest_review.reviewer == "manager_user" + assert detail.latest_review.reviewer == "finance_user" def test_expense_rule_runtime_uses_published_version_instead_of_working_version() -> None: @@ -438,10 +449,7 @@ def test_expense_rule_runtime_uses_published_version_instead_of_working_version( rule.id, AgentAssetVersionCreate( version="v1.1.1", - content=( - "# 工作稿\n\n" - '```expense-rule\n{"kind":"travel_policy","version":1}\n```' - ), + content=('# 工作稿\n\n```expense-rule\n{"kind":"travel_policy","version":1}\n```'), content_type=AgentAssetContentType.MARKDOWN, change_note="未上线草稿", created_by="finance_user", @@ -488,7 +496,7 @@ def test_spreadsheet_upload_records_sheet_and_cell_changes_without_versions() -> service.upload_rule_spreadsheet( rule.id, filename="公司差旅费报销规则.xlsx", - content=build_workbook_bytes([["城市", "住宿"], ["北京", 500]]), + content=build_workbook_bytes([["城市", "住宿"], ["北京", 500]]), actor="finance_user", ) service.upload_rule_spreadsheet( @@ -504,12 +512,10 @@ def test_spreadsheet_upload_records_sheet_and_cell_changes_without_versions() -> assert latest.changed_sheet_count == 1 assert latest.changed_cell_count == 3 assert any( - item.cell == "B2" and item.change_type == "modified" - for item in latest.cell_changes + item.cell == "B2" and item.change_type == "modified" for item in latest.cell_changes ) assert any( - item.cell == "A3" and item.change_type == "added" - for item in latest.cell_changes + item.cell == "A3" and item.change_type == "added" for item in latest.cell_changes ) assert not hasattr(latest, "version") @@ -634,7 +640,7 @@ def test_spreadsheet_change_records_include_all_modified_sheets() -> None: assert "填表说明" in latest.summary -def test_editable_spreadsheet_onlyoffice_config_enables_forcesave(monkeypatch) -> None: +def test_platform_spreadsheet_onlyoffice_requires_platform_admin_to_edit(monkeypatch) -> None: with build_session() as db: monkeypatch.setattr( "app.services.agent_asset_onlyoffice.resolve_onlyoffice_settings", @@ -642,7 +648,7 @@ def test_editable_spreadsheet_onlyoffice_config_enables_forcesave(monkeypatch) - enabled=True, public_url="http://onlyoffice.example.com", backend_url="http://backend.example.com", - jwt_secret="secret", + jwt_secret="onlyoffice-test-secret-at-least-32-bytes", ), ) @@ -653,21 +659,83 @@ def test_editable_spreadsheet_onlyoffice_config_enables_forcesave(monkeypatch) - if item.code == "rule.expense.company_travel_expense_reimbursement" ) - config = service.build_rule_spreadsheet_onlyoffice_config( + finance_config = service.build_rule_spreadsheet_onlyoffice_config( rule.id, CurrentUserContext( username="finance_user", name="财务人员", role_codes=["finance"], is_admin=False, + tenant_id="default", + ), + ) + admin_config = service.build_rule_spreadsheet_onlyoffice_config( + rule.id, + CurrentUserContext( + username="platform_admin", + name="平台管理员", + role_codes=["manager"], + is_admin=True, + tenant_id="default", ), ) - customization = config.config["editorConfig"]["customization"] - assert config.config["editorConfig"]["mode"] == "edit" - assert customization["forcesave"] is True - assert "version=" not in config.config["document"]["url"] - assert "version=" not in config.config["editorConfig"]["callbackUrl"] + assert finance_config.config["editorConfig"]["mode"] == "view" + assert finance_config.config["editorConfig"]["customization"]["forcesave"] is False + assert admin_config.config["editorConfig"]["mode"] == "edit" + assert admin_config.config["editorConfig"]["customization"]["forcesave"] is True + assert "version=" not in admin_config.config["document"]["url"] + assert "version=" not in admin_config.config["editorConfig"]["callbackUrl"] + + finance_callback_token = parse_qs( + urlsplit(finance_config.config["editorConfig"]["callbackUrl"]).query + )["access_token"][0] + admin_callback_token = parse_qs( + urlsplit(admin_config.config["editorConfig"]["callbackUrl"]).query + )["access_token"][0] + finance_session = service.validate_rule_spreadsheet_access_token( + rule.id, + finance_callback_token, + expected_scope=AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + ) + admin_session = service.validate_rule_spreadsheet_access_token( + rule.id, + admin_callback_token, + expected_scope=AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + ) + assert finance_session.writable is False + assert admin_session.writable is True + assert admin_session.tenant_id == "platform" + assert admin_session.resource_scope == "platform" + assert admin_session.actor == "username:platform_admin" + with pytest.raises(AgentAssetOnlyOfficeSecurityError, match="不匹配"): + service.validate_rule_spreadsheet_access_token( + "different-asset", + admin_callback_token, + expected_scope=AGENT_ASSET_ONLYOFFICE_CALLBACK_SCOPE, + ) + with pytest.raises(AgentAssetOnlyOfficeSecurityError, match="只读"): + service._onlyoffice_session_service().claim_callback( + asset_id=rule.id, + token=finance_callback_token, + payload_document_key=finance_session.document_key, + ) + + claimed = service._onlyoffice_session_service().claim_callback( + asset_id=rule.id, + token=admin_callback_token, + payload_document_key=admin_session.document_key, + ) + with pytest.raises(AgentAssetOnlyOfficeReplayError, match="已被使用"): + service._onlyoffice_session_service().claim_callback( + asset_id=rule.id, + token=admin_callback_token, + payload_document_key=admin_session.document_key, + ) + service._onlyoffice_session_service().finish_callback( + claimed.jti, + succeeded=True, + ) def test_version_timeline_contains_created_review_and_publish_events() -> None: @@ -789,6 +857,9 @@ def test_expense_rule_runtime_reads_amount_standards_from_travel_spreadsheet() - def test_travel_reimbursement_calculator_uses_finance_spreadsheet_amounts() -> None: with build_session() as db: + # 生产环境在应用启动阶段初始化规则资产；计算阶段必须保持只读， + # 测试需显式模拟该启动前置条件，不能依赖计算器隐式提交事务。 + AgentAssetService(db).list_assets(asset_type=AgentAssetType.RULE.value) db.add( Employee( employee_no="E9001", @@ -803,6 +874,7 @@ def test_travel_reimbursement_calculator_uses_finance_spreadsheet_amounts() -> N result = TravelReimbursementCalculatorService(db).calculate( TravelReimbursementCalculatorRequest(days=3, location="北京市朝阳区"), CurrentUserContext( + tenant_id="default", username="traveler@example.com", name="测试员工", role_codes=[], @@ -829,8 +901,58 @@ def test_travel_reimbursement_calculator_uses_finance_spreadsheet_amounts() -> N assert "申请预算占用参考总金额为 2690.00 元" in result.summary_text +def test_travel_reimbursement_calculator_resolves_employee_inside_authenticated_tenant() -> None: + with build_session() as db: + db.add( + Tenant( + tenant_id="tenant-travel-other", + tenant_code="tenant-travel-other", + name="差旅隔离对照企业", + ) + ) + other_employee = Employee( + id="travel-employee-other", + tenant_id="tenant-travel-other", + employee_no="TRAVEL-SAME-001", + name="同名差旅员工", + email="same-traveler@example.com", + position="其他企业员工", + grade="P8", + location="北京", + ) + current_employee = Employee( + id="travel-employee-current", + tenant_id="default", + employee_no="TRAVEL-SAME-001", + name="同名差旅员工", + email="same-traveler@example.com", + position="当前企业员工", + grade="P4", + location="上海", + ) + db.add_all([other_employee, current_employee]) + db.commit() + + resolved = TravelReimbursementCalculatorService(db)._resolve_current_employee( + CurrentUserContext( + tenant_id="default", + employee_id=current_employee.id, + employee_no=current_employee.employee_no, + username=current_employee.email, + name=current_employee.name, + role_codes=[], + is_admin=False, + ) + ) + + assert resolved is not None + assert resolved.id == current_employee.id + assert resolved.tenant_id == "default" + + def test_travel_reimbursement_calculator_uses_other_region_for_known_unlisted_location() -> None: with build_session() as db: + AgentAssetService(db).list_assets(asset_type=AgentAssetType.RULE.value) db.add( Employee( employee_no="E9002", @@ -845,6 +967,7 @@ def test_travel_reimbursement_calculator_uses_other_region_for_known_unlisted_lo result = TravelReimbursementCalculatorService(db).calculate( TravelReimbursementCalculatorRequest(days=2, location="吉林延边"), CurrentUserContext( + tenant_id="default", username="other-region@example.com", name="其他地区员工", role_codes=[], @@ -880,6 +1003,7 @@ def test_travel_reimbursement_calculator_rejects_unrecognized_location() -> None TravelReimbursementCalculatorService(db).calculate( TravelReimbursementCalculatorRequest(days=2, location="背景"), CurrentUserContext( + tenant_id="default", username="invalid-location@example.com", name="无效地点员工", role_codes=[], @@ -904,6 +1028,7 @@ def test_travel_reimbursement_calculator_normalizes_location_mixed_with_business result = TravelReimbursementCalculatorService(db).calculate( TravelReimbursementCalculatorRequest(days=4, location="上海辅助国网仿生产服务器"), CurrentUserContext( + tenant_id="default", username="mixed-location@example.com", name="混合地点员工", role_codes=[], diff --git a/server/tests/test_agent_asset_spreadsheet_import.py b/server/tests/test_agent_asset_spreadsheet_import.py index eaeb493..3c9966b 100644 --- a/server/tests/test_agent_asset_spreadsheet_import.py +++ b/server/tests/test_agent_asset_spreadsheet_import.py @@ -25,9 +25,9 @@ def test_rebuild_from_uploaded_content_preserves_sheet_values() -> None: assert workbook.sheetnames == ["差旅标准", "补贴标准"] assert workbook["差旅标准"]["A2"].value == "北京" - assert workbook["差旅标准"]["B2"].value == "500" + assert workbook["差旅标准"]["B2"].value == 500 assert workbook["补贴标准"]["A2"].value == "直辖市" - assert workbook["补贴标准"]["B2"].value == "75" + assert workbook["补贴标准"]["B2"].value == 75 def build_workbook_bytes(rows: list[list[object]], *, sheet_name: str = "规则表") -> bytes: diff --git a/server/tests/test_agent_asset_tenant_security.py b/server/tests/test_agent_asset_tenant_security.py new file mode 100644 index 0000000..b8db7c0 --- /dev/null +++ b/server/tests/test_agent_asset_tenant_security.py @@ -0,0 +1,414 @@ +from __future__ import annotations + +from collections.abc import Generator +from datetime import UTC, datetime +from pathlib import Path +from types import MethodType + +import pytest +from auth_helpers import install_legacy_header_auth_override +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine, event, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.api.deps import CurrentUserContext, get_db +from app.api.v1.endpoints.agent_asset_releases import _actor as release_actor +from app.api.v1.endpoints.agent_assets import router as agent_assets_router +from app.db.base import Base +from app.models.agent_asset import AgentAsset, AgentAssetTestRun, AgentAssetVersion +from app.models.financial_record import ExpenseClaim +from app.models.tenant import Tenant +from app.schemas.agent_asset import AgentAssetRiskRuleScenarioTestRequest +from app.services.agent_asset_access import stable_user_principal +from app.services.agent_asset_onlyoffice_security import ( + AgentAssetOnlyOfficeSessionService, +) +from app.services.agent_assets import AgentAssetService +from app.services.settings import OnlyOfficeRuntimeConfig + + +def _factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False) + + +def _seed(db: Session) -> dict[str, AgentAsset]: + db.add_all( + [ + Tenant(tenant_id="platform", tenant_code="platform", name="平台管理域"), + Tenant(tenant_id="tenant-a", tenant_code="tenant-a", name="租户 A"), + Tenant(tenant_id="tenant-b", tenant_code="tenant-b", name="租户 B"), + ] + ) + assets = { + "platform": AgentAsset( + tenant_id="platform", + scope="platform", + asset_type="rule", + code="rule.shared", + name="平台规则", + domain="expense", + owner="platform", + status="draft", + current_version="v1.0.0", + working_version="v1.0.0", + ), + "tenant-a": AgentAsset( + tenant_id="tenant-a", + scope="tenant", + asset_type="rule", + code="rule.tenant", + name="租户 A 规则", + domain="expense", + owner="tenant-a", + status="draft", + current_version="v1.0.0", + working_version="v1.0.0", + ), + "tenant-b": AgentAsset( + tenant_id="tenant-b", + scope="tenant", + asset_type="rule", + code="rule.tenant", + name="租户 B 规则", + domain="expense", + owner="tenant-b", + status="draft", + current_version="v1.0.0", + working_version="v1.0.0", + ), + } + db.add_all(assets.values()) + db.flush() + for asset in assets.values(): + db.add( + AgentAssetVersion( + tenant_id=asset.tenant_id, + scope=asset.scope, + asset_id=asset.id, + version="v1.0.0", + content="# v1", + content_type="markdown", + created_by="seed", + ) + ) + db.commit() + return assets + + +def _client(factory: sessionmaker[Session], *, auth_override: bool = True) -> TestClient: + app = FastAPI() + app.include_router(agent_assets_router) + if auth_override: + install_legacy_header_auth_override(app) + + def override_db() -> Generator[Session, None, None]: + with factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + return TestClient(app) + + +def _headers( + tenant_id: str, + *, + employee_id: str = "employee-a", + is_admin: bool = False, +) -> dict[str, str]: + return { + "X-Auth-Username": f"user-{tenant_id}", + "X-Auth-Name": "Mutable Display Name", + "X-Auth-Employee-Id": employee_id, + "X-Auth-Role-Codes": "finance,manager", + "X-Auth-Is-Admin": "true" if is_admin else "false", + "X-Auth-Tenant-Id": tenant_id, + } + + +@pytest.fixture(autouse=True) +def _skip_foundation_bootstrap(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr( + "app.services.agent_foundation.AgentFoundationService.ensure_foundation_ready", + lambda _self: None, + ) + + +def test_asset_reads_are_authenticated_and_tenant_scoped() -> None: + factory = _factory() + with factory() as db: + assets = _seed(db) + ids = {key: value.id for key, value in assets.items()} + client = _client(factory) + + listing = client.get("/agent-assets", headers=_headers("tenant-a")) + assert listing.status_code == 200 + visible_ids = {item["id"] for item in listing.json()} + assert ids["platform"] in visible_ids + assert ids["tenant-a"] in visible_ids + assert ids["tenant-b"] not in visible_ids + + assert ( + client.get( + f"/agent-assets/{ids['tenant-b']}", headers=_headers("tenant-a") + ).status_code + == 404 + ) + assert ( + client.get( + f"/agent-assets/{ids['tenant-b']}/versions", + headers=_headers("tenant-a"), + ).status_code + == 404 + ) + + anonymous = _client(factory, auth_override=False).get("/agent-assets") + assert anonymous.status_code == 401 + + +def test_version_write_uses_stable_principal_and_blocks_cross_tenant_or_platform() -> None: + factory = _factory() + with factory() as db: + assets = _seed(db) + ids = {key: value.id for key, value in assets.items()} + client = _client(factory) + body = { + "version": "v1.0.1", + "content": "# tenant update", + "content_type": "markdown", + "created_by": "spoofed-display-name", + } + + cross_tenant = client.post( + f"/agent-assets/{ids['tenant-b']}/versions", + json=body, + headers=_headers("tenant-a"), + ) + assert cross_tenant.status_code == 404 + + platform_by_editor = client.post( + f"/agent-assets/{ids['platform']}/versions", + json=body, + headers=_headers("tenant-a"), + ) + assert platform_by_editor.status_code == 400 + + own = client.post( + f"/agent-assets/{ids['tenant-a']}/versions", + json=body, + headers=_headers("tenant-a", employee_id="employee-stable"), + ) + assert own.status_code == 201 + assert own.json()["created_by"] == "employee:employee-stable" + assert own.json()["tenant_id"] == "tenant-a" + assert own.json()["scope"] == "tenant" + + platform_admin = client.post( + f"/agent-assets/{ids['platform']}/versions", + json=body, + headers=_headers("tenant-a", is_admin=True), + ) + assert platform_admin.status_code == 201 + assert platform_admin.json()["scope"] == "platform" + + +def test_scenario_samples_filter_tenant_in_sql_and_persist_target_tenant() -> None: + factory = _factory() + with factory() as db: + assets = _seed(db) + platform_asset = assets["platform"] + db.add_all( + [ + _claim("tenant-a", "A-001"), + _claim("tenant-b", "B-001"), + ] + ) + db.commit() + statements: list[str] = [] + + @event.listens_for(db.get_bind(), "before_cursor_execute") + def _capture(_conn, _cursor, statement, _parameters, _context, _executemany): + if "FROM expense_claims" in statement: + statements.append(statement) + + current_user = CurrentUserContext( + username="reviewer-a", + name="显示名一", + role_codes=["manager"], + is_admin=True, + tenant_id="tenant-a", + employee_id="employee-reviewer-a", + ) + service = AgentAssetService(db, current_user=current_user) + + def _load(_self, _asset_id: str, _version: str | None): + return platform_asset, "v1.0.0", {} + + def _run(_self, _manifest: dict, claim: ExpenseClaim): + return {"claim_id": claim.id, "hit": False, "severity": "none"} + + service._load_risk_rule_for_test = MethodType(_load, service) + service._run_claim_scenario = MethodType(_run, service) + result = service.run_risk_rule_scenario_test( + platform_asset.id, + AgentAssetRiskRuleScenarioTestRequest( + target_tenant_id="tenant-a", + intent="最近 30 天", + ), + actor="employee:employee-reviewer-a", + ) + + assert result.result_json["total_count"] == 1 + assert statements + assert "expense_claims.tenant_id = ?" in statements[0] + run = db.scalar(select(AgentAssetTestRun).where(AgentAssetTestRun.id == result.id)) + assert run is not None + assert run.tenant_id == "tenant-a" + assert run.scope == "tenant" + assert run.input_json["target_tenant_id"] == "tenant-a" + + with pytest.raises(LookupError, match="Asset not found"): + service.run_risk_rule_scenario_test( + platform_asset.id, + AgentAssetRiskRuleScenarioTestRequest( + target_tenant_id="tenant-b", + intent="最近 30 天", + ), + actor="employee:employee-reviewer-a", + ) + + +def test_release_reviewer_identity_is_stable_when_display_name_changes() -> None: + first = CurrentUserContext( + username="reviewer", + name="显示名一", + role_codes=["manager"], + is_admin=False, + tenant_id="tenant-a", + employee_id="employee-reviewer", + ) + renamed = CurrentUserContext( + username="reviewer-renamed", + name="显示名二", + role_codes=["manager"], + is_admin=False, + tenant_id="tenant-a", + employee_id="employee-reviewer", + ) + assert stable_user_principal(first) == "employee:employee-reviewer" + assert stable_user_principal(first) == stable_user_principal(renamed) + assert release_actor(first) == release_actor(renamed) + + +def test_onlyoffice_callback_is_tenant_bound_and_one_time( + monkeypatch: pytest.MonkeyPatch, +) -> None: + factory = _factory() + jwt_secret = "agent-asset-onlyoffice-test-secret-32-bytes" + with factory() as db: + assets = _seed(db) + tenant_asset_id = assets["tenant-a"].id + other_asset_id = assets["tenant-b"].id + tokens = AgentAssetOnlyOfficeSessionService( + db, + jwt_secret=jwt_secret, + ).issue( + tenant_id="tenant-a", + resource_scope="tenant", + asset_id=tenant_asset_id, + document_key="tenant-a-document-key", + document_version="v1.0.0", + document_fingerprint="tenant-a-fingerprint", + writable=True, + actor="employee:onlyoffice-editor", + ) + + monkeypatch.setattr( + "app.services.agent_asset_onlyoffice.resolve_onlyoffice_settings", + lambda: OnlyOfficeRuntimeConfig( + enabled=True, + public_url="https://onlyoffice.example.com", + backend_url="https://backend.example.com", + jwt_secret=jwt_secret, + ), + ) + + def _save_scoped_callback(self, *, claimed, download_url): + assert download_url == "https://onlyoffice.example.com/download/tenant-a.xlsx" + assert claimed.tenant_id == "tenant-a" + assert self.repository.get(tenant_asset_id) is not None + assert self.repository.get(other_asset_id) is None + + monkeypatch.setattr( + AgentAssetService, + "_save_current_rule_spreadsheet_callback", + _save_scoped_callback, + ) + client = _client(factory) + payload = { + "status": 2, + "url": "https://onlyoffice.example.com/download/tenant-a.xlsx", + "key": "tenant-a-document-key", + } + + missing_token = client.post( + f"/agent-assets/{tenant_asset_id}/spreadsheet/onlyoffice/callback", + json=payload, + ) + assert missing_token.status_code == 422 + cross_asset = client.post( + f"/agent-assets/{other_asset_id}/spreadsheet/onlyoffice/callback", + params={"access_token": tokens.callback_token}, + json=payload, + ) + assert cross_asset.status_code == 401 + + first = client.post( + f"/agent-assets/{tenant_asset_id}/spreadsheet/onlyoffice/callback", + params={"access_token": tokens.callback_token}, + json=payload, + ) + assert first.status_code == 200 + replay = client.post( + f"/agent-assets/{tenant_asset_id}/spreadsheet/onlyoffice/callback", + params={"access_token": tokens.callback_token}, + json=payload, + ) + assert replay.status_code == 409 + + +def test_tenant_security_migration_has_fixed_ancestry_and_fail_closed_downgrade() -> None: + migration = Path( + "/app/server/alembic/versions/20260717_0026_agent_asset_tenant_security.py" + ).read_text(encoding="utf-8") + assert 'revision: str = "20260717_0026"' in migration + assert 'down_revision: str | None = "20260717_0025"' in migration + assert "uq_agent_assets_tenant_scope_code" in migration + assert "agent_asset_onlyoffice_sessions" in migration + assert "target_tenant_id" in migration + assert "_require_lossless_downgrade()" in migration + assert "_require_no_onlyoffice_sessions()" in migration + + +def _claim(tenant_id: str, claim_no: str) -> ExpenseClaim: + return ExpenseClaim( + tenant_id=tenant_id, + claim_no=claim_no, + employee_name="测试员工", + department_name="测试部门", + expense_type="差旅费", + reason="真实场景测试", + location="北京", + amount=100, + currency="CNY", + invoice_count=0, + occurred_at=datetime.now(UTC), + status="submitted", + ) diff --git a/server/tests/test_agent_run_tenant_security.py b/server/tests/test_agent_run_tenant_security.py new file mode 100644 index 0000000..496cfc8 --- /dev/null +++ b/server/tests/test_agent_run_tenant_security.py @@ -0,0 +1,301 @@ +from __future__ import annotations + +import json +from collections.abc import Generator +from datetime import UTC, datetime, timedelta + +import pytest +from auth_helpers import install_legacy_header_auth_override +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.api.deps import get_db +from app.api.v1.endpoints.agent_runs import router as agent_runs_router +from app.db.base import Base +from app.models.agent_run import AgentRun, AgentToolCall, SemanticParseLog +from app.services.agent_runs import AgentRunService +from app.services.finance_dashboard_scope import ( + FINANCE_DASHBOARD_TASK_TYPE, + resolve_finance_dashboard_data_scope, +) + + +def _session_factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False) + + +def _run( + *, + run_id: str, + started_at: datetime, + route_tenant_id: str | None, + ontology_tenant_id: str | None, + status: str = "succeeded", + error_message: str | None = None, +) -> AgentRun: + route_json: dict[str, object] = {"stage": "tenant-security-test"} + ontology_json: dict[str, object] = { + "scenario": "expense", + "intent": "query", + } + if route_tenant_id is not None: + route_json["tenant_id"] = route_tenant_id + if ontology_tenant_id is not None: + ontology_json["tenant_id"] = ontology_tenant_id + return AgentRun( + run_id=run_id, + agent="orchestrator", + source="user_message", + user_id=f"user-{run_id}", + ontology_json=ontology_json, + route_json=route_json, + permission_level="read", + status=status, + result_summary=f"summary-{run_id}", + error_message=error_message, + started_at=started_at, + finished_at=started_at + timedelta(seconds=1), + ) + + +def _seed_runs(db: Session) -> None: + now = datetime.now(UTC) + tenant_a = _run( + run_id="run-normal-tenant-a", + started_at=now - timedelta(minutes=4), + route_tenant_id="tenant-a", + ontology_tenant_id="tenant-a", + ) + tenant_b = _run( + run_id="run-normal-tenant-b", + started_at=now - timedelta(minutes=3), + route_tenant_id="tenant-b", + ontology_tenant_id="tenant-b", + status="failed", + error_message="tenant-b-run-secret", + ) + legacy_unscoped = _run( + run_id="run-normal-unscoped", + started_at=now - timedelta(minutes=2), + route_tenant_id=None, + ontology_tenant_id=None, + status="failed", + error_message="legacy-unscoped-secret", + ) + partial_scope = _run( + run_id="run-normal-partial-scope", + started_at=now - timedelta(seconds=90), + route_tenant_id="tenant-a", + ontology_tenant_id=None, + status="failed", + error_message="partial-scope-secret", + ) + conflicting = _run( + run_id="run-normal-conflicting", + started_at=now - timedelta(minutes=1), + route_tenant_id="tenant-a", + ontology_tenant_id="tenant-b", + status="failed", + error_message="conflicting-scope-secret", + ) + finance_scope = resolve_finance_dashboard_data_scope("tenant-a") + finance_snapshot = _run( + run_id="run-finance-tenant-a", + started_at=now, + route_tenant_id="tenant-a", + ontology_tenant_id="tenant-a", + status="failed", + error_message="finance-snapshot-secret", + ) + finance_snapshot.route_json.update( + { + "task_type": FINANCE_DASHBOARD_TASK_TYPE, + "data_scope": finance_scope, + } + ) + finance_snapshot.ontology_json["data_scope"] = finance_scope + db.add_all( + [ + tenant_a, + tenant_b, + legacy_unscoped, + partial_scope, + conflicting, + finance_snapshot, + ] + ) + db.flush() + db.add_all( + [ + AgentToolCall( + run_id=tenant_a.run_id, + tool_type="database", + tool_name="tenant-a.tool", + request_json={"private": "tenant-a-request"}, + response_json={"private": "tenant-a-response"}, + status="succeeded", + duration_ms=3, + ), + AgentToolCall( + run_id=tenant_b.run_id, + tool_type="database", + tool_name="tenant-b.tool", + request_json={"private": "tenant-b-request-secret"}, + response_json={"private": "tenant-b-response-secret"}, + status="failed", + duration_ms=5, + error_message="tenant-b-tool-secret", + ), + SemanticParseLog( + run_id=tenant_a.run_id, + user_id="tenant-a-user", + raw_query="tenant-a-raw-query", + scenario="expense", + intent="query", + confidence=0.9, + ), + SemanticParseLog( + run_id=tenant_b.run_id, + user_id="tenant-b-user", + raw_query="tenant-b-raw-query-secret", + scenario="expense", + intent="query", + confidence=0.9, + ), + ] + ) + db.commit() + + +def _client( + session_factory: sessionmaker[Session], +) -> TestClient: + app = FastAPI() + app.include_router(agent_runs_router) + install_legacy_header_auth_override(app) + + def override_db() -> Generator[Session, None, None]: + with session_factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + return TestClient(app) + + +def _headers(tenant_id: str, *, is_admin: bool = False) -> dict[str, str]: + return { + "X-Auth-Username": f"user-{tenant_id}", + "X-Auth-Role-Codes": "user", + "X-Auth-Is-Admin": "true" if is_admin else "false", + "X-Auth-Tenant-Id": tenant_id, + } + + +def test_agent_run_endpoints_fail_closed_for_normal_cross_tenant_runs() -> None: + session_factory = _session_factory() + with session_factory() as db: + _seed_runs(db) + client = _client(session_factory) + + tenant_a_list = client.get( + "/agent-runs", + params={"limit": 1}, + headers=_headers("tenant-a"), + ) + tenant_a_summary = client.get( + "/agent-runs/summary", + params={"limit": 1}, + headers=_headers("tenant-a"), + ) + tenant_b_list = client.get( + "/agent-runs", + headers=_headers("tenant-b"), + ) + + assert tenant_a_list.status_code == 200 + assert [item["run_id"] for item in tenant_a_list.json()] == ["run-normal-tenant-a"] + assert tenant_a_summary.status_code == 200 + assert tenant_a_summary.json()["total_runs"] == 1 + assert tenant_a_summary.json()["failed_runs"] == 0 + assert tenant_a_summary.json()["recent_errors"] == [] + assert tenant_b_list.status_code == 200 + assert [item["run_id"] for item in tenant_b_list.json()] == ["run-normal-tenant-b"] + + tenant_a_payload = json.dumps( + [tenant_a_list.json(), tenant_a_summary.json()], + ensure_ascii=False, + ) + assert "tenant-b" not in tenant_a_payload + assert "legacy-unscoped-secret" not in tenant_a_payload + assert "partial-scope-secret" not in tenant_a_payload + assert "conflicting-scope-secret" not in tenant_a_payload + assert "finance-snapshot-secret" not in tenant_a_payload + + own_detail = client.get( + "/agent-runs/run-normal-tenant-a", + headers=_headers("tenant-a"), + ) + assert own_detail.status_code == 200 + assert own_detail.json()["tool_calls"][0]["request_json"] == {"private": "tenant-a-request"} + assert own_detail.json()["semantic_parse"]["raw_query"] == "tenant-a-raw-query" + + hidden_run_ids = [ + "run-normal-tenant-b", + "run-normal-unscoped", + "run-normal-partial-scope", + "run-normal-conflicting", + ] + for run_id in hidden_run_ids: + response = client.get( + f"/agent-runs/{run_id}", + headers=_headers("tenant-a", is_admin=True), + ) + assert response.status_code == 404 + response_body = json.dumps(response.json(), ensure_ascii=False) + assert "tenant-b-request-secret" not in response_body + assert "tenant-b-response-secret" not in response_body + assert "tenant-b-raw-query-secret" not in response_body + + +def test_agent_run_creation_stamps_both_payloads_and_rejects_conflicts() -> None: + session_factory = _session_factory() + with session_factory() as db: + service = AgentRunService(db) + created = service.create_run( + agent="orchestrator", + source="user_message", + tenant_id="tenant-a", + ontology_json={"scenario": "expense"}, + route_json={"stage": "created"}, + status="running", + ) + + assert created.ontology_json["tenant_id"] == "tenant-a" + assert created.route_json["tenant_id"] == "tenant-a" + + updated = service.update_run( + created.run_id, + ontology_json={"scenario": "expense", "intent": "query"}, + route_json={"stage": "finished"}, + status="succeeded", + ) + assert updated.ontology_json["tenant_id"] == "tenant-a" + assert updated.route_json["tenant_id"] == "tenant-a" + + with pytest.raises(ValueError, match="tenant_id 与业务上下文冲突"): + service.create_run( + agent="orchestrator", + source="user_message", + tenant_id="tenant-a", + route_json={"tenant_id": "tenant-b"}, + status="running", + ) diff --git a/server/tests/test_alembic_migrations.py b/server/tests/test_alembic_migrations.py index 6a70b16..36d38b2 100644 --- a/server/tests/test_alembic_migrations.py +++ b/server/tests/test_alembic_migrations.py @@ -10,6 +10,22 @@ from typing import Any import pytest from alembic.config import Config +from commercial_migration_assertions import ( + _assert_commercial_head_schema, + _assert_commercial_runtime_invariants, +) +from financial_connector_migration_assertions import ( + _assert_financial_connector_head_schema, + _assert_financial_connector_runtime_invariants, +) +from release_telemetry_migration_assertions import ( + _assert_release_telemetry_head_schema, + _assert_release_telemetry_runtime_invariants, +) +from savings_migration_assertions import ( + _assert_savings_head_schema, + _assert_savings_runtime_invariants, +) from sqlalchemy import create_engine, inspect, text from sqlalchemy.engine import Engine, make_url from sqlalchemy.exc import IntegrityError @@ -30,7 +46,7 @@ from app.models.risk_observation import RiskObservation MIGRATION_TEST_DATABASE_URL = os.getenv("MIGRATION_TEST_DATABASE_URL", "").strip() LEGACY_PROBE_TABLE = "legacy_migration_probe_records" -HEAD_REVISION = "20260716_0014" +HEAD_REVISION = "20260717_0028" SERVER_DIR = Path(__file__).resolve().parents[1] ALEMBIC_INI_PATH = SERVER_DIR / "alembic.ini" @@ -734,6 +750,10 @@ def _assert_head_schema(engine: Engine) -> None: "attachment_association_jobs", "ck_attachment_association_jobs_generation", ) + _assert_savings_head_schema(engine) + _assert_commercial_head_schema(engine) + _assert_financial_connector_head_schema(engine) + _assert_release_telemetry_head_schema(engine) _assert_indexes( engine, @@ -1486,6 +1506,86 @@ def _assert_historical_case_downgrade_probe(engine: Engine) -> None: ) +def _create_legacy_connector_payload_probe(engine: Engine) -> None: + with engine.begin() as connection: + connection.execute( + text( + """ + INSERT INTO financial_connector_configs ( + id, tenant_id, provider, environment, key_version, secret_ref, + allowed_event_types_json, clock_skew_seconds, status, created_by + ) VALUES ( + 'legacy-connector-config', 'tenant-legacy', 'legacy-bank', + 'mock', 'v1', 'server/legacy', '["payment_settled"]', + 300, 'active', 'migration-probe' + ) + """ + ) + ) + connection.execute( + text( + """ + INSERT INTO financial_connector_events ( + id, tenant_id, config_id, provider, environment, direction, + external_event_id, event_type, occurred_at, key_version, + verification_level, request_fingerprint, content_hash, + processing_status, correlation_id, + normalized_payload_json, response_json + ) VALUES ( + 'legacy-connector-event', 'tenant-legacy', + 'legacy-connector-config', 'legacy-bank', 'mock', 'inbound', + 'legacy-external', 'payment_settled', now(), 'v1', 'simulated', + 'sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + 'sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb', + 'processed', 'legacy-correlation', + '{"claim_reference": "BX-SENSITIVE-001", "amount": "66.00"}', + '{"accepted": true, "reconciliation_case_id": "legacy-case"}' + ) + """ + ) + ) + + +def _assert_and_delete_legacy_connector_payload_probe(engine: Engine) -> None: + with engine.begin() as connection: + row = connection.execute( + text( + """ + SELECT config.version, + event.normalized_payload_json::jsonb ? 'claim_reference', + event.normalized_payload_json ->> 'amount', + event.response_json ->> 'projection_scope' + FROM financial_connector_configs AS config + JOIN financial_connector_events AS event + ON event.config_id = config.id + WHERE config.id = 'legacy-connector-config' + """ + ) + ).one() + assert row == (1, False, "66.00", "legacy_nonproduction_effect_unknown") + connection.execute( + text( + "ALTER TABLE financial_connector_events " + "DISABLE TRIGGER trg_financial_connector_events_append_only" + ) + ) + connection.execute( + text("DELETE FROM financial_connector_events WHERE id = 'legacy-connector-event'") + ) + connection.execute( + text( + "ALTER TABLE financial_connector_events " + "ENABLE TRIGGER trg_financial_connector_events_append_only" + ) + ) + connection.execute( + text( + "DELETE FROM financial_connector_configs " + "WHERE id = 'legacy-connector-config'" + ) + ) + + def _assert_legacy_sentinel(engine: Engine) -> None: assert LEGACY_PROBE_TABLE in _table_names(engine) with engine.connect() as connection: @@ -1524,6 +1624,34 @@ def _assert_base_schema(engine: Engine) -> None: ("20260716_0013_approval_tasks.py", "downgrade"), ("20260716_0014_risk_waiver_decision.py", "upgrade"), ("20260716_0014_risk_waiver_decision.py", "downgrade"), + ("20260716_0015_savings_value_ledger.py", "upgrade"), + ("20260716_0015_savings_value_ledger.py", "downgrade"), + ("20260716_0016_commercial_metering.py", "upgrade"), + ("20260716_0016_commercial_metering.py", "downgrade"), + ("20260716_0017_financial_connector_reconciliation.py", "upgrade"), + ("20260716_0017_financial_connector_reconciliation.py", "downgrade"), + ("20260716_0018_agent_asset_release_telemetry.py", "upgrade"), + ("20260716_0018_agent_asset_release_telemetry.py", "downgrade"), + ("20260716_0019_commercial_runtime_reservations.py", "upgrade"), + ("20260716_0019_commercial_runtime_reservations.py", "downgrade"), + ("20260716_0020_financial_connector_config_lifecycle.py", "upgrade"), + ("20260716_0020_financial_connector_config_lifecycle.py", "downgrade"), + ("20260716_0021_commercial_billing_periods.py", "upgrade"), + ("20260716_0021_commercial_billing_periods.py", "downgrade"), + ("20260716_0022_financial_connector_operational_events.py", "upgrade"), + ("20260716_0022_financial_connector_operational_events.py", "downgrade"), + ("20260716_0023_agent_asset_release_blind_audit.py", "upgrade"), + ("20260716_0023_agent_asset_release_blind_audit.py", "downgrade"), + ("20260717_0024_commercial_resource_quantity_bases.py", "upgrade"), + ("20260717_0024_commercial_resource_quantity_bases.py", "downgrade"), + ("20260717_0025_tenant_identity_foundation.py", "upgrade"), + ("20260717_0025_tenant_identity_foundation.py", "downgrade"), + ("20260717_0026_agent_asset_tenant_security.py", "upgrade"), + ("20260717_0026_agent_asset_tenant_security.py", "downgrade"), + ("20260717_0027_knowledge_tenant_security.py", "upgrade"), + ("20260717_0027_knowledge_tenant_security.py", "downgrade"), + ("20260717_0028_hermes_ontology_tenant_security.py", "upgrade"), + ("20260717_0028_hermes_ontology_tenant_security.py", "downgrade"), ], ) def test_postgresql_only_migrations_reject_other_dialects_before_mutation( @@ -1576,6 +1704,142 @@ def test_risk_waiver_migration_refuses_lossy_audit_downgrade() -> None: assert operation_guard.mutation_calls == [] +def test_savings_ledger_migration_refuses_non_empty_fact_downgrade() -> None: + migration = _load_migration_module("20260716_0015_savings_value_ledger.py") + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="immutable value facts exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_commercial_migration_refuses_non_empty_contract_or_fact_downgrade() -> None: + migration = _load_migration_module("20260716_0016_commercial_metering.py") + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="contracts or immutable facts exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_financial_connector_migration_refuses_non_empty_fact_downgrade() -> None: + migration = _load_migration_module( + "20260716_0017_financial_connector_reconciliation.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="configurations or immutable facts exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_release_telemetry_migration_refuses_non_empty_fact_downgrade() -> None: + migration = _load_migration_module( + "20260716_0018_agent_asset_release_telemetry.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="immutable observations or labels exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_runtime_reservation_migration_refuses_non_empty_hold_downgrade() -> None: + migration = _load_migration_module( + "20260716_0019_commercial_runtime_reservations.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="operational quota holds exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_connector_lifecycle_migration_refuses_lossy_downgrade() -> None: + migration = _load_migration_module( + "20260716_0020_financial_connector_config_lifecycle.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="versioned configuration state"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_commercial_billing_migration_refuses_non_empty_history_downgrade() -> None: + migration = _load_migration_module( + "20260716_0021_commercial_billing_periods.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="immutable periods or audit facts exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_connector_operational_migration_refuses_non_empty_history_downgrade() -> None: + migration = _load_migration_module( + "20260716_0022_financial_connector_operational_events.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="immutable operational facts exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_release_blind_audit_migration_refuses_non_empty_evidence_downgrade() -> None: + migration = _load_migration_module( + "20260716_0023_agent_asset_release_blind_audit.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="immutable audit evidence exists"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + +def test_commercial_resource_basis_migration_refuses_lossy_downgrade() -> None: + migration = _load_migration_module( + "20260717_0024_commercial_resource_quantity_bases.py" + ) + operation_guard = _UnsupportedDialectOperationGuard(dialect_name="postgresql") + operation_guard.bind.scalar = lambda _statement: 1 + migration.op = operation_guard + + with pytest.raises(RuntimeError, match="resource reservations exist"): + migration.downgrade() + + assert operation_guard.mutation_calls == [] + + def test_head_model_declares_soft_claim_reference_and_organization_only_active_index() -> None: claim_column = RiskObservation.__table__.c.claim_id assert not claim_column.foreign_keys @@ -1707,9 +1971,51 @@ def test_alembic_migration_cycle_on_disposable_postgres( }.isdisjoint(memory_columns) _delete_duplicate_active_organization_memory_probe(engine) + _upgrade_revision(migration_database_url, "20260716_0019") + _create_legacy_connector_payload_probe(engine) _upgrade_head(migration_database_url) + _assert_and_delete_legacy_connector_payload_probe(engine) _assert_head_schema(engine) assert validate_migration_state(engine).revision == HEAD_REVISION + _assert_savings_runtime_invariants(engine) + _assert_commercial_runtime_invariants(engine) + _assert_financial_connector_runtime_invariants(engine) + _assert_release_telemetry_runtime_invariants(engine) + with engine.begin() as connection: + connection.execute( + text( + """ + INSERT INTO profile_baseline_snapshots ( + id, tenant_id, baseline_key, baseline_type, dimension_type, + dimension_id, metric_key, unit, baseline_value, sample_count, + method, query_fingerprint, data_quality_status, + data_quality_score, algorithm_version, frozen_at, frozen_by + ) VALUES ( + 'downgrade-refusal-baseline', 'tenant-a', + 'downgrade-refusal-baseline', 'manual', 'tenant', 'tenant-a', + 'expense_amount', 'currency', 1, 0, 'migration_probe', + 'downgrade-refusal-fingerprint', 'complete', 1, + 'migration-probe-v1', now(), 'migration-probe' + ) + """ + ) + ) + with pytest.raises(RuntimeError, match="immutable value facts exist"): + _downgrade_revision(migration_database_url, "20260716_0014") + assert validate_migration_state(engine).revision == HEAD_REVISION + with engine.begin() as connection: + assert connection.scalar( + text( + "SELECT COUNT(*) FROM profile_baseline_snapshots " + "WHERE id = 'downgrade-refusal-baseline'" + ) + ) == 1 + connection.execute( + text( + "DELETE FROM profile_baseline_snapshots " + "WHERE id = 'downgrade-refusal-baseline'" + ) + ) create_legacy_schema(engine) assert "expense_claims" in _table_names(engine) diff --git a/server/tests/test_approval_risk_concurrency_postgres.py b/server/tests/test_approval_risk_concurrency_postgres.py index 2e51a65..f5b0460 100644 --- a/server/tests/test_approval_risk_concurrency_postgres.py +++ b/server/tests/test_approval_risk_concurrency_postgres.py @@ -42,6 +42,7 @@ def test_disposition_reopen_and_approval_share_claim_lock( disposition_id = f"disposition-lock-{suffix}" manager_email = f"manager-{suffix}@example.com" manager_user = CurrentUserContext( + tenant_id="default", username=manager_email, name="并发审批经理", role_codes=["manager"], @@ -146,12 +147,14 @@ def _seed_locked_risk_case( ) -> None: manager = Employee( id=f"manager-risk-lock-{suffix}", + tenant_id="default", employee_no=f"M-RISK-LOCK-{suffix}", name="并发审批经理", email=manager_email, ) employee = Employee( id=f"employee-risk-lock-{suffix}", + tenant_id="default", employee_no=f"E-RISK-LOCK-{suffix}", name="并发风险员工", email=f"employee-{suffix}@example.com", @@ -160,6 +163,7 @@ def _seed_locked_risk_case( now = datetime.now(UTC) claim = ExpenseClaim( id=claim_id, + tenant_id="default", claim_no=f"EXP-RISK-LOCK-{suffix}", employee=employee, employee_name=employee.name, diff --git a/server/tests/test_approval_task_actions.py b/server/tests/test_approval_task_actions.py index 7ec1948..7d95c86 100644 --- a/server/tests/test_approval_task_actions.py +++ b/server/tests/test_approval_task_actions.py @@ -59,6 +59,7 @@ def _seed_root_task( ) manager = Employee( id="employee-task-manager", + tenant_id=tenant_id, employee_no="M-TASK-001", name="李经理", email="manager-task@example.com", @@ -67,6 +68,7 @@ def _seed_root_task( participants = [ Employee( id=f"employee-task-participant-{index}", + tenant_id=tenant_id, employee_no=f"M-TASK-{index + 1:03d}", name=f"加签经理{index}", email=f"participant-task-{index}@example.com", @@ -76,6 +78,7 @@ def _seed_root_task( ] employee = Employee( id="employee-task-owner", + tenant_id=tenant_id, employee_no="E-TASK-001", name="张三", email="owner-task@example.com", @@ -84,6 +87,7 @@ def _seed_root_task( occurred_at = datetime.now(UTC) - timedelta(hours=1) claim = ExpenseClaim( id="claim-task-001", + tenant_id=tenant_id, claim_no="RE-TASK-001", employee=employee, employee_name=employee.name, @@ -154,6 +158,7 @@ def test_queue_is_tenant_safe_and_admin_has_no_implicit_approval() -> None: admin = Employee( id="employee-task-admin", + tenant_id="tenant-approval-task", employee_no="A-TASK-001", name="平台管理员", email="admin-task@example.com", diff --git a/server/tests/test_approval_task_backfill.py b/server/tests/test_approval_task_backfill.py index 92438d7..2eca522 100644 --- a/server/tests/test_approval_task_backfill.py +++ b/server/tests/test_approval_task_backfill.py @@ -65,6 +65,7 @@ def _persist_claim( manager = ( Employee( id=f"manager-{suffix}", + tenant_id=tenant_id, employee_no=f"M-{suffix}", name=f"经理{suffix}", email=f"manager-{suffix}@example.com", @@ -74,6 +75,7 @@ def _persist_claim( ) employee = Employee( id=f"employee-{suffix}", + tenant_id=tenant_id, employee_no=f"E-{suffix}", name=f"员工{suffix}", email=f"employee-{suffix}@example.com", @@ -81,6 +83,7 @@ def _persist_claim( ) claim = ExpenseClaim( id=f"claim-{suffix}", + tenant_id=tenant_id, claim_no=f"RE-BACKFILL-{suffix}", employee=employee, employee_name=employee.name, @@ -326,6 +329,7 @@ def test_entered_at_fallback_order_is_explicit_and_deterministic() -> None: submitted_at = datetime(2026, 7, 14, 10, 0, tzinfo=UTC) claim = ExpenseClaim( id="claim-time-fallback", + tenant_id="tenant-a", claim_no="RE-TIME-FALLBACK", employee_name="张三", department_name="市场部", diff --git a/server/tests/test_approval_task_concurrency_postgres.py b/server/tests/test_approval_task_concurrency_postgres.py index f3f85e8..fdb0631 100644 --- a/server/tests/test_approval_task_concurrency_postgres.py +++ b/server/tests/test_approval_task_concurrency_postgres.py @@ -17,6 +17,7 @@ from app.models.approval_task import ApprovalTask, ApprovalTaskEvent from app.models.employee import Employee from app.models.financial_record import ExpenseClaim from app.models.role import Role +from app.models.tenant import Tenant from app.schemas.approval_task import ApprovalTaskAssignmentAction from app.services.approval_task_actions import ApprovalTaskActionService from app.services.approval_task_lifecycle import ApprovalTaskLifecycleService @@ -138,6 +139,16 @@ def _seed_case( delegates: int, ) -> tuple[str, CurrentUserContext, list[str]]: with factory() as db: + if db.get(Tenant, TENANT_ID) is None: + db.add( + Tenant( + tenant_id=TENANT_ID, + tenant_code=TENANT_ID, + name="审批任务并发探针租户", + status="active", + ) + ) + db.flush() role = db.scalar(select(Role).where(Role.role_code == "manager")) if role is None: role = Role( @@ -147,6 +158,7 @@ def _seed_case( ) manager = Employee( id=f"manager-{suffix}", + tenant_id=TENANT_ID, employee_no=f"M-{suffix}", name="并发审批经理", email=f"manager-{suffix}@example.com", @@ -155,6 +167,7 @@ def _seed_case( delegate_rows = [ Employee( id=f"delegate-{suffix}-{index}", + tenant_id=TENANT_ID, employee_no=f"D-{suffix}-{index}", name=f"委托审批人{index + 1}", email=f"delegate-{suffix}-{index}@example.com", @@ -164,6 +177,7 @@ def _seed_case( ] claimant = Employee( id=f"claimant-{suffix}", + tenant_id=TENANT_ID, employee_no=f"E-{suffix}", name="并发报销申请人", email=f"claimant-{suffix}@example.com", @@ -172,6 +186,7 @@ def _seed_case( occurred_at = datetime.now(UTC) - timedelta(hours=1) claim = ExpenseClaim( id=f"claim-{suffix}", + tenant_id=TENANT_ID, claim_no=f"RE-CONCURRENT-{suffix}", employee=claimant, employee_name=claimant.name, diff --git a/server/tests/test_approval_task_query_and_batch.py b/server/tests/test_approval_task_query_and_batch.py index d21503c..a176a73 100644 --- a/server/tests/test_approval_task_query_and_batch.py +++ b/server/tests/test_approval_task_query_and_batch.py @@ -126,6 +126,7 @@ def _build_queue_task( selected_risk = risk_level or ("high" if index % 5 == 0 else "low") claim = ExpenseClaim( id=claim_id, + tenant_id=tenant_id, claim_no=f"RE-QUEUE-{tenant_id[-1].upper()}-{index:03d}", employee_id=None, employee_name=f"申请人{index:03d}", @@ -370,6 +371,7 @@ def _seed_batch( ) finance = Employee( id="batch-finance", + tenant_id="tenant-a", employee_no="BATCH-FINANCE", name="财务审批人", email="batch-finance@example.com", @@ -377,6 +379,7 @@ def _seed_batch( ) owner = Employee( id="batch-owner", + tenant_id="tenant-a", employee_no="BATCH-OWNER", name="批处理申请人", email="batch-owner@example.com", @@ -399,6 +402,7 @@ def _seed_batch( claim_id = f"batch-claim-{index}" claim = ExpenseClaim( id=claim_id, + tenant_id="tenant-a", claim_no=f"RE-BATCH-{index:03d}", employee=owner, employee_name=owner.name, diff --git a/server/tests/test_attachment_association_jobs.py b/server/tests/test_attachment_association_jobs.py index 0f24f49..0345731 100644 --- a/server/tests/test_attachment_association_jobs.py +++ b/server/tests/test_attachment_association_jobs.py @@ -347,6 +347,7 @@ def test_attachment_association_job_links_receipts_after_conversation_exit( try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -446,9 +447,7 @@ def test_attachment_association_job_links_receipts_after_conversation_exit( "attachment_associated", } expense_case = db.scalar( - select(ExpenseCase).where( - ExpenseCase.id == receipt_links[0].expense_case_id - ) + select(ExpenseCase).where(ExpenseCase.id == receipt_links[0].expense_case_id) ) assert expense_case is not None assert expense_case.current_stage == "claiming" @@ -480,6 +479,7 @@ def test_attachment_association_keeps_receipt_folder_preview_and_fields_after_ca try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -497,7 +497,10 @@ def test_attachment_association_keeps_receipt_folder_preview_and_fields_after_ca document=OcrRecognizeDocumentRead( filename="2月20 武汉-上海.pdf", media_type="application/pdf", - text="电子发票（铁路电子客票） 武汉站 G458 上海虹桥站 2026年02月20日 07:55开 二等座 票价 354.00", + text=( + "电子发票（铁路电子客票） 武汉站 G458 上海虹桥站 " + "2026年02月20日 07:55开 二等座 票价 354.00" + ), summary="铁路电子客票，武汉-上海，票价 354 元。", avg_score=0.96, line_count=1, @@ -597,6 +600,7 @@ def test_attachment_meta_repairs_existing_pdf_fallback_from_source_receipt( ) try: current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -617,7 +621,10 @@ def test_attachment_meta_repairs_existing_pdf_fallback_from_source_receipt( document=OcrRecognizeDocumentRead( filename="2月20 武汉-上海.pdf", media_type="application/pdf", - text="电子发票（铁路电子客票） 武汉站 G458 上海虹桥站 2026年02月20日 07:55开 二等座 票价 354.00", + text=( + "电子发票（铁路电子客票） 武汉站 G458 上海虹桥站 " + "2026年02月20日 07:55开 二等座 票价 354.00" + ), summary="铁路电子客票，武汉-上海，票价 354 元。", avg_score=0.96, line_count=1, @@ -708,6 +715,7 @@ def test_attachment_association_job_requests_confirmation_without_editable_claim try: client, _session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -766,6 +774,7 @@ def test_confirmation_job_is_re_evaluated_after_draft_is_created( try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -823,9 +832,7 @@ def test_confirmation_job_is_re_evaluated_after_draft_is_created( with session_factory() as db: jobs = list( db.scalars( - select(AttachmentAssociationJob).order_by( - AttachmentAssociationJob.generation - ) + select(AttachmentAssociationJob).order_by(AttachmentAssociationJob.generation) ).all() ) assert [(job.generation, job.resolution) for job in jobs] == [ @@ -847,6 +854,7 @@ def test_attachment_association_job_returns_application_candidate_without_creati try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -905,6 +913,7 @@ def test_attachment_association_job_is_idempotent_for_same_receipt(monkeypatch, try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -980,6 +989,7 @@ def test_attachment_association_ambiguous_match_has_no_business_writes( try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1056,6 +1066,7 @@ def test_attachment_association_rolls_back_when_event_write_fails(monkeypatch, t try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1130,6 +1141,7 @@ def test_attachment_association_failure_keeps_preexisting_application_case( try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1294,6 +1306,7 @@ def test_confirmation_match_does_not_repair_or_commit_unrelated_claim( db.add_all([employee, claim]) db.commit() current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1345,6 +1358,7 @@ def test_mixed_receipt_batch_requires_confirmation_without_writes( try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1422,6 +1436,7 @@ def test_attachment_write_failure_restores_previous_directory_and_business_state try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1494,6 +1509,7 @@ def test_concurrent_jobs_for_same_receipt_are_serialized(monkeypatch, tmp_path) try: _client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1592,6 +1608,7 @@ def test_concurrent_receipts_for_same_claim_are_serialized(monkeypatch, tmp_path try: _client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1657,9 +1674,7 @@ def test_concurrent_receipts_for_same_claim_are_serialized(monkeypatch, tmp_path assert not thread.is_alive() with session_factory() as db: - results = [ - get_attachment_association_job(job.job_id, current_user, db) for job in jobs - ] + results = [get_attachment_association_job(job.job_id, current_user, db) for job in jobs] claim = db.scalar( select(ExpenseClaim) .options(selectinload(ExpenseClaim.items)) @@ -1672,9 +1687,7 @@ def test_concurrent_receipts_for_same_claim_are_serialized(monkeypatch, tmp_path ) receipt_links = list( db.scalars( - select(ExpenseCaseLink).where( - ExpenseCaseLink.resource_type == "receipt" - ) + select(ExpenseCaseLink).where(ExpenseCaseLink.resource_type == "receipt") ).all() ) @@ -1694,9 +1707,7 @@ def test_concurrent_receipts_for_same_claim_are_serialized(monkeypatch, tmp_path ) assert len(events) == 4 assert len(receipt_links) == 2 - assert { - (event.aggregate_id, event.event_type) for event in events - } == { + assert {(event.aggregate_id, event.event_type) for event in events} == { (receipt_id, event_type) for receipt_id in receipt_ids for event_type in ("receipt_received", "attachment_associated") @@ -1706,15 +1717,16 @@ def test_concurrent_receipts_for_same_claim_are_serialized(monkeypatch, tmp_path for receipt_id in receipt_ids ] assert all(receipt.status == "linked" for receipt in linked_receipts) - assert all( - receipt.linked_claim_id == "claim-bg-association" for receipt in linked_receipts + assert all(receipt.linked_claim_id == "claim-bg-association" for receipt in linked_receipts) + assert ( + len( + { + str((receipt.raw_meta or {}).get("linked_item_id") or "") + for receipt in linked_receipts + } + ) + == 2 ) - assert len( - { - str((receipt.raw_meta or {}).get("linked_item_id") or "") - for receipt in linked_receipts - } - ) == 2 finally: clear_attachment_association_jobs_for_tests() get_settings.cache_clear() @@ -1731,6 +1743,7 @@ def test_persistent_job_resumes_after_process_state_is_cleared(monkeypatch, tmp_ try: client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1789,6 +1802,7 @@ def test_expired_worker_cannot_overwrite_newer_job_attempt(monkeypatch) -> None: try: _client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1870,6 +1884,7 @@ def test_failed_job_creates_new_generation_without_rewriting_history(monkeypatch try: _client, session_factory = build_client(monkeypatch) current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["user"], @@ -1895,9 +1910,7 @@ def test_failed_job_creates_new_generation_without_rewriting_history(monkeypatch second = create_attachment_association_job(payload, current_user, db) jobs = list( db.scalars( - select(AttachmentAssociationJob).order_by( - AttachmentAssociationJob.generation - ) + select(AttachmentAssociationJob).order_by(AttachmentAssociationJob.generation) ).all() ) diff --git a/server/tests/test_auth_service.py b/server/tests/test_auth_service.py index f53d93d..27a83c3 100644 --- a/server/tests/test_auth_service.py +++ b/server/tests/test_auth_service.py @@ -29,7 +29,7 @@ def build_session() -> Session: def test_employee_can_login_with_seed_default_password() -> None: with build_session() as db: - employee = EmployeeService(db).list_employees()[0] + employee = EmployeeService(db, tenant_id="default").list_employees()[0] result = AuthService(db).login( LoginRequest(username=employee.email, password="123456") ) @@ -50,8 +50,11 @@ def test_employee_can_login_with_seed_default_password() -> None: def test_current_user_snapshot_refreshes_employee_position() -> None: with build_session() as db: - employee = EmployeeService(db).list_employees()[0] - result = AuthService(db).get_user_snapshot(employee.email) + employee = EmployeeService(db, tenant_id="default").list_employees()[0] + result = AuthService(db).get_user_snapshot( + employee.email, + tenant_id="default", + ) assert result is not None assert result.username == employee.email @@ -83,7 +86,7 @@ def test_admin_can_login_with_database_password() -> None: def test_disabled_employee_cannot_login() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] service.disable_employee(employee.id) @@ -97,7 +100,7 @@ def test_disabled_employee_cannot_login() -> None: def test_reenabled_employee_can_login_again() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] service.disable_employee(employee.id) service.enable_employee(employee.id) @@ -125,7 +128,7 @@ def test_employee_login_skips_directory_bootstrap_when_employee_exists(monkeypat monkeypatch.setattr( AuthService, "_find_employee_by_email", - lambda self, _: ExistingEmployee(), + lambda self, _, requested_tenant: ExistingEmployee(), ) monkeypatch.setattr( "app.services.auth.verify_password", @@ -150,11 +153,16 @@ def test_employee_login_skips_directory_bootstrap_when_employee_exists(monkeypat role_codes=["user"], email=employee.email, avatar="D", + tenant_id="default", ), ) monkeypatch.setattr(EmployeeService, "ensure_directory_ready", fail_if_bootstrapped) - user = service._authenticate_employee("demo@example.com", "123456") + user = service._authenticate_employee( + "demo@example.com", + "123456", + requested_tenant="default", + ) assert user is not None assert user.username == "demo@example.com" @@ -179,6 +187,7 @@ def test_login_session_write_rolls_back_metric_when_token_issue_fails(monkeypatc role_codes=["user"], email="rollback@example.com", avatar="R", + tenant_id="default", ) def fail_issue(*args, **kwargs): diff --git a/server/tests/test_auth_session_endpoints.py b/server/tests/test_auth_session_endpoints.py index 3b742d8..30a8abd 100644 --- a/server/tests/test_auth_session_endpoints.py +++ b/server/tests/test_auth_session_endpoints.py @@ -136,6 +136,7 @@ def test_expired_and_revoked_tokens_are_rejected() -> None: def test_manager_role_is_not_platform_admin() -> None: manager = CurrentUserContext( + tenant_id="default", username="manager@example.com", name="Manager", role_codes=["manager"], diff --git a/server/tests/test_automation_eligibility.py b/server/tests/test_automation_eligibility.py new file mode 100644 index 0000000..ae77999 --- /dev/null +++ b/server/tests/test_automation_eligibility.py @@ -0,0 +1,166 @@ +from __future__ import annotations + +from decimal import Decimal + +from app.services.automation_eligibility import ( + AutomationEligibilityCalculator, + AutomationEligibilityInput, + AutomationLevel, + AutomationPolicy, +) + + +def _enabled_policy(**updates) -> AutomationPolicy: + values = { + "enabled": True, + "enterprise_max_level": AutomationLevel.L4_LOW_RISK_STRAIGHT_THROUGH, + "enterprise_whitelist": frozenset({"expense.prefill_fields"}), + "amount_cap": Decimal("1000"), + "min_confidence": 0.95, + "min_evidence_completeness": 0.95, + "min_historical_precision": 0.98, + "min_historical_samples": 100, + "minimum_sampling_rate": 0.1, + "release_stage": "active", + } + values.update(updates) + return AutomationPolicy(**values) + + +def _qualified_input(**updates) -> AutomationEligibilityInput: + values = { + "action_key": "expense.prefill_fields", + "action_risk": "low", + "amount": Decimal("0"), + "confidence": 0.99, + "evidence_completeness": 1.0, + "reversible": True, + "historical_precision": 0.995, + "historical_samples": 1000, + "sampling_rate": 0.2, + } + values.update(updates) + return AutomationEligibilityInput(**values) + + +def test_default_policy_is_shadow_only() -> None: + decision = AutomationEligibilityCalculator().calculate(_qualified_input()) + + assert decision.eligible_level == AutomationLevel.L1_RECOMMEND + assert decision.eligible_for_auto_execution is False + assert decision.requires_human_confirmation is True + + +def test_hard_whitelist_cannot_be_extended_by_enterprise() -> None: + decision = AutomationEligibilityCalculator().calculate( + _qualified_input(action_key="expense.unknown_action"), + _enabled_policy(enterprise_whitelist=frozenset({"expense.unknown_action"})), + ) + + assert decision.eligible_level == AutomationLevel.L1_RECOMMEND + assert "action_not_in_hard_whitelist" in decision.reasons + + +def test_payment_and_high_risk_policy_actions_never_auto_execute() -> None: + calculator = AutomationEligibilityCalculator() + payment = calculator.calculate( + _qualified_input(action_key="expense.execute_payment", moves_money=True), + _enabled_policy(enterprise_whitelist=frozenset({"expense.execute_payment"})), + ) + policy_change = calculator.calculate( + _qualified_input( + action_key="expense.publish_policy", + changes_policy=True, + action_risk="high", + ), + _enabled_policy(enterprise_whitelist=frozenset({"expense.publish_policy"})), + ) + + assert payment.eligible_level == AutomationLevel.L5_HUMAN_CONTROLLED + assert policy_change.eligible_level == AutomationLevel.L5_HUMAN_CONTROLLED + assert not payment.eligible_for_auto_execution + assert not policy_change.eligible_for_auto_execution + + +def test_all_controls_can_reach_enterprise_cap() -> None: + decision = AutomationEligibilityCalculator().calculate( + _qualified_input(), + _enabled_policy(enterprise_max_level=AutomationLevel.L3_REVERSIBLE_AUTO), + ) + + assert decision.eligible_level == AutomationLevel.L3_REVERSIBLE_AUTO + assert decision.eligible_for_auto_execution is True + assert "clamped_by_enterprise_level" in decision.reasons + + +def test_each_safety_gap_clamps_to_human_confirmation() -> None: + calculator = AutomationEligibilityCalculator() + candidates = [ + _qualified_input(amount=Decimal("1001")), + _qualified_input(action_risk="medium"), + _qualified_input(reversible=False), + _qualified_input(confidence=0.7), + _qualified_input(evidence_completeness=0.7), + _qualified_input(historical_precision=None), + _qualified_input(historical_samples=10), + ] + + for candidate in candidates: + decision = calculator.calculate(candidate, _enabled_policy()) + assert decision.eligible_level <= AutomationLevel.L2_PREFILL + assert decision.requires_human_confirmation is True + + +def test_low_sampling_rate_prevents_all_auto_execution() -> None: + decision = AutomationEligibilityCalculator().calculate( + _qualified_input(sampling_rate=0.01), + _enabled_policy(), + ) + + assert decision.eligible_level == AutomationLevel.L2_PREFILL + assert decision.eligible_for_auto_execution is False + assert "sampling_rate_below_threshold" in decision.reasons + + +def test_invalid_amount_and_enterprise_l0_cap_fail_closed() -> None: + calculator = AutomationEligibilityCalculator() + invalid_amount = calculator.calculate( + _qualified_input(amount=Decimal("NaN")), + _enabled_policy(), + ) + human_only = calculator.calculate( + _qualified_input(), + _enabled_policy(enterprise_max_level=AutomationLevel.L0_EXPLAIN), + ) + + assert invalid_amount.eligible_level == AutomationLevel.L2_PREFILL + assert "amount_invalid" in invalid_amount.reasons + assert human_only.eligible_level == AutomationLevel.L0_EXPLAIN + + +def test_even_qualified_action_stays_non_executing_in_shadow() -> None: + decision = AutomationEligibilityCalculator().calculate( + _qualified_input(), + _enabled_policy(release_stage="shadow"), + ) + + assert decision.eligible_level == AutomationLevel.L4_LOW_RISK_STRAIGHT_THROUGH + assert decision.mode == "shadow_low_risk_straight_through" + assert decision.eligible_for_auto_execution is False + + +def test_malformed_levels_and_sample_counts_fail_closed_without_runtime_error() -> None: + decision = AutomationEligibilityCalculator().calculate( + _qualified_input(historical_samples="not-a-number"), + _enabled_policy( + enterprise_max_level="not-a-level", + min_historical_samples="not-a-count", + ), + ) + + assert decision.eligible_level == AutomationLevel.L1_RECOMMEND + assert decision.eligible_for_auto_execution is False + assert decision.checks["historical_samples"] == 0 + assert decision.checks["historical_samples_valid"] is False + assert decision.checks["minimum_historical_samples_valid"] is False + assert decision.checks["enterprise_max_level"] == 1 diff --git a/server/tests/test_cfo_value_analytics.py b/server/tests/test_cfo_value_analytics.py new file mode 100644 index 0000000..ef1cfae --- /dev/null +++ b/server/tests/test_cfo_value_analytics.py @@ -0,0 +1,319 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, date, datetime, timedelta +from decimal import Decimal + +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext +from app.db.base_class import Base +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.schemas.cfo_value import CfoValueFiltersRead +from app.schemas.savings import ( + SavingsEvidenceCreate, + SavingsRealizationActionCreate, + SavingsRealizationCreate, +) +from app.services.cfo_value_analytics import CfoValueAnalyticsService +from app.services.savings_access_policy import SavingsPermissionError +from app.services.savings_discovery import SavingsDiscoveryService +from app.services.savings_realization import SavingsRealizationService + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_cfo_value_counts_only_confirmed_canonical_and_replays_as_of(db: Session) -> None: + operator = _user("finance-operator", roles=["finance"]) + confirmer = _user("finance-confirmer", roles=["finance"]) + cny_opportunity = _discover(db, operator, currency="CNY", saving=Decimal("200")) + usd_opportunity = _discover(db, operator, currency="USD", saving=Decimal("50")) + db.commit() + + cny_realization = ( + SavingsRealizationService(db) + .record( + cny_opportunity, + _record_payload("record-cny-001", Decimal("200"), "CNY"), + operator, + ) + .response.realization + ) + usd_realization = ( + SavingsRealizationService(db) + .record( + usd_opportunity, + _record_payload("record-usd-001", Decimal("50"), "USD"), + operator, + ) + .response.realization + ) + + start = datetime.now(UTC) - timedelta(days=2) + end = datetime.now(UTC) + timedelta(days=2) + pending_dashboard = CfoValueAnalyticsService(db).build_dashboard( + confirmer, + start=start, + end=end, + as_of=datetime.now(UTC), + filters=CfoValueFiltersRead(), + ) + assert pending_dashboard.kpis.verified_cash.status == "empty" + assert pending_dashboard.data_quality.pending_confirmation_count == 2 + assert _stage(pending_dashboard, "actual_pending").count == 2 + + confirmed_cny = ( + SavingsRealizationService(db) + .execute_action( + cny_realization.id, + SavingsRealizationActionCreate( + action="confirm", + request_id="confirm-cny-001", + expected_version=1, + comment="独立复核 CNY 结果与归因", + ), + confirmer, + ) + .response + ) + SavingsRealizationService(db).execute_action( + usd_realization.id, + SavingsRealizationActionCreate( + action="confirm", + request_id="confirm-usd-001", + expected_version=1, + comment="独立复核 USD 结果与归因", + ), + confirmer, + ) + confirmed_as_of = confirmed_cny.event.occurred_at + + current = CfoValueAnalyticsService(db).build_dashboard( + confirmer, + start=start, + end=end, + as_of=datetime.now(UTC), + filters=CfoValueFiltersRead(), + ) + assert _money(current.kpis.verified_cash.values) == { + "CNY": Decimal("200.0000"), + "USD": Decimal("50.0000"), + } + assert current.kpis.releasable_labor.status == "collecting" + assert current.kpis.safe_straight_through.status == "collecting" + assert ( + next( + item for item in current.guardrails if item.key == "confirmed_high_risk_exposure" + ).status + == "unavailable" + ) + + SavingsRealizationService(db).execute_action( + cny_realization.id, + SavingsRealizationActionCreate( + action="reverse", + request_id="reverse-cny-001", + expected_version=2, + comment="补付后全额冲回 CNY 节省", + reversal_amount=Decimal("200"), + ), + confirmer, + ) + latest = CfoValueAnalyticsService(db).build_dashboard( + confirmer, + start=start, + end=end, + as_of=datetime.now(UTC), + filters=CfoValueFiltersRead(), + ) + assert _money(latest.kpis.verified_cash.values) == { + "CNY": Decimal("0.0000"), + "USD": Decimal("50.0000"), + } + assert _stage(latest, "reversed").count == 1 + + historical = CfoValueAnalyticsService(db).build_dashboard( + confirmer, + start=start, + end=end, + as_of=confirmed_as_of, + filters=CfoValueFiltersRead(), + ) + assert _money(historical.kpis.verified_cash.values)["CNY"] == Decimal("200.0000") + + with pytest.raises(SavingsPermissionError): + CfoValueAnalyticsService(db).build_dashboard( + _user("employee"), + start=start, + end=end, + as_of=datetime.now(UTC), + filters=CfoValueFiltersRead(), + ) + + +def test_cfo_value_budget_monitor_is_limited_to_own_department(db: Session) -> None: + operator = _user("finance-operator", roles=["finance"]) + _discover( + db, + operator, + currency="CNY", + saving=Decimal("100"), + department_name="销售部", + ) + _discover( + db, + operator, + currency="CNY", + saving=Decimal("300"), + department_name="研发部", + ) + db.commit() + dashboard = CfoValueAnalyticsService(db).build_dashboard( + _user("budget-sales", roles=["budget_monitor"], department_name="销售部"), + start=datetime.now(UTC) - timedelta(days=1), + end=datetime.now(UTC) + timedelta(days=1), + as_of=datetime.now(UTC), + filters=CfoValueFiltersRead(), + ) + assert dashboard.source.opportunity_count == 1 + assert _money(_stage(dashboard, "estimated").values) == { + "CNY": Decimal("100.0000") + } + + +def _discover( + db: Session, + current_user: CurrentUserContext, + *, + currency: str, + saving: Decimal, + department_name: str = "销售部", +) -> str: + original = Decimal("1000") + target = original - saving + claim = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no=f"BX-{uuid.uuid4().hex[:12]}", + employee_name="测试员工", + department_name=department_name, + project_code="PROJECT-CFO", + expense_type="hotel", + reason="客户现场差旅", + location="上海", + amount=original, + currency=currency, + invoice_count=1, + occurred_at=datetime.now(UTC), + status="draft", + risk_flags_json=[], + ) + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim=claim, + item_date=date.today(), + item_type="hotel", + item_reason="住宿", + item_location="上海", + item_note="", + item_amount=original, + ) + db.add(claim) + db.flush() + return ( + SavingsDiscoveryService(db) + .discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[ + { + "item_id": item.id, + "message": "CFO 聚合测试政策差额", + "original_amount": str(original), + "reimbursable_amount": str(target), + "employee_absorbed_amount": str(saving), + "policy_rule_version": f"cfo-{currency.lower()}-v1", + "policy_grade": "P6", + "policy_matched_city": "上海", + "calculation_fingerprint": "sha256:" + uuid.uuid4().hex * 2, + } + ], + current_user=current_user, + request_id=f"discover-{currency.lower()}-{uuid.uuid4().hex[:8]}", + )[0] + .id + ) + + +def _record_payload( + request_id: str, + amount: Decimal, + currency: str, +) -> SavingsRealizationCreate: + return SavingsRealizationCreate( + request_id=request_id, + expected_version=1, + comment="登记待独立确认的实际结果", + actual_gross=amount, + incremental_cost=Decimal("0"), + currency=currency, + realized_at=datetime.now(UTC), + attribution_method="server_policy_counterfactual", + attribution_ratio=Decimal("1"), + evidence_level="business_state", + evidence=[ + SavingsEvidenceCreate( + evidence_key=f"evidence-{request_id}", + evidence_role="payment_business_state", + resource_type="business_event", + resource_id=f"payment-{request_id}", + source_system="x-financial", + external_event_id=f"payment-{request_id}", + content_hash="b" * 64, + occurred_at=datetime.now(UTC), + verification_status="unverified", + metadata_json={"source": "cfo-test"}, + ) + ], + ) + + +def _user( + username: str, + *, + roles: list[str] | None = None, + department_name: str = "", +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=False, + tenant_id="default", + employee_id=username, + department_name=department_name, + ) + + +def _money(items) -> dict[str, Decimal]: + return {item.currency: item.amount for item in items} + + +def _stage(dashboard, key: str): + return next(item for item in dashboard.funnel.stages if item.key == key) diff --git a/server/tests/test_commercial_billing_periods.py b/server/tests/test_commercial_billing_periods.py new file mode 100644 index 0000000..799bf1e --- /dev/null +++ b/server/tests/test_commercial_billing_periods.py @@ -0,0 +1,230 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.db.base_class import Base +from app.models.commercial_billing import CommercialAdminEvent, CommercialBillingPeriod +from app.schemas.commercial import CommercialPlanCreate, CommercialSubscriptionCreate +from app.services.commercial_admin import CommercialAdminService +from app.services.commercial_subscription_rollover import ( + CommercialSubscriptionRolloverService, +) + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_subscription_creation_issues_period_and_redacted_audit(db: Session) -> None: + start = datetime(2026, 1, 1, tzinfo=UTC) + end = datetime(2026, 2, 1, tzinfo=UTC) + _, subscription = _seed_subscription( + db, + "tenant-a", + start=start, + end=end, + request_prefix="initial", + ) + + periods = list( + db.scalars( + select(CommercialBillingPeriod).where(CommercialBillingPeriod.tenant_id == "tenant-a") + ).all() + ) + assert len(periods) == 1 + assert periods[0].subscription_id == subscription.id + assert periods[0].period_start.replace(tzinfo=UTC) == start + assert periods[0].period_end.replace(tzinfo=UTC) == end + assert periods[0].base_fee_snapshot == Decimal("100.0000") + + events = list( + db.scalars( + select(CommercialAdminEvent).where(CommercialAdminEvent.tenant_id == "tenant-a") + ).all() + ) + assert {event.action for event in events} >= { + "plan_created", + "plan_activated", + "subscription_created", + "billing_period_created", + } + serialized = str([(event.before_json, event.after_json) for event in events]).casefold() + for forbidden in ( + "contract_terms_json", + "metadata_json", + "external_subscription_id", + "secret", + "password", + "token", + ): + assert forbidden not in serialized + + +def test_auto_renew_rollover_is_idempotent_and_keeps_month_end_anchor(db: Session) -> None: + start = datetime(2027, 1, 31, tzinfo=UTC) + due = datetime(2027, 2, 28, tzinfo=UTC) + _, subscription = _seed_subscription( + db, + "tenant-rollover", + start=start, + end=due, + auto_renew=True, + request_prefix="rollover", + ) + + first = CommercialSubscriptionRolloverService(db).rollover_due( + "tenant-rollover", + subscription.id, + as_of=due, + ) + replay = CommercialSubscriptionRolloverService(db).rollover_due( + "tenant-rollover", + subscription.id, + as_of=due, + ) + + assert first.status == "rolled_over" + assert len(first.created_period_ids) == 1 + assert first.current_period_start == due + assert first.current_period_end == datetime(2027, 3, 31, tzinfo=UTC) + assert replay.status == "not_due" + periods = list( + db.scalars( + select(CommercialBillingPeriod) + .where(CommercialBillingPeriod.subscription_id == subscription.id) + .order_by(CommercialBillingPeriod.period_sequence) + ).all() + ) + assert [period.period_sequence for period in periods] == [1, 2] + assert periods[0].period_start.replace(tzinfo=UTC) == start + assert periods[0].period_end.replace(tzinfo=UTC) == due + assert periods[1].source == "auto_renew" + assert subscription.version == 2 + actions = set( + db.scalars( + select(CommercialAdminEvent.action).where( + CommercialAdminEvent.tenant_id == "tenant-rollover" + ) + ).all() + ) + assert {"billing_period_created", "subscription_rolled_over"} <= actions + + +@pytest.mark.parametrize( + ("interval", "contract_end", "reason_code"), + [ + ("contract", None, "contract_interval_requires_explicit_renewal"), + ( + "monthly", + datetime(2026, 2, 15, tzinfo=UTC), + "contract_boundary_requires_renewal", + ), + ], +) +def test_rollover_fails_closed_at_unprovable_contract_boundary( + db: Session, + interval: str, + contract_end: datetime | None, + reason_code: str, +) -> None: + tenant_id = f"tenant-{interval}" + _, subscription = _seed_subscription( + db, + tenant_id, + start=datetime(2026, 1, 1, tzinfo=UTC), + end=datetime(2026, 2, 1, tzinfo=UTC), + billing_interval=interval, + auto_renew=True, + contract_end=contract_end, + request_prefix=interval, + ) + + result = CommercialSubscriptionRolloverService(db).rollover_due( + tenant_id, + subscription.id, + as_of=datetime(2026, 2, 1, tzinfo=UTC), + ) + + assert result.status == "ineligible" + assert result.reason_code == reason_code + assert ( + db.scalar( + select(CommercialBillingPeriod.period_sequence).where( + CommercialBillingPeriod.subscription_id == subscription.id + ) + ) + == 1 + ) + + +def _seed_subscription( + db: Session, + tenant_id: str, + *, + start: datetime, + end: datetime, + billing_interval: str = "monthly", + auto_renew: bool = False, + contract_end: datetime | None = None, + request_prefix: str, +): + admin = CommercialAdminService(db) + plan = admin.create_plan( + tenant_id, + CommercialPlanCreate( + plan_code="billing", + name="不可变账期测试套餐", + pricing_model="subscription", + billing_interval=billing_interval, + currency="CNY", + base_fee=Decimal("100"), + effective_from=start, + contract_terms_json={"secret": "must-not-enter-audit"}, + ), + actor_id="platform-admin", + request_id=f"{request_prefix}-plan-create", + ) + admin.activate_plan( + tenant_id, + plan.id, + expected_version=plan.version, + actor_id="platform-admin", + request_id=f"{request_prefix}-plan-activate", + reason="测试激活套餐", + ) + subscription = admin.create_subscription( + tenant_id, + CommercialSubscriptionCreate( + subscription_key=f"{tenant_id}-subscription", + plan_id=plan.id, + starts_at=start, + ends_at=contract_end, + current_period_start=start, + current_period_end=end, + seats=3, + auto_renew=auto_renew, + metadata_json={"token": "must-not-enter-audit"}, + ), + actor_id="platform-admin", + request_id=f"{request_prefix}-subscription-create", + ) + db.flush() + return plan, subscription diff --git a/server/tests/test_commercial_concurrency_postgres.py b/server/tests/test_commercial_concurrency_postgres.py new file mode 100644 index 0000000..136984c --- /dev/null +++ b/server/tests/test_commercial_concurrency_postgres.py @@ -0,0 +1,387 @@ +from __future__ import annotations + +import threading +import uuid +from concurrent.futures import ThreadPoolExecutor +from datetime import UTC, datetime, timedelta +from decimal import Decimal + +from savings_postgres_testkit import ( # noqa: F401 - 注册 pg_factory fixture + _pg_factory_fixture, +) +from sqlalchemy import func, select +from sqlalchemy.orm import Session, sessionmaker + +from app.models.commercial import CommercialCostEvent, UsageMeterEvent +from app.models.commercial_billing import CommercialBillingPeriod +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.schemas.commercial import ( + CommercialCostEventCreate, + CommercialEntitlementUpsert, + CommercialPlanCreate, + CommercialSubscriptionCreate, + UsageMeterEventCreate, +) +from app.services.commercial_access_policy import CommercialConflictError +from app.services.commercial_admin import CommercialAdminService +from app.services.commercial_metering import CommercialMeteringService +from app.services.commercial_runtime_reservations import ( + CommercialRuntimeReservationService, +) +from app.services.commercial_subscription_rollover import ( + CommercialSubscriptionRolloverService, +) + + +def test_concurrent_usage_replay_creates_one_event( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, subscription_id, entitlement_id = _seed_account(pg_factory) + payload = UsageMeterEventCreate( + subscription_id=subscription_id, + entitlement_id=entitlement_id, + quantity=Decimal("2"), + occurred_at=datetime.now(UTC), + source_system="postgres-runtime", + idempotency_key=f"usage-{uuid.uuid4().hex}", + ) + ready = threading.Barrier(2) + + def record_once() -> bool: + with pg_factory() as db: + ready.wait(timeout=5) + _, created = CommercialMeteringService(db).record_usage( + tenant_id, + payload, + actor_type="system", + actor_id="postgres-runtime", + ) + db.commit() + return created + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(record_once), pool.submit(record_once)) + ] + assert sorted(outcomes) == [False, True] + with pg_factory() as db: + assert ( + db.scalar( + select(func.count()) + .select_from(UsageMeterEvent) + .where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.idempotency_key == payload.idempotency_key, + ) + ) + == 1 + ) + + +def test_concurrent_usage_cannot_bypass_hard_limit( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, subscription_id, entitlement_id = _seed_account(pg_factory) + ready = threading.Barrier(2) + + def consume_once(suffix: str) -> str: + payload = UsageMeterEventCreate( + subscription_id=subscription_id, + entitlement_id=entitlement_id, + quantity=Decimal("4"), + occurred_at=datetime.now(UTC), + source_system="postgres-runtime", + idempotency_key=f"quota-{suffix}-{uuid.uuid4().hex}", + ) + with pg_factory() as db: + ready.wait(timeout=5) + try: + CommercialMeteringService(db).record_usage( + tenant_id, + payload, + actor_type="system", + actor_id="postgres-runtime", + ) + db.commit() + return "created" + except CommercialConflictError: + db.rollback() + return "blocked" + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(consume_once, "a"), pool.submit(consume_once, "b")) + ] + assert sorted(outcomes) == ["blocked", "created"] + with pg_factory() as db: + used = db.scalar( + select(func.sum(UsageMeterEvent.quantity)).where( + UsageMeterEvent.tenant_id == tenant_id, + UsageMeterEvent.entitlement_id == entitlement_id, + ) + ) + assert Decimal(used or 0) == Decimal("4.000000") + + +def test_concurrent_cost_reversal_has_one_winner( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id, subscription_id, _ = _seed_account(pg_factory) + original_payload = _cost_payload( + subscription_id, + idempotency_key=f"cost-{uuid.uuid4().hex}", + ) + with pg_factory() as db: + original, _ = CommercialMeteringService(db).record_cost(tenant_id, original_payload) + db.commit() + original_id = original.id + ready = threading.Barrier(2) + + def reverse_once(suffix: str) -> str: + payload = original_payload.model_copy( + update={ + "event_type": "reversal", + "idempotency_key": f"reversal-{suffix}-{uuid.uuid4().hex}", + "reversal_of_cost_event_id": original_id, + "occurred_at": datetime.now(UTC), + } + ) + with pg_factory() as db: + ready.wait(timeout=5) + try: + CommercialMeteringService(db).record_cost(tenant_id, payload) + db.commit() + return "created" + except CommercialConflictError: + db.rollback() + return "blocked" + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(reverse_once, "a"), pool.submit(reverse_once, "b")) + ] + assert sorted(outcomes) == ["blocked", "created"] + with pg_factory() as db: + assert ( + db.scalar( + select(func.count()) + .select_from(CommercialCostEvent) + .where( + CommercialCostEvent.tenant_id == tenant_id, + CommercialCostEvent.reversal_of_cost_event_id == original_id, + ) + ) + == 1 + ) + + +def test_concurrent_runtime_reservations_cannot_oversell_hard_quota( + pg_factory: sessionmaker[Session], +) -> None: + runtime_meter = { + "enabled": True, + "tool_type": "llm", + "tool_name": "chat.completions", + "quantity_basis": "call", + } + tenant_id, subscription_id, entitlement_id = _seed_account( + pg_factory, + hard_limit=Decimal("1"), + runtime_meter=runtime_meter, + ) + ready = threading.Barrier(2) + + def reserve_once(suffix: str) -> str: + with pg_factory() as db: + ready.wait(timeout=5) + try: + CommercialRuntimeReservationService(db).reserve( + tenant_id=tenant_id, + entitlement_id=entitlement_id, + run_id=f"run-{suffix}-{uuid.uuid4().hex}", + tool_call_id=str(uuid.uuid4()), + tool_type="llm", + tool_name="chat.completions", + quantity_basis="call", + reserved_quantity=Decimal("1"), + meter_config=runtime_meter, + ) + db.commit() + return "reserved" + except CommercialConflictError: + db.rollback() + return "blocked" + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(reserve_once, "a"), pool.submit(reserve_once, "b")) + ] + assert sorted(outcomes) == ["blocked", "reserved"] + with pg_factory() as db: + held = db.scalar( + select(func.sum(CommercialRuntimeReservation.reserved_quantity)).where( + CommercialRuntimeReservation.tenant_id == tenant_id, + CommercialRuntimeReservation.subscription_id == subscription_id, + CommercialRuntimeReservation.entitlement_id == entitlement_id, + CommercialRuntimeReservation.status == "reserved", + ) + ) + assert Decimal(held or 0) == Decimal("1.000000") + assert ( + db.scalar( + select(func.count()) + .select_from(UsageMeterEvent) + .where(UsageMeterEvent.tenant_id == tenant_id) + ) + == 0 + ) + + +def test_concurrent_subscription_rollover_creates_one_period( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id = f"tenant-rollover-{uuid.uuid4().hex}" + period_start = datetime(2026, 1, 1, tzinfo=UTC) + period_end = datetime(2026, 2, 1, tzinfo=UTC) + with pg_factory() as db: + admin = CommercialAdminService(db) + plan = admin.create_plan( + tenant_id, + CommercialPlanCreate( + plan_code="rollover", + name="并发续期版", + pricing_model="subscription", + billing_interval="monthly", + currency="CNY", + base_fee=Decimal("100"), + effective_from=datetime(2025, 12, 1, tzinfo=UTC), + ), + actor_id="postgres-test", + ) + admin.activate_plan(tenant_id, plan.id, expected_version=plan.version) + subscription = admin.create_subscription( + tenant_id, + CommercialSubscriptionCreate( + subscription_key=f"rollover-{uuid.uuid4().hex}", + plan_id=plan.id, + starts_at=period_start, + current_period_start=period_start, + current_period_end=period_end, + seats=1, + auto_renew=True, + ), + actor_id="postgres-test", + ) + db.commit() + subscription_id = subscription.id + ready = threading.Barrier(2) + + def rollover_once() -> str: + with pg_factory() as db: + ready.wait(timeout=5) + result = CommercialSubscriptionRolloverService(db).rollover_due( + tenant_id, + subscription_id, + as_of=period_end, + ) + db.commit() + return result.status + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(rollover_once), pool.submit(rollover_once)) + ] + assert sorted(outcomes) == ["not_due", "rolled_over"] + with pg_factory() as db: + count = db.scalar( + select(func.count()) + .select_from(CommercialBillingPeriod) + .where( + CommercialBillingPeriod.tenant_id == tenant_id, + CommercialBillingPeriod.subscription_id == subscription_id, + ) + ) + assert count == 2 + + +def _seed_account( + factory: sessionmaker[Session], + *, + hard_limit: Decimal = Decimal("6"), + runtime_meter: dict[str, object] | None = None, +) -> tuple[str, str, str]: + tenant_id = f"tenant-commercial-{uuid.uuid4().hex}" + now = datetime.now(UTC) + with factory() as db: + service = CommercialAdminService(db) + plan = service.create_plan( + tenant_id, + CommercialPlanCreate( + plan_code="concurrency", + name="并发验证版", + pricing_model="usage", + billing_interval="monthly", + currency="CNY", + base_fee=Decimal("0"), + overage_enabled=False, + effective_from=now - timedelta(days=30), + ), + actor_id="postgres-test", + ) + service.activate_plan(tenant_id, plan.id, expected_version=plan.version) + subscription = service.create_subscription( + tenant_id, + CommercialSubscriptionCreate( + subscription_key=f"subscription-{uuid.uuid4().hex}", + plan_id=plan.id, + starts_at=now - timedelta(days=5), + current_period_start=now - timedelta(days=1), + current_period_end=now + timedelta(days=29), + seats=1, + ), + actor_id="postgres-test", + ) + entitlement = service.upsert_entitlement( + tenant_id, + CommercialEntitlementUpsert( + subscription_id=subscription.id, + entitlement_key="concurrency_usage", + metric_key="concurrency_usage", + entitlement_type="metered", + unit="run", + included_quantity=hard_limit, + hard_limit_quantity=hard_limit, + reset_interval="monthly", + overage_policy="block", + effective_from=now - timedelta(days=5), + config_json={"runtime_meter": runtime_meter} if runtime_meter else {}, + ), + ) + db.commit() + return tenant_id, subscription.id, entitlement.id + + +def _cost_payload( + subscription_id: str, + *, + idempotency_key: str, +) -> CommercialCostEventCreate: + return CommercialCostEventCreate( + subscription_id=subscription_id, + cost_category="ai_inference", + quantity=Decimal("10"), + unit="1k_tokens", + unit_cost=Decimal("1"), + original_currency="CNY", + reporting_currency="CNY", + fx_rate=Decimal("1"), + allocation_key="postgres-concurrency", + occurred_at=datetime.now(UTC), + source_system="postgres-provider", + idempotency_key=idempotency_key, + ) diff --git a/server/tests/test_commercial_direct_operation.py b/server/tests/test_commercial_direct_operation.py new file mode 100644 index 0000000..63af087 --- /dev/null +++ b/server/tests/test_commercial_direct_operation.py @@ -0,0 +1,397 @@ +from __future__ import annotations + +import json +from datetime import UTC, datetime, timedelta +from decimal import Decimal + +import pytest +from commercial_runtime_testkit import seed_meter +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.db.base_class import Base +from app.models.commercial import CommercialCostEvent, UsageMeterEvent +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.commercial_direct_operation import ( + CommercialDirectOperationBridge, + DirectOperationIdentity, +) + + +@pytest.fixture() +def factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + result = sessionmaker(bind=engine, expire_on_commit=False) + try: + yield result + finally: + Base.metadata.drop_all(engine) + engine.dispose() + + +def _identity(now: datetime, suffix: str = "a") -> DirectOperationIdentity: + return DirectOperationIdentity( + tenant_id="tenant-a", + operation_key=f"SENSITIVE-OPERATION-{suffix}", + run_key=f"SENSITIVE-RUN-{suffix}", + tool_type="llm", + tool_name="chat.completions", + provider="OpenAI", + model_name="gpt-test", + started_at=now, + ) + + +def test_unconfigured_direct_operation_is_compatible_and_writes_no_fact( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + bridge = CommercialDirectOperationBridge(factory) + + permit = bridge.permit(_identity(now)) + + assert permit.enforced is False + assert permit.allowed is True + assert permit.reason_code == "runtime_meter_not_configured" + with factory() as db: + assert db.query(CommercialRuntimeReservation).count() == 0 + assert db.query(UsageMeterEvent).count() == 0 + + +def test_authoritative_tokens_settle_once_with_redacted_identity_and_cost( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis="total_tokens", + preflight_quantity=Decimal("20"), + internal_cost={ + "enabled": True, + "cost_category": "ai_inference", + "unit": "token", + "unit_cost": "0.002", + "original_currency": "CNY", + "reporting_currency": "CNY", + "fx_rate": "1", + "provider": "OpenAI", + "model_name": "gpt-test", + }, + ) + db.commit() + + identity = _identity(now, "tokens") + bridge = CommercialDirectOperationBridge(factory) + permit = bridge.permit(identity) + result = bridge.complete( + identity, + outcome="succeeded", + authoritative_quantities={ + "input_tokens": 3, + "output_tokens": 2, + "total_tokens": 5, + }, + completed_at=now + timedelta(seconds=1), + usage_source="openai_usage", + usage_availability="available", + ) + replay = bridge.complete( + identity, + outcome="succeeded", + authoritative_quantities={ + "input_tokens": 3, + "output_tokens": 2, + "total_tokens": 5, + }, + completed_at=now + timedelta(seconds=1), + usage_source="openai_usage", + usage_availability="available", + ) + + assert permit.allowed is True and permit.reservation_id + assert permit.reserved_quantity == Decimal("20") + assert result.status == "created" + assert result.quantity == Decimal("5") + assert replay.status == "replayed" + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + usage = db.scalars(select(UsageMeterEvent)).one() + cost = db.scalars(select(CommercialCostEvent)).one() + assert reservation.status == "committed" + assert Decimal(reservation.actual_quantity or 0) == Decimal("5") + assert Decimal(usage.quantity) == Decimal("5") + assert Decimal(cost.cost_amount) == Decimal("0.0100") + assert usage.billing_period_id == reservation.billing_period_id + assert cost.billing_period_id == reservation.billing_period_id + serialized = json.dumps(usage.metadata_json, ensure_ascii=False) + assert "SENSITIVE-OPERATION" not in serialized + assert "SENSITIVE-RUN" not in serialized + assert db.query(UsageMeterEvent).count() == 1 + assert db.query(CommercialCostEvent).count() == 1 + + +@pytest.mark.parametrize( + ("basis", "tool_type", "tool_name"), + [ + ("bytes", "storage", "attachment.upload"), + ("pages", "ocr", "receipt.extract"), + ("objects", "storage", "attachment.persist"), + ("events", "connector", "financial.ingest"), + ], +) +def test_authoritative_resource_quantities_settle_without_estimation( + factory: sessionmaker[Session], + basis: str, + tool_type: str, + tool_name: str, +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis=basis, + tool_type=tool_type, + tool_name=tool_name, + preflight_quantity=Decimal("100"), + ) + db.commit() + identity = DirectOperationIdentity( + tenant_id="tenant-a", + operation_key=f"resource-operation:{basis}", + run_key=f"resource-run:{basis}", + tool_type=tool_type, + tool_name=tool_name, + started_at=now, + ) + bridge = CommercialDirectOperationBridge(factory) + + permit = bridge.permit(identity, requested_quantity=7) + assert permit.allowed is True + assert permit.reserved_quantity == Decimal("7") + result = bridge.complete( + identity, + outcome="succeeded", + authoritative_quantities={basis: 7}, + completed_at=now + timedelta(seconds=1), + usage_source=f"authoritative_{basis}", + usage_availability="available", + ) + + assert result.status == "created" + assert result.quantity_basis == basis + assert result.quantity == Decimal("7") + with factory() as db: + usage = db.scalars(select(UsageMeterEvent)).one() + assert Decimal(usage.quantity) == Decimal("7") + assert usage.metadata_json["usage_source"] == f"authoritative_{basis}" + + +@pytest.mark.parametrize( + ("outcome", "quantities", "reason_code", "actual"), + [ + ("outcome_unknown", {}, "authoritative_usage_unavailable", None), + ( + "provider_rejected", + {"total_tokens": 25}, + "actual_exceeds_preflight_reservation", + Decimal("25"), + ), + ], +) +def test_uncertain_or_over_reservation_call_enters_durable_reconciliation( + factory: sessionmaker[Session], + outcome: str, + quantities: dict[str, int], + reason_code: str, + actual: Decimal | None, +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis="total_tokens", + preflight_quantity=Decimal("20"), + ) + db.commit() + identity = _identity(now, outcome) + bridge = CommercialDirectOperationBridge(factory) + assert bridge.permit(identity).allowed is True + + result = bridge.complete( + identity, + outcome=outcome, # type: ignore[arg-type] + authoritative_quantities=quantities, + completed_at=now + timedelta(seconds=1), + usage_source="unavailable" if not quantities else "openai_usage", + usage_availability="unavailable" if not quantities else "available", + ) + + assert result.status == "reconciliation_required" + assert result.reason_code == reason_code + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + assert reservation.status == "reconciliation_required" + assert ( + Decimal(reservation.actual_quantity) + if reservation.actual_quantity is not None + else None + ) == actual + assert db.query(UsageMeterEvent).count() == 0 + assert db.query(CommercialCostEvent).count() == 0 + + +def test_not_sent_releases_reservation_and_never_meters( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter(db, "tenant-a", now, basis="call") + db.commit() + identity = _identity(now, "not-sent") + bridge = CommercialDirectOperationBridge(factory) + assert bridge.permit(identity).allowed is True + + result = bridge.complete( + identity, + outcome="not_sent", + authoritative_quantities={}, + completed_at=now + timedelta(milliseconds=10), + usage_source="unavailable", + usage_availability="unavailable", + ) + + assert result.status == "released" + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + assert reservation.status == "released" + assert db.query(UsageMeterEvent).count() == 0 + + +def test_released_operation_can_be_preflighted_again_after_business_rollback( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter(db, "tenant-a", now, basis="call") + db.commit() + identity = _identity(now, "released-retry") + bridge = CommercialDirectOperationBridge(factory) + assert bridge.permit(identity).allowed is True + bridge.complete( + identity, + outcome="not_sent", + authoritative_quantities={}, + completed_at=now + timedelta(milliseconds=10), + usage_source="business_transaction_rolled_back", + usage_availability="unavailable", + ) + + retry = bridge.permit(identity) + result = bridge.complete( + identity, + outcome="succeeded", + authoritative_quantities={"call": 1}, + completed_at=now + timedelta(milliseconds=20), + usage_source="accepted_retry", + usage_availability="available", + ) + + assert retry.allowed is True + assert result.status == "created" + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + assert reservation.status == "committed" + assert db.query(UsageMeterEvent).count() == 1 + + +def test_variable_meter_without_explicit_hard_preflight_is_denied( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter(db, "tenant-a", now, basis="total_tokens") + db.commit() + + permit = CommercialDirectOperationBridge(factory).permit(_identity(now, "no-max")) + + assert permit.enforced is True + assert permit.allowed is False + assert "preflight_quantity" in permit.reason + with factory() as db: + assert db.query(CommercialRuntimeReservation).count() == 0 + + +def test_known_resource_quantity_above_configured_maximum_is_denied_before_call( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis="pages", + tool_type="ocr", + tool_name="receipt.extract", + preflight_quantity=Decimal("10"), + ) + db.commit() + identity = DirectOperationIdentity( + tenant_id="tenant-a", + operation_key="resource-operation:too-many-pages", + run_key="resource-run:too-many-pages", + tool_type="ocr", + tool_name="receipt.extract", + started_at=now, + ) + + permit = CommercialDirectOperationBridge(factory).permit( + identity, + requested_quantity=11, + ) + + assert permit.allowed is False + assert "硬上限" in permit.reason + with factory() as db: + assert db.query(CommercialRuntimeReservation).count() == 0 + + +def test_call_without_observed_permit_creates_reconciliation_backlog( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter(db, "tenant-a", now, basis="call") + db.commit() + identity = _identity(now, "missing-permit") + + result = CommercialDirectOperationBridge(factory).complete( + identity, + outcome="provider_rejected", + authoritative_quantities={}, + completed_at=now + timedelta(milliseconds=50), + usage_source="unavailable", + usage_availability="unavailable", + ) + + assert result.status == "reconciliation_required" + assert result.reason_code == "missing_pre_execution_reservation" + assert result.quantity == Decimal("1") + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + assert reservation.status == "reconciliation_required" + assert reservation.resolution_code == "missing_pre_execution_reservation" diff --git a/server/tests/test_commercial_endpoints.py b/server/tests/test_commercial_endpoints.py new file mode 100644 index 0000000..6bd3b19 --- /dev/null +++ b/server/tests/test_commercial_endpoints.py @@ -0,0 +1,347 @@ +from __future__ import annotations + +from collections.abc import Generator +from datetime import UTC, datetime, timedelta + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.api.v1.endpoints.commercial import router +from app.api.v1.endpoints.commercial_billing import router as billing_router +from app.db.base_class import Base + + +@pytest.fixture() +def http_context() -> Generator[ + tuple[TestClient, dict[str, CurrentUserContext]], + None, + None, +]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + app = FastAPI() + app.include_router(router, prefix="/api/v1") + app.include_router(billing_router, prefix="/api/v1") + user_box = {"current": _user("platform-admin", "platform", is_admin=True)} + + def override_db() -> Generator[Session, None, None]: + with factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: user_box["current"] + client = TestClient(app) + client.headers.update({"X-Request-Id": "commercial-endpoint-test"}) + try: + yield client, user_box + finally: + client.close() + app.dependency_overrides.clear() + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_commercial_http_admin_config_metering_and_tenant_read_scope( + http_context: tuple[TestClient, dict[str, CurrentUserContext]], +) -> None: + client, user_box = http_context + now = datetime.now(UTC) + plan_response = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/plans", + json={ + "plan_code": "enterprise", + "name": "企业版", + "pricing_model": "subscription", + "billing_interval": "monthly", + "currency": "CNY", + "base_fee": "500.0000", + "included_seats": 50, + "effective_from": (now - timedelta(days=30)).isoformat(), + }, + ) + assert plan_response.status_code == 201 + plan = plan_response.json() + assert plan["status"] == "draft" + activate_response = client.post( + f"/api/v1/commercial/admin/tenants/tenant-a/plans/{plan['id']}/activate", + json={"expected_version": plan["version"], "reason": "启用企业套餐"}, + ) + assert activate_response.status_code == 200 + assert activate_response.json()["plan"]["status"] == "active" + + subscription_response = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/subscriptions", + json={ + "subscription_key": "tenant-a-2026", + "plan_id": plan["id"], + "status": "active", + "starts_at": (now - timedelta(days=5)).isoformat(), + "current_period_start": (now - timedelta(days=1)).isoformat(), + "current_period_end": (now + timedelta(days=29)).isoformat(), + "seats": 20, + }, + ) + assert subscription_response.status_code == 201 + subscription = subscription_response.json() + entitlement_response = client.put( + "/api/v1/commercial/admin/tenants/tenant-a/entitlements", + json={ + "subscription_id": subscription["id"], + "entitlement_key": "ocr", + "metric_key": "ocr_pages", + "entitlement_type": "metered", + "unit": "page", + "included_quantity": "100", + "hard_limit_quantity": "120", + "reset_interval": "monthly", + "overage_policy": "block", + "status": "active", + "effective_from": (now - timedelta(days=5)).isoformat(), + }, + ) + assert entitlement_response.status_code == 200 + entitlement = entitlement_response.json() + + usage_payload = { + "subscription_id": subscription["id"], + "entitlement_id": entitlement["id"], + "event_type": "usage", + "quantity": "10", + "occurred_at": now.isoformat(), + "source_system": "ocr-runtime", + "idempotency_key": "ocr-usage-001", + } + first_usage = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/usage-events", + json=usage_payload, + ) + replay_usage = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/usage-events", + json=usage_payload, + ) + assert first_usage.status_code == 200 + assert first_usage.json()["created"] is True + assert replay_usage.status_code == 200 + assert replay_usage.json()["created"] is False + conflict = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/usage-events", + json={**usage_payload, "quantity": "11"}, + ) + assert conflict.status_code == 409 + + assert ( + client.get("/api/v1/commercial/admin/tenants/tenant-a/plans").json()[0]["id"] == plan["id"] + ) + assert ( + client.get("/api/v1/commercial/admin/tenants/tenant-a/subscriptions").json()[0]["id"] + == subscription["id"] + ) + assert ( + client.get("/api/v1/commercial/admin/tenants/tenant-a/entitlements").json()[0]["id"] + == entitlement["id"] + ) + assert ( + client.get("/api/v1/commercial/admin/tenants/tenant-a/usage-events").json()[0]["id"] + == first_usage.json()["usage_event"]["id"] + ) + suspended = client.post( + f"/api/v1/commercial/admin/tenants/tenant-a/subscriptions/{subscription['id']}/transition", + json={ + "expected_version": subscription["version"], + "target_status": "suspended", + "reason": "客户主动暂停商业服务", + }, + ) + assert suspended.status_code == 200 + assert suspended.json()["status"] == "suspended" + resumed = client.post( + f"/api/v1/commercial/admin/tenants/tenant-a/subscriptions/{subscription['id']}/activate", + json={ + "expected_version": suspended.json()["version"], + "reason": "客户确认恢复商业服务", + }, + ) + assert resumed.status_code == 200 + assert resumed.json()["status"] == "active" + periods = client.get("/api/v1/commercial/admin/tenants/tenant-a/billing-periods") + assert periods.status_code == 200 + assert periods.json()[0]["subscription_id"] == subscription["id"] + assert periods.json()[0]["temporal_state"] == "current" + audit_events = client.get("/api/v1/commercial/admin/tenants/tenant-a/admin-events") + assert audit_events.status_code == 200 + assert {item["action"] for item in audit_events.json()} >= { + "plan_created", + "subscription_created", + "billing_period_created", + } + + user_box["current"] = _user("finance-a", "tenant-a", roles=["finance"]) + account = client.get("/api/v1/commercial/account") + assert account.status_code == 200 + assert account.json()["tenant_id"] == "tenant-a" + assert account.json()["plan"]["id"] == plan["id"] + assert account.json()["quotas"][0]["used_quantity"] == "10.000000" + assert client.get("/api/v1/commercial/billing-periods").status_code == 200 + assert ( + client.get( + "/api/v1/commercial/admin/tenants/tenant-a/admin-events" + ).status_code + == 403 + ) + assert ( + client.post( + "/api/v1/commercial/admin/tenants/tenant-a/cost-events", + json=_cost_json(subscription["id"], now), + ).status_code + == 403 + ) + + user_box["current"] = _user("finance-b", "tenant-b", roles=["finance"]) + other_account = client.get("/api/v1/commercial/account") + assert other_account.status_code == 200 + assert other_account.json()["tenant_id"] == "tenant-b" + assert other_account.json()["subscription"] is None + assert other_account.json()["quotas"] == [] + assert client.get("/api/v1/commercial/billing-periods").json() == [] + + user_box["current"] = _user("employee-a", "tenant-a") + assert client.get("/api/v1/commercial/account").status_code == 403 + assert client.get("/api/v1/commercial/billing-periods").status_code == 403 + user_box["current"] = _user("manager-a", "tenant-a", roles=["manager"]) + assert client.get("/api/v1/commercial/account").status_code == 403 + assert ( + client.post( + "/api/v1/commercial/admin/tenants/tenant-a/plans", + json={}, + ).status_code + == 403 + ) + + +def test_commercial_http_cost_is_admin_only_and_analytics_never_fakes_missing_value( + http_context: tuple[TestClient, dict[str, CurrentUserContext]], +) -> None: + client, user_box = http_context + now = datetime.now(UTC) + plan = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/plans", + json={ + "plan_code": "pilot", + "name": "试点版", + "pricing_model": "pilot", + "billing_interval": "contract", + "currency": "CNY", + "base_fee": "1000", + "effective_from": (now - timedelta(days=10)).isoformat(), + }, + ).json() + client.post( + f"/api/v1/commercial/admin/tenants/tenant-a/plans/{plan['id']}/activate", + json={"expected_version": plan["version"], "reason": "启用试点套餐"}, + ) + subscription = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/subscriptions", + json={ + "subscription_key": "pilot-a", + "plan_id": plan["id"], + "starts_at": (now - timedelta(days=3)).isoformat(), + "current_period_start": (now - timedelta(days=3)).isoformat(), + "current_period_end": (now + timedelta(days=87)).isoformat(), + "seats": 5, + }, + ).json() + + cost = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/cost-events", + json=_cost_json(subscription["id"], now), + ) + replay = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/cost-events", + json=_cost_json(subscription["id"], now), + ) + assert cost.status_code == 200 + assert cost.json()["created"] is True + assert cost.json()["cost_event"]["cost_amount"] == "25.0000" + assert replay.status_code == 200 + assert replay.json()["created"] is False + + analytics = client.get( + "/api/v1/commercial/admin/tenants/tenant-a/analytics", + params={ + "start": (now - timedelta(days=5)).isoformat(), + "end": (now + timedelta(days=1)).isoformat(), + "as_of": (now + timedelta(hours=1)).isoformat(), + }, + ) + assert analytics.status_code == 200 + body = analytics.json() + assert body["customer_charges"]["status"] == "partial" + assert body["internal_costs"]["status"] == "available" + assert body["contribution_margin"]["status"] == "partial" + assert body["verified_cash_savings"]["status"] == "unavailable" + assert body["verified_cash_savings"]["values"] == [] + assert body["customer_roi"]["status"] == "unavailable" + assert body["customer_labor_value"]["status"] == "unavailable" + + pricing = client.post( + "/api/v1/commercial/admin/tenants/tenant-a/pricing-scenarios", + json={ + "start": (now - timedelta(days=5)).isoformat(), + "end": (now + timedelta(days=1)).isoformat(), + "as_of": (now + timedelta(hours=1)).isoformat(), + "target_contribution_margin_rate": "0.65", + "max_verified_savings_share": "0.25", + }, + ) + assert pricing.status_code == 200 + assert pricing.json()["recommended_model"] == "subscription" + assert pricing.json()["scenarios"][0]["status"] == "cost_only" + + user_box["current"] = _user("finance-a", "tenant-a", roles=["finance"]) + assert client.get("/api/v1/commercial/admin/tenants/tenant-a/analytics").status_code == 403 + + +def _cost_json(subscription_id: str, now: datetime) -> dict[str, object]: + return { + "subscription_id": subscription_id, + "event_type": "incurred", + "cost_category": "ocr", + "quantity": "100", + "unit": "page", + "unit_cost": "0.25", + "original_currency": "CNY", + "reporting_currency": "CNY", + "fx_rate": "1", + "allocation_key": "tenant-a:ocr", + "occurred_at": now.isoformat(), + "source_system": "ocr-provider", + "idempotency_key": "ocr-cost-001", + } + + +def _user( + username: str, + tenant_id: str, + *, + roles: list[str] | None = None, + is_admin: bool = False, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=is_admin, + tenant_id=tenant_id, + employee_id=username, + ) diff --git a/server/tests/test_commercial_models.py b/server/tests/test_commercial_models.py new file mode 100644 index 0000000..ce4fadd --- /dev/null +++ b/server/tests/test_commercial_models.py @@ -0,0 +1,166 @@ +from __future__ import annotations + +from sqlalchemy import ForeignKeyConstraint + +from app.db.base import Base +from app.models.commercial import ( + CommercialCostEvent, + CommercialEntitlement, + TenantCommercialPlan, + TenantSubscription, + UsageMeterEvent, +) + +COMMERCIAL_TABLES = { + "tenant_commercial_plans", + "tenant_subscriptions", + "commercial_entitlements", + "usage_meter_events", + "commercial_cost_events", +} + + +def _constraint_names(model: type[object]) -> set[str]: + return { + str(constraint.name) + for constraint in model.__table__.constraints # type: ignore[attr-defined] + if constraint.name is not None + } + + +def _foreign_key_targets(model: type[object]) -> set[str]: + return { + element.target_fullname.split(".", maxsplit=1)[0] + for constraint in model.__table__.constraints # type: ignore[attr-defined] + if isinstance(constraint, ForeignKeyConstraint) + for element in constraint.elements + } + + +def test_commercial_models_are_registered_as_separate_tenant_domain() -> None: + models = ( + TenantCommercialPlan, + TenantSubscription, + CommercialEntitlement, + UsageMeterEvent, + CommercialCostEvent, + ) + assert {model.__table__.name for model in models} == COMMERCIAL_TABLES + assert COMMERCIAL_TABLES.issubset(Base.metadata.tables) + assert all("tenant_id" in model.__table__.c for model in models) + + +def test_plan_subscription_and_entitlement_declare_contract_invariants() -> None: + assert { + "uq_tenant_commercial_plans_tenant_id", + "uq_tenant_commercial_plans_tenant_code_version", + "ck_tenant_commercial_plans_pricing_model", + "ck_tenant_commercial_plans_billing_interval", + "ck_tenant_commercial_plans_status", + "ck_tenant_commercial_plans_values", + "ck_tenant_commercial_plans_effective_window", + }.issubset(_constraint_names(TenantCommercialPlan)) + assert { + "uq_tenant_subscriptions_tenant_id", + "uq_tenant_subscriptions_tenant_key", + "uq_tenant_subscriptions_external_ref", + "fk_tenant_subscriptions_tenant_plan", + "ck_tenant_subscriptions_status", + "ck_tenant_subscriptions_period", + "ck_tenant_subscriptions_external_pair", + "ck_tenant_subscriptions_cancellation", + }.issubset(_constraint_names(TenantSubscription)) + assert { + "uq_commercial_entitlements_tenant_id", + "uq_commercial_entitlements_tenant_subscription_id", + "uq_commercial_entitlements_subscription_key", + "fk_commercial_entitlements_tenant_subscription", + "ck_commercial_entitlements_type", + "ck_commercial_entitlements_quota_shape", + "ck_commercial_entitlements_overage_policy", + "ck_commercial_entitlements_effective_window", + }.issubset(_constraint_names(CommercialEntitlement)) + + active_plan_index = next( + index + for index in TenantCommercialPlan.__table__.indexes + if index.name == "uq_tenant_commercial_plans_active_code" + ) + assert active_plan_index.unique is True + assert "status = 'active'" in str( + active_plan_index.dialect_options["postgresql"]["where"] + ) + current_subscription_index = next( + index + for index in TenantSubscription.__table__.indexes + if index.name == "uq_tenant_subscriptions_current" + ) + assert current_subscription_index.unique is True + assert "trialing" in str( + current_subscription_index.dialect_options["postgresql"]["where"] + ) + + +def test_usage_and_cost_events_declare_idempotency_and_tenant_safe_links() -> None: + assert { + "uq_usage_meter_events_tenant_id", + "uq_usage_meter_events_tenant_subscription_id", + "uq_usage_meter_events_entitlement_id", + "uq_usage_meter_events_source_request", + "fk_usage_meter_events_tenant_subscription", + "fk_usage_meter_events_tenant_entitlement", + "fk_usage_meter_events_tenant_reversal", + "ck_usage_meter_events_quantity", + "ck_usage_meter_events_reversal", + "ck_usage_meter_events_keys", + }.issubset(_constraint_names(UsageMeterEvent)) + assert { + "uq_commercial_cost_events_tenant_id", + "uq_commercial_cost_events_source_request", + "fk_commercial_cost_events_tenant_subscription", + "fk_commercial_cost_events_tenant_usage", + "fk_commercial_cost_events_tenant_reversal", + "ck_commercial_cost_events_category", + "ck_commercial_cost_events_amount_direction", + "ck_commercial_cost_events_usage_pair", + "ck_commercial_cost_events_keys", + }.issubset(_constraint_names(CommercialCostEvent)) + + usage_constraints = { + constraint.name: constraint + for constraint in UsageMeterEvent.__table__.constraints + if isinstance(constraint, ForeignKeyConstraint) + } + assert tuple( + usage_constraints["fk_usage_meter_events_tenant_entitlement"].column_keys + ) == ("tenant_id", "subscription_id", "entitlement_id") + assert tuple(usage_constraints["fk_usage_meter_events_tenant_reversal"].column_keys) == ( + "tenant_id", + "subscription_id", + "entitlement_id", + "reversal_of_event_id", + ) + cost_constraints = { + constraint.name: constraint + for constraint in CommercialCostEvent.__table__.constraints + if isinstance(constraint, ForeignKeyConstraint) + } + assert tuple(cost_constraints["fk_commercial_cost_events_tenant_usage"].column_keys) == ( + "tenant_id", + "subscription_id", + "usage_event_id", + ) + + +def test_internal_cost_facts_are_physically_separate_from_customer_value_facts() -> None: + savings_tables = { + "profile_baseline_snapshots", + "savings_opportunities", + "savings_realizations", + "savings_evidence_links", + "savings_events", + } + assert _foreign_key_targets(CommercialCostEvent).isdisjoint(savings_tables) + assert _foreign_key_targets(UsageMeterEvent).isdisjoint(savings_tables) + assert "updated_at" not in UsageMeterEvent.__table__.c + assert "updated_at" not in CommercialCostEvent.__table__.c diff --git a/server/tests/test_commercial_resource_boundaries.py b/server/tests/test_commercial_resource_boundaries.py new file mode 100644 index 0000000..ca13918 --- /dev/null +++ b/server/tests/test_commercial_resource_boundaries.py @@ -0,0 +1,462 @@ +from __future__ import annotations + +import json +import time +import uuid +from datetime import UTC, datetime +from decimal import Decimal +from pathlib import Path + +import pytest +from commercial_runtime_testkit import seed_meter +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext +from app.db.base_class import Base +from app.models.commercial import UsageMeterEvent +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.models.financial_connector import FinancialConnectorConfig, FinancialConnectorEvent +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.schemas.financial_connector import FinancialEventEnvelope +from app.services.expense_claim_attachment_commercial import ( + ExpenseClaimAttachmentCommercialAccessDenied, + stage_attachment_deletion, + stage_attachment_replacement, + stage_claim_attachment_deletion, +) +from app.services.expense_claim_attachment_storage import ExpenseClaimAttachmentStorage +from app.services.financial_connector_auth import ( + FinancialConnectorAuthError, + FinancialConnectorSecretResolver, + sign_financial_event, +) +from app.services.financial_connector_commercial import ( + FinancialConnectorCommercialAccessDenied, +) +from app.services.financial_connector_ingestion import ( + FinancialConnectorConflictError, + FinancialConnectorIngestionService, +) + + +@pytest.fixture() +def factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + result = sessionmaker(bind=engine, expire_on_commit=False) + try: + yield result + finally: + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_connector_only_meters_new_authenticated_committed_event( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + timestamp = int(now.timestamp()) + with factory() as db: + claim = _seed_connector_boundary(db, now=now, hard_limit=Decimal("10")) + db.commit() + envelope = _envelope(claim, "SENSITIVE-EXTERNAL-EVENT-001", now) + + first = _ingest(db, envelope, timestamp) + assert first.replayed is False + assert db.scalar(select(func.count(UsageMeterEvent.id))) == 0 + db.commit() + + replay = _ingest(db, envelope, timestamp) + db.commit() + assert replay.replayed is True + + conflicting = envelope.model_copy( + update={"payload": {**envelope.payload, "amount": "66.01"}} + ) + with pytest.raises(FinancialConnectorConflictError): + _ingest(db, conflicting, timestamp) + db.rollback() + + with pytest.raises(FinancialConnectorAuthError): + _ingest(db, envelope, timestamp, signature="sha256=" + "0" * 64) + db.rollback() + + usage = db.scalars(select(UsageMeterEvent)).one() + reservations = list(db.scalars(select(CommercialRuntimeReservation)).all()) + assert Decimal(usage.quantity) == Decimal("1") + assert len(reservations) == 1 and reservations[0].status == "committed" + serialized = json.dumps(usage.metadata_json, ensure_ascii=False) + assert envelope.external_event_id not in serialized + assert envelope.correlation_id not in serialized + assert claim.claim_no not in serialized + + +def test_connector_rollback_releases_reservation_and_writes_no_usage( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + claim = _seed_connector_boundary(db, now=now, hard_limit=Decimal("10")) + db.commit() + envelope = _envelope(claim, "rollback-event", now) + + _ingest(db, envelope, int(now.timestamp())) + db.rollback() + + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + assert reservation.status == "released" + assert db.scalar(select(func.count(UsageMeterEvent.id))) == 0 + assert db.scalar(select(func.count(FinancialConnectorEvent.id))) == 0 + + retry = _ingest(db, envelope, int(now.timestamp())) + db.commit() + assert retry.replayed is False + assert db.scalar(select(func.count(FinancialConnectorEvent.id))) == 1 + assert db.scalar(select(func.count(UsageMeterEvent.id))) == 1 + db.refresh(reservation) + assert reservation.status == "committed" + + +def test_connector_quota_denial_happens_before_second_event_persistence( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + claim = _seed_connector_boundary(db, now=now, hard_limit=Decimal("1")) + db.commit() + _ingest(db, _envelope(claim, "quota-first", now), int(now.timestamp())) + db.commit() + + with pytest.raises(FinancialConnectorCommercialAccessDenied): + _ingest(db, _envelope(claim, "quota-second", now), int(now.timestamp())) + db.rollback() + + assert db.scalar(select(func.count(FinancialConnectorEvent.id))) == 1 + assert db.scalar(select(func.count(UsageMeterEvent.id))) == 1 + + +def test_attachment_bytes_commit_and_rollback_follow_business_transaction( + factory: sessionmaker[Session], + tmp_path: Path, +) -> None: + now = datetime.now(UTC) + storage = _Storage(tmp_path / "expense-claims") + item = ExpenseClaimItem(id="item-a", claim_id="claim-a", invoice_id=None) + current_user = _user() + old_path = storage.build_item_dir("claim-a", "item-a") / "old.txt" + old_path.parent.mkdir(parents=True) + old_path.write_bytes(b"old") + item.invoice_id = storage.to_storage_key(old_path) + + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis="bytes", + tool_type="storage", + tool_name="attachment.upload", + hard_limit=Decimal("100"), + preflight_quantity=Decimal("100"), + ) + db.commit() + + stage_attachment_replacement( + db, + storage=storage, + item=item, + current_user=current_user, + claim_id="claim-a", + content=b"new-content", + request_id="upload-rollback", + ) + new_path = storage.build_item_dir("claim-a", "item-a") / "new.txt" + new_path.parent.mkdir(parents=True) + new_path.write_bytes(b"new-content") + db.rollback() + + assert old_path.read_bytes() == b"old" + assert not new_path.exists() + assert db.scalar(select(func.count(UsageMeterEvent.id))) == 0 + assert db.scalars(select(CommercialRuntimeReservation)).one().status == "released" + + stage_attachment_replacement( + db, + storage=storage, + item=item, + current_user=current_user, + claim_id="claim-a", + content=b"accepted", + request_id="upload-commit", + ) + accepted_path = storage.build_item_dir("claim-a", "item-a") / "accepted.txt" + accepted_path.parent.mkdir(parents=True) + accepted_path.write_bytes(b"accepted") + db.commit() + + assert accepted_path.read_bytes() == b"accepted" + assert not old_path.exists() + usage = db.scalars(select(UsageMeterEvent)).one() + assert Decimal(usage.quantity) == Decimal(len(b"accepted")) + assert usage.metadata_json["quantity_basis"] == "bytes" + + +@pytest.mark.parametrize( + ("basis", "hard_limit"), + [("bytes", Decimal("2")), ("objects", Decimal("100"))], +) +def test_attachment_quota_or_wrong_basis_denies_before_existing_file_moves( + factory: sessionmaker[Session], + tmp_path: Path, + basis: str, + hard_limit: Decimal, +) -> None: + now = datetime.now(UTC) + storage = _Storage(tmp_path / "expense-claims") + item = ExpenseClaimItem(id="item-a", claim_id="claim-a", invoice_id=None) + old_path = storage.build_item_dir("claim-a", "item-a") / "old.txt" + old_path.parent.mkdir(parents=True) + old_path.write_bytes(b"keep-me") + item.invoice_id = storage.to_storage_key(old_path) + + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis=basis, + tool_type="storage", + tool_name="attachment.upload", + hard_limit=hard_limit, + preflight_quantity=Decimal("100"), + ) + db.commit() + + with pytest.raises(ExpenseClaimAttachmentCommercialAccessDenied): + stage_attachment_replacement( + db, + storage=storage, + item=item, + current_user=_user(), + claim_id="claim-a", + content=b"three", + request_id="quota-denied", + ) + + assert old_path.read_bytes() == b"keep-me" + assert db.scalar(select(func.count(CommercialRuntimeReservation.id))) == 0 + assert db.scalar(select(func.count(UsageMeterEvent.id))) == 0 + + +def test_attachment_delete_restores_on_rollback_and_finalizes_on_commit( + factory: sessionmaker[Session], + tmp_path: Path, +) -> None: + storage = _Storage(tmp_path / "expense-claims") + item = ExpenseClaimItem(id="item-a", claim_id="claim-a", invoice_id=None) + file_path = storage.build_item_dir("claim-a", "item-a") / "invoice.pdf" + file_path.parent.mkdir(parents=True) + file_path.write_bytes(b"invoice") + item.invoice_id = storage.to_storage_key(file_path) + + with factory() as db: + stage_attachment_deletion(db, storage=storage, item=item) + assert not file_path.exists() + db.rollback() + assert file_path.read_bytes() == b"invoice" + + stage_attachment_deletion(db, storage=storage, item=item) + db.commit() + assert not file_path.exists() + + +def test_claim_attachment_tree_delete_follows_business_transaction( + factory: sessionmaker[Session], + tmp_path: Path, +) -> None: + storage = _Storage(tmp_path / "expense-claims") + file_path = storage.build_item_dir("claim-a", "item-a") / "invoice.pdf" + file_path.parent.mkdir(parents=True) + file_path.write_bytes(b"invoice") + + with factory() as db: + stage_claim_attachment_deletion(db, storage=storage, claim_id="claim-a") + assert not file_path.exists() + db.rollback() + assert file_path.read_bytes() == b"invoice" + + stage_claim_attachment_deletion(db, storage=storage, claim_id="claim-a") + db.commit() + assert not file_path.exists() + + +def test_multiple_attachment_replacements_restore_in_reverse_order( + factory: sessionmaker[Session], + tmp_path: Path, +) -> None: + storage = _Storage(tmp_path / "expense-claims") + item = ExpenseClaimItem(id="item-a", claim_id="claim-a", invoice_id=None) + original_path = storage.build_item_dir("claim-a", "item-a") / "original.txt" + original_path.parent.mkdir(parents=True) + original_path.write_bytes(b"original") + item.invoice_id = storage.to_storage_key(original_path) + + with factory() as db: + stage_attachment_replacement( + db, + storage=storage, + item=item, + current_user=_user(), + claim_id="claim-a", + content=b"middle", + request_id="replacement-one", + ) + middle_path = storage.build_item_dir("claim-a", "item-a") / "middle.txt" + middle_path.parent.mkdir(parents=True) + middle_path.write_bytes(b"middle") + item.invoice_id = storage.to_storage_key(middle_path) + + stage_attachment_replacement( + db, + storage=storage, + item=item, + current_user=_user(), + claim_id="claim-a", + content=b"latest", + request_id="replacement-two", + ) + latest_path = storage.build_item_dir("claim-a", "item-a") / "latest.txt" + latest_path.parent.mkdir(parents=True) + latest_path.write_bytes(b"latest") + db.rollback() + + assert original_path.read_bytes() == b"original" + assert not middle_path.exists() + assert not latest_path.exists() + + +class _Storage(ExpenseClaimAttachmentStorage): + def __init__(self, root: Path) -> None: + self._root = root + + def root(self) -> Path: + return self._root.resolve() + + +def _seed_connector_boundary( + db: Session, + *, + now: datetime, + hard_limit: Decimal, +) -> ExpenseClaim: + seed_meter( + db, + "tenant-a", + now, + basis="events", + tool_type="connector", + tool_name="financial.ingest", + hard_limit=hard_limit, + preflight_quantity=Decimal("1"), + ) + db.add( + FinancialConnectorConfig( + id=str(uuid.uuid4()), + tenant_id="tenant-a", + provider="metered-bank", + environment="mock", + key_version="v1", + secret_ref="connector/metered-test", + allowed_event_types_json=["payment_settled"], + clock_skew_seconds=300, + status="active", + created_by="platform-admin", + ) + ) + claim = ExpenseClaim( + id=str(uuid.uuid4()), + tenant_id="tenant-a", + claim_no=f"BX-METER-{uuid.uuid4().hex[:8]}", + employee_name="计量测试员工", + department_name="财务部", + expense_type="travel", + reason="连接器资源边界测试", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=now, + submitted_at=now, + status="pending_payment", + approval_stage="待付款", + risk_flags_json=[], + ) + db.add(claim) + db.flush() + return claim + + +def _envelope(claim: ExpenseClaim, event_id: str, now: datetime) -> FinancialEventEnvelope: + return FinancialEventEnvelope( + tenant_id="tenant-a", + external_event_id=event_id, + event_type="payment_settled", + occurred_at=now, + correlation_id=f"sensitive-correlation-{event_id}"[:64], + payload={ + "claim_id": claim.id, + "claim_reference": claim.claim_no, + "amount": str(claim.amount), + "currency": claim.currency, + "external_payment_reference": f"SENSITIVE-PAYMENT-{event_id}", + }, + ) + + +def _ingest( + db: Session, + envelope: FinancialEventEnvelope, + timestamp: int, + *, + signature: str = "", +): + resolved_signature = signature or sign_financial_event( + envelope, + timestamp=timestamp, + secret="metered-server-secret", + tenant_id="tenant-a", + provider="metered-bank", + key_version="v1", + ) + return FinancialConnectorIngestionService( + db, + secrets=FinancialConnectorSecretResolver( + {"connector/metered-test": "metered-server-secret"} + ), + now_epoch=timestamp, + ).ingest( + envelope, + tenant_header="tenant-a", + provider_header="metered-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header=resolved_signature, + ) + + +def _user() -> CurrentUserContext: + return CurrentUserContext( + username="employee-a", + name="员工甲", + role_codes=["employee"], + is_admin=False, + tenant_id="tenant-a", + auth_session_id=f"session-{time.time_ns()}", + ) diff --git a/server/tests/test_commercial_rollover_scheduler.py b/server/tests/test_commercial_rollover_scheduler.py new file mode 100644 index 0000000..2441856 --- /dev/null +++ b/server/tests/test_commercial_rollover_scheduler.py @@ -0,0 +1,90 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal + +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.db.base_class import Base +from app.models.commercial_billing import CommercialBillingPeriod +from app.schemas.commercial import CommercialPlanCreate, CommercialSubscriptionCreate +from app.services.commercial_admin import CommercialAdminService +from app.services.commercial_rollover_scheduler import CommercialRolloverScheduler + + +def test_scheduler_rolls_due_subscription_once_and_respects_leader_lease() -> None: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + due = datetime(2026, 2, 1, tzinfo=UTC) + with factory() as db: + subscription_id = _seed_due_subscription(db) + db.commit() + + scheduler = CommercialRolloverScheduler(session_factory=factory) + first = scheduler._run_once(as_of=due) + replay = scheduler._run_once(as_of=due) + + assert first["scanned"] == 1 + assert first["rolled_over"] == 1 + assert first["periods_created"] == 1 + assert replay["scanned"] == 0 + with factory() as db: + assert ( + db.scalar( + select(func.count()) + .select_from(CommercialBillingPeriod) + .where(CommercialBillingPeriod.subscription_id == subscription_id) + ) + == 2 + ) + + scheduler._try_acquire_lease = lambda _db: False # type: ignore[method-assign] + skipped = scheduler._run_once(as_of=due) + assert skipped["leader_skipped"] == 1 + engine.dispose() + + +def _seed_due_subscription(db: Session) -> str: + start = datetime(2026, 1, 1, tzinfo=UTC) + due = datetime(2026, 2, 1, tzinfo=UTC) + admin = CommercialAdminService(db) + plan = admin.create_plan( + "tenant-scheduler", + CommercialPlanCreate( + plan_code="scheduler", + name="调度器测试套餐", + pricing_model="subscription", + billing_interval="monthly", + currency="CNY", + base_fee=Decimal("100"), + effective_from=start, + ), + actor_id="scheduler-test", + ) + admin.activate_plan( + "tenant-scheduler", + plan.id, + expected_version=plan.version, + ) + subscription = admin.create_subscription( + "tenant-scheduler", + CommercialSubscriptionCreate( + subscription_key="scheduler-subscription", + plan_id=plan.id, + starts_at=start, + current_period_start=start, + current_period_end=due, + seats=1, + auto_renew=True, + ), + actor_id="scheduler-test", + ) + return subscription.id diff --git a/server/tests/test_commercial_runtime_metering.py b/server/tests/test_commercial_runtime_metering.py new file mode 100644 index 0000000..e8549ff --- /dev/null +++ b/server/tests/test_commercial_runtime_metering.py @@ -0,0 +1,690 @@ +from __future__ import annotations + +import json +import uuid +from datetime import UTC, datetime, timedelta +from decimal import Decimal +from typing import Any + +import pytest +from commercial_runtime_testkit import seed_meter as _seed_meter +from commercial_runtime_testkit import seed_run as _seed_run +from commercial_runtime_testkit import seed_tool_call as _seed_tool_call +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.db.base_class import Base +from app.models.agent_run import AgentToolCall +from app.models.commercial import CommercialCostEvent, UsageMeterEvent +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.agent_runs import AgentRunService +from app.services.commercial_access_policy import CommercialConflictError +from app.services.commercial_entitlements import CommercialEntitlementService +from app.services.commercial_metering import CommercialMeteringService +from app.services.commercial_runtime_bridge import CommercialRuntimeBridge +from app.services.commercial_runtime_metering import CommercialRuntimeMeteringService +from app.services.commercial_runtime_reservations import ( + CommercialRuntimeReservationService, +) +from app.services.orchestrator_execution import OrchestratorExecutionEngine + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_real_tokens_create_redacted_usage_and_linked_cost_idempotently( + db: Session, +) -> None: + now = datetime.now(UTC) + _, entitlement = _seed_meter( + db, + "tenant-a", + now, + basis="total_tokens", + internal_cost={ + "enabled": True, + "cost_category": "ai_inference", + "unit": "token", + "unit_cost": "0.002", + "original_currency": "CNY", + "reporting_currency": "CNY", + "fx_rate": "1", + "provider": "provider-a", + "model_name": "model-a", + }, + ) + _, tool_call = _seed_tool_call( + db, + now, + route_json={"tenant_id": "tenant-a"}, + request_json={ + "usage": {"input_tokens": 12}, + "prompt": "REQUEST-TOP-SECRET", + "tenant_id": "tenant-spoofed", + "user_id": "raw-user-in-request", + }, + response_json={ + "usage": {"output_tokens": 8}, + "answer": "RESPONSE-TOP-SECRET", + }, + user_id="raw-run-user", + ) + + result = CommercialRuntimeMeteringService(db).sync_tool_call(tool_call.id) + replay = CommercialRuntimeMeteringService(db).sync_tool_call(tool_call.id) + usage = db.scalars(select(UsageMeterEvent)).one() + cost = db.scalars(select(CommercialCostEvent)).one() + + assert result.status == "created" + assert result.quantity == Decimal("20") + assert replay.status == "replayed" + assert replay.usage_event_id == usage.id + assert replay.cost_event_id == cost.id + assert usage.tenant_id == "tenant-a" + assert usage.entitlement_id == entitlement.id + assert Decimal(usage.quantity) == Decimal("20") + assert cost.usage_event_id == usage.id + assert Decimal(cost.quantity) == Decimal("20") + assert Decimal(cost.cost_amount) == Decimal("0.0400") + assert db.query(UsageMeterEvent).count() == 1 + assert db.query(CommercialCostEvent).count() == 1 + + serialized_metadata = json.dumps( + {"usage": usage.metadata_json, "cost": cost.metadata_json}, + ensure_ascii=False, + ) + for forbidden in ( + "REQUEST-TOP-SECRET", + "RESPONSE-TOP-SECRET", + "raw-run-user", + "raw-user-in-request", + "tenant-spoofed", + "request_json", + "response_json", + ): + assert forbidden not in serialized_metadata + + +@pytest.mark.parametrize( + ("basis", "request_json", "response_json", "duration_ms", "expected"), + [ + ("input_tokens", {"token_usage": {"prompt_tokens": 11}}, {}, 0, Decimal("11")), + ("output_tokens", {}, {"metrics": {"completion_tokens": 7}}, 0, Decimal("7")), + ("duration_ms", {}, {}, 321, Decimal("321")), + ], +) +def test_supported_measured_quantity_bases( + db: Session, + basis: str, + request_json: dict[str, Any], + response_json: dict[str, Any], + duration_ms: int, + expected: Decimal, +) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis=basis) + _, tool_call = _seed_tool_call( + db, + now, + route_json={"tenant_id": "tenant-a"}, + request_json=request_json, + response_json=response_json, + duration_ms=duration_ms, + ) + + result = CommercialRuntimeMeteringService(db).sync_tool_call(tool_call.id) + + assert result.status == "created" + assert result.quantity_basis == basis + assert result.quantity == expected + assert Decimal(db.scalars(select(UsageMeterEvent)).one().quantity) == expected + + +def test_call_basis_uses_route_tenant_and_isolated_entitlement(db: Session) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="call") + _, tenant_b_entitlement = _seed_meter( + db, + "tenant-b", + now, + basis="call", + subscription_status="trialing", + ) + _, tool_call = _seed_tool_call( + db, + now, + route_json={"tenant_id": "tenant-b"}, + request_json={"tenant_id": "tenant-a", "user_id": "tenant-a-user"}, + response_json={"tenant_id": "tenant-a"}, + ) + + result = CommercialRuntimeMeteringService(db).sync_tool_call(tool_call.id) + usage = db.scalars(select(UsageMeterEvent)).one() + + assert result.status == "created" + assert result.quantity == Decimal("1") + assert usage.tenant_id == "tenant-b" + assert usage.entitlement_id == tenant_b_entitlement.id + assert ( + db.scalar(select(UsageMeterEvent.id).where(UsageMeterEvent.tenant_id == "tenant-a")) is None + ) + + +def test_missing_route_tenant_and_missing_tokens_are_collecting_not_estimated( + db: Session, +) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="total_tokens") + _, unowned_call = _seed_tool_call( + db, + now, + route_json={}, + request_json={"tenant_id": "tenant-a", "usage": {"input_tokens": 9}}, + response_json={"usage": {"output_tokens": 3}}, + ) + _, unmeasured_call = _seed_tool_call( + db, + now + timedelta(microseconds=1), + route_json={"tenant_id": "tenant-a"}, + request_json={"prompt": "x" * 20_000}, + response_json={"answer": "y" * 20_000}, + ) + service = CommercialRuntimeMeteringService(db) + + unowned = service.sync_tool_call(unowned_call.id) + unmeasured = service.sync_tool_call(unmeasured_call.id) + + assert unowned.status == "skipped" + assert unowned.reason_code == "missing_tenant" + assert unowned.business_call_occurred is True + assert unowned.requires_reconciliation is True + assert unmeasured.status == "skipped" + assert unmeasured.reason_code == "collecting_missing_tokens" + assert unmeasured.collecting is True + assert "正文长度" in unmeasured.reason + assert db.query(UsageMeterEvent).count() == 0 + + +def test_only_active_exact_runtime_meter_is_eligible(db: Session) -> None: + now = datetime.now(UTC) + subscription, entitlement = _seed_meter(db, "tenant-a", now, basis="call") + service = CommercialRuntimeMeteringService(db) + _, wrong_name = _seed_tool_call( + db, + now, + route_json={"tenant_id": "tenant-a"}, + tool_name="llm.responses", + ) + assert service.sync_tool_call(wrong_name.id).reason_code == "no_matching_runtime_meter" + + entitlement.status = "suspended" + db.flush() + _, suspended_entitlement = _seed_tool_call( + db, + now + timedelta(microseconds=1), + route_json={"tenant_id": "tenant-a"}, + ) + result = service.sync_tool_call(suspended_entitlement.id) + assert result.reason_code == "no_matching_runtime_meter" + + entitlement.status = "active" + subscription.status = "past_due" + db.flush() + _, inactive_subscription = _seed_tool_call( + db, + now + timedelta(microseconds=2), + route_json={"tenant_id": "tenant-a"}, + ) + result = service.sync_tool_call(inactive_subscription.id) + assert result.reason_code == "no_active_subscription" + assert db.query(UsageMeterEvent).count() == 0 + + +def test_preflight_blocks_quota_and_post_call_error_requires_reconciliation( + db: Session, +) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="call", hard_limit=Decimal("2")) + run, first = _seed_tool_call(db, now, route_json={"tenant_id": "tenant-a"}) + _, second = _seed_tool_call( + db, + now + timedelta(microseconds=1), + route_json={"tenant_id": "tenant-a"}, + run=run, + ) + _, third = _seed_tool_call( + db, + now + timedelta(microseconds=2), + route_json={"tenant_id": "tenant-a"}, + run=run, + ) + service = CommercialRuntimeMeteringService(db) + + allowed = service.preflight_run_tool( + run.run_id, + tool_type="llm", + tool_name="chat.completions", + ) + assert allowed.allowed is True + assert ( + service.assert_run_tool_allowed( + run.run_id, + tool_type="llm", + tool_name="chat.completions", + ).allowed + is True + ) + oversized = service.preflight_run_tool( + run.run_id, + tool_type="llm", + tool_name="chat.completions", + requested_quantity=Decimal("3"), + ) + assert oversized.allowed is False + with pytest.raises(CommercialConflictError): + service.assert_run_tool_allowed( + run.run_id, + tool_type="llm", + tool_name="chat.completions", + requested_quantity=Decimal("3"), + ) + + assert service.sync_tool_call(first.id).status == "created" + assert service.sync_tool_call(second.id).status == "created" + exhausted = service.preflight_run_tool( + run.run_id, + tool_type="llm", + tool_name="chat.completions", + ) + assert exhausted.allowed is False + failed_fact = service.sync_tool_call(third.id) + assert failed_fact.status == "error" + assert failed_fact.reason_code == "metering_failed_after_business_call" + assert failed_fact.business_call_occurred is True + assert failed_fact.requires_reconciliation is True + assert db.query(UsageMeterEvent).count() == 2 + + +def test_zero_internal_cost_never_writes_fake_usage_or_cost(db: Session) -> None: + now = datetime.now(UTC) + _seed_meter( + db, + "tenant-a", + now, + basis="call", + internal_cost={ + "enabled": True, + "cost_category": "ai_inference", + "unit_cost": "0", + "fx_rate": "1", + "original_currency": "CNY", + "reporting_currency": "CNY", + }, + ) + _, tool_call = _seed_tool_call(db, now, route_json={"tenant_id": "tenant-a"}) + + result = CommercialRuntimeMeteringService(db).sync_tool_call(tool_call.id) + + assert result.status == "error" + assert result.business_call_occurred is True + assert result.requires_reconciliation is True + assert db.query(UsageMeterEvent).count() == 0 + assert db.query(CommercialCostEvent).count() == 0 + + +def test_batch_is_bounded_cursor_based_and_keeps_error_semantics(db: Session) -> None: + now = datetime.now(UTC) - timedelta(seconds=1) + _seed_meter(db, "tenant-a", now, basis="input_tokens") + _seed_tool_call( + db, + now, + route_json={"tenant_id": "tenant-a"}, + request_json={"usage": {"input_tokens": 3}}, + ) + _seed_tool_call( + db, + now + timedelta(microseconds=1), + route_json={"tenant_id": "tenant-a"}, + request_json={"prompt": "not-a-token-count"}, + ) + _seed_tool_call( + db, + now + timedelta(microseconds=2), + route_json={"tenant_id": "tenant-a"}, + request_json={"usage": {"input_tokens": "3"}}, + ) + service = CommercialRuntimeMeteringService(db) + + first_page = service.sync_batch(limit=2) + assert first_page.created == 1 + assert first_page.skipped == 1 + assert first_page.errors == 0 + assert first_page.has_more is True + assert first_page.next_cursor is not None + + second_page = service.sync_batch(limit=2, cursor=first_page.next_cursor) + assert second_page.created == 0 + assert second_page.skipped == 0 + assert second_page.errors == 1 + assert second_page.items[0].business_call_occurred is True + assert second_page.items[0].requires_reconciliation is True + assert second_page.has_more is False + assert db.query(UsageMeterEvent).count() == 1 + with pytest.raises(ValueError, match="200"): + service.sync_batch(limit=201) + + +def test_non_successful_tool_calls_never_enter_commercial_ledger(db: Session) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="call") + calls = [ + _seed_tool_call( + db, + now + timedelta(microseconds=index), + route_json={"tenant_id": "tenant-a"}, + status=status, + )[1] + for index, status in enumerate(("running", "blocked", "failed")) + ] + service = CommercialRuntimeMeteringService(db) + + batch = service.sync_batch(limit=10) + + assert [item.reason_code for item in batch.items] == [ + "tool_call_not_terminal", + "tool_call_not_billable", + "tool_call_not_billable", + ] + assert batch.items[0].collecting is True + assert batch.items[0].requires_reconciliation is True + assert batch.items[1].business_call_occurred is False + assert batch.items[2].business_call_occurred is True + assert db.query(UsageMeterEvent).count() == 0 + + calls[0].status = "succeeded" + db.flush() + completed = service.sync_tool_call(calls[0].id) + + assert completed.status == "created" + assert db.query(UsageMeterEvent).count() == 1 + + +def test_runtime_bridge_bypasses_unconfigured_tool_without_fake_fact(db: Session) -> None: + now = datetime.now(UTC) + run = _seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + bridge = CommercialRuntimeBridge(db) + + gate = bridge.preflight(run.run_id, tool_type="llm", tool_name="chat.completions") + _, call = _seed_tool_call( + db, + now, + route_json={"tenant_id": "tenant-a"}, + run=run, + ) + result = bridge.sync_tool_call(call.id) + + assert gate.enforced is False + assert gate.allowed is True + assert gate.reason_code == "runtime_meter_not_configured" + assert result.status == "skipped" + assert result.requires_reconciliation is False + assert db.query(UsageMeterEvent).count() == 0 + + +def test_agent_run_service_without_preflight_creates_durable_backlog( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="call") + run = _seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + + call = run_service.record_tool_call( + run_id=run.run_id, + tool_type="llm", + tool_name="chat.completions", + request_json={"prompt": "not persisted to commercial metadata"}, + response_json={"answer": "ok"}, + status="succeeded", + duration_ms=12, + ) + replay = CommercialRuntimeBridge(db).sync_tool_call(call.id) + + backlog = db.scalars(select(CommercialRuntimeReservation)).one() + assert replay.status == "error" + assert replay.reason_code == "reservation_not_settleable" + assert backlog.status == "reconciliation_required" + assert backlog.tool_call_id == call.id + assert db.query(UsageMeterEvent).count() == 0 + + +def test_running_direct_tool_call_requires_pre_execution_reservation_on_update( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="duration_ms") + run = _seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + + call = run_service.record_tool_call( + run_id=run.run_id, + tool_type="llm", + tool_name="chat.completions", + status="running", + ) + assert db.query(UsageMeterEvent).count() == 0 + + run_service.update_tool_call( + call.id, + response_json={"answer": "done"}, + status="succeeded", + duration_ms=321, + ) + + backlog = db.scalars(select(CommercialRuntimeReservation)).one() + assert backlog.status == "reconciliation_required" + assert Decimal(backlog.actual_quantity or 0) == Decimal("321") + assert db.query(UsageMeterEvent).count() == 0 + + +def test_metering_failure_keeps_tool_call_and_supports_idempotent_reconciliation( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="call") + run = _seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + call_id = str(uuid.uuid4()) + permit = CommercialRuntimeBridge(db).reserve_tool( + run.run_id, + tool_call_id=call_id, + tool_type="llm", + tool_name="chat.completions", + ) + assert permit.reservation_id is not None + original_sync = CommercialRuntimeMeteringService.sync_reserved_tool_call + + def fail_metering(self, tool_call_id: str, reservation): # noqa: ANN001 + del self, tool_call_id, reservation + raise RuntimeError("simulated metering outage") + + monkeypatch.setattr( + CommercialRuntimeMeteringService, + "sync_reserved_tool_call", + fail_metering, + ) + call = run_service.record_tool_call( + run_id=run.run_id, + tool_call_id=call_id, + tool_type="llm", + tool_name="chat.completions", + status="succeeded", + ) + + assert db.get(AgentToolCall, call.id) is not None + assert db.query(UsageMeterEvent).count() == 0 + + monkeypatch.setattr( + CommercialRuntimeMeteringService, + "sync_reserved_tool_call", + original_sync, + ) + reconciled = CommercialRuntimeBridge(db).sync_tool_call(call.id) + replay = CommercialRuntimeBridge(db).sync_tool_call(call.id) + + assert reconciled.status == "created" + assert replay.status == "replayed" + assert db.query(UsageMeterEvent).count() == 1 + + +def test_cost_failure_commits_usage_then_reconciliation_appends_missing_cost( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + _seed_meter( + db, + "tenant-a", + now, + basis="call", + internal_cost={ + "enabled": True, + "cost_category": "ai_inference", + "unit": "call", + "unit_cost": "0.5", + "original_currency": "CNY", + "reporting_currency": "CNY", + "fx_rate": "1", + }, + ) + run = _seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + call_id = str(uuid.uuid4()) + permit = CommercialRuntimeBridge(db).reserve_tool( + run.run_id, + tool_call_id=call_id, + tool_type="llm", + tool_name="chat.completions", + ) + assert permit.reservation_id is not None + original_record_cost = CommercialMeteringService.record_cost + + def fail_cost(self, tenant_id, payload): # noqa: ANN001 + del self, tenant_id, payload + raise RuntimeError("simulated cost ledger outage") + + monkeypatch.setattr(CommercialMeteringService, "record_cost", fail_cost) + call = run_service.record_tool_call( + run_id=run.run_id, + tool_call_id=call_id, + tool_type="llm", + tool_name="chat.completions", + status="succeeded", + ) + + assert db.query(UsageMeterEvent).count() == 1 + assert db.query(CommercialCostEvent).count() == 0 + reservation = db.get(CommercialRuntimeReservation, permit.reservation_id) + assert reservation is not None + assert reservation.status == "committed_reconciliation_required" + assert reservation.resolution_code == "cost_metering_failed" + candidates = CommercialRuntimeReservationService(db).reconciliation_candidates() + quota = CommercialEntitlementService(db).get_account_for_tenant("tenant-a").quotas[0] + assert [row.id for row in candidates] == [reservation.id] + assert quota.used_quantity == Decimal("1") + assert quota.reserved_quantity == Decimal("0") + + monkeypatch.setattr(CommercialMeteringService, "record_cost", original_record_cost) + reconciled = CommercialRuntimeBridge(db).sync_tool_call(call.id) + db.refresh(reservation) + + assert reconciled.status == "created" + assert reconciled.usage_created is False + assert reconciled.cost_created is True + assert db.query(UsageMeterEvent).count() == 1 + assert db.query(CommercialCostEvent).count() == 1 + assert reservation.status == "committed" + assert reservation.resolution_code is None + + +def test_orchestrator_preflight_blocks_executor_after_quota_exhaustion( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + _seed_meter(db, "tenant-a", now, basis="call", hard_limit=Decimal("1")) + run = _seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + first_call_id = str(uuid.uuid4()) + permit = CommercialRuntimeBridge(db).reserve_tool( + run.run_id, + tool_call_id=first_call_id, + tool_type="llm", + tool_name="chat.completions", + ) + assert permit.reservation_id is not None + run_service.record_tool_call( + run_id=run.run_id, + tool_call_id=first_call_id, + tool_type="llm", + tool_name="chat.completions", + status="succeeded", + ) + engine = OrchestratorExecutionEngine( + db=db, + run_service=run_service, + expense_claim_service=None, + knowledge_service=None, + user_agent_service=None, + database_query_builder=None, + ) + executed = False + + def executor() -> dict[str, Any]: + nonlocal executed + executed = True + return {"answer": "should not run"} + + response, degraded = engine._invoke_tool( + run_id=run.run_id, + tool_type="llm", + tool_name="chat.completions", + request_json={"prompt": "quota check"}, + context_json={}, + executor=executor, + fallback_factory=lambda error: {"error": str(error)}, + ) + + assert executed is False + assert degraded is True + assert "配额" in response["error"] + assert db.query(UsageMeterEvent).count() == 1 + statuses = sorted( + db.scalars( + select(AgentToolCall.status) + .where(AgentToolCall.run_id == run.run_id) + ).all() + ) + assert statuses == ["blocked", "succeeded"] diff --git a/server/tests/test_commercial_runtime_reservations.py b/server/tests/test_commercial_runtime_reservations.py new file mode 100644 index 0000000..b37adfb --- /dev/null +++ b/server/tests/test_commercial_runtime_reservations.py @@ -0,0 +1,348 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, datetime, timedelta +from decimal import Decimal + +import pytest +from commercial_runtime_testkit import seed_meter, seed_run, seed_tool_call +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.db.base_class import Base +from app.models.commercial import UsageMeterEvent +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.agent_runs import AgentRunService +from app.services.commercial_entitlements import CommercialEntitlementService +from app.services.commercial_runtime_bridge import CommercialRuntimeBridge +from app.services.commercial_runtime_reconciler import CommercialRuntimeReconciler + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_reserved_success_settles_real_usage_and_replays_idempotently( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + _, entitlement = seed_meter(db, "tenant-a", now, basis="call", hard_limit=Decimal("2")) + run = seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + tool_call_id = str(uuid.uuid4()) + bridge = CommercialRuntimeBridge(db) + + permit = bridge.reserve_tool( + run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + ) + reservation = db.get(CommercialRuntimeReservation, permit.reservation_id) + assert permit.gate.allowed is True + assert reservation is not None and reservation.status == "reserved" + assert db.query(UsageMeterEvent).count() == 0 + + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + run_service.record_tool_call( + run_id=run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + status="succeeded", + ) + replay = bridge.sync_tool_call(tool_call_id) + db.refresh(reservation) + quota = CommercialEntitlementService(db).get_account_for_tenant("tenant-a").quotas[0] + + assert reservation.status == "committed" + assert Decimal(reservation.actual_quantity or 0) == Decimal("1") + assert replay.status == "replayed" + assert db.query(UsageMeterEvent).count() == 1 + assert quota.entitlement.id == entitlement.id + assert quota.used_quantity == Decimal("1") + assert quota.reserved_quantity == Decimal("0") + + +@pytest.mark.parametrize( + ("tool_status", "business_occurred"), + [("failed", True), ("blocked", False)], +) +def test_failed_or_blocked_tool_releases_reservation_without_usage( + db: Session, + monkeypatch: pytest.MonkeyPatch, + tool_status: str, + business_occurred: bool, +) -> None: + now = datetime.now(UTC) + seed_meter(db, "tenant-a", now, basis="call", hard_limit=Decimal("1")) + run = seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + tool_call_id = str(uuid.uuid4()) + permit = CommercialRuntimeBridge(db).reserve_tool( + run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + ) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + + run_service.record_tool_call( + run_id=run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + status=tool_status, + ) + result = CommercialRuntimeBridge(db).sync_tool_call(tool_call_id) + reservation = db.get(CommercialRuntimeReservation, permit.reservation_id) + + assert reservation is not None and reservation.status == "released" + assert result.reason_code == "tool_call_not_billable" + assert result.business_call_occurred is business_occurred + assert db.query(UsageMeterEvent).count() == 0 + + +def test_active_reservation_holds_hard_quota_before_execution(db: Session) -> None: + now = datetime.now(UTC) + seed_meter(db, "tenant-a", now, basis="call", hard_limit=Decimal("1")) + run = seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + bridge = CommercialRuntimeBridge(db) + + first = bridge.reserve_tool( + run.run_id, + tool_call_id=str(uuid.uuid4()), + tool_type="llm", + tool_name="chat.completions", + ) + second = bridge.reserve_tool( + run.run_id, + tool_call_id=str(uuid.uuid4()), + tool_type="llm", + tool_name="chat.completions", + ) + quota = CommercialEntitlementService(db).get_account_for_tenant("tenant-a").quotas[0] + + assert first.reservation_id is not None + assert second.reservation_id is None + assert second.gate.allowed is False + assert quota.reserved_quantity == Decimal("1") + assert quota.hard_limit_remaining == Decimal("0") + + +def test_same_reservation_request_replays_without_consuming_its_own_quota( + db: Session, +) -> None: + now = datetime.now(UTC) + seed_meter(db, "tenant-a", now, basis="call", hard_limit=Decimal("1")) + run = seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + bridge = CommercialRuntimeBridge(db) + tool_call_id = str(uuid.uuid4()) + + first = bridge.reserve_tool( + run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + ) + replay = bridge.reserve_tool( + run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + ) + + assert replay.gate.allowed is True + assert replay.gate.reason_code == "reservation_replayed" + assert replay.reservation_id == first.reservation_id + assert db.query(CommercialRuntimeReservation).count() == 1 + + +def test_variable_quantity_requires_hard_max_and_refuses_actual_over_reservation( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + seed_meter(db, "tenant-a", now, basis="duration_ms", hard_limit=Decimal("1000")) + run = seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + bridge = CommercialRuntimeBridge(db) + rejected = bridge.reserve_tool( + run.run_id, + tool_call_id=str(uuid.uuid4()), + tool_type="llm", + tool_name="chat.completions", + requested_quantity=Decimal("500"), + ) + assert rejected.gate.reason_code == "hard_max_required" + + tool_call_id = str(uuid.uuid4()) + permit = bridge.reserve_tool( + run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + requested_quantity=Decimal("500"), + hard_max_confirmed=True, + ) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + run_service.record_tool_call( + run_id=run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + status="succeeded", + duration_ms=501, + ) + failed = bridge.sync_tool_call(tool_call_id) + reservation = db.get(CommercialRuntimeReservation, permit.reservation_id) + assert failed.status == "error" + assert failed.requires_reconciliation is True + assert reservation is not None and reservation.status == "reserved" + assert db.query(UsageMeterEvent).count() == 0 + + run_service.update_tool_call(tool_call_id, duration_ms=400, status="succeeded") + db.refresh(reservation) + assert reservation.status == "committed" + assert Decimal(db.scalars(select(UsageMeterEvent)).one().quantity) == Decimal("400") + + +def test_expired_historical_meter_does_not_enable_runtime_enforcement(db: Session) -> None: + now = datetime.now(UTC) + subscription, _ = seed_meter(db, "tenant-a", now, basis="call") + subscription.current_period_start = now - timedelta(days=31) + subscription.current_period_end = now - timedelta(days=1) + db.commit() + run = seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + + permit = CommercialRuntimeBridge(db).reserve_tool( + run.run_id, + tool_call_id=str(uuid.uuid4()), + tool_type="llm", + tool_name="chat.completions", + ) + + assert permit.gate.enforced is False + assert permit.gate.allowed is True + assert permit.gate.reason_code == "runtime_meter_not_configured" + assert permit.reservation_id is None + + +def test_legacy_run_without_tenant_skips_bridge_without_reconciliation(db: Session) -> None: + now = datetime.now(UTC) + _, tool_call = seed_tool_call(db, now, route_json={}) + + result = CommercialRuntimeBridge(db).sync_tool_call(tool_call.id) + + assert result.status == "skipped" + assert result.reason_code == "legacy_run_without_tenant" + assert result.requires_reconciliation is False + assert db.query(CommercialRuntimeReservation).count() == 0 + + +def test_direct_call_on_suspended_contract_is_persisted_for_reconciliation( + db: Session, + monkeypatch: pytest.MonkeyPatch, +) -> None: + now = datetime.now(UTC) + subscription, _ = seed_meter( + db, + "tenant-a", + now, + basis="call", + ) + subscription.status = "suspended" + db.flush() + run = seed_run(db, now, route_json={"tenant_id": "tenant-a"}) + run_service = AgentRunService(db) + monkeypatch.setattr(run_service, "_ensure_ready", lambda: None) + + call = run_service.record_tool_call( + run_id=run.run_id, + tool_type="llm", + tool_name="chat.completions", + status="succeeded", + ) + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + + assert reservation.tool_call_id == call.id + assert reservation.status == "reconciliation_required" + assert db.query(UsageMeterEvent).count() == 0 + + +def test_expired_reconciler_settles_releases_and_preserves_uncertain_holds( + db: Session, +) -> None: + now = datetime.now(UTC) + seed_meter(db, "tenant-a", now, basis="call", hard_limit=Decimal("10")) + bridge = CommercialRuntimeBridge(db) + reservations: dict[str, CommercialRuntimeReservation] = {} + for scenario, run_status in ( + ("success", "running"), + ("failure", "running"), + ("terminal_without_call", "failed"), + ("uncertain", "running"), + ): + run = seed_run( + db, + now, + route_json={"tenant_id": "tenant-a"}, + status=run_status, + ) + tool_call_id = str(uuid.uuid4()) + permit = bridge.reserve_tool( + run.run_id, + tool_call_id=tool_call_id, + tool_type="llm", + tool_name="chat.completions", + ) + reservation = db.get(CommercialRuntimeReservation, permit.reservation_id) + assert reservation is not None + reservation.created_at = now - timedelta(hours=1) + reservation.expires_at = now - timedelta(minutes=30) + reservations[scenario] = reservation + if scenario == "success": + seed_tool_call( + db, + now - timedelta(minutes=45), + route_json={"tenant_id": "tenant-a"}, + run=run, + status="succeeded", + )[1].id = tool_call_id + elif scenario == "failure": + seed_tool_call( + db, + now - timedelta(minutes=45), + route_json={"tenant_id": "tenant-a"}, + run=run, + status="failed", + )[1].id = tool_call_id + db.commit() + + batch = CommercialRuntimeReconciler(db).reconcile_expired(as_of=now, limit=10) + for row in reservations.values(): + db.refresh(row) + + assert batch.settled == 1 + assert batch.released == 2 + assert batch.deferred == 1 + assert batch.errors == 0 + assert reservations["success"].status == "committed" + assert reservations["failure"].status == "released" + assert reservations["terminal_without_call"].status == "expired" + assert reservations["uncertain"].status == "reserved" + assert db.query(UsageMeterEvent).count() == 1 diff --git a/server/tests/test_commercial_services.py b/server/tests/test_commercial_services.py new file mode 100644 index 0000000..f9a3660 --- /dev/null +++ b/server/tests/test_commercial_services.py @@ -0,0 +1,557 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, date, datetime, timedelta +from decimal import Decimal + +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext +from app.db.base_class import Base +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.schemas.commercial import ( + CommercialCostEventCreate, + CommercialEntitlementUpsert, + CommercialPlanCreate, + CommercialPricingScenarioWrite, + CommercialSubscriptionCreate, + CommercialSubscriptionTransition, + UsageMeterEventCreate, +) +from app.schemas.savings import ( + SavingsEvidenceCreate, + SavingsRealizationActionCreate, + SavingsRealizationCreate, +) +from app.services.commercial_access_policy import CommercialConflictError +from app.services.commercial_admin import CommercialAdminService +from app.services.commercial_analytics import CommercialAnalyticsService +from app.services.commercial_entitlements import CommercialEntitlementService +from app.services.commercial_metering import CommercialMeteringService +from app.services.commercial_pricing import CommercialPricingService +from app.services.commercial_queries import CommercialQueryService +from app.services.savings_discovery import SavingsDiscoveryService +from app.services.savings_realization import SavingsRealizationService + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_commercial_account_quota_idempotency_and_security_gate(db: Session) -> None: + now = datetime.now(UTC) + _, subscription, entitlement = _seed_commercial_account(db, "tenant-a", now) + service = CommercialMeteringService(db) + payload = UsageMeterEventCreate( + subscription_id=subscription.id, + entitlement_id=entitlement.id, + quantity=Decimal("2"), + occurred_at=now, + source_system="agent-runtime", + idempotency_key="usage-001", + subject_type="agent_run", + subject_id="run-001", + ) + created, was_created = service.record_usage( + "tenant-a", + payload, + actor_type="system", + actor_id="agent-runtime", + ) + replay, replay_created = service.record_usage( + "tenant-a", + payload, + actor_type="system", + actor_id="agent-runtime", + ) + assert was_created is True + assert replay_created is False + assert replay.id == created.id + + with pytest.raises(CommercialConflictError, match="幂等键"): + service.record_usage( + "tenant-a", + payload.model_copy(update={"quantity": Decimal("3")}), + actor_type="system", + actor_id="agent-runtime", + ) + + account = CommercialEntitlementService(db).get_account( + _user("finance-a", tenant_id="tenant-a", roles=["finance"]), + as_of=now + timedelta(seconds=1), + ) + assert account.tenant_id == "tenant-a" + assert account.quotas[0].used_quantity == Decimal("2") + assert account.quotas[0].hard_limit_remaining == Decimal("4") + + commercial_gate = CommercialEntitlementService(db).check( + "tenant-a", + entitlement_key="ai_precheck", + requested_quantity=Decimal("1"), + security_decision="human_review", + as_of=now + timedelta(seconds=1), + ) + assert commercial_gate.commercial_allowed is True + assert commercial_gate.final_allowed is False + assert "人工审核" in commercial_gate.reason + + with pytest.raises(CommercialConflictError, match="硬配额"): + service.record_usage( + "tenant-a", + payload.model_copy( + update={ + "quantity": Decimal("5"), + "idempotency_key": "usage-over-limit", + } + ), + actor_type="system", + actor_id="agent-runtime", + ) + + with pytest.raises(CommercialConflictError, match="不能回改配额"): + CommercialAdminService(db).upsert_entitlement( + "tenant-a", + CommercialEntitlementUpsert( + subscription_id=subscription.id, + entitlement_key="ai_precheck", + metric_key="ai_precheck_runs", + entitlement_type="metered", + unit="run", + included_quantity=Decimal("5"), + hard_limit_quantity=Decimal("8"), + reset_interval="monthly", + overage_policy="block", + effective_from=now - timedelta(days=10), + ), + ) + with pytest.raises(LookupError): + service.record_usage( + "tenant-b", + payload.model_copy(update={"idempotency_key": "cross-tenant"}), + actor_type="system", + actor_id="agent-runtime", + ) + + +def test_cost_events_are_server_derived_idempotent_and_reversible(db: Session) -> None: + now = datetime.now(UTC) + _, subscription, _ = _seed_commercial_account(db, "tenant-a", now) + payload = CommercialCostEventCreate( + subscription_id=subscription.id, + event_type="incurred", + cost_category="ai_inference", + quantity=Decimal("200"), + unit="1k_tokens", + unit_cost=Decimal("0.1"), + original_currency="CNY", + reporting_currency="CNY", + fx_rate=Decimal("1"), + provider="provider-a", + model_name="model-a", + allocation_key="tenant-a:ai", + occurred_at=now, + source_system="provider-billing", + idempotency_key="cost-001", + ) + service = CommercialMeteringService(db) + event, created = service.record_cost("tenant-a", payload) + replay, replay_created = service.record_cost("tenant-a", payload) + assert created is True + assert replay_created is False + assert replay.id == event.id + assert event.cost_amount == Decimal("20.0000") + assert event.reporting_amount == Decimal("20.0000") + + with pytest.raises(CommercialConflictError, match="幂等键"): + service.record_cost( + "tenant-a", + payload.model_copy(update={"quantity": Decimal("201")}), + ) + + reversal_payload = payload.model_copy( + update={ + "event_type": "reversal", + "idempotency_key": "cost-reversal-001", + "reversal_of_cost_event_id": event.id, + "occurred_at": now + timedelta(minutes=1), + } + ) + reversal, _ = service.record_cost("tenant-a", reversal_payload) + assert reversal.cost_amount == Decimal("-20.0000") + assert reversal.reporting_amount == Decimal("-20.0000") + with pytest.raises(CommercialConflictError, match="已经冲回"): + service.record_cost( + "tenant-a", + reversal_payload.model_copy(update={"idempotency_key": "cost-reversal-002"}), + ) + + +def test_commercial_analytics_separates_charge_cost_savings_and_currency(db: Session) -> None: + now = datetime.now(UTC) + _, subscription, _ = _seed_commercial_account(db, "tenant-a", now) + CommercialMeteringService(db).record_cost( + "tenant-a", + _cost_payload(subscription.id, now, "CNY", Decimal("20"), "cost-cny"), + ) + CommercialMeteringService(db).record_cost( + "tenant-a", + _cost_payload(subscription.id, now, "USD", Decimal("5"), "cost-usd"), + ) + _seed_verified_savings(db, "tenant-a", now, Decimal("200"), "CNY") + db.commit() + + result = CommercialAnalyticsService(db).build( + "tenant-a", + start=now - timedelta(days=2), + end=now + timedelta(days=2), + as_of=now + timedelta(days=1), + ) + assert result.customer_charges.status == "partial" + assert _money(result.customer_charges) == {"CNY": Decimal("100.0000")} + assert _money(result.internal_costs) == { + "CNY": Decimal("20.0000"), + "USD": Decimal("5.0000"), + } + assert _money(result.contribution_margin) == {"CNY": Decimal("80.0000")} + assert result.contribution_margin.status == "partial" + assert result.verified_cash_savings.status == "available" + assert _money(result.verified_cash_savings) == {"CNY": Decimal("200.0000")} + assert result.customer_roi.ratios[0].currency == "CNY" + assert result.customer_roi.ratios[0].ratio == Decimal("1.000000") + assert result.customer_roi.ratios[0].numerator == Decimal("100.0000") + assert result.customer_labor_value.status == "unavailable" + assert any("币种" in note for note in result.contribution_margin.notes) + + unavailable = CommercialAnalyticsService(db).build( + "tenant-b", + start=now - timedelta(days=2), + end=now + timedelta(days=2), + as_of=now + timedelta(days=1), + ) + assert unavailable.customer_charges.status == "unavailable" + assert unavailable.customer_charges.values == [] + assert unavailable.internal_costs.status == "unavailable" + assert unavailable.customer_roi.status == "unavailable" + + +def test_commercial_write_contract_rejects_ambiguous_naive_timestamps() -> None: + with pytest.raises(ValueError, match="时区"): + UsageMeterEventCreate( + subscription_id=str(uuid.uuid4()), + entitlement_id=str(uuid.uuid4()), + quantity=Decimal("1"), + occurred_at=datetime(2026, 7, 16, 12, 0), + source_system="ambiguous-clock", + idempotency_key="naive-time", + ) + + +def test_subscription_lifecycle_and_history_queries_are_operable(db: Session) -> None: + now = datetime.now(UTC) + plan, subscription, entitlement = _seed_commercial_account(db, "tenant-a", now) + admin = CommercialAdminService(db) + suspended = admin.transition_subscription( + "tenant-a", + subscription.id, + CommercialSubscriptionTransition( + expected_version=subscription.version, + target_status="suspended", + reason="客户要求暂时停用自动续费与用量消费", + ), + actor_id="platform-admin", + ) + assert suspended.status == "suspended" + assert suspended.metadata_json["status_history"][-1]["to"] == "suspended" + resumed = admin.activate_subscription( + "tenant-a", + subscription.id, + expected_version=suspended.version, + ) + assert resumed.status == "active" + canceled = admin.transition_subscription( + "tenant-a", + subscription.id, + CommercialSubscriptionTransition( + expected_version=resumed.version, + target_status="canceled", + reason="试点合同到期且客户确认不再续约", + ), + actor_id="platform-admin", + ) + assert canceled.status == "canceled" + assert canceled.canceled_at is not None + assert canceled.ends_at is not None + with pytest.raises(CommercialConflictError, match="不能从"): + admin.transition_subscription( + "tenant-a", + subscription.id, + CommercialSubscriptionTransition( + expected_version=canceled.version, + target_status="suspended", + reason="终态不可恢复", + ), + actor_id="platform-admin", + ) + + query = CommercialQueryService(db) + assert [row.id for row in query.list_plans("tenant-a")] == [plan.id] + assert [row.id for row in query.list_subscriptions("tenant-a")] == [subscription.id] + assert [row.id for row in query.list_entitlements("tenant-a")] == [entitlement.id] + assert query.list_plans("tenant-b") == [] + with pytest.raises(ValueError, match="时区"): + query.list_usage_events( + "tenant-a", + start=datetime(2026, 7, 16, 0, 0), + ) + + +def test_pricing_scenario_uses_cost_floor_and_verified_value_ceiling(db: Session) -> None: + now = datetime.now(UTC) + _, subscription, _ = _seed_commercial_account(db, "tenant-a", now) + CommercialMeteringService(db).record_cost( + "tenant-a", + _cost_payload(subscription.id, now, "CNY", Decimal("20"), "pricing-cost"), + ) + _seed_verified_savings(db, "tenant-a", now, Decimal("200"), "CNY") + db.commit() + + result = CommercialPricingService(db).build( + "tenant-a", + CommercialPricingScenarioWrite( + start=now - timedelta(days=2), + end=now + timedelta(days=2), + as_of=now + timedelta(days=1), + target_contribution_margin_rate=Decimal("0.5"), + max_verified_savings_share=Decimal("0.25"), + ), + ) + assert result.recommended_model == "hybrid" + assert result.evidence_status == "complete" + scenario = result.scenarios[0] + assert scenario.status == "feasible" + assert scenario.minimum_sustainable_charge == Decimal("40.0000") + assert scenario.maximum_value_aligned_charge == Decimal("50.0000") + assert scenario.maximum_success_fee == Decimal("10.0000") + assert scenario.customer_roi_at_minimum_charge == Decimal("4.000000") + assert scenario.contribution_margin_at_value_ceiling == Decimal("0.600000") + + conflict = CommercialPricingService(db).build( + "tenant-a", + CommercialPricingScenarioWrite( + start=now - timedelta(days=2), + end=now + timedelta(days=2), + as_of=now + timedelta(days=1), + target_contribution_margin_rate=Decimal("0.5"), + max_verified_savings_share=Decimal("0.05"), + ), + ) + assert conflict.recommended_model == "optimize_unit_economics" + assert conflict.scenarios[0].status == "insufficient_value" + + +def _seed_commercial_account(db: Session, tenant_id: str, now: datetime): + admin = CommercialAdminService(db) + plan = admin.create_plan( + tenant_id, + CommercialPlanCreate( + plan_code="growth", + name="成长版", + pricing_model="subscription", + billing_interval="monthly", + currency="CNY", + base_fee=Decimal("100"), + included_seats=10, + effective_from=now - timedelta(days=30), + ), + actor_id="platform-admin", + ) + admin.activate_plan(tenant_id, plan.id, expected_version=plan.version) + subscription = admin.create_subscription( + tenant_id, + CommercialSubscriptionCreate( + subscription_key=f"{tenant_id}-2026", + plan_id=plan.id, + starts_at=now - timedelta(days=10), + current_period_start=now - timedelta(days=1), + current_period_end=now + timedelta(days=29), + seats=5, + ), + actor_id="platform-admin", + ) + entitlement = admin.upsert_entitlement( + tenant_id, + CommercialEntitlementUpsert( + subscription_id=subscription.id, + entitlement_key="ai_precheck", + metric_key="ai_precheck_runs", + entitlement_type="metered", + unit="run", + included_quantity=Decimal("5"), + hard_limit_quantity=Decimal("6"), + reset_interval="monthly", + overage_policy="block", + effective_from=now - timedelta(days=10), + ), + ) + db.flush() + return plan, subscription, entitlement + + +def _cost_payload( + subscription_id: str, + now: datetime, + currency: str, + amount: Decimal, + key: str, +) -> CommercialCostEventCreate: + return CommercialCostEventCreate( + subscription_id=subscription_id, + cost_category="infrastructure", + quantity=Decimal("1"), + unit="allocation", + unit_cost=amount, + original_currency=currency, + reporting_currency=currency, + fx_rate=Decimal("1"), + allocation_key=key, + occurred_at=now, + source_system="cost-ledger", + idempotency_key=key, + ) + + +def _seed_verified_savings( + db: Session, + tenant_id: str, + now: datetime, + amount: Decimal, + currency: str, +) -> None: + operator = _user("finance-operator", tenant_id=tenant_id, roles=["finance"]) + confirmer = _user("finance-confirmer", tenant_id=tenant_id, roles=["finance"]) + original = Decimal("1000") + claim = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no=f"BX-{uuid.uuid4().hex[:12]}", + employee_name="测试员工", + department_name="销售部", + project_code="COMMERCIAL-ROI", + expense_type="hotel", + reason="客户现场差旅", + location="上海", + amount=original, + currency=currency, + invoice_count=1, + occurred_at=now, + status="draft", + risk_flags_json=[], + ) + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim=claim, + item_date=date.today(), + item_type="hotel", + item_reason="住宿", + item_location="上海", + item_note="", + item_amount=original, + ) + db.add(claim) + db.flush() + opportunity = SavingsDiscoveryService(db).discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[ + { + "item_id": item.id, + "message": "商业 ROI 测试政策差额", + "original_amount": str(original), + "reimbursable_amount": str(original - amount), + "employee_absorbed_amount": str(amount), + "policy_rule_version": "commercial-roi-v1", + "policy_grade": "P6", + "policy_matched_city": "上海", + "calculation_fingerprint": "sha256:" + uuid.uuid4().hex * 2, + } + ], + current_user=operator, + request_id=f"discover-{uuid.uuid4().hex[:8]}", + )[0] + realization = ( + SavingsRealizationService(db) + .record( + opportunity.id, + SavingsRealizationCreate( + request_id=f"record-{uuid.uuid4().hex[:8]}", + expected_version=1, + comment="登记可追溯实际节省", + actual_gross=amount, + incremental_cost=Decimal("0"), + currency=currency, + realized_at=now, + attribution_method="server_policy_counterfactual", + attribution_ratio=Decimal("1"), + evidence_level="business_state", + evidence=[ + SavingsEvidenceCreate( + evidence_key=f"payment-{uuid.uuid4().hex}", + evidence_role="payment_business_state", + resource_type="business_event", + resource_id=f"payment-{uuid.uuid4().hex}", + source_system="x-financial", + content_hash="c" * 64, + occurred_at=now, + verification_status="unverified", + ) + ], + ), + operator, + ) + .response.realization + ) + SavingsRealizationService(db).execute_action( + realization.id, + SavingsRealizationActionCreate( + action="confirm", + request_id=f"confirm-{uuid.uuid4().hex[:8]}", + expected_version=1, + comment="独立财务确认商业 ROI 事实", + ), + confirmer, + ) + + +def _user( + username: str, + *, + tenant_id: str, + roles: list[str] | None = None, + is_admin: bool = False, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=is_admin, + tenant_id=tenant_id, + employee_id=username, + ) + + +def _money(metric) -> dict[str, Decimal]: + return {item.currency: item.amount for item in metric.values} diff --git a/server/tests/test_digital_employee_dashboard_service.py b/server/tests/test_digital_employee_dashboard_service.py index d0e0cc9..b05e63a 100644 --- a/server/tests/test_digital_employee_dashboard_service.py +++ b/server/tests/test_digital_employee_dashboard_service.py @@ -34,7 +34,11 @@ def test_digital_employee_dashboard_aggregates_daily_work_from_agent_runs() -> N source="schedule", user_id="system", status="succeeded", - route_json={"task_code": "task.hermes.global_risk_scan"}, + route_json={ + "tenant_id": "default", + "task_code": "task.hermes.global_risk_scan", + }, + ontology_json={"tenant_id": "default"}, result_summary="财务风险图谱巡检完成。", started_at=now - timedelta(hours=4), finished_at=now - timedelta(hours=4, minutes=-3), @@ -60,7 +64,11 @@ def test_digital_employee_dashboard_aggregates_daily_work_from_agent_runs() -> N source="schedule", user_id="system", status="failed", - route_json={"report_type": "risk_clue_collect"}, + route_json={ + "tenant_id": "default", + "report_type": "risk_clue_collect", + }, + ontology_json={"tenant_id": "default"}, result_summary="风险线索归集失败。", started_at=now - timedelta(hours=3), finished_at=now - timedelta(hours=3, minutes=-1), @@ -89,9 +97,11 @@ def test_digital_employee_dashboard_aggregates_daily_work_from_agent_runs() -> N user_id="admin", status="running", route_json={ + "tenant_id": "default", "job_type": "knowledge_index_sync", "requested_document_ids": ["doc-1", "doc-2"], }, + ontology_json={"tenant_id": "default"}, result_summary="知识归纳任务已入队。", started_at=now - timedelta(hours=1), ), @@ -170,7 +180,11 @@ def test_digital_employee_dashboard_counts_finance_dashboard_snapshots() -> None source="schedule", user_id="digital_employee", status="succeeded", - route_json={"task_type": "finance_dashboard_snapshot"}, + route_json={ + "tenant_id": "default", + "task_type": "finance_dashboard_snapshot", + }, + ontology_json={"tenant_id": "default"}, result_summary="finance dashboard snapshot generated", started_at=now - timedelta(minutes=3), finished_at=now - timedelta(minutes=2), @@ -214,7 +228,11 @@ def test_digital_employee_dashboard_counts_reminder_outputs() -> None: source="schedule", user_id="digital_employee", status="succeeded", - route_json={"task_type": "digital_employee_reminder_scan"}, + route_json={ + "tenant_id": "default", + "task_type": "digital_employee_reminder_scan", + }, + ontology_json={"tenant_id": "default"}, result_summary="reminder scan generated", started_at=now - timedelta(minutes=3), finished_at=now - timedelta(minutes=2), diff --git a/server/tests/test_digital_employee_reminder_task.py b/server/tests/test_digital_employee_reminder_task.py index 86e614a..5c68e59 100644 --- a/server/tests/test_digital_employee_reminder_task.py +++ b/server/tests/test_digital_employee_reminder_task.py @@ -32,7 +32,10 @@ def test_digital_employee_reminder_task_generates_actionable_report() -> None: with build_session() as db: _seed_reminder_data(db, now) - result = DigitalEmployeeReminderTaskService(db).refresh_reminders(now=now) + result = DigitalEmployeeReminderTaskService( + db, + tenant_id="default", + ).refresh_reminders(now=now) summary = result["summary"] report = result["report"] @@ -56,7 +59,7 @@ def test_digital_employee_reminder_task_generates_actionable_report() -> None: "reimbursement_overdue", }.issubset(reminder_types) - dashboard = DigitalEmployeeDashboardService(db).build_dashboard(days=7) + dashboard = DigitalEmployeeDashboardService(db).build_dashboard(days=7, now=now) assert dashboard.totals["reminders"] >= 4 assert dashboard.totals["businessOutputs"] >= 4 assert dashboard.task_distribution[0]["taskType"] == "digital_employee_reminder_scan" diff --git a/server/tests/test_employee_behavior_profile_service.py b/server/tests/test_employee_behavior_profile_service.py index f9176b5..7b65261 100644 --- a/server/tests/test_employee_behavior_profile_service.py +++ b/server/tests/test_employee_behavior_profile_service.py @@ -105,6 +105,8 @@ def seed_profile_data(db: Session) -> None: agent="hermes", source="user_message", user_id=employee.email, + route_json={"tenant_id": "default"}, + ontology_json={"tenant_id": "default"}, status="success", result_summary="AI 已辅助生成报销说明。", started_at=now - timedelta(days=2), @@ -269,7 +271,11 @@ def test_latest_profile_endpoint_returns_approval_payload() -> None: "window_days": 90, "expense_type_scope": "travel", }, - headers={"x-auth-username": "auditor", "x-auth-name": "auditor"}, + headers={ + "x-auth-username": "auditor", + "x-auth-name": "auditor", + "x-auth-is-admin": "true", + }, ) assert response.status_code == 200 @@ -353,6 +359,8 @@ def test_current_admin_profile_endpoint_returns_account_usage_profile() -> None: agent="user_agent", source="user_message", user_id="admin", + route_json={"tenant_id": "default"}, + ontology_json={"tenant_id": "default"}, status="success", result_summary="管理员查看运行概览。", started_at=started_at, @@ -482,6 +490,7 @@ def test_finish_session_endpoint_closes_active_session() -> None: client = TestClient(app) response = client.post( "/api/v1/auth/sessions/session-active-finish/finish", + headers={"X-Auth-Username": "zhangsan@example.com"}, json={ "reason": "manual", "lastActivityAt": datetime.now(UTC).isoformat(), @@ -500,6 +509,51 @@ def test_finish_session_endpoint_closes_active_session() -> None: assert session.activity_event_count == 5 +def test_finish_session_endpoint_does_not_close_another_users_session() -> None: + session_factory = build_session_factory() + login_at = datetime.now(UTC) - timedelta(minutes=9) + with session_factory() as db: + db.add( + UserSessionMetric( + session_id="session-owned-by-zhangsan", + username="zhangsan@example.com", + display_name="张三", + employee_no="E1001", + email="zhangsan@example.com", + login_at=login_at, + last_activity_at=login_at, + status="active", + ) + ) + db.commit() + + app = create_app() + install_legacy_header_auth_override(app) + + def override_db() -> Generator[Session, None, None]: + db = session_factory() + try: + yield db + finally: + db.close() + + app.dependency_overrides[get_db] = override_db + response = TestClient(app).post( + "/api/v1/auth/sessions/session-owned-by-zhangsan/finish", + headers={"X-Auth-Username": "lisi@example.com"}, + json={"reason": "manual", "activityEventCount": 1}, + ) + + assert response.status_code == 200 + assert response.json()["durationMs"] == 0 + with session_factory() as db: + session = db.query(UserSessionMetric).filter_by( + session_id="session-owned-by-zhangsan" + ).one() + assert session.status == "active" + assert session.logout_at is None + + def test_hermes_scheduler_parses_weekly_profile_cron() -> None: scheduler = HermesScheduler() diff --git a/server/tests/test_employee_service.py b/server/tests/test_employee_service.py index 15fd7a3..cce2e93 100644 --- a/server/tests/test_employee_service.py +++ b/server/tests/test_employee_service.py @@ -32,7 +32,7 @@ def build_session() -> Session: def test_employee_directory_seeds_rich_employee_data() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employees = service.list_employees() meta = service.get_employee_meta() @@ -59,7 +59,7 @@ def test_employee_directory_seeds_rich_employee_data() -> None: def test_employee_detail_contains_department_and_roles() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] detail = service.get_employee(employee.id) @@ -72,7 +72,7 @@ def test_employee_detail_contains_department_and_roles() -> None: def test_update_employee_persists_changes_and_hashes_password() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] updated = service.update_employee( @@ -115,7 +115,7 @@ def test_update_employee_persists_changes_and_hashes_password() -> None: def test_disable_employee_marks_status_and_logs_change() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = next(item for item in service.list_employees() if item.status != "停用") updated = service.disable_employee(employee.id) @@ -131,7 +131,7 @@ def test_disable_employee_marks_status_and_logs_change() -> None: def test_enable_employee_restores_status_and_logs_change() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = next(item for item in service.list_employees() if item.status != "停用") service.disable_employee(employee.id) @@ -148,7 +148,7 @@ def test_enable_employee_restores_status_and_logs_change() -> None: def test_profile_repairs_do_not_run_on_every_list() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] updated = service.update_employee( @@ -163,7 +163,7 @@ def test_profile_repairs_do_not_run_on_every_list() -> None: def test_role_update_appends_recent_history() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] current_codes = list(employee.roleCodes) next_codes = ["finance", "user"] if "finance" not in current_codes else ["user"] @@ -175,7 +175,7 @@ def test_role_update_appends_recent_history() -> None: def test_employee_change_logs_keep_only_latest_five() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] persisted = db.get(Employee, employee.id) assert persisted is not None @@ -198,7 +198,7 @@ def test_employee_change_logs_keep_only_latest_five() -> None: def test_employee_meta_includes_organization_options() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") meta = service.get_employee_meta() assert meta.organizationOptions @@ -215,7 +215,7 @@ def test_employee_meta_includes_organization_options() -> None: def test_employee_directory_normalizes_legacy_departments() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") service.list_employees() legacy_department = OrganizationUnit( @@ -239,11 +239,17 @@ def test_employee_directory_normalizes_legacy_departments() -> None: assert refreshed.organization is not None assert refreshed.organization.code == "TECH-DEPT" assert "RND-CENTER" not in {item.code for item in meta.organizationOptions} + persisted = db.execute( + select(Employee).where(Employee.employee_no == "E11745") + ).scalar_one() + assert persisted.tenant_id == "default" + assert persisted.organization_unit is not None + assert persisted.organization_unit.unit_code == "TECH-DEPT" def test_update_employee_changes_organization() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] organizations = service.repository.list_organization_units() current_code = employee.organization.code if employee.organization else None @@ -266,7 +272,7 @@ def test_update_employee_changes_organization() -> None: def test_update_employee_rejects_unknown_organization() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] with pytest.raises(ValueError, match="部门编码"): @@ -278,7 +284,7 @@ def test_update_employee_rejects_unknown_organization() -> None: def test_update_employee_changes_manager() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employees = service.list_employees() employee = employees[0] manager = next(item for item in employees if item.id != employee.id) @@ -302,7 +308,7 @@ def test_format_history_datetime_uses_local_timezone_without_seconds() -> None: def test_update_employee_rejects_invalid_date_format() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] with pytest.raises(ValueError, match="出生日期格式必须为 YYYY-MM-DD。"): diff --git a/server/tests/test_employee_spreadsheet_import.py b/server/tests/test_employee_spreadsheet_import.py index 3432ab8..23e1a36 100644 --- a/server/tests/test_employee_spreadsheet_import.py +++ b/server/tests/test_employee_spreadsheet_import.py @@ -2,6 +2,7 @@ from __future__ import annotations from io import BytesIO +import pytest from openpyxl import Workbook from sqlalchemy import create_engine, select from sqlalchemy.orm import Session, sessionmaker @@ -39,7 +40,7 @@ def build_workbook_bytes(rows: list[list[object]]) -> bytes: def test_import_employees_rejects_invalid_row_without_writing() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") first = service.list_employees()[0] content = build_workbook_bytes( @@ -80,7 +81,7 @@ def test_import_employees_rejects_invalid_row_without_writing() -> None: def test_import_employees_updates_existing_employee() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") employee = service.list_employees()[0] new_name = f"{employee.name}-导入" @@ -121,11 +122,14 @@ def test_import_employees_updates_existing_employee() -> None: assert updated.bankAccountName == "导入户名" assert updated.bankName == "招商银行上海分行" assert updated.bankAccountNo == "622588000000000002" + persisted = db.execute(select(Employee).where(Employee.id == employee.id)).scalar_one() + assert persisted.tenant_id == "default" + assert persisted.manager_id is None def test_import_employees_creates_new_employee() -> None: with build_session() as db: - service = EmployeeService(db) + service = EmployeeService(db, tenant_id="default") service.list_employees() content = build_workbook_bytes( @@ -158,11 +162,50 @@ def test_import_employees_creates_new_employee() -> None: assert result.success is True assert result.summary.created == 1 - imported = db.execute( - select(Employee).where(Employee.employee_no == "E90001") - ).scalar_one() + imported = db.execute(select(Employee).where(Employee.employee_no == "E90001")).scalar_one() assert imported.name == "导入新员工" assert imported.email == "import.new.user@xfinance.com" assert imported.bank_account_name == "导入新员工" assert imported.bank_name assert imported.bank_account_no + + +def test_import_employees_rolls_back_when_membership_sync_fails(monkeypatch) -> None: + with build_session() as db: + service = EmployeeService(db, tenant_id="default") + service.list_employees() + content = build_workbook_bytes( + [ + [ + "E-ROLLBACK-001", + "导入回滚员工", + "import.rollback@xfinance.com", + "女", + "", + "", + "", + "上海", + "业务专员", + "P3", + "FINANCE-DEPT", + "", + "", + "CC-ROLLBACK", + "", + "", + "", + "在职", + "user", + ] + ] + ) + monkeypatch.setattr( + service, + "_ensure_tenant_memberships", + lambda: (_ for _ in ()).throw(RuntimeError("membership sync failed")), + ) + + with pytest.raises(RuntimeError, match="membership sync failed"): + service.import_employees(content) + + assert db.scalar(select(Employee).where(Employee.employee_no == "E-ROLLBACK-001")) is None diff --git a/server/tests/test_expense_application_memory.py b/server/tests/test_expense_application_memory.py index c12f8d2..358adb1 100644 --- a/server/tests/test_expense_application_memory.py +++ b/server/tests/test_expense_application_memory.py @@ -14,6 +14,7 @@ from app.api.deps import CurrentUserContext, get_db from app.main import create_app from app.models.ai_learning import AIDecision, AIDecisionFeedback, WorkflowOutcome from app.models.ai_memory import MemoryEntry, MemoryEvidenceLink +from app.models.employee import Employee from app.models.expense_case import BusinessEvent, ExpenseCase from app.models.financial_record import ExpenseClaim from app.schemas.expense_application_preview import ExpenseApplicationPreviewDecisionCreate @@ -345,9 +346,10 @@ def test_invalidated_feedback_and_reversed_outcome_remove_active_evidence() -> N reversed_outcome.outcome_status = "reversed" db.flush() - assert ExpenseApplicationMemoryService(db).apply_active_transport_memory( - {}, current_user - ) == [] + assert ( + ExpenseApplicationMemoryService(db).apply_active_transport_memory({}, current_user) + == [] + ) db.refresh(entry) assert entry.status == "suppressed" assert entry.evidence_count == 1 @@ -369,9 +371,10 @@ def test_opposite_evidence_immediately_suppresses_old_active_memory() -> None: assert old_entry.status == "suppressed" assert old_entry.suppressed_at is not None assert new_entry.status == "candidate" - assert ExpenseApplicationMemoryService(db).apply_active_transport_memory( - {}, current_user - ) == [] + assert ( + ExpenseApplicationMemoryService(db).apply_active_transport_memory({}, current_user) + == [] + ) def test_non_whitelisted_correction_suppresses_old_active_without_storing_value() -> None: @@ -392,9 +395,10 @@ def test_non_whitelisted_correction_suppresses_old_active_without_storing_value( assert old_entry.status == "suppressed" assert old_entry.suppressed_at is not None assert len(list(db.scalars(select(MemoryEntry)).all())) == entry_count - assert ExpenseApplicationMemoryService(db).apply_active_transport_memory( - {}, current_user - ) == [] + assert ( + ExpenseApplicationMemoryService(db).apply_active_transport_memory({}, current_user) + == [] + ) def test_expiry_sets_expired_at_and_replay_does_not_extend_ttl() -> None: @@ -428,9 +432,7 @@ def test_expiry_sets_expired_at_and_replay_does_not_extend_ttl() -> None: entry.created_at = old_created_at entry.candidate_expires_at = datetime.now(UTC) - timedelta(days=1) db.flush() - memories = ExpenseApplicationMemoryService(db).list_current_user_memories( - current_user - ) + memories = ExpenseApplicationMemoryService(db).list_current_user_memories(current_user) db.refresh(entry) assert memories.items[0].status == "expired" assert entry.status == "expired" @@ -472,10 +474,13 @@ def test_replay_after_revoke_or_suppression_never_recreates_memory() -> None: ) claim = db.get(ExpenseClaim, decision.expense_claim_id) assert claim is not None - assert ExpenseApplicationMemoryService(db).revoke_current_user_memory( - revoked_entry.id, - current_user, - ) is not None + assert ( + ExpenseApplicationMemoryService(db).revoke_current_user_memory( + revoked_entry.id, + current_user, + ) + is not None + ) entry_count = len(list(db.scalars(select(MemoryEntry)).all())) revoked_replay = ExpenseApplicationMemoryService(db).record_transport_edit_evidence( @@ -515,9 +520,7 @@ def test_replay_after_revoke_or_suppression_never_recreates_memory() -> None: assert suppressed_entry.status == "suppressed" entry_count = len(list(db.scalars(select(MemoryEntry)).all())) - suppressed_replay = ExpenseApplicationMemoryService( - db - ).record_transport_edit_evidence( + suppressed_replay = ExpenseApplicationMemoryService(db).record_transport_edit_evidence( current_user=current_user, decision=decision, feedback=feedback, @@ -537,13 +540,14 @@ def test_memory_isolated_by_tenant_and_owner_and_revoke_clears_value() -> None: service = ExpenseApplicationMemoryService(db) assert service.list_current_user_memories(_user(employee_id="employee-other")).items == [] - assert service.list_current_user_memories( - _user(tenant_id="tenant-other") - ).items == [] - assert service.revoke_current_user_memory( - entry.id, - _user(employee_id="employee-other"), - ) is None + assert service.list_current_user_memories(_user(tenant_id="tenant-other")).items == [] + assert ( + service.revoke_current_user_memory( + entry.id, + _user(employee_id="employee-other"), + ) + is None + ) revoked = service.revoke_current_user_memory(entry.id, owner) assert revoked is not None @@ -576,10 +580,7 @@ def test_memory_rejects_same_tenant_evidence_from_another_owner() -> None: transport_mode="火车", ) assert ( - ExpenseApplicationMemoryService(db) - .list_current_user_memories(other_owner) - .items - == [] + ExpenseApplicationMemoryService(db).list_current_user_memories(other_owner).items == [] ) @@ -638,13 +639,29 @@ def test_memory_read_failure_degrades_without_mutating_facts( def test_preview_and_orchestrator_apply_active_memory() -> None: with build_in_memory_session_factory()() as db: current_user = _user() + manager = Employee( + id="employee-memory-manager", + tenant_id=current_user.tenant_id, + employee_no="E-MEMORY-MANAGER", + name="记忆测试经理", + email="employee-memory-manager@example.com", + ) + employee = Employee( + id=current_user.employee_id, + tenant_id=current_user.tenant_id, + employee_no=current_user.employee_no, + name=current_user.name, + email=current_user.username, + manager=manager, + ) + db.add_all([manager, employee]) + db.flush() _seed_active_memory(db, current_user=current_user) preview = ExpenseApplicationPreviewWorkflow(db).issue( ExpenseApplicationPreviewDecisionCreate( message=( - "申请时间：2026-07-20 至 2026-07-22\n" - "地点：上海\n事由：客户现场实施\n天数：3天" + "申请时间：2026-07-20 至 2026-07-22\n地点：上海\n事由：客户现场实施\n天数：3天" ), conversation_id="conversation-memory-preview", request_id="request-memory-preview", @@ -685,9 +702,7 @@ def test_preview_and_orchestrator_apply_active_memory() -> None: ) assert outcome is not None assert outcome.result["application_preview"]["fields"]["transportMode"] == "火车" - assert outcome.result["application_preview"]["memoryApplications"][0][ - "status" - ] == "applied" + assert outcome.result["application_preview"]["memoryApplications"][0]["status"] == "applied" assert "补充出行方式" not in str(outcome.result) edited_fields = dict(preview.application_preview["fields"]) @@ -749,10 +764,13 @@ def test_memory_api_enforces_owner_and_allows_owner_revoke() -> None: "/api/v1/expense-application-memories/me", headers=other_headers, ).json() == {"items": []} - assert client.delete( - f"/api/v1/expense-application-memories/{memory_id}", - headers=other_headers, - ).status_code == 404 + assert ( + client.delete( + f"/api/v1/expense-application-memories/{memory_id}", + headers=other_headers, + ).status_code + == 404 + ) response = client.delete( f"/api/v1/expense-application-memories/{memory_id}", diff --git a/server/tests/test_expense_application_preview_decisions.py b/server/tests/test_expense_application_preview_decisions.py index a7fb8b9..282d995 100644 --- a/server/tests/test_expense_application_preview_decisions.py +++ b/server/tests/test_expense_application_preview_decisions.py @@ -275,6 +275,32 @@ def test_server_preview_decision_detects_field_added_after_issuance() -> None: assert {item["field_key"] for item in feedback.changed_fields_json} >= {"transport_mode"} +def test_server_preview_decision_records_cleared_suggestion_as_rejected() -> None: + client, session_factory = build_client() + with session_factory() as db: + seed_employee(db) + + issued = issue_preview(client, request_id="issue-preview-rejected-field") + payload = build_action_payload(issued) + payload["message"] = payload["message"].replace("出行方式：飞机\n", "") + payload["context_json"]["application_preview"]["fields"]["transportMode"] = "" + response = client.post( + "/api/v1/reimbursements/application-preview-action", + headers=auth_headers(), + json=payload, + ) + + assert response.status_code == 200, response.text + with session_factory() as db: + feedback = db.scalar(select(AIDecisionFeedback)) + assert feedback is not None + assert feedback.feedback_type == "rejected" + assert feedback.verification_status == "server_verified" + assert {item["field_key"] for item in feedback.changed_fields_json} == { + "transport_mode" + } + + def test_server_preview_decision_rejects_cross_session_replay_without_writes() -> None: client, session_factory = build_client() with session_factory() as db: diff --git a/server/tests/test_expense_case_service.py b/server/tests/test_expense_case_service.py index d5dd9af..0b2825c 100644 --- a/server/tests/test_expense_case_service.py +++ b/server/tests/test_expense_case_service.py @@ -25,6 +25,7 @@ from app.services.expense_claim_workflow_constants import ( DIRECT_MANAGER_APPROVAL_STAGE, ) from app.services.expense_claims import ExpenseClaimService +from app.services.tenant_registry import TenantRegistryService def build_session() -> Session: @@ -316,12 +317,15 @@ def test_submit_blocks_fixable_pre_review_before_budget_and_submission_event( ).all() ) assert len(pre_review_events) == 1 - assert db.scalar( - select(BusinessEvent).where( - BusinessEvent.aggregate_id == claim.id, - BusinessEvent.event_type == "claim_submitted", + assert ( + db.scalar( + select(BusinessEvent).where( + BusinessEvent.aggregate_id == claim.id, + BusinessEvent.event_type == "claim_submitted", + ) ) - ) is None + is None + ) assert db.scalar(select(BudgetReservation)) is None assert db.scalar(select(BudgetTransaction)) is None @@ -400,12 +404,15 @@ def test_submit_rechecks_dynamic_risk_context_and_rejects_stale_ready_review( assert error_info.value.review["review_id"] != ready_flag["review_id"] assert error_info.value.review["decision"] == "needs_fix" assert db.get(ExpenseClaim, claim.id).status == "draft" - assert db.scalar( - select(BusinessEvent).where( - BusinessEvent.aggregate_id == claim.id, - BusinessEvent.event_type == "claim_submitted", + assert ( + db.scalar( + select(BusinessEvent).where( + BusinessEvent.aggregate_id == claim.id, + BusinessEvent.event_type == "claim_submitted", + ) ) - ) is None + is None + ) pre_review_events = list( db.scalars( select(BusinessEvent).where( @@ -429,6 +436,7 @@ def test_legacy_bootstrap_excludes_migration_owned_tables( monkeypatch.setattr(service, "_seed_agent_assets", lambda: None) monkeypatch.setattr(service, "_sync_demo_financial_records", lambda: None) monkeypatch.setattr(service, "_seed_runs_and_logs", lambda: None) + monkeypatch.setattr(TenantRegistryService, "ensure_builtin", lambda self: None) service._prepare_foundation() @@ -473,6 +481,7 @@ def test_event_write_is_idempotent_for_same_business_operation() -> None: def test_submit_claim_creates_case_link_and_structured_event() -> None: current_user = CurrentUserContext( + tenant_id="default", username="employee-case@example.com", name="张三", role_codes=[], @@ -536,6 +545,7 @@ def test_payment_event_failure_rolls_back_payment_archive_and_nested_audit( monkeypatch: pytest.MonkeyPatch, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-case@example.com", name="财务付款", role_codes=["finance"], @@ -639,6 +649,7 @@ def test_application_approval_links_generated_reimbursement_to_same_case() -> No approved = ExpenseClaimService(db).approve_claim( application_claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -680,6 +691,7 @@ def test_application_approval_links_generated_reimbursement_to_same_case() -> No def test_payment_records_application_archive_event_in_same_case() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-archive@example.com", name="财务付款", role_codes=["finance"], diff --git a/server/tests/test_expense_claim_action_protocol.py b/server/tests/test_expense_claim_action_protocol.py index ff8db3a..98d8f74 100644 --- a/server/tests/test_expense_claim_action_protocol.py +++ b/server/tests/test_expense_claim_action_protocol.py @@ -25,6 +25,7 @@ from app.services.expense_claims import ExpenseClaimService def _manager_user() -> CurrentUserContext: return CurrentUserContext( + tenant_id="default", username="manager-action@example.com", name="李经理", role_codes=["manager"], @@ -34,6 +35,7 @@ def _manager_user() -> CurrentUserContext: def _finance_user() -> CurrentUserContext: return CurrentUserContext( + tenant_id="default", username="finance-action@example.com", name="王财务", role_codes=["finance"], @@ -173,9 +175,7 @@ def test_approve_replay_returns_original_snapshot_after_claim_moves_forward( assert replay is not None assert ExpenseClaimRead.model_validate(replay).model_dump(mode="json") == first_json assert replay.approval_stage == "财务审批" - assert all( - item.get("source") != "future_state" for item in replay.risk_flags_json or [] - ) + assert all(item.get("source") != "future_state" for item in replay.risk_flags_json or []) persisted = db.get(ExpenseClaim, claim.id) assert persisted is not None assert persisted.status == "pending_payment" @@ -283,6 +283,7 @@ def test_legacy_stage_repair_cannot_commit_inside_action_protocol( ] db.commit() admin_user = CurrentUserContext( + tenant_id="default", username="admin-action@example.com", name="审批管理员", role_codes=["admin"], diff --git a/server/tests/test_expense_claim_approval_routing.py b/server/tests/test_expense_claim_approval_routing.py index 60a63d1..f8725ef 100644 --- a/server/tests/test_expense_claim_approval_routing.py +++ b/server/tests/test_expense_claim_approval_routing.py @@ -82,7 +82,9 @@ def _seed_budget_allocation( db.flush() -def _seed_people(db: Session, *, suffix: str) -> tuple[OrganizationUnit, Employee, Employee, Employee]: +def _seed_people( + db: Session, *, suffix: str +) -> tuple[OrganizationUnit, Employee, Employee, Employee]: budget_role = _seed_budget_monitor_role(db) department = OrganizationUnit( unit_code=f"ROUTE-{suffix}", @@ -148,6 +150,7 @@ def test_low_risk_application_skips_budget_manager_and_generates_draft() -> None approved = ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -219,6 +222,7 @@ def test_budget_warning_application_still_skips_budget_manager_when_not_over_bud approved = ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -279,6 +283,7 @@ def test_application_routes_to_budget_manager_when_usage_reaches_90_percent() -> routed = ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -303,6 +308,7 @@ def test_application_routes_to_budget_manager_when_usage_reaches_90_percent() -> ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=budget_manager.email, name=budget_manager.name, role_codes=["budget_monitor"], @@ -355,6 +361,7 @@ def test_high_risk_application_under_90_percent_routes_to_budget_manager() -> No approved = ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -570,6 +577,7 @@ def test_application_route_ignores_reimbursement_stage_current_risks() -> None: approved = ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -633,6 +641,7 @@ def test_risky_reimbursement_routes_to_budget_then_finance() -> None: routed = service.approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -656,6 +665,7 @@ def test_risky_reimbursement_routes_to_budget_then_finance() -> None: budget_approved = service.approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=budget_manager.email, name=budget_manager.name, role_codes=["budget_monitor"], @@ -722,6 +732,7 @@ def test_resolved_high_risk_application_does_not_route_to_budget_manager() -> No routed = ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -769,6 +780,7 @@ def test_budget_manager_blank_opinion_defaults_to_agree_when_budget_under_warnin approved = ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=budget_manager.email, name=budget_manager.name, role_codes=["budget_monitor"], diff --git a/server/tests/test_expense_claim_platform_risk_stage.py b/server/tests/test_expense_claim_platform_risk_stage.py index d251e9b..0148b97 100644 --- a/server/tests/test_expense_claim_platform_risk_stage.py +++ b/server/tests/test_expense_claim_platform_risk_stage.py @@ -18,7 +18,6 @@ from app.services.expense_claims import ExpenseClaimService from app.services.risk_rule_generation_interpreter import COMPOSITE_RULE_TEMPLATE_KEY from app.test_helpers.db import build_in_memory_session - build_session = build_in_memory_session @@ -208,6 +207,7 @@ def _write_attachment_meta(storage_root, invoice_id: str, meta: dict[str, Any]) def _build_claim(*, claim_no: str, expense_type: str, status: str = "draft") -> ExpenseClaim: return ExpenseClaim( + tenant_id="default", claim_no=claim_no, employee_name="张三", department_name="研发部", @@ -283,6 +283,8 @@ def test_platform_risk_rules_are_filtered_by_business_stage_and_category( claim_no="CLM-TEST-001", expense_type="travel", ) + db.add_all([application_claim, reimbursement_claim]) + db.commit() application_review = service.evaluate_platform_risk_rules( application_claim, @@ -326,6 +328,7 @@ def test_expense_application_pre_review_runs_stage_rules(tmp_path, monkeypatch) db.commit() current_user = CurrentUserContext( + tenant_id="default", username="张三", name="张三", role_codes=[], @@ -338,8 +341,7 @@ def test_expense_application_pre_review_runs_stage_rules(tmp_path, monkeypatch) rule_flags = [ flag for flag in reviewed.risk_flags_json - if isinstance(flag, dict) - and flag.get("rule_code") == "application.pre.review.rule" + if isinstance(flag, dict) and flag.get("rule_code") == "application.pre.review.rule" ] assert len(rule_flags) == 1 assert rule_flags[0]["message"] == "申请预审规则命中" @@ -361,6 +363,8 @@ def test_preapproval_amount_rules_run_from_rule_library() -> None: with build_session() as db: claim = _build_claim(claim_no="RE-PREAPPROVAL-MEAL", expense_type="meal") claim.amount = Decimal("501.00") + db.add(claim) + db.commit() flags = ExpenseClaimService(db).evaluate_platform_risk_rules( claim, @@ -430,8 +434,7 @@ def test_reimbursement_item_sync_persists_rule_center_risk_preview( rule_flags = [ flag for flag in claim.risk_flags_json - if isinstance(flag, dict) - and flag.get("rule_code") == "reimbursement.preview.rule" + if isinstance(flag, dict) and flag.get("rule_code") == "reimbursement.preview.rule" ] assert len(rule_flags) == 1 assert rule_flags[0]["message"] == "报销风险预判命中" diff --git a/server/tests/test_expense_claim_risk_gate.py b/server/tests/test_expense_claim_risk_gate.py index 5ae06fe..2c961d6 100644 --- a/server/tests/test_expense_claim_risk_gate.py +++ b/server/tests/test_expense_claim_risk_gate.py @@ -180,6 +180,7 @@ def test_blocked_approval_rolls_back_action_ledger_and_claim_mutation() -> None: ExpenseClaimService(db).approve_claim( claim.id, CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], diff --git a/server/tests/test_expense_claim_service.py b/server/tests/test_expense_claim_service.py index 414b808..cea7c9f 100644 --- a/server/tests/test_expense_claim_service.py +++ b/server/tests/test_expense_claim_service.py @@ -1,12 +1,13 @@ from __future__ import annotations import re +import threading import uuid from datetime import UTC, date, datetime, timedelta from decimal import Decimal import pytest -from sqlalchemy import create_engine, or_ +from sqlalchemy import create_engine, or_, select from sqlalchemy.orm import Session, sessionmaker from sqlalchemy.pool import StaticPool @@ -18,6 +19,12 @@ from app.models.employee import Employee from app.models.financial_record import ExpenseClaim, ExpenseClaimItem from app.models.organization import OrganizationUnit from app.models.role import Role +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) from app.schemas.ocr import OcrRecognizeBatchRead, OcrRecognizeDocumentRead from app.schemas.ontology import OntologyParseRequest from app.schemas.reimbursement import ( @@ -43,6 +50,7 @@ from app.services.expense_claims import ExpenseClaimService from app.services.ocr import OcrService from app.services.ontology import SemanticOntologyService from app.services.receipt_folder import ReceiptFolderService +from app.services.savings_discovery import SavingsDiscoveryService def build_claim(*, expense_type: str, location: str) -> ExpenseClaim: @@ -202,6 +210,32 @@ def _seed_budget_allocation( return allocation +def _seed_submitter_with_manager( + db: Session, + *, + employee_id: str, + email: str, + name: str = "张三", +) -> Employee: + manager = Employee( + tenant_id="default", + employee_no=f"{employee_id}-manager", + name="李经理", + email=f"{employee_id}-manager@example.com", + ) + employee = Employee( + id=employee_id, + tenant_id="default", + employee_no=f"{employee_id}-owner", + name=name, + email=email, + manager=manager, + ) + db.add_all([manager, employee]) + db.flush() + return employee + + def _seed_budget_monitor_role(db: Session) -> Role: role = db.query(Role).filter(Role.role_code == "budget_monitor").one_or_none() if role is not None: @@ -379,6 +413,7 @@ def test_save_or_submit_preview_does_not_create_claim_without_explicit_action() message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "预览员工", "user_input_text": message, }, @@ -418,6 +453,7 @@ def test_save_or_submit_persists_claim_only_after_save_draft_action() -> None: message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "保存员工", "user_input_text": message, "review_action": "save_draft", @@ -455,6 +491,7 @@ def test_save_draft_persists_user_changed_expense_category() -> None: message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "分类员工", "user_input_text": message, "review_action": "save_draft", @@ -506,6 +543,7 @@ def test_upsert_draft_from_ontology_persists_linked_application_context() -> Non message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "关联员工", "user_input_text": message, "review_action": "save_draft", @@ -542,7 +580,8 @@ def test_upsert_linked_application_draft_without_receipts_has_no_placeholder_ite user_id = "linked-application-no-receipt@example.com" message = ( "报销类型：差旅费\n" - "关联申请单：AP-202606-001 / 支撑国网仿生产服务器部署 / 2026-02-20 至 2026-02-23 / 上海 / ¥3,000\n" + "关联申请单：AP-202606-001 / 支撑国网仿生产服务器部署 / " + "2026-02-20 至 2026-02-23 / 上海 / ¥3,000\n" "报销票据：草稿生成后在详情中上传" ) @@ -576,6 +615,7 @@ def test_upsert_linked_application_draft_without_receipts_has_no_placeholder_ite message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "关联员工", "user_input_text": message, "review_action": "save_draft", @@ -705,6 +745,7 @@ def test_upsert_linked_application_draft_clears_existing_placeholder_item() -> N message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "关联员工", "draft_claim_id": existing_claim.id, "user_input_text": message, @@ -765,6 +806,7 @@ def test_upsert_linked_application_requires_approved_application() -> None: message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "Linked Employee", "user_input_text": message, "review_action": "save_draft", @@ -837,6 +879,7 @@ def test_upsert_linked_application_rejects_duplicate_reimbursement_draft() -> No message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "Linked Employee", "user_input_text": message, "review_action": "save_draft", @@ -980,13 +1023,14 @@ def test_unsaved_conversation_expires_after_retention_but_saved_conversation_sta conversation_id="conv-unsaved-expire", user_id="expire@example.com", source="user_message", - context_json={"session_type": "expense"}, + context_json={"tenant_id": "default", "session_type": "expense"}, ) saved = service.get_or_create_conversation( conversation_id="conv-saved-keep", user_id="expire@example.com", source="user_message", context_json={ + "tenant_id": "default", "session_type": "expense", "draft_claim_id": "claim-saved", }, @@ -1077,6 +1121,7 @@ def test_upsert_draft_from_ontology_defers_multi_document_association_choice() - message="我上传了两张交通票据，帮我生成报销草稿", ontology=ontology, context_json={ + "tenant_id": "default", "name": "张三", "attachment_names": ["didi-trip.png", "parking-ticket.jpg"], "attachment_count": 2, @@ -1147,6 +1192,7 @@ def test_linked_document_supplement_keeps_existing_claim_expense_type() -> None: db.commit() context_json = { + "tenant_id": "default", "name": "类型锁定员工", "review_action": "link_to_existing_draft", "draft_claim_id": existing_claim.id, @@ -1209,9 +1255,13 @@ def test_upsert_draft_from_ontology_keeps_reason_missing_for_attachment_only_upl result = service.upsert_draft_from_ontology( run_id=ontology.run_id, user_id=user_id, - message="我上传了 1 份票据，请结合附件名称给出报销建议并尽量生成草稿。\n附件名称：didi-trip.png", + message=( + "我上传了 1 份票据，请结合附件名称给出报销建议并尽量生成草稿。\n" + "附件名称：didi-trip.png" + ), ontology=ontology, context_json={ + "tenant_id": "default", "name": "王五", "user_input_text": "", "attachment_names": ["didi-trip.png"], @@ -1258,6 +1308,7 @@ def test_upsert_draft_from_ontology_strips_recognized_business_time_from_reason( message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "赵六", "user_input_text": message, }, @@ -1325,6 +1376,7 @@ def test_upsert_draft_from_ontology_supports_link_or_create_for_multi_documents( ) service = ExpenseClaimService(db) context_json = { + "tenant_id": "default", "name": "李四", "attachment_names": ["didi-trip.png", "parking-ticket.jpg"], "attachment_count": 2, @@ -1357,6 +1409,7 @@ def test_upsert_draft_from_ontology_supports_link_or_create_for_multi_documents( message="把这两张票据关联到已有草稿", ontology=ontology, context_json={ + "tenant_id": "default", **context_json, "review_action": "link_to_existing_draft", }, @@ -1374,6 +1427,7 @@ def test_upsert_draft_from_ontology_supports_link_or_create_for_multi_documents( message="单独新建一张报销单", ontology=ontology, context_json={ + "tenant_id": "default", **context_json, "review_action": "create_new_claim_from_documents", }, @@ -1430,6 +1484,7 @@ def test_link_existing_draft_blocks_duplicate_uploaded_invoice() -> None: db.commit() context_json = { + "tenant_id": "default", "name": "重复票据员工", "review_action": "link_to_existing_draft", "draft_claim_id": existing_claim.id, @@ -1497,6 +1552,7 @@ def test_upsert_travel_draft_uses_ticket_item_types_and_auto_allowance() -> None message="我去北京出差 3 天，上传了火车票，帮我生成差旅费报销草稿", ontology=ontology, context_json={ + "tenant_id": "default", "name": "差旅员工", "grade": "P4", "attachment_names": ["train-ticket.png"], @@ -1550,6 +1606,7 @@ def test_upsert_travel_draft_uses_ticket_item_types_and_auto_allowance() -> None item_id=allowance_item.id, payload=ExpenseClaimItemUpdate(item_amount=Decimal("1.00")), current_user=CurrentUserContext( + tenant_id="default", username=user_id, name="差旅员工", role_codes=[], @@ -1560,7 +1617,10 @@ def test_upsert_travel_draft_uses_ticket_item_types_and_auto_allowance() -> None def test_upsert_travel_draft_uses_explicit_text_days_for_allowance() -> None: user_id = "travel-explicit-days@example.com" - message = "业务发生时间:2026-05-20 至 2026-05-23，去上海支撑上海电力服务器部署，出差3天，申请差旅费报销" + message = ( + "业务发生时间:2026-05-20 至 2026-05-23，" + "去上海支撑上海电力服务器部署，出差3天，申请差旅费报销" + ) with build_session() as db: employee = Employee( @@ -1585,6 +1645,7 @@ def test_upsert_travel_draft_uses_explicit_text_days_for_allowance() -> None: message=message, ontology=ontology, context_json={ + "tenant_id": "default", "name": "文本差旅员工", "grade": "P4", "user_input_text": message, @@ -1661,6 +1722,7 @@ def test_sync_travel_claim_adds_allowance_from_manual_ticket_dates() -> None: def test_update_claim_item_allows_placeholder_date_reason_and_amount() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -1759,6 +1821,7 @@ def test_upsert_draft_from_ontology_updates_returned_claim_and_preserves_return_ message="我补充了交通票据，更新这张退回单据", ontology=ontology, context_json={ + "tenant_id": "default", "name": "赵六", "draft_claim_id": existing_claim.id, "attachment_names": ["new-trip.png"], @@ -1882,6 +1945,7 @@ def test_upsert_draft_from_ontology_retries_claim_no_conflict() -> None: message="帮我生成报销草稿，我昨天交通费 13.4 元", ontology=ontology, context_json={ + "tenant_id": "default", "name": "赵六", "user_input_text": "帮我生成报销草稿，我昨天交通费 13.4 元", }, @@ -1895,6 +1959,7 @@ def test_upsert_draft_from_ontology_retries_claim_no_conflict() -> None: def test_create_claim_item_adds_blank_draft_row_without_forcing_attachment() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -1929,6 +1994,7 @@ def test_create_claim_item_adds_blank_draft_row_without_forcing_attachment() -> def test_update_claim_reason_only_allows_draft_pending_submission() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -1965,6 +2031,7 @@ def test_update_claim_reason_only_allows_draft_pending_submission() -> None: def test_update_claim_item_reanalyzes_existing_attachment(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -2053,6 +2120,7 @@ def test_update_claim_item_reanalyzes_existing_attachment(monkeypatch, tmp_path) def test_upload_attachment_refreshes_claim_pre_review(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="submitter", role_codes=[], @@ -2125,6 +2193,7 @@ def test_upload_attachment_refreshes_claim_pre_review(monkeypatch, tmp_path) -> def test_upload_train_ticket_attachment_backfills_item_amount(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -2224,6 +2293,7 @@ def test_upload_auto_collected_attachment_uses_source_receipt_ocr_result( get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="auto-collect-travel@example.com", name="张三", role_codes=[], @@ -2375,6 +2445,7 @@ def test_upload_attachment_response_includes_refreshed_rule_center_risk_flags( tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -2467,6 +2538,7 @@ def test_upload_attachment_runs_rule_center_city_risk_from_origin_destination_fi tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -2541,6 +2613,7 @@ def test_upload_attachment_uses_linked_application_business_time_for_date_risk( tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -2616,6 +2689,7 @@ def test_upload_attachment_uses_linked_application_business_time_for_date_risk( def test_upload_hotel_attachment_audits_date_like_amount(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -2701,6 +2775,7 @@ def test_upload_hotel_attachment_audits_date_like_amount(monkeypatch, tmp_path) def test_upload_hotel_attachment_flags_amount_over_travel_policy(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-hotel-risk@example.com", name="张三", role_codes=[], @@ -2805,6 +2880,7 @@ def test_upload_hotel_attachment_does_not_add_generic_auto_review_summary( tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-hotel-summary@example.com", name="张三", role_codes=[], @@ -2902,6 +2978,7 @@ def test_delete_claim_item_attachment_removes_attachment_analysis_risk( monkeypatch, tmp_path ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-hotel-risk@example.com", name="张三", role_codes=[], @@ -3093,6 +3170,7 @@ def test_attachment_risk_flag_message_uses_specific_points(monkeypatch, tmp_path def test_upload_ride_receipt_backfills_item_reason_from_addresses(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -3162,6 +3240,7 @@ def test_upload_ride_receipt_backfills_item_reason_from_addresses(monkeypatch, t def test_delete_claim_item_removes_row_and_attachment_files(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -3232,6 +3311,7 @@ def test_delete_claim_item_removes_row_and_attachment_files(monkeypatch, tmp_pat def test_delete_claim_removes_all_claim_attachment_files(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="admin", name="张三", role_codes=["admin"], @@ -3256,6 +3336,7 @@ def test_delete_claim_removes_all_claim_attachment_files(monkeypatch, tmp_path) user_id=current_user.username, source="user_message", context_json={ + "tenant_id": "default", "session_type": "expense", "draft_claim_id": claim.id, }, @@ -3273,6 +3354,7 @@ def test_delete_claim_removes_all_claim_attachment_files(monkeypatch, tmp_path) def test_applicant_can_delete_own_editable_draft_claim(monkeypatch, tmp_path) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -3296,6 +3378,7 @@ def test_attachment_preview_resolves_legacy_filename_in_claim_item_directory( monkeypatch, tmp_path ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -3336,6 +3419,7 @@ def test_attachment_pdf_preview_falls_back_to_source_when_render_fonts_missing( monkeypatch, tmp_path ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-1", name="张三", role_codes=[], @@ -3402,6 +3486,7 @@ def test_attachment_pdf_preview_falls_back_to_source_when_render_fonts_missing( def test_submit_claim_runs_ai_review_and_routes_to_direct_manager() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-submit@example.com", name="张三", role_codes=[], @@ -3437,6 +3522,7 @@ def test_submit_claim_runs_ai_review_and_routes_to_direct_manager() -> None: def test_submit_claim_refreshes_legacy_pre_review_without_fingerprint(monkeypatch) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-submit@example.com", name="submitter", role_codes=[], @@ -3504,6 +3590,7 @@ def test_submit_claim_refreshes_legacy_pre_review_without_fingerprint(monkeypatc def test_accept_standard_adjustment_recalculates_claim_amount_and_preserves_on_submit() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-standard@example.com", name="张三", role_codes=[], @@ -3561,6 +3648,28 @@ def test_accept_standard_adjustment_recalculates_claim_amount_and_preserves_on_s assert standard_flag["reimbursable_amount"] == "450.00" assert standard_flag["employee_absorbed_amount"] == "430.00" assert standard_flag["visibility_scope"] == "leader" + opportunity = db.scalar( + select(SavingsOpportunity).where(SavingsOpportunity.claim_id == claim.id) + ) + assert opportunity is not None + assert opportunity.status == "in_progress" + assert opportunity.baseline_amount == Decimal("880.0000") + assert opportunity.target_amount == Decimal("450.0000") + assert opportunity.estimated_net == Decimal("430.0000") + assert opportunity.discovery_business_event_id + baseline = db.get(ProfileBaselineSnapshot, opportunity.baseline_snapshot_id) + assert baseline is not None + assert baseline.policy_version == standard_flag["policy_rule_version"] + evidence_links = list( + db.scalars( + select(SavingsEvidenceLink).where( + SavingsEvidenceLink.tenant_id == "default", + SavingsEvidenceLink.entity_id.in_([baseline.id, opportunity.id]), + ) + ).all() + ) + assert len(evidence_links) == 2 + assert all(item.verification_status == "verified" for item in evidence_links) submitted = service.submit_claim(claim.id, current_user) @@ -3573,8 +3682,9 @@ def test_accept_standard_adjustment_recalculates_claim_amount_and_preserves_on_s ) -def test_accept_standard_adjustment_uses_policy_amount_when_payload_has_no_downgrade() -> None: +def test_accept_standard_adjustment_ignores_client_amount_and_days_snapshots() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-policy-standard@example.com", name="张三", role_codes=[], @@ -3615,9 +3725,9 @@ def test_accept_standard_adjustment_uses_policy_amount_when_payload_has_no_downg "item_id": claim.items[0].id, "title": "住宿超标待说明", "risk": "住宿票据金额超过职级标准。", - "application_days": 2, - "original_amount": Decimal("1000.00"), - "reimbursable_amount": Decimal("1000.00"), + "application_days": 99, + "original_amount": Decimal("1.00"), + "reimbursable_amount": Decimal("999999.00"), } ] ), @@ -3625,20 +3735,313 @@ def test_accept_standard_adjustment_uses_policy_amount_when_payload_has_no_downg ) assert adjusted is not None - assert adjusted.amount == Decimal("900.00") + assert adjusted.amount == Decimal("450.00") + assert adjusted.items[0].item_amount == Decimal("1000.00") standard_flag = next( flag for flag in adjusted.risk_flags_json if isinstance(flag, dict) and flag.get("source") == "reimbursement_standard_adjustment" ) assert standard_flag["original_amount"] == "1000.00" - assert standard_flag["reimbursable_amount"] == "900.00" - assert standard_flag["employee_absorbed_amount"] == "100.00" + assert standard_flag["reimbursable_amount"] == "450.00" + assert standard_flag["employee_absorbed_amount"] == "550.00" + assert standard_flag["calculation_source"] == "server_policy" + assert standard_flag["policy_days"] == 1 + assert standard_flag["policy_location"] == "北京市" + assert standard_flag["policy_matched_city"] == "北京" + assert standard_flag["policy_grade"] == "P4" + assert standard_flag["policy_hotel_rate"] == "450.00" + assert standard_flag["policy_hotel_amount"] == "450.00" + assert standard_flag["policy_rule_name"] + assert standard_flag["policy_rule_version"] + assert standard_flag["calculation_fingerprint"].startswith("sha256:") assert standard_flag["visibility_scope"] == "leader" +def test_accept_standard_adjustment_fails_closed_without_server_policy() -> None: + current_user = CurrentUserContext( + tenant_id="default", + username="emp-policy-missing@example.com", + name="张三", + role_codes=[], + is_admin=False, + grade="P4", + ) + + with build_session() as db: + employee = Employee( + employee_no="E7034", + name="张三", + email="emp-policy-missing@example.com", + grade="P4", + ) + claim = build_claim(expense_type="hotel", location="待补充") + claim.employee = employee + claim.employee_id = employee.id + claim.amount = Decimal("1000.00") + claim.location = "待补充" + claim.items[0].item_type = "hotel_ticket" + claim.items[0].item_reason = "住宿" + claim.items[0].item_location = "待补充" + claim.items[0].item_amount = Decimal("1000.00") + db.add_all([employee, claim]) + db.commit() + + with pytest.raises(ValueError, match="暂未取得服务端可验证的报销标准"): + ExpenseClaimService(db).accept_standard_adjustment( + claim_id=claim.id, + payload=ExpenseClaimStandardAdjustmentPayload( + risks=[ + { + "item_id": claim.items[0].id, + "application_days": 2, + "original_amount": Decimal("1.00"), + "reimbursable_amount": Decimal("1.00"), + } + ] + ), + current_user=current_user, + ) + + db.refresh(claim) + assert claim.amount == Decimal("1000.00") + assert claim.items[0].item_amount == Decimal("1000.00") + assert not any( + isinstance(flag, dict) and flag.get("source") == "reimbursement_standard_adjustment" + for flag in claim.risk_flags_json + ) + + +def test_accept_standard_adjustment_replays_same_request_without_rewriting_snapshot() -> None: + current_user = CurrentUserContext( + tenant_id="default", + username="emp-standard-replay@example.com", + name="张三", + role_codes=[], + is_admin=False, + grade="P4", + ) + + with build_session() as db: + employee = Employee( + employee_no="E7035", + name="张三", + email="emp-standard-replay@example.com", + grade="P4", + ) + claim = build_claim(expense_type="hotel", location="北京") + claim.employee = employee + claim.employee_id = employee.id + claim.amount = Decimal("1000.00") + claim.items[0].item_type = "hotel_ticket" + claim.items[0].item_reason = "北京住宿" + claim.items[0].item_location = "北京" + claim.items[0].item_amount = Decimal("1000.00") + db.add_all([employee, claim]) + db.commit() + + payload = ExpenseClaimStandardAdjustmentPayload( + request_id="standard-adjustment-replay-1", + expected_updated_at=claim.updated_at, + risks=[{"item_id": claim.items[0].id, "risk_id": "risk-replay-1"}], + ) + service = ExpenseClaimService(db) + first = service.accept_standard_adjustment( + claim_id=claim.id, + payload=payload, + current_user=current_user, + ) + assert first is not None + first_flag = next( + flag + for flag in first.risk_flags_json + if isinstance(flag, dict) and flag.get("source") == "reimbursement_standard_adjustment" + ) + first_created_at = first_flag["created_at"] + first_calculation_fingerprint = first_flag["calculation_fingerprint"] + + replayed = service.accept_standard_adjustment( + claim_id=claim.id, + payload=payload, + current_user=current_user, + ) + + assert replayed is not None + replayed_flags = [ + flag + for flag in replayed.risk_flags_json + if isinstance(flag, dict) and flag.get("source") == "reimbursement_standard_adjustment" + ] + assert len(replayed_flags) == 1 + assert replayed_flags[0]["created_at"] == first_created_at + assert replayed_flags[0]["calculation_fingerprint"] == first_calculation_fingerprint + assert replayed.amount == Decimal("450.00") + + +def test_standard_adjustment_fallback_lock_serializes_same_tenant_claim() -> None: + first_service = ExpenseClaimService(build_session()) + second_service = ExpenseClaimService(build_session()) + current_user = CurrentUserContext( + username="emp-standard-lock@example.com", + name="张三", + role_codes=[], + is_admin=False, + tenant_id="tenant-standard-lock", + ) + contender_started = threading.Event() + contender_acquired = threading.Event() + + def contend_for_lock() -> None: + contender_started.set() + with second_service._serialize_standard_adjustment( + claim_id="claim-standard-lock", + current_user=current_user, + ): + contender_acquired.set() + + try: + with first_service._serialize_standard_adjustment( + claim_id="claim-standard-lock", + current_user=current_user, + ): + contender = threading.Thread(target=contend_for_lock) + contender.start() + assert contender_started.wait(timeout=1) + assert not contender_acquired.wait(timeout=0.05) + + contender.join(timeout=1) + assert not contender.is_alive() + assert contender_acquired.is_set() + finally: + first_service.db.close() + second_service.db.close() + + +def test_accept_standard_adjustment_rejects_stale_claim_version() -> None: + current_user = CurrentUserContext( + tenant_id="default", + username="emp-standard-stale@example.com", + name="张三", + role_codes=[], + is_admin=False, + grade="P4", + ) + + with build_session() as db: + employee = Employee( + employee_no="E7036", + name="张三", + email="emp-standard-stale@example.com", + grade="P4", + ) + claim = build_claim(expense_type="hotel", location="北京") + claim.employee = employee + claim.employee_id = employee.id + claim.items[0].item_type = "hotel_ticket" + claim.items[0].item_location = "北京" + claim.items[0].item_amount = Decimal("1000.00") + db.add_all([employee, claim]) + db.commit() + + with pytest.raises(ValueError, match="已被其他操作更新"): + ExpenseClaimService(db).accept_standard_adjustment( + claim_id=claim.id, + payload=ExpenseClaimStandardAdjustmentPayload( + request_id="standard-adjustment-stale-1", + expected_updated_at=datetime(2020, 1, 1, tzinfo=UTC), + risks=[{"item_id": claim.items[0].id}], + ), + current_user=current_user, + ) + + +def test_accept_standard_adjustment_preserves_other_item_snapshot() -> None: + current_user = CurrentUserContext( + tenant_id="default", + username="emp-standard-multi@example.com", + name="张三", + role_codes=[], + is_admin=False, + grade="P4", + ) + + with build_session() as db: + employee = Employee( + employee_no="E7037", + name="张三", + email="emp-standard-multi@example.com", + grade="P4", + ) + claim = build_claim(expense_type="hotel", location="北京") + claim.employee = employee + claim.employee_id = employee.id + first_item = claim.items[0] + first_item.item_type = "hotel_ticket" + first_item.item_reason = "北京住宿第一晚" + first_item.item_location = "北京" + first_item.item_amount = Decimal("1000.00") + second_item = ExpenseClaimItem( + id="item-standard-second", + claim_id=claim.id, + item_date=date(2026, 5, 14), + item_type="hotel_ticket", + item_reason="北京住宿第二晚", + item_location="北京", + item_amount=Decimal("800.00"), + invoice_id="invoice-standard-second", + ) + claim.items.append(second_item) + claim.amount = Decimal("1800.00") + db.add_all([employee, claim]) + db.commit() + + service = ExpenseClaimService(db) + first = service.accept_standard_adjustment( + claim_id=claim.id, + payload=ExpenseClaimStandardAdjustmentPayload( + request_id="standard-adjustment-multi-1", + risks=[{"item_id": first_item.id}], + ), + current_user=current_user, + ) + assert first is not None + assert first.amount == Decimal("1250.00") + + with pytest.raises(ValueError, match="request_id 已用于另一组"): + service.accept_standard_adjustment( + claim_id=claim.id, + payload=ExpenseClaimStandardAdjustmentPayload( + request_id="standard-adjustment-multi-1", + risks=[{"item_id": second_item.id}], + ), + current_user=current_user, + ) + + second = service.accept_standard_adjustment( + claim_id=claim.id, + payload=ExpenseClaimStandardAdjustmentPayload( + request_id="standard-adjustment-multi-2", + risks=[{"item_id": second_item.id}], + ), + current_user=current_user, + ) + + assert second is not None + standard_flags = [ + flag + for flag in second.risk_flags_json + if isinstance(flag, dict) and flag.get("source") == "reimbursement_standard_adjustment" + ] + assert {flag["item_id"] for flag in standard_flags} == { + first_item.id, + second_item.id, + } + assert second.amount == Decimal("900.00") + + def test_pre_review_claim_records_ai_result_without_submitting() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-pre-review@example.com", name="张三", role_codes=[], @@ -3689,6 +4092,7 @@ def test_pre_review_claim_records_ai_result_without_submitting() -> None: def test_submit_claim_allows_returned_claim_to_be_resubmitted() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-submit@example.com", name="张三", role_codes=[], @@ -3726,6 +4130,7 @@ def test_submit_claim_allows_returned_claim_to_be_resubmitted() -> None: def test_submit_claim_backfills_department_from_current_employee() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-dept@example.com", name="张三", role_codes=[], @@ -3773,6 +4178,7 @@ def test_submit_claim_blocks_high_risk_attachment_until_submitter_fixes_it( tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-risk@example.com", name="张三", role_codes=[], @@ -3853,6 +4259,7 @@ def test_submit_claim_blocks_travel_route_mismatch_until_submitter_explains_it( tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-travel@example.com", name="张三", role_codes=[], @@ -3992,7 +4399,10 @@ def test_submit_claim_blocks_travel_route_mismatch_until_submitter_explains_it( "rule_code": "risk.travel.medium.multi_city_no_reason", "severity": "medium", "label": "多城市行程缺少说明中风险", - "message": "本次报销识别到多城市行程（上海、武汉、成都），但事由中未说明中转、多地拜访或改签原因。", + "message": ( + "本次报销识别到多城市行程（上海、武汉、成都），" + "但事由中未说明中转、多地拜访或改签原因。" + ), "item_ids": ["travel-item-2"], "business_stage": "reimbursement", } @@ -4027,6 +4437,7 @@ def test_submit_claim_allows_round_trip_ticket_origin_inferred_from_route( tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-round-trip@example.com", name="张三", role_codes=[], @@ -4175,6 +4586,7 @@ def test_submit_claim_blocks_hotel_amount_over_policy_until_standard_adjustment( tmp_path, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-hotel@example.com", name="张三", role_codes=[], @@ -4324,6 +4736,7 @@ def test_submit_claim_blocks_hotel_amount_over_policy_until_standard_adjustment( def test_list_claims_scopes_to_current_user_id_even_when_names_duplicate() -> None: current_user = CurrentUserContext( + tenant_id="default", username="zhangsan1@example.com", name="张三", role_codes=["manager"], @@ -4393,6 +4806,7 @@ def test_list_claims_scopes_to_current_user_id_even_when_names_duplicate() -> No def test_list_claims_resolves_short_username_to_unique_employee_email_prefix() -> None: current_user = CurrentUserContext( + tenant_id="default", username="caoxiaozhu", name="caoxiaozhu", role_codes=["employee"], @@ -4437,6 +4851,7 @@ def test_list_claims_resolves_short_username_to_unique_employee_email_prefix() - def test_list_claims_limits_finance_to_personal_records() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance@example.com", name="财务", role_codes=["finance"], @@ -4492,6 +4907,7 @@ def test_list_claims_limits_finance_to_personal_records() -> None: def test_list_claims_returns_company_reimbursements_for_finance_document_center() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance@example.com", name="财务", role_codes=["finance"], @@ -4569,6 +4985,7 @@ def test_list_claims_returns_company_reimbursements_for_finance_document_center( def test_list_claims_returns_all_active_documents_for_admin_document_center() -> None: current_user = CurrentUserContext( + tenant_id="default", username="admin", name="admin", role_codes=["admin"], @@ -4664,6 +5081,7 @@ def test_list_claims_returns_all_active_documents_for_admin_document_center() -> def test_list_claims_limits_executive_to_personal_records() -> None: current_user = CurrentUserContext( + tenant_id="default", username="executive@example.com", name="高管", role_codes=["executive"], @@ -4719,6 +5137,7 @@ def test_list_claims_limits_executive_to_personal_records() -> None: def test_list_claims_keeps_own_archived_claim_for_finance_applicant() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance@example.com", name="财务", role_codes=["finance"], @@ -4755,6 +5174,7 @@ def test_list_claims_keeps_own_archived_claim_for_finance_applicant() -> None: def test_list_archived_claims_returns_company_archived_records_for_finance() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance@example.com", name="财务", role_codes=["finance"], @@ -4881,6 +5301,7 @@ def test_list_archived_claims_returns_company_archived_records_for_finance() -> def test_list_archived_claims_returns_only_own_records_for_regular_employee() -> None: current_user = CurrentUserContext( + tenant_id="default", username="zhangsan@example.com", name="张三", role_codes=["employee"], @@ -4935,6 +5356,7 @@ def test_list_archived_claims_returns_only_own_records_for_regular_employee() -> def test_finance_can_return_but_cannot_delete_submitted_claim() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance@example.com", name="财务", role_codes=["finance"], @@ -4984,6 +5406,7 @@ def test_finance_can_return_but_cannot_delete_submitted_claim() -> None: def test_executive_cannot_delete_submitted_claim_without_admin_role() -> None: current_user = CurrentUserContext( + tenant_id="default", username="executive-delete@example.com", name="高管", role_codes=["executive"], @@ -5020,6 +5443,7 @@ def test_executive_cannot_delete_submitted_claim_without_admin_role() -> None: def test_direct_manager_cannot_delete_application_claim() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager-delete-application@example.com", name="李经理", role_codes=["manager"], @@ -5070,6 +5494,7 @@ def test_direct_manager_cannot_delete_application_claim() -> None: def test_applicant_can_delete_returned_application_claim() -> None: current_user = CurrentUserContext( + tenant_id="default", username="zhangsan-application-return-delete@example.com", name="张三", role_codes=[], @@ -5121,6 +5546,7 @@ def test_applicant_can_delete_returned_application_claim() -> None: def test_admin_can_delete_application_claim() -> None: current_user = CurrentUserContext( + tenant_id="default", username="superadmin", name="系统管理员", role_codes=["manager"], @@ -5158,6 +5584,7 @@ def test_admin_can_delete_application_claim() -> None: def test_executive_cannot_delete_archived_claim() -> None: current_user = CurrentUserContext( + tenant_id="default", username="executive-archive-delete@example.com", name="高管", role_codes=["executive"], @@ -5194,6 +5621,7 @@ def test_executive_cannot_delete_archived_claim() -> None: def test_admin_can_delete_archived_claim() -> None: current_user = CurrentUserContext( + tenant_id="default", username="superadmin", name="系统管理员", role_codes=["manager"], @@ -5234,12 +5662,14 @@ def test_admin_delete_claim_unlinks_receipt_folder_items(monkeypatch, tmp_path) get_settings.cache_clear() try: receipt_owner = CurrentUserContext( + tenant_id="default", username="emp-1", name="Employee", role_codes=[], is_admin=False, ) admin_user = CurrentUserContext( + tenant_id="default", username="superadmin", name="Admin", role_codes=["manager"], @@ -5290,6 +5720,7 @@ def test_admin_delete_claim_unlinks_receipt_folder_items(monkeypatch, tmp_path) def test_admin_delete_linked_reimbursement_resets_application_link_status() -> None: admin_user = CurrentUserContext( + tenant_id="default", username="superadmin", name="系统管理员", role_codes=["admin"], @@ -5390,6 +5821,7 @@ def test_admin_delete_linked_reimbursement_resets_application_link_status() -> N def test_direct_manager_can_return_subordinate_claim_to_pending_submission() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager-return@example.com", name="李经理", role_codes=["manager"], @@ -5450,6 +5882,7 @@ def test_direct_manager_can_return_subordinate_claim_to_pending_submission() -> def test_direct_manager_can_approve_subordinate_claim_to_finance_review() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager-approve@example.com", name="李经理", role_codes=["manager"], @@ -5515,6 +5948,7 @@ def test_direct_manager_can_approve_subordinate_claim_to_finance_review() -> Non def test_manager_cannot_operate_own_claim_submitted_to_direct_manager() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager-own-approval@example.com", name="李经理", role_codes=["manager"], @@ -5558,10 +5992,10 @@ def test_manager_cannot_operate_own_claim_submitted_to_direct_manager() -> None: claim_id = claim.id service = ExpenseClaimService(db) - with pytest.raises(ValueError, match="当前直属领导审批人"): + with pytest.raises(LookupError, match="Approval task not found"): service.approve_claim(claim_id, current_user, opinion="同意") - with pytest.raises(ValueError, match="当前审批人"): + with pytest.raises(LookupError, match="Approval task not found"): service.return_claim(claim_id, current_user, reason="退回") db.refresh(claim) @@ -5572,6 +6006,7 @@ def test_manager_cannot_operate_own_claim_submitted_to_direct_manager() -> None: def test_direct_manager_budget_monitor_routes_reimbursement_directly_to_finance() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager-budget-monitor-reimbursement@example.com", name="李预算经理", role_codes=["manager", "budget_monitor", "executive"], @@ -5663,6 +6098,7 @@ def test_direct_manager_budget_monitor_routes_reimbursement_directly_to_finance( def test_legacy_duplicate_budget_stage_is_not_mutated_by_read() -> None: admin_user = CurrentUserContext( + tenant_id="default", username="admin", name="admin", role_codes=["admin"], @@ -5747,6 +6183,7 @@ def test_application_submit_skips_ai_review_and_receipt_requirements( monkeypatch: pytest.MonkeyPatch, ) -> None: current_user = CurrentUserContext( + tenant_id="default", username="application-owner@example.com", name="张三", role_codes=["employee"], @@ -5754,7 +6191,21 @@ def test_application_submit_skips_ai_review_and_receipt_requirements( ) with build_session() as db: + manager = Employee( + tenant_id="default", + employee_no="APP-SUBMIT-MANAGER", + name="李经理", + email="application-submit-manager@example.com", + ) + employee = Employee( + tenant_id="default", + employee_no="APP-SUBMIT-OWNER", + name="张三", + email=current_user.username, + manager=manager, + ) claim = ExpenseClaim( + tenant_id="default", claim_no="APP-20260525-SUBMIT", employee_name="张三", department_name="交付部", @@ -5777,7 +6228,8 @@ def test_application_submit_skips_ai_review_and_receipt_requirements( } ], ) - db.add(claim) + claim.employee = employee + db.add_all([manager, employee, claim]) db.commit() claim_id = claim.id service = ExpenseClaimService(db) @@ -5809,6 +6261,7 @@ def test_application_submit_skips_ai_review_and_receipt_requirements( def test_application_submit_reserves_budget_once() -> None: current_user = CurrentUserContext( + tenant_id="default", username="application-budget-owner@example.com", name="张三", role_codes=["employee"], @@ -5822,7 +6275,13 @@ def test_application_submit_reserves_budget_once() -> None: department_name="交付部", amount=Decimal("50000.00"), ) + _seed_submitter_with_manager( + db, + employee_id="emp-budget", + email=current_user.username, + ) claim = ExpenseClaim( + tenant_id="default", claim_no="APP-20260525-BUDGET", employee_id="emp-budget", employee_name="张三", @@ -5864,6 +6323,7 @@ def test_application_submit_reserves_budget_once() -> None: def test_application_submit_blocks_when_budget_insufficient_without_state_change() -> None: current_user = CurrentUserContext( + tenant_id="default", username="application-budget-block@example.com", name="张三", role_codes=["employee"], @@ -5910,6 +6370,7 @@ def test_application_submit_blocks_when_budget_insufficient_without_state_change def test_reimbursement_submit_keeps_budget_insufficient_as_review_risk() -> None: current_user = CurrentUserContext( + tenant_id="default", username="reimbursement-budget-risk@example.com", name="张三", role_codes=["employee"], @@ -5924,7 +6385,13 @@ def test_reimbursement_submit_keeps_budget_insufficient_as_review_risk() -> None subject_code="office", amount=Decimal("1000.00"), ) + employee = _seed_submitter_with_manager( + db, + employee_id="emp-reimbursement-budget-risk", + email=current_user.username, + ) claim = build_claim(expense_type="office", location="待补充") + claim.employee = employee claim.amount = Decimal("1200.00") claim.items[0].item_amount = Decimal("1200.00") db.add(claim) @@ -5948,6 +6415,7 @@ def test_reimbursement_submit_keeps_budget_insufficient_as_review_risk() -> None def test_application_submit_skips_budget_for_non_demo_subject() -> None: current_user = CurrentUserContext( + tenant_id="default", username="application-budget-skip@example.com", name="张三", role_codes=["employee"], @@ -5961,7 +6429,13 @@ def test_application_submit_skips_budget_for_non_demo_subject() -> None: department_name="交付部", amount=Decimal("1000.00"), ) + _seed_submitter_with_manager( + db, + employee_id="emp-budget-skip", + email=current_user.username, + ) claim = ExpenseClaim( + tenant_id="default", claim_no="APP-20260525-SKIP", employee_id="emp-budget-skip", employee_name="张三", @@ -5996,16 +6470,16 @@ def test_application_submit_skips_budget_for_non_demo_subject() -> None: ) -def test_direct_manager_can_route_application_claim_to_budget_approval_then_budget_manager_creates_draft() -> ( - None -): +def test_direct_manager_routes_application_then_budget_manager_creates_draft() -> None: manager_user = CurrentUserContext( + tenant_id="default", username="manager-application-approve@example.com", name="李经理", role_codes=["manager"], is_admin=False, ) budget_user = CurrentUserContext( + tenant_id="default", username="budget-p8-application-approve@example.com", name="赵预算", role_codes=["budget_monitor"], @@ -6125,6 +6599,7 @@ def test_direct_manager_can_route_application_claim_to_budget_approval_then_budg assert approved.approval_stage == "关联单据状态" archived_claims = ExpenseClaimService(db).list_archived_claims( CurrentUserContext( + tenant_id="default", username="finance-archive@example.com", name="财务归档员", role_codes=["finance"], @@ -6185,12 +6660,14 @@ def test_direct_manager_can_route_application_claim_to_budget_approval_then_budg def test_application_routes_to_department_p8_executive_with_approver_name() -> None: manager_user = CurrentUserContext( + tenant_id="default", username="manager-executive-route@example.com", name="Manager", role_codes=["manager"], is_admin=False, ) budget_user = CurrentUserContext( + tenant_id="default", username="p8-executive-route@example.com", name="P8 Executive", role_codes=["executive"], @@ -6293,6 +6770,7 @@ def test_application_routes_to_department_p8_executive_with_approver_name() -> N def test_direct_manager_cannot_route_application_to_missing_budget_approver() -> None: manager_user = CurrentUserContext( + tenant_id="default", username="manager-missing-budget@example.com", name="Manager", role_codes=["manager"], @@ -6368,6 +6846,7 @@ def test_direct_manager_p8_executive_completes_application_without_duplicate_bud None ): manager_user = CurrentUserContext( + tenant_id="default", username="manager-executive-merged@example.com", name="P8 Manager", role_codes=["manager"], @@ -6456,10 +6935,9 @@ def test_direct_manager_p8_executive_completes_application_without_duplicate_bud ) -def test_direct_manager_budget_monitor_completes_application_claim_without_duplicate_budget_approval() -> ( - None -): +def test_direct_manager_budget_monitor_avoids_duplicate_budget_approval() -> None: manager_user = CurrentUserContext( + tenant_id="default", username="manager-budget-monitor-application@example.com", name="李预算经理", role_codes=["manager", "budget_monitor", "executive"], @@ -6556,6 +7034,7 @@ def test_direct_manager_budget_monitor_completes_application_claim_without_dupli assert all(claim.claim_no != generated_draft.claim_no for claim in reviewer_claims) applicant_claims = ExpenseClaimService(db).list_claims( CurrentUserContext( + tenant_id="default", username="zhangsan-budget-monitor-application@example.com", name="张三", role_codes=[], @@ -6576,6 +7055,7 @@ def test_direct_manager_budget_monitor_completes_application_claim_without_dupli def test_direct_manager_return_application_claim_records_return_node_and_opinion() -> None: manager_user = CurrentUserContext( + tenant_id="default", username="manager-application-return@example.com", name="李经理", role_codes=["manager"], @@ -6659,18 +7139,21 @@ def test_direct_manager_return_application_claim_records_return_node_and_opinion def test_application_approval_transfers_budget_reservation_to_reimbursement_draft() -> None: owner = CurrentUserContext( + tenant_id="default", username="application-budget-owner-approve@example.com", name="张三", role_codes=["employee"], is_admin=False, ) manager_user = CurrentUserContext( + tenant_id="default", username="manager-application-budget@example.com", name="李经理", role_codes=["manager"], is_admin=False, ) budget_user = CurrentUserContext( + tenant_id="default", username="budget-p8-transfer@example.com", name="赵预算", role_codes=["budget_monitor"], @@ -6774,6 +7257,7 @@ def test_application_approval_transfers_budget_reservation_to_reimbursement_draf deleted = service.delete_claim( generated_draft.id, CurrentUserContext( + tenant_id="default", username="browser-session-user", name="", role_codes=["user"], @@ -6791,6 +7275,7 @@ def test_application_approval_transfers_budget_reservation_to_reimbursement_draf def test_direct_manager_approval_defaults_blank_opinion_to_agree() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager-application-required-opinion@example.com", name="李经理", role_codes=["manager"], @@ -6854,6 +7339,7 @@ def test_direct_manager_approval_defaults_blank_opinion_to_agree() -> None: def test_budget_analysis_uses_current_application_reservation_without_double_counting() -> None: owner = CurrentUserContext( + tenant_id="default", username="application-budget-analysis-owner@example.com", name="张三", role_codes=["employee"], @@ -6861,12 +7347,20 @@ def test_budget_analysis_uses_current_application_reservation_without_double_cou ) with build_session() as db: + manager = Employee( + tenant_id="default", + employee_no="E-BUDGET-ANALYSIS-MANAGER", + name="李经理", + email="application-budget-analysis-manager@example.com", + ) employee = Employee( + tenant_id="default", employee_no="E-BUDGET-ANALYSIS", name="张三", email="application-budget-analysis-owner@example.com", + manager=manager, ) - db.add(employee) + db.add_all([manager, employee]) db.flush() _seed_budget_allocation( db, @@ -6908,6 +7402,7 @@ def test_budget_analysis_uses_current_application_reservation_without_double_cou def test_finance_approve_reimbursement_consumes_budget_reservation() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-budget-approve@example.com", name="财务", role_codes=["finance"], @@ -6977,6 +7472,7 @@ def test_finance_approve_reimbursement_consumes_budget_reservation() -> None: def test_finance_cannot_operate_own_claim_in_finance_stage() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-own-approval@example.com", name="财务", role_codes=["finance"], @@ -7013,9 +7509,9 @@ def test_finance_cannot_operate_own_claim_in_finance_stage() -> None: db.commit() service = ExpenseClaimService(db) - with pytest.raises(ValueError, match="财务终审"): + with pytest.raises(ValueError, match="申请人不能审批自己的费用单"): service.approve_claim(claim.id, current_user, opinion="同意入账") - with pytest.raises(ValueError, match="可以退回"): + with pytest.raises(ValueError, match="申请人不能审批自己的费用单"): service.return_claim(claim.id, current_user, reason="退回") db.refresh(claim) @@ -7026,6 +7522,7 @@ def test_finance_cannot_operate_own_claim_in_finance_stage() -> None: def test_finance_can_approve_claim_to_pending_payment_stage() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-approve@example.com", name="财务复核", role_codes=["finance"], @@ -7077,6 +7574,7 @@ def test_finance_can_approve_claim_to_pending_payment_stage() -> None: def test_finance_can_mark_pending_payment_claim_as_paid() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-pay@example.com", name="财务付款", role_codes=["finance"], @@ -7101,9 +7599,40 @@ def test_finance_can_mark_pending_payment_claim_as_paid() -> None: approval_stage="待付款", risk_flags_json=[], ) + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim=claim, + item_date=date(2026, 5, 12), + item_type="hotel", + item_reason="上海住宿", + item_location="上海", + item_note="", + item_amount=Decimal("100.00"), + ) db.add(claim) db.commit() + opportunities = SavingsDiscoveryService(db).discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[ + { + "item_id": item.id, + "message": "付款集成测试标准调整", + "original_amount": "100.00", + "reimbursable_amount": "66.00", + "employee_absorbed_amount": "34.00", + "policy_rule_version": "payment-integration-v1", + "policy_grade": "P6", + "policy_matched_city": "上海", + "calculation_fingerprint": "sha256:" + "b" * 64, + } + ], + current_user=current_user, + request_id="payment-integration-adjustment", + ) + db.commit() + paid = ExpenseClaimService(db).mark_claim_paid(claim.id, current_user) assert paid is not None @@ -7118,10 +7647,21 @@ def test_finance_can_mark_pending_payment_claim_as_paid() -> None: and flag.get("next_approval_stage") == "已付款" for flag in paid.risk_flags_json ) + realization = db.scalar( + select(SavingsRealization).where( + SavingsRealization.opportunity_id == opportunities[0].id + ) + ) + assert realization is not None + assert realization.status == "pending_confirmation" + assert realization.actual_net == Decimal("34.0000") + db.refresh(opportunities[0]) + assert opportunities[0].status == "realized" def test_marking_linked_reimbursement_paid_archives_application_claim() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-pay-linked-application@example.com", name="财务付款", role_codes=["finance"], @@ -7210,6 +7750,7 @@ def test_marking_linked_reimbursement_paid_archives_application_claim() -> None: def test_return_claim_rejects_already_returned_claim_without_adding_event() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-returned@example.com", name="财务", role_codes=["finance"], @@ -7246,7 +7787,7 @@ def test_return_claim_rejects_already_returned_claim_without_adding_event() -> N db.commit() claim_id = claim.id - with pytest.raises(ValueError, match="无需重复退回"): + with pytest.raises(ValueError, match="没有可执行的审批任务"): ExpenseClaimService(db).return_claim(claim_id, current_user, reason="重复退回") db.refresh(claim) @@ -7260,12 +7801,14 @@ def test_return_claim_rejects_already_returned_claim_without_adding_event() -> N def test_return_claim_records_each_return_event_with_stage_reason_and_counts() -> None: manager_user = CurrentUserContext( + tenant_id="default", username="manager-return-count@example.com", name="李经理", role_codes=["manager"], is_admin=False, ) finance_user = CurrentUserContext( + tenant_id="default", username="finance-return@example.com", name="财务复核", role_codes=["finance"], @@ -7354,6 +7897,7 @@ def test_return_claim_records_each_return_event_with_stage_reason_and_counts() - def test_submit_returned_claim_preserves_manual_return_events() -> None: current_user = CurrentUserContext( + tenant_id="default", username="emp-submit-returned@example.com", name="张三", role_codes=[], @@ -7397,6 +7941,7 @@ def test_submit_returned_claim_preserves_manual_return_events() -> None: user_id=current_user.username, source="user_message", context_json={ + "tenant_id": "default", "session_type": "expense", "draft_claim_id": claim.id, }, @@ -7419,6 +7964,7 @@ def test_submit_returned_claim_preserves_manual_return_events() -> None: def test_manager_personal_claims_exclude_subordinate_pending_approval_claims() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager-personal@example.com", name="李经理", role_codes=["manager"], @@ -7491,6 +8037,7 @@ def test_manager_personal_claims_exclude_subordinate_pending_approval_claims() - def test_list_approval_claims_allows_direct_manager_to_view_pending_claims_for_approval() -> None: current_user = CurrentUserContext( + tenant_id="default", username="manager@example.com", name="李经理", role_codes=["manager"], @@ -7572,6 +8119,7 @@ def test_list_approval_claims_allows_direct_manager_to_view_pending_claims_for_a def test_list_approval_claims_limits_finance_to_finance_stage_claims() -> None: current_user = CurrentUserContext( + tenant_id="default", username="finance-approval-list@example.com", name="财务", role_codes=["finance"], @@ -7626,12 +8174,14 @@ def test_list_approval_claims_limits_finance_to_finance_stage_claims() -> None: def test_list_approval_claims_allows_budget_monitor_to_view_budget_stage_applications() -> None: current_user = CurrentUserContext( + tenant_id="default", username="budget-p8-list@example.com", name="赵预算", role_codes=["budget_monitor"], is_admin=False, ) p8_without_budget_role = CurrentUserContext( + tenant_id="default", username="p8-without-budget-list@example.com", name="budget manager", role_codes=["manager"], diff --git a/server/tests/test_expense_claim_tenant_and_case_stage.py b/server/tests/test_expense_claim_tenant_and_case_stage.py index 142529a..296a8f0 100644 --- a/server/tests/test_expense_claim_tenant_and_case_stage.py +++ b/server/tests/test_expense_claim_tenant_and_case_stage.py @@ -220,17 +220,20 @@ def test_steward_reimbursement_context_uses_current_tenant() -> None: def test_application_approval_keeps_case_at_approved_to_spend() -> None: with build_session() as db: department = OrganizationUnit( + tenant_id="tenant-case-stage", unit_code="TENANT-CASE-TRAVEL", name="租户差旅部", unit_type="department", ) manager = Employee( + tenant_id="tenant-case-stage", employee_no="TENANT-CASE-MANAGER", name="租户差旅经理", email="tenant-case-manager@example.com", organization_unit=department, ) employee = Employee( + tenant_id="tenant-case-stage", employee_no="TENANT-CASE-EMPLOYEE", name="租户差旅员工", email="tenant-case-employee@example.com", @@ -259,6 +262,7 @@ def test_application_approval_keeps_case_at_approved_to_spend() -> None: ) ) application = ExpenseClaim( + tenant_id="tenant-case-stage", claim_no="AP-TENANT-CASE-GENERATE", employee_id=employee.id, employee_name=employee.name, diff --git a/server/tests/test_expense_claim_tenant_scope.py b/server/tests/test_expense_claim_tenant_scope.py index 472bc16..359d248 100644 --- a/server/tests/test_expense_claim_tenant_scope.py +++ b/server/tests/test_expense_claim_tenant_scope.py @@ -14,13 +14,20 @@ from app.services.expense_claims import ExpenseClaimService from app.test_helpers.db import build_in_memory_session_factory -def _build_claim(*, claim_id: str, claim_no: str, employee: Employee) -> ExpenseClaim: +def _build_claim( + *, + claim_id: str, + claim_no: str, + tenant_id: str, + employee: Employee, +) -> ExpenseClaim: return ExpenseClaim( id=claim_id, + tenant_id=tenant_id, claim_no=claim_no, employee_id=employee.id, employee_name=employee.name, - department_id="tenant-scope-department", + department_id=None, department_name="租户隔离部", project_code=None, expense_type="office", @@ -37,6 +44,16 @@ def _build_claim(*, claim_id: str, claim_no: str, employee: Employee) -> Expense ) +def _build_employee(*, tenant_id: str, suffix: str, name: str, email: str) -> Employee: + return Employee( + id=f"tenant-{suffix}-employee", + tenant_id=tenant_id, + employee_no=f"TENANT-{suffix.upper()}-001", + name=name, + email=email, + ) + + def _current_user(tenant_id: str) -> CurrentUserContext: return CurrentUserContext( username="same-owner@example.com", @@ -50,28 +67,52 @@ def _current_user(tenant_id: str) -> CurrentUserContext: def test_same_identity_claims_are_isolated_by_server_tenant() -> None: session_factory = build_in_memory_session_factory() with session_factory() as db: - employee = Employee( - id="tenant-scope-employee", - employee_no="TENANT-SCOPE-001", + employee_a = _build_employee( + tenant_id="tenant-a", + suffix="scope-a", + name="同名员工", + email="same-owner@example.com", + ) + employee_b = _build_employee( + tenant_id="tenant-b", + suffix="scope-b", + name="同名员工", + email="same-owner@example.com", + ) + default_employee = _build_employee( + tenant_id="default", + suffix="scope-default", name="同名员工", email="same-owner@example.com", ) tenant_a_claim = _build_claim( claim_id="tenant-a-claim", claim_no="RE-TENANT-A", - employee=employee, + tenant_id="tenant-a", + employee=employee_a, ) tenant_b_claim = _build_claim( claim_id="tenant-b-claim", claim_no="RE-TENANT-B", - employee=employee, + tenant_id="tenant-b", + employee=employee_b, ) legacy_default_claim = _build_claim( claim_id="legacy-default-claim", claim_no="RE-LEGACY-DEFAULT", - employee=employee, + tenant_id="default", + employee=default_employee, + ) + db.add_all( + [ + employee_a, + employee_b, + default_employee, + tenant_a_claim, + tenant_b_claim, + legacy_default_claim, + ] ) - db.add_all([employee, tenant_a_claim, tenant_b_claim, legacy_default_claim]) db.flush() case_service = ExpenseCaseService(db) case_service.ensure_case_for_claim(tenant_a_claim, tenant_id="tenant-a") @@ -83,12 +124,8 @@ def test_same_identity_claims_are_isolated_by_server_tenant() -> None: tenant_b_user = _current_user("tenant-b") default_user = _current_user("default") - assert {claim.id for claim in service.list_claims(tenant_a_user)} == { - tenant_a_claim.id - } - assert {claim.id for claim in service.list_claims(tenant_b_user)} == { - tenant_b_claim.id - } + assert {claim.id for claim in service.list_claims(tenant_a_user)} == {tenant_a_claim.id} + assert {claim.id for claim in service.list_claims(tenant_b_user)} == {tenant_b_claim.id} assert {claim.id for claim in service.list_claims(default_user)} == { legacy_default_claim.id } @@ -108,21 +145,29 @@ def test_same_identity_claims_are_isolated_by_server_tenant() -> None: def _seed_cross_tenant_risk_history(db): - employee = Employee( - id="tenant-history-employee", - employee_no="TENANT-HISTORY-001", + employee_a = _build_employee( + tenant_id="tenant-a", + suffix="history-a", + name="同名风险员工", + email="same-risk-owner@example.com", + ) + employee_b = _build_employee( + tenant_id="tenant-b", + suffix="history-b", name="同名风险员工", email="same-risk-owner@example.com", ) clean_claim = _build_claim( claim_id="tenant-a-clean-claim", claim_no="RE-TENANT-A-CLEAN", - employee=employee, + tenant_id="tenant-a", + employee=employee_a, ) risky_claim = _build_claim( claim_id="tenant-b-risky-claim", claim_no="RE-TENANT-B-RISKY", - employee=employee, + tenant_id="tenant-b", + employee=employee_b, ) risky_claim.risk_flags_json = [ { @@ -132,7 +177,7 @@ def _seed_cross_tenant_risk_history(db): "message": "该风险只属于 tenant-b。", } ] - db.add_all([employee, clean_claim, risky_claim]) + db.add_all([employee_a, employee_b, clean_claim, risky_claim]) db.flush() case_service = ExpenseCaseService(db) case_service.ensure_case_for_claim(clean_claim, tenant_id="tenant-a") @@ -207,28 +252,52 @@ def test_cross_tenant_risk_history_does_not_route_to_p8(monkeypatch) -> None: def test_draft_lookup_never_returns_cross_tenant_claim_and_keeps_default_legacy() -> None: session_factory = build_in_memory_session_factory() with session_factory() as db: - employee = Employee( - id="tenant-draft-employee", - employee_no="TENANT-DRAFT-001", + employee_a = _build_employee( + tenant_id="tenant-a", + suffix="draft-a", + name="同名草稿员工", + email="same-draft-owner@example.com", + ) + employee_b = _build_employee( + tenant_id="tenant-b", + suffix="draft-b", + name="同名草稿员工", + email="same-draft-owner@example.com", + ) + default_employee = _build_employee( + tenant_id="default", + suffix="draft-default", name="同名草稿员工", email="same-draft-owner@example.com", ) tenant_a_claim = _build_claim( claim_id="tenant-a-draft", claim_no="RE-TENANT-A-DRAFT", - employee=employee, + tenant_id="tenant-a", + employee=employee_a, ) tenant_b_claim = _build_claim( claim_id="tenant-b-draft", claim_no="RE-TENANT-B-DRAFT", - employee=employee, + tenant_id="tenant-b", + employee=employee_b, ) legacy_default_claim = _build_claim( claim_id="legacy-default-draft", claim_no="RE-LEGACY-DEFAULT-DRAFT", - employee=employee, + tenant_id="default", + employee=default_employee, + ) + db.add_all( + [ + employee_a, + employee_b, + default_employee, + tenant_a_claim, + tenant_b_claim, + legacy_default_claim, + ] ) - db.add_all([employee, tenant_a_claim, tenant_b_claim, legacy_default_claim]) db.flush() case_service = ExpenseCaseService(db) case_service.ensure_case_for_claim(tenant_a_claim, tenant_id="tenant-a") @@ -272,8 +341,8 @@ def test_draft_lookup_never_returns_cross_tenant_claim_and_keeps_default_legacy( "tenant_id": "tenant-a", "draft_claim_id": tenant_b_claim.id, }, - user_id=employee.email, - employee=employee, + user_id=employee_a.email, + employee=employee_a, ) assert association_candidate is not None assert association_candidate.id == tenant_a_claim.id diff --git a/server/tests/test_expense_financial_value_chain_e2e.py b/server/tests/test_expense_financial_value_chain_e2e.py new file mode 100644 index 0000000..d1ebcc5 --- /dev/null +++ b/server/tests/test_expense_financial_value_chain_e2e.py @@ -0,0 +1,591 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, date, datetime, timedelta +from decimal import Decimal + +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext +from app.db.base_class import Base +from app.models.budget import BudgetAllocation +from app.models.employee import Employee +from app.models.expense_case import BusinessEvent +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorEvent, + PaymentReconciliationCase, +) +from app.models.financial_record import ExpenseClaim +from app.models.organization import OrganizationUnit +from app.models.savings import SavingsEvidenceLink, SavingsRealization +from app.schemas.commercial import ( + CommercialCostEventCreate, + CommercialPlanCreate, + CommercialPricingScenarioWrite, + CommercialSubscriptionCreate, +) +from app.schemas.financial_connector import FinancialEventEnvelope +from app.schemas.reimbursement import ExpenseClaimItemCreate +from app.schemas.savings import SavingsRealizationActionCreate +from app.services.commercial_admin import CommercialAdminService +from app.services.commercial_analytics import CommercialAnalyticsService +from app.services.commercial_metering import CommercialMeteringService +from app.services.commercial_pricing import CommercialPricingService +from app.services.expense_claims import ExpenseClaimService +from app.services.financial_connector_auth import ( + FinancialConnectorSecretResolver, + sign_financial_event, +) +from app.services.financial_connector_ingestion import FinancialConnectorIngestionService +from app.services.savings_discovery import SavingsDiscoveryService +from app.services.savings_realization import SavingsRealizationService + + +def test_expense_application_to_refund_and_value_evidence_chain() -> None: + """正式服务贯穿申请、审批、外部财务事实和商业价值口径。""" + + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + try: + with factory() as db: + _run_chain(db) + finally: + Base.metadata.drop_all(engine) + engine.dispose() + + +def _run_chain(db: Session) -> None: + now = datetime.now(UTC) + users, application = _seed_application_context(db, now=now) + claims = ExpenseClaimService(db) + + submitted_application = claims.submit_claim( + application.id, + users["employee"], + correlation_id="value-chain-application-submit", + ) + assert submitted_application is not None + assert (submitted_application.status, submitted_application.approval_stage) == ( + "submitted", + "直属领导审批", + ) + + approved_application = claims.approve_claim( + application.id, + users["manager"], + opinion="差旅必要且预算充足，同意生成报销草稿。", + request_id="value-chain-application-approve", + expected_status="submitted", + expected_approval_stage="直属领导审批", + ) + assert approved_application is not None + assert (approved_application.status, approved_application.approval_stage) == ( + "approved", + "关联单据状态", + ) + reimbursement = _generated_reimbursement(db, approved_application) + + reimbursement = claims.create_claim_item( + claim_id=reimbursement.id, + payload=ExpenseClaimItemCreate( + item_date=date.today(), + item_type="hotel", + item_reason="上海客户现场住宿", + item_location="上海", + item_amount=Decimal("66.00"), + invoice_id="invoice-value-chain-001", + ), + current_user=users["employee"], + ) + assert reimbursement is not None + reimbursable_item = next( + item for item in reimbursement.items if item.invoice_id == "invoice-value-chain-001" + ) + opportunity = SavingsDiscoveryService(db).discover_standard_adjustments( + claim=reimbursement, + items_by_id={reimbursable_item.id: reimbursable_item}, + adjustment_flags=[ + { + "item_id": reimbursable_item.id, + "message": "已发布住宿政策将原始 100 元锁定为可报 66 元。", + "original_amount": "100.00", + "reimbursable_amount": "66.00", + "employee_absorbed_amount": "34.00", + "policy_rule_version": "value-chain-hotel-v1", + "policy_rule_version_source": "published", + "policy_grade": "P6", + "policy_matched_city": "上海", + "calculation_fingerprint": "sha256:" + "a" * 64, + } + ], + current_user=users["employee"], + request_id="value-chain-saving-discovery", + )[0] + db.commit() + + submitted = claims.submit_claim( + reimbursement.id, + users["employee"], + correlation_id="value-chain-reimbursement-submit", + ) + assert submitted is not None and submitted.approval_stage == "直属领导审批" + manager_approved = claims.approve_claim( + reimbursement.id, + users["manager"], + opinion="业务真实，同意进入财务审核。", + request_id="value-chain-manager-approve", + expected_status="submitted", + expected_approval_stage="直属领导审批", + ) + assert manager_approved is not None and manager_approved.approval_stage == "财务审批" + finance_approved = claims.approve_claim( + reimbursement.id, + users["finance"], + opinion="票据、政策调整和预算均已复核。", + request_id="value-chain-finance-approve", + expected_status="submitted", + expected_approval_stage="财务审批", + ) + assert finance_approved is not None + assert (finance_approved.status, finance_approved.approval_stage) == ( + "pending_payment", + "待付款", + ) + + _seed_commercial_account(db, now=now) + _seed_connector(db) + db.commit() + timestamp = 1_800_000_000 + + before_payment_events = _event_count(db, "payment_completed") + mismatch = _financial_event( + reimbursement, + event_id="value-chain-settlement-mismatch", + extra={"amount": "999.00"}, + ) + mismatch_result = _ingest(db, mismatch, timestamp) + db.commit() + db.refresh(reimbursement) + assert mismatch_result.error_code == "amount_mismatch" + assert reimbursement.status == "pending_payment" + assert _event_count(db, "payment_completed") == before_payment_events + assert ( + db.scalar( + select(func.count(SavingsRealization.id)).where( + SavingsRealization.opportunity_id == opportunity.id + ) + ) + == 0 + ) + + settlement = _financial_event(reimbursement, event_id="value-chain-settlement-001") + settled = _ingest(db, settlement, timestamp + 1) + db.commit() + replay = _ingest(db, settlement, timestamp + 1) + db.commit() + db.refresh(reimbursement) + db.refresh(application) + assert settled.processing_status == "processed" and replay.replayed is True + assert settled.verification_level == "production_verified" + assert settled.evidence_classification == "external_cash" + assert reimbursement.status == "paid" + assert application.approval_stage == "申请归档" + assert _event_count(db, "payment_completed") == before_payment_events + 1 + + realization = db.scalar( + select(SavingsRealization).where( + SavingsRealization.opportunity_id == opportunity.id, + SavingsRealization.realization_type == "actual", + ) + ) + assert realization is not None and realization.actual_net == Decimal("34.0000") + confirmation = SavingsRealizationActionCreate( + action="confirm", + request_id="value-chain-saving-confirm", + expected_version=1, + comment="独立财务已复核生产银行回执、政策基线和归因键。", + ) + confirmed = SavingsRealizationService(db).execute_action( + realization.id, + confirmation, + users["confirmer"], + ) + confirmed_replay = SavingsRealizationService(db).execute_action( + realization.id, + confirmation, + users["confirmer"], + ) + assert confirmed.response.replayed is False + assert confirmed_replay.response.replayed is True + evidence = db.scalar( + select(SavingsEvidenceLink).where( + SavingsEvidenceLink.realization_id == realization.id, + SavingsEvidenceLink.resource_type == "financial_connector_event", + ) + ) + assert evidence is not None and evidence.verification_status == "verified" + + erp = _financial_event( + reimbursement, + event_id="value-chain-erp-001", + event_type="erp_posted", + extra={ + "origin_external_event_id": settlement.external_event_id, + "erp_document_number": "ERP-VALUE-CHAIN-000001", + "accounting_period": "2026-07", + }, + ) + erp_result = _ingest(db, erp, timestamp + 2) + db.commit() + case = db.get(PaymentReconciliationCase, settled.reconciliation_case_id) + assert erp_result.processing_status == "processed" + assert case is not None and case.erp_status == "posted" + assert case.erp_document_tail == "N-000001" + + window = { + "start": now - timedelta(days=2), + "end": now + timedelta(days=2), + "as_of": now + timedelta(days=1), + } + before_refund = CommercialAnalyticsService(db).build("default", **window) + assert _money(before_refund.verified_cash_savings) == {"CNY": Decimal("34.0000")} + pricing = CommercialPricingService(db).build( + "default", + CommercialPricingScenarioWrite( + **window, + target_contribution_margin_rate=Decimal("0.50"), + max_verified_savings_share=Decimal("0.50"), + ), + ) + assert pricing.recommended_model == "hybrid" + assert pricing.scenarios[0].minimum_sustainable_charge == Decimal("10.0000") + assert pricing.scenarios[0].maximum_value_aligned_charge == Decimal("17.0000") + + refund = _financial_event( + reimbursement, + event_id="value-chain-refund-001", + event_type="payment_refunded", + extra={"origin_external_event_id": settlement.external_event_id}, + ) + refunded = _ingest(db, refund, timestamp + 3) + db.commit() + refund_replay = _ingest(db, refund, timestamp + 3) + db.commit() + db.refresh(reimbursement) + assert refunded.processing_status == "processed" and refund_replay.replayed is True + assert reimbursement.status == "pending_payment" + realizations = list( + db.scalars( + select(SavingsRealization).where(SavingsRealization.opportunity_id == opportunity.id) + ).all() + ) + assert len(realizations) == 2 + assert sum((row.reporting_amount for row in realizations), Decimal("0")) == Decimal("0") + + duplicate_reversal = _financial_event( + reimbursement, + event_id="value-chain-reversal-after-refund", + event_type="payment_reversed", + extra={"origin_external_event_id": settlement.external_event_id}, + ) + duplicate_result = _ingest(db, duplicate_reversal, timestamp + 4) + db.commit() + assert duplicate_result.processing_status == "exception" + assert duplicate_result.error_code == "claim_not_paid" + assert ( + db.scalar( + select(func.count(SavingsRealization.id)).where( + SavingsRealization.opportunity_id == opportunity.id + ) + ) + == 2 + ) + + after_refund = CommercialAnalyticsService(db).build("default", **window) + assert _money(after_refund.verified_cash_savings) == {"CNY": Decimal("0.0000")} + pricing_after_refund = CommercialPricingService(db).build( + "default", + CommercialPricingScenarioWrite( + **window, + target_contribution_margin_rate=Decimal("0.50"), + max_verified_savings_share=Decimal("0.50"), + ), + ) + assert pricing_after_refund.recommended_model == "optimize_unit_economics" + assert pricing_after_refund.scenarios[0].maximum_value_aligned_charge == Decimal("0.0000") + assert ( + CommercialAnalyticsService(db) + .build( + "tenant-other", + **window, + ) + .verified_cash_savings.values + == [] + ) + assert ( + db.scalar( + select(func.count(FinancialConnectorEvent.id)).where( + FinancialConnectorEvent.external_event_id == refund.external_event_id + ) + ) + == 1 + ) + + +def _seed_application_context( + db: Session, + *, + now: datetime, +) -> tuple[dict[str, CurrentUserContext], ExpenseClaim]: + department = OrganizationUnit( + unit_code="VALUE-CHAIN-DEPT", + name="价值链试点部", + unit_type="department", + ) + manager = Employee( + employee_no="VALUE-MANAGER", + name="价值链经理", + email="value-manager@example.com", + organization_unit=department, + ) + employee = Employee( + employee_no="VALUE-EMPLOYEE", + name="价值链员工", + email="value-employee@example.com", + organization_unit=department, + manager=manager, + ) + db.add_all([department, manager, employee]) + db.flush() + db.add( + BudgetAllocation( + budget_no="BUD-VALUE-CHAIN-2026Q3", + fiscal_year=2026, + period_type="quarter", + period_key="2026Q3", + department_id=department.id, + department_name=department.name, + subject_code="travel", + subject_name="差旅", + original_amount=Decimal("50000.00"), + adjusted_amount=Decimal("0.00"), + status="active", + warning_threshold=Decimal("80.00"), + control_action="block", + ) + ) + application = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no="APP-VALUE-CHAIN-20260716", + employee_id=employee.id, + employee_name=employee.name, + department_id=department.id, + department_name=department.name, + expense_type="travel_application", + reason="上海客户现场差旅", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=0, + occurred_at=now, + status="draft", + approval_stage="待提交", + risk_flags_json=[], + ) + db.add(application) + db.commit() + return ( + { + "employee": _user( + employee.email, + name=employee.name, + employee_id=employee.id, + ), + "manager": _user( + manager.email, + name=manager.name, + employee_id=manager.id, + roles=["manager"], + ), + "finance": _user("value-finance@example.com", roles=["finance"]), + "confirmer": _user("value-confirmer@example.com", roles=["finance"]), + }, + application, + ) + + +def _generated_reimbursement( + db: Session, + application: ExpenseClaim, +) -> ExpenseClaim: + draft_id = next( + str(flag.get("generated_draft_claim_id")) + for flag in application.risk_flags_json + if isinstance(flag, dict) and flag.get("generated_draft_claim_id") + ) + draft = db.get(ExpenseClaim, draft_id) + assert draft is not None and draft.status == "draft" + return draft + + +def _seed_commercial_account(db: Session, *, now: datetime) -> None: + admin = CommercialAdminService(db) + plan = admin.create_plan( + "default", + CommercialPlanCreate( + plan_code="value-pilot", + name="价值证明试点", + pricing_model="hybrid", + billing_interval="monthly", + currency="CNY", + base_fee=Decimal("8.00"), + included_seats=10, + effective_from=now - timedelta(days=30), + ), + actor_id="platform-admin", + ) + admin.activate_plan("default", plan.id, expected_version=plan.version) + subscription = admin.create_subscription( + "default", + CommercialSubscriptionCreate( + subscription_key="default-value-pilot-2026", + plan_id=plan.id, + starts_at=now - timedelta(days=10), + current_period_start=now - timedelta(days=1), + current_period_end=now + timedelta(days=29), + seats=5, + ), + actor_id="platform-admin", + ) + CommercialMeteringService(db).record_cost( + "default", + CommercialCostEventCreate( + subscription_id=subscription.id, + cost_category="connector", + quantity=Decimal("1"), + unit="workflow", + unit_cost=Decimal("5.00"), + original_currency="CNY", + reporting_currency="CNY", + fx_rate=Decimal("1"), + allocation_key="value-chain-workflow", + occurred_at=now, + source_system="cost-ledger", + idempotency_key="value-chain-cost-001", + ), + ) + + +def _seed_connector(db: Session) -> None: + db.add( + FinancialConnectorConfig( + id=str(uuid.uuid4()), + tenant_id="default", + provider="value-bank", + environment="production", + key_version="v1", + secret_ref="connector/value-chain", + allowed_event_types_json=[ + "payment_settled", + "payment_failed", + "erp_posted", + "erp_posting_failed", + "payment_refunded", + "payment_reversed", + ], + clock_skew_seconds=300, + status="active", + created_by="platform-admin", + ) + ) + + +def _financial_event( + claim: ExpenseClaim, + *, + event_id: str, + event_type: str = "payment_settled", + extra: dict[str, str] | None = None, +) -> FinancialEventEnvelope: + return FinancialEventEnvelope( + tenant_id="default", + external_event_id=event_id, + event_type=event_type, + occurred_at=datetime.now(UTC), + correlation_id=f"corr-{event_id}", + payload={ + "claim_id": claim.id, + "claim_reference": claim.claim_no, + "amount": str(claim.amount), + "currency": claim.currency, + "external_payment_reference": f"PAY-{claim.claim_no}", + **dict(extra or {}), + }, + ) + + +def _ingest( + db: Session, + envelope: FinancialEventEnvelope, + timestamp: int, +): + secret = "value-chain-server-secret" + signature = sign_financial_event( + envelope, + timestamp=timestamp, + secret=secret, + tenant_id="default", + provider="value-bank", + key_version="v1", + ) + return FinancialConnectorIngestionService( + db, + secrets=FinancialConnectorSecretResolver({"connector/value-chain": secret}), + now_epoch=timestamp, + ).ingest( + envelope, + tenant_header="default", + provider_header="value-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header=signature, + ) + + +def _event_count(db: Session, event_type: str) -> int: + return int( + db.scalar( + select(func.count(BusinessEvent.id)).where(BusinessEvent.event_type == event_type) + ) + or 0 + ) + + +def _money(metric) -> dict[str, Decimal]: + return {item.currency: item.amount for item in metric.values} + + +def _user( + username: str, + *, + name: str | None = None, + employee_id: str = "", + roles: list[str] | None = None, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=name or username, + employee_id=employee_id, + role_codes=list(roles or []), + is_admin=False, + tenant_id="default", + ) diff --git a/server/tests/test_expense_workflow_learning.py b/server/tests/test_expense_workflow_learning.py new file mode 100644 index 0000000..82fde8f --- /dev/null +++ b/server/tests/test_expense_workflow_learning.py @@ -0,0 +1,447 @@ +from __future__ import annotations + +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.db.base import Base +from app.models.ai_learning import AIDecision, AIDecisionFeedback, WorkflowOutcome +from app.models.expense_case import BusinessEvent +from app.models.financial_record import ExpenseClaim +from app.models.risk_observation import RiskObservation +from app.schemas.risk_disposition import RiskDispositionActionCreate +from app.services.expense_cases import ExpenseCaseService +from app.services.expense_workflow_learning import ExpenseWorkflowLearningService +from app.services.risk_dispositions import RiskDispositionService + + +def test_workflow_events_create_idempotent_verified_learning_evidence() -> None: + with _build_session() as db: + claim, decision, pre_review = _seed_submitted_decision(db, tenant_id="tenant-a") + service = ExpenseWorkflowLearningService(db) + + service.record_prior_events_for_decision(decision) + service.record_prior_events_for_decision(decision) + + audit_outcome = db.scalar( + select(WorkflowOutcome).where(WorkflowOutcome.business_event_id == pre_review.id) + ) + audit_feedback = db.scalar( + select(AIDecisionFeedback).where( + AIDecisionFeedback.correlation_id == pre_review.correlation_id, + AIDecisionFeedback.action_type == "audit_cleared", + ) + ) + assert audit_outcome is not None + assert audit_outcome.outcome_type == "audit_cleared" + assert audit_outcome.outcome_status == "verified" + assert audit_feedback is not None + assert audit_feedback.feedback_type == "accepted" + assert audit_feedback.verification_status == "server_verified" + assert audit_feedback.training_eligible is True + assert decision.training_eligible is True + assert "客户端伪造审计备注" not in str(audit_outcome.result_json) + assert "客户端伪造审计备注" not in str(audit_feedback.final_value_json) + + claim.status = "returned" + claim.approval_stage = "待提交" + _, returned = ExpenseCaseService(db).record_claim_event( + claim, + event_type="application_returned", + actor_id="manager-a", + tenant_id="tenant-a", + correlation_id="return-flow-1", + idempotency_key="return-flow-1", + extra_payload={"reason": "不要把自由文本变成高置信记忆"}, + ) + _, replayed = ExpenseCaseService(db).record_claim_event( + claim, + event_type="application_returned", + actor_id="manager-a", + tenant_id="tenant-a", + correlation_id="return-flow-1", + idempotency_key="return-flow-1", + extra_payload={"reason": "重放时也不得进入学习证据"}, + ) + assert replayed.id == returned.id + + return_outcome = db.scalar( + select(WorkflowOutcome).where(WorkflowOutcome.business_event_id == returned.id) + ) + return_feedback = db.scalar( + select(AIDecisionFeedback).where(AIDecisionFeedback.action_type == "workflow_returned") + ) + assert return_outcome is not None + assert return_outcome.outcome_type == "workflow_returned" + assert "自由文本" not in str(return_outcome.result_json) + assert return_feedback is not None + assert return_feedback.feedback_type == "rejected" + assert return_feedback.verification_status == "human_verified" + assert return_feedback.changed_fields_json == [] + assert db.scalar(select(func.count()).select_from(WorkflowOutcome)) == 2 + assert db.scalar(select(func.count()).select_from(AIDecisionFeedback)) == 2 + + +def test_approval_pass_override_and_payment_are_separate_outcomes() -> None: + with _build_session() as db: + claim, decision, _ = _seed_submitted_decision(db, tenant_id="tenant-a") + case_service = ExpenseCaseService(db) + + claim.approval_stage = "财务审批" + _, passed = case_service.record_claim_event( + claim, + event_type="approval_stage_completed", + actor_id="manager-a", + tenant_id="tenant-a", + correlation_id="approval-pass-1", + idempotency_key="approval-pass-1", + extra_payload={"opinion": "自由审批意见不进入反馈"}, + ) + _, overridden = case_service.record_claim_event( + claim, + event_type="approval_stage_completed", + actor_id="finance-a", + tenant_id="tenant-a", + correlation_id="approval-override-1", + idempotency_key="approval-override-1", + extra_payload={ + "ai_decision_overridden": True, + "opinion": "覆盖原因也不进入高置信证据", + }, + ) + claim.status = "paid" + claim.approval_stage = "已付款" + _, paid = case_service.record_claim_event( + claim, + event_type="payment_completed", + actor_id="finance-a", + tenant_id="tenant-a", + correlation_id="payment-1", + idempotency_key="payment-1", + extra_payload={"payment_note": "付款备注不得被学习"}, + ) + + outcomes = list( + db.scalars( + select(WorkflowOutcome).where( + WorkflowOutcome.business_event_id.in_([passed.id, overridden.id, paid.id]) + ) + ).all() + ) + assert {item.outcome_type for item in outcomes} == { + "approval_passed", + "approval_overridden", + "payment_completed", + } + assert all(item.tenant_id == "tenant-a" for item in outcomes) + assert all(item.decision_id == decision.id for item in outcomes) + assert "审批意见" not in str([item.result_json for item in outcomes]) + assert "付款备注" not in str([item.result_json for item in outcomes]) + + feedback = list( + db.scalars( + select(AIDecisionFeedback).where( + AIDecisionFeedback.correlation_id.in_( + ["approval-pass-1", "approval-override-1", "payment-1"] + ) + ) + ).all() + ) + assert {(item.action_type, item.feedback_type) for item in feedback} == { + ("approval_passed", "accepted"), + ("approval_overridden", "rejected"), + } + + +def test_trusted_outcome_does_not_promote_unverified_client_decision_for_training() -> None: + with _build_session() as db: + _, decision, pre_review = _seed_submitted_decision( + db, + tenant_id="tenant-client-observed", + ) + decision.evidence_json = {"trust_level": "behavioral_analytics_only"} + db.flush() + + ExpenseWorkflowLearningService(db).record_prior_events_for_decision(decision) + + outcome = db.scalar( + select(WorkflowOutcome).where(WorkflowOutcome.business_event_id == pre_review.id) + ) + feedback = db.scalar( + select(AIDecisionFeedback).where( + AIDecisionFeedback.correlation_id == pre_review.correlation_id + ) + ) + assert outcome is not None + assert outcome.outcome_status == "verified" + assert feedback is not None + assert feedback.verification_status == "server_verified" + assert feedback.training_eligible is False + assert decision.training_eligible is False + + +def test_later_workflow_event_only_updates_latest_submitted_ai_decision() -> None: + with _build_session() as db: + claim, first_decision, _ = _seed_submitted_decision(db, tenant_id="tenant-latest") + case_service = ExpenseCaseService(db) + _, second_submission = case_service.record_claim_event( + claim, + event_type="application_submitted", + actor_id="owner-a", + tenant_id="tenant-latest", + correlation_id="submit-flow-2", + idempotency_key="application-submit-2", + ) + second_decision = AIDecision( + id="decision-tenant-latest-2", + tenant_id="tenant-latest", + expense_case_id=first_decision.expense_case_id, + business_event_id=second_submission.id, + expense_claim_id=claim.id, + correlation_id=second_submission.correlation_id, + subject_type="expense_claim", + subject_id=claim.id, + decision_type="expense_application_prefill", + decision_source="hybrid", + status="executed", + automation_mode="prefill", + confidence=Decimal("0.9500"), + suggestion_json={"value_fingerprint": "hmac-sha256:second"}, + evidence_json={"trust_level": "server_snapshot_verified"}, + version_json={"schema_version": 1}, + schema_version=1, + training_eligible=False, + idempotency_key="decision-tenant-latest-2-key", + content_fingerprint="sha256:" + "2" * 64, + created_at=datetime.now(UTC), + ) + db.add(second_decision) + db.flush() + + claim.status = "returned" + _, returned = case_service.record_claim_event( + claim, + event_type="application_returned", + actor_id="manager-a", + tenant_id="tenant-latest", + correlation_id="return-latest", + idempotency_key="return-latest", + ) + + outcomes = list( + db.scalars( + select(WorkflowOutcome).where(WorkflowOutcome.business_event_id == returned.id) + ).all() + ) + assert [item.decision_id for item in outcomes] == [second_decision.id] + + +def test_learning_bridge_rejects_forged_cross_tenant_event() -> None: + with _build_session() as db: + claim, _, _ = _seed_submitted_decision(db, tenant_id="tenant-a") + claim.status = "returned" + _, trusted = ExpenseCaseService(db).record_claim_event( + claim, + event_type="application_returned", + actor_id="manager-a", + tenant_id="tenant-a", + correlation_id="trusted-return", + idempotency_key="trusted-return", + ) + forged = BusinessEvent( + id=trusted.id, + tenant_id="tenant-b", + expense_case_id=trusted.expense_case_id, + aggregate_type=trusted.aggregate_type, + aggregate_id=trusted.aggregate_id, + event_type=trusted.event_type, + event_version=trusted.event_version, + idempotency_key=trusted.idempotency_key, + correlation_id=trusted.correlation_id, + actor_id=trusted.actor_id, + actor_type=trusted.actor_type, + payload_json=trusted.payload_json, + delivery_status="pending", + occurred_at=trusted.occurred_at, + ) + + with pytest.raises(PermissionError, match="不存在或不属于"): + ExpenseWorkflowLearningService(db).record_event(forged) + + assert all( + outcome.tenant_id == "tenant-a" for outcome in db.scalars(select(WorkflowOutcome)).all() + ) + + +def test_typed_risk_disposition_emits_audit_outcome_without_comment() -> None: + with _build_session() as db: + claim, decision, _ = _seed_submitted_decision(db, tenant_id="tenant-a") + observation = RiskObservation( + id="risk-observation-1", + tenant_id="tenant-a", + observation_key="risk:tenant-a:claim-1", + subject_type="expense_claim", + subject_key=f"claim:{claim.id}", + subject_label=claim.claim_no, + claim_id=claim.id, + claim_no=claim.claim_no, + risk_type="duplicate_invoice", + risk_signal="duplicate_invoice", + title="重复票据", + description="风险观察描述", + risk_score=88, + risk_level="high", + confidence_score=0.91, + control_stage="reimbursement", + control_mode="risk_observation", + automation_mode="semi_auto_review", + source="test", + algorithm_version="risk.v1", + status="pending_review", + feedback_status="unreviewed", + ) + db.add(observation) + db.commit() + + service = RiskDispositionService(db) + payload = RiskDispositionActionCreate( + action="confirm", + expected_version=0, + request_id="risk-confirm-1", + comment="该人工评论不得进入 AI 学习账本", + ) + first = service.execute_action( + observation.id, + payload, + tenant_id="tenant-a", + actor_id="finance-a", + actor_name="财务甲", + ) + replay = service.execute_action( + observation.id, + payload, + tenant_id="tenant-a", + actor_id="finance-a", + actor_name="财务甲", + ) + assert replay.event.id == first.event.id + + audit_events = list( + db.scalars( + select(BusinessEvent).where( + BusinessEvent.tenant_id == "tenant-a", + BusinessEvent.event_type == "audit_conclusion_recorded", + ) + ).all() + ) + assert len(audit_events) == 1 + assert audit_events[0].payload_json["audit_decision"] == "confirm" + assert "人工评论" not in str(audit_events[0].payload_json) + outcome = db.scalar( + select(WorkflowOutcome).where(WorkflowOutcome.business_event_id == audit_events[0].id) + ) + assert outcome is not None + assert outcome.decision_id == decision.id + assert outcome.outcome_type == "audit_confirmed" + assert "人工评论" not in str(outcome.result_json) + assert ( + db.scalar( + select(func.count()) + .select_from(WorkflowOutcome) + .where(WorkflowOutcome.business_event_id == audit_events[0].id) + ) + == 1 + ) + + +def _seed_submitted_decision( + db: Session, + *, + tenant_id: str, +) -> tuple[ExpenseClaim, AIDecision, BusinessEvent]: + now = datetime(2026, 7, 16, 9, 0, tzinfo=UTC) + claim = ExpenseClaim( + id=f"claim-{tenant_id}", + tenant_id=tenant_id, + claim_no=f"AP-{tenant_id.upper()}-001", + employee_name="测试员工", + department_name="研发部", + expense_type="travel_application", + reason="客户拜访", + location="上海", + amount=Decimal("1200.00"), + currency="CNY", + invoice_count=0, + occurred_at=now, + status="draft", + approval_stage="待提交", + risk_flags_json=[], + ) + db.add(claim) + db.flush() + case_service = ExpenseCaseService(db) + _, pre_review = case_service.record_claim_event( + claim, + event_type="application_pre_review_completed", + actor_id="owner-a", + tenant_id=tenant_id, + correlation_id="submit-flow-1", + idempotency_key="pre-review-1", + update_case_state=False, + extra_payload={ + "decision": "ready", + "comment": "客户端伪造审计备注", + }, + ) + claim.status = "submitted" + claim.approval_stage = "直属领导审批" + claim.submitted_at = now + expense_case, submitted = case_service.record_claim_event( + claim, + event_type="application_submitted", + actor_id="owner-a", + tenant_id=tenant_id, + correlation_id="submit-flow-1", + idempotency_key="application-submit-1", + ) + decision = AIDecision( + id=f"decision-{tenant_id}", + tenant_id=tenant_id, + expense_case_id=expense_case.id, + business_event_id=submitted.id, + expense_claim_id=claim.id, + correlation_id=submitted.correlation_id, + subject_type="expense_claim", + subject_id=claim.id, + decision_type="expense_application_prefill", + decision_source="hybrid", + status="executed", + automation_mode="prefill", + confidence=Decimal("0.9000"), + suggestion_json={"value_fingerprint": "hmac-sha256:test"}, + evidence_json={"trust_level": "server_snapshot_verified"}, + version_json={"schema_version": 1}, + schema_version=1, + training_eligible=False, + idempotency_key=f"decision-{tenant_id}-idempotency", + content_fingerprint="sha256:" + "1" * 64, + created_at=now, + ) + db.add(decision) + db.flush() + return claim, decision, pre_review + + +def _build_session() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + return factory() diff --git a/server/tests/test_finance_dashboard_tenant_security.py b/server/tests/test_finance_dashboard_tenant_security.py new file mode 100644 index 0000000..43283cf --- /dev/null +++ b/server/tests/test_finance_dashboard_tenant_security.py @@ -0,0 +1,530 @@ +from __future__ import annotations + +from collections.abc import Generator +from datetime import UTC, datetime, timedelta +from decimal import Decimal + +import pytest +from auth_helpers import install_legacy_header_auth_override +from fastapi import FastAPI, HTTPException +from fastapi.testclient import TestClient +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.api.deps import CurrentUserContext, get_db +from app.api.v1.endpoints.agent_runs import router as agent_runs_router +from app.api.v1.endpoints.analytics import router as analytics_router +from app.db.base import Base +from app.models.agent_run import AgentRun, AgentToolCall +from app.models.budget import BudgetAllocation +from app.models.expense_case import ExpenseCase, ExpenseCaseLink +from app.models.financial_record import ExpenseClaim +from app.services.finance_dashboard import FinanceDashboardService +from app.services.finance_dashboard_access_policy import FinanceDashboardAccessPolicy +from app.services.finance_dashboard_scope import resolve_finance_dashboard_data_scope +from app.services.finance_dashboard_snapshot import ( + FINANCE_DASHBOARD_TASK_TYPE, + FinanceDashboardSnapshotService, +) + + +def _session_factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False) + + +def _claim( + *, + claim_id: str, + claim_no: str, + amount: str, + tenant_id: str = "default", +) -> ExpenseClaim: + now = datetime.now(UTC) + return ExpenseClaim( + id=claim_id, + tenant_id=tenant_id, + claim_no=claim_no, + employee_name=f"employee-{claim_id}", + department_name="财务部", + expense_type="travel", + reason="tenant scope test", + location="上海", + amount=Decimal(amount), + invoice_count=1, + occurred_at=now - timedelta(hours=2), + submitted_at=now - timedelta(hours=1), + status="paid", + approval_stage="payment", + risk_flags_json=[], + hermes_risk_flag=False, + created_at=now - timedelta(hours=2), + updated_at=now - timedelta(minutes=30), + ) + + +def _link_claim(db: Session, claim: ExpenseClaim, *, tenant_id: str) -> None: + expense_case = ExpenseCase( + id=f"case-{claim.id}", + tenant_id=tenant_id, + case_no=f"CASE-{claim.claim_no}", + scene_code="travel", + title=f"{claim.claim_no} case", + current_stage="completed", + status="completed", + ) + db.add(expense_case) + db.flush() + db.add( + ExpenseCaseLink( + tenant_id=tenant_id, + expense_case_id=expense_case.id, + resource_type="expense_claim", + resource_id=claim.id, + relation_type="reimbursement", + ) + ) + + +def _seed_tenant_claim( + db: Session, + *, + tenant_id: str, + claim_id: str, + claim_no: str, + amount: str, +) -> ExpenseClaim: + claim = _claim( + claim_id=claim_id, + claim_no=claim_no, + amount=amount, + tenant_id=tenant_id, + ) + db.add(claim) + db.flush() + _link_claim(db, claim, tenant_id=tenant_id) + return claim + + +def _seed_finance_snapshot_run( + db: Session, + *, + run_id: str, + tenant_id: str | None, + amount: str, + route_data_scope: str | None = None, +) -> None: + route_json = { + "task_type": FINANCE_DASHBOARD_TASK_TYPE, + "snapshot_key": f"snapshot-{run_id}", + "snapshot_payload": { + "tenant_marker": tenant_id or "legacy-unscoped", + "amount": amount, + }, + } + ontology_json = {"scenario": "finance_dashboard"} + if tenant_id is not None: + expected_scope = resolve_finance_dashboard_data_scope(tenant_id) + route_json["tenant_id"] = tenant_id + route_json["data_scope"] = route_data_scope or expected_scope + ontology_json["tenant_id"] = tenant_id + ontology_json["data_scope"] = expected_scope + + db.add( + AgentRun( + run_id=run_id, + agent="hermes", + source="system_event", + user_id="digital_employee", + ontology_json=ontology_json, + route_json=route_json, + permission_level="read", + status="succeeded", + result_summary=f"sensitive amount {amount}", + started_at=datetime.now(UTC), + finished_at=datetime.now(UTC), + ) + ) + db.flush() + db.add( + AgentToolCall( + run_id=run_id, + tool_type="database", + tool_name="digital_employee.finance_dashboard.snapshot", + request_json={"tenant_id": tenant_id, "amount": amount}, + response_json={"secret_amount": amount}, + status="succeeded", + duration_ms=1, + ) + ) + + +def test_finance_dashboard_isolates_claims_and_hides_legacy_budget_outside_default() -> None: + now = datetime.now(UTC) + session_factory = _session_factory() + + with session_factory() as db: + db.add(_claim(claim_id="legacy-default", claim_no="CLM-LEGACY-001", amount="100.00")) + _seed_tenant_claim( + db, + tenant_id="tenant-a", + claim_id="claim-a", + claim_no="CLM-TENANT-A-001", + amount="200.00", + ) + _seed_tenant_claim( + db, + tenant_id="tenant-b", + claim_id="claim-b", + claim_no="CLM-TENANT-B-001", + amount="300.00", + ) + db.add( + BudgetAllocation( + budget_no="BUD-LEGACY-DEFAULT-001", + fiscal_year=now.year, + period_type="year", + period_key=str(now.year), + department_name="财务部", + subject_code="travel", + subject_name="差旅费", + original_amount=Decimal("10000.00"), + adjusted_amount=Decimal("0.00"), + status="active", + warning_threshold=Decimal("80.00"), + control_action="warn", + ) + ) + db.commit() + + default_dashboard = FinanceDashboardService(db).build_dashboard() + tenant_a_dashboard = FinanceDashboardService( + db, + tenant_id="tenant-a", + ).build_dashboard() + tenant_b_dashboard = FinanceDashboardService( + db, + tenant_id="tenant-b", + ).build_dashboard() + + assert default_dashboard.totals["reimbursementCount"] == 1 + assert default_dashboard.totals["reimbursementAmount"] == 100.0 + assert default_dashboard.budget_summary["included"] is True + assert default_dashboard.budget_summary["total"] == "¥10,000" + + assert tenant_a_dashboard.totals["reimbursementCount"] == 1 + assert tenant_a_dashboard.totals["reimbursementAmount"] == 200.0 + assert "CLM-TENANT-B-001" not in str(tenant_a_dashboard.top_claims) + assert tenant_a_dashboard.budget_summary == { + "ratio": 0.0, + "total": "¥0", + "used": "¥0", + "left": "¥0", + "included": False, + "scope": "unavailable", + "reason": "当前租户尚未接入独立预算池，预算指标未纳入统计。", + } + assert tenant_a_dashboard.totals["budgetUsageRate"] == 0.0 + assert all( + metric["detail"] == "当前租户未接入独立预算池" and metric["tone"] == "neutral" + for metric in tenant_a_dashboard.budget_metrics + ) + assert "预算超支" not in {item["name"] for item in tenant_a_dashboard.bottlenecks} + + assert tenant_b_dashboard.totals["reimbursementCount"] == 1 + assert tenant_b_dashboard.totals["reimbursementAmount"] == 300.0 + assert tenant_b_dashboard.budget_summary["included"] is False + + +def test_finance_dashboard_uses_structured_claim_tenant_when_legacy_link_disagrees() -> None: + session_factory = _session_factory() + + with session_factory() as db: + claim = _claim( + tenant_id="tenant-a", + claim_id="structured-tenant-claim", + claim_no="CLM-STRUCTURED-TENANT-001", + amount="456.00", + ) + db.add(claim) + db.flush() + # Case Link 是历史关联索引，不能覆盖 Claim 自身的结构化租户归属。 + _link_claim(db, claim, tenant_id="tenant-b") + db.commit() + + tenant_a_dashboard = FinanceDashboardService( + db, + tenant_id="tenant-a", + ).build_dashboard() + tenant_b_dashboard = FinanceDashboardService( + db, + tenant_id="tenant-b", + ).build_dashboard() + + assert tenant_a_dashboard.totals["reimbursementCount"] == 1 + assert tenant_a_dashboard.totals["reimbursementAmount"] == 456.0 + assert tenant_b_dashboard.totals["reimbursementCount"] == 0 + + +def test_finance_dashboard_snapshot_cache_is_partitioned_by_tenant_and_data_scope() -> None: + session_factory = _session_factory() + + with session_factory() as db: + _seed_tenant_claim( + db, + tenant_id="tenant-a", + claim_id="snapshot-claim-a", + claim_no="CLM-SNAPSHOT-A-001", + amount="880.00", + ) + _seed_tenant_claim( + db, + tenant_id="tenant-b", + claim_id="snapshot-claim-b", + claim_no="CLM-SNAPSHOT-B-001", + amount="990.00", + ) + db.commit() + + tenant_a_service = FinanceDashboardSnapshotService(db, tenant_id="tenant-a") + tenant_b_service = FinanceDashboardSnapshotService(db, tenant_id="tenant-b") + first_a = tenant_a_service.build_dashboard() + first_b = tenant_b_service.build_dashboard() + second_a = tenant_a_service.build_dashboard() + + runs = list( + db.scalars( + select(AgentRun).where( + AgentRun.route_json["task_type"].as_string() == FINANCE_DASHBOARD_TASK_TYPE + ) + ).all() + ) + routes = [run.route_json or {} for run in runs] + + assert first_a.totals["reimbursementAmount"] == 880.0 + assert first_b.totals["reimbursementAmount"] == 990.0 + assert second_a.generated_at == first_a.generated_at + assert len(runs) == 2 + assert {route["tenant_id"] for route in routes} == {"tenant-a", "tenant-b"} + assert all(route["data_scope"] for route in routes) + assert len({route["snapshot_key"] for route in routes}) == 2 + assert all(route["tenant_id"] in route["snapshot_key"] for route in routes) + + +def test_default_scheduler_snapshot_rejects_non_default_tenant_scope() -> None: + session_factory = _session_factory() + + with session_factory() as db: + service = FinanceDashboardSnapshotService(db, tenant_id="tenant-a") + with pytest.raises(ValueError, match="default 系统租户"): + service.refresh_default_snapshot() + + +@pytest.mark.parametrize("tenant_id", ["", " "]) +def test_finance_dashboard_access_policy_rejects_missing_tenant_scope( + tenant_id: str, +) -> None: + current_user = CurrentUserContext( + username="finance-without-tenant", + name="Finance Without Tenant", + role_codes=["finance"], + is_admin=False, + tenant_id=tenant_id, + ) + + assert FinanceDashboardAccessPolicy.can_read(current_user) is False + with pytest.raises(HTTPException) as exc_info: + FinanceDashboardAccessPolicy.require_read(current_user) + assert exc_info.value.status_code == 403 + + +def test_finance_dashboard_endpoint_requires_finance_read_role_and_passes_tenant() -> None: + session_factory = _session_factory() + with session_factory() as db: + _seed_tenant_claim( + db, + tenant_id="tenant-a", + claim_id="endpoint-claim-a", + claim_no="CLM-ENDPOINT-A-001", + amount="1234.00", + ) + db.commit() + + app = FastAPI() + app.include_router(analytics_router) + install_legacy_header_auth_override(app) + + def override_db() -> Generator[Session, None, None]: + with session_factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + client = TestClient(app) + + ordinary_response = client.get( + "/analytics/finance-dashboard", + headers={ + "X-Auth-Username": "ordinary", + "X-Auth-Role-Codes": "user", + "X-Auth-Tenant-Id": "tenant-a", + }, + ) + budget_monitor_response = client.get( + "/analytics/finance-dashboard", + headers={ + "X-Auth-Username": "budget-monitor", + "X-Auth-Role-Codes": "budget_monitor", + "X-Auth-Tenant-Id": "tenant-a", + }, + ) + finance_response = client.get( + "/analytics/finance-dashboard", + headers={ + "X-Auth-Username": "finance", + "X-Auth-Role-Codes": "finance", + "X-Auth-Tenant-Id": "tenant-a", + }, + ) + executive_other_tenant_response = client.get( + "/analytics/finance-dashboard", + headers={ + "X-Auth-Username": "executive", + "X-Auth-Role-Codes": "executive", + "X-Auth-Tenant-Id": "tenant-b", + }, + ) + admin_response = client.get( + "/analytics/finance-dashboard", + headers={ + "X-Auth-Username": "admin-reader", + "X-Auth-Is-Admin": "true", + "X-Auth-Tenant-Id": "tenant-b", + }, + ) + + assert ordinary_response.status_code == 403 + assert budget_monitor_response.status_code == 403 + assert finance_response.status_code == 200 + assert finance_response.json()["totals"]["reimbursementAmount"] == 1234.0 + assert executive_other_tenant_response.status_code == 200 + assert executive_other_tenant_response.json()["totals"]["reimbursementCount"] == 0 + assert admin_response.status_code == 200 + + +def test_agent_run_endpoint_cannot_bypass_finance_snapshot_tenant_and_role_scope() -> None: + session_factory = _session_factory() + with session_factory() as db: + _seed_finance_snapshot_run( + db, + run_id="run-finance-tenant-a", + tenant_id="tenant-a", + amount="111.00", + ) + _seed_finance_snapshot_run( + db, + run_id="run-finance-tenant-b", + tenant_id="tenant-b", + amount="222.00", + ) + _seed_finance_snapshot_run( + db, + run_id="run-finance-legacy-unscoped", + tenant_id=None, + amount="999.00", + ) + _seed_finance_snapshot_run( + db, + run_id="run-finance-corrupt-scope", + tenant_id="tenant-a", + amount="777.00", + route_data_scope="claims:corrupt;budget:corrupt", + ) + db.commit() + + app = FastAPI() + app.include_router(agent_runs_router) + install_legacy_header_auth_override(app) + + def override_db() -> Generator[Session, None, None]: + with session_factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + client = TestClient(app) + finance_a_headers = { + "X-Auth-Username": "finance-a", + "X-Auth-Role-Codes": "finance", + "X-Auth-Tenant-Id": "tenant-a", + } + finance_b_headers = { + "X-Auth-Username": "finance-b", + "X-Auth-Role-Codes": "finance", + "X-Auth-Tenant-Id": "tenant-b", + } + + finance_a_list = client.get("/agent-runs", headers=finance_a_headers) + ordinary_a_list = client.get( + "/agent-runs", + headers={ + "X-Auth-Username": "ordinary-a", + "X-Auth-Role-Codes": "user", + "X-Auth-Tenant-Id": "tenant-a", + }, + ) + assert finance_a_list.status_code == 200 + assert [item["run_id"] for item in finance_a_list.json()] == ["run-finance-tenant-a"] + assert ordinary_a_list.status_code == 200 + assert ordinary_a_list.json() == [] + + allowed_detail = client.get( + "/agent-runs/run-finance-tenant-a", + headers=finance_a_headers, + ) + assert allowed_detail.status_code == 200 + assert allowed_detail.json()["route_json"]["snapshot_payload"]["amount"] == "111.00" + assert allowed_detail.json()["tool_calls"][0]["response_json"] == {"secret_amount": "111.00"} + + cross_tenant_detail = client.get( + "/agent-runs/run-finance-tenant-a", + headers=finance_b_headers, + ) + cross_tenant_admin_detail = client.get( + "/agent-runs/run-finance-tenant-a", + headers={ + "X-Auth-Username": "admin-b", + "X-Auth-Is-Admin": "true", + "X-Auth-Tenant-Id": "tenant-b", + }, + ) + same_tenant_ordinary_detail = client.get( + "/agent-runs/run-finance-tenant-a", + headers={ + "X-Auth-Username": "ordinary-a", + "X-Auth-Role-Codes": "user", + "X-Auth-Tenant-Id": "tenant-a", + }, + ) + legacy_unscoped_detail = client.get( + "/agent-runs/run-finance-legacy-unscoped", + headers={ + "X-Auth-Username": "finance-default", + "X-Auth-Role-Codes": "finance", + "X-Auth-Tenant-Id": "default", + }, + ) + corrupt_scope_detail = client.get( + "/agent-runs/run-finance-corrupt-scope", + headers=finance_a_headers, + ) + + assert cross_tenant_detail.status_code == 404 + assert cross_tenant_admin_detail.status_code == 404 + assert same_tenant_ordinary_detail.status_code == 403 + assert legacy_unscoped_detail.status_code == 404 + assert corrupt_scope_detail.status_code == 404 diff --git a/server/tests/test_finance_report_task.py b/server/tests/test_finance_report_task.py index 8ccd3e8..e4f0b8e 100644 --- a/server/tests/test_finance_report_task.py +++ b/server/tests/test_finance_report_task.py @@ -75,6 +75,8 @@ def test_finance_report_task_generates_pdf_and_agent_record(monkeypatch, tmp_pat result = DigitalEmployeeFinanceReportTaskService(db).generate_report( report_type="weekly", + start_date=(now - timedelta(days=7)).date(), + end_date=now.date(), send_email=True, dry_run_email=True, ) diff --git a/server/tests/test_financial_connector_concurrency_postgres.py b/server/tests/test_financial_connector_concurrency_postgres.py new file mode 100644 index 0000000..39bc025 --- /dev/null +++ b/server/tests/test_financial_connector_concurrency_postgres.py @@ -0,0 +1,442 @@ +from __future__ import annotations + +import threading +import uuid +from concurrent.futures import ThreadPoolExecutor +from datetime import UTC, datetime +from decimal import Decimal + +from savings_postgres_testkit import ( # noqa: F401 - 注册 pg_factory fixture + _pg_factory_fixture, +) +from sqlalchemy import func, select, text +from sqlalchemy.exc import DBAPIError +from sqlalchemy.orm import Session, sessionmaker + +from app.models.expense_case import BusinessEvent +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorConfigEvent, + FinancialConnectorEvent, + FinancialConnectorOperationalEvent, +) +from app.models.financial_record import ExpenseClaim +from app.models.tenant import Tenant +from app.schemas.financial_connector import ( + FinancialConnectorConfigLifecycleAction, + FinancialEventEnvelope, +) +from app.services.expense_cases import ExpenseCaseService +from app.services.financial_connector_auth import ( + FinancialConnectorSecretResolver, + sign_financial_event, +) +from app.services.financial_connector_config_lifecycle import ( + FinancialConnectorConfigConflictError, + FinancialConnectorConfigLifecycleService, +) +from app.services.financial_connector_ingestion import ( + FinancialConnectorConflictError, + FinancialConnectorIngestionService, +) +from app.services.financial_connector_operational_events import ( + FinancialConnectorOperationalContext, + FinancialConnectorOperationalEventService, + operational_event_candidate, +) + + +def test_concurrent_identical_settlement_has_one_fact_and_one_payment( + pg_factory: sessionmaker[Session], +) -> None: + seeded = _seed(pg_factory) + timestamp = 1_800_000_000 + envelope = _envelope(seeded, event_id=f"settled-{uuid.uuid4().hex}") + ready = threading.Barrier(2) + + def ingest_once() -> bool: + with pg_factory() as db: + ready.wait(timeout=5) + result = _ingest(db, envelope, timestamp) + db.commit() + return result.replayed + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(ingest_once), pool.submit(ingest_once)) + ] + assert sorted(outcomes) == [False, True] + with pg_factory() as db: + claim = db.get(ExpenseClaim, seeded[1]) + assert claim is not None and claim.status == "paid" + assert ( + db.scalar( + select(func.count(FinancialConnectorEvent.id)).where( + FinancialConnectorEvent.tenant_id == seeded[0], + FinancialConnectorEvent.external_event_id == envelope.external_event_id, + ) + ) + == 1 + ) + assert ( + db.scalar( + select(func.count(BusinessEvent.id)).where( + BusinessEvent.tenant_id == seeded[0], + BusinessEvent.aggregate_id == seeded[1], + BusinessEvent.event_type == "payment_completed", + ) + ) + == 1 + ) + + refund = _envelope( + seeded, + event_id=f"refund-{uuid.uuid4().hex}", + event_type="payment_refunded", + origin_external_event_id=envelope.external_event_id, + ) + with pg_factory() as db: + _ingest(db, refund, timestamp + 1) + db.commit() + claim = db.get(ExpenseClaim, seeded[1]) + assert claim is not None and claim.status == "pending_payment" + second_settlement = _envelope( + seeded, + event_id=f"settled-{uuid.uuid4().hex}", + ) + with pg_factory() as db: + _ingest(db, second_settlement, timestamp + 2) + db.commit() + claim = db.get(ExpenseClaim, seeded[1]) + assert claim is not None and claim.status == "paid" + + +def test_concurrent_conflicting_payload_has_one_winner_and_facts_are_append_only( + pg_factory: sessionmaker[Session], +) -> None: + seeded = _seed(pg_factory) + timestamp = 1_800_000_000 + event_id = f"failed-{uuid.uuid4().hex}" + first = _envelope(seeded, event_id=event_id, event_type="payment_failed") + second = first.model_copy( + update={"payload": {**first.payload, "failure_code": "different_failure"}} + ) + ready = threading.Barrier(2) + + def ingest_once(envelope: FinancialEventEnvelope) -> str: + with pg_factory() as db: + ready.wait(timeout=5) + try: + _ingest(db, envelope, timestamp) + db.commit() + return "created" + except FinancialConnectorConflictError as error: + db.rollback() + FinancialConnectorOperationalEventService(db).record(error.operational_event) + db.commit() + return "conflict" + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(ingest_once, first), pool.submit(ingest_once, second)) + ] + assert sorted(outcomes) == ["conflict", "created"] + with pg_factory() as db: + event = db.scalar( + select(FinancialConnectorEvent).where( + FinancialConnectorEvent.tenant_id == seeded[0], + FinancialConnectorEvent.external_event_id == event_id, + ) + ) + assert event is not None + assert ( + db.scalar( + select(func.count(FinancialConnectorOperationalEvent.id)).where( + FinancialConnectorOperationalEvent.tenant_id == seeded[0], + FinancialConnectorOperationalEvent.event_type == "payload_conflict", + ) + ) + == 1 + ) + savepoint = db.begin_nested() + try: + try: + db.execute( + text( + "UPDATE financial_connector_events SET error_code = 'tampered' " + "WHERE id = :event_id" + ), + {"event_id": event.id}, + ) + except DBAPIError: + pass + else: # pragma: no cover - PostgreSQL trigger must reject + raise AssertionError("append-only event unexpectedly accepted UPDATE") + finally: + savepoint.rollback() + + +def test_concurrent_operational_candidate_has_one_append_only_fact( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id = f"tenant-operational-{uuid.uuid4().hex}" + config_id = str(uuid.uuid4()) + with pg_factory() as db: + db.add( + FinancialConnectorConfig( + id=config_id, + tenant_id=tenant_id, + provider="operational-bank", + environment="production", + key_version="v1", + secret_ref=f"connector/{tenant_id}", + allowed_event_types_json=["payment_settled"], + clock_skew_seconds=300, + status="active", + created_by="postgres-test", + ) + ) + db.commit() + context = FinancialConnectorOperationalContext( + tenant_id=tenant_id, + config_id=config_id, + provider="operational-bank", + environment="production", + request_fingerprint="hmac-sha256:" + "a" * 64, + external_event_fingerprint="hmac-sha256:" + "b" * 64, + ) + occurred_at = datetime.now(UTC) + candidate = operational_event_candidate( + context, + event_type="replay", + reason_code="duplicate_external_event", + occurred_at=occurred_at, + ) + ready = threading.Barrier(2) + + def record_once() -> None: + with pg_factory() as db: + ready.wait(timeout=5) + FinancialConnectorOperationalEventService(db).record(candidate) + db.commit() + + with ThreadPoolExecutor(max_workers=2) as pool: + for future in (pool.submit(record_once), pool.submit(record_once)): + future.result(timeout=10) + + with pg_factory() as db: + rows = list( + db.scalars( + select(FinancialConnectorOperationalEvent).where( + FinancialConnectorOperationalEvent.tenant_id == tenant_id + ) + ).all() + ) + assert len(rows) == 1 + savepoint = db.begin_nested() + try: + try: + db.execute( + text("DELETE FROM financial_connector_operational_events WHERE id = :event_id"), + {"event_id": rows[0].id}, + ) + except DBAPIError: + pass + else: # pragma: no cover - PostgreSQL trigger must reject + raise AssertionError("append-only operational event accepted DELETE") + finally: + savepoint.rollback() + + +def test_concurrent_config_activation_has_one_version_winner( + pg_factory: sessionmaker[Session], +) -> None: + tenant_id = f"tenant-config-{uuid.uuid4().hex}" + config_id = str(uuid.uuid4()) + secret_ref = f"connector/{tenant_id}" + secret = f"strong-secret-{tenant_id}" + with pg_factory() as db: + db.add( + FinancialConnectorConfig( + id=config_id, + tenant_id=tenant_id, + provider="versioned-bank", + environment="production", + key_version="v1", + secret_ref=secret_ref, + allowed_event_types_json=["payment_settled"], + clock_skew_seconds=300, + status="disabled", + version=1, + created_by="postgres-test", + ) + ) + db.commit() + ready = threading.Barrier(2) + + def activate_once(index: int) -> str: + with pg_factory() as db: + ready.wait(timeout=5) + try: + FinancialConnectorConfigLifecycleService( + db, + secrets=FinancialConnectorSecretResolver({secret_ref: secret}), + ).activate( + tenant_id=tenant_id, + config_id=config_id, + payload=FinancialConnectorConfigLifecycleAction( + expected_version=1, + request_id=f"config-activate-{index:03d}", + reason="并发激活只能有一个版本胜者。", + ), + actor_id=f"postgres-operator-{index}", + ) + db.commit() + return "activated" + except FinancialConnectorConfigConflictError: + db.rollback() + return "conflict" + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(activate_once, 1), pool.submit(activate_once, 2)) + ] + assert sorted(outcomes) == ["activated", "conflict"] + with pg_factory() as db: + config = db.get(FinancialConnectorConfig, config_id) + assert config is not None + assert config.status == "active" and config.version == 2 + assert ( + db.scalar( + select(func.count(FinancialConnectorConfigEvent.id)).where( + FinancialConnectorConfigEvent.tenant_id == tenant_id, + FinancialConnectorConfigEvent.config_id == config_id, + FinancialConnectorConfigEvent.action == "activated", + ) + ) + == 1 + ) + + +def _seed(factory: sessionmaker[Session]) -> tuple[str, str, str, str]: + tenant_id = f"tenant-connector-{uuid.uuid4().hex}" + claim_id = str(uuid.uuid4()) + config_id = str(uuid.uuid4()) + with factory() as db: + db.add( + Tenant( + tenant_id=tenant_id, + tenant_code=tenant_id, + name="连接器并发探针租户", + status="active", + ) + ) + db.flush() + claim = ExpenseClaim( + id=claim_id, + tenant_id=tenant_id, + claim_no=f"BX-PG-{uuid.uuid4().hex[:12]}", + employee_name="PostgreSQL 测试员工", + department_name="测试部", + expense_type="travel", + reason="连接器并发验证", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime.now(UTC), + submitted_at=datetime.now(UTC), + status="pending_payment", + approval_stage="待付款", + risk_flags_json=[], + ) + db.add(claim) + db.flush() + ExpenseCaseService(db).ensure_case_for_claim(claim, tenant_id=tenant_id) + db.add( + FinancialConnectorConfig( + id=config_id, + tenant_id=tenant_id, + provider="postgres-bank", + environment="production", + key_version="v1", + secret_ref=f"connector/{tenant_id}", + allowed_event_types_json=[ + "payment_settled", + "payment_failed", + "payment_refunded", + ], + clock_skew_seconds=300, + status="active", + created_by="postgres-test", + ) + ) + db.commit() + return tenant_id, claim_id, config_id, claim.claim_no + + +def _envelope( + seeded: tuple[str, str, str, str], + *, + event_id: str, + event_type: str = "payment_settled", + origin_external_event_id: str | None = None, +) -> FinancialEventEnvelope: + tenant_id, claim_id, _, claim_no = seeded + return FinancialEventEnvelope( + tenant_id=tenant_id, + external_event_id=event_id, + event_type=event_type, + occurred_at=datetime.now(UTC), + correlation_id=f"correlation-{event_id}"[:64], + payload={ + "claim_id": claim_id, + "claim_reference": claim_no, + "amount": "66.00", + "currency": "CNY", + "external_payment_reference": f"PAY-{event_id}", + "failure_code": "provider_rejected", + **( + {"origin_external_event_id": origin_external_event_id} + if origin_external_event_id + else {} + ), + }, + ) + + +def _ingest( + db: Session, + envelope: FinancialEventEnvelope, + timestamp: int, +): + claim = db.get(ExpenseClaim, str(envelope.payload["claim_id"])) + assert claim is not None + envelope = envelope.model_copy( + update={"payload": {**envelope.payload, "claim_reference": claim.claim_no}} + ) + secret_ref = f"connector/{envelope.tenant_id}" + secret = f"secret-{envelope.tenant_id}" + signature = sign_financial_event( + envelope, + timestamp=timestamp, + secret=secret, + tenant_id=envelope.tenant_id, + provider="postgres-bank", + key_version="v1", + ) + return FinancialConnectorIngestionService( + db, + secrets=FinancialConnectorSecretResolver({secret_ref: secret}), + now_epoch=timestamp, + ).ingest( + envelope, + tenant_header=envelope.tenant_id, + provider_header="postgres-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header=signature, + ) diff --git a/server/tests/test_financial_connector_config_lifecycle.py b/server/tests/test_financial_connector_config_lifecycle.py new file mode 100644 index 0000000..56851fa --- /dev/null +++ b/server/tests/test_financial_connector_config_lifecycle.py @@ -0,0 +1,299 @@ +from __future__ import annotations + +import json +import time +import uuid +from collections.abc import Generator +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.api.v1.endpoints.financial_connectors import router +from app.core.config import get_settings +from app.db.base_class import Base +from app.models.financial_record import ExpenseClaim +from app.schemas.financial_connector import FinancialEventEnvelope +from app.services.financial_connector_auth import sign_financial_event + + +@pytest.fixture() +def lifecycle_http(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setenv( + "FINANCIAL_CONNECTOR_HMAC_KEYS_JSON", + json.dumps( + { + "connector/valid": "v" * 32, + "connector/next": "n" * 32, + "connector/weak": "short", + "connector/blank": " " * 32, + } + ), + ) + get_settings.cache_clear() + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + app = FastAPI() + app.include_router(router, prefix="/api/v1") + + def override_db() -> Generator[Session, None, None]: + with factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: CurrentUserContext( + username="lifecycle-admin", + name="连接器管理员", + role_codes=["platform_admin"], + is_admin=True, + tenant_id="default", + ) + with factory() as db: + claim = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no="BX-CONFIG-LIFECYCLE-001", + employee_name="配置回归员工", + department_name="财务测试部", + expense_type="travel", + reason="连接器配置生命周期回归", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime.now(UTC), + submitted_at=datetime.now(UTC), + status="pending_payment", + approval_stage="待付款", + risk_flags_json=[], + ) + db.add(claim) + db.commit() + client = TestClient(app) + try: + yield client, claim + finally: + client.close() + app.dependency_overrides.clear() + Base.metadata.drop_all(engine) + engine.dispose() + get_settings.cache_clear() + + +def test_activate_rotate_disable_is_versioned_and_audited(lifecycle_http) -> None: + client, claim = lifecycle_http + bypass = client.post( + "/api/v1/financial-connectors/admin/tenants/default/configs", + json={ + "provider": "bypass-bank", + "environment": "production", + "key_version": "v1", + "secret_ref": "connector/valid", + "allowed_event_types": ["payment_settled"], + "status": "active", + "request_id": "lifecycle-bypass-001", + "reason": "创建时不得绕过显式激活和服务端密钥校验。", + }, + ) + assert bypass.status_code == 422 + created = _create_config(client, secret_ref="connector/valid") + config_id = created["id"] + assert created["status"] == "disabled" and created["version"] == 1 + + activated = client.post( + f"/api/v1/financial-connectors/admin/tenants/default/configs/{config_id}/activate", + json={ + "expected_version": 1, + "request_id": "lifecycle-activate-001", + "reason": "生产切换前完成服务端密钥解析与强度校验。", + }, + ) + assert activated.status_code == 200 + assert activated.json()["status"] == "active" + assert activated.json()["version"] == 2 + + stale = client.post( + f"/api/v1/financial-connectors/admin/tenants/default/configs/{config_id}/disable", + json={ + "expected_version": 1, + "request_id": "lifecycle-disable-stale-001", + "reason": "使用过期版本停用必须被拒绝。", + }, + ) + assert stale.status_code == 409 + + rotated = client.post( + f"/api/v1/financial-connectors/admin/tenants/default/configs/{config_id}/rotate", + json={ + "expected_version": 2, + "request_id": "lifecycle-rotate-001", + "reason": "按季度轮换服务端 HMAC 密钥。", + "new_key_version": "v2", + "new_secret_ref": "connector/next", + }, + ) + assert rotated.status_code == 200 + previous = rotated.json()["previous"] + replacement = rotated.json()["replacement"] + assert previous["status"] == "rotating" and previous["version"] == 3 + assert replacement["status"] == "active" and replacement["version"] == 1 + assert replacement["key_version"] == "v2" + + envelope = FinancialEventEnvelope( + tenant_id="default", + external_event_id="lifecycle-simulation-001", + event_type="payment_settled", + occurred_at=datetime.now(UTC), + correlation_id="lifecycle-correlation-001", + payload={ + "claim_id": claim.id, + "claim_reference": claim.claim_no, + "amount": "66.00", + "currency": "CNY", + "external_payment_reference": "LIFECYCLE-PAYMENT-001", + }, + ) + timestamp = int(time.time()) + old_response = client.post( + "/api/v1/integrations/financial-events", + json=envelope.model_dump(mode="json"), + headers=_headers(envelope, timestamp, version="v1", secret="v" * 32), + ) + assert old_response.status_code == 401 + assert old_response.json()["detail"]["code"] == "connector_inactive" + new_response = client.post( + "/api/v1/integrations/financial-events", + json=envelope.model_dump(mode="json"), + headers=_headers(envelope, timestamp, version="v2", secret="n" * 32), + ) + assert new_response.status_code == 200 + assert new_response.json()["projection_scope"] == "simulation_only" + assert new_response.json()["claim_status"] == "pending_payment" + + disabled = client.post( + f"/api/v1/financial-connectors/admin/tenants/default/configs/{config_id}/disable", + json={ + "expected_version": 3, + "request_id": "lifecycle-disable-001", + "reason": "新版本已稳定接管，停用旧轮换版本。", + }, + ) + assert disabled.status_code == 200 + assert disabled.json()["status"] == "disabled" + assert disabled.json()["version"] == 4 + + events = client.get( + "/api/v1/financial-connectors/admin/tenants/default/config-events" + ) + assert events.status_code == 200 + payload = events.json() + assert [item["action"] for item in payload] == [ + "created", + "activated", + "rotation_started", + "rotation_replacement_created", + "disabled", + ] + assert all(item["actor_id"] == "lifecycle-admin" for item in payload) + serialized = json.dumps(payload).lower() + assert "secret_ref" not in serialized + assert "connector/valid" not in serialized + assert "connector/next" not in serialized + + +@pytest.mark.parametrize( + ("secret_ref", "error_code"), + [ + ("connector/missing", "secret_unavailable"), + ("connector/weak", "secret_too_short"), + ("connector/blank", "secret_too_short"), + ], +) +def test_activate_fails_closed_when_server_secret_is_unusable( + lifecycle_http, + secret_ref: str, + error_code: str, +) -> None: + client, _ = lifecycle_http + created = _create_config( + client, + provider=f"invalid-{error_code}", + secret_ref=secret_ref, + request_id=f"create-{error_code}-001", + ) + response = client.post( + "/api/v1/financial-connectors/admin/tenants/default/configs/" + f"{created['id']}/activate", + json={ + "expected_version": 1, + "request_id": f"activate-{error_code}-001", + "reason": "不可用的服务端密钥不得进入激活态。", + }, + ) + assert response.status_code == 400 + assert response.json()["detail"]["code"] == error_code + configs = client.get( + "/api/v1/financial-connectors/admin/tenants/default/configs" + ).json() + current = next(item for item in configs if item["id"] == created["id"]) + assert current["status"] == "disabled" + assert current["version"] == 1 + + +def _create_config( + client: TestClient, + *, + provider: str = "lifecycle-bank", + secret_ref: str, + request_id: str = "lifecycle-create-001", +) -> dict[str, object]: + response = client.post( + "/api/v1/financial-connectors/admin/tenants/default/configs", + json={ + "provider": provider, + "environment": "mock", + "key_version": "v1", + "secret_ref": secret_ref, + "allowed_event_types": ["payment_settled"], + "clock_skew_seconds": 300, + "status": "disabled", + "request_id": request_id, + "reason": "创建连接器配置并等待显式激活。", + }, + ) + assert response.status_code == 201 + return response.json() + + +def _headers( + envelope: FinancialEventEnvelope, + timestamp: int, + *, + version: str, + secret: str, +) -> dict[str, str]: + return { + "X-Financial-Tenant": "default", + "X-Financial-Provider": "lifecycle-bank", + "X-Financial-Key-Version": version, + "X-Financial-Timestamp": str(timestamp), + "X-Financial-Signature": sign_financial_event( + envelope, + timestamp=timestamp, + secret=secret, + tenant_id="default", + provider="lifecycle-bank", + key_version=version, + ), + } diff --git a/server/tests/test_financial_connector_endpoints.py b/server/tests/test_financial_connector_endpoints.py new file mode 100644 index 0000000..0ec5a36 --- /dev/null +++ b/server/tests/test_financial_connector_endpoints.py @@ -0,0 +1,239 @@ +from __future__ import annotations + +import json +import time +import uuid +from collections.abc import Generator +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.api.v1.endpoints.financial_connectors import router +from app.core.config import get_settings +from app.db.base_class import Base +from app.models.financial_record import ExpenseClaim +from app.schemas.financial_connector import FinancialEventEnvelope +from app.services.financial_connector_auth import sign_financial_event + + +@pytest.fixture() +def http_context(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setenv( + "FINANCIAL_CONNECTOR_HMAC_KEYS_JSON", + json.dumps({"connector/http-test": "http-server-secret"}), + ) + get_settings.cache_clear() + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + app = FastAPI() + app.include_router(router, prefix="/api/v1") + user_box = {"current": _user("platform-admin", is_admin=True)} + + def override_db() -> Generator[Session, None, None]: + with factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: user_box["current"] + with factory() as db: + claim = _claim(db) + db.commit() + client = TestClient(app) + try: + yield client, user_box, claim + finally: + client.close() + app.dependency_overrides.clear() + Base.metadata.drop_all(engine) + engine.dispose() + get_settings.cache_clear() + + +def test_financial_connector_http_auth_config_and_reconciliation_scope(http_context) -> None: + client, user_box, claim = http_context + config = client.post( + "/api/v1/financial-connectors/admin/tenants/default/configs", + json={ + "provider": "http-bank", + "environment": "mock", + "key_version": "v1", + "secret_ref": "connector/http-test", + "allowed_event_types": ["payment_settled"], + "clock_skew_seconds": 300, + "status": "disabled", + "request_id": "http-config-create-001", + "reason": "创建 HTTP 回归测试连接器配置。", + }, + ) + assert config.status_code == 201 + assert config.json()["status"] == "disabled" + assert config.json()["version"] == 1 + assert "secret_ref" not in config.json() + assert "secret" not in json.dumps(config.json()) + activated = client.post( + "/api/v1/financial-connectors/admin/tenants/default/configs/" + f"{config.json()['id']}/activate", + json={ + "expected_version": 1, + "request_id": "http-config-activate-001", + "reason": "完成服务端密钥校验后启用模拟连接器。", + }, + ) + assert activated.status_code == 200 + assert activated.json()["status"] == "active" + assert activated.json()["version"] == 2 + + envelope = FinancialEventEnvelope( + tenant_id="default", + external_event_id="http-settlement-001", + event_type="payment_settled", + occurred_at=datetime.now(UTC), + correlation_id="http-correlation-001", + payload={ + "claim_id": claim.id, + "claim_reference": claim.claim_no, + "amount": "66.00", + "currency": "CNY", + "external_payment_reference": "HTTP-PAYMENT-00001", + }, + ) + timestamp = int(time.time()) + headers = { + "X-Financial-Tenant": "default", + "X-Financial-Provider": "http-bank", + "X-Financial-Key-Version": "v1", + "X-Financial-Timestamp": str(timestamp), + "X-Financial-Signature": sign_financial_event( + envelope, + timestamp=timestamp, + secret="http-server-secret", + tenant_id="default", + provider="http-bank", + key_version="v1", + ), + } + bad = client.post( + "/api/v1/integrations/financial-events", + json=envelope.model_dump(mode="json"), + headers={**headers, "X-Financial-Signature": "sha256=" + "0" * 64}, + ) + assert bad.status_code == 401 + accepted = client.post( + "/api/v1/integrations/financial-events", + json=envelope.model_dump(mode="json"), + headers=headers, + ) + replay = client.post( + "/api/v1/integrations/financial-events", + json=envelope.model_dump(mode="json"), + headers=headers, + ) + conflicting_envelope = envelope.model_copy( + update={"payload": {**envelope.payload, "amount": "66.01"}} + ) + conflicting_headers = { + **headers, + "X-Financial-Signature": sign_financial_event( + conflicting_envelope, + timestamp=timestamp, + secret="http-server-secret", + tenant_id="default", + provider="http-bank", + key_version="v1", + ), + } + conflict = client.post( + "/api/v1/integrations/financial-events", + json=conflicting_envelope.model_dump(mode="json"), + headers=conflicting_headers, + ) + assert accepted.status_code == 200 + assert accepted.json()["claim_status"] == "pending_payment" + assert accepted.json()["evidence_classification"] == "simulated_connector" + assert accepted.json()["projection_scope"] == "simulation_only" + assert accepted.json()["reconciliation_case_id"] is None + assert replay.status_code == 200 and replay.json()["replayed"] is True + assert conflict.status_code == 409 + + assert client.get("/api/v1/financial-reconciliation/cases").status_code == 403 + user_box["current"] = _user("finance-default", roles=["finance"]) + listed = client.get("/api/v1/financial-reconciliation/cases") + observed = client.get("/api/v1/financial-connectors/observability") + assert listed.status_code == 200 + assert listed.json()["total"] == 0 + assert observed.status_code == 200 + assert observed.json()["summary"]["retry_count"] == 1 + assert observed.json()["summary"]["auth_failure_count"] == 1 + assert observed.json()["summary"]["signature_failure_count"] == 1 + assert observed.json()["summary"]["payload_conflict_count"] == 1 + assert observed.json()["window_hours"] == 24 + assert observed.json()["source_revision"] == "20260716_0022" + assert observed.json()["summary"]["latest_replay_at"] is not None + assert observed.json()["summary"]["latest_auth_failure_at"] is not None + assert observed.json()["summary"]["latest_payload_conflict_at"] is not None + + user_box["current"] = _user("finance-other", tenant_id="tenant-other", roles=["finance"]) + assert client.get("/api/v1/financial-reconciliation/cases").json()["total"] == 0 + other_observability = client.get("/api/v1/financial-connectors/observability") + assert other_observability.json()["summary"]["auth_failure_count"] == 0 + assert other_observability.json()["summary"]["payload_conflict_count"] == 0 + + user_box["current"] = _user("platform-admin", is_admin=True) + events = client.get( + "/api/v1/financial-connectors/admin/tenants/default/config-events" + ) + assert events.status_code == 200 + assert [item["action"] for item in events.json()] == ["created", "activated"] + assert "secret" not in json.dumps(events.json()).lower() + + +def _claim(db: Session) -> ExpenseClaim: + row = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no="BX-CONNECTOR-HTTP-001", + employee_name="接口测试员工", + department_name="财务测试部", + expense_type="travel", + reason="接口验收", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime.now(UTC), + submitted_at=datetime.now(UTC), + status="pending_payment", + approval_stage="待付款", + risk_flags_json=[], + ) + db.add(row) + db.flush() + return row + + +def _user( + username: str, + *, + tenant_id: str = "default", + roles: list[str] | None = None, + is_admin: bool = False, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=is_admin, + tenant_id=tenant_id, + ) diff --git a/server/tests/test_financial_connector_mock_observability.py b/server/tests/test_financial_connector_mock_observability.py new file mode 100644 index 0000000..81dc6cc --- /dev/null +++ b/server/tests/test_financial_connector_mock_observability.py @@ -0,0 +1,449 @@ +from __future__ import annotations + +import json +import uuid +from collections.abc import Generator +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.api.v1.endpoints.financial_connectors import router +from app.core.config import get_settings +from app.db.base_class import Base +from app.models.expense_case import BusinessEvent +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorEvent, + PaymentReconciliationCase, +) +from app.models.financial_record import ExpenseClaim +from app.models.savings import SavingsRealization +from app.schemas.financial_connector import FinancialConnectorSimulationCreate +from app.services.expense_cases import ExpenseCaseService +from app.services.financial_connector_auth import FinancialConnectorSecretResolver +from app.services.financial_connector_mock_adapter import ( + FinancialConnectorMockAdapter, + FinancialConnectorMockAdapterError, +) +from app.services.financial_connector_observability import ( + FinancialConnectorObservabilityPermissionError, + FinancialConnectorObservabilityService, +) +from app.services.financial_connector_payment_evidence import ( + FinancialConnectorPaymentEvidenceService, +) + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_mock_adapter_covers_deterministic_scenarios_without_core_side_effects( + db: Session, +) -> None: + config = _config(db) + _config(db, provider=config.provider, key_version="v0", status="rotating") + claim = _claim(db) + db.commit() + adapter = FinancialConnectorMockAdapter( + db, + secrets=FinancialConnectorSecretResolver( + {"connector/runtime-test": "runtime-server-only-secret"} + ), + now=datetime(2026, 7, 16, 12, 0, tzinfo=UTC), + ) + expected_outcomes = { + "success": ["accepted"], + "failure": ["expected_exception"], + "out_of_order": ["expected_exception"], + "duplicate": ["accepted", "replayed"], + "conflict": ["accepted", "conflict"], + "refund": ["accepted", "accepted"], + "erp_receipt": ["accepted", "accepted"], + } + + results = {} + for scenario, outcomes in expected_outcomes.items(): + result = adapter.run( + tenant_id="default", + config_id=config.id, + payload=FinancialConnectorSimulationCreate( + claim_id=claim.id, + scenario=scenario, + request_id=f"runtime-{scenario}-request-001", + ), + ) + db.commit() + results[scenario] = result + assert [step.outcome for step in result.steps] == outcomes + assert result.projection_scope == "simulation_only" + assert result.core_side_effects_allowed is False + serialized = result.model_dump_json().lower() + assert '"payload":' not in serialized + assert '"signature":' not in serialized + + repeated = adapter.run( + tenant_id="default", + config_id=config.id, + payload=FinancialConnectorSimulationCreate( + claim_id=claim.id, + scenario="success", + request_id="runtime-success-request-001", + ), + ) + db.commit() + + assert repeated.request_fingerprint == results["success"].request_fingerprint + assert repeated.steps[0].outcome == "replayed" + assert db.scalar(select(func.count(FinancialConnectorEvent.id))) == 9 + assert db.scalar(select(func.count(PaymentReconciliationCase.id))) == 0 + assert db.scalar(select(func.count(BusinessEvent.id))) == 0 + assert db.scalar(select(func.count(SavingsRealization.id))) == 0 + db.refresh(claim) + assert claim.status == "pending_payment" + assert claim.approval_stage == "待付款" + assert claim.risk_flags_json == [] + assert all( + event.response_json["projection_scope"] == "simulation_only" + for event in db.scalars(select(FinancialConnectorEvent)).all() + ) + + +def test_mock_adapter_rejects_cross_tenant_production_and_inactive_configs( + db: Session, +) -> None: + tenant_a = _config(db, tenant_id="tenant-a", provider="tenant-a-bank") + tenant_b_claim = _claim(db, tenant_id="tenant-b") + production = _config(db, provider="production-bank", environment="production") + inactive = _config(db, provider="disabled-bank", status="disabled") + db.commit() + adapter = FinancialConnectorMockAdapter( + db, + secrets=FinancialConnectorSecretResolver( + {"connector/runtime-test": "runtime-server-only-secret"} + ), + ) + payload = FinancialConnectorSimulationCreate( + claim_id=tenant_b_claim.id, + scenario="success", + request_id="runtime-boundary-request-001", + ) + + with pytest.raises(LookupError, match="报销单不存在"): + adapter.run(tenant_id="tenant-a", config_id=tenant_a.id, payload=payload) + with pytest.raises(FinancialConnectorMockAdapterError, match="生产"): + adapter.run(tenant_id="default", config_id=production.id, payload=payload) + with pytest.raises(FinancialConnectorMockAdapterError, match="激活"): + adapter.run(tenant_id="default", config_id=inactive.id, payload=payload) + + +def test_observability_aggregates_real_facts_and_durable_zero_metrics( + db: Session, +) -> None: + config = _config(db) + claim = _claim(db) + adapter = FinancialConnectorMockAdapter( + db, + secrets=FinancialConnectorSecretResolver( + {"connector/runtime-test": "runtime-server-only-secret"} + ), + now=datetime(2026, 7, 16, 12, 0, tzinfo=UTC), + ) + for scenario in ("success", "failure", "out_of_order"): + adapter.run( + tenant_id="default", + config_id=config.id, + payload=FinancialConnectorSimulationCreate( + claim_id=claim.id, + scenario=scenario, + request_id=f"runtime-observe-{scenario}-001", + ), + ) + db.add( + PaymentReconciliationCase( + id=str(uuid.uuid4()), + tenant_id="default", + provider=config.provider, + claim_id=claim.id, + expected_amount=claim.amount, + actual_amount=claim.amount, + amount_difference=Decimal("0.00"), + expected_currency="CNY", + actual_currency="CNY", + expected_reference=claim.claim_no, + status="exception", + exception_code="test_anomaly", + erp_status="pending_posting", + last_connector_event_id="test-observability-event", + version=1, + created_at=datetime(2026, 7, 16, 12, 0, tzinfo=UTC), + updated_at=datetime(2026, 7, 16, 12, 0, tzinfo=UTC), + ) + ) + db.commit() + + service = FinancialConnectorObservabilityService( + db, + now=datetime(2026, 7, 16, 12, 5, tzinfo=UTC), + ) + result = service.read_for_current_user(_user("finance", roles=["finance"]), window_hours=24) + other = service.read_for_tenant("tenant-other", window_hours=24) + + assert result.summary.event_count == 3 + assert result.summary.processed_event_count == 1 + assert result.summary.failed_event_count == 2 + assert result.summary.failure_rate == pytest.approx(0.6667) + assert result.summary.backlog_count == 0 + assert result.summary.reconciliation_anomaly_count == 1 + assert result.summary.retry_count == 0 + assert result.summary.auth_failure_count == 0 + assert result.summary.signature_failure_count == 0 + assert result.summary.payload_conflict_count == 0 + assert result.retry_metric.status == "available" + assert result.auth_failure_metric.status == "available" + assert result.signature_failure_metric.status == "available" + assert result.payload_conflict_metric.status == "available" + assert result.source_revision == "20260716_0022" + assert result.as_of == datetime(2026, 7, 16, 12, 5, tzinfo=UTC) + assert all(item.evidence_classification == "simulated_connector" for item in result.items) + assert sum(item.reconciliation_anomaly_count for item in result.items) == 1 + assert other.summary.event_count == 0 + assert other.items == [] + with pytest.raises(FinancialConnectorObservabilityPermissionError): + service.read_for_current_user(_user("ordinary"), window_hours=24) + + +def test_payment_evidence_distinguishes_manual_external_and_unpaid() -> None: + manual = _claim_object(status="paid") + manual.risk_flags_json = [ + { + "source": "payment", + "event_type": "expense_claim_payment_completed", + "created_at": "2026-07-16T10:00:00+00:00", + } + ] + external = _claim_object(status="paid") + external.risk_flags_json = [ + { + "source": "external_payment", + "event_type": "expense_claim_external_payment_settled", + "provider": "verified-bank", + "verification_level": "production_verified", + "evidence_classification": "external_cash", + "external_reference_tail": "12345678", + "created_at": "2026-07-16T11:00:00+00:00", + } + ] + unpaid = _claim_object(status="pending_payment") + + manual_read = FinancialConnectorPaymentEvidenceService.read(manual) + external_read = FinancialConnectorPaymentEvidenceService.read(external) + unpaid_read = FinancialConnectorPaymentEvidenceService.read(unpaid) + + assert manual_read.evidence_classification == "internal_manual_payment" + assert manual_read.trust_level == "low" + assert external_read.evidence_classification == "external_cash" + assert external_read.trust_level == "high" + assert external_read.external_reference_tail == "12345678" + assert unpaid_read.evidence_classification == "none" + + +@pytest.fixture() +def http_context(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setenv( + "FINANCIAL_CONNECTOR_HMAC_KEYS_JSON", + json.dumps({"connector/runtime-test": "runtime-server-only-secret"}), + ) + get_settings.cache_clear() + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + config = _config(session) + claim = _claim(session) + claim.status = "paid" + claim.approval_stage = "已付款" + claim.risk_flags_json = [ + { + "source": "payment", + "event_type": "expense_claim_payment_completed", + "created_at": "2026-07-16T10:00:00+00:00", + } + ] + session.commit() + app = FastAPI() + app.include_router(router, prefix="/api/v1") + user_box = {"current": _user("platform-admin", is_admin=True)} + + def override_db() -> Generator[Session, None, None]: + with factory() as session: + yield session + + app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: user_box["current"] + client = TestClient(app) + try: + yield client, user_box, config.id, claim.id + finally: + client.close() + app.dependency_overrides.clear() + Base.metadata.drop_all(engine) + engine.dispose() + get_settings.cache_clear() + + +def test_runtime_http_permissions_tenant_scope_and_replay(http_context) -> None: + client, user_box, config_id, claim_id = http_context + simulation = client.post( + f"/api/v1/financial-connectors/admin/tenants/default/configs/{config_id}/simulate", + json={ + "claim_id": claim_id, + "scenario": "success", + "request_id": "runtime-http-simulation-001", + }, + ) + replay = client.post( + f"/api/v1/financial-connectors/admin/tenants/default/configs/{config_id}/simulate", + json={ + "claim_id": claim_id, + "scenario": "success", + "request_id": "runtime-http-simulation-001", + }, + ) + assert simulation.status_code == 200 + assert simulation.json()["core_side_effects_allowed"] is False + assert replay.json()["steps"][0]["outcome"] == "replayed" + + user_box["current"] = _user("finance", roles=["finance"]) + observed = client.get("/api/v1/financial-connectors/observability?window_hours=24") + evidence = client.get(f"/api/v1/financial-connectors/payment-evidence/{claim_id}") + assert observed.status_code == 200 + assert observed.json()["summary"]["event_count"] == 1 + assert observed.json()["summary"]["retry_count"] == 1 + assert observed.json()["retry_metric"]["status"] == "available" + assert observed.json()["source_revision"] == "20260716_0022" + assert evidence.status_code == 200 + assert evidence.json()["evidence_classification"] == "internal_manual_payment" + + forbidden_simulation = client.post( + f"/api/v1/financial-connectors/admin/tenants/default/configs/{config_id}/simulate", + json={ + "claim_id": claim_id, + "scenario": "success", + "request_id": "runtime-http-forbidden-001", + }, + ) + assert forbidden_simulation.status_code == 403 + + user_box["current"] = _user("other-finance", tenant_id="tenant-other", roles=["finance"]) + assert ( + client.get(f"/api/v1/financial-connectors/payment-evidence/{claim_id}").status_code == 404 + ) + assert ( + client.get("/api/v1/financial-connectors/observability").json()["tenant_id"] + == "tenant-other" + ) + + user_box["current"] = _user("ordinary") + assert client.get("/api/v1/financial-connectors/observability").status_code == 403 + + +def _config( + db: Session, + *, + tenant_id: str = "default", + provider: str = "runtime-bank", + environment: str = "mock", + status: str = "active", + key_version: str = "v1", +) -> FinancialConnectorConfig: + row = FinancialConnectorConfig( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + provider=provider, + environment=environment, + key_version=key_version, + secret_ref="connector/runtime-test", + allowed_event_types_json=[ + "payment_settled", + "payment_failed", + "erp_posted", + "erp_posting_failed", + "payment_refunded", + "payment_reversed", + ], + clock_skew_seconds=300, + status=status, + created_by="platform-admin", + ) + db.add(row) + db.flush() + return row + + +def _claim(db: Session, *, tenant_id: str = "default") -> ExpenseClaim: + row = _claim_object() + db.add(row) + db.flush() + if tenant_id != "default": + ExpenseCaseService(db).ensure_case_for_claim(row, tenant_id=tenant_id) + return row + + +def _claim_object(*, status: str = "pending_payment") -> ExpenseClaim: + return ExpenseClaim( + id=str(uuid.uuid4()), + claim_no=f"BX-RUNTIME-{uuid.uuid4().hex[:10].upper()}", + employee_name="连接器运行测试员工", + department_name="财务测试部", + expense_type="travel", + reason="连接器运行测试", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime.now(UTC), + submitted_at=datetime.now(UTC), + status=status, + approval_stage="待付款" if status == "pending_payment" else "已付款", + risk_flags_json=[], + ) + + +def _user( + username: str, + *, + tenant_id: str = "default", + roles: list[str] | None = None, + is_admin: bool = False, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=is_admin, + tenant_id=tenant_id, + ) diff --git a/server/tests/test_financial_connector_operational_events.py b/server/tests/test_financial_connector_operational_events.py new file mode 100644 index 0000000..1337b44 --- /dev/null +++ b/server/tests/test_financial_connector_operational_events.py @@ -0,0 +1,211 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, datetime, timedelta + +import pytest +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.db.base_class import Base +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorOperationalEvent, +) +from app.schemas.financial_connector import FinancialEventEnvelope +from app.services.financial_connector_auth import ( + FinancialConnectorAuthenticator, + FinancialConnectorAuthError, + FinancialConnectorSecretResolver, +) +from app.services.financial_connector_observability import ( + FinancialConnectorObservabilityService, +) +from app.services.financial_connector_operational_events import ( + FinancialConnectorOperationalContext, + FinancialConnectorOperationalEventService, + operational_event_candidate, +) + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_operational_attempts_are_counted_but_candidate_retry_is_idempotent( + db: Session, +) -> None: + config = _config(db) + context = _context(config) + as_of = datetime(2026, 7, 17, 10, 2, tzinfo=UTC) + first = operational_event_candidate( + context, + event_type="replay", + reason_code="duplicate_external_event", + occurred_at=as_of - timedelta(minutes=2), + ) + second = operational_event_candidate( + context, + event_type="replay", + reason_code="duplicate_external_event", + occurred_at=as_of - timedelta(minutes=1), + ) + outside_window = operational_event_candidate( + context, + event_type="auth_failure", + reason_code="signature_invalid", + occurred_at=as_of - timedelta(hours=2), + ) + service = FinancialConnectorOperationalEventService(db) + + service.record(first) + service.record(first) + service.record(second) + service.record(outside_window) + db.commit() + + assert first.idempotency_key != second.idempotency_key + assert db.scalar(select(func.count(FinancialConnectorOperationalEvent.id))) == 3 + observed = FinancialConnectorObservabilityService(db, now=as_of).read_for_tenant( + config.tenant_id, + window_hours=1, + ) + assert observed.window_started_at == as_of - timedelta(hours=1) + assert observed.as_of == as_of + assert observed.source_revision == "20260716_0022" + assert observed.summary.retry_count == 2 + assert observed.summary.auth_failure_count == 0 + assert observed.summary.latest_replay_at == second.occurred_at + + +def test_auth_failure_is_attributed_only_after_trusted_config_and_secret_resolution( + db: Session, +) -> None: + config = _config(db) + secret = "trusted-server-secret" + now_epoch = 1_800_000_000 + envelope = FinancialEventEnvelope( + tenant_id=config.tenant_id, + external_event_id="raw-external-event-must-not-be-stored", + event_type="payment_settled", + occurred_at=datetime.fromtimestamp(now_epoch, UTC), + correlation_id="raw-correlation-must-not-be-stored", + payload={ + "claim_id": str(uuid.uuid4()), + "claim_reference": "BX-RAW-MUST-NOT-BE-STORED", + "amount": "66.00", + "currency": "CNY", + "external_payment_reference": "RAW-PAYMENT-MUST-NOT-BE-STORED", + }, + ) + authenticator = FinancialConnectorAuthenticator( + db, + secrets=FinancialConnectorSecretResolver({config.secret_ref: secret}), + now_epoch=now_epoch, + ) + + with pytest.raises(FinancialConnectorAuthError) as forged_tenant: + authenticator.verify( + envelope, + tenant_header="tenant-forged", + provider_header=config.provider, + key_version_header=config.key_version, + timestamp_header=str(now_epoch), + signature_header="sha256=" + "0" * 64, + ) + assert forged_tenant.value.operational_context is None + + with pytest.raises(FinancialConnectorAuthError) as forged_provider: + authenticator.verify( + envelope, + tenant_header=config.tenant_id, + provider_header="provider-forged", + key_version_header=config.key_version, + timestamp_header=str(now_epoch), + signature_header="sha256=" + "0" * 64, + ) + assert forged_provider.value.operational_context is None + + with pytest.raises(FinancialConnectorAuthError) as trusted_failure: + authenticator.verify( + envelope, + tenant_header=config.tenant_id, + provider_header=config.provider, + key_version_header=config.key_version, + timestamp_header=str(now_epoch), + signature_header="sha256=" + "0" * 64, + ) + context = trusted_failure.value.operational_context + assert context is not None + assert context.request_fingerprint.startswith("hmac-sha256:") + assert context.external_event_fingerprint.startswith("hmac-sha256:") + + candidate = operational_event_candidate( + context, + event_type="auth_failure", + reason_code=trusted_failure.value.code, + occurred_at=datetime.fromtimestamp(now_epoch, UTC), + ) + FinancialConnectorOperationalEventService(db).record(candidate) + db.commit() + row = db.scalar(select(FinancialConnectorOperationalEvent)) + assert row is not None + stored = "|".join( + ( + row.provider, + row.reason_code, + row.request_fingerprint, + row.external_event_fingerprint, + row.idempotency_key, + ) + ) + for raw_value in ( + secret, + envelope.external_event_id, + envelope.correlation_id, + envelope.payload["claim_reference"], + envelope.payload["external_payment_reference"], + ): + assert str(raw_value) not in stored + + +def _config(db: Session) -> FinancialConnectorConfig: + row = FinancialConnectorConfig( + id=str(uuid.uuid4()), + tenant_id="tenant-operational", + provider="operational-bank", + environment="production", + key_version="v1", + secret_ref="connector/operational-test", + allowed_event_types_json=["payment_settled"], + clock_skew_seconds=300, + status="active", + created_by="operational-test", + ) + db.add(row) + db.flush() + return row + + +def _context(config: FinancialConnectorConfig) -> FinancialConnectorOperationalContext: + return FinancialConnectorOperationalContext( + tenant_id=config.tenant_id, + config_id=config.id, + provider=config.provider, + environment=config.environment, + request_fingerprint="hmac-sha256:" + "a" * 64, + external_event_fingerprint="hmac-sha256:" + "b" * 64, + ) diff --git a/server/tests/test_financial_connector_services.py b/server/tests/test_financial_connector_services.py new file mode 100644 index 0000000..a5d4670 --- /dev/null +++ b/server/tests/test_financial_connector_services.py @@ -0,0 +1,788 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, date, datetime +from decimal import Decimal + +import pytest +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext +from app.db.base_class import Base +from app.models.expense_case import BusinessEvent +from app.models.financial_connector import ( + FinancialConnectorConfig, + FinancialConnectorEvent, + PaymentReconciliationCase, + PaymentReconciliationEvent, +) +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.savings import SavingsEvidenceLink, SavingsRealization +from app.schemas.financial_connector import ( + FinancialEventEnvelope, + PaymentReconciliationActionCreate, +) +from app.schemas.savings import SavingsRealizationActionCreate +from app.services.expense_cases import ExpenseCaseService +from app.services.financial_connector_auth import ( + FinancialConnectorAuthError, + FinancialConnectorSecretResolver, + sign_financial_event, +) +from app.services.financial_connector_ingestion import ( + FinancialConnectorConflictError, + FinancialConnectorIngestionService, +) +from app.services.financial_connector_projection import ( + FinancialConnectorProjectionService, + FinancialReconciliationPermissionError, +) +from app.services.savings_discovery import SavingsDiscoveryService +from app.services.savings_realization import SavingsRealizationService + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_hmac_is_tenant_bound_time_bounded_and_secret_never_persisted(db: Session) -> None: + _config(db, tenant_id="tenant-a", environment="mock") + claim = _claim(db, tenant_id="tenant-a") + envelope = _event(claim, tenant_id="tenant-a", event_id="settled-auth-001") + timestamp = 1_800_000_000 + signature = sign_financial_event( + envelope, + timestamp=timestamp, + secret="server-only-secret", + tenant_id="tenant-a", + provider="mock-bank", + key_version="v1", + ) + + with pytest.raises(FinancialConnectorAuthError, match="租户"): + _service(db, timestamp).ingest( + envelope, + tenant_header="tenant-b", + provider_header="mock-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header=signature, + ) + with pytest.raises(FinancialConnectorAuthError) as expired: + _service(db, timestamp + 301).ingest( + envelope, + tenant_header="tenant-a", + provider_header="mock-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header=signature, + ) + assert expired.value.code == "timestamp_outside_window" + with pytest.raises(FinancialConnectorAuthError) as invalid: + _service(db, timestamp).ingest( + envelope, + tenant_header="tenant-a", + provider_header="mock-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header="sha256=" + "0" * 64, + ) + assert invalid.value.code == "signature_invalid" + config = db.scalar(select(FinancialConnectorConfig)) + assert config is not None + assert config.secret_ref == "connector/test-key" + assert "server-only-secret" not in str(config.__dict__) + with pytest.raises(ValueError, match="未允许字段"): + FinancialEventEnvelope( + **{ + **envelope.model_dump(), + "payload": {**envelope.payload, "bank_account_number": "sensitive"}, + } + ) + + +def test_hmac_binds_provider_and_key_version_even_when_secret_is_shared( + db: Session, +) -> None: + _config(db, tenant_id="tenant-a", provider="mock-bank", key_version="v1") + _config(db, tenant_id="tenant-a", provider="alias-bank", key_version="v2") + claim = _claim(db, tenant_id="tenant-a") + envelope = _event(claim, tenant_id="tenant-a", event_id="source-bound-001") + timestamp = 1_800_000_000 + signature = sign_financial_event( + envelope, + timestamp=timestamp, + secret="server-only-secret", + tenant_id="tenant-a", + provider="mock-bank", + key_version="v1", + ) + service = _service(db, timestamp) + + with pytest.raises(FinancialConnectorAuthError) as replayed_to_alias: + service.ingest( + envelope, + tenant_header="tenant-a", + provider_header="alias-bank", + key_version_header="v2", + timestamp_header=str(timestamp), + signature_header=signature, + ) + + assert replayed_to_alias.value.code == "signature_invalid" + + +def test_settlement_replay_erp_reversal_and_savings_evidence_e2e(db: Session) -> None: + _config(db, environment="production") + claim = _claim(db) + application = _linked_application(db, claim) + _savings_opportunity(db, claim) + db.commit() + timestamp = 1_800_000_000 + settlement = _event(claim, event_id="settled-e2e-001") + + first = _ingest(db, settlement, timestamp) + db.commit() + replay = _ingest(db, settlement, timestamp) + db.commit() + db.refresh(claim) + assert first.replayed is False + assert replay.replayed is True + assert first.event_id == replay.event_id + assert first.processing_status == "processed" + assert first.verification_level == "production_verified" + assert first.evidence_classification == "external_cash" + assert first.projection_scope == "canonical" + assert claim.status == "paid" + db.refresh(application) + assert application.approval_stage == "申请归档" + assert db.scalar(select(func.count(FinancialConnectorEvent.id))) == 1 + assert db.scalar(select(func.count(PaymentReconciliationEvent.id))) == 1 + connector_fact = db.get(FinancialConnectorEvent, first.event_id) + assert connector_fact is not None + assert "claim_reference" not in connector_fact.normalized_payload_json + assert claim.claim_no not in str(connector_fact.normalized_payload_json) + + realization = db.scalar( + select(SavingsRealization).where(SavingsRealization.claim_id == claim.id) + ) + assert realization is not None + evidence = db.scalar( + select(SavingsEvidenceLink).where( + SavingsEvidenceLink.realization_id == realization.id, + SavingsEvidenceLink.resource_type == "financial_connector_event", + ) + ) + assert evidence is not None + assert evidence.resource_id == first.event_id + assert evidence.source_system == "mock-bank" + assert evidence.metadata_json["evidence_level"] == "external_cash" + SavingsRealizationService(db).execute_action( + realization.id, + SavingsRealizationActionCreate( + action="confirm", + request_id="connector-confirm-001", + expected_version=1, + comment="独立财务已核验生产连接器回执与政策基线。", + ), + _user("independent-confirmer", roles=["finance"]), + ) + + invalid_erp = _event( + claim, + event_id="erp-e2e-invalid-001", + event_type="erp_posted", + extra={ + "amount": "65.00", + "origin_external_event_id": settlement.external_event_id, + "erp_document_number": "ERP-VOUCHER-INVALID", + }, + ) + invalid_erp_result = _ingest(db, invalid_erp, timestamp + 1) + db.commit() + db.refresh(claim) + assert invalid_erp_result.processing_status == "exception" + assert invalid_erp_result.error_code == "erp_amount_mismatch" + assert claim.status == "paid" + assert "erp_posted" not in set(db.scalars(select(BusinessEvent.event_type)).all()) + + erp = _event( + claim, + event_id="erp-e2e-001", + event_type="erp_posted", + extra={ + "origin_external_event_id": settlement.external_event_id, + "erp_document_number": "ERP-VOUCHER-2026-000991", + "accounting_period": "2026-07", + }, + ) + erp = FinancialEventEnvelope.model_validate( + { + **erp.model_dump(), + "payload": { + key: value + for key, value in erp.payload.items() + if key != "external_payment_reference" + }, + } + ) + erp_result = _ingest(db, erp, timestamp + 2) + db.commit() + db.refresh(claim) + case = db.get(PaymentReconciliationCase, first.reconciliation_case_id) + assert erp_result.processing_status == "processed" + assert claim.status == "paid" + assert case is not None and case.erp_status == "posted" + assert case.erp_document_tail == "6-000991" + assert "ERP-VOUCHER" not in str(case.__dict__) + + reversal = _event( + claim, + event_id="refund-e2e-001", + event_type="payment_refunded", + extra={"origin_external_event_id": settlement.external_event_id}, + ) + reversed_result = _ingest(db, reversal, timestamp + 3) + db.commit() + db.refresh(claim) + db.refresh(case) + assert reversed_result.processing_status == "processed" + assert claim.status == "pending_payment" + assert case.status == "reopened" + db.refresh(realization) + reversal_realization = db.scalar( + select(SavingsRealization).where( + SavingsRealization.reversal_of_realization_id == realization.id + ) + ) + assert realization.reversed_at is not None + assert reversal_realization is not None + assert reversal_realization.actual_net == -realization.actual_net + assert any( + item.get("role") == "external_payment_reversal" + and item.get("verification_status") == "verified" + for item in reversal_realization.evidence_json + ) + event_types = set(db.scalars(select(BusinessEvent.event_type)).all()) + assert {"payment_completed", "erp_posted", "payment_reversed"} <= event_types + second_settlement = _event(claim, event_id="settled-e2e-002") + second_result = _ingest(db, second_settlement, timestamp + 4) + db.commit() + db.refresh(claim) + db.refresh(case) + assert second_result.processing_status == "processed" + assert claim.status == "paid" + assert case.status == "matched" + assert case.erp_status == "pending_posting" + assert db.scalar(select(func.count(FinancialConnectorEvent.id))) == 5 + + +def test_mismatch_failure_and_conflicting_replay_have_no_payment_side_effect(db: Session) -> None: + _config(db) + claim = _claim(db) + db.commit() + timestamp = 1_800_000_000 + mismatch = _event( + claim, + event_id="settled-mismatch-001", + extra={"amount": "999.00"}, + ) + result = _ingest(db, mismatch, timestamp) + db.commit() + db.refresh(claim) + assert result.processing_status == "exception" + assert result.error_code == "amount_mismatch" + assert claim.status == "pending_payment" + assert db.scalar(select(func.count(BusinessEvent.id))) == 0 + + conflict = _event( + claim, + event_id="settled-mismatch-001", + extra={"amount": "998.00"}, + ) + with pytest.raises(FinancialConnectorConflictError): + _ingest(db, conflict, timestamp + 1) + db.rollback() + db.refresh(claim) + assert claim.status == "pending_payment" + assert db.scalar(select(func.count(FinancialConnectorEvent.id))) == 1 + + failed = _event( + claim, + event_id="payment-failed-001", + event_type="payment_failed", + extra={"failure_code": "provider_timeout"}, + ) + failed_result = _ingest(db, failed, timestamp + 2) + db.commit() + db.refresh(claim) + assert failed_result.error_code == "external_payment_failed" + assert claim.status == "pending_payment" + + +@pytest.mark.parametrize("environment", ["test", "mock", "staging"]) +def test_non_production_events_only_create_simulation_facts( + db: Session, + environment: str, +) -> None: + _config(db, environment=environment) + claim = _claim(db) + application = _linked_application(db, claim) + _savings_opportunity(db, claim) + db.commit() + initial_business_event_count = int( + db.scalar(select(func.count(BusinessEvent.id))) or 0 + ) + timestamp = 1_800_000_000 + settlement = _event(claim, event_id="pending-saving-settled-001") + settlement_result = _ingest(db, settlement, timestamp) + db.commit() + db.refresh(claim) + db.refresh(application) + assert settlement_result.processing_status == "processed" + assert settlement_result.projection_scope == "simulation_only" + assert settlement_result.reconciliation_case_id is None + assert settlement_result.claim_status == "pending_payment" + assert claim.status == "pending_payment" + assert application.approval_stage == "审批完成" + + failed = _event( + claim, + event_id="pending-saving-failed-001", + event_type="payment_failed", + extra={"failure_code": "simulated_provider_failure"}, + ) + failed_result = _ingest(db, failed, timestamp + 1) + db.commit() + assert failed_result.processing_status == "exception" + assert failed_result.error_code == "external_payment_failed" + assert failed_result.projection_scope == "simulation_only" + + erp = _event( + claim, + event_id="pending-saving-erp-001", + event_type="erp_posted", + extra={ + "origin_external_event_id": settlement.external_event_id, + "erp_document_number": "ERP-SIMULATED-001", + }, + ) + erp_result = _ingest(db, erp, timestamp + 2) + db.commit() + assert erp_result.processing_status == "processed" + assert erp_result.projection_scope == "simulation_only" + + erp_failed = _event( + claim, + event_id="pending-saving-erp-failed-001", + event_type="erp_posting_failed", + extra={ + "origin_external_event_id": settlement.external_event_id, + "failure_code": "simulated_erp_failure", + }, + ) + erp_failed_result = _ingest(db, erp_failed, timestamp + 3) + db.commit() + assert erp_failed_result.processing_status == "exception" + assert erp_failed_result.error_code == "erp_posting_failed" + assert erp_failed_result.projection_scope == "simulation_only" + + refund = _event( + claim, + event_id="pending-saving-refund-001", + event_type="payment_refunded", + extra={"origin_external_event_id": settlement.external_event_id}, + ) + refund_result = _ingest(db, refund, timestamp + 4) + db.commit() + db.refresh(claim) + assert refund_result.processing_status == "processed" + assert refund_result.projection_scope == "simulation_only" + reversal = _event( + claim, + event_id="pending-saving-reversal-001", + event_type="payment_reversed", + extra={"origin_external_event_id": settlement.external_event_id}, + ) + reversal_result = _ingest(db, reversal, timestamp + 5) + db.commit() + assert reversal_result.processing_status == "processed" + assert reversal_result.projection_scope == "simulation_only" + assert claim.status == "pending_payment" + assert db.scalar(select(func.count(PaymentReconciliationCase.id))) == 0 + assert db.scalar(select(func.count(PaymentReconciliationEvent.id))) == 0 + assert db.scalar(select(func.count(BusinessEvent.id))) == initial_business_event_count + assert db.scalar(select(func.count(SavingsRealization.id))) == 0 + facts = list(db.scalars(select(FinancialConnectorEvent)).all()) + assert len(facts) == 6 + assert all(item.expense_case_id is None for item in facts) + assert all("claim_reference" not in item.normalized_payload_json for item in facts) + + +def test_cross_tenant_claim_is_hidden_and_finance_actions_are_separated(db: Session) -> None: + _config(db, tenant_id="tenant-a") + _config(db, tenant_id="tenant-b", secret_ref="connector/tenant-b") + claim = _claim(db, tenant_id="tenant-a", employee_name="申请人甲") + db.commit() + timestamp = 1_800_000_000 + cross = _event(claim, tenant_id="tenant-b", event_id="cross-tenant-001") + signature = sign_financial_event( + cross, + timestamp=timestamp, + secret="tenant-b-secret-strong", + tenant_id="tenant-b", + provider="mock-bank", + key_version="v1", + ) + result = FinancialConnectorIngestionService( + db, + secrets=FinancialConnectorSecretResolver( + { + "connector/test-key": "server-only-secret", + "connector/tenant-b": "tenant-b-secret-strong", + } + ), + now_epoch=timestamp, + ).ingest( + cross, + tenant_header="tenant-b", + provider_header="mock-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header=signature, + ) + db.commit() + db.refresh(claim) + assert result.error_code == "claim_not_found" + assert result.reconciliation_case_id is None + assert claim.status == "pending_payment" + + finance = _user("finance-a", tenant_id="tenant-a", roles=["finance"]) + outsider = _user("employee-a", tenant_id="tenant-a") + with pytest.raises(FinancialReconciliationPermissionError): + FinancialConnectorProjectionService(db).list_cases( + outsider, + status_filter=None, + page=1, + page_size=20, + ) + tenant_a_cases = FinancialConnectorProjectionService(db).list_cases( + finance, + status_filter=None, + page=1, + page_size=20, + ) + assert tenant_a_cases.total == 0 + + +def test_production_reversal_cannot_reference_simulation_origin(db: Session) -> None: + _config(db, environment="mock", key_version="v1") + _config(db, environment="production", key_version="v2") + claim = _claim(db) + db.commit() + timestamp = 1_800_000_000 + simulated = _event(claim, event_id="simulated-origin-001") + simulated_result = _ingest(db, simulated, timestamp) + db.commit() + assert simulated_result.projection_scope == "simulation_only" + + reversal = _event( + claim, + event_id="production-reversal-001", + event_type="payment_reversed", + extra={"origin_external_event_id": simulated.external_event_id}, + ) + signature = sign_financial_event( + reversal, + timestamp=timestamp + 1, + secret="server-only-secret", + tenant_id="default", + provider="mock-bank", + key_version="v2", + ) + result = FinancialConnectorIngestionService( + db, + secrets=FinancialConnectorSecretResolver( + {"connector/test-key": "server-only-secret"} + ), + now_epoch=timestamp + 1, + ).ingest( + reversal, + tenant_header="default", + provider_header="mock-bank", + key_version_header="v2", + timestamp_header=str(timestamp + 1), + signature_header=signature, + ) + db.commit() + db.refresh(claim) + assert result.projection_scope == "canonical" + assert result.error_code == "origin_settlement_not_found" + assert claim.status == "pending_payment" + assert result.reconciliation_case_id is None + + +def test_applicant_cannot_confirm_own_exception(db: Session) -> None: + _config(db, environment="production") + employee_id = str(uuid.uuid4()) + claim = _claim(db, employee_id=employee_id, employee_name="申请人甲") + db.commit() + mismatch = _event(claim, event_id="own-exception-001", extra={"amount": "2.00"}) + result = _ingest(db, mismatch, 1_800_000_000) + db.commit() + case = db.get(PaymentReconciliationCase, result.reconciliation_case_id) + assert case is not None + applicant_finance = _user( + "applicant-finance", + roles=["finance"], + employee_id=employee_id, + name="申请人甲", + ) + with pytest.raises(FinancialReconciliationPermissionError): + FinancialConnectorProjectionService(db).resolve( + case.id, + PaymentReconciliationActionCreate( + expected_version=case.version, + reason="本人尝试确认自己的异常", + ), + applicant_finance, + action="confirmed", + ) + db.rollback() + db.refresh(case) + independent_finance = _user("independent-finance", roles=["finance"]) + resolved = FinancialConnectorProjectionService(db).resolve( + case.id, + PaymentReconciliationActionCreate( + expected_version=case.version, + reason="独立财务确认该差异仅作为处置结论", + ), + independent_finance, + action="confirmed", + ) + db.commit() + db.refresh(claim) + assert resolved is not None and resolved.status == "confirmed" + assert claim.status == "pending_payment" + + +def _service(db: Session, timestamp: int) -> FinancialConnectorIngestionService: + return FinancialConnectorIngestionService( + db, + secrets=FinancialConnectorSecretResolver( + {"connector/test-key": "server-only-secret"} + ), + now_epoch=timestamp, + ) + + +def _ingest( + db: Session, + envelope: FinancialEventEnvelope, + timestamp: int, +): + signature = sign_financial_event( + envelope, + timestamp=timestamp, + secret="server-only-secret", + tenant_id=envelope.tenant_id, + provider="mock-bank", + key_version="v1", + ) + return _service(db, timestamp).ingest( + envelope, + tenant_header=envelope.tenant_id, + provider_header="mock-bank", + key_version_header="v1", + timestamp_header=str(timestamp), + signature_header=signature, + ) + + +def _config( + db: Session, + *, + tenant_id: str = "default", + environment: str = "mock", + secret_ref: str = "connector/test-key", + provider: str = "mock-bank", + key_version: str = "v1", +) -> FinancialConnectorConfig: + row = FinancialConnectorConfig( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + provider=provider, + environment=environment, + key_version=key_version, + secret_ref=secret_ref, + allowed_event_types_json=[ + "payment_settled", + "payment_failed", + "erp_posted", + "erp_posting_failed", + "payment_refunded", + "payment_reversed", + ], + clock_skew_seconds=300, + status="active", + created_by="platform-admin", + ) + db.add(row) + db.flush() + return row + + +def _claim( + db: Session, + *, + tenant_id: str = "default", + employee_id: str | None = None, + employee_name: str = "测试员工", +) -> ExpenseClaim: + claim = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no=f"BX-CONNECTOR-{uuid.uuid4().hex[:10].upper()}", + employee_id=employee_id, + employee_name=employee_name, + department_name="财务测试部", + expense_type="travel", + reason="连接器端到端测试", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime.now(UTC), + submitted_at=datetime.now(UTC), + status="pending_payment", + approval_stage="待付款", + risk_flags_json=[], + ) + db.add(claim) + db.flush() + if tenant_id != "default": + ExpenseCaseService(db).ensure_case_for_claim(claim, tenant_id=tenant_id) + return claim + + +def _event( + claim: ExpenseClaim, + *, + tenant_id: str = "default", + event_id: str, + event_type: str = "payment_settled", + extra: dict[str, str] | None = None, +) -> FinancialEventEnvelope: + payload = { + "claim_id": claim.id, + "claim_reference": claim.claim_no, + "amount": str(claim.amount), + "currency": claim.currency, + "external_payment_reference": f"PAY-{claim.claim_no}-0001", + **dict(extra or {}), + } + return FinancialEventEnvelope( + tenant_id=tenant_id, + external_event_id=event_id, + event_type=event_type, + occurred_at=datetime.now(UTC), + correlation_id=f"corr-{event_id}"[:64], + payload=payload, + ) + + +def _savings_opportunity(db: Session, claim: ExpenseClaim) -> None: + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim=claim, + item_date=date.today(), + item_type="hotel", + item_reason="住宿", + item_location="上海", + item_note="", + item_amount=Decimal("100.00"), + ) + db.add(item) + db.flush() + SavingsDiscoveryService(db).discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[ + { + "item_id": item.id, + "message": "政策调整", + "original_amount": "100.00", + "reimbursable_amount": "66.00", + "employee_absorbed_amount": "34.00", + "policy_rule_version": "connector-policy-v1", + "policy_rule_version_source": "published", + "policy_grade": "P6", + "policy_matched_city": "上海", + "calculation_fingerprint": "sha256:" + "d" * 64, + } + ], + current_user=_user("finance-payer", roles=["finance"]), + request_id="connector-discovery-001", + ) + + +def _linked_application(db: Session, reimbursement: ExpenseClaim) -> ExpenseClaim: + application = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no=f"AP-CONNECTOR-{uuid.uuid4().hex[:10].upper()}", + employee_name=reimbursement.employee_name, + department_name=reimbursement.department_name, + expense_type="travel_application", + reason="连接器端到端申请", + location=reimbursement.location, + amount=reimbursement.amount, + currency=reimbursement.currency, + invoice_count=0, + occurred_at=datetime.now(UTC), + submitted_at=datetime.now(UTC), + status="approved", + approval_stage="审批完成", + risk_flags_json=[], + ) + db.add(application) + db.flush() + reimbursement.risk_flags_json = [ + { + "source": "application_handoff", + "event_type": "expense_application_to_reimbursement_draft", + "application_claim_id": application.id, + "application_claim_no": application.claim_no, + } + ] + return application + + +def _user( + username: str, + *, + tenant_id: str = "default", + roles: list[str] | None = None, + employee_id: str = "", + name: str | None = None, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=name or username, + role_codes=list(roles or []), + is_admin=False, + tenant_id=tenant_id, + employee_id=employee_id, + ) diff --git a/server/tests/test_hermes_finance_tenant_security.py b/server/tests/test_hermes_finance_tenant_security.py new file mode 100644 index 0000000..b3649c9 --- /dev/null +++ b/server/tests/test_hermes_finance_tenant_security.py @@ -0,0 +1,305 @@ +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from decimal import Decimal +from importlib.util import module_from_spec, spec_from_file_location +from pathlib import Path +from typing import Any + +import pytest +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.core.config import get_settings +from app.db.base import Base +from app.models.agent_run import AgentRun, AgentToolCall +from app.models.financial_record import ExpenseClaim +from app.models.tenant import Tenant +from app.models.tenant_finance_report import TenantFinanceReportRun +from app.services.digital_employee_dashboard import DigitalEmployeeDashboardService +from app.services.digital_employee_finance_report_task import ( + DigitalEmployeeFinanceReportTaskService, +) +from app.services.finance_report_tenant import TenantFinanceReportConfigService +from app.services.hermes_risk_clue_collector import HermesRiskClueCollectorService +from app.services.hermes_risk_scanner import HermesRiskScannerService + + +def _session_factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False) + + +def _tenant(tenant_id: str) -> Tenant: + return Tenant( + tenant_id=tenant_id, + tenant_code=tenant_id, + name=f"{tenant_id} 公司", + status="active", + ) + + +def _claim( + *, + tenant_id: str, + claim_id: str, + claim_no: str, + amount: str, + now: datetime, +) -> ExpenseClaim: + return ExpenseClaim( + id=claim_id, + tenant_id=tenant_id, + claim_no=claim_no, + employee_name=f"{tenant_id} 员工", + department_name=f"{tenant_id} 财务部", + expense_type="travel", + reason="客户拜访", + location="上海", + amount=Decimal(amount), + invoice_count=1, + occurred_at=now - timedelta(days=1), + submitted_at=now - timedelta(days=1), + status="submitted", + approval_stage="直属领导审批", + risk_flags_json=[], + hermes_risk_flag=False, + created_at=now - timedelta(days=1), + updated_at=now, + ) + + +def _digital_run( + *, + tenant_id: str, + run_id: str, + scanned_claim_count: int, + now: datetime, +) -> AgentRun: + return AgentRun( + run_id=run_id, + agent="hermes", + source="schedule", + user_id="digital_employee", + status="succeeded", + route_json={ + "tenant_id": tenant_id, + "task_type": "global_risk_scan", + }, + ontology_json={"tenant_id": tenant_id}, + result_summary="风险扫描完成。", + started_at=now - timedelta(minutes=2), + finished_at=now - timedelta(minutes=1), + tool_calls=[ + AgentToolCall( + run_id=run_id, + tool_type="rule_engine", + tool_name="digital_employee.financial_risk_graph.scan", + request_json={"tenant_id": tenant_id}, + response_json={"scanned_claim_count": scanned_claim_count}, + status="succeeded", + duration_ms=10, + created_at=now - timedelta(minutes=2), + ) + ], + ) + + +def test_hermes_scanners_and_dashboard_do_not_mix_tenants() -> None: + factory = _session_factory() + now = datetime.now(UTC) + with factory() as db: + db.add_all([_tenant("tenant-a"), _tenant("tenant-b")]) + db.add_all( + [ + _claim( + tenant_id="tenant-a", + claim_id="claim-a", + claim_no="RE-A-001", + amount="100.00", + now=now, + ), + _claim( + tenant_id="tenant-b", + claim_id="claim-b", + claim_no="RE-B-SECRET", + amount="9000.00", + now=now, + ), + _digital_run( + tenant_id="tenant-a", + run_id="run-a", + scanned_claim_count=1, + now=now, + ), + _digital_run( + tenant_id="tenant-b", + run_id="run-b", + scanned_claim_count=99, + now=now, + ), + ] + ) + db.commit() + + fetched = HermesRiskScannerService(db)._fetch_unscanned_claims(tenant_id="tenant-a") + assert [row.id for row in fetched] == ["claim-a"] + + clues = HermesRiskClueCollectorService(db).collect_risk_clues(tenant_id="tenant-a") + assert clues["tenant_id"] == "tenant-a" + assert [row["claim_no"] for row in clues["facts"]] == ["RE-A-001"] + + dashboard = DigitalEmployeeDashboardService( + db, + tenant_id="tenant-a", + ).build_dashboard(days=7) + assert dashboard.totals["totalRuns"] == 1 + assert dashboard.totals["riskObservations"] == 0 + assert [row["runId"] for row in dashboard.recent_runs] == ["run-a"] + + +def test_finance_report_recipients_data_path_and_idempotency_are_tenant_bound( + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + monkeypatch.setenv("STORAGE_ROOT_DIR", str(tmp_path)) + get_settings.cache_clear() + factory = _session_factory() + now = datetime.now(UTC) + start_date = (now - timedelta(days=3)).date() + end_date = now.date() + try: + with factory() as db: + db.add_all([_tenant("tenant-a"), _tenant("tenant-b")]) + db.add_all( + [ + _claim( + tenant_id="tenant-a", + claim_id="claim-a", + claim_no="RE-A-001", + amount="100.00", + now=now, + ), + _claim( + tenant_id="tenant-b", + claim_id="claim-b", + claim_no="RE-B-001", + amount="9000.00", + now=now, + ), + ] + ) + db.commit() + + config_service = TenantFinanceReportConfigService(db) + config_service.upsert( + tenant_id="tenant-a", + recipients=["finance-a@example.com"], + delivery_enabled=True, + updated_by="finance-a", + ) + config_service.upsert( + tenant_id="tenant-b", + recipients=["finance-b@example.com"], + delivery_enabled=True, + updated_by="finance-b", + ) + assert config_service.configured_recipients(tenant_id="tenant-a") == [ + "finance-a@example.com" + ] + assert ( + config_service.configured_recipients( + tenant_id="tenant-a", + requested=["finance-b@example.com"], + ) + == [] + ) + assert config_service.configured_recipients(tenant_id="missing") == [] + + task = DigitalEmployeeFinanceReportTaskService(db) + report_a = task.generate_report( + report_type="weekly", + start_date=start_date, + end_date=end_date, + tenant_id="tenant-a", + send_email=False, + ) + replay_a = task.generate_report( + report_type="weekly", + start_date=start_date, + end_date=end_date, + tenant_id="tenant-a", + send_email=False, + ) + report_b = task.generate_report( + report_type="weekly", + start_date=start_date, + end_date=end_date, + tenant_id="tenant-b", + send_email=False, + ) + + assert report_a["summary"]["reimbursement_count"] == 1 + assert report_a["summary"]["reimbursement_amount"] == 100.0 + assert report_b["summary"]["reimbursement_count"] == 1 + assert report_b["summary"]["reimbursement_amount"] == 9000.0 + assert replay_a["idempotent_replay"] is True + assert report_a["pdf"]["storage_key"] != report_b["pdf"]["storage_key"] + assert "/tenants/" in report_a["pdf"]["storage_key"] + assert "tenant-a" not in report_a["pdf"]["storage_key"] + + ledger_rows = list(db.scalars(select(TenantFinanceReportRun)).all()) + assert len(ledger_rows) == 2 + assert {row.tenant_id for row in ledger_rows} == {"tenant-a", "tenant-b"} + assert len({row.idempotency_key for row in ledger_rows}) == 2 + + runs = list(db.scalars(select(AgentRun).where(AgentRun.agent == "hermes"))) + tenant_a_runs = [ + row for row in runs if (row.route_json or {}).get("tenant_id") == "tenant-a" + ] + assert len(tenant_a_runs) == 1 + assert tenant_a_runs[0].ontology_json["tenant_id"] == "tenant-a" + finally: + get_settings.cache_clear() + + +class _UnsupportedDialectOperationGuard: + def __init__(self) -> None: + self.bind = type("Bind", (), {"dialect": type("Dialect", (), {"name": "sqlite"})()})() + self.mutation_calls: list[str] = [] + + def get_bind(self) -> Any: + return self.bind + + def __getattr__(self, name: str) -> Any: + self.mutation_calls.append(name) + raise AssertionError(f"unsupported dialect attempted migration operation: {name}") + + +@pytest.mark.parametrize("direction", ["upgrade", "downgrade"]) +def test_hermes_tenant_migration_rejects_unsupported_dialect_before_mutation( + direction: str, +) -> None: + migration_path = ( + Path(__file__).resolve().parents[1] + / "alembic" + / "versions" + / "20260717_0028_hermes_ontology_tenant_security.py" + ) + spec = spec_from_file_location("migration_20260717_0028_test", migration_path) + assert spec is not None and spec.loader is not None + module = module_from_spec(spec) + spec.loader.exec_module(module) + guard = _UnsupportedDialectOperationGuard() + module.op = guard + + with pytest.raises(RuntimeError, match="only supports PostgreSQL"): + getattr(module, direction)() + + assert guard.mutation_calls == [] diff --git a/server/tests/test_hierarchical_expense_memory_foundation.py b/server/tests/test_hierarchical_expense_memory_foundation.py index 064a1e8..3a06c75 100644 --- a/server/tests/test_hierarchical_expense_memory_foundation.py +++ b/server/tests/test_hierarchical_expense_memory_foundation.py @@ -13,6 +13,7 @@ from app.api.deps import _authenticate_bearer_user from app.db.base import Base from app.models.ai_memory import MemoryEntry from app.models.employee import Employee +from app.models.tenant import Tenant from app.services.auth import AuthService from app.services.auth_sessions import AuthSessionService from app.services.employee import EmployeeService @@ -149,11 +150,22 @@ def test_admin_managed_memory_rejects_partially_missing_audit_fields() -> None: def test_authenticated_session_restores_tenant_and_stable_department_id() -> None: with _build_session() as db: - employee_snapshot = EmployeeService(db).list_employees()[0] + db.add( + Tenant( + tenant_id="tenant-session-a", + tenant_code="tenant-session-a", + name="会话租户 A", + status="active", + ) + ) + db.flush() + employee_snapshot = EmployeeService( + db, + tenant_id="tenant-session-a", + ).list_employees()[0] employee = db.get(Employee, employee_snapshot.id) assert employee is not None authenticated_at_login = AuthService(db)._build_employee_user(employee) - authenticated_at_login.tenant_id = "tenant-session-a" access_token, auth_session = AuthSessionService(db).issue( authenticated_at_login, metric_session_id="metric-session-a", diff --git a/server/tests/test_knowledge_onlyoffice_config.py b/server/tests/test_knowledge_onlyoffice_config.py index dd6fa7e..37df7ab 100644 --- a/server/tests/test_knowledge_onlyoffice_config.py +++ b/server/tests/test_knowledge_onlyoffice_config.py @@ -6,12 +6,15 @@ from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker from app.api.deps import CurrentUserContext -from app.core.config import Settings, get_settings from app.core import secret_box +from app.core.config import Settings, get_settings from app.db.base import Base +from app.models.hermes_config import HermesTaskConfig +from app.models.knowledge_security import KnowledgeOnlyOfficeSession from app.models.system_model_setting import SystemModelSetting from app.models.system_setting import SystemSetting from app.models.system_setting_secret import SystemSettingSecret +from app.models.tenant import Tenant from app.schemas.settings import SettingsWrite from app.services.knowledge import KnowledgeService from app.services.settings import SettingsService @@ -25,9 +28,24 @@ def build_session_factory(db_file: Path): SystemSetting.__table__.create(bind=engine) SystemSettingSecret.__table__.create(bind=engine) SystemModelSetting.__table__.create(bind=engine) + Tenant.__table__.create(bind=engine) + KnowledgeOnlyOfficeSession.__table__.create(bind=engine) + HermesTaskConfig.__table__.create(bind=engine) return sessionmaker(bind=engine, autoflush=False, autocommit=False) +def seed_tenant(db) -> None: + db.add( + Tenant( + tenant_id="tenant-a", + tenant_code="TENANT-A", + name="租户 A", + status="active", + ) + ) + db.commit() + + def test_onlyoffice_config_is_read_only_for_admin_users(tmp_path, monkeypatch) -> None: env_file = tmp_path / ".env" env_file.write_text( @@ -36,7 +54,7 @@ def test_onlyoffice_config_is_read_only_for_admin_users(tmp_path, monkeypatch) - "ONLYOFFICE_ENABLED=true", "ONLYOFFICE_PUBLIC_URL=http://10.10.10.122:8082", "ONLYOFFICE_BACKEND_URL=http://main:8000", - "ONLYOFFICE_JWT_SECRET=change-me-onlyoffice", + "ONLYOFFICE_JWT_SECRET=change-me-onlyoffice-secret-32bytes", ] ) + "\n", @@ -47,32 +65,42 @@ def test_onlyoffice_config_is_read_only_for_admin_users(tmp_path, monkeypatch) - get_settings.cache_clear() try: - service = KnowledgeService(storage_root=tmp_path) - service.ensure_library_ready() + session_factory = build_session_factory(tmp_path / "onlyoffice.db") + with session_factory() as db: + seed_tenant(db) + service = KnowledgeService( + storage_root=tmp_path, + db=db, + tenant_id="tenant-a", + ) + service.ensure_library_ready() - document_id = "readonly-docx" - folder = "制度政策" - stored_name = f"{document_id}__制度预览.docx" - target_path = tmp_path / "knowledge" / folder / stored_name - target_path.write_bytes(b"fake-docx-content") + document_id = "readonly-docx" + folder = "制度政策" + stored_name = f"{document_id}__制度预览.docx" + target_path = ( + tmp_path / "knowledge" / "tenants" / "tenant-a" / folder / stored_name + ) + target_path.write_bytes(b"fake-docx-content") - current_user = CurrentUserContext( - username="admin", - name="管理员", - role_codes=["manager"], - is_admin=True, - ) + current_user = CurrentUserContext( + username="admin", + name="管理员", + role_codes=["manager"], + is_admin=True, + tenant_id="tenant-a", + ) - config = service.build_onlyoffice_config(document_id, current_user) - permissions = config.config["document"]["permissions"] - customization = config.config["editorConfig"]["customization"] + config = service.build_onlyoffice_config(document_id, current_user) + permissions = config.config["document"]["permissions"] + customization = config.config["editorConfig"]["customization"] - assert config.documentServerUrl == "http://10.10.10.122:8082" - assert config.config["editorConfig"]["mode"] == "view" - assert permissions["edit"] is False - assert permissions["download"] is True - assert customization["autosave"] is False - assert customization["forcesave"] is False + assert config.documentServerUrl == "http://10.10.10.122:8082" + assert config.config["editorConfig"]["mode"] == "view" + assert permissions["edit"] is False + assert permissions["download"] is True + assert customization["autosave"] is False + assert customization["forcesave"] is False finally: monkeypatch.setitem(Settings.model_config, "env_file", original_env_file) get_settings.cache_clear() @@ -102,35 +130,45 @@ def test_onlyoffice_config_prefers_saved_settings_snapshot(tmp_path, monkeypatch try: with session_factory() as db: + seed_tenant(db) service = SettingsService(db) payload = service.get_settings_snapshot().model_dump() payload["renderForm"]["enabled"] = True payload["renderForm"]["publicUrl"] = "http://10.10.10.122:8082" - payload["renderForm"]["jwtSecret"] = "change-me-onlyoffice" + payload["renderForm"]["jwtSecret"] = "change-me-onlyoffice-secret-32bytes" service.save_settings_snapshot(SettingsWrite(**payload)) - service = KnowledgeService(storage_root=tmp_path) - service.ensure_library_ready() + with session_factory() as db: + service = KnowledgeService( + storage_root=tmp_path, + db=db, + tenant_id="tenant-a", + ) + service.ensure_library_ready() - document_id = "db-backed-docx" - folder = "制度政策" - stored_name = f"{document_id}__制度预览.docx" - target_path = tmp_path / "knowledge" / folder / stored_name - target_path.write_bytes(b"fake-docx-content") + document_id = "db-backed-docx" + folder = "制度政策" + stored_name = f"{document_id}__制度预览.docx" + target_path = ( + tmp_path / "knowledge" / "tenants" / "tenant-a" / folder / stored_name + ) + target_path.write_bytes(b"fake-docx-content") - current_user = CurrentUserContext( - username="admin", - name="管理员", - role_codes=["manager"], - is_admin=True, - ) + current_user = CurrentUserContext( + username="admin", + name="管理员", + role_codes=["manager"], + is_admin=True, + tenant_id="tenant-a", + ) - config = service.build_onlyoffice_config(document_id, current_user) + config = service.build_onlyoffice_config(document_id, current_user) - assert config.documentServerUrl == "http://10.10.10.122:8082" - assert config.config["document"]["url"].startswith( - "http://main:8000/api/v1/knowledge/documents/db-backed-docx/onlyoffice/content?access_token=" - ) + assert config.documentServerUrl == "http://10.10.10.122:8082" + assert config.config["document"]["url"].startswith( + "http://main:8000/api/v1/knowledge/documents/" + "db-backed-docx/onlyoffice/content?access_token=" + ) finally: monkeypatch.setitem(Settings.model_config, "env_file", original_env_file) get_settings.cache_clear() diff --git a/server/tests/test_knowledge_onlyoffice_tenant_security.py b/server/tests/test_knowledge_onlyoffice_tenant_security.py new file mode 100644 index 0000000..5dbf035 --- /dev/null +++ b/server/tests/test_knowledge_onlyoffice_tenant_security.py @@ -0,0 +1,587 @@ +from __future__ import annotations + +import json +from datetime import UTC, datetime, timedelta +from io import BytesIO +from urllib.parse import parse_qs, urlsplit +from zipfile import ZIP_DEFLATED, ZipFile + +import jwt +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import sessionmaker +from sqlalchemy.pool import StaticPool + +from app.api.deps import CurrentUserContext +from app.db.base import Base +from app.models.knowledge_security import KnowledgeOnlyOfficeSession +from app.models.tenant import Tenant +from app.services import knowledge_onlyoffice_security as security_module +from app.services.knowledge import KnowledgeService +from app.services.knowledge_onlyoffice_callback import ( + handle_onlyoffice_callback, + resolve_onlyoffice_content, +) +from app.services.knowledge_onlyoffice_security import ( + ONLYOFFICE_TOKEN_AUDIENCE, + KnowledgeOnlyOfficeSessionService, + OnlyOfficeReplayError, + OnlyOfficeSecurityError, + download_onlyoffice_document, +) +from app.services.knowledge_rag import KnowledgeRagService +from app.services.knowledge_tenant_scope import PLATFORM_KNOWLEDGE_SCOPE +from app.services.settings import OnlyOfficeRuntimeConfig + +JWT_SECRET = "test-onlyoffice-security-secret-32bytes" + + +def _docx_bytes(text: str) -> bytes: + stream = BytesIO() + with ZipFile(stream, mode="w", compression=ZIP_DEFLATED) as archive: + archive.writestr( + "[Content_Types].xml", + """ + + +""", + ) + archive.writestr( + "word/document.xml", + """ + + """ + + text + + """ +""", + ) + return stream.getvalue() + + +def _user(tenant_id: str, *, admin: bool = True) -> CurrentUserContext: + return CurrentUserContext( + username=f"user-{tenant_id}", + name=f"用户 {tenant_id}", + role_codes=["manager"] if admin else ["employee"], + is_admin=admin, + tenant_id=tenant_id, + ) + + +def _factory(): + assert "tenants" in Base.metadata.tables + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Tenant.__table__.create(bind=engine) + KnowledgeOnlyOfficeSession.__table__.create(bind=engine) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as db: + db.add_all( + [ + Tenant(tenant_id="tenant-a", tenant_code="A", name="A", status="active"), + Tenant(tenant_id="tenant-b", tenant_code="B", name="B", status="active"), + ] + ) + db.commit() + return factory + + +def _configure_onlyoffice(monkeypatch) -> OnlyOfficeRuntimeConfig: + runtime = OnlyOfficeRuntimeConfig( + enabled=True, + public_url="https://docs.example.com", + backend_url="https://app.example.com", + jwt_secret=JWT_SECRET, + ) + monkeypatch.setattr( + "app.services.knowledge_onlyoffice.resolve_onlyoffice_settings", + lambda *_args, **_kwargs: runtime, + ) + monkeypatch.setattr( + security_module, + "resolve_onlyoffice_settings", + lambda *_args, **_kwargs: runtime, + ) + monkeypatch.setattr( + KnowledgeRagService, + "get_document_status_map", + lambda _self, _document_ids: {}, + ) + monkeypatch.setattr(KnowledgeRagService, "delete_document", lambda *_args: None) + return runtime + + +def _token_from_url(url: str, parameter: str) -> str: + return parse_qs(urlsplit(url).query)[parameter][0] + + +def test_onlyoffice_tokens_bind_tenant_resource_key_version_and_audience( + tmp_path, + monkeypatch, +) -> None: + _configure_onlyoffice(monkeypatch) + factory = _factory() + with factory() as db: + service = KnowledgeService(storage_root=tmp_path, db=db, tenant_id="tenant-a") + uploaded = service.upload_document( + "制度政策", + "制度.docx", + _docx_bytes("version one"), + _user("tenant-a"), + ) + config = service.build_onlyoffice_config(uploaded.id, _user("tenant-a")) + content_token = _token_from_url(config.config["document"]["url"], "access_token") + callback_token = _token_from_url( + config.config["editorConfig"]["callbackUrl"], + "callback_token", + ) + claims = jwt.decode( + content_token, + JWT_SECRET, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + ) + callback_claims = jwt.decode( + callback_token, + JWT_SECRET, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + ) + + assert claims["tenant_id"] == "tenant-a" + assert claims["document_id"] == uploaded.id + assert claims["document_key"] == config.config["document"]["key"] + assert claims["document_version"] == 1 + assert claims["editable"] is False + assert callback_claims["jti"] == claims["jti"] + assert callback_claims["exp"] - claims["exp"] >= 3 * 60 * 60 + row = db.get(KnowledgeOnlyOfficeSession, claims["jti"]) + assert row is not None and row.tenant_id == "tenant-a" and row.status == "active" + content_path, _, _ = resolve_onlyoffice_content( + db=db, + storage_root=tmp_path, + document_id=uploaded.id, + access_token=content_token, + ) + assert content_path.is_relative_to( + tmp_path / "knowledge" / "tenants" / "tenant-a" + ) + with ZipFile(content_path) as archive: + assert b"version one" in archive.read("word/document.xml") + + tampered_claims = dict(claims) + tampered_claims["tenant_id"] = "tenant-b" + tampered_token = jwt.encode(tampered_claims, JWT_SECRET, algorithm="HS256") + with pytest.raises(OnlyOfficeSecurityError, match="不匹配"): + KnowledgeOnlyOfficeSessionService(db).validate_content( + document_id=uploaded.id, + token=tampered_token, + ) + assert callback_token + + +def test_platform_document_session_is_tenant_bound_and_strictly_read_only( + tmp_path, + monkeypatch, +) -> None: + _configure_onlyoffice(monkeypatch) + document_id = "platform-doc" + content = _docx_bytes("platform policy") + filename = "平台制度.docx" + stored_name = f"{document_id}__{filename}" + platform_root = tmp_path / "knowledge" / "platform" + folder_root = platform_root / "制度政策" + folder_root.mkdir(parents=True) + (folder_root / stored_name).write_bytes(content) + (platform_root / ".index.json").write_text( + json.dumps( + { + "version": 1, + "documents": [ + { + "id": document_id, + "folder": "制度政策", + "original_name": filename, + "stored_name": stored_name, + "mime_type": ( + "application/vnd.openxmlformats-officedocument." + "wordprocessingml.document" + ), + "extension": "docx", + "size_bytes": len(content), + "sha256": "platform-checksum", + "created_at": "2026-07-17T00:00:00+00:00", + "updated_at": "2026-07-17T00:00:00+00:00", + "uploaded_by": "平台", + "version_number": 1, + "ingest_status": 1, + } + ], + }, + ensure_ascii=False, + ), + encoding="utf-8", + ) + factory = _factory() + with factory() as db: + service = KnowledgeService(storage_root=tmp_path, db=db, tenant_id="tenant-a") + config = service.build_onlyoffice_config(document_id, _user("tenant-a")) + content_token = _token_from_url(config.config["document"]["url"], "access_token") + claims = jwt.decode( + content_token, + JWT_SECRET, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + ) + assert claims["tenant_id"] == "tenant-a" + assert claims["resource_scope"] == PLATFORM_KNOWLEDGE_SCOPE + assert claims["editable"] is False + resolved, _, _ = resolve_onlyoffice_content( + db=db, + storage_root=tmp_path, + document_id=document_id, + access_token=content_token, + ) + assert resolved.read_bytes() == content + assert resolved.is_relative_to(platform_root) + with pytest.raises(ValueError, match="只读"): + service.build_onlyoffice_config( + document_id, + _user("tenant-a"), + editable=True, + ) + + +def test_view_session_never_writes_and_wrong_key_does_not_claim_session( + tmp_path, + monkeypatch, +) -> None: + _configure_onlyoffice(monkeypatch) + factory = _factory() + with factory() as db: + service = KnowledgeService(storage_root=tmp_path, db=db, tenant_id="tenant-a") + uploaded = service.upload_document( + "制度政策", + "制度.docx", + _docx_bytes("original"), + _user("tenant-a"), + ) + view_config = service.build_onlyoffice_config(uploaded.id, _user("tenant-a")) + view_token = _token_from_url( + view_config.config["editorConfig"]["callbackUrl"], + "callback_token", + ) + with pytest.raises(OnlyOfficeSecurityError, match="只读"): + handle_onlyoffice_callback( + db=db, + storage_root=tmp_path, + document_id=uploaded.id, + callback_token=view_token, + payload={ + "status": 2, + "key": view_config.config["document"]["key"], + "url": "https://docs.example.com/download/view", + }, + ) + + edit_config = service.build_onlyoffice_config( + uploaded.id, + _user("tenant-a"), + editable=True, + ) + edit_token = _token_from_url( + edit_config.config["editorConfig"]["callbackUrl"], + "callback_token", + ) + with pytest.raises(OnlyOfficeSecurityError, match="key"): + handle_onlyoffice_callback( + db=db, + storage_root=tmp_path, + document_id=uploaded.id, + callback_token=edit_token, + payload={ + "status": 2, + "key": "wrong-key", + "url": "https://docs.example.com/download/edit", + }, + ) + edit_claims = jwt.decode( + edit_token, + JWT_SECRET, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + ) + assert db.get(KnowledgeOnlyOfficeSession, edit_claims["jti"]).status == "active" + assert service.get_document_entry(uploaded.id)["version_number"] == 1 + + +def test_edit_callback_is_one_time_and_replay_is_rejected(tmp_path, monkeypatch) -> None: + _configure_onlyoffice(monkeypatch) + replacement = _docx_bytes("replacement") + monkeypatch.setattr( + "app.services.knowledge_onlyoffice_callback.download_onlyoffice_document", + lambda _url, *, expected_filename: replacement, + ) + factory = _factory() + with factory() as db: + service = KnowledgeService(storage_root=tmp_path, db=db, tenant_id="tenant-a") + uploaded = service.upload_document( + "制度政策", + "制度.docx", + _docx_bytes("original"), + _user("tenant-a"), + ) + config = service.build_onlyoffice_config( + uploaded.id, + _user("tenant-a"), + editable=True, + ) + callback_token = _token_from_url( + config.config["editorConfig"]["callbackUrl"], + "callback_token", + ) + payload = { + "status": 2, + "key": config.config["document"]["key"], + "url": "https://docs.example.com/download/final", + "users": ["editor"], + } + handle_onlyoffice_callback( + db=db, + storage_root=tmp_path, + document_id=uploaded.id, + callback_token=callback_token, + payload=payload, + ) + + claims = jwt.decode( + callback_token, + JWT_SECRET, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + ) + assert db.get(KnowledgeOnlyOfficeSession, claims["jti"]).status == "consumed" + assert service.get_document_entry(uploaded.id)["version_number"] == 2 + assert service.get_document_content(uploaded.id)[0].read_bytes() == replacement + with pytest.raises(OnlyOfficeReplayError): + handle_onlyoffice_callback( + db=db, + storage_root=tmp_path, + document_id=uploaded.id, + callback_token=callback_token, + payload=payload, + ) + + +def test_expired_or_stale_version_session_is_rejected(tmp_path, monkeypatch) -> None: + _configure_onlyoffice(monkeypatch) + factory = _factory() + with factory() as db: + service = KnowledgeService(storage_root=tmp_path, db=db, tenant_id="tenant-a") + uploaded = service.upload_document( + "制度政策", + "制度.docx", + _docx_bytes("original"), + _user("tenant-a"), + ) + config = service.build_onlyoffice_config(uploaded.id, _user("tenant-a")) + content_token = _token_from_url(config.config["document"]["url"], "access_token") + claims = jwt.decode( + content_token, + JWT_SECRET, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + ) + row = db.get(KnowledgeOnlyOfficeSession, claims["jti"]) + row.expires_at = datetime.now(UTC) - timedelta(seconds=1) + db.commit() + with pytest.raises(OnlyOfficeSecurityError): + resolve_onlyoffice_content( + db=db, + storage_root=tmp_path, + document_id=uploaded.id, + access_token=content_token, + ) + + +def test_callback_ssrf_attempt_fails_session_without_overwriting_document( + tmp_path, + monkeypatch, +) -> None: + _configure_onlyoffice(monkeypatch) + factory = _factory() + with factory() as db: + service = KnowledgeService(storage_root=tmp_path, db=db, tenant_id="tenant-a") + original = _docx_bytes("original") + uploaded = service.upload_document( + "制度政策", + "制度.docx", + original, + _user("tenant-a"), + ) + config = service.build_onlyoffice_config( + uploaded.id, + _user("tenant-a"), + editable=True, + ) + callback_token = _token_from_url( + config.config["editorConfig"]["callbackUrl"], + "callback_token", + ) + with pytest.raises(OnlyOfficeSecurityError, match="不属于"): + handle_onlyoffice_callback( + db=db, + storage_root=tmp_path, + document_id=uploaded.id, + callback_token=callback_token, + payload={ + "status": 2, + "key": config.config["document"]["key"], + "url": "https://evil.example/internal-metadata", + }, + ) + claims = jwt.decode( + callback_token, + JWT_SECRET, + algorithms=["HS256"], + audience=ONLYOFFICE_TOKEN_AUDIENCE, + ) + assert db.get(KnowledgeOnlyOfficeSession, claims["jti"]).status == "failed" + assert service.get_document_content(uploaded.id)[0].read_bytes() == original + assert service.get_document_entry(uploaded.id)["version_number"] == 1 + + +def test_download_target_rejects_wrong_origin_private_dns_and_redirects(monkeypatch) -> None: + _configure_onlyoffice(monkeypatch) + with pytest.raises(OnlyOfficeSecurityError, match="不属于"): + security_module._validate_download_target("https://evil.example/download") + + monkeypatch.setattr( + security_module.socket, + "getaddrinfo", + lambda *_args, **_kwargs: [ + (2, 1, 6, "", ("127.0.0.1", 443)), + ], + ) + with pytest.raises(OnlyOfficeSecurityError, match="非公网"): + security_module._validate_download_target("https://docs.example.com/download") + + monkeypatch.setattr( + security_module.socket, + "getaddrinfo", + lambda *_args, **_kwargs: [ + (2, 1, 6, "", ("8.8.8.8", 443)), + ], + ) + + class RedirectResponse: + status = 302 + + @staticmethod + def getheader(_name): + return None + + class FakeConnection: + def request(self, *_args, **_kwargs) -> None: + pass + + @staticmethod + def getresponse(): + return RedirectResponse() + + def close(self) -> None: + pass + + pinned: list[str] = [] + + def fake_open(_parsed, resolved_ip): + pinned.append(resolved_ip) + return FakeConnection() + + monkeypatch.setattr(security_module, "_open_pinned_connection", fake_open) + with pytest.raises(OnlyOfficeSecurityError, match="状态码 302"): + download_onlyoffice_document( + "https://docs.example.com/download", + expected_filename="制度.docx", + ) + assert pinned == ["8.8.8.8"] + + +def test_download_enforces_mime_size_and_ooxml_structure(monkeypatch) -> None: + _configure_onlyoffice(monkeypatch) + monkeypatch.setattr( + security_module.socket, + "getaddrinfo", + lambda *_args, **_kwargs: [(2, 1, 6, "", ("8.8.8.8", 443))], + ) + + class Response: + status = 200 + + def __init__(self, body: bytes, content_type: str, declared_size: int | None = None): + self.body = body + self.content_type = content_type + self.declared_size = declared_size + + def getheader(self, name): + if name == "Content-Type": + return self.content_type + if name == "Content-Length": + return None if self.declared_size is None else str(self.declared_size) + return None + + def read(self, limit): + return self.body[:limit] + + class Connection: + def __init__(self, response): + self.response = response + + def request(self, *_args, **_kwargs) -> None: + pass + + def getresponse(self): + return self.response + + def close(self) -> None: + pass + + response = Response( + _docx_bytes("safe"), + "application/vnd.openxmlformats-officedocument.wordprocessingml.document", + ) + monkeypatch.setattr( + security_module, + "_open_pinned_connection", + lambda *_args: Connection(response), + ) + downloaded = download_onlyoffice_document( + "https://docs.example.com/download", + expected_filename="制度.docx", + ) + with ZipFile(BytesIO(downloaded)) as archive: + assert b"safe" in archive.read("word/document.xml") + + response.content_type = "text/html" + with pytest.raises(OnlyOfficeSecurityError, match="MIME"): + download_onlyoffice_document( + "https://docs.example.com/download", + expected_filename="制度.docx", + ) + response.content_type = "application/octet-stream" + response.declared_size = 200 * 1024 * 1024 + with pytest.raises(OnlyOfficeSecurityError, match="大小"): + download_onlyoffice_document( + "https://docs.example.com/download", + expected_filename="制度.docx", + ) + response.declared_size = None + response.body = b"not-a-zip" + with pytest.raises(OnlyOfficeSecurityError, match="OOXML"): + download_onlyoffice_document( + "https://docs.example.com/download", + expected_filename="制度.docx", + ) diff --git a/server/tests/test_knowledge_rag_service.py b/server/tests/test_knowledge_rag_service.py index fd2941d..00ad658 100644 --- a/server/tests/test_knowledge_rag_service.py +++ b/server/tests/test_knowledge_rag_service.py @@ -155,7 +155,8 @@ def test_query_local_text_chunks_prioritizes_relevant_policy_chunk(tmp_path) -> def test_query_knowledge_uses_local_chunks_before_lightrag_runtime(tmp_path, monkeypatch) -> None: - workspace = tmp_path / "knowledge" / ".lightrag" / "x_financial_knowledge" + service = KnowledgeRagService(storage_root=tmp_path, tenant_id="tenant-a") + workspace = service.storage_scope.lightrag_root / service.storage_scope.workspace workspace.mkdir(parents=True) (workspace / "kv_store_text_chunks.json").write_text( json.dumps( @@ -180,7 +181,7 @@ def test_query_knowledge_uses_local_chunks_before_lightrag_runtime(tmp_path, mon monkeypatch.setattr(KnowledgeRagService, "_get_runtime", fail_if_runtime_is_used) - payload = KnowledgeRagService(storage_root=tmp_path).query_knowledge( + payload = service.query_knowledge( "费用发生后多久内必须报销？超过三个月还能不能报？", limit=3, ) @@ -257,14 +258,14 @@ def test_runtime_cache_uses_dedicated_instance_across_calling_threads(monkeypatc lambda self: (("same-config",), {}), ) - service = KnowledgeRagService() + service = KnowledgeRagService(tenant_id="tenant-a") main_runtime = service._get_runtime() assert service._get_runtime() is main_runtime worker_runtimes = [] def load_worker_runtime() -> None: - worker_runtimes.append(KnowledgeRagService()._get_runtime()) + worker_runtimes.append(KnowledgeRagService(tenant_id="tenant-a")._get_runtime()) thread = threading.Thread(target=load_worker_runtime) thread.start() diff --git a/server/tests/test_knowledge_security_migration.py b/server/tests/test_knowledge_security_migration.py new file mode 100644 index 0000000..031d76c --- /dev/null +++ b/server/tests/test_knowledge_security_migration.py @@ -0,0 +1,26 @@ +from __future__ import annotations + +import importlib.util +from pathlib import Path + +MIGRATION_PATH = ( + Path(__file__).resolve().parents[1] + / "alembic" + / "versions" + / "20260717_0027_knowledge_tenant_security.py" +) + + +def test_knowledge_security_migration_has_fixed_ancestry_and_fail_closed_downgrade() -> None: + spec = importlib.util.spec_from_file_location("knowledge_security_0027", MIGRATION_PATH) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + + assert module.revision == "20260717_0027" + assert module.down_revision == "20260717_0026" + source = MIGRATION_PATH.read_text(encoding="utf-8") + assert "knowledge_onlyoffice_sessions" in source + assert "fk_knowledge_onlyoffice_sessions_tenant" in source + assert "ck_knowledge_onlyoffice_sessions_scope_tenant" in source + assert "OnlyOffice session evidence exists" in source diff --git a/server/tests/test_knowledge_service.py b/server/tests/test_knowledge_service.py index 0867038..85fe38a 100644 --- a/server/tests/test_knowledge_service.py +++ b/server/tests/test_knowledge_service.py @@ -30,7 +30,7 @@ def build_session() -> Session: def test_list_library_returns_closed_folder_icons_by_default(tmp_path) -> None: - service = KnowledgeService(storage_root=tmp_path) + service = KnowledgeService(storage_root=tmp_path, tenant_id="tenant-a") library = service.list_library() @@ -43,7 +43,7 @@ def test_reconcile_document_ingest_status_keeps_failed_when_linked_run_failed( monkeypatch, ) -> None: with build_session() as db: - service = KnowledgeService(storage_root=tmp_path, db=db) + service = KnowledgeService(storage_root=tmp_path, db=db, tenant_id="tenant-a") uploaded = service.upload_document( "报销制度", "demo.txt", @@ -53,6 +53,7 @@ def test_reconcile_document_ingest_status_keeps_failed_when_linked_run_failed( name="管理员", role_codes=["manager"], is_admin=True, + tenant_id="tenant-a", ), ) @@ -60,6 +61,7 @@ def test_reconcile_document_ingest_status_keeps_failed_when_linked_run_failed( agent=AgentName.HERMES.value, source=AgentRunSource.USER_MESSAGE.value, status=AgentRunStatus.FAILED.value, + tenant_id="tenant-a", route_json={"job_type": "knowledge_index_sync"}, ) service.set_document_ingest_statuses( @@ -95,7 +97,7 @@ def test_reconcile_document_ingest_status_preserves_ingested_when_status_map_mis tmp_path, monkeypatch, ) -> None: - service = KnowledgeService(storage_root=tmp_path) + service = KnowledgeService(storage_root=tmp_path, tenant_id="tenant-a") uploaded = service.upload_document( "报销制度", "demo.txt", @@ -105,6 +107,7 @@ def test_reconcile_document_ingest_status_preserves_ingested_when_status_map_mis name="管理员", role_codes=["manager"], is_admin=True, + tenant_id="tenant-a", ), ) service.set_document_ingest_statuses( diff --git a/server/tests/test_knowledge_sync.py b/server/tests/test_knowledge_sync.py index a43c30b..edff31e 100644 --- a/server/tests/test_knowledge_sync.py +++ b/server/tests/test_knowledge_sync.py @@ -28,10 +28,15 @@ def test_force_sync_queues_ingested_documents_and_creates_hermes_run(tmp_path, m name="管理员", role_codes=["manager"], is_admin=True, + tenant_id="tenant-a", ) with build_session() as db: - knowledge_service = KnowledgeService(storage_root=tmp_path, db=db) + knowledge_service = KnowledgeService( + storage_root=tmp_path, + db=db, + tenant_id="tenant-a", + ) uploaded = knowledge_service.upload_document("报销制度", "demo.txt", b"hello", user) document_id = uploaded.id knowledge_service.set_document_ingest_statuses( diff --git a/server/tests/test_knowledge_tenant_security.py b/server/tests/test_knowledge_tenant_security.py new file mode 100644 index 0000000..3158a26 --- /dev/null +++ b/server/tests/test_knowledge_tenant_security.py @@ -0,0 +1,230 @@ +from __future__ import annotations + +import json + +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import sessionmaker +from sqlalchemy.pool import StaticPool + +from app.api.deps import CurrentUserContext +from app.db.base import Base +from app.models.agent_run import AgentRun +from app.models.tenant import Tenant +from app.services.knowledge import KnowledgeService +from app.services.knowledge_rag import KnowledgeRagService +from app.services.knowledge_run_scope import resolve_trusted_knowledge_run_tenant +from app.services.knowledge_scheduler import KnowledgeIndexScheduler +from app.services.knowledge_tenant_scope import PLATFORM_KNOWLEDGE_SCOPE + + +def _user(tenant_id: str) -> CurrentUserContext: + return CurrentUserContext( + username=f"admin-{tenant_id}", + name=f"管理员 {tenant_id}", + role_codes=["manager"], + is_admin=True, + tenant_id=tenant_id, + ) + + +def test_tenant_document_metadata_paths_and_reads_are_isolated(tmp_path, monkeypatch) -> None: + monkeypatch.setattr( + KnowledgeRagService, + "get_document_status_map", + lambda _self, _document_ids: {}, + ) + monkeypatch.setattr(KnowledgeRagService, "delete_document", lambda *_args: None) + tenant_a = KnowledgeService(storage_root=tmp_path, tenant_id="tenant-a") + tenant_b = KnowledgeService(storage_root=tmp_path, tenant_id="tenant-b") + + doc_a = tenant_a.upload_document("报销制度", "制度.txt", b"tenant-a", _user("tenant-a")) + doc_b = tenant_b.upload_document("报销制度", "制度.txt", b"tenant-b", _user("tenant-b")) + + assert doc_a.id != doc_b.id + path_a, _, _ = tenant_a.get_document_content(doc_a.id) + path_b, _, _ = tenant_b.get_document_content(doc_b.id) + assert path_a.read_bytes() == b"tenant-a" + assert path_b.read_bytes() == b"tenant-b" + assert path_a.is_relative_to(tmp_path / "knowledge" / "tenants" / "tenant-a") + assert path_b.is_relative_to(tmp_path / "knowledge" / "tenants" / "tenant-b") + with pytest.raises(FileNotFoundError): + tenant_a.get_document_content(doc_b.id) + with pytest.raises(FileNotFoundError): + tenant_b.get_document_detail(doc_a.id) + + +def test_platform_documents_are_explicitly_read_only_and_visible_to_tenants( + tmp_path, + monkeypatch, +) -> None: + legacy_folder = tmp_path / "knowledge" / "制度政策" + legacy_folder.mkdir(parents=True) + legacy_file = legacy_folder / "platform-doc__平台制度.txt" + legacy_file.write_bytes(b"platform-policy") + (tmp_path / "knowledge" / ".index.json").write_text( + json.dumps( + { + "version": 1, + "documents": [ + { + "id": "platform-doc", + "folder": "制度政策", + "original_name": "平台制度.txt", + "stored_name": legacy_file.name, + "mime_type": "text/plain", + "extension": "txt", + "size_bytes": len(b"platform-policy"), + "sha256": "", + "created_at": "2026-07-17T00:00:00+00:00", + "updated_at": "2026-07-17T00:00:00+00:00", + "uploaded_by": "平台", + "version_number": 1, + "ingest_status": 1, + } + ], + }, + ensure_ascii=False, + ), + encoding="utf-8", + ) + monkeypatch.setattr( + KnowledgeRagService, + "get_document_status_map", + lambda _self, _document_ids: {}, + ) + platform = KnowledgeService(storage_root=tmp_path, scope=PLATFORM_KNOWLEDGE_SCOPE) + platform.ensure_library_ready() + tenant = KnowledgeService(storage_root=tmp_path, tenant_id="tenant-a") + + listed = {item.id: item for item in tenant.list_library().documents} + assert listed["platform-doc"].scope == PLATFORM_KNOWLEDGE_SCOPE + assert listed["platform-doc"].readOnly is True + assert tenant.get_document_content("platform-doc")[0].read_bytes() == b"platform-policy" + with pytest.raises(FileNotFoundError): + tenant.delete_document("platform-doc") + with pytest.raises(ValueError, match="只读"): + platform.upload_document("制度政策", "new.txt", b"x", _user("tenant-a")) + + +def test_unscoped_knowledge_services_fail_closed(tmp_path) -> None: + with pytest.raises(ValueError, match="显式提供"): + KnowledgeService(storage_root=tmp_path) + with pytest.raises(ValueError, match="显式提供"): + KnowledgeRagService(storage_root=tmp_path) + + +def test_lightrag_workspace_cache_and_local_chunks_are_tenant_namespaced(tmp_path) -> None: + service_a = KnowledgeRagService(storage_root=tmp_path, tenant_id="tenant-a") + service_b = KnowledgeRagService(storage_root=tmp_path, tenant_id="tenant-b") + assert service_a.storage_scope.workspace != service_b.storage_scope.workspace + assert service_a.storage_scope.runtime_cache_key != service_b.storage_scope.runtime_cache_key + assert "tenant-a" not in service_a.storage_scope.workspace + + for service, marker in ((service_a, "A 租户专属限额"), (service_b, "B 租户专属限额")): + workspace = service.storage_scope.lightrag_root / service.storage_scope.workspace + workspace.mkdir(parents=True) + (workspace / "kv_store_text_chunks.json").write_text( + json.dumps( + { + f"chunk-{marker[0]}": { + "_id": f"chunk-{marker[0]}", + "full_doc_id": f"doc-{marker[0]}", + "chunk_order_index": 1, + "file_path": f"/tmp/doc-{marker[0]}__制度.txt", + "content": f"报销限额规定：{marker}，提交前必须校验。", + } + }, + ensure_ascii=False, + ), + encoding="utf-8", + ) + + result_a = service_a.query_knowledge("A 租户专属限额是什么？", limit=2) + result_b = service_b.query_knowledge("B 租户专属限额是什么？", limit=2) + assert "A 租户专属限额" in result_a["hits"][0]["content"] + assert "B 租户专属限额" in result_b["hits"][0]["content"] + assert all("B 租户" not in item["content"] for item in result_a["hits"]) + assert all("A 租户" not in item["content"] for item in result_b["hits"]) + + +def test_scheduler_enumerates_only_active_tenant_registry_rows(monkeypatch) -> None: + assert "tenants" in Base.metadata.tables + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Tenant.__table__.create(bind=engine) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as db: + db.add_all( + [ + Tenant(tenant_id="tenant-a", tenant_code="A", name="A", status="active"), + Tenant( + tenant_id="tenant-b", + tenant_code="B", + name="B", + status="suspended", + ), + ] + ) + db.commit() + + seen_tenants: list[str] = [] + + class FakeDispatch: + def __init__(self, _db) -> None: + pass + + def queue_sync(self, *, current_user, **_kwargs): + seen_tenants.append(current_user.tenant_id) + return type( + "Result", + (), + { + "agent_run_id": "", + "document_ids": [], + "reused": False, + "summary": "no changes", + }, + )() + + monkeypatch.setattr( + "app.services.knowledge_scheduler.get_session_factory", + lambda: factory, + ) + monkeypatch.setattr( + "app.services.knowledge_scheduler.KnowledgeSyncDispatchService", + FakeDispatch, + ) + KnowledgeIndexScheduler()._run_incremental_sync() + assert seen_tenants == ["tenant-a"] + + +def test_index_worker_rederives_tenant_from_trusted_agent_run() -> None: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + AgentRun.__table__.create(bind=engine) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as db: + run = AgentRun( + run_id="run-tenant-bound", + agent="hermes", + source="schedule", + route_json={"tenant_id": "tenant-a"}, + ontology_json={"tenant_id": "tenant-a"}, + permission_level="read", + status="running", + ) + db.add(run) + db.commit() + assert resolve_trusted_knowledge_run_tenant(db, run.run_id) == "tenant-a" + + run.ontology_json = {"tenant_id": "tenant-b"} + db.commit() + with pytest.raises(ValueError, match="冲突"): + resolve_trusted_knowledge_run_tenant(db, run.run_id) diff --git a/server/tests/test_linked_reimbursement_draft_jobs.py b/server/tests/test_linked_reimbursement_draft_jobs.py index 92dea55..a119267 100644 --- a/server/tests/test_linked_reimbursement_draft_jobs.py +++ b/server/tests/test_linked_reimbursement_draft_jobs.py @@ -26,6 +26,7 @@ from app.test_helpers.db import build_in_memory_session_factory def seed_employee_and_application(db: Session) -> None: employee = Employee( id="emp-linked-draft-fast", + tenant_id="tenant-a", employee_no="E10001", name="张三", email="zhangsan@example.com", @@ -34,6 +35,7 @@ def seed_employee_and_application(db: Session) -> None: ) application = ExpenseClaim( id="application-linked-draft-fast", + tenant_id="tenant-a", claim_no="AP-202606-FAST", employee_id=employee.id, employee_name=employee.name, @@ -76,9 +78,11 @@ def build_client(monkeypatch) -> tuple[TestClient, object]: def test_linked_reimbursement_draft_job_runs_after_conversation_leaves(monkeypatch) -> None: clear_linked_reimbursement_draft_jobs_for_tests() captured_messages = [] + captured_users = [] - def fake_run(self, payload): + def fake_run(self, payload, *, current_user=None): captured_messages.append(payload.message) + captured_users.append(current_user) return OrchestratorResponse( run_id="run-linked-draft-job", conversation_id=None, @@ -148,7 +152,13 @@ def test_linked_reimbursement_draft_job_runs_after_conversation_leaves(monkeypat assert payload["status"] == "succeeded" assert payload["draft_payload"]["claim_no"] == "RE-202606-009" assert payload["run_id"] == "run-linked-draft-job" - assert captured_messages == ["我要报销\n用户选择报销场景：差旅费\n关联申请单：AP-202606-001"] + assert captured_messages == [ + "我要报销\n用户选择报销场景：差旅费\n关联申请单：AP-202606-001" + ] + assert len(captured_users) == 1 + assert captured_users[0] is not None + assert captured_users[0].username == "zhangsan@example.com" + assert captured_users[0].tenant_id == "default" finally: clear_linked_reimbursement_draft_jobs_for_tests() @@ -158,9 +168,11 @@ def test_linked_job_overrides_forged_tenant_and_rejects_same_owner_cross_tenant( ) -> None: clear_linked_reimbursement_draft_jobs_for_tests() captured_contexts = [] + captured_users = [] - def fake_run(self, payload): + def fake_run(self, payload, *, current_user=None): captured_contexts.append(dict(payload.context_json or {})) + captured_users.append(current_user) return OrchestratorResponse( run_id="run-linked-tenant-guard", conversation_id=None, @@ -217,6 +229,10 @@ def test_linked_job_overrides_forged_tenant_and_rejects_same_owner_cross_tenant( "session_type": "expense", } ] + assert len(captured_users) == 1 + assert captured_users[0] is not None + assert captured_users[0].tenant_id == "tenant-a" + assert captured_users[0].tenant_id != "default" job_id = response.json()["job_id"] cross_tenant_response = client.get( f"/api/v1/reimbursements/linked-reimbursement-draft-jobs/{job_id}", @@ -286,6 +302,7 @@ def test_linked_reimbursement_draft_job_uses_direct_save_path(monkeypatch) -> No "x-auth-name": "Zhang San", "x-auth-employee-no": "E10001", "x-auth-role-codes": "user", + "x-auth-tenant-id": "tenant-a", } response = client.post( @@ -345,11 +362,15 @@ def test_linked_reimbursement_draft_job_uses_direct_save_path(monkeypatch) -> No clear_linked_reimbursement_draft_jobs_for_tests() -def test_linked_reimbursement_draft_job_uses_direct_save_path_with_application_no_only(monkeypatch) -> None: +def test_linked_reimbursement_draft_job_uses_direct_save_path_with_application_no_only( + monkeypatch, +) -> None: clear_linked_reimbursement_draft_jobs_for_tests() def fail_if_orchestrator_runs(self, payload): - raise AssertionError("linked draft job should resolve application no without full orchestrator") + raise AssertionError( + "linked draft job should resolve application no without full orchestrator" + ) monkeypatch.setattr(OrchestratorService, "run", fail_if_orchestrator_runs) try: @@ -362,6 +383,7 @@ def test_linked_reimbursement_draft_job_uses_direct_save_path_with_application_n "x-auth-name": "Zhang San", "x-auth-employee-no": "E10001", "x-auth-role-codes": "user", + "x-auth-tenant-id": "tenant-a", } response = client.post( @@ -409,9 +431,7 @@ def test_linked_reimbursement_draft_job_uses_direct_save_path_with_application_n draft = db.get(ExpenseClaim, payload["draft_payload"]["claim_id"]) assert draft is not None link_flag = next( - flag - for flag in draft.risk_flags_json - if flag.get("source") == "application_link" + flag for flag in draft.risk_flags_json if flag.get("source") == "application_link" ) assert link_flag["application_claim_no"] == "AP-202606-FAST" assert link_flag["application_claim_id"] == "application-linked-draft-fast" diff --git a/server/tests/test_migration_preflight.py b/server/tests/test_migration_preflight.py index 1d9703d..da279aa 100644 --- a/server/tests/test_migration_preflight.py +++ b/server/tests/test_migration_preflight.py @@ -69,7 +69,10 @@ def test_revision_0007_accepts_partial_legacy_historical_case_tables(engine: Eng @pytest.mark.parametrize( "owned_table", - sorted(MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0007"]), + sorted( + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0023"] + - LEGACY_ADOPTABLE_HISTORICAL_CASE_TABLES + ), ) def test_unversioned_database_with_any_migration_owned_table_is_rejected( engine: Engine, @@ -157,6 +160,64 @@ def test_known_revision_requires_and_accepts_its_exact_owned_table_set( "20260716_0014", MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0014"] - {"risk_disposition_events"}, ), + ( + "20260716_0015", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0015"] - {"savings_events"}, + ), + ( + "20260716_0016", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0016"] + - {"commercial_cost_events"}, + ), + ( + "20260716_0017", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0017"] + - {"financial_connector_events"}, + ), + ( + "20260716_0018", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0018"] + - {"agent_asset_release_observations"}, + ), + ( + "20260716_0019", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0019"] + - {"commercial_runtime_reservations"}, + ), + ( + "20260716_0020", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0020"] + - {"financial_connector_config_events"}, + ), + ( + "20260716_0021", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0021"] + - {"commercial_billing_periods"}, + ), + ( + "20260716_0022", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0022"] + - {"financial_connector_operational_events"}, + ), + ( + "20260716_0023", + MIGRATION_OWNED_TABLES_BY_REVISION["20260716_0023"] + - {"agent_asset_release_audit_samples"}, + ), + ( + "20260717_0025", + MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0025"] - {"tenants"}, + ), + ( + "20260717_0027", + MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0027"] + - {"knowledge_onlyoffice_sessions"}, + ), + ( + "20260717_0028", + MIGRATION_OWNED_TABLES_BY_REVISION["20260717_0028"] + - {"tenant_finance_report_runs"}, + ), ], ) def test_known_revision_with_missing_or_unexpected_owned_tables_is_rejected( diff --git a/server/tests/test_notification_states.py b/server/tests/test_notification_states.py index 32b59ac..d4a8376 100644 --- a/server/tests/test_notification_states.py +++ b/server/tests/test_notification_states.py @@ -51,8 +51,12 @@ def build_client() -> TestClient: def test_notification_state_service_persists_user_scoped_read_and_hidden_state() -> None: with build_session() as db: service = NotificationStateService(db) - user = CurrentUserContext(username="alice", name="Alice", role_codes=[], is_admin=False) - other_user = CurrentUserContext(username="bob", name="Bob", role_codes=[], is_admin=False) + user = CurrentUserContext( + tenant_id="default", username="alice", name="Alice", role_codes=[], is_admin=False + ) + other_user = CurrentUserContext( + tenant_id="default", username="bob", name="Bob", role_codes=[], is_admin=False + ) saved = service.patch_states( NotificationStateBatchPatch( @@ -90,7 +94,9 @@ def test_notification_state_service_persists_user_scoped_read_and_hidden_state() def test_notification_state_storage_ready_runs_once_per_database_bind(monkeypatch) -> None: with build_session() as db: service = NotificationStateService(db) - user = CurrentUserContext(username="alice", name="Alice", role_codes=[], is_admin=False) + user = CurrentUserContext( + tenant_id="default", username="alice", name="Alice", role_codes=[], is_admin=False + ) calls: list[object] = [] original_create_all = Base.metadata.create_all diff --git a/server/tests/test_ocr_commercial.py b/server/tests/test_ocr_commercial.py new file mode 100644 index 0000000..48b45fd --- /dev/null +++ b/server/tests/test_ocr_commercial.py @@ -0,0 +1,256 @@ +from __future__ import annotations + +import json +import subprocess +from datetime import UTC, datetime +from decimal import Decimal +from pathlib import Path +from threading import Semaphore +from types import SimpleNamespace + +import pytest +from commercial_runtime_testkit import seed_meter +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.core.config import get_settings +from app.db.base_class import Base +from app.models.commercial import UsageMeterEvent +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.commercial_direct_operation import CommercialDirectOperationBridge +from app.services.ocr import WORKER_JSON_PREFIX, OcrService +from app.services.ocr_commercial import ( + OcrCommercialAccessDenied, + OcrCommercialObserver, + OcrOperationContext, +) +from app.services.ocr_worker_runtime import invoke_ocr_worker + + +@pytest.fixture() +def factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + result = sessionmaker(bind=engine, expire_on_commit=False) + try: + yield result + finally: + Base.metadata.drop_all(engine) + engine.dispose() + + +def _seed_ocr_meter( + factory: sessionmaker[Session], + *, + preflight_quantity: Decimal = Decimal("5"), + hard_limit: Decimal = Decimal("20"), +) -> None: + with factory() as db: + seed_meter( + db, + "tenant-a", + datetime.now(UTC), + basis="pages", + tool_type="ocr", + tool_name="paddle.worker", + preflight_quantity=preflight_quantity, + hard_limit=hard_limit, + ) + db.commit() + + +def _context(suffix: str = "one") -> OcrOperationContext: + return OcrOperationContext( + tenant_id="tenant-a", + operation_id=f"ocr-operation-{suffix}", + run_id=f"ocr-run-{suffix}", + ) + + +def _settings() -> SimpleNamespace: + return SimpleNamespace( + ocr_language="ch", + ocr_device="", + ocr_text_detection_model="PP-OCRv5_mobile_det", + ocr_text_recognition_model="PP-OCRv5_mobile_rec", + ocr_timeout_seconds=10, + ) + + +def _invoke( + *, + observer: OcrCommercialObserver, + context: OcrOperationContext, + input_paths: list[Path], +) -> dict: + return invoke_ocr_worker( + settings=_settings(), + python_bin="python", + worker_path="worker.py", + input_paths=input_paths, + semaphore=Semaphore(1), + parse_stdout=_parse_json, + commercial_observer=observer, + operation_context=context, + ) + + +def _parse_json(value: str) -> dict | None: + try: + payload = json.loads(value) + except json.JSONDecodeError: + return None + return payload if isinstance(payload, dict) else None + + +def test_ocr_worker_settles_exact_prepared_page_count_once( + factory: sessionmaker[Session], + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + _seed_ocr_meter(factory) + observer = OcrCommercialObserver(CommercialDirectOperationBridge(factory)) + inputs = [tmp_path / "page-1.png", tmp_path / "page-2.png"] + monkeypatch.setattr( + "app.services.ocr_worker_runtime.subprocess.run", + lambda *args, **kwargs: subprocess.CompletedProcess( + args=args[0], returncode=0, stdout='{"documents": []}', stderr="" + ), + ) + + payload = _invoke(observer=observer, context=_context(), input_paths=inputs) + + assert payload == {"documents": []} + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + usage = db.scalars(select(UsageMeterEvent)).one() + assert reservation.status == "committed" + assert Decimal(reservation.reserved_quantity) == Decimal("2") + assert Decimal(reservation.actual_quantity or 0) == Decimal("2") + assert Decimal(usage.quantity) == Decimal("2") + + +@pytest.mark.parametrize( + ("returncode", "stdout", "expected_error"), + [ + (3, "", "OCR 执行失败"), + (0, "not-json", "JSON"), + ], +) +def test_ocr_worker_failure_after_send_still_records_real_pages( + factory: sessionmaker[Session], + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, + returncode: int, + stdout: str, + expected_error: str, +) -> None: + _seed_ocr_meter(factory) + observer = OcrCommercialObserver(CommercialDirectOperationBridge(factory)) + monkeypatch.setattr( + "app.services.ocr_worker_runtime.subprocess.run", + lambda *args, **kwargs: subprocess.CompletedProcess( + args=args[0], returncode=returncode, stdout=stdout, stderr="provider-error" + ), + ) + + with pytest.raises((RuntimeError, json.JSONDecodeError), match=expected_error): + _invoke( + observer=observer, + context=_context(f"failure-{returncode}"), + input_paths=[tmp_path / "page.png"], + ) + + with factory() as db: + usage = db.scalars(select(UsageMeterEvent)).one() + assert Decimal(usage.quantity) == Decimal("1") + + +def test_ocr_quota_is_checked_before_subprocess( + factory: sessionmaker[Session], + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + _seed_ocr_meter(factory, preflight_quantity=Decimal("1")) + observer = OcrCommercialObserver(CommercialDirectOperationBridge(factory)) + called = False + + def fail_if_called(*args, **kwargs): + nonlocal called + called = True + raise AssertionError("额度拒绝后不应启动 OCR 子进程。") + + monkeypatch.setattr("app.services.ocr_worker_runtime.subprocess.run", fail_if_called) + + with pytest.raises(OcrCommercialAccessDenied, match="preflight_quantity"): + _invoke( + observer=observer, + context=_context("over-limit"), + input_paths=[tmp_path / "page-1.png", tmp_path / "page-2.png"], + ) + + assert called is False + with factory() as db: + assert db.query(CommercialRuntimeReservation).count() == 0 + assert db.query(UsageMeterEvent).count() == 0 + + +def test_ocr_cache_hit_does_not_start_or_bill_worker_again( + factory: sessionmaker[Session], + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + _seed_ocr_meter(factory) + calls = 0 + + def fake_run(command, **kwargs): + nonlocal calls + calls += 1 + input_path = command[command.index("--input") + 1] + payload = { + "engine": "paddleocr_mobile", + "model": "PP-OCRv5_mobile_rec", + "documents": [ + { + "input_path": input_path, + "text": "增值税发票 金额 100 元", + "summary": "增值税发票", + "line_count": 1, + "page_count": 1, + "lines": [{"text": "增值税发票 金额 100 元", "score": 0.98}], + } + ], + } + return subprocess.CompletedProcess( + args=command, + returncode=0, + stdout=f"{WORKER_JSON_PREFIX}{json.dumps(payload, ensure_ascii=False)}\n", + stderr="", + ) + + monkeypatch.setenv("STORAGE_ROOT_DIR", str(tmp_path / "storage")) + monkeypatch.setattr("app.services.ocr_worker_runtime.subprocess.run", fake_run) + monkeypatch.setattr(OcrService, "_resolve_python_bin", lambda self: "python") + monkeypatch.setattr(OcrService, "_resolve_worker_path", lambda self: "worker.py") + get_settings.cache_clear() + OcrService.clear_result_cache() + content = b"same-real-image" + try: + with factory() as db: + first = OcrService(db, operation_context=_context("cache-first")) + second = OcrService(db, operation_context=_context("cache-second")) + first.recognize_files([("invoice.png", content, "image/png")]) + second.recognize_files([("renamed.png", content, "image/png")]) + finally: + OcrService.clear_result_cache() + get_settings.cache_clear() + + assert calls == 1 + with factory() as db: + assert db.query(UsageMeterEvent).count() == 1 diff --git a/server/tests/test_onlyoffice_callback_summary.py b/server/tests/test_onlyoffice_callback_summary.py index 3c685f0..0f55183 100644 --- a/server/tests/test_onlyoffice_callback_summary.py +++ b/server/tests/test_onlyoffice_callback_summary.py @@ -1,78 +1,83 @@ -import pytest -from unittest.mock import MagicMock, patch from io import BytesIO -from openpyxl import Workbook -from app.services.agent_assets import AgentAssetService -from app.schemas.agent_asset import AgentAssetOnlyOfficeCallbackWrite +from unittest.mock import MagicMock, patch -def test_onlyoffice_callback_generates_summary_note(): - # Setup mock DB and repository +from openpyxl import Workbook + +from app.services.agent_asset_onlyoffice_security import ( + AgentAssetOnlyOfficeValidatedSession, +) +from app.services.agent_assets import AgentAssetService + + +def test_onlyoffice_callback_delegates_downloaded_workbook_to_upload(): db = MagicMock() service = AgentAssetService(db) - service.repository = MagicMock() - service.spreadsheet_manager = MagicMock() service._ensure_ready = MagicMock() - - # Mock asset and metadata + asset = MagicMock() asset.id = "test-asset" - asset.name = "测试规则" + asset.tenant_id = "tenant-a" + asset.scope = "tenant" service._require_spreadsheet_rule = MagicMock(return_value=asset) - service._resolve_working_version = MagicMock(return_value="v1") - - base_meta = MagicMock() - base_meta.file_name = "test.xlsx" - base_meta.storage_key = "old-key" - base_meta.checksum = "old-checksum" - service._resolve_spreadsheet_version_meta = MagicMock(return_value=("v1", base_meta)) - - # Create base workbook - base_wb = Workbook() - base_ws = base_wb.active - base_ws["A1"] = "old value" - - # Mock loading base workbook - service._load_spreadsheet_for_compare = MagicMock(return_value=base_wb) - service.spreadsheet_manager.resolve_storage_path = MagicMock() - - # Create new content (modified) + current_metadata = MagicMock(file_name="test.xlsx", checksum="old-checksum") + service._resolve_current_spreadsheet_meta = MagicMock( + return_value=("current", current_metadata) + ) + new_wb = Workbook() new_ws = new_wb.active - new_ws["A1"] = "new value" # 1 cell changed - new_ws["B2"] = "added" # 1 more cell changed - - # Mock URL open to return new content + new_ws["A1"] = "new value" + new_ws["B2"] = "added" new_content_bio = BytesIO() new_wb.save(new_content_bio) new_content = new_content_bio.getvalue() - - with patch("app.services.agent_assets.urlopen") as mock_urlopen: - mock_response = MagicMock() - mock_response.read.return_value = new_content - mock_response.__enter__.return_value = mock_response - mock_urlopen.return_value = mock_response - - # Mock upload_rule_spreadsheet + document_key = service._build_onlyoffice_document_key("test-asset", current_metadata) + claimed = AgentAssetOnlyOfficeValidatedSession( + jti="callback-jti", + tenant_id="tenant-a", + resource_scope="tenant", + asset_id="test-asset", + document_key=document_key, + document_version="current", + document_fingerprint="old-checksum", + writable=True, + actor="username:test_user", + status="processing", + ) + session_service = MagicMock() + session_service.claim_callback.return_value = claimed + service._onlyoffice_session_service = MagicMock(return_value=session_service) + + with patch( + "app.services.agent_asset_onlyoffice.download_onlyoffice_document", + return_value=new_content, + ) as secure_download: service.upload_rule_spreadsheet = MagicMock() - - # Execute callback handler payload = { "status": 2, "url": "http://onlyoffice/download", - "users": ["test_user"] + "key": document_key, } - + service.handle_rule_spreadsheet_onlyoffice_callback( "test-asset", - version="v1", - payload=payload + version="current", + payload=payload, + callback_token="signed-callback-token", ) - - # Verify upload_rule_spreadsheet was called with correct change_note - service.upload_rule_spreadsheet.assert_called_once() - call_args = service.upload_rule_spreadsheet.call_args[1] - assert "涉及 1 个 Sheet，共 2 处改动" in call_args["change_note"] - assert call_args["actor"] == "test_user" -if __name__ == "__main__": - pytest.main([__file__]) + secure_download.assert_called_once_with( + "http://onlyoffice/download", + expected_filename="test.xlsx", + ) + service.upload_rule_spreadsheet.assert_called_once_with( + "test-asset", + filename="test.xlsx", + content=new_content, + actor="username:test_user", + source="onlyoffice", + ) + session_service.finish_callback.assert_called_once_with( + "callback-jti", + succeeded=True, + ) diff --git a/server/tests/test_ontology_employee_tenant_security.py b/server/tests/test_ontology_employee_tenant_security.py new file mode 100644 index 0000000..d4d99a3 --- /dev/null +++ b/server/tests/test_ontology_employee_tenant_security.py @@ -0,0 +1,407 @@ +from __future__ import annotations + +from collections.abc import Generator +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from fastapi.testclient import TestClient +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.db.base import Base +from app.main import create_app +from app.models.agent_run import AgentRun +from app.models.employee import Employee +from app.models.financial_record import ( + AccountsPayableRecord, + AccountsReceivableRecord, + ExpenseClaim, +) +from app.models.organization import OrganizationUnit +from app.models.tenant import Tenant +from app.schemas.ontology import OntologyParseRequest +from app.schemas.orchestrator import OrchestratorRequest +from app.services.ontology import SemanticOntologyService +from app.services.orchestrator import OrchestratorService + + +def _session_factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False) + + +def _tenant(tenant_id: str) -> Tenant: + return Tenant( + tenant_id=tenant_id, + tenant_code=tenant_id, + name=f"{tenant_id} 公司", + status="active", + ) + + +def _employee( + *, + tenant_id: str, + employee_id: str, + employee_no: str, + name: str, + email: str, + manager_id: str | None = None, +) -> Employee: + return Employee( + id=employee_id, + tenant_id=tenant_id, + employee_no=employee_no, + name=name, + email=email, + manager_id=manager_id, + ) + + +def _claim( + *, + tenant_id: str, + claim_id: str, + claim_no: str, + employee_id: str, + employee_name: str, + department_id: str, + department_name: str, + project_code: str, +) -> ExpenseClaim: + now = datetime.now(UTC) + return ExpenseClaim( + id=claim_id, + tenant_id=tenant_id, + claim_no=claim_no, + employee_id=employee_id, + employee_name=employee_name, + department_id=department_id, + department_name=department_name, + project_code=project_code, + expense_type="travel", + reason="客户拜访", + location="上海", + amount=Decimal("100.00"), + invoice_count=1, + occurred_at=now, + submitted_at=now, + status="submitted", + approval_stage="直属领导审批", + risk_flags_json=[], + ) + + +def test_ontology_reference_catalog_never_reads_another_tenant() -> None: + factory = _session_factory() + today = datetime.now(UTC).date() + with factory() as db: + db.add_all([_tenant("tenant-a"), _tenant("tenant-b")]) + db.add_all( + [ + OrganizationUnit( + id="dept-a", + tenant_id="tenant-a", + unit_code="A-FIN", + name="甲方财务部", + ), + OrganizationUnit( + id="dept-b", + tenant_id="tenant-b", + unit_code="B-FIN", + name="乙方机密部门", + ), + _employee( + tenant_id="tenant-a", + employee_id="employee-a", + employee_no="A001", + name="甲方员工", + email="employee-a@example.com", + ), + _employee( + tenant_id="tenant-b", + employee_id="employee-b", + employee_no="B001", + name="乙方机密员工", + email="employee-b@example.com", + ), + ] + ) + db.add_all( + [ + _claim( + tenant_id="tenant-a", + claim_id="claim-a", + claim_no="RE-A-001", + employee_id="employee-a", + employee_name="甲方员工", + department_id="dept-a", + department_name="甲方财务部", + project_code="PROJECT-A", + ), + _claim( + tenant_id="tenant-b", + claim_id="claim-b", + claim_no="RE-B-001", + employee_id="employee-b", + employee_name="乙方机密员工", + department_id="dept-b", + department_name="乙方机密部门", + project_code="PROJECT-B-SECRET", + ), + AccountsReceivableRecord( + tenant_id="tenant-a", + receivable_no="AR-A-001", + customer_id="customer-a", + customer_name="甲方客户", + amount_receivable=Decimal("100"), + amount_received=Decimal("0"), + amount_outstanding=Decimal("100"), + posting_date=today, + due_date=today, + status="open", + ), + AccountsReceivableRecord( + tenant_id="tenant-b", + receivable_no="AR-B-001", + customer_id="customer-b", + customer_name="乙方机密客户", + amount_receivable=Decimal("200"), + amount_received=Decimal("0"), + amount_outstanding=Decimal("200"), + posting_date=today, + due_date=today, + status="open", + ), + AccountsPayableRecord( + tenant_id="tenant-a", + payable_no="AP-A-001", + vendor_id="vendor-a", + vendor_name="甲方供应商", + amount_payable=Decimal("100"), + amount_paid=Decimal("0"), + amount_outstanding=Decimal("100"), + posting_date=today, + due_date=today, + status="open", + ), + AccountsPayableRecord( + tenant_id="tenant-b", + payable_no="AP-B-001", + vendor_id="vendor-b", + vendor_name="乙方机密供应商", + amount_payable=Decimal("200"), + amount_paid=Decimal("0"), + amount_outstanding=Decimal("200"), + posting_date=today, + due_date=today, + status="open", + ), + ] + ) + db.commit() + + catalog = SemanticOntologyService(db)._load_reference_catalog(tenant_id="tenant-a") + + assert catalog.employees == ["甲方员工"] + assert catalog.departments == ["甲方财务部"] + assert catalog.customers == ["甲方客户"] + assert catalog.vendors == ["甲方供应商"] + assert catalog.projects == ["PROJECT-A"] + all_values = [ + *catalog.employees, + *catalog.departments, + *catalog.customers, + *catalog.vendors, + *catalog.projects, + ] + assert all("机密" not in value for value in all_values) + + +def test_ontology_creates_tenant_run_before_model_and_persists_failure( + monkeypatch: pytest.MonkeyPatch, +) -> None: + factory = _session_factory() + with factory() as db: + db.add(_tenant("tenant-a")) + db.commit() + service = SemanticOntologyService(db) + observed: dict[str, str] = {} + + def fake_model_parse(**kwargs): + context = kwargs["operation_context"] + run = db.scalar(select(AgentRun).where(AgentRun.run_id == context.run_id)) + assert run is not None + assert run.status == "running" + assert run.route_json["tenant_id"] == "tenant-a" + assert run.route_json["phase"] == "pending_model_analysis" + assert context.tenant_id == "tenant-a" + observed["run_id"] = context.run_id + return None, [], None + + monkeypatch.setattr(service, "_parse_with_model", fake_model_parse) + result = service.parse( + OntologyParseRequest(query="查询本月报销金额", user_id="employee-a"), + tenant_id="tenant-a", + ) + + assert result.run_id == observed["run_id"] + succeeded = db.scalar(select(AgentRun).where(AgentRun.run_id == result.run_id)) + assert succeeded is not None + assert succeeded.route_json["tenant_id"] == "tenant-a" + assert succeeded.ontology_json["tenant_id"] == "tenant-a" + + with pytest.raises(ValueError, match="仅支持财务业务"): + service.parse( + OntologyParseRequest(query="今天天气怎么样", user_id="employee-a"), + tenant_id="tenant-a", + ) + failed = db.scalars( + select(AgentRun).where(AgentRun.status == "failed").order_by(AgentRun.started_at.desc()) + ).first() + assert failed is not None + assert failed.route_json["tenant_id"] == "tenant-a" + assert failed.route_json["phase"] == "failed" + + +def test_employee_profile_api_hides_cross_tenant_and_enforces_manager_scope() -> None: + factory = _session_factory() + with factory() as db: + db.add_all([_tenant("tenant-a"), _tenant("tenant-b")]) + db.add_all( + [ + _employee( + tenant_id="tenant-a", + employee_id="manager-a", + employee_no="A-MGR", + name="甲方经理", + email="manager-a@example.com", + ), + _employee( + tenant_id="tenant-a", + employee_id="employee-a", + employee_no="A001", + name="甲方员工", + email="employee-a@example.com", + manager_id="manager-a", + ), + _employee( + tenant_id="tenant-b", + employee_id="employee-b", + employee_no="B001", + name="乙方员工", + email="employee-b@example.com", + ), + ] + ) + db.add( + _claim( + tenant_id="tenant-b", + claim_id="claim-b", + claim_no="RE-B-001", + employee_id="employee-b", + employee_name="乙方员工", + department_id="dept-b", + department_name="乙方部门", + project_code="PROJECT-B", + ) + ) + db.commit() + + current = { + "user": CurrentUserContext( + username="manager-a@example.com", + name="甲方经理", + role_codes=["manager"], + is_admin=False, + tenant_id="tenant-a", + employee_id="manager-a", + ) + } + app = create_app() + + def override_db() -> Generator[Session, None, None]: + db = factory() + try: + yield db + finally: + db.close() + + app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: current["user"] + client = TestClient(app) + + assert client.get("/api/v1/employee-profiles/employee-a/latest").status_code == 200 + assert client.get("/api/v1/employee-profiles/employee-b/latest").status_code == 404 + assert ( + client.get( + "/api/v1/employee-profiles/employee-a/latest", + params={"claim_id": "claim-b"}, + ).status_code + == 404 + ) + + current["user"] = CurrentUserContext( + username="employee-a@example.com", + name="甲方员工", + role_codes=[], + is_admin=False, + tenant_id="tenant-a", + employee_id="employee-a", + ) + assert client.get("/api/v1/employee-profiles/manager-a/latest").status_code == 404 + assert client.get("/api/v1/employee-profiles/employee-a/latest").status_code == 200 + + +def test_orchestrator_rejects_untrusted_or_mismatched_tenant_before_run() -> None: + factory = _session_factory() + payload = OrchestratorRequest( + source="user_message", + user_id="employee-a@example.com", + message="查询本月报销金额", + context_json={"tenant_id": "forged-tenant"}, + ) + with factory() as db: + db.add(_tenant("tenant-a")) + db.add( + Tenant( + tenant_id="tenant-suspended", + tenant_code="tenant-suspended", + name="已停用公司", + status="suspended", + ) + ) + db.commit() + service = OrchestratorService(db) + + with pytest.raises(ValueError, match="缺少可信租户上下文"): + service.run(payload) + with pytest.raises(ValueError, match="不存在或未启用"): + service.run(payload, trusted_tenant_id="unknown-tenant") + with pytest.raises(ValueError, match="不存在或未启用"): + service.run(payload, trusted_tenant_id="tenant-suspended") + + current_user = CurrentUserContext( + username="employee-a@example.com", + name="甲方员工", + role_codes=[], + is_admin=False, + tenant_id="tenant-a", + employee_id="employee-a", + ) + with pytest.raises(ValueError, match="租户不一致"): + service.run( + payload, + current_user=current_user, + trusted_tenant_id="tenant-b", + ) + + assert db.scalar(select(AgentRun.id)) is None diff --git a/server/tests/test_ontology_service.py b/server/tests/test_ontology_service.py index a0d5d2a..cc5fcfa 100644 --- a/server/tests/test_ontology_service.py +++ b/server/tests/test_ontology_service.py @@ -1026,11 +1026,12 @@ def test_semantic_ontology_service_records_model_call_errors_for_statistics(monk session_factory = build_session_factory() with session_factory() as db: service = SemanticOntologyService(db) - run = service.run_service.create_run( - agent=AgentName.ORCHESTRATOR.value, - source=AgentRunSource.USER_MESSAGE.value, - status=AgentRunStatus.RUNNING.value, - ) + run = service.run_service.create_run( + agent=AgentName.ORCHESTRATOR.value, + source=AgentRunSource.USER_MESSAGE.value, + tenant_id="default", + status=AgentRunStatus.RUNNING.value, + ) monkeypatch.setattr( service.runtime_chat_service, diff --git a/server/tests/test_orchestrator_review_flow.py b/server/tests/test_orchestrator_review_flow.py index 2b627f0..4d573f7 100644 --- a/server/tests/test_orchestrator_review_flow.py +++ b/server/tests/test_orchestrator_review_flow.py @@ -16,11 +16,14 @@ from app.models.ai_application_preview import AIApplicationPreviewDecision from app.models.ai_learning import AIDecision, AIDecisionFeedback from app.models.employee import Employee from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.tenant import Tenant from app.schemas.ontology import OntologyParseResult, OntologyPermission from app.schemas.orchestrator import OrchestratorRequest from app.services.agent_conversations import AgentConversationService from app.services.orchestrator import OrchestratorService +FIXTURE_TENANT_ID = "default" + def build_session_factory() -> sessionmaker[Session]: engine = create_engine( @@ -29,7 +32,47 @@ def build_session_factory() -> sessionmaker[Session]: poolclass=StaticPool, ) Base.metadata.create_all(bind=engine) - return sessionmaker(bind=engine, autoflush=False, autocommit=False) + factory = sessionmaker(bind=engine, autoflush=False, autocommit=False) + with factory() as db: + db.add( + Tenant( + tenant_id=FIXTURE_TENANT_ID, + tenant_code="orchestrator-review-fixture", + name="Orchestrator 回归测试租户", + status="active", + ) + ) + db.commit() + return factory + + +def run_for_fixture_tenant( + service: OrchestratorService, + payload: OrchestratorRequest, +): + """显式使用已注册 fixture 租户，不提供生产默认租户回退。""" + + return service.run(payload, trusted_tenant_id=FIXTURE_TENANT_ID) + + +def seed_application_employee(db: Session, *, email: str) -> None: + """为申请提交回归提供可唯一解析的真实直属领导关系。""" + + manager = Employee( + tenant_id=FIXTURE_TENANT_ID, + employee_no="E-APP-MANAGER", + name="陈硕", + email="application-manager@example.com", + ) + employee = Employee( + tenant_id=FIXTURE_TENANT_ID, + employee_no="E-APP-EMPLOYEE", + name="申请员工", + email=email, + manager=manager, + ) + db.add_all([manager, employee]) + db.commit() @pytest.fixture(autouse=True) @@ -92,7 +135,10 @@ def test_schedule_digital_employee_task_runs_real_service( run_id=run_id, ) - monkeypatch.setattr("app.services.ontology.SemanticOntologyService.parse_for_run", parse_for_run) + monkeypatch.setattr( + "app.services.ontology.SemanticOntologyService.parse_for_run", + parse_for_run, + ) monkeypatch.setattr(method_path, lambda self, **kwargs: dict(summary)) session_factory = build_session_factory() @@ -114,8 +160,9 @@ def test_schedule_digital_employee_task_runs_real_service( db.add(task) db.commit() - response = OrchestratorService(db).run( - OrchestratorRequest(source="schedule", task_id=task.id, message=task.name) + response = run_for_fixture_tenant( + OrchestratorService(db), + OrchestratorRequest(source="schedule", task_id=task.id, message=task.name), ) run = db.query(AgentRun).filter_by(run_id=response.run_id).one() @@ -176,7 +223,8 @@ def test_review_next_step_run_submits_existing_claim_and_returns_draft_payload( db.add_all([manager, employee, claim]) db.commit() - response = OrchestratorService(db).run( + response = run_for_fixture_tenant( + OrchestratorService(db), OrchestratorRequest( source="user_message", user_id="emp-next@example.com", @@ -187,7 +235,7 @@ def test_review_next_step_run_submits_existing_claim_and_returns_draft_payload( "attachment_count": 1, "name": "张三", }, - ) + ), ) db.refresh(claim) @@ -246,7 +294,8 @@ def test_review_next_step_blocked_returns_reasons_and_removes_next_step_action( db.add_all([employee, claim]) db.commit() - response = OrchestratorService(db).run( + response = run_for_fixture_tenant( + OrchestratorService(db), OrchestratorRequest( source="user_message", user_id="emp-blocked@example.com", @@ -257,7 +306,7 @@ def test_review_next_step_blocked_returns_reasons_and_removes_next_step_action( "attachment_count": 1, "name": "张三", }, - ) + ), ) result = response.result @@ -490,7 +539,8 @@ def test_orchestrator_history_query_filters_location_time_and_returns_real_amoun db.add_all([employee, beijing_claim, shanghai_claim, current_year_claim]) db.commit() - response = OrchestratorService(db).run( + response = run_for_fixture_tenant( + OrchestratorService(db), OrchestratorRequest( source="user_message", user_id="history-query@example.com", @@ -499,7 +549,7 @@ def test_orchestrator_history_query_filters_location_time_and_returns_real_amoun "client_now_iso": "2026-05-21T04:00:00.000Z", "client_timezone_offset_minutes": -480, }, - ) + ), ) query_payload = response.result["query_payload"] @@ -570,7 +620,8 @@ def test_orchestrator_archive_query_filters_archived_claims_and_limits_preview( db.add_all([employee, *claims, draft_claim]) db.commit() - response = OrchestratorService(db).run( + response = run_for_fixture_tenant( + OrchestratorService(db), OrchestratorRequest( source="user_message", user_id="archive-query@example.com", @@ -579,7 +630,7 @@ def test_orchestrator_archive_query_filters_archived_claims_and_limits_preview( "client_now_iso": "2026-05-21T04:00:00.000Z", "client_timezone_offset_minutes": -480, }, - ) + ), ) query_payload = response.result["query_payload"] @@ -588,9 +639,11 @@ def test_orchestrator_archive_query_filters_archived_claims_and_limits_preview( assert query_payload["record_count"] == 6 assert query_payload["preview_count"] == 5 assert query_payload["preview_limit"] == 5 - assert query_payload["title"] == "最近 5 条你的归档报销单" + assert query_payload["title"] == "最近 5 条您的归档报销单" assert all(record["status"] == "approved" for record in query_payload["records"]) - assert "EXP-ARCHIVE-DRAFT" not in [record["claim_no"] for record in query_payload["records"]] + assert "EXP-ARCHIVE-DRAFT" not in [ + record["claim_no"] for record in query_payload["records"] + ] assert response.result["suggested_actions"] == [] assert "下面先列出最近 5 条记录" in response.result["answer"] @@ -612,22 +665,23 @@ def test_orchestrator_expense_preview_does_not_persist_claim_before_user_action( db.add(employee) db.commit() - response = OrchestratorService(db).run( + response = run_for_fixture_tenant( + OrchestratorService(db), OrchestratorRequest( source="user_message", user_id="preview-orchestrator@example.com", message="业务发生时间:2026-03-04，打车去客户现场，交通费32元，请帮我看看怎么报", context_json={ "name": "预览员工", - "user_input_text": "业务发生时间:2026-03-04，打车去客户现场，交通费32元，请帮我看看怎么报", + "user_input_text": ( + "业务发生时间:2026-03-04，打车去客户现场，交通费32元，请帮我看看怎么报" + ), }, - ) + ), ) user_claims = [ - claim - for claim in db.query(ExpenseClaim).all() - if claim.employee_name == "预览员工" + claim for claim in db.query(ExpenseClaim).all() if claim.employee_name == "预览员工" ] assert response.status == "succeeded" assert response.result.get("review_payload") is not None @@ -660,25 +714,32 @@ def test_orchestrator_prompts_scene_choices_before_review_for_fresh_ambiguous_ex }, ) - response = OrchestratorService(db).run( + response = run_for_fixture_tenant( + OrchestratorService(db), OrchestratorRequest( source="user_message", user_id="emp-scene-choice@example.com", conversation_id=conversation.conversation_id, - message="业务发生时间:2026-02-20 至 2026-02-23，去上海支持上海电力部署项目，申请报销", + message=( + "业务发生时间:2026-02-20 至 2026-02-23，去上海支持上海电力部署项目，申请报销" + ), context_json={ "session_type": "expense", "draft_claim_id": "claim-old", }, - ) + ), ) result = response.result - assert response.status == "succeeded" + assert response.status == "blocked" assert result.get("review_payload") is None assert result.get("draft_payload") is None assert "请先在下面选择报销场景" in result["answer"] - assert [item["label"] for item in result["suggested_actions"][:3]] == ["差旅费", "交通费", "住宿费"] + assert [item["label"] for item in result["suggested_actions"][:3]] == [ + "差旅费", + "交通费", + "住宿费", + ] def test_orchestrator_application_session_does_not_use_reimbursement_scene_prompt( @@ -689,14 +750,10 @@ def test_orchestrator_application_session_does_not_use_reimbursement_scene_promp lambda *_args, **_kwargs: None, ) session_factory = build_session_factory() - message = ( - "发生时间：2026-05-25\n" - "地点：上海\n" - "事由：支持上海国网服务器部署\n" - "天数：3天" - ) + message = "发生时间：2026-05-25\n地点：上海\n事由：支持上海国网服务器部署\n天数：3天" with session_factory() as db: - response = OrchestratorService(db).run( + response = run_for_fixture_tenant( + OrchestratorService(db), OrchestratorRequest( source="user_message", user_id="application-session@example.com", @@ -706,7 +763,7 @@ def test_orchestrator_application_session_does_not_use_reimbursement_scene_promp "entry_source": "application", "name": "申请员工", }, - ) + ), ) result = response.result @@ -727,51 +784,53 @@ def test_orchestrator_application_session_guides_transport_estimate_and_submit( lambda *_args, **_kwargs: None, ) session_factory = build_session_factory() - initial_message = ( - "发生时间：2026-05-25\n" - "地点：上海\n" - "事由：支持上海国网服务器部署\n" - "天数：3天" - ) + initial_message = "发生时间：2026-05-25\n地点：上海\n事由：支持上海国网服务器部署\n天数：3天" context_json = { "session_type": "application", "entry_source": "application", "name": "申请员工", + "department_name": "交付部", "manager_name": "陈硕", } with session_factory() as db: + seed_application_employee(db, email="application-flow@example.com") service = OrchestratorService(db) - first = service.run( + first = run_for_fixture_tenant( + service, OrchestratorRequest( source="user_message", user_id="application-flow@example.com", message=initial_message, context_json=context_json, - ) + ), ) - second = service.run( + second = run_for_fixture_tenant( + service, OrchestratorRequest( source="user_message", user_id="application-flow@example.com", conversation_id=first.conversation_id, message="飞机", context_json=context_json, - ) + ), ) - third = service.run( + third = run_for_fixture_tenant( + service, OrchestratorRequest( source="user_message", user_id="application-flow@example.com", conversation_id=first.conversation_id, message="确认提交", context_json=context_json, - ) + ), ) assert first.status == "blocked" assert "当前还需要补充：出行方式" in first.result["answer"] - assert [item["label"] for item in first.result["suggested_actions"]] == ["一次性补充申请信息"] + assert [item["label"] for item in first.result["suggested_actions"]] == [ + "一次性补充申请信息" + ] assert first.result["suggested_actions"][0]["payload"]["prompt_prefill"] == "出行方式：" assert "这是费用申请核对结果" in second.result["answer"] @@ -793,11 +852,7 @@ def test_orchestrator_application_session_guides_transport_estimate_and_submit( assert "申请单据已生成，并已进入审批流程" in third.result["answer"] assert "系统已推送给 陈硕 审核，当前节点：陈硕审核中" in third.result["answer"] assert third.result["suggested_actions"] == [] - application_claims = [ - claim - for claim in db.query(ExpenseClaim).all() - if claim.claim_no.startswith("AP-") - ] + application_claims = db.query(ExpenseClaim).all() assert len(application_claims) == 1 assert application_claims[0].status == "submitted" assert application_claims[0].approval_stage == "直属领导审批" @@ -812,37 +867,36 @@ def test_orchestrator_application_submit_bypasses_generic_operation_block( lambda *_args, **_kwargs: None, ) session_factory = build_session_factory() - initial_message = ( - "发生时间：2026-05-25\n" - "地点：上海\n" - "事由：支持上海国网服务器部署\n" - "天数：3天" - ) + initial_message = "发生时间：2026-05-25\n地点：上海\n事由：支持上海国网服务器部署\n天数：3天" context_json = { "session_type": "application", "entry_source": "application", "name": "申请员工", + "department_name": "交付部", "manager_name": "陈硕", } with session_factory() as db: + seed_application_employee(db, email="application-approval-required@example.com") service = OrchestratorService(db) - first = service.run( + first = run_for_fixture_tenant( + service, OrchestratorRequest( source="user_message", user_id="application-approval-required@example.com", message=initial_message, context_json=context_json, - ) + ), ) - preview = service.run( + preview = run_for_fixture_tenant( + service, OrchestratorRequest( source="user_message", user_id="application-approval-required@example.com", conversation_id=first.conversation_id, message="飞机", context_json=context_json, - ) + ), ) def approval_required_parse_for_run(self, request, run_id): # noqa: ANN001 @@ -867,14 +921,15 @@ def test_orchestrator_application_submit_bypasses_generic_operation_block( "app.services.ontology.SemanticOntologyService.parse_for_run", approval_required_parse_for_run, ) - submitted = service.run( + submitted = run_for_fixture_tenant( + service, OrchestratorRequest( source="user_message", user_id="application-approval-required@example.com", conversation_id=first.conversation_id, message="确认提交", context_json=context_json, - ) + ), ) assert preview.status == "blocked" @@ -946,10 +1001,7 @@ def test_authenticated_orchestrator_application_draft_consumes_server_preview_de source="user_message", user_id="forged@example.com", message=( - "发生时间：2026-05-25\n" - "地点：上海\n" - "事由：支持上海国网服务器部署\n" - "天数：3天" + "发生时间：2026-05-25\n地点：上海\n事由：支持上海国网服务器部署\n天数：3天" ), context_json=context_json, ), @@ -1010,9 +1062,7 @@ def test_authenticated_orchestrator_application_draft_consumes_server_preview_de ) assert learning_decision is not None feedback = db.scalar( - select(AIDecisionFeedback).where( - AIDecisionFeedback.decision_id == learning_decision.id - ) + select(AIDecisionFeedback).where(AIDecisionFeedback.decision_id == learning_decision.id) ) assert feedback is not None assert feedback.feedback_type == "accepted" diff --git a/server/tests/test_receipt_folder_service.py b/server/tests/test_receipt_folder_service.py index c515fc5..e88b525 100644 --- a/server/tests/test_receipt_folder_service.py +++ b/server/tests/test_receipt_folder_service.py @@ -9,11 +9,14 @@ from app.services.document_preview import DocumentPreviewAssets from app.services.receipt_folder import ReceiptFolderService -def test_receipt_folder_train_ticket_uses_invoice_date_and_enriches_fields(monkeypatch, tmp_path) -> None: +def test_receipt_folder_train_ticket_uses_invoice_date_and_enriches_fields( + monkeypatch, tmp_path +) -> None: monkeypatch.setenv("STORAGE_ROOT_DIR", str(tmp_path / "storage")) get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -77,6 +80,7 @@ def test_receipt_folder_pdf_save_eagerly_renders_image_preview(monkeypatch, tmp_ get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -87,7 +91,9 @@ def test_receipt_folder_pdf_save_eagerly_renders_image_preview(monkeypatch, tmp_ preview_path.write_bytes(b"rendered-preview") return preview_path - monkeypatch.setattr(DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page) + monkeypatch.setattr( + DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page + ) service = ReceiptFolderService() receipt = service.save_receipt( @@ -121,11 +127,14 @@ def test_receipt_folder_pdf_save_eagerly_renders_image_preview(monkeypatch, tmp_ get_settings.cache_clear() -def test_receipt_folder_persist_enriches_pdf_ocr_document_with_image_preview(monkeypatch, tmp_path) -> None: +def test_receipt_folder_persist_enriches_pdf_ocr_document_with_image_preview( + monkeypatch, tmp_path +) -> None: monkeypatch.setenv("STORAGE_ROOT_DIR", str(tmp_path / "storage")) get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -136,7 +145,9 @@ def test_receipt_folder_persist_enriches_pdf_ocr_document_with_image_preview(mon preview_path.write_bytes(b"rendered-preview") return preview_path - monkeypatch.setattr(DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page) + monkeypatch.setattr( + DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page + ) service = ReceiptFolderService() result = service.persist_ocr_batch( @@ -162,7 +173,9 @@ def test_receipt_folder_persist_enriches_pdf_ocr_document_with_image_preview(mon document = result.documents[0] assert document.receipt_id - assert document.receipt_preview_url.endswith(f"/receipt-folder/{document.receipt_id}/preview") + assert document.receipt_preview_url.endswith( + f"/receipt-folder/{document.receipt_id}/preview" + ) assert document.preview_kind == "image" finally: get_settings.cache_clear() @@ -173,13 +186,16 @@ def test_receipt_folder_pdf_preview_regenerates_stale_cached_image(monkeypatch, get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], is_admin=False, ) stale_preview = b"stale-preview" - preview_data_url = f"data:image/png;base64,{base64.b64encode(stale_preview).decode('ascii')}" + preview_data_url = ( + f"data:image/png;base64,{base64.b64encode(stale_preview).decode('ascii')}" + ) service = ReceiptFolderService() receipt = service.save_receipt( filename="2月20_武汉-上海.pdf", @@ -205,7 +221,9 @@ def test_receipt_folder_pdf_preview_regenerates_stale_cached_image(monkeypatch, preview_path.write_bytes(b"refreshed-preview") return preview_path - monkeypatch.setattr(DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page) + monkeypatch.setattr( + DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page + ) resolved_path, media_type, file_name = service.resolve_preview(receipt.id, current_user) @@ -227,13 +245,16 @@ def test_receipt_folder_pdf_preview_falls_back_to_source_when_render_fonts_missi get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], is_admin=False, ) stale_preview = b"broken-preview" - preview_data_url = f"data:image/png;base64,{base64.b64encode(stale_preview).decode('ascii')}" + preview_data_url = ( + f"data:image/png;base64,{base64.b64encode(stale_preview).decode('ascii')}" + ) service = ReceiptFolderService() receipt = service.save_receipt( filename="2月20_武汉-上海.pdf", @@ -256,7 +277,9 @@ def test_receipt_folder_pdf_preview_falls_back_to_source_when_render_fonts_missi def fake_render_pdf_first_page(*, pdf_path, preview_path, timeout_seconds): raise RuntimeError("Missing language pack for 'Adobe-GB1' mapping") - monkeypatch.setattr(DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page) + monkeypatch.setattr( + DocumentPreviewAssets, "render_pdf_first_page", fake_render_pdf_first_page + ) resolved_path, media_type, file_name = service.resolve_preview(receipt.id, current_user) @@ -280,6 +303,7 @@ def test_receipt_folder_train_ticket_extracts_passenger_from_id_line_and_purchas get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -314,9 +338,13 @@ def test_receipt_folder_train_ticket_extracts_passenger_from_id_line_and_purchas scene_code="travel", scene_label="差旅票据", document_fields=[ - OcrRecognizeFieldRead(key="merchant_name", label="商户", value="电子发票（铁路"), + OcrRecognizeFieldRead( + key="merchant_name", label="商户", value="电子发票（铁路" + ), OcrRecognizeFieldRead(key="amount", label="金额", value="354元"), - OcrRecognizeFieldRead(key="date", label="列车出发时间", value="2026-02-20 07:55"), + OcrRecognizeFieldRead( + key="date", label="列车出发时间", value="2026-02-20 07:55" + ), OcrRecognizeFieldRead(key="trip_no", label="车次", value="G458"), OcrRecognizeFieldRead(key="route", label="行程", value="武汉-上海"), ], @@ -349,6 +377,7 @@ def test_receipt_folder_train_ticket_repairs_invalid_generated_fields_from_ocr_t get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -401,12 +430,20 @@ def test_receipt_folder_train_ticket_repairs_invalid_generated_fields_from_ocr_t scene_label="差旅票据", document_fields=[ OcrRecognizeFieldRead(key="amount", label="金额", value="438元"), - OcrRecognizeFieldRead(key="date", label="列车出发时间", value="2026-02-21 08:30"), - OcrRecognizeFieldRead(key="invoice_number", label="票据号码", value="DEMO202602210001"), + OcrRecognizeFieldRead( + key="date", label="列车出发时间", value="2026-02-21 08:30" + ), + OcrRecognizeFieldRead( + key="invoice_number", label="票据号码", value="DEMO202602210001" + ), OcrRecognizeFieldRead(key="trip_no", label="车次", value="G999"), OcrRecognizeFieldRead(key="route", label="行程", value="上海-深圳"), - OcrRecognizeFieldRead(key="departure_station", label="出发地点", value="二等座"), - OcrRecognizeFieldRead(key="arrival_station", label="到达地点", value="扫码无效"), + OcrRecognizeFieldRead( + key="departure_station", label="出发地点", value="二等座" + ), + OcrRecognizeFieldRead( + key="arrival_station", label="到达地点", value="扫码无效" + ), OcrRecognizeFieldRead(key="passenger_name", label="乘车人", value="席别二等座"), ], ), @@ -431,6 +468,7 @@ def test_receipt_folder_delete_removes_duplicate_marker(monkeypatch, tmp_path) - get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -480,11 +518,14 @@ def test_receipt_folder_delete_removes_duplicate_marker(monkeypatch, tmp_path) - get_settings.cache_clear() -def test_receipt_folder_duplicate_uses_newer_ocr_when_existing_meta_is_weaker(monkeypatch, tmp_path) -> None: +def test_receipt_folder_duplicate_uses_newer_ocr_when_existing_meta_is_weaker( + monkeypatch, tmp_path +) -> None: monkeypatch.setenv("STORAGE_ROOT_DIR", str(tmp_path / "storage")) get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -538,7 +579,9 @@ def test_receipt_folder_duplicate_uses_newer_ocr_when_existing_meta_is_weaker(mo assert document.receipt_id == stale_receipt.id assert document.document_type == "train_ticket" assert document.document_type_label == "火车/高铁票" - assert any(field.label == "金额" and field.value == "354元" for field in document.document_fields) + assert any( + field.label == "金额" and field.value == "354元" for field in document.document_fields + ) assert any("重复上传" in warning for warning in document.warnings) repaired = service.get_receipt(stale_receipt.id, current_user) @@ -549,11 +592,14 @@ def test_receipt_folder_duplicate_uses_newer_ocr_when_existing_meta_is_weaker(mo get_settings.cache_clear() -def test_receipt_folder_recovers_train_ticket_detail_from_other_english_ocr(monkeypatch, tmp_path) -> None: +def test_receipt_folder_recovers_train_ticket_detail_from_other_english_ocr( + monkeypatch, tmp_path +) -> None: monkeypatch.setenv("STORAGE_ROOT_DIR", str(tmp_path / "storage")) get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], @@ -610,11 +656,14 @@ def test_receipt_folder_recovers_train_ticket_detail_from_other_english_ocr(monk get_settings.cache_clear() -def test_receipt_folder_unlink_receipts_for_claim_marks_linked_receipts_unlinked(monkeypatch, tmp_path) -> None: +def test_receipt_folder_unlink_receipts_for_claim_marks_linked_receipts_unlinked( + monkeypatch, tmp_path +) -> None: monkeypatch.setenv("STORAGE_ROOT_DIR", str(tmp_path / "storage")) get_settings.cache_clear() try: current_user = CurrentUserContext( + tenant_id="default", username="pytest", name="Py Test", role_codes=[], diff --git a/server/tests/test_reimbursement_endpoints.py b/server/tests/test_reimbursement_endpoints.py index d1dcb8c..9a622f0 100644 --- a/server/tests/test_reimbursement_endpoints.py +++ b/server/tests/test_reimbursement_endpoints.py @@ -439,26 +439,34 @@ def test_claim_standard_adjustment_endpoint_recalculates_and_marks_reviewer_noti "item_id": item_id, "title": "住宿超标待说明", "risk": "住宿票据金额超过职级标准。", - "application_days": 2, - "original_amount": "1000.00", - "reimbursable_amount": "1000.00", + "application_days": 99, + "original_amount": "1.00", + "reimbursable_amount": "999999.00", } - ] + ], + "request_id": "standard-adjustment-endpoint-1", }, headers={"x-auth-username": "emp-1", "x-auth-name": "Zhang San", "x-auth-grade": "P4"}, ) assert response.status_code == 200 payload = response.json() - assert payload["amount"] == "900.00" + assert payload["amount"] == "450.00" standard_flag = next( flag for flag in payload["risk_flags_json"] if isinstance(flag, dict) and flag.get("source") == "reimbursement_standard_adjustment" ) assert standard_flag["original_amount"] == "1000.00" - assert standard_flag["reimbursable_amount"] == "900.00" - assert standard_flag["employee_absorbed_amount"] == "100.00" + assert standard_flag["reimbursable_amount"] == "450.00" + assert standard_flag["employee_absorbed_amount"] == "550.00" + assert standard_flag["calculation_source"] == "server_policy" + assert standard_flag["policy_days"] == 1 + assert standard_flag["policy_grade"] == "P4" + assert standard_flag["policy_matched_city"] == "北京" + assert standard_flag["policy_hotel_rate"] == "450.00" + assert standard_flag["policy_rule_name"] + assert standard_flag["policy_rule_version"] assert standard_flag["visibility_scope"] == "leader" @@ -781,6 +789,82 @@ def test_approve_claim_endpoint_routes_direct_manager_claim_to_finance_review() assert ledgers[0].completed_at is not None +def test_claim_owner_cannot_discover_or_operate_manager_approval_task() -> None: + client, session_factory = build_client() + with session_factory() as db: + superior = Employee( + id="manager-self-approval-superior", + employee_no="E-SELF-APPROVAL-SUPERIOR", + name="王总", + email="superior-self-approval-api@example.com", + ) + manager = Employee( + id="manager-self-approval-owner", + employee_no="E-SELF-APPROVAL-OWNER", + name="李经理", + email="manager-self-approval-api@example.com", + manager=superior, + ) + claim = ExpenseClaim( + id="claim-self-approval-api", + claim_no="EXP-SELF-APPROVAL-API", + employee_id=manager.id, + employee_name=manager.name, + department_name="市场部", + expense_type="transport", + reason="交通报销", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime(2026, 5, 13, tzinfo=UTC), + submitted_at=datetime(2026, 5, 13, 10, 0, tzinfo=UTC), + status="submitted", + approval_stage="直属领导审批", + risk_flags_json=[], + ) + db.add_all([superior, manager, claim]) + db.commit() + + headers = { + "X-Auth-Username": "manager-self-approval-api@example.com", + "X-Auth-Name": "manager-self-approval-api@example.com", + "X-Auth-Role-Codes": "manager", + } + approve_response = client.post( + "/api/v1/reimbursements/claims/claim-self-approval-api/approve", + json={ + "opinion": "同意", + "request_id": "claim-self-approval-approve", + "expected_status": "submitted", + "expected_approval_stage": "直属领导审批", + }, + headers=headers, + ) + return_response = client.post( + "/api/v1/reimbursements/claims/claim-self-approval-api/return", + json={ + "reason": "退回", + "request_id": "claim-self-approval-return", + "expected_status": "submitted", + "expected_approval_stage": "直属领导审批", + }, + headers=headers, + ) + + assert approve_response.status_code == 404 + assert return_response.status_code == 404 + assert approve_response.json()["detail"] == "Approval task not found." + assert return_response.json()["detail"] == "Approval task not found." + with session_factory() as db: + claim = db.get(ExpenseClaim, "claim-self-approval-api") + assert claim is not None + assert claim.status == "submitted" + assert claim.approval_stage == "直属领导审批" + assert claim.risk_flags_json == [] + assert db.scalar(select(func.count(ApprovalActionLedger.id))) == 0 + + def test_approve_claim_endpoint_blocks_open_high_risk_with_machine_readable_detail() -> None: client, session_factory = build_client() with session_factory() as db: diff --git a/server/tests/test_risk_dispositions.py b/server/tests/test_risk_dispositions.py index 83bd176..b2aa245 100644 --- a/server/tests/test_risk_dispositions.py +++ b/server/tests/test_risk_dispositions.py @@ -14,12 +14,19 @@ from sqlalchemy.pool import StaticPool from app.api.deps import CurrentUserContext, get_db from app.api.v1.endpoints.risk_observations import router as risk_observations_router +from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus, AgentAssetType from app.db.base import Base +from app.models.agent_asset import AgentAsset +from app.models.agent_asset_release_telemetry import AgentAssetReleaseLabel from app.models.employee import Employee from app.models.financial_record import ExpenseClaim from app.models.risk_disposition import RiskDispositionEvent from app.models.risk_observation import RiskObservationFeedback from app.schemas.risk_disposition import RiskDispositionActionCreate +from app.services.agent_asset_release_telemetry import ( + AgentAssetReleaseTelemetryService, + ReleaseObservationInput, +) from app.services.risk_dispositions import ( RiskDispositionConflictError, RiskDispositionIdempotencyConflictError, @@ -120,6 +127,121 @@ def test_risk_disposition_separates_adjudication_and_lifecycle_with_audit_events ) +def test_typed_risk_disposition_appends_current_release_label_automatically( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("FEW_SHOT_INJECTION_ENABLED", "false") + with _build_session() as db: + db.add(_claim()) + asset = AgentAsset( + id="risk-disposition-release-asset", + tenant_id="default", + scope="tenant", + asset_type=AgentAssetType.RULE.value, + code="risk.disposition.release", + name="处置学习发布规则", + description="", + domain=AgentAssetDomain.EXPENSE.value, + scenario_json=["travel"], + owner="finance", + status=AgentAssetStatus.ACTIVE.value, + current_version="v1", + published_version="v1", + working_version="v2", + config_json={ + "tenant_id": "default", + "detail_mode": "json_risk", + "stable_marker": "stable-risk-disposition-release", + "release_guard": { + "release_id": "risk-disposition-release-1", + "stage": "shadow", + "candidate_version": "v2", + "previous_version": "v1", + "previous_config": { + "tenant_id": "default", + "detail_mode": "json_risk", + "enabled": True, + "stable_marker": "stable-risk-disposition-release", + }, + "policy": { + "shadow_min_samples": 1, + "canary_min_samples": 1, + "max_error_rate": 1.0, + "min_precision": 0.9, + "max_precision_drop": 1.0, + "canary_traffic_percent": 10, + }, + }, + }, + ) + db.add(asset) + db.commit() + telemetry = AgentAssetReleaseTelemetryService(db).record_observation( + ReleaseObservationInput( + tenant_id="default", + asset_id=asset.id, + release_id="risk-disposition-release-1", + stage="shadow", + version="v2", + rule_code=asset.code, + source_key="claim-risk-1", + candidate_hit=True, + baseline_hit=True, + business_stage="reimbursement", + ) + ) + observation = RiskObservationService(db).upsert_observation( + { + **_observation_payload("risk:typed:release-telemetry"), + "risk_signal": asset.code, + "algorithm_version": "v1", + "policy_refs": [asset.code], + "decision_trace": {"rule_code": asset.code}, + } + ) + db.commit() + + mutation = RiskDispositionService(db).execute_action( + observation.id, + _action("confirm", version=0, request_id="request-release-label-001"), + tenant_id="default", + actor_id="finance-secret-account", + actor_name="财务甲", + ) + + label = db.scalar( + select(AgentAssetReleaseLabel).where( + AgentAssetReleaseLabel.observation_id == telemetry.id + ) + ) + assert mutation.disposition.adjudication == "confirmed" + assert label is not None + assert label.label == "confirmed" + assert label.verification_source == "typed_risk_disposition" + assert "finance-secret-account" not in str(label.__dict__) + + corrected = RiskDispositionService(db).execute_action( + observation.id, + _action( + "false_positive", + version=1, + request_id="request-release-label-correction-001", + ), + tenant_id="default", + actor_id="finance-secret-account", + actor_name="财务甲", + ) + db.expire_all() + refreshed_asset = db.get(AgentAsset, asset.id) + + assert corrected.disposition.adjudication == "false_positive" + assert refreshed_asset is not None + assert refreshed_asset.published_version == "v1" + assert refreshed_asset.config_json["stable_marker"] == ("stable-risk-disposition-release") + assert refreshed_asset.config_json["release_guard"]["stage"] == "rolled_back" + assert refreshed_asset.config_json["release_guard"]["rollback"]["automatic"] is True + + def test_risk_disposition_requires_confirmation_before_remediation_or_resolution( monkeypatch: pytest.MonkeyPatch, ) -> None: @@ -558,6 +680,7 @@ def test_current_claim_approver_can_manage_disposition_without_pool_access() -> ) db.commit() current_user = CurrentUserContext( + tenant_id="default", username="risk.manager@example.com", name="风险主管", role_codes=["approver"], @@ -571,12 +694,14 @@ def test_current_claim_approver_can_manage_disposition_without_pool_access() -> assert policy.can_manage_disposition(observation, current_user) is True unrelated_manager = CurrentUserContext( + tenant_id="default", username="unrelated.manager@example.com", name="其他经理", role_codes=["manager"], is_admin=False, ) finance_outside_stage = CurrentUserContext( + tenant_id="default", username="finance@example.com", name="财务甲", role_codes=["finance"], @@ -618,6 +743,7 @@ def test_disposition_rechecks_current_approver_after_claim_stage_changes() -> No ) db.commit() current_user = CurrentUserContext( + tenant_id="default", username=manager.email, name=manager.name, role_codes=["manager"], @@ -986,11 +1112,7 @@ def _action( action=action, expected_version=version, request_id=request_id, - comment=( - "风险处置说明" - if action in {"false_positive", "request_supplement"} - else None - ), + comment=("风险处置说明" if action in {"false_positive", "request_supplement"} else None), **waiver_fields, ) diff --git a/server/tests/test_risk_observations_service.py b/server/tests/test_risk_observations_service.py index 6976876..9f4ea31 100644 --- a/server/tests/test_risk_observations_service.py +++ b/server/tests/test_risk_observations_service.py @@ -421,6 +421,8 @@ def test_hermes_global_scan_builds_graphs_inside_each_tenant( _claim_orm("claim-tenant-a", "BX-TENANT-A"), _claim_orm("claim-tenant-b", "BX-TENANT-B"), ] + claims[0].tenant_id = "tenant-a" + claims[1].tenant_id = "tenant-b" cases = [ ExpenseCase( id=f"case-tenant-{suffix}", @@ -459,21 +461,27 @@ def test_hermes_global_scan_builds_graphs_inside_each_tenant( return [] scanner = HermesRiskScannerService(db) - monkeypatch.setattr(scanner, "_fetch_unscanned_claims", lambda: claims) + monkeypatch.setattr( + scanner, + "_fetch_unscanned_claims", + lambda *, tenant_id: [claim for claim in claims if claim.tenant_id == tenant_id], + ) monkeypatch.setattr( "app.services.hermes_risk_scanner.evaluate_financial_risk_graph", fake_evaluate, ) monkeypatch.setattr(RiskObservationService, "build_history_stats", fake_history) - summary = scanner.scan_global_risks() + summary_a = scanner.scan_global_risks(tenant_id="tenant-a") + summary_b = scanner.scan_global_risks(tenant_id="tenant-b") assert evaluated_claim_sets == [ {"claim-tenant-a"}, {"claim-tenant-b"}, ] assert history_tenants == ["tenant-a", "tenant-b"] - assert summary["scanned_claim_count"] == 2 + assert summary_a["scanned_claim_count"] == 1 + assert summary_b["scanned_claim_count"] == 1 def test_risk_scan_discards_snapshot_after_claim_changes_during_evaluation( @@ -493,7 +501,11 @@ def test_risk_scan_discards_snapshot_after_claim_changes_during_evaluation( return SimpleNamespace(observations=[], nodes=[], edges=[]) scanner = HermesRiskScannerService(db) - monkeypatch.setattr(scanner, "_fetch_unscanned_claims", lambda: [claim]) + monkeypatch.setattr( + scanner, + "_fetch_unscanned_claims", + lambda *, tenant_id: [claim] if claim.tenant_id == tenant_id else [], + ) monkeypatch.setattr( "app.services.hermes_risk_scanner.evaluate_financial_risk_graph", fake_evaluate, diff --git a/server/tests/test_risk_rule_feedback.py b/server/tests/test_risk_rule_feedback.py index 87bc93d..bd58441 100644 --- a/server/tests/test_risk_rule_feedback.py +++ b/server/tests/test_risk_rule_feedback.py @@ -115,7 +115,7 @@ def test_risk_rule_feedback_endpoint_allows_ordinary_user_and_manager_list(tmp_p ) assert response.status_code == 201 - assert response.json()["created_by"] == "employee" + assert response.json()["created_by"] == "username:employee" assert response.json()["status"] == "open" list_response = client.get( diff --git a/server/tests/test_risk_rule_generation.py b/server/tests/test_risk_rule_generation.py index 320e607..852e5e8 100644 --- a/server/tests/test_risk_rule_generation.py +++ b/server/tests/test_risk_rule_generation.py @@ -11,6 +11,7 @@ from sqlalchemy import create_engine from sqlalchemy.orm import Session, sessionmaker from sqlalchemy.pool import StaticPool +from app.api.deps import CurrentUserContext from app.core.agent_enums import ( AgentAssetDomain, AgentAssetStatus, @@ -21,6 +22,7 @@ from app.db.base import Base from app.models.agent_asset import AgentAsset from app.models.employee import Employee from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.golden_case import GoldenCase from app.schemas.agent_asset import ( AgentAssetReviewCreate, AgentAssetRiskRuleGenerateRequest, @@ -29,18 +31,22 @@ from app.schemas.agent_asset import ( AgentAssetRiskRuleScenarioTestRequest, AgentAssetRiskRuleSimulationRequest, ) +from app.services.agent_asset_release_guard import ( + AgentAssetReleaseGuardService, + ReleaseEvaluationInput, +) from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager from app.services.agent_asset_spreadsheet import RISK_RULES_LIBRARY from app.services.agent_assets import AgentAssetService from app.services.agent_foundation_risk_rules import AgentFoundationRiskRuleMixin from app.services.expense_claim_platform_risk import ExpenseClaimPlatformRiskMixin -from app.services.risk_rule_manifest_classifier import is_budget_risk_manifest from app.services.risk_rule_flow_diagram import ( RiskRuleFlowDiagramRenderer, RiskRuleFlowDiagramSpec, ) from app.services.risk_rule_generation import RiskRuleGenerationService from app.services.risk_rule_generation_jobs import RiskRuleGenerationJobService +from app.services.risk_rule_manifest_classifier import is_budget_risk_manifest from app.services.risk_rule_manifest_normalizer import normalize_risk_rule_manifest from app.services.risk_rule_scoring import calculate_risk_rule_score, risk_level_from_score from app.services.risk_rule_template_executor import RiskRuleTemplateExecutor @@ -1135,9 +1141,7 @@ def test_legacy_city_route_keyword_manifest_is_normalized_before_display_and_exe "outcomes": {"fail": {"severity": "high"}}, "metadata": { "condition_summary": "检查住宿城市、申报地点、行程城市是否出现规则描述中的风险关键词", - "flow": { - "decision": "检查住宿城市、申报地点、行程城市是否出现规则描述中的风险关键词" - }, + "flow": {"decision": "检查住宿城市、申报地点、行程城市是否出现规则描述中的风险关键词"}, }, "flow_diagram_svg": ( '' @@ -1293,7 +1297,16 @@ def test_simulation_uses_current_rule_manifest_for_ticket_city_mismatch(tmp_path ), actor="pytest", ) - service = AgentAssetService(db) + service = AgentAssetService( + db, + current_user=CurrentUserContext( + username="pytest", + name="pytest", + role_codes=["manager"], + is_admin=True, + tenant_id="default", + ), + ) service.rule_library_manager = manager asset = db.get(AgentAsset, asset_id) assert asset is not None @@ -1357,7 +1370,16 @@ def test_risk_rule_requires_test_report_before_review_and_publish(tmp_path) -> N ), actor="pytest", ) - service = AgentAssetService(db) + service = AgentAssetService( + db, + current_user=CurrentUserContext( + username="pytest", + name="pytest", + role_codes=["manager"], + is_admin=True, + tenant_id="default", + ), + ) service.rule_library_manager = manager asset = db.get(AgentAsset, asset_id) @@ -1430,7 +1452,10 @@ def test_risk_rule_requires_test_report_before_review_and_publish(tmp_path) -> N scenario = service.run_risk_rule_scenario_test( asset_id, - AgentAssetRiskRuleScenarioTestRequest(intent="用最近30天的住宿报销单试运行"), + AgentAssetRiskRuleScenarioTestRequest( + target_tenant_id="default", + intent="用最近30天的住宿报销单试运行", + ), actor="pytest", ) assert scenario.passed is True @@ -1443,6 +1468,21 @@ def test_risk_rule_requires_test_report_before_review_and_publish(tmp_path) -> N ) assert report.passed is True + for case in sample.input_json["cases"]: + db.add( + GoldenCase( + case_key=f"{asset.code}:{case['case_id']}", + rule_code=asset.code, + name=case["name"], + values_json=case["values"], + expected_hit=case["expected_hit"], + expected_severity=case["expected_severity"], + status="active", + source="pytest", + ) + ) + db.commit() + review = service.create_review( asset_id, AgentAssetReviewCreate( @@ -1455,8 +1495,65 @@ def test_risk_rule_requires_test_report_before_review_and_publish(tmp_path) -> N ) assert review.review_status == AgentReviewStatus.PENDING.value published = service.publish_risk_rule(asset_id, actor="manager") + assert published.status == AgentAssetStatus.REVIEW.value + assert published.published_version is None + assert published.config_json["release_guard"]["stage"] == "shadow" + approved = service.create_review( + asset_id, + AgentAssetReviewCreate( + version=asset.working_version or "v0.1.0", + reviewer="manager", + review_status=AgentReviewStatus.APPROVED, + review_note="批准进入灰度发布", + ), + actor="manager", + ) + assert approved.review_status == AgentReviewStatus.APPROVED.value + release_guard = AgentAssetReleaseGuardService( + db, + rule_library_manager=manager, + ) + release_guard.record_evaluation( + asset_id, + ReleaseEvaluationInput( + total=20, + failure_count=0, + precision=1.0, + details={ + "metric_source": "release_runtime_telemetry", + "negative_ground_truth_status": "available", + "recall_lower_bound": 1.0, + }, + ), + actor="monitor", + ) + release_guard.promote(asset_id, actor="manager") + release_guard.record_evaluation( + asset_id, + ReleaseEvaluationInput( + total=100, + failure_count=0, + precision=1.0, + details={ + "metric_source": "release_runtime_telemetry", + "negative_ground_truth_status": "available", + "recall_lower_bound": 1.0, + }, + ), + actor="monitor", + ) + release_guard.promote(asset_id, actor="manager") + published = db.get(AgentAsset, asset_id) + assert published is not None assert published.status == AgentAssetStatus.ACTIVE.value assert published.published_version == asset.working_version + golden_run = service.repository.get_latest_test_run( + asset_id, + version=asset.working_version, + test_type="golden", + ) + assert golden_run is not None + assert golden_run.passed is True disabled = service.set_risk_rule_enabled( asset_id, diff --git a/server/tests/test_risk_rule_golden_evaluator.py b/server/tests/test_risk_rule_golden_evaluator.py index 6acf1ff..91df9f0 100644 --- a/server/tests/test_risk_rule_golden_evaluator.py +++ b/server/tests/test_risk_rule_golden_evaluator.py @@ -1,9 +1,6 @@ from __future__ import annotations -from collections.abc import Generator -from datetime import datetime -from decimal import Decimal -from unittest.mock import MagicMock, patch +from unittest.mock import patch import pytest from sqlalchemy import create_engine @@ -12,11 +9,8 @@ from sqlalchemy.pool import StaticPool from app.db.base import Base from app.models.agent_asset import AgentAsset, AgentAssetTestRun -from app.models.employee import Employee -from app.models.financial_record import ExpenseClaim from app.models.golden_case import GoldenCase from app.services.risk_rule_golden_evaluator import ( - GoldenEvalReport, RiskRuleGoldenEvaluator, _aggregate, _run_single_case, @@ -141,19 +135,36 @@ def test_aggregate_with_failure() -> None: results = [ GoldenCaseResult("1", "a", True, True, "high", "high", True), - GoldenCaseResult("2", "b", True, False, "high", "none", False), # FP + GoldenCaseResult("2", "b", True, False, "high", "none", False), # FN ] report = _aggregate(results) assert report.passed_count == 1 assert report.failed_count == 1 assert report.accuracy == 0.5 assert report.all_passed is False - assert report.precision == 0.5 # 1/(1+1) + assert report.precision == 1.0 + assert report.recall == 0.5 # 1/(1+1) + + +def test_aggregate_false_positive_reduces_precision_not_recall() -> None: + from app.services.risk_rule_golden_evaluator import GoldenCaseResult + + report = _aggregate( + [ + GoldenCaseResult("1", "a", True, True, "high", "high", True), + GoldenCaseResult("2", "b", False, True, "", "high", False), + ] + ) + + assert report.precision == 0.5 + assert report.recall == 1.0 def test_evaluate_for_rule_empty_returns_passed() -> None: with _build_session() as db: - report = RiskRuleGoldenEvaluator().evaluate_for_rule(db, _keyword_manifest(), "risk.test.keyword") + report = RiskRuleGoldenEvaluator().evaluate_for_rule( + db, _keyword_manifest(), "risk.test.keyword" + ) assert report.total == 0 assert report.all_passed is True @@ -163,7 +174,9 @@ def test_evaluate_for_rule_all_pass() -> None: db.add(_golden_case("g1", reason="虚假发票", expected_hit=True)) db.add(_golden_case("g2", reason="正常报销", expected_hit=False)) db.commit() - report = RiskRuleGoldenEvaluator().evaluate_for_rule(db, _keyword_manifest(), "risk.test.keyword") + report = RiskRuleGoldenEvaluator().evaluate_for_rule( + db, _keyword_manifest(), "risk.test.keyword" + ) assert report.total == 2 assert report.all_passed is True assert report.accuracy == 1.0 @@ -172,9 +185,11 @@ def test_evaluate_for_rule_all_pass() -> None: def test_evaluate_for_rule_with_failure() -> None: with _build_session() as db: db.add(_golden_case("g1", reason="虚假发票", expected_hit=False)) # 期望不命中但实际命中 - db.add(_golden_case("g2", reason="正常报销", expected_hit=True)) # 期望命中但实际不命中 + db.add(_golden_case("g2", reason="正常报销", expected_hit=True)) # 期望命中但实际不命中 db.commit() - report = RiskRuleGoldenEvaluator().evaluate_for_rule(db, _keyword_manifest(), "risk.test.keyword") + report = RiskRuleGoldenEvaluator().evaluate_for_rule( + db, _keyword_manifest(), "risk.test.keyword" + ) assert report.total == 2 assert report.all_passed is False assert report.failed_count == 2 @@ -206,6 +221,7 @@ def test_require_pass_passes_when_all_green() -> None: # 应写一条 test_type='golden' 记录 run = db.query(AgentAssetTestRun).filter_by(asset_id="a1", test_type="golden").one() assert run.passed is True + assert run.status == "passed" def test_require_pass_raises_on_failure() -> None: @@ -220,18 +236,22 @@ def test_require_pass_raises_on_failure() -> None: ) run = db.query(AgentAssetTestRun).filter_by(asset_id="a2", test_type="golden").one() assert run.passed is False + assert run.status == "failed" + assert run.result_json["failure_reason"] == "golden_case_regression" -def test_require_pass_empty_golden_set_passes() -> None: +def test_require_pass_empty_golden_set_fails_closed() -> None: with _build_session() as db: asset = _asset("a3", "R3") db.add(asset) db.commit() - report = RiskRuleGoldenEvaluator().require_pass( - db, asset, "v1", _keyword_manifest(), "risk.test.keyword", actor="tester" - ) - assert report.total == 0 - assert report.all_passed is True + with pytest.raises(PermissionError, match="缺少必要"): + RiskRuleGoldenEvaluator().require_pass( + db, asset, "v1", _keyword_manifest(), "risk.test.keyword", actor="tester" + ) + run = db.query(AgentAssetTestRun).filter_by(asset_id="a3", test_type="golden").one() + assert run.passed is False + assert run.result_json["failure_reason"] == "missing_active_golden_cases" def test_require_pass_respects_feature_flag(monkeypatch: pytest.MonkeyPatch) -> None: @@ -246,17 +266,45 @@ def test_require_pass_respects_feature_flag(monkeypatch: pytest.MonkeyPatch) -> db, asset, "v1", _keyword_manifest(), "risk.test.keyword", actor="tester" ) assert report.total == 0 + assert report.gate_status == "skipped" + run = db.query(AgentAssetTestRun).filter_by(asset_id="a4", test_type="golden").one() + assert run.status == "skipped" + assert run.result_json["failure_reason"] == "gate_explicitly_disabled" -def test_require_pass_swallows_evaluator_exception() -> None: +def test_require_pass_records_and_blocks_precondition_error() -> None: + with _build_session() as db: + asset = _asset("a-precondition", "R-PRECONDITION") + db.add(asset) + db.commit() + + with pytest.raises(PermissionError, match="前置条件失败"): + RiskRuleGoldenEvaluator().require_pass( + db, + asset, + "v1", + {}, + "", + actor="tester", + precondition_error="missing_rule_document", + ) + + run = db.query(AgentAssetTestRun).filter_by(asset_id=asset.id).one() + assert run.status == "failed" + assert run.result_json["failure_reason"] == "missing_rule_document" + + +def test_require_pass_fails_closed_on_evaluator_exception() -> None: with _build_session() as db: asset = _asset("a5", "R5") db.add(asset) db.commit() evaluator = RiskRuleGoldenEvaluator() with patch.object(evaluator, "evaluate_for_rule", side_effect=RuntimeError("boom")): - report = evaluator.require_pass( - db, asset, "v1", _keyword_manifest(), "risk.test.keyword", actor="tester" - ) - assert report.total == 0 - assert report.all_passed is True # 降级放行 + with pytest.raises(PermissionError, match="fail-closed"): + evaluator.require_pass( + db, asset, "v1", _keyword_manifest(), "risk.test.keyword", actor="tester" + ) + run = db.query(AgentAssetTestRun).filter_by(asset_id="a5", test_type="golden").one() + assert run.passed is False + assert run.result_json["failure_reason"] == "evaluation_error:RuntimeError" diff --git a/server/tests/test_risk_rule_revision_endpoints.py b/server/tests/test_risk_rule_revision_endpoints.py index 5b80c55..51dbb56 100644 --- a/server/tests/test_risk_rule_revision_endpoints.py +++ b/server/tests/test_risk_rule_revision_endpoints.py @@ -111,7 +111,10 @@ def test_create_risk_rule_revision_endpoint_keeps_active_version(tmp_path) -> No assert payload["working_version"] == "v0.1.1" assert revision["version"] == "v0.1.1" assert revision["base_version"] == "v0.1.0" - assert revision["generation_request"]["natural_language"] == "票据城市与申报目的地不一致时，要求补充说明。" + assert ( + revision["generation_request"]["natural_language"] + == "票据城市与申报目的地不一致时，要求补充说明。" + ) assert payload["config_json"]["last_operation"]["action"] == "create_revision" @@ -133,7 +136,11 @@ def test_regenerate_risk_rule_endpoint_returns_updated_detail(tmp_path, monkeypa assert asset is not None config = dict(asset.config_json or {}) config["generation_status"] = "completed" - config["last_operation"] = {"action": "regenerate", "actor": actor, "at": "2026-05-30T00:00:00+00:00"} + config["last_operation"] = { + "action": "regenerate", + "actor": actor, + "at": "2026-05-30T00:00:00+00:00", + } asset.config_json = config self.db.add(asset) self.db.flush() @@ -209,7 +216,7 @@ def test_manager_can_toggle_risk_rule_enabled_endpoint(tmp_path, monkeypatch) -> assert response.status_code == 200 assert response.json()["config_json"]["enabled"] is False - assert response.json()["config_json"]["last_operation"]["actor"] == "manager" + assert response.json()["config_json"]["last_operation"]["actor"] == "username:manager" def _create_rule(session_factory: sessionmaker[Session], tmp_path) -> str: @@ -225,6 +232,7 @@ def _create_rule(session_factory: sessionmaker[Session], tmp_path) -> str: rule_title="差旅规则草稿", natural_language="差旅报销事由缺失时，提示补充说明。", ), + tenant_id="default", actor="pytest", ) diff --git a/server/tests/test_risk_rule_revision_service.py b/server/tests/test_risk_rule_revision_service.py index 547223a..5249bf8 100644 --- a/server/tests/test_risk_rule_revision_service.py +++ b/server/tests/test_risk_rule_revision_service.py @@ -8,17 +8,22 @@ from sqlalchemy.pool import StaticPool from app.core.agent_enums import AgentAssetDomain, AgentAssetStatus from app.db.base import Base from app.models.agent_asset import AgentAsset, AgentAssetTestRun, AgentAssetVersion +from app.models.golden_case import GoldenCase from app.schemas.agent_asset import ( AgentAssetRiskRuleDraftUpdate, AgentAssetRiskRuleGenerateRequest, AgentAssetRiskRuleRegenerateRequest, AgentAssetRiskRuleRevisionCreate, ) -from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager +from app.services.agent_asset_release_guard import ( + AgentAssetReleaseGuardService, + ReleaseEvaluationInput, +) from app.services.agent_asset_risk_rule_regeneration import AgentAssetRiskRuleRegenerationService +from app.services.agent_asset_risk_rule_revision import AgentAssetRiskRuleRevisionService +from app.services.agent_asset_rule_library import AgentAssetRuleLibraryManager from app.services.agent_assets import AgentAssetService from app.services.risk_rule_generation import RiskRuleGenerationService -from app.services.agent_asset_risk_rule_revision import AgentAssetRiskRuleRevisionService class NullRuntimeChatService: @@ -76,7 +81,9 @@ def test_update_published_rule_requires_revision(tmp_path) -> None: ) -def test_create_revision_draft_for_published_rule_does_not_overwrite_active_version(tmp_path) -> None: +def test_create_revision_draft_for_published_rule_does_not_overwrite_active_version( + tmp_path, +) -> None: with build_session() as db: asset_id = _create_rule(db, tmp_path) asset = db.get(AgentAsset, asset_id) @@ -105,7 +112,10 @@ def test_create_revision_draft_for_published_rule_does_not_overwrite_active_vers assert updated.working_version == "v0.1.1" assert revision["version"] == "v0.1.1" assert revision["base_version"] == "v0.1.0" - assert revision["generation_request"]["natural_language"] == "票据城市与申报目的地不一致时，要求补充说明。" + assert ( + revision["generation_request"]["natural_language"] + == "票据城市与申报目的地不一致时，要求补充说明。" + ) assert updated.config_json["last_operation"]["action"] == "create_revision" assert db.query(AgentAssetVersion).filter_by(asset_id=asset_id, version="v0.1.1").one() @@ -251,21 +261,87 @@ def test_publish_regenerated_revision_replaces_online_document(tmp_path) -> None service = AgentAssetService(db) service.rule_library_manager = manager + revision_manifest = manager.read_rule_library_json( + library="risk-rules", + file_name=revision["rule_document"]["file_name"], + ) + for case in service._build_default_sample_cases(revision_manifest): + db.add( + GoldenCase( + case_key=f"{asset.code}:{case.case_id}", + rule_code=asset.code, + name=case.name, + values_json=case.values, + expected_hit=case.expected_hit, + expected_severity=case.expected_severity, + status="active", + source="pytest", + ) + ) + db.commit() published = service.publish_risk_rule(asset_id, actor="manager") assert published.status == AgentAssetStatus.ACTIVE.value + assert published.current_version == "v0.1.0" + assert published.published_version == "v0.1.0" + assert published.config_json["release_guard"]["stage"] == "shadow" + release_guard = AgentAssetReleaseGuardService( + db, + rule_library_manager=manager, + ) + release_guard.record_evaluation( + asset_id, + ReleaseEvaluationInput( + total=20, + failure_count=0, + precision=1.0, + details={ + "metric_source": "release_runtime_telemetry", + "negative_ground_truth_status": "available", + "recall_lower_bound": 1.0, + }, + ), + actor="monitor", + ) + release_guard.promote(asset_id, actor="manager") + release_guard.record_evaluation( + asset_id, + ReleaseEvaluationInput( + total=100, + failure_count=0, + precision=1.0, + details={ + "metric_source": "release_runtime_telemetry", + "negative_ground_truth_status": "available", + "recall_lower_bound": 1.0, + }, + ), + actor="monitor", + ) + release_guard.promote(asset_id, actor="manager") + published = db.get(AgentAsset, asset_id) + assert published is not None assert published.current_version == "v0.1.1" assert published.published_version == "v0.1.1" assert "revision_draft" not in published.config_json assert published.config_json["rule_document"] == revision["rule_document"] - assert published.config_json["revision_history"][0]["previous_rule_document"] == old_document - assert published.config_json["last_operation"]["action"] == "publish_revision" + assert ( + published.config_json["revision_history"][0]["previous_rule_document"] == old_document + ) + assert published.config_json["last_operation"]["action"] == "activate_staged_release" manifest = manager.read_rule_library_json( library="risk-rules", file_name=published.config_json["rule_document"]["file_name"], ) - assert manifest["enabled"] is True + assert manifest["enabled"] is False assert manifest["rule_code"] == published.code + golden_run = service.repository.get_latest_test_run( + asset_id, + version="v0.1.1", + test_type="golden", + ) + assert golden_run is not None + assert golden_run.passed is True def _create_rule( diff --git a/server/tests/test_runtime_chat_attempts.py b/server/tests/test_runtime_chat_attempts.py new file mode 100644 index 0000000..b71b188 --- /dev/null +++ b/server/tests/test_runtime_chat_attempts.py @@ -0,0 +1,716 @@ +from __future__ import annotations + +import pytest +from runtime_chat_testkit import ( + DenyingAttemptObserver, + FailingCompletionObserver, + FailingPermitObserver, + RecordingAttemptObserver, + build_operation_context, + patch_single_slot, +) +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +from app.db.base import Base +from app.services import runtime_chat as runtime_chat_module +from app.services.model_connectivity import ConnectivityCheckError +from app.services.runtime_chat import ( + RuntimeChatOperationContext, + RuntimeChatService, +) + + +def build_session_factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + return sessionmaker(bind=engine, autoflush=False, autocommit=False) + + +def _clear_runtime_chat_cooldown() -> None: + runtime_chat_module._slot_failure_until.clear() + + +@pytest.mark.parametrize( + ("provider", "configured_model", "response_payload", "expected_usage"), + [ + ( + "OpenAI Compatible", + "gpt-configured", + { + "id": "chatcmpl-openai-001", + "model": "gpt-response", + "choices": [{"message": {"content": "openai answer"}}], + "usage": { + "prompt_tokens": 17, + "completion_tokens": 5, + "total_tokens": 22, + }, + }, + { + "prompt_tokens": 17, + "completion_tokens": 5, + "total_tokens": 22, + "prompt_eval_count": None, + "eval_count": None, + }, + ), + ( + "Azure OpenAI", + "azure-deployment", + { + "id": "chatcmpl-azure-001", + "model": "azure-response-model", + "choices": [{"message": {"content": "azure answer"}}], + "usage": { + "prompt_tokens": 23, + "completion_tokens": 7, + "total_tokens": 30, + }, + }, + { + "prompt_tokens": 23, + "completion_tokens": 7, + "total_tokens": 30, + "prompt_eval_count": None, + "eval_count": None, + }, + ), + ( + "Ollama", + "llama-configured", + { + "model": "llama-response", + "message": {"content": "ollama answer"}, + "prompt_eval_count": 31, + "eval_count": 11, + }, + { + "prompt_tokens": None, + "completion_tokens": None, + "total_tokens": None, + "prompt_eval_count": 31, + "eval_count": 11, + }, + ), + ], +) +def test_runtime_chat_preserves_authoritative_provider_usage( + monkeypatch, + provider: str, + configured_model: str, + response_payload: dict[str, object], + expected_usage: dict[str, int | None], +) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + observer = RecordingAttemptObserver() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider=provider, + model=configured_model, + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: (200, response_payload), + ) + + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + assert result.text is not None + assert len(result.calls) == 1 + trace = result.calls[0] + assert trace.provider == provider + assert trace.model == configured_model + assert trace.attempt == 1 + assert trace.response_id == response_payload.get("id") + assert trace.response_model == response_payload.get("model") + assert trace.outcome == "succeeded" + assert trace.started_at is not None + assert trace.completed_at is not None + assert trace.completed_at >= trace.started_at + assert trace.usage.availability == "available" + for field_name, expected_value in expected_usage.items(): + assert getattr(trace.usage, field_name) == expected_value + assert trace.observer_status == "completed_notified" + assert len(observer.permits) == 1 + assert len(observer.completed) == 1 + assert observer.completed[0].identity.tenant_id == "tenant-runtime-chat" + assert observer.completed[0].identity.invocation_seq == 7 + assert observer.completed[0].usage == trace.usage + + +def test_runtime_chat_keeps_missing_usage_unavailable_without_estimation( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + observer = RecordingAttemptObserver() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-no-usage", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: ( + 200, + { + "id": "chatcmpl-no-usage", + "model": "gpt-no-usage", + "choices": [{"message": {"content": "answer"}}], + }, + ), + ) + + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_tokens=9876, + max_attempts=1, + operation_context=build_operation_context(), + ) + + usage = result.calls[0].usage + assert usage.availability == "unavailable" + assert usage.source == "unavailable" + assert usage.prompt_tokens is None + assert usage.completion_tokens is None + assert usage.total_tokens is None + assert observer.completed[0].usage == usage + + +def test_runtime_chat_notifies_every_real_retry_attempt_and_marks_timeout_unknown( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + timeline: list[str] = [] + observer = RecordingAttemptObserver(timeline) + request_count = 0 + + def fake_send_json_request(*_args, **_kwargs): + nonlocal request_count + request_count += 1 + timeline.append(f"send:{request_count}") + if request_count == 1: + raise TimeoutError("provider response timed out") + return 200, { + "id": "chatcmpl-retry-002", + "model": "gpt-retry-response", + "choices": [ + { + "message": { + "tool_calls": [ + { + "id": "call-retry-002", + "function": { + "name": "submit_plan", + "arguments": "{}", + }, + } + ] + } + } + ], + "usage": { + "prompt_tokens": 41, + "completion_tokens": 3, + "total_tokens": 44, + }, + } + + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-retry", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + fake_send_json_request, + ) + monkeypatch.setattr("app.services.runtime_chat.sleep", lambda *_args: None) + + result = service.complete_with_tool_call( + [{"role": "user", "content": "build plan"}], + tools=[{"type": "function", "function": {"name": "submit_plan"}}], + slot_priority=("main",), + max_attempts=2, + use_failure_cooldown=False, + operation_context=build_operation_context(), + ) + + assert result.tool_call is not None + assert [item.attempt for item in result.calls] == [1, 2] + assert [item.outcome for item in result.calls] == [ + "outcome_unknown", + "succeeded", + ] + assert result.calls[0].usage.availability == "unavailable" + assert result.calls[1].usage.total_tokens == 44 + assert [item.identity.attempt for item in observer.permits] == [1, 2] + assert [item.identity.attempt for item in observer.completed] == [1, 2] + assert observer.completed[0].request_may_have_been_sent is True + assert observer.completed[0].requires_reconciliation is True + assert result.calls[0].requires_reconciliation is True + assert observer.permits[0].identity.attempt_key != observer.permits[1].identity.attempt_key + assert timeline == [ + "permit:1", + "send:1", + "completed:1:outcome_unknown", + "permit:2", + "send:2", + "completed:2:succeeded", + ] + + +def test_runtime_chat_preserves_usage_when_tool_response_postprocessing_fails( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + observer = RecordingAttemptObserver() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-postprocess", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: ( + 200, + { + "id": "chatcmpl-postprocess", + "model": "gpt-postprocess-response", + "choices": [ + { + "message": { + "tool_calls": [ + { + "id": "call-invalid-json", + "function": { + "name": "submit_plan", + "arguments": "{invalid-json", + }, + } + ] + } + } + ], + "usage": { + "prompt_tokens": 13, + "completion_tokens": 2, + "total_tokens": 15, + }, + }, + ), + ) + + result = service.complete_with_tool_call( + [{"role": "user", "content": "build plan"}], + tools=[{"type": "function", "function": {"name": "submit_plan"}}], + slot_priority=("main",), + max_attempts=1, + use_failure_cooldown=False, + operation_context=build_operation_context(), + ) + + assert result.tool_call is None + assert result.calls[0].outcome == "postprocess_failed" + assert result.calls[0].response_id == "chatcmpl-postprocess" + assert result.calls[0].usage.total_tokens == 15 + assert observer.completed[0].outcome == "postprocess_failed" + assert observer.completed[0].usage.total_tokens == 15 + + +def test_runtime_chat_observer_failure_keeps_success_and_reports_reconciliation( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + observer = FailingCompletionObserver() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-observer-failure", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: ( + 200, + { + "id": "chatcmpl-observer-failure", + "model": "gpt-observer-response", + "choices": [{"message": {"content": "valid answer"}}], + "usage": { + "prompt_tokens": 8, + "completion_tokens": 2, + "total_tokens": 10, + }, + }, + ), + ) + + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + assert result.text == "valid answer" + trace = result.calls[0] + assert trace.outcome == "succeeded" + assert trace.usage.total_tokens == 10 + assert trace.observer_status == "reconciliation_required" + assert [item.phase for item in trace.observer_failures] == ["completion"] + assert len(observer.failures) == 1 + assert observer.failures[0].identity.operation_id == "operation-runtime-chat-001" + + +def test_runtime_chat_without_operation_context_does_not_invoke_observer( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + observer = RecordingAttemptObserver() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-no-context", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: ( + 200, + { + "id": "chatcmpl-no-context", + "choices": [{"message": {"content": "legacy answer"}}], + "usage": { + "prompt_tokens": 5, + "completion_tokens": 2, + "total_tokens": 7, + }, + }, + ), + ) + + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + ) + + assert result.text == "legacy answer" + assert result.calls[0].usage.total_tokens == 7 + assert result.calls[0].observer_status == "not_applicable" + assert observer.permits == [] + assert observer.completed == [] + assert observer.failures == [] + + +def test_runtime_chat_explicit_permit_denial_is_not_sent(monkeypatch) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + observer = DenyingAttemptObserver() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-denied", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: pytest.fail("permit 拒绝后不应发送请求"), + ) + + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + assert result.text is None + assert result.calls[0].status == "blocked" + assert result.calls[0].outcome == "not_sent" + assert observer.completed[0].request_may_have_been_sent is False + assert observer.completed[0].usage.availability == "unavailable" + + +def test_runtime_chat_provider_http_error_is_rejected_with_status( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + observer = RecordingAttemptObserver() + session_factory = build_session_factory() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-rate-limited", + ) + + def reject_request(*_args, **_kwargs): + raise ConnectivityCheckError("rate limited", status_code=429) + + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + reject_request, + ) + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + assert result.text is None + trace = result.calls[0] + assert trace.outcome == "provider_rejected" + assert trace.provider_status_code == 429 + assert trace.requires_reconciliation is False + assert observer.completed[0].provider_status_code == 429 + assert observer.completed[0].request_may_have_been_sent is True + + +def test_runtime_chat_rejected_response_preserves_authoritative_usage( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + observer = RecordingAttemptObserver() + session_factory = build_session_factory() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-rejected-with-usage", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: ( + 429, + { + "id": "chatcmpl-rejected", + "model": "gpt-rejected-response", + "usage": { + "prompt_tokens": 9, + "completion_tokens": 0, + "total_tokens": 9, + }, + }, + ), + ) + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + trace = result.calls[0] + assert trace.outcome == "provider_rejected" + assert trace.provider_status_code == 429 + assert trace.response_id == "chatcmpl-rejected" + assert trace.response_model == "gpt-rejected-response" + assert trace.usage.availability == "available" + assert trace.usage.total_tokens == 9 + assert observer.completed[0].usage == trace.usage + + +def test_runtime_chat_pre_send_adapter_failure_is_not_sent(monkeypatch) -> None: + _clear_runtime_chat_cooldown() + observer = RecordingAttemptObserver() + session_factory = build_session_factory() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-invalid-endpoint", + ) + monkeypatch.setattr( + "app.services.runtime_chat.request_openai_compatible_completion", + lambda **_kwargs: (_ for _ in ()).throw(ValueError("invalid endpoint")), + ) + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + assert result.calls[0].outcome == "not_sent" + assert result.calls[0].provider_status_code is None + assert observer.completed[0].request_may_have_been_sent is False + + +def test_runtime_chat_permit_observer_failure_blocks_provider_and_reconciles( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + observer = FailingPermitObserver() + session_factory = build_session_factory() + with session_factory() as db: + service = RuntimeChatService(db, attempt_observer=observer) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-permit-failure", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: pytest.fail( + "permit 状态未知时不应发送 provider 请求" + ), + ) + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + trace = result.calls[0] + assert result.text is None + assert trace.status == "blocked" + assert trace.outcome == "not_sent" + assert trace.observer_status == "reconciliation_required" + assert trace.requires_reconciliation is True + assert [failure.phase for failure in trace.observer_failures] == ["permit"] + assert len(observer.completed) == 1 + assert observer.completed[0].request_may_have_been_sent is False + + +def test_runtime_chat_trusted_context_without_observer_requires_reconciliation( + monkeypatch, +) -> None: + _clear_runtime_chat_cooldown() + session_factory = build_session_factory() + with session_factory() as db: + service = RuntimeChatService(db) + patch_single_slot( + monkeypatch, + service, + provider="OpenAI Compatible", + model="gpt-unobserved", + ) + monkeypatch.setattr( + "app.services.runtime_chat._send_json_request", + lambda *_args, **_kwargs: ( + 200, + { + "id": "chatcmpl-unobserved", + "choices": [{"message": {"content": "answer"}}], + "usage": { + "prompt_tokens": 3, + "completion_tokens": 1, + "total_tokens": 4, + }, + }, + ), + ) + result = service.complete_with_trace( + [{"role": "user", "content": "hello"}], + slot_priority=("main",), + max_attempts=1, + operation_context=build_operation_context(), + ) + + assert result.text == "answer" + assert result.calls[0].observer_status == "not_configured" + assert result.calls[0].requires_reconciliation is True + + +@pytest.mark.parametrize( + "context_kwargs", + [ + {"tenant_id": ""}, + {"operation_id": ""}, + {"invocation_seq": 0}, + {"invocation_seq": True}, + {"invocation_seq": "1"}, + {"attempt_scope": ""}, + ], +) +def test_runtime_chat_operation_context_rejects_untrusted_empty_identity( + context_kwargs, +) -> None: + values = { + "tenant_id": "tenant-trusted", + "operation_id": "operation-trusted", + "invocation_seq": 1, + "attempt_scope": "trusted-entry", + } + values.update(context_kwargs) + with pytest.raises(ValueError): + RuntimeChatOperationContext(**values) + + +def test_runtime_chat_attempt_key_has_unambiguous_opaque_identity() -> None: + common = { + "run_id": "run", + "invocation_seq": 1, + "attempt_scope": "scope", + } + first = RuntimeChatOperationContext( + tenant_id="tenant|operation", + operation_id="id", + **common, + ).build_attempt_identity( + slot="main", + provider="OpenAI Compatible", + model="gpt-test", + attempt=1, + ) + second = RuntimeChatOperationContext( + tenant_id="tenant", + operation_id="operation|id", + **common, + ).build_attempt_identity( + slot="main", + provider="OpenAI Compatible", + model="gpt-test", + attempt=1, + ) + + assert first.attempt_key != second.attempt_key + assert first.attempt_key.startswith("runtime-chat-attempt:v1:") + assert "tenant" not in first.attempt_key + assert "operation" not in first.attempt_key diff --git a/server/tests/test_runtime_chat_commercial.py b/server/tests/test_runtime_chat_commercial.py new file mode 100644 index 0000000..21efa3e --- /dev/null +++ b/server/tests/test_runtime_chat_commercial.py @@ -0,0 +1,213 @@ +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from decimal import Decimal + +import pytest +from commercial_runtime_testkit import seed_meter +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.db.base_class import Base +from app.models.agent_run import AgentRun +from app.models.commercial import UsageMeterEvent +from app.models.commercial_runtime import CommercialRuntimeReservation +from app.services.commercial_direct_operation import CommercialDirectOperationBridge +from app.services.runtime_chat_attempts import ( + RuntimeChatAttemptCompletedEvent, + RuntimeChatAttemptPermitEvent, + RuntimeChatAuthoritativeUsage, + RuntimeChatOperationContext, +) +from app.services.runtime_chat_commercial import ( + CommercialDirectReconciliationRequired, + CommercialRuntimeChatAttemptObserver, + trusted_runtime_chat_operation_context, +) + + +@pytest.fixture() +def factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + result = sessionmaker(bind=engine, expire_on_commit=False) + try: + yield result + finally: + Base.metadata.drop_all(engine) + engine.dispose() + + +def _identity(*, tenant_id: str = "tenant-a", operation_id: str = "operation-a"): + return RuntimeChatOperationContext( + tenant_id=tenant_id, + operation_id=operation_id, + run_id="run-a", + invocation_seq=3, + attempt_scope="user-agent-response", + ).build_attempt_identity( + slot="main", + provider="Ollama", + model="qwen-test", + attempt=1, + ) + + +def test_runtime_chat_observer_settles_authoritative_provider_tokens( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis="total_tokens", + preflight_quantity=Decimal("50"), + ) + db.commit() + observer = CommercialRuntimeChatAttemptObserver( + CommercialDirectOperationBridge(factory) + ) + identity = _identity() + + permit = observer.on_permit( + RuntimeChatAttemptPermitEvent(identity=identity, started_at=now) + ) + observer.on_completed( + RuntimeChatAttemptCompletedEvent( + identity=identity, + started_at=now, + completed_at=now + timedelta(milliseconds=250), + outcome="succeeded", + response_id=None, + response_model="qwen-test", + provider_status_code=200, + usage=RuntimeChatAuthoritativeUsage( + source="ollama_response", + availability="available", + prompt_eval_count=12, + eval_count=8, + ), + ) + ) + + assert permit.allowed is True + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + usage = db.scalars(select(UsageMeterEvent)).one() + assert reservation.status == "committed" + assert Decimal(reservation.actual_quantity or 0) == Decimal("20") + assert Decimal(usage.quantity) == Decimal("20") + assert usage.metadata_json["usage_source"] == "ollama_response" + + +def test_runtime_chat_observer_persists_reconciliation_when_usage_is_missing( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + seed_meter( + db, + "tenant-a", + now, + basis="total_tokens", + preflight_quantity=Decimal("50"), + ) + db.commit() + observer = CommercialRuntimeChatAttemptObserver( + CommercialDirectOperationBridge(factory) + ) + identity = _identity(operation_id="operation-missing-usage") + assert observer.on_permit( + RuntimeChatAttemptPermitEvent(identity=identity, started_at=now) + ).allowed + + with pytest.raises( + CommercialDirectReconciliationRequired, + match="authoritative_usage_unavailable", + ): + observer.on_completed( + RuntimeChatAttemptCompletedEvent( + identity=identity, + started_at=now, + completed_at=now + timedelta(seconds=1), + outcome="succeeded", + response_id="response-without-usage", + response_model="qwen-test", + provider_status_code=200, + usage=RuntimeChatAuthoritativeUsage(), + ) + ) + + with factory() as db: + reservation = db.scalars(select(CommercialRuntimeReservation)).one() + assert reservation.status == "reconciliation_required" + assert reservation.resolution_code == "authoritative_usage_unavailable" + assert db.query(UsageMeterEvent).count() == 0 + + +def test_operation_context_uses_only_persisted_agent_run_tenant( + factory: sessionmaker[Session], +) -> None: + now = datetime.now(UTC) + with factory() as db: + db.add_all( + [ + AgentRun( + run_id="trusted-run", + agent="user_agent", + source="chat", + user_id="user-a", + route_json={"tenant_id": "tenant-trusted"}, + permission_level="write", + status="running", + started_at=now, + ), + AgentRun( + run_id="unscoped-run", + agent="user_agent", + source="chat", + user_id="user-a", + route_json={"context_json": {"tenant_id": "tenant-spoofed"}}, + permission_level="write", + status="running", + started_at=now, + ), + ] + ) + db.commit() + + context = trusted_runtime_chat_operation_context( + db, + run_id="trusted-run", + attempt_scope="user-agent-response", + invocation_seq=2, + ) + + assert context is not None + assert context.tenant_id == "tenant-trusted" + assert context.operation_id == "agent-run:trusted-run" + assert context.invocation_seq == 2 + assert ( + trusted_runtime_chat_operation_context( + db, + run_id="unscoped-run", + attempt_scope="user-agent-response", + ) + is None + ) + assert ( + trusted_runtime_chat_operation_context( + db, + run_id="missing-run", + attempt_scope="user-agent-response", + ) + is None + ) diff --git a/server/tests/test_runtime_chat_service.py b/server/tests/test_runtime_chat_service.py index 7179e3f..5130798 100644 --- a/server/tests/test_runtime_chat_service.py +++ b/server/tests/test_runtime_chat_service.py @@ -39,7 +39,9 @@ def test_runtime_chat_fails_over_to_backup_before_retrying_main(monkeypatch) -> "apiKey": "secret", } - def fake_request_chat_completion(config, messages, *, max_tokens, temperature, timeout_seconds): + def fake_request_chat_completion( + config, messages, *, max_tokens, temperature, timeout_seconds + ): del messages, max_tokens, temperature, timeout_seconds calls.append(config["slot"]) if config["slot"] == "main": @@ -70,7 +72,9 @@ def test_runtime_chat_complete_with_trace_records_slot_failover(monkeypatch) -> "apiKey": "secret", } - def fake_request_chat_completion(config, messages, *, max_tokens, temperature, timeout_seconds): + def fake_request_chat_completion( + config, messages, *, max_tokens, temperature, timeout_seconds + ): del messages, max_tokens, temperature, timeout_seconds if config["slot"] == "main": raise RuntimeError("incorrect api key") @@ -104,7 +108,9 @@ def test_runtime_chat_does_not_rehit_failed_slots_during_cooldown(monkeypatch) - "apiKey": "secret", } - def fake_request_chat_completion(config, messages, *, max_tokens, temperature, timeout_seconds): + def fake_request_chat_completion( + config, messages, *, max_tokens, temperature, timeout_seconds + ): del messages, max_tokens, temperature, timeout_seconds calls.append(config["slot"]) raise RuntimeError("unavailable") @@ -134,7 +140,7 @@ def test_runtime_chat_disables_glm_thinking_for_direct_user_answers(monkeypatch) monkeypatch.setattr("app.services.runtime_chat._send_json_request", fake_send_json_request) - answer = service._request_openai_compatible( + provider_response = service._request_openai_compatible( provider="GLM", endpoint="https://open.bigmodel.cn/api/paas/v4/", model="glm-5.1", @@ -145,7 +151,7 @@ def test_runtime_chat_disables_glm_thinking_for_direct_user_answers(monkeypatch) timeout_seconds=17, ) - assert answer == "ok" + assert provider_response.output == "ok" assert captured["payload"]["thinking"] == {"type": "disabled"} assert captured["timeout_seconds"] == 17 @@ -184,7 +190,7 @@ def test_runtime_chat_openai_compatible_tool_call_payload(monkeypatch) -> None: monkeypatch.setattr("app.services.runtime_chat._send_json_request", fake_send_json_request) - tool_call = service._request_openai_compatible_tool_call( + provider_response = service._request_openai_compatible_tool_call( provider="OpenAI Compatible", endpoint="https://api.example.com/v1", model="gpt-test", @@ -197,12 +203,16 @@ def test_runtime_chat_openai_compatible_tool_call_payload(monkeypatch) -> None: timeout_seconds=19, ) + tool_call = provider_response.output assert tool_call is not None assert tool_call.name == "submit_steward_intent_plan" assert tool_call.arguments == {"tasks": []} assert captured["url"] == "https://api.example.com/v1/chat/completions" assert captured["payload"]["tools"][0]["function"]["name"] == "submit_steward_intent_plan" - assert captured["payload"]["tool_choice"]["function"]["name"] == "submit_steward_intent_plan" + assert ( + captured["payload"]["tool_choice"]["function"]["name"] + == "submit_steward_intent_plan" + ) assert captured["headers"]["Authorization"] == "Bearer secret" @@ -222,7 +232,9 @@ def test_runtime_chat_supports_single_pass_fast_failover(monkeypatch) -> None: "apiKey": "secret", } - def fake_request_chat_completion(config, messages, *, max_tokens, temperature, timeout_seconds): + def fake_request_chat_completion( + config, messages, *, max_tokens, temperature, timeout_seconds + ): del messages, max_tokens, temperature calls.append((config["slot"], timeout_seconds)) raise RuntimeError("unavailable") @@ -242,7 +254,9 @@ def test_runtime_chat_supports_single_pass_fast_failover(monkeypatch) -> None: assert calls == [("main", 8), ("backup", 20)] -def test_runtime_chat_complete_with_tool_call_fails_over_to_backup_before_retrying_main(monkeypatch) -> None: +def test_runtime_chat_tool_call_fails_over_to_backup_before_retrying_main( + monkeypatch, +) -> None: _clear_runtime_chat_cooldown() session_factory = build_session_factory() with session_factory() as db: @@ -258,7 +272,16 @@ def test_runtime_chat_complete_with_tool_call_fails_over_to_backup_before_retryi "apiKey": "secret", } - def fake_request_chat_tool_call(config, messages, *, tools, tool_choice, max_tokens, temperature, timeout_seconds): + def fake_request_chat_tool_call( + config, + messages, + *, + tools, + tool_choice, + max_tokens, + temperature, + timeout_seconds, + ): del messages, tools, tool_choice, max_tokens, temperature, timeout_seconds calls.append(config["slot"]) if config["slot"] == "main": @@ -302,7 +325,9 @@ def test_runtime_chat_skips_slot_during_cooldown(monkeypatch) -> None: "apiKey": "secret", } - def fake_request_chat_completion(config, messages, *, max_tokens, temperature, timeout_seconds): + def fake_request_chat_completion( + config, messages, *, max_tokens, temperature, timeout_seconds + ): del messages, max_tokens, temperature, timeout_seconds calls.append(config["slot"]) if config["slot"] == "main": @@ -312,8 +337,14 @@ def test_runtime_chat_skips_slot_during_cooldown(monkeypatch) -> None: monkeypatch.setattr(service, "_load_chat_slot", fake_load_chat_slot) monkeypatch.setattr(service, "_request_chat_completion", fake_request_chat_completion) - assert service.complete([{"role": "user", "content": "hello"}], max_attempts=1) == "backup answer" - assert service.complete([{"role": "user", "content": "hello again"}], max_attempts=1) == "backup answer" + first = service.complete( + [{"role": "user", "content": "hello"}], max_attempts=1 + ) + second = service.complete( + [{"role": "user", "content": "hello again"}], max_attempts=1 + ) + assert first == "backup answer" + assert second == "backup answer" assert calls == ["main", "backup", "backup"] diff --git a/server/tests/test_savings_baseline_insights.py b/server/tests/test_savings_baseline_insights.py new file mode 100644 index 0000000..7cf9430 --- /dev/null +++ b/server/tests/test_savings_baseline_insights.py @@ -0,0 +1,750 @@ +from __future__ import annotations + +import uuid +from collections.abc import Generator +from datetime import UTC, date, datetime, timedelta +from decimal import Decimal + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.api.v1.endpoints.savings import router +from app.db.base_class import Base +from app.models.budget import BudgetAllocation, BudgetTransaction +from app.models.expense_case import BusinessEvent, ExpenseCaseLink +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, +) +from app.schemas.savings_insights import ( + SavingsBaselineGenerateRequest, + SavingsInsightAnalyzeRequest, +) +from app.services.expense_cases import ExpenseCaseService +from app.services.savings_access_policy import SavingsPermissionError +from app.services.savings_baseline_generation import SavingsBaselineGenerationService +from app.services.savings_insight_analysis import SavingsInsightAnalysisService + + +@pytest.fixture() +def db() -> Generator[Session, None, None]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_baselines_cover_six_dimensions_replay_and_exclude_other_tenant( + db: Session, +) -> None: + for index in range(5): + _seed_claim( + db, + tenant_id="tenant-a", + suffix=f"A-{index}", + amount=Decimal("50.00"), + occurred_on=date(2026, 6, index + 1), + ) + _seed_claim( + db, + tenant_id="tenant-b", + suffix=f"B-{index}", + amount=Decimal("900.00"), + occurred_on=date(2026, 6, index + 1), + workflow_elapsed_minutes=600, + ) + application = _seed_claim( + db, + tenant_id="tenant-a", + suffix="APP-EXCLUDED", + amount=Decimal("5000.00"), + occurred_on=date(2026, 6, 10), + ) + application.claim_no = f"AP-{uuid.uuid4().hex[:8]}" + application.expense_type = "application" + application.approval_stage = "申请归档" + application.status = "approved" + db.commit() + request = _baseline_request("baseline-request-tenant-a") + user = _user("finance-a", tenant_id="tenant-a", roles=["finance"]) + + first = SavingsBaselineGenerationService(db).generate(request, user) + snapshot_ids = {row.id for row in first.snapshots} + assert first.replayed is False + assert first.source_claim_count == 5 + assert first.source_item_count == 5 + assert first.source_workflow_cycle_count == 5 + assert {row.dimension_type for row in first.snapshots} == { + "employee", + "department", + "expense_type", + "city", + "project", + "workflow", + } + amount_snapshots = [row for row in first.snapshots if row.dimension_type != "workflow"] + workflow_snapshot = next(row for row in first.snapshots if row.dimension_type == "workflow") + assert {row.baseline_value for row in amount_snapshots} == {Decimal("50.0000")} + assert workflow_snapshot.baseline_value == Decimal("60.0000") + assert workflow_snapshot.metric_key == "median_submission_to_payment_elapsed_minutes" + assert workflow_snapshot.unit == "minutes" + assert workflow_snapshot.original_currency is None + assert all(row.sample_count == 5 for row in first.snapshots) + assert all(row.data_quality_status == "complete" for row in first.snapshots) + assert all( + row.method == "median_archived_expense_facts_tenant_scope" for row in amount_snapshots + ) + assert workflow_snapshot.method == "median_completed_workflow_elapsed_tenant_scope" + assert any(issue.code == "supplier_dimension_unavailable" for issue in first.quality_issues) + assert any(issue.code == "workflow_active_labor_unavailable" for issue in first.quality_issues) + assert db.scalar(select(func.count(SavingsEvidenceLink.id))) == 6 + assert db.scalar(select(func.count(SavingsEvent.id))) == 6 + + replay = SavingsBaselineGenerationService(db).generate(request, user) + assert replay.replayed is True + assert {row.id for row in replay.snapshots} == snapshot_ids + assert db.scalar(select(func.count(ProfileBaselineSnapshot.id))) == 6 + assert db.scalar(select(func.count(SavingsEvent.id))) == 6 + + stricter = SavingsBaselineGenerationService(db).generate( + request.model_copy( + update={ + "request_id": "baseline-request-stricter-threshold", + "minimum_complete_samples": 10, + } + ), + user, + ) + assert {row.id for row in stricter.snapshots}.isdisjoint(snapshot_ids) + assert all(row.data_quality_status == "partial" for row in stricter.snapshots) + + +def test_scoped_baseline_only_uses_authorized_department(db: Session) -> None: + _seed_claim( + db, + tenant_id="tenant-a", + suffix="SCOPE-A", + amount=Decimal("80.00"), + occurred_on=date(2026, 6, 1), + department_name="研发部", + ) + _seed_claim( + db, + tenant_id="tenant-a", + suffix="SCOPE-B", + amount=Decimal("800.00"), + occurred_on=date(2026, 6, 2), + department_name="销售部", + ) + db.commit() + + result = SavingsBaselineGenerationService(db).generate( + _baseline_request( + "baseline-scoped-request", + dimensions=["department", "expense_type"], + ), + _user( + "budget-user", + tenant_id="tenant-a", + roles=["budget_monitor"], + department_name="研发部", + ), + ) + + assert result.data_scope == "department" + assert result.source_claim_count == 1 + assert {row.baseline_value for row in result.snapshots} == {Decimal("80.0000")} + assert all( + row.method == "median_archived_expense_facts_department_scope" for row in result.snapshots + ) + assert all(row.data_quality_status == "insufficient" for row in result.snapshots) + assert all( + any(issue["code"] == "baseline_sample_insufficient" for issue in row.quality_issues_json) + for row in result.snapshots + ) + assert db.scalar(select(func.count(SavingsOpportunity.id))) == 0 + assert {row.dimension_id for row in result.snapshots if row.dimension_type == "department"} == { + "name:研发部" + } + + +def test_workflow_baseline_is_department_scoped_and_excludes_future_completion( + db: Session, +) -> None: + _seed_claim( + db, + tenant_id="tenant-a", + suffix="WORKFLOW-SCOPE-A", + amount=Decimal("80.00"), + occurred_on=date(2026, 6, 1), + department_name="研发部", + workflow_elapsed_minutes=90, + ) + _seed_claim( + db, + tenant_id="tenant-a", + suffix="WORKFLOW-SCOPE-B", + amount=Decimal("800.00"), + occurred_on=date(2026, 6, 2), + department_name="销售部", + workflow_elapsed_minutes=900, + ) + _seed_claim( + db, + tenant_id="tenant-a", + suffix="WORKFLOW-FUTURE", + amount=Decimal("100.00"), + occurred_on=date(2026, 6, 3), + department_name="研发部", + payment_completed_at=datetime(2026, 7, 2, tzinfo=UTC), + ) + db.commit() + + result = SavingsBaselineGenerationService(db).generate( + _baseline_request( + "workflow-scoped-request", + dimensions=["workflow"], + ), + _user( + "budget-user", + tenant_id="tenant-a", + roles=["budget_monitor"], + department_name="研发部", + ), + ) + + assert result.data_scope == "department" + assert result.source_claim_count == 2 + assert result.source_workflow_cycle_count == 1 + assert len(result.snapshots) == 1 + assert result.snapshots[0].baseline_value == Decimal("90.0000") + assert result.snapshots[0].sample_count == 1 + assert result.snapshots[0].data_quality_status == "insufficient" + assert all( + evidence.metadata_json["metric_semantics"] == "elapsed_cycle_not_active_labor" + for evidence in db.scalars(select(SavingsEvidenceLink)).all() + ) + + +def test_workflow_completion_event_must_belong_to_claim_case(db: Session) -> None: + mismatched_claim = _seed_claim( + db, + tenant_id="tenant-a", + suffix="WORKFLOW-WRONG-CASE", + amount=Decimal("80.00"), + occurred_on=date(2026, 6, 1), + workflow_elapsed_minutes=30, + ) + valid_claim = _seed_claim( + db, + tenant_id="tenant-a", + suffix="WORKFLOW-VALID-CASE", + amount=Decimal("90.00"), + occurred_on=date(2026, 6, 2), + workflow_elapsed_minutes=120, + ) + valid_case_id = db.scalar( + select(ExpenseCaseLink.expense_case_id).where( + ExpenseCaseLink.resource_type == "expense_claim", + ExpenseCaseLink.resource_id == valid_claim.id, + ) + ) + mismatched_event = db.scalar( + select(BusinessEvent).where(BusinessEvent.aggregate_id == mismatched_claim.id) + ) + assert valid_case_id and mismatched_event is not None + mismatched_event.expense_case_id = valid_case_id + db.commit() + + result = SavingsBaselineGenerationService(db).generate( + _baseline_request("workflow-case-binding", dimensions=["workflow"]), + _user("finance-a", tenant_id="tenant-a", roles=["finance"]), + ) + + assert result.source_workflow_cycle_count == 1 + assert result.snapshots[0].baseline_value == Decimal("120.0000") + + +def test_analysis_returns_evidence_attribution_and_policy_candidates_without_monetizing( + db: Session, +) -> None: + for index in range(5): + _seed_claim( + db, + tenant_id="default", + suffix=f"BASE-{index}", + amount=Decimal("50.00"), + occurred_on=date(2026, 6, index + 1), + ) + db.commit() + user = _user("finance", roles=["finance"]) + baseline_result = SavingsBaselineGenerationService(db).generate( + _baseline_request("baseline-for-insight"), + user, + ) + assert baseline_result.snapshots + + for index in range(3): + _seed_claim( + db, + tenant_id="default", + suffix=f"OBS-{index}", + amount=Decimal("100.00"), + occurred_on=date(2026, 7, index + 2), + ) + _seed_overrun_budget(db) + db.commit() + before_count = db.scalar(select(func.count(SavingsOpportunity.id))) + + result = SavingsInsightAnalysisService(db).analyze( + SavingsInsightAnalyzeRequest( + request_id="insight-analysis-request", + window_start=datetime(2026, 7, 1, tzinfo=UTC), + window_end=datetime(2026, 7, 31, 23, 59, tzinfo=UTC), + as_of=datetime(2026, 8, 1, tzinfo=UTC), + small_amount_threshold=Decimal("200.00"), + minimum_repeat_count=3, + price_deviation_ratio=Decimal("1.2500"), + ), + user, + ) + + insight_types = {candidate.insight_type for candidate in result.candidates} + assert insight_types == { + "budget_forecast_variance", + "repeated_small_expense_pattern", + "historical_price_deviation", + "anomaly_driver_attribution", + "policy_simulation_candidate", + } + assert result.source_claim_count == 3 + assert all(candidate.evidence_sufficient_for_signal for candidate in result.candidates) + assert all(candidate.evidence for candidate in result.candidates) + assert all(candidate.estimated_savings is None for candidate in result.candidates) + assert all( + candidate.monetization_status == "withheld_no_counterfactual" + for candidate in result.candidates + ) + budget_candidate = next( + candidate + for candidate in result.candidates + if candidate.insight_type == "budget_forecast_variance" + ) + assert budget_candidate.currency is None + assert any( + issue.code == "budget_currency_unavailable" for issue in budget_candidate.quality_issues + ) + attribution = next( + candidate + for candidate in result.candidates + if candidate.insight_type == "anomaly_driver_attribution" + ) + assert attribution.dimension_json["attribution_kind"] == ( + "descriptive_concentration_not_causal" + ) + assert any( + issue.code == "descriptive_attribution_not_causal" for issue in attribution.quality_issues + ) + policy_candidate = next( + candidate + for candidate in result.candidates + if candidate.insight_type == "policy_simulation_candidate" + ) + assert policy_candidate.dimension_json["simulation_action"] == ( + "run_versioned_policy_counterfactual" + ) + assert policy_candidate.dimension_json["write_mode"] == ("read_only_no_opportunity_creation") + assert result.created_opportunity_ids == [] + assert result.monetized_opportunity_count == 0 + assert db.scalar(select(func.count(SavingsOpportunity.id))) == before_count + assert any(issue.code == "supplier_price_drift_unavailable" for issue in result.quality_issues) + + replay = SavingsInsightAnalysisService(db).analyze( + SavingsInsightAnalyzeRequest( + request_id="insight-analysis-request", + window_start=datetime(2026, 7, 1, tzinfo=UTC), + window_end=datetime(2026, 7, 31, 23, 59, tzinfo=UTC), + as_of=datetime(2026, 8, 1, tzinfo=UTC), + small_amount_threshold=Decimal("200.00"), + minimum_repeat_count=3, + price_deviation_ratio=Decimal("1.2500"), + ), + user, + ) + assert replay.request_fingerprint == result.request_fingerprint + assert [candidate.candidate_key for candidate in replay.candidates] == [ + candidate.candidate_key for candidate in result.candidates + ] + assert db.scalar(select(func.count(SavingsOpportunity.id))) == before_count + + +def test_budget_forecast_excludes_transactions_after_window_cutoff( + db: Session, +) -> None: + _seed_claim( + db, + tenant_id="default", + suffix="BUDGET-CUTOFF", + amount=Decimal("100.00"), + occurred_on=date(2026, 7, 3), + ) + _seed_overrun_budget(db) + transactions = list( + db.scalars(select(BudgetTransaction).order_by(BudgetTransaction.created_at)).all() + ) + transactions[-1].created_at = datetime(2026, 8, 1, tzinfo=UTC) + db.commit() + + result = SavingsInsightAnalysisService(db).analyze( + SavingsInsightAnalyzeRequest( + request_id="budget-window-cutoff-request", + window_start=datetime(2026, 7, 1, tzinfo=UTC), + window_end=datetime(2026, 7, 31, 23, 59, tzinfo=UTC), + as_of=datetime(2026, 8, 2, tzinfo=UTC), + ), + _user("finance", roles=["finance"]), + ) + + assert not any( + candidate.insight_type == "budget_forecast_variance" for candidate in result.candidates + ) + assert any( + issue.code == "budget_forecast_sample_insufficient" for issue in result.quality_issues + ) + + +def test_budget_forecast_excludes_allocation_modified_after_as_of( + db: Session, +) -> None: + _seed_overrun_budget(db) + allocation = db.scalar(select(BudgetAllocation)) + assert allocation is not None + allocation.updated_at = datetime(2026, 8, 2, tzinfo=UTC) + db.commit() + + result = SavingsInsightAnalysisService(db).analyze( + SavingsInsightAnalyzeRequest( + request_id="budget-allocation-as-of-request", + window_start=datetime(2026, 7, 1, tzinfo=UTC), + window_end=datetime(2026, 7, 31, 23, 59, tzinfo=UTC), + as_of=datetime(2026, 8, 1, tzinfo=UTC), + ), + _user("finance", roles=["finance"]), + ) + + assert not any( + candidate.insight_type == "budget_forecast_variance" for candidate in result.candidates + ) + assert any(issue.code == "budget_allocation_unavailable" for issue in result.quality_issues) + + +def test_non_default_tenant_never_reads_legacy_budget(db: Session) -> None: + _seed_claim( + db, + tenant_id="tenant-a", + suffix="NONDEFAULT", + amount=Decimal("100.00"), + occurred_on=date(2026, 7, 3), + ) + _seed_overrun_budget(db) + db.commit() + + result = SavingsInsightAnalysisService(db).analyze( + SavingsInsightAnalyzeRequest( + request_id="tenant-budget-boundary", + window_start=datetime(2026, 7, 1, tzinfo=UTC), + window_end=datetime(2026, 7, 31, 23, 59, tzinfo=UTC), + as_of=datetime(2026, 8, 1, tzinfo=UTC), + ), + _user("finance-a", tenant_id="tenant-a", roles=["finance"]), + ) + + assert not any( + candidate.insight_type == "budget_forecast_variance" for candidate in result.candidates + ) + assert any(issue.code == "tenant_budget_scope_unavailable" for issue in result.quality_issues) + + +def test_analysis_does_not_time_travel_into_baseline_frozen_after_as_of( + db: Session, +) -> None: + for index in range(5): + _seed_claim( + db, + tenant_id="tenant-a", + suffix=f"TEMPORAL-BASE-{index}", + amount=Decimal("50.00"), + occurred_on=date(2026, 6, index + 1), + ) + db.commit() + user = _user("finance-a", tenant_id="tenant-a", roles=["finance"]) + generated = SavingsBaselineGenerationService(db).generate( + _baseline_request("temporal-baseline-request"), + user, + ) + assert generated.snapshots + for snapshot in db.scalars(select(ProfileBaselineSnapshot)).all(): + snapshot.frozen_at = datetime(2026, 8, 2, tzinfo=UTC) + _seed_claim( + db, + tenant_id="tenant-a", + suffix="TEMPORAL-OBSERVED", + amount=Decimal("100.00"), + occurred_on=date(2026, 7, 2), + ) + db.commit() + + result = SavingsInsightAnalysisService(db).analyze( + SavingsInsightAnalyzeRequest( + request_id="temporal-analysis-request", + window_start=datetime(2026, 7, 1, tzinfo=UTC), + window_end=datetime(2026, 7, 31, 23, 59, tzinfo=UTC), + as_of=datetime(2026, 8, 1, tzinfo=UTC), + ), + user, + ) + + assert not any( + candidate.insight_type == "historical_price_deviation" for candidate in result.candidates + ) + assert any(issue.code == "historical_baseline_unavailable" for issue in result.quality_issues) + + +def test_baseline_endpoint_enforces_savings_access_policy(db: Session) -> None: + with pytest.raises(SavingsPermissionError): + SavingsBaselineGenerationService(db).generate( + _baseline_request("ordinary-user-request"), + _user("ordinary"), + ) + + +def test_baseline_and_insight_http_contracts(db: Session) -> None: + _seed_claim( + db, + tenant_id="default", + suffix="HTTP", + amount=Decimal("60.00"), + occurred_on=date(2026, 6, 3), + ) + db.commit() + app = FastAPI() + app.include_router(router, prefix="/api/v1") + user_box = {"current": _user("finance", roles=["finance"])} + + def override_db() -> Generator[Session, None, None]: + yield db + + app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: user_box["current"] + with TestClient(app) as client: + baseline_response = client.post( + "/api/v1/savings/baselines/generate", + json=_baseline_request("http-baseline-request").model_dump(mode="json"), + ) + assert baseline_response.status_code == 200 + assert baseline_response.json()["source_claim_count"] == 1 + assert baseline_response.json()["snapshots"] + + insight_response = client.post( + "/api/v1/savings/insights/analyze", + json={ + "request_id": "http-insight-request", + "window_start": "2026-06-01T00:00:00Z", + "window_end": "2026-06-30T23:59:00Z", + "as_of": "2026-07-01T00:00:00Z", + }, + ) + assert insight_response.status_code == 200 + assert insight_response.json()["monetized_opportunity_count"] == 0 + + user_box["current"] = _user("ordinary") + forbidden = client.post( + "/api/v1/savings/baselines/generate", + json=_baseline_request("http-forbidden-request").model_dump(mode="json"), + ) + assert forbidden.status_code == 403 + + +def _seed_claim( + db: Session, + *, + tenant_id: str, + suffix: str, + amount: Decimal, + occurred_on: date, + department_name: str = "研发部", + workflow_elapsed_minutes: int = 60, + payment_completed_at: datetime | None = None, +) -> ExpenseClaim: + occurred_at = datetime.combine(occurred_on, datetime.min.time(), tzinfo=UTC) + claim = ExpenseClaim( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + claim_no=f"BX-{suffix}-{uuid.uuid4().hex[:6]}", + employee_name="张三", + department_name=department_name, + project_code="PROJECT-A", + expense_type="taxi", + reason="客户现场交通", + location="上海", + amount=amount, + currency="CNY", + invoice_count=1, + occurred_at=occurred_at, + submitted_at=occurred_at, + status="paid", + approval_stage="已付款", + risk_flags_json=[], + created_at=occurred_at, + updated_at=occurred_at, + ) + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim=claim, + item_date=occurred_on, + item_type="taxi", + item_reason="客户现场交通", + item_location="上海", + item_note="", + item_amount=amount, + created_at=occurred_at, + updated_at=occurred_at, + ) + db.add(claim) + db.flush() + expense_case = ExpenseCaseService(db).ensure_case_for_claim( + claim, + tenant_id=tenant_id, + ) + completion_time = payment_completed_at or ( + occurred_at + timedelta(minutes=workflow_elapsed_minutes) + ) + expense_case.created_at = occurred_at + expense_case.updated_at = completion_time + event_id = str(uuid.uuid4()) + db.add( + BusinessEvent( + id=event_id, + tenant_id=tenant_id, + expense_case_id=expense_case.id, + aggregate_type="expense_claim", + aggregate_id=claim.id, + event_type="payment_completed", + event_version=1, + idempotency_key=f"payment:{event_id}", + correlation_id=event_id, + causation_id=None, + actor_id="finance", + actor_type="user", + payload_json={"source": "test_business_fact"}, + delivery_status="published", + delivery_attempts=0, + occurred_at=completion_time, + published_at=completion_time, + ) + ) + assert item.claim_id == claim.id + return claim + + +def _seed_overrun_budget(db: Session) -> None: + allocation = BudgetAllocation( + id=str(uuid.uuid4()), + budget_no=f"BUD-{uuid.uuid4().hex[:8]}", + fiscal_year=2026, + period_type="quarter", + period_key="2026Q3", + department_name="研发部", + cost_center="CC-100", + project_code="PROJECT-A", + subject_code="travel", + subject_name="差旅费", + original_amount=Decimal("500.00"), + adjusted_amount=Decimal("0.00"), + status="active", + warning_threshold=Decimal("80.00"), + control_action="warn", + created_at=datetime(2026, 7, 1, tzinfo=UTC), + updated_at=datetime(2026, 7, 1, tzinfo=UTC), + ) + db.add(allocation) + db.flush() + for index, created_at in enumerate( + (datetime(2026, 7, 10, tzinfo=UTC), datetime(2026, 7, 20, tzinfo=UTC)) + ): + db.add( + BudgetTransaction( + id=str(uuid.uuid4()), + transaction_no=f"BTX-{uuid.uuid4().hex[:8]}", + allocation_id=allocation.id, + source_type="claim", + source_id=f"budget-source-{index}", + source_no=f"BX-BUDGET-{index}", + transaction_type="consume", + amount=Decimal("150.00"), + before_available_amount=Decimal("500.00") - Decimal("150.00") * index, + after_available_amount=Decimal("350.00") - Decimal("150.00") * index, + operator="finance", + reason="已付款单据核销", + context_json={}, + created_at=created_at, + ) + ) + + +def _baseline_request( + request_id: str, + *, + dimensions: list[str] | None = None, +) -> SavingsBaselineGenerateRequest: + return SavingsBaselineGenerateRequest( + request_id=request_id, + window_start=datetime(2026, 6, 1, tzinfo=UTC), + window_end=datetime(2026, 6, 30, 23, 59, tzinfo=UTC), + as_of=datetime(2026, 7, 1, tzinfo=UTC), + dimensions=dimensions + or [ + "employee", + "department", + "expense_type", + "city", + "project", + "workflow", + "supplier", + ], + minimum_complete_samples=5, + ) + + +def _user( + username: str, + *, + tenant_id: str = "default", + roles: list[str] | None = None, + department_name: str = "", +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=False, + tenant_id=tenant_id, + employee_id=username, + department_name=department_name, + ) diff --git a/server/tests/test_savings_concurrency_postgres.py b/server/tests/test_savings_concurrency_postgres.py new file mode 100644 index 0000000..64940dc --- /dev/null +++ b/server/tests/test_savings_concurrency_postgres.py @@ -0,0 +1,406 @@ +from __future__ import annotations + +import threading +import uuid +from concurrent.futures import ThreadPoolExecutor + +import pytest +from savings_postgres_testkit import ( + _pg_factory_fixture, # noqa: F401 - 注册名为 pg_factory 的 pytest fixture + _seed_opportunity, + _seed_payment_case, + _seed_pending_realization, + _SeededRealization, + _user, +) +from sqlalchemy import func, select, text +from sqlalchemy.exc import DBAPIError +from sqlalchemy.orm import Session, sessionmaker + +from app.models.expense_case import BusinessEvent +from app.models.financial_record import ExpenseClaim +from app.models.savings import ( + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) +from app.schemas.savings import ( + SavingsOpportunityActionCreate, + SavingsRealizationActionCreate, +) +from app.services.savings_access_policy import SavingsPermissionError +from app.services.savings_actions import SavingsActionService +from app.services.savings_realization import ( + SavingsRealizationError, + SavingsRealizationService, +) + + +def test_concurrent_identical_request_replays_one_immutable_event( + pg_factory: sessionmaker[Session], +) -> None: + seed = _seed_opportunity(pg_factory, status="identified", owner_id="owner-idempotent") + actor = _user("owner-idempotent", seed.tenant_id, employee_id="owner-idempotent") + payload = SavingsOpportunityActionCreate( + action="accept", + request_id=f"accept-concurrent-{seed.suffix}", + expected_version=1, + comment="并发接受同一个节省机会", + ) + ready = threading.Barrier(2) + + def accept_once(): + with pg_factory() as db: + ready.wait(timeout=5) + return SavingsActionService(db).execute(seed.opportunity_id, payload, actor).response + + with ThreadPoolExecutor(max_workers=2) as pool: + results = [ + future.result(timeout=10) + for future in (pool.submit(accept_once), pool.submit(accept_once)) + ] + + assert sorted(item.replayed for item in results) == [False, True] + assert results[0].opportunity.model_dump(mode="json") == results[1].opportunity.model_dump( + mode="json" + ) + with pg_factory() as db: + opportunity = db.get(SavingsOpportunity, seed.opportunity_id) + assert opportunity is not None and opportunity.status == "accepted" + assert opportunity.version == 2 + assert ( + db.scalar( + select(func.count()) + .select_from(SavingsEvent) + .where( + SavingsEvent.tenant_id == seed.tenant_id, + SavingsEvent.actor_id == "owner-idempotent", + SavingsEvent.request_id == payload.request_id, + ) + ) + == 1 + ) + + +def test_same_benefit_concurrent_confirmation_has_one_canonical_winner( + pg_factory: sessionmaker[Session], + monkeypatch: pytest.MonkeyPatch, +) -> None: + benefit_key = f"benefit-shared-{uuid.uuid4().hex}" + first = _seed_pending_realization( + pg_factory, + benefit_key=benefit_key, + owner_id="owner-canonical-a", + recorder_id="payer-canonical-a", + ) + second = _seed_pending_realization( + pg_factory, + benefit_key=benefit_key, + owner_id="owner-canonical-b", + recorder_id="payer-canonical-b", + ) + ready = threading.Barrier(2) + original_confirm = SavingsRealizationService._confirm + + def confirm_after_duplicate_probe(self, *args, **kwargs): + original_confirm(self, *args, **kwargs) + ready.wait(timeout=5) + + monkeypatch.setattr(SavingsRealizationService, "_confirm", confirm_after_duplicate_probe) + + def confirm_once(seed: _SeededRealization, actor_id: str): + payload = SavingsRealizationActionCreate( + action="confirm", + request_id=f"confirm-race-{seed.suffix}", + expected_version=1, + comment="并发确认相同经济收益", + ) + with pg_factory() as db: + try: + return ( + SavingsRealizationService(db) + .execute_action( + seed.realization_id, + payload, + _user(actor_id, seed.tenant_id, employee_id=actor_id, roles=["finance"]), + ) + .response.realization.id + ) + except SavingsRealizationError as error: + return error + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in ( + pool.submit(confirm_once, first, "finance-canonical-a"), + pool.submit(confirm_once, second, "finance-canonical-b"), + ) + ] + + assert sum(isinstance(item, str) for item in outcomes) == 1 + assert sum(isinstance(item, SavingsRealizationError) for item in outcomes) == 1 + with pg_factory() as db: + realizations = list( + db.scalars( + select(SavingsRealization).where( + SavingsRealization.tenant_id == first.tenant_id, + SavingsRealization.benefit_key == benefit_key, + ) + ) + ) + assert sorted((row.status, row.dedupe_status) for row in realizations) == [ + ("finance_confirmed", "canonical"), + ("pending_confirmation", "pending_review"), + ] + winner = next(row for row in realizations if row.status == "finance_confirmed") + loser = next(row for row in realizations if row.status == "pending_confirmation") + loser_evidence = list( + db.scalars( + select(SavingsEvidenceLink).where( + SavingsEvidenceLink.realization_id == loser.id, + ) + ) + ) + assert loser_evidence and {item.verification_status for item in loser_evidence} == { + "unverified" + } + assert ( + db.scalar( + select(func.count()) + .select_from(SavingsEvent) + .where( + SavingsEvent.realization_id.in_([first.realization_id, second.realization_id]), + SavingsEvent.action == "confirm", + ) + ) + == 1 + ) + assert ( + db.scalar( + select(func.count()) + .select_from(BusinessEvent) + .where( + BusinessEvent.aggregate_type == "savings_realization", + BusinessEvent.aggregate_id == winner.id, + BusinessEvent.event_type == "saving_confirmed", + ) + ) + == 1 + ) + assert ( + db.scalar( + select(func.count()) + .select_from(SavingsRealization) + .where( + SavingsRealization.tenant_id == first.tenant_id, + SavingsRealization.benefit_key == benefit_key, + SavingsRealization.realization_type == "actual", + SavingsRealization.dedupe_status == "canonical", + SavingsRealization.status == "finance_confirmed", + ) + ) + == 1 + ) + + +def test_cross_tenant_action_is_rejected_without_side_effect( + pg_factory: sessionmaker[Session], +) -> None: + seed = _seed_opportunity( + pg_factory, tenant_id="tenant-savings-private", owner_id="owner-private" + ) + realization_seed = _seed_pending_realization( + pg_factory, + tenant_id="tenant-savings-private", + owner_id="owner-private-realization", + recorder_id="payer-private-realization", + ) + request_id = f"cross-tenant-{seed.suffix}" + realization_request_id = f"cross-tenant-confirm-{realization_seed.suffix}" + with pg_factory() as db: + with pytest.raises(LookupError, match="不存在"): + SavingsActionService(db).execute( + seed.opportunity_id, + SavingsOpportunityActionCreate( + action="accept", + request_id=request_id, + expected_version=1, + comment="尝试跨租户改变机会", + ), + _user("finance-outsider", "tenant-savings-outsider", roles=["finance"]), + ) + with pytest.raises(LookupError, match="不存在"): + SavingsRealizationService(db).execute_action( + realization_seed.realization_id, + SavingsRealizationActionCreate( + action="confirm", + request_id=realization_request_id, + expected_version=1, + comment="尝试跨租户确认实际节省", + ), + _user("finance-outsider", "tenant-savings-outsider", roles=["finance"]), + ) + + with pg_factory() as db: + opportunity = db.get(SavingsOpportunity, seed.opportunity_id) + assert opportunity is not None and opportunity.status == "identified" + assert opportunity.version == 1 + realization = db.get(SavingsRealization, realization_seed.realization_id) + assert realization is not None and realization.status == "pending_confirmation" + assert realization.version == 1 + assert ( + db.scalar( + select(func.count()) + .select_from(SavingsEvent) + .where(SavingsEvent.request_id.in_([request_id, realization_request_id])) + ) + == 0 + ) + + +def test_confirmation_requires_actor_independent_from_owner_and_recorder( + pg_factory: sessionmaker[Session], +) -> None: + seed = _seed_pending_realization( + pg_factory, + owner_id="finance-owner-independent", + recorder_id="finance-recorder-independent", + ) + + for actor_id in ("finance-owner-independent", "finance-recorder-independent"): + with pg_factory() as db: + with pytest.raises(SavingsPermissionError, match="不能确认自己"): + SavingsRealizationService(db).execute_action( + seed.realization_id, + SavingsRealizationActionCreate( + action="confirm", + request_id=f"self-confirm-{actor_id}-{seed.suffix}", + expected_version=1, + comment="不应允许自我确认", + ), + _user(actor_id, seed.tenant_id, employee_id=actor_id, roles=["finance"]), + ) + + confirmer_id = "finance-independent-reviewer" + with pg_factory() as db: + result = SavingsRealizationService(db).execute_action( + seed.realization_id, + SavingsRealizationActionCreate( + action="confirm", + request_id=f"independent-confirm-{seed.suffix}", + expected_version=1, + comment="独立复核付款与政策证据", + ), + _user(confirmer_id, seed.tenant_id, employee_id=confirmer_id, roles=["finance"]), + ) + assert result.response.realization.status == "finance_confirmed" + assert result.response.realization.finance_confirmer_id == confirmer_id + assert result.response.realization.recorded_by_id != confirmer_id + evidence_statuses = set( + db.scalars( + select(SavingsEvidenceLink.verification_status).where( + SavingsEvidenceLink.realization_id == seed.realization_id, + ) + ) + ) + assert evidence_statuses == {"verified"} + + +def test_concurrent_same_payment_event_persists_one_realization( + pg_factory: sessionmaker[Session], +) -> None: + seed = _seed_payment_case(pg_factory) + ready = threading.Barrier(2) + + def realize_once(): + with pg_factory() as db: + claim = db.get(ExpenseClaim, seed.claim_id) + event = db.get(BusinessEvent, seed.payment_event_id) + assert claim is not None and event is not None + ready.wait(timeout=5) + created = SavingsRealizationService(db).realize_paid_claim( + claim, + event, + _user( + "payment-actor", seed.tenant_id, employee_id="payment-actor", roles=["finance"] + ), + ) + db.commit() + return len(created) + + with ThreadPoolExecutor(max_workers=2) as pool: + outcomes = [ + future.result(timeout=10) + for future in (pool.submit(realize_once), pool.submit(realize_once)) + ] + + assert sorted(outcomes) == [0, 1] + with pg_factory() as db: + rows = list( + db.scalars( + select(SavingsRealization).where( + SavingsRealization.tenant_id == seed.tenant_id, + SavingsRealization.opportunity_id == seed.opportunity_id, + SavingsRealization.business_event_id == seed.payment_event_id, + ) + ) + ) + assert len(rows) == 1 + assert rows[0].realization_key == (f"payment:{seed.payment_event_id}:{seed.opportunity_id}") + opportunity = db.get(SavingsOpportunity, seed.opportunity_id) + assert opportunity is not None and opportunity.status == "realized" + assert opportunity.version == 2 + assert ( + db.scalar( + select(func.count()) + .select_from(BusinessEvent) + .where( + BusinessEvent.tenant_id == seed.tenant_id, + BusinessEvent.aggregate_type == "savings_realization", + BusinessEvent.aggregate_id == rows[0].id, + BusinessEvent.event_type == "saving_action_completed", + BusinessEvent.causation_id == seed.payment_event_id, + ) + ) + == 1 + ) + + +def test_savings_event_is_append_only_at_database_boundary( + pg_factory: sessionmaker[Session], +) -> None: + seed = _seed_opportunity(pg_factory, owner_id="owner-append-only") + request_id = f"append-only-{seed.suffix}" + with pg_factory() as db: + response = ( + SavingsActionService(db) + .execute( + seed.opportunity_id, + SavingsOpportunityActionCreate( + action="accept", + request_id=request_id, + expected_version=1, + comment="创建不可变审计事件", + ), + _user("owner-append-only", seed.tenant_id, employee_id="owner-append-only"), + ) + .response + ) + event_id = response.event.id + + for statement in ( + "UPDATE savings_events SET action = 'tampered' WHERE id = :event_id", + "DELETE FROM savings_events WHERE id = :event_id", + ): + with pg_factory() as db: + with pytest.raises(DBAPIError, match="append-only"): + db.execute(text(statement), {"event_id": event_id}) + db.commit() + db.rollback() + + with pg_factory() as db: + event = db.get(SavingsEvent, event_id) + assert event is not None and event.action == "accept" + assert event.response_json["event"]["id"] == event_id diff --git a/server/tests/test_savings_endpoints.py b/server/tests/test_savings_endpoints.py new file mode 100644 index 0000000..2bb18f5 --- /dev/null +++ b/server/tests/test_savings_endpoints.py @@ -0,0 +1,194 @@ +from __future__ import annotations + +import uuid +from collections.abc import Generator +from datetime import UTC, date, datetime +from decimal import Decimal + +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient +from sqlalchemy import create_engine +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.api.v1.endpoints.cfo_value import router as cfo_value_router +from app.api.v1.endpoints.savings import router +from app.db.base_class import Base +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.services.savings_discovery import SavingsDiscoveryService + + +@pytest.fixture() +def http_context() -> Generator[ + tuple[TestClient, sessionmaker[Session], dict[str, CurrentUserContext]], + None, + None, +]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + app = FastAPI() + app.include_router(router, prefix="/api/v1") + app.include_router(cfo_value_router, prefix="/api/v1") + user_box = {"current": _user("finance-a", roles=["finance"])} + + def override_db() -> Generator[Session, None, None]: + with factory() as db: + yield db + + app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: user_box["current"] + client = TestClient(app) + try: + yield client, factory, user_box + finally: + client.close() + app.dependency_overrides.clear() + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_savings_http_list_detail_record_and_tenant_boundary( + http_context: tuple[ + TestClient, + sessionmaker[Session], + dict[str, CurrentUserContext], + ], +) -> None: + client, factory, user_box = http_context + with factory() as db: + opportunity_id = _seed_discovered_opportunity(db, user_box["current"]) + db.commit() + + list_response = client.get("/api/v1/savings/opportunities") + assert list_response.status_code == 200 + assert list_response.json()["total"] == 1 + assert list_response.json()["items"][0]["id"] == opportunity_id + + detail_response = client.get(f"/api/v1/savings/opportunities/{opportunity_id}") + assert detail_response.status_code == 200 + assert detail_response.json()["baseline"]["policy_version"] == "endpoint-policy-v1" + assert len(detail_response.json()["evidence"]) == 1 + + record_response = client.post( + f"/api/v1/savings/opportunities/{opportunity_id}/realizations", + json={ + "request_id": "endpoint-record-001", + "expected_version": 1, + "comment": "付款状态完成，登记待确认结果", + "actual_gross": "200.00", + "incremental_cost": "0.00", + "currency": "CNY", + "realized_at": datetime.now(UTC).isoformat(), + "attribution_method": "server_policy_counterfactual", + "attribution_ratio": "1.0", + "evidence_level": "business_state", + "evidence": [ + { + "evidence_key": "endpoint-payment-state-001", + "evidence_role": "payment_business_state", + "resource_type": "business_event", + "resource_id": "payment-event-endpoint-001", + "source_system": "x-financial", + "external_event_id": "payment-event-endpoint-001", + "content_hash": "a" * 64, + "occurred_at": datetime.now(UTC).isoformat(), + "verification_status": "unverified", + "metadata_json": {"source": "endpoint-test"}, + } + ], + }, + ) + assert record_response.status_code == 200 + assert record_response.json()["realization"]["status"] == "pending_confirmation" + assert record_response.json()["opportunity"]["status"] == "realized" + + cfo_response = client.get("/api/v1/analytics/cfo-value") + assert cfo_response.status_code == 200 + assert cfo_response.json()["kpis"]["verified_cash"]["status"] == "empty" + assert cfo_response.json()["data_quality"]["pending_confirmation_count"] == 1 + + user_box["current"] = _user("ordinary-user") + assert client.get("/api/v1/analytics/cfo-value").status_code == 403 + + user_box["current"] = _user("finance-other", tenant_id="tenant-b", roles=["finance"]) + cross_tenant = client.get(f"/api/v1/savings/opportunities/{opportunity_id}") + assert cross_tenant.status_code == 404 + assert client.get("/api/v1/savings/opportunities").json()["total"] == 0 + + +def _seed_discovered_opportunity( + db: Session, + current_user: CurrentUserContext, +) -> str: + claim = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no=f"BX-{uuid.uuid4().hex[:10]}", + employee_name="测试员工", + department_name="销售部", + project_code="PROJECT-1", + expense_type="hotel", + reason="客户现场差旅", + location="深圳", + amount=Decimal("1000.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime.now(UTC), + status="draft", + risk_flags_json=[], + ) + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim=claim, + item_date=date(2026, 7, 15), + item_type="hotel", + item_reason="深圳住宿", + item_location="深圳", + item_note="", + item_amount=Decimal("1000.00"), + ) + db.add(claim) + db.flush() + opportunity = SavingsDiscoveryService(db).discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[ + { + "item_id": item.id, + "message": "服务端政策重算", + "original_amount": "1000.00", + "reimbursable_amount": "800.00", + "employee_absorbed_amount": "200.00", + "policy_rule_version": "endpoint-policy-v1", + "policy_grade": "P6", + "policy_matched_city": "深圳", + "calculation_fingerprint": "sha256:" + "c" * 64, + } + ], + current_user=current_user, + request_id="endpoint-discovery-001", + )[0] + return opportunity.id + + +def _user( + username: str, + *, + tenant_id: str = "default", + roles: list[str] | None = None, +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=False, + tenant_id=tenant_id, + employee_id=username, + ) diff --git a/server/tests/test_savings_ledger_services.py b/server/tests/test_savings_ledger_services.py new file mode 100644 index 0000000..7a9533e --- /dev/null +++ b/server/tests/test_savings_ledger_services.py @@ -0,0 +1,466 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, date, datetime +from decimal import Decimal + +import pytest +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext +from app.db.base_class import Base +from app.models.expense_case import BusinessEvent, ExpenseCase +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsOpportunity, + SavingsRealization, +) +from app.schemas.savings import ( + SavingsEvidenceCreate, + SavingsOpportunityActionCreate, + SavingsRealizationActionCreate, + SavingsRealizationCreate, +) +from app.services.savings_access_policy import SavingsPermissionError +from app.services.savings_actions import SavingsActionService +from app.services.savings_discovery import SavingsDiscoveryService +from app.services.savings_protocol import SavingsIdempotencyConflictError +from app.services.savings_query import SavingsQueryService +from app.services.savings_realization import SavingsRealizationService + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_query_is_tenant_scoped_and_owner_scoped(db: Session) -> None: + first = _seed_opportunity(db, tenant_id="tenant-a", owner_id="owner-a") + _seed_opportunity( + db, + tenant_id="tenant-a", + owner_id="owner-b", + department_id="D-2", + ) + _seed_opportunity(db, tenant_id="tenant-b", owner_id="owner-a") + db.commit() + + finance_result = SavingsQueryService(db).list_opportunities( + _user("finance-a", tenant_id="tenant-a", roles=["finance"]) + ) + assert finance_result.total == 2 + assert first.id in {item.id for item in finance_result.items} + + owner_result = SavingsQueryService(db).list_opportunities( + _user("owner-a", tenant_id="tenant-a", employee_id="owner-a") + ) + assert owner_result.total == 1 + scoped_result = SavingsQueryService(db).list_opportunities( + _user( + "budget-a", + tenant_id="tenant-a", + roles=["budget_monitor"], + department_id="D-1", + ) + ) + assert scoped_result.total == 1 + assert scoped_result.items[0].id == first.id + assert ( + SavingsQueryService(db).get_opportunity( + first.id, + _user("outsider", tenant_id="tenant-b", roles=["finance"]), + ) + is None + ) + + +def test_opportunity_action_is_versioned_and_idempotent(db: Session) -> None: + opportunity = _seed_opportunity(db, status="identified", owner_id="owner-a") + db.commit() + owner = _user("owner-a", employee_id="owner-a") + payload = SavingsOpportunityActionCreate( + action="accept", + request_id="accept-request-001", + expected_version=1, + comment="确认接受该节省机会", + ) + + first = SavingsActionService(db).execute(opportunity.id, payload, owner) + replay = SavingsActionService(db).execute(opportunity.id, payload, owner) + + assert first.response.opportunity.status == "accepted" + assert first.response.opportunity.version == 2 + assert replay.response.replayed is True + assert db.scalar( + select(func.count(SavingsEvent.id)).where( + SavingsEvent.actor_id == "owner-a", + SavingsEvent.request_id == payload.request_id, + ) + ) == 1 + + with pytest.raises(SavingsIdempotencyConflictError): + SavingsActionService(db).execute( + opportunity.id, + payload.model_copy(update={"comment": "使用同一请求号篡改内容"}), + owner, + ) + + +def test_actual_requires_independent_finance_confirmation_and_appends_reversal( + db: Session, +) -> None: + opportunity = _seed_opportunity( + db, + status="in_progress", + owner_id="owner-a", + accepted_at=datetime.now(UTC), + started_at=datetime.now(UTC), + ) + db.commit() + owner = _user("owner-a", employee_id="owner-a") + record_payload = SavingsRealizationCreate( + request_id="record-request-001", + expected_version=1, + comment="付款完成后登记实际结果", + actual_gross=Decimal("100.00"), + incremental_cost=Decimal("10.00"), + currency="CNY", + realized_at=datetime.now(UTC), + attribution_method="server_policy_counterfactual", + attribution_ratio=Decimal("1"), + evidence_level="business_state", + evidence=[_result_evidence("record-request-001")], + ) + recorded = SavingsRealizationService(db).record( + opportunity.id, + record_payload, + owner, + ) + realization_id = recorded.response.realization.id + assert recorded.response.realization.status == "pending_confirmation" + assert recorded.response.opportunity.status == "realized" + + confirm_payload = SavingsRealizationActionCreate( + action="confirm", + request_id="confirm-request-001", + expected_version=1, + comment="已复核政策、付款业务状态及归因", + ) + with pytest.raises(SavingsPermissionError): + SavingsRealizationService(db).execute_action( + realization_id, + confirm_payload, + owner, + ) + with pytest.raises(SavingsPermissionError): + SavingsRealizationService(db).execute_action( + realization_id, + confirm_payload, + _user("platform-admin", is_admin=True), + ) + + confirmer = _user("finance-b", employee_id="finance-b", roles=["finance"]) + confirmed = SavingsRealizationService(db).execute_action( + realization_id, + confirm_payload, + confirmer, + ) + assert confirmed.response.realization.status == "finance_confirmed" + assert confirmed.response.realization.dedupe_status == "canonical" + assert confirmed.response.realization.evidence_json[0]["verification_status"] == "verified" + assert confirmed.response.opportunity.status == "verified" + + reversed_result = SavingsRealizationService(db).execute_action( + realization_id, + SavingsRealizationActionCreate( + action="reverse", + request_id="reverse-request-001", + expected_version=2, + comment="员工申诉补付，全额冲回原节省", + reversal_amount=Decimal("90.00"), + ), + confirmer, + ) + original = db.get(SavingsRealization, realization_id) + reversal = db.get(SavingsRealization, reversed_result.response.realization.id) + assert original is not None and original.status == "finance_confirmed" + assert original.reversed_at is not None + assert reversal is not None and reversal.realization_type == "reversal" + assert reversal.actual_net == Decimal("-90.0000") + assert reversed_result.response.opportunity.status == "reversed" + + +def test_payment_realization_is_idempotent_and_stays_pending_confirmation( + db: Session, +) -> None: + opportunity = _seed_opportunity( + db, + status="in_progress", + owner_id="finance", + accepted_at=datetime.now(UTC), + started_at=datetime.now(UTC), + ) + claim = _seed_claim(db, claim_id=opportunity.claim_id) + payment_event = BusinessEvent( + id=str(uuid.uuid4()), + tenant_id="default", + expense_case_id=opportunity.expense_case_id, + aggregate_type="expense_claim", + aggregate_id=claim.id, + event_type="payment_completed", + event_version=1, + idempotency_key="payment-event-001", + correlation_id="payment-event-001", + actor_id="payer", + actor_type="user", + payload_json={}, + delivery_status="pending", + occurred_at=datetime.now(UTC), + ) + db.add(payment_event) + db.commit() + payer = _user("payer", employee_id="payer", roles=["finance"]) + + first = SavingsRealizationService(db).realize_paid_claim( + claim, + payment_event, + payer, + ) + second = SavingsRealizationService(db).realize_paid_claim( + claim, + payment_event, + payer, + ) + db.commit() + + assert len(first) == 1 + assert second == [] + assert first[0].status == "pending_confirmation" + assert first[0].dedupe_status == "pending_review" + assert db.scalar( + select(func.count(SavingsRealization.id)).where( + SavingsRealization.opportunity_id == opportunity.id + ) + ) == 1 + + +def test_standard_adjustment_discovery_freezes_baseline_and_evidence(db: Session) -> None: + claim = _seed_claim(db) + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim_id=claim.id, + item_date=date(2026, 7, 10), + item_type="hotel", + item_reason="上海住宿 2 晚", + item_location="上海", + item_note="", + item_amount=Decimal("1200.00"), + ) + db.add(item) + db.flush() + flag = { + "item_id": item.id, + "message": "服务端按政策把 1200 元调整为 800 元", + "original_amount": "1200.00", + "reimbursable_amount": "800.00", + "employee_absorbed_amount": "400.00", + "policy_rule_version": "v1.2.0", + "policy_rule_version_source": "published", + "policy_grade": "P6", + "policy_matched_city": "上海", + "calculation_fingerprint": "sha256:" + "a" * 64, + } + + opportunities = SavingsDiscoveryService(db).discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[flag], + current_user=_user("employee-a", employee_id="employee-a"), + request_id="standard-adjustment-001", + ) + replay = SavingsDiscoveryService(db).discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[flag], + current_user=_user("employee-a", employee_id="employee-a"), + request_id="standard-adjustment-001", + ) + db.commit() + + assert len(opportunities) == 1 + assert replay[0].id == opportunities[0].id + assert opportunities[0].status == "in_progress" + assert opportunities[0].estimated_net == Decimal("400.0000") + assert opportunities[0].baseline_snapshot.policy_version == "v1.2.0" + assert opportunities[0].baseline_snapshot.baseline_value == Decimal("1200.0000") + assert opportunities[0].baseline_snapshot.data_quality_status == "complete" + assert len(opportunities[0].evidence_links) == 1 + + +def _seed_opportunity( + db: Session, + *, + tenant_id: str = "default", + status: str = "identified", + owner_id: str = "finance", + accepted_at: datetime | None = None, + started_at: datetime | None = None, + department_id: str = "D-1", +) -> SavingsOpportunity: + now = datetime.now(UTC) + case = ExpenseCase( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + case_no=f"CASE-{uuid.uuid4().hex[:12]}", + scene_code="travel", + title="节省测试费用事件", + current_stage="claiming", + status="active", + created_at=now, + updated_at=now, + ) + baseline = ProfileBaselineSnapshot( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + baseline_key=f"baseline-{uuid.uuid4()}", + baseline_type="policy_counterfactual", + dimension_type="expense_claim_item", + dimension_id=str(uuid.uuid4()), + metric_key="pre_adjustment_reimbursable_amount", + unit="currency", + original_currency="CNY", + baseline_value=Decimal("100.00"), + sample_count=1, + method="test_policy", + query_fingerprint="sha256:" + uuid.uuid4().hex, + data_quality_status="complete", + data_quality_score=Decimal("1"), + quality_issues_json=[], + algorithm_version="test-v1", + policy_version="policy-v1", + policy_effective_from=date(2026, 1, 1), + target_resource_type="expense_claim_item", + target_resource_id=str(uuid.uuid4()), + frozen_at=now, + frozen_by="test", + version=1, + created_at=now, + ) + claim_id = str(uuid.uuid4()) + opportunity = SavingsOpportunity( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + opportunity_key=f"opportunity-{uuid.uuid4()}", + benefit_key=f"benefit-{uuid.uuid4()}", + expense_case_id=case.id, + claim_id=claim_id, + claim_no_snapshot=f"BX-{uuid.uuid4().hex[:8]}", + source_type="standard_adjustment", + source_id=str(uuid.uuid4()), + category="policy_compliance", + value_kind="cash", + title="住宿标准重算", + description="测试机会", + exposure_amount=Decimal("100.00"), + baseline_snapshot_id=baseline.id, + baseline_amount=Decimal("100.00"), + target_amount=Decimal("0.00"), + estimated_gross=Decimal("100.00"), + estimated_cost=Decimal("0.00"), + estimated_net=Decimal("100.00"), + estimated_low=Decimal("100.00"), + estimated_high=Decimal("100.00"), + confidence=Decimal("1"), + currency="CNY", + reporting_currency="CNY", + attribution_method="server_policy_counterfactual", + suggested_action="完成付款后登记实际结果", + owner_id=owner_id, + owner_name=owner_id, + owner_role="finance", + status=status, + version=1, + dimension_json={"department_id": department_id, "city": "上海"}, + baseline_snapshot_json={"baseline_value": "100.00"}, + evidence_json=[], + accepted_at=accepted_at, + started_at=started_at, + created_at=now, + updated_at=now, + ) + db.add_all([case, baseline, opportunity]) + db.flush() + return opportunity + + +def _seed_claim(db: Session, *, claim_id: str | None = None) -> ExpenseClaim: + now = datetime.now(UTC) + claim = ExpenseClaim( + id=claim_id or str(uuid.uuid4()), + claim_no=f"BX-{uuid.uuid4().hex[:10]}", + employee_name="测试员工", + department_name="财务部", + project_code="P-001", + expense_type="travel", + reason="差旅", + location="上海", + amount=Decimal("1200.00"), + currency="CNY", + invoice_count=1, + occurred_at=now, + status="draft", + risk_flags_json=[], + ) + db.add(claim) + db.flush() + return claim + + +def _result_evidence(key: str) -> SavingsEvidenceCreate: + return SavingsEvidenceCreate( + evidence_key=f"evidence-{key}", + evidence_role="payment_business_state", + resource_type="business_event", + resource_id=f"payment-{key}", + source_system="x-financial", + external_event_id=f"payment-{key}", + content_hash="c" * 64, + occurred_at=datetime.now(UTC), + verification_status="unverified", + metadata_json={"source": "ledger-service-test"}, + ) + + +def _user( + username: str, + *, + tenant_id: str = "default", + employee_id: str = "", + roles: list[str] | None = None, + is_admin: bool = False, + department_id: str = "", +) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=list(roles or []), + is_admin=is_admin, + tenant_id=tenant_id, + employee_id=employee_id, + department_id=department_id, + ) diff --git a/server/tests/test_savings_models.py b/server/tests/test_savings_models.py new file mode 100644 index 0000000..e189491 --- /dev/null +++ b/server/tests/test_savings_models.py @@ -0,0 +1,131 @@ +from __future__ import annotations + +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, + SavingsRealization, +) + + +def _constraint_names(model: type[object]) -> set[str]: + return { + str(constraint.name) + for constraint in model.__table__.constraints # type: ignore[attr-defined] + if constraint.name is not None + } + + +def test_savings_models_declare_tenant_safe_fact_and_audit_invariants() -> None: + assert { + ProfileBaselineSnapshot.__table__.name, + SavingsOpportunity.__table__.name, + SavingsRealization.__table__.name, + SavingsEvidenceLink.__table__.name, + SavingsEvent.__table__.name, + } == { + "profile_baseline_snapshots", + "savings_opportunities", + "savings_realizations", + "savings_evidence_links", + "savings_events", + } + + assert { + "uq_profile_baseline_snapshots_tenant_id", + "uq_profile_baseline_snapshots_tenant_key", + "ck_profile_baseline_snapshots_historical_shape", + "ck_profile_baseline_snapshots_policy_shape", + "ck_profile_baseline_snapshots_quality_status", + "ck_profile_baseline_snapshots_keys", + }.issubset(_constraint_names(ProfileBaselineSnapshot)) + assert { + "uq_savings_opportunities_tenant_id", + "uq_savings_opportunities_tenant_key", + "fk_savings_opportunities_tenant_case", + "fk_savings_opportunities_tenant_event", + "fk_savings_opportunities_tenant_baseline", + "fk_savings_opportunities_tenant_ai_decision", + "ck_savings_opportunities_net_math", + "ck_savings_opportunities_interval", + "ck_savings_opportunities_currencies", + "ck_savings_opportunities_acceptance", + "ck_savings_opportunities_started", + "ck_savings_opportunities_realized", + "ck_savings_opportunities_verified", + "ck_savings_opportunities_closed", + }.issubset(_constraint_names(SavingsOpportunity)) + assert { + "uq_savings_realizations_tenant_id", + "uq_savings_realizations_tenant_key", + "uq_savings_realizations_tenant_opportunity_benefit_id", + "uq_savings_realizations_tenant_benefit_id", + "fk_savings_realizations_tenant_opportunity", + "fk_savings_realizations_tenant_case", + "fk_savings_realizations_tenant_event", + "fk_savings_realizations_tenant_reversal", + "fk_savings_realizations_tenant_canonical", + "ck_savings_realizations_amount_direction", + "ck_savings_realizations_duplicate_target", + "ck_savings_realizations_confirmation", + "ck_savings_realizations_net_math", + "ck_savings_realizations_currencies", + }.issubset(_constraint_names(SavingsRealization)) + assert { + "uq_savings_evidence_links_tenant_id", + "uq_savings_evidence_links_tenant_key", + "fk_savings_evidence_links_tenant_baseline", + "fk_savings_evidence_links_tenant_opportunity", + "fk_savings_evidence_links_tenant_realization", + "ck_savings_evidence_links_entity_shape", + "ck_savings_evidence_links_verifier", + "ck_savings_evidence_links_keys", + }.issubset(_constraint_names(SavingsEvidenceLink)) + assert { + "uq_savings_events_tenant_id", + "uq_savings_events_actor_request", + "uq_savings_events_aggregate_version", + "fk_savings_events_tenant_baseline", + "fk_savings_events_tenant_opportunity", + "fk_savings_events_tenant_realization", + "ck_savings_events_aggregate_shape", + "ck_savings_events_version", + "ck_savings_events_request", + }.issubset(_constraint_names(SavingsEvent)) + + assert not SavingsOpportunity.__table__.c.claim_id.foreign_keys + assert not SavingsOpportunity.__table__.c.claim_item_id.foreign_keys + assert not SavingsRealization.__table__.c.claim_id.foreign_keys + assert not SavingsRealization.__table__.c.claim_item_id.foreign_keys + + realization_constraints = { + constraint.name: constraint for constraint in SavingsRealization.__table__.constraints + } + reversal_fk = realization_constraints["fk_savings_realizations_tenant_reversal"] + assert tuple(reversal_fk.column_keys) == ( + "tenant_id", + "opportunity_id", + "benefit_key", + "reversal_of_realization_id", + ) + canonical_fk = realization_constraints["fk_savings_realizations_tenant_canonical"] + assert tuple(canonical_fk.column_keys) == ( + "tenant_id", + "benefit_key", + "canonical_realization_id", + ) + confirmation = realization_constraints["ck_savings_realizations_confirmation"] + assert "finance_confirmer_id <> recorded_by_id" in str(confirmation.sqltext) + assert "realization_type = 'reversal'" in str(confirmation.sqltext) + + canonical_index = next( + index + for index in SavingsRealization.__table__.indexes + if index.name == "uq_savings_realizations_actual_canonical_benefit" + ) + assert canonical_index.unique is True + predicate = str(canonical_index.dialect_options["postgresql"]["where"]) + assert "realization_type = 'actual'" in predicate + assert "dedupe_status = 'canonical'" in predicate + assert SavingsEvent.__table__.c.response_json.nullable is False diff --git a/server/tests/test_savings_value_e2e.py b/server/tests/test_savings_value_e2e.py new file mode 100644 index 0000000..9fd09c9 --- /dev/null +++ b/server/tests/test_savings_value_e2e.py @@ -0,0 +1,193 @@ +from __future__ import annotations + +import uuid +from datetime import UTC, date, datetime, timedelta +from decimal import Decimal + +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.api.deps import CurrentUserContext +from app.db.base_class import Base +from app.models.expense_case import BusinessEvent +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.savings import SavingsEvidenceLink, SavingsRealization +from app.schemas.cfo_value import CfoValueFiltersRead +from app.schemas.savings import SavingsRealizationActionCreate +from app.services.cfo_value_analytics import CfoValueAnalyticsService +from app.services.expense_claims import ExpenseClaimService +from app.services.savings_discovery import SavingsDiscoveryService +from app.services.savings_realization import SavingsRealizationService + + +def test_standard_adjustment_payment_confirmation_and_reversal_value_e2e() -> None: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + try: + with factory() as db: + _run_value_e2e(db) + finally: + Base.metadata.drop_all(engine) + engine.dispose() + + +def _run_value_e2e(db: Session) -> None: + payer = _finance_user("finance-payer") + confirmer = _finance_user("finance-confirmer") + claim = ExpenseClaim( + id=str(uuid.uuid4()), + claim_no="BX-SAVINGS-E2E-001", + employee_name="测试员工", + department_name="销售部", + project_code="PROJECT-E2E", + expense_type="hotel", + reason="上海客户现场差旅", + location="上海", + amount=Decimal("66.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime.now(UTC), + submitted_at=datetime.now(UTC), + status="pending_payment", + approval_stage="待付款", + risk_flags_json=[], + ) + item = ExpenseClaimItem( + id=str(uuid.uuid4()), + claim=claim, + item_date=date.today(), + item_type="hotel", + item_reason="上海住宿", + item_location="上海", + item_note="", + item_amount=Decimal("100.00"), + ) + db.add(claim) + db.flush() + opportunity = SavingsDiscoveryService(db).discover_standard_adjustments( + claim=claim, + items_by_id={item.id: item}, + adjustment_flags=[ + { + "item_id": item.id, + "message": "服务端按已发布住宿政策从 100 元调整为 66 元", + "original_amount": "100.00", + "reimbursable_amount": "66.00", + "employee_absorbed_amount": "34.00", + "policy_rule_version": "hotel-policy-e2e-v1", + "policy_rule_version_source": "published", + "policy_grade": "P6", + "policy_matched_city": "上海", + "calculation_fingerprint": "sha256:" + "d" * 64, + } + ], + current_user=payer, + request_id="e2e-discovery-001", + )[0] + db.commit() + + ExpenseClaimService(db).mark_claim_paid( + claim.id, + payer, + request_id="e2e-payment-001", + expected_status="pending_payment", + expected_approval_stage="待付款", + ) + realization = db.scalar( + select(SavingsRealization).where( + SavingsRealization.opportunity_id == opportunity.id, + SavingsRealization.realization_type == "actual", + ) + ) + assert realization is not None + assert realization.status == "pending_confirmation" + assert realization.actual_net == Decimal("34.0000") + assert _verified_cash(db, confirmer) == {} + + confirmed = SavingsRealizationService(db).execute_action( + realization.id, + SavingsRealizationActionCreate( + action="confirm", + request_id="e2e-confirm-001", + expected_version=1, + comment="独立复核政策基线、付款事件、归因和去重键。", + ), + confirmer, + ) + confirmed_as_of = confirmed.response.event.occurred_at + assert _verified_cash(db, confirmer) == {"CNY": Decimal("34.0000")} + evidence = list( + db.scalars( + select(SavingsEvidenceLink).where( + SavingsEvidenceLink.realization_id == realization.id + ) + ).all() + ) + assert evidence and all(item.verification_status == "verified" for item in evidence) + + SavingsRealizationService(db).execute_action( + realization.id, + SavingsRealizationActionCreate( + action="reverse", + request_id="e2e-reverse-001", + expected_version=2, + comment="员工申诉后完成补付，全额冲回原节省。", + reversal_amount=Decimal("34.00"), + ), + confirmer, + ) + assert _verified_cash(db, confirmer) == {"CNY": Decimal("0.0000")} + assert _verified_cash(db, confirmer, as_of=confirmed_as_of) == { + "CNY": Decimal("34.0000") + } + event_types = set( + db.scalars( + select(BusinessEvent.event_type).where( + BusinessEvent.expense_case_id == opportunity.expense_case_id + ) + ).all() + ) + assert { + "saving_opportunity_created", + "payment_completed", + "saving_action_completed", + "saving_confirmed", + "saving_reversed", + } <= event_types + + +def _verified_cash( + db: Session, + current_user: CurrentUserContext, + *, + as_of: datetime | None = None, +) -> dict[str, Decimal]: + now = datetime.now(UTC) + dashboard = CfoValueAnalyticsService(db).build_dashboard( + current_user, + start=now - timedelta(days=1), + end=now + timedelta(days=1), + as_of=as_of or now, + filters=CfoValueFiltersRead(), + ) + return { + item.currency: item.amount for item in dashboard.kpis.verified_cash.values + } + + +def _finance_user(username: str) -> CurrentUserContext: + return CurrentUserContext( + username=username, + name=username, + role_codes=["finance"], + is_admin=False, + tenant_id="default", + employee_id=username, + ) diff --git a/server/tests/test_schema_ownership.py b/server/tests/test_schema_ownership.py index 4f853cf..d7ae7a0 100644 --- a/server/tests/test_schema_ownership.py +++ b/server/tests/test_schema_ownership.py @@ -22,17 +22,43 @@ def test_create_legacy_schema_never_creates_migration_owned_tables() -> None: "approval_action_ledgers", "approval_task_events", "approval_tasks", + "agent_asset_release_audit_samples", + "agent_asset_release_labels", + "agent_asset_release_observations", "auth_sessions", "attachment_association_jobs", "business_events", + "commercial_cost_events", + "commercial_admin_events", + "commercial_billing_periods", + "commercial_entitlements", + "commercial_runtime_reservations", "expense_case_links", "expense_cases", + "financial_connector_configs", + "financial_connector_config_events", + "financial_connector_events", + "financial_connector_operational_events", + "knowledge_onlyoffice_sessions", "memory_entries", "memory_evidence_links", "risk_observations", "risk_observation_feedback", "risk_disposition_events", "risk_dispositions", + "profile_baseline_snapshots", + "payment_reconciliation_cases", + "payment_reconciliation_events", + "savings_evidence_links", + "savings_events", + "savings_opportunities", + "savings_realizations", + "tenant_commercial_plans", + "tenant_finance_report_configs", + "tenant_finance_report_runs", + "tenant_subscriptions", + "tenants", + "usage_meter_events", "few_shot_samples", "workflow_outcomes", } diff --git a/server/tests/test_standard_adjustment_savings_backfill.py b/server/tests/test_standard_adjustment_savings_backfill.py new file mode 100644 index 0000000..9d1092f --- /dev/null +++ b/server/tests/test_standard_adjustment_savings_backfill.py @@ -0,0 +1,354 @@ +from __future__ import annotations + +import argparse +import importlib.util +import uuid +from datetime import UTC, date, datetime +from decimal import Decimal +from pathlib import Path + +import pytest +from sqlalchemy import create_engine, func, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.cli.savings_standard_adjustment_backfill import ( + StandardAdjustmentBackfillDisposition, + StandardAdjustmentSavingsBackfillService, +) +from app.db.base_class import Base +from app.models.expense_case import BusinessEvent, ExpenseCase, ExpenseCaseLink +from app.models.financial_record import ExpenseClaim, ExpenseClaimItem +from app.models.savings import ( + ProfileBaselineSnapshot, + SavingsEvent, + SavingsEvidenceLink, + SavingsOpportunity, +) +from app.services.expense_claim_standard_adjustment import ExpenseClaimStandardAdjustmentMixin + + +@pytest.fixture() +def db() -> Session: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + factory = sessionmaker(bind=engine, expire_on_commit=False) + with factory() as session: + yield session + Base.metadata.drop_all(engine) + engine.dispose() + + +def test_dry_run_reports_eligible_without_any_write(db: Session) -> None: + _seed_adjusted_claim(db, tenant_id="tenant-a") + db.commit() + before = _write_counts(db) + + preview = StandardAdjustmentSavingsBackfillService( + db, + tenant_id="tenant-a", + ).preview() + + assert preview.claims_inspected == 1 + assert preview.flags_inspected == 1 + assert preview.eligible == 1 + assert preview.replayed == 0 + assert preview.skipped == 0 + assert preview.reasons == {"eligible": 1} + assert _write_counts(db) == before + assert not db.new + assert not db.dirty + + +def test_apply_creates_once_and_second_apply_is_replayed(db: Session) -> None: + claim, item = _seed_adjusted_claim(db, tenant_id="tenant-a") + db.commit() + service = StandardAdjustmentSavingsBackfillService(db, tenant_id="tenant-a") + + applied = service.apply_batch(run_id="run-001") + db.commit() + + assert applied.created == 1 + assert applied.replayed == 0 + assert applied.skipped == 0 + opportunity = db.scalar( + select(SavingsOpportunity).where( + SavingsOpportunity.tenant_id == "tenant-a", + SavingsOpportunity.claim_id == claim.id, + ) + ) + assert opportunity is not None + assert opportunity.claim_item_id == item.id + assert opportunity.estimated_net == Decimal("400.0000") + assert opportunity.status == "in_progress" + assert db.scalar( + select(func.count(SavingsEvidenceLink.id)).where( + SavingsEvidenceLink.tenant_id == "tenant-a" + ) + ) == 2 + counts_after_first = _write_counts(db) + + replayed = service.apply_batch(run_id="run-002") + db.commit() + + assert replayed.created == 0 + assert replayed.replayed == 1 + assert replayed.skipped == 0 + assert replayed.items[0].opportunity_id == opportunity.id + assert _write_counts(db) == counts_after_first + + +def test_apply_skips_cross_tenant_missing_case_and_missing_evidence(db: Session) -> None: + _seed_adjusted_claim(db, tenant_id="tenant-b") + _seed_adjusted_claim(db, tenant_id=None) + _seed_adjusted_claim( + db, + tenant_id="tenant-a", + flag_overrides={"policy_rule_version": "", "calculation_fingerprint": ""}, + ) + db.commit() + service = StandardAdjustmentSavingsBackfillService(db, tenant_id="tenant-a") + + preview = service.preview() + + dispositions = {item.disposition for item in preview.items} + assert StandardAdjustmentBackfillDisposition.TENANT_CONFLICT in dispositions + assert StandardAdjustmentBackfillDisposition.MISSING_CASE_LINK in dispositions + assert StandardAdjustmentBackfillDisposition.MISSING_POLICY_VERSION in dispositions + assert preview.eligible == 0 + assert preview.skipped == 3 + + applied = service.apply_batch(run_id="run-quality-report") + db.commit() + + assert applied.created == 0 + assert applied.replayed == 0 + assert applied.skipped == 3 + assert db.scalar(select(func.count(SavingsOpportunity.id))) == 0 + assert db.scalar(select(func.count(ProfileBaselineSnapshot.id))) == 0 + + +def test_tampered_amount_or_policy_snapshot_is_never_monetized(db: Session) -> None: + _seed_adjusted_claim( + db, + tenant_id="tenant-a", + flag_overrides={"original_amount": "9999.00"}, + ) + _seed_adjusted_claim( + db, + tenant_id="tenant-a", + flag_overrides={"policy_hotel_rate": "451.00"}, + ) + db.commit() + + preview = StandardAdjustmentSavingsBackfillService( + db, + tenant_id="tenant-a", + ).preview() + + dispositions = [item.disposition for item in preview.items] + assert StandardAdjustmentBackfillDisposition.ORIGINAL_AMOUNT_MISMATCH in dispositions + assert ( + StandardAdjustmentBackfillDisposition.CALCULATION_FINGERPRINT_MISMATCH + in dispositions + ) + assert preview.eligible == 0 + + +def test_cli_defaults_to_dry_run_and_apply_requires_target_confirmation( + monkeypatch: pytest.MonkeyPatch, +) -> None: + module = _load_cli() + args = module.build_parser().parse_args( + [ + "--tenant-id", + "tenant-a", + "--created-before", + "2026-07-16T00:00:00+08:00", + "--expected-host", + "migration-probe", + "--expected-database", + "migration_probe", + ] + ) + assert args.apply is False + assert args.created_before == datetime(2026, 7, 15, 16, 0, tzinfo=UTC) + + apply_args = module.build_parser().parse_args( + [ + "--apply", + "--tenant-id", + "tenant-a", + "--created-before", + "2026-07-16T00:00:00Z", + "--expected-host", + "migration-probe", + "--expected-database", + "migration_probe", + ] + ) + monkeypatch.delenv("DATABASE_URL", raising=False) + with pytest.raises(module.BackfillCommandError) as exc_info: + module.run(apply_args) + assert exc_info.value.code == "confirm_target_required" + + +@pytest.mark.parametrize( + ("current_revision", "expected"), + [ + ("20260716_0015", True), + ("20260716_0016", True), + ("20260716_0017", True), + ("20260716_0014", False), + (None, False), + ], +) +def test_cli_accepts_any_migration_descended_from_required_savings_revision( + current_revision: str | None, + expected: bool, +) -> None: + module = _load_cli() + + assert module.revision_contains_required(current_revision) is expected + + +@pytest.mark.parametrize("value", ["", "2026-07-16T00:00:00", "not-a-time"]) +def test_cli_rejects_timestamp_without_explicit_timezone(value: str) -> None: + with pytest.raises(argparse.ArgumentTypeError): + _load_cli().parse_timestamp(value) + + +def _seed_adjusted_claim( + db: Session, + *, + tenant_id: str | None, + flag_overrides: dict[str, object] | None = None, +) -> tuple[ExpenseClaim, ExpenseClaimItem]: + claim_id = str(uuid.uuid4()) + item_id = str(uuid.uuid4()) + suffix = uuid.uuid4().hex[:10] + policy_result = { + "days": 2, + "location": "上海市", + "matched_city": "上海", + "grade": "P6", + "grade_band": "P6-P7", + "hotel_rate": Decimal("400.00"), + "hotel_amount": Decimal("800.00"), + "rule_name": "差旅住宿标准", + "rule_version": "hotel-policy-v3", + } + flag: dict[str, object] = { + "source": "reimbursement_standard_adjustment", + "event_type": "standard_adjustment_accepted", + "calculation_source": "server_policy", + "item_id": item_id, + "original_amount": "1200.00", + "reimbursable_amount": "800.00", + "employee_absorbed_amount": "400.00", + "policy_days": policy_result["days"], + "policy_location": policy_result["location"], + "policy_matched_city": policy_result["matched_city"], + "policy_grade": policy_result["grade"], + "policy_grade_band": policy_result["grade_band"], + "policy_hotel_rate": "400.00", + "policy_hotel_amount": "800.00", + "policy_rule_name": policy_result["rule_name"], + "policy_rule_version": policy_result["rule_version"], + "policy_rule_version_source": "finance_rules_content_hash", + } + flag["calculation_fingerprint"] = ( + ExpenseClaimStandardAdjustmentMixin._standard_adjustment_calculation_fingerprint( + item_id=item_id, + original_amount=Decimal("1200.00"), + reimbursable_amount=Decimal("800.00"), + policy_result=policy_result, + ) + ) + flag.update(flag_overrides or {}) + claim = ExpenseClaim( + id=claim_id, + claim_no=f"BX-BACKFILL-{suffix}", + employee_name="历史员工", + department_name="销售部", + project_code="PRJ-BACKFILL", + expense_type="hotel", + reason="历史住宿报销", + location="上海", + amount=Decimal("800.00"), + currency="CNY", + invoice_count=1, + occurred_at=datetime(2026, 6, 1, 9, 0, tzinfo=UTC), + submitted_at=datetime(2026, 6, 2, 9, 0, tzinfo=UTC), + status="submitted", + approval_stage="财务审批", + risk_flags_json=[flag], + created_at=datetime(2026, 6, 1, 8, 0, tzinfo=UTC), + updated_at=datetime(2026, 6, 2, 8, 0, tzinfo=UTC), + ) + item = ExpenseClaimItem( + id=item_id, + claim=claim, + item_date=date(2026, 6, 1), + item_type="hotel", + item_reason="上海住宿 2 晚", + item_location="上海", + item_note="", + item_amount=Decimal("1200.00"), + ) + db.add(claim) + if tenant_id is not None: + expense_case = ExpenseCase( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + case_no=f"CASE-{suffix}", + scene_code="travel", + title="历史住宿费用事件", + current_stage="claiming", + status="active", + ) + db.add_all( + [ + expense_case, + ExpenseCaseLink( + id=str(uuid.uuid4()), + tenant_id=tenant_id, + expense_case_id=expense_case.id, + resource_type="expense_claim", + resource_id=claim.id, + relation_type="reimbursement", + ), + ] + ) + return claim, item + + +def _write_counts(db: Session) -> tuple[int, ...]: + return tuple( + int(db.scalar(select(func.count()).select_from(model)) or 0) + for model in ( + SavingsOpportunity, + ProfileBaselineSnapshot, + SavingsEvidenceLink, + SavingsEvent, + BusinessEvent, + ) + ) + + +def _load_cli(): + path = ( + Path(__file__).resolve().parents[1] + / "scripts" + / "backfill_standard_adjustment_savings.py" + ) + spec = importlib.util.spec_from_file_location("backfill_standard_adjustment_savings_cli", path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module diff --git a/server/tests/test_steward_action_executor.py b/server/tests/test_steward_action_executor.py index 9ee2913..29f02cd 100644 --- a/server/tests/test_steward_action_executor.py +++ b/server/tests/test_steward_action_executor.py @@ -50,6 +50,7 @@ def build_client() -> tuple[TestClient, sessionmaker[Session]]: def seed_employee(db: Session) -> None: manager = Employee( id="steward-action-manager", + tenant_id="tenant-steward-action", employee_no="E90000", name="李总", email="leader@example.com", @@ -58,6 +59,7 @@ def seed_employee(db: Session) -> None: ) employee = Employee( id="steward-action-employee", + tenant_id="tenant-steward-action", employee_no="E90001", name="张三", email="zhangsan@example.com", @@ -166,6 +168,7 @@ def issue_application_preview( def seed_approved_application(db: Session) -> None: application = ExpenseClaim( id="application-action-approved", + tenant_id="tenant-steward-action", claim_no="AAPPROVED1", employee_id="steward-action-employee", employee_name="张三", @@ -252,7 +255,10 @@ def test_steward_action_executor_records_pending_interrupt_in_conversation_state headers=auth_headers(), json={ "action_type": "submit_application", - "message": "2026-02-20 至 2026-02-23，去上海出差，辅助国网仿生产服务器部署，交通火车，直接提交", + "message": ( + "2026-02-20 至 2026-02-23，去上海出差，" + "辅助国网仿生产服务器部署，交通火车，直接提交" + ), "conversation_id": "conv-action-submit", "client_trace_id": "trace-submit-pending", "task": base_application_task("submit"), @@ -368,7 +374,8 @@ def test_steward_action_checkpoint_does_not_replay_across_tenants() -> None: } -def test_steward_action_executor_reuses_checkpoint_for_duplicate_trace_without_duplicate_draft() -> None: +def test_steward_action_executor_reuses_checkpoint_for_duplicate_trace_without_duplicate_draft( +) -> None: client, session_factory = build_client() with session_factory() as db: seed_employee(db) @@ -380,7 +387,10 @@ def test_steward_action_executor_reuses_checkpoint_for_duplicate_trace_without_d ) request_payload = { "action_type": "save_application_draft", - "message": "2026-02-20 至 2026-02-23，去上海出差，辅助国网仿生产服务器部署，交通火车，保存草稿", + "message": ( + "2026-02-20 至 2026-02-23，去上海出差，" + "辅助国网仿生产服务器部署，交通火车，保存草稿" + ), "conversation_id": "conv-action-draft", "client_trace_id": "trace-save-draft", "decision_id": issued["decision_id"], @@ -425,7 +435,10 @@ def test_steward_action_executor_requires_confirmation_before_submit_side_effect headers=auth_headers(), json={ "action_type": "submit_application", - "message": "2026-02-20 至 2026-02-23，去上海出差，辅助国网仿生产服务器部署，交通火车，直接提交", + "message": ( + "2026-02-20 至 2026-02-23，去上海出差，" + "辅助国网仿生产服务器部署，交通火车，直接提交" + ), "task": base_application_task("submit"), "confirmed": False, "context_json": { @@ -460,7 +473,10 @@ def test_steward_action_executor_saves_application_draft_from_action_step() -> N headers=auth_headers(), json={ "action_type": "save_application_draft", - "message": "2026-02-20 至 2026-02-23，去上海出差，辅助国网仿生产服务器部署，交通火车，保存草稿", + "message": ( + "2026-02-20 至 2026-02-23，去上海出差，" + "辅助国网仿生产服务器部署，交通火车，保存草稿" + ), "conversation_id": "conv-action-save", "client_trace_id": "trace-save-application-action", "decision_id": issued["decision_id"], @@ -541,7 +557,8 @@ def test_steward_action_executor_blocks_save_without_decision_and_stable_trace() assert claim_count(db) == 0 -def test_steward_action_executor_submits_verified_application_after_confirmation_and_precheck() -> None: +def test_steward_action_executor_submits_verified_application_after_confirmation_and_precheck( +) -> None: client, session_factory = build_client() with session_factory() as db: seed_employee(db) diff --git a/server/tests/test_steward_planner.py b/server/tests/test_steward_planner.py index b1e25a8..85637e4 100644 --- a/server/tests/test_steward_planner.py +++ b/server/tests/test_steward_planner.py @@ -9,7 +9,7 @@ from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker from sqlalchemy.pool import StaticPool -from app.api.deps import get_db +from app.api.deps import CurrentUserContext, get_current_user, get_db from app.db.base import Base from app.main import create_app from app.models.financial_record import ExpenseClaim @@ -299,6 +299,16 @@ def _create_steward_test_client_with_db(): db.close() app.dependency_overrides[get_db] = override_db + app.dependency_overrides[get_current_user] = lambda: CurrentUserContext( + username="zhang.xiaoqing", + name="张小青", + role_codes=["user"], + is_admin=False, + tenant_id="default", + department_name="产品交付部", + employee_no="E-STEW-001", + employee_id="employee-steward-001", + ) return TestClient(app), TestingSessionLocal, app @@ -756,22 +766,25 @@ def test_steward_planner_builds_travel_attachment_group_with_exclusions() -> Non def test_steward_stream_endpoint_emits_thinking_before_plan(monkeypatch) -> None: _patch_steward_endpoint_planner(monkeypatch) - client = TestClient(create_app()) + client, _session_factory, app = _create_steward_test_client_with_db() - with client.stream( - "POST", - "/api/v1/steward/plans/stream", - json={ - "message": "我要报销昨天的交通费", - "client_now_iso": "2026-06-04T09:30:00+08:00", - }, - ) as response: - assert response.status_code == 200 - events = [ - json.loads(line.decode("utf-8") if isinstance(line, bytes) else line) - for line in response.iter_lines() - if line - ] + try: + with client.stream( + "POST", + "/api/v1/steward/plans/stream", + json={ + "message": "我要报销昨天的交通费", + "client_now_iso": "2026-06-04T09:30:00+08:00", + }, + ) as response: + assert response.status_code == 200 + events = [ + json.loads(line.decode("utf-8") if isinstance(line, bytes) else line) + for line in response.iter_lines() + if line + ] + finally: + app.dependency_overrides.clear() assert [event["event"] for event in events][:2] == ["thinking", "thinking"] assert events[0]["data"]["stage"] == "stream_start" @@ -781,17 +794,20 @@ def test_steward_stream_endpoint_emits_thinking_before_plan(monkeypatch) -> None def test_steward_plan_endpoint_persists_application_and_reimbursement_state(monkeypatch) -> None: _patch_steward_endpoint_planner(monkeypatch) - client = TestClient(create_app()) + client, _session_factory, app = _create_steward_test_client_with_db() - response = client.post( - "/api/v1/steward/plans", - json={ - "message": "我想申请7月2日去北京出差，并且我要报销昨天的交通费", - "user_id": "u-steward-state", - "client_now_iso": "2026-06-04T09:30:00+08:00", - "context_json": {"session_type": "steward", "entry_source": "personal_workbench"}, - }, - ) + try: + response = client.post( + "/api/v1/steward/plans", + json={ + "message": "我想申请7月2日去北京出差，并且我要报销昨天的交通费", + "user_id": "u-steward-state", + "client_now_iso": "2026-06-04T09:30:00+08:00", + "context_json": {"session_type": "steward", "entry_source": "personal_workbench"}, + }, + ) + finally: + app.dependency_overrides.clear() assert response.status_code == 200 payload = response.json() diff --git a/server/tests/test_steward_tenant_security.py b/server/tests/test_steward_tenant_security.py new file mode 100644 index 0000000..8fd776a --- /dev/null +++ b/server/tests/test_steward_tenant_security.py @@ -0,0 +1,367 @@ +from __future__ import annotations + +from collections.abc import Generator +from datetime import UTC, datetime +from decimal import Decimal + +import pytest +from fastapi.testclient import TestClient +from sqlalchemy import select +from sqlalchemy.orm import Session + +from app.api.deps import CurrentUserContext, get_current_user, get_db +from app.api.v1.endpoints import steward as steward_endpoint +from app.main import create_app +from app.models.agent_conversation import AgentConversation +from app.models.financial_record import ExpenseClaim +from app.schemas.steward import ( + StewardPlanResponse, + StewardRuntimeDecisionResponse, + StewardSlotDecisionResponse, +) +from app.services.expense_cases import ExpenseCaseService +from app.test_helpers.db import build_in_memory_session_factory + + +def _current_user(*, tenant_id: str = "tenant-a") -> CurrentUserContext: + return CurrentUserContext( + username="trusted.user@example.com", + name="可信用户", + role_codes=["user"], + is_admin=False, + tenant_id=tenant_id, + department_name="交付部", + department_id="dept-delivery", + cost_center="CC-TRUSTED", + position="实施顾问", + grade="P6", + employee_no="E-TRUSTED", + employee_id="employee-trusted", + manager_name="可信经理", + auth_session_id="session-secret", + ) + + +def _build_client( + *, + current_user: CurrentUserContext | None, +) -> tuple[TestClient, object, object]: + session_factory = build_in_memory_session_factory() + app = create_app() + + def override_db() -> Generator[Session, None, None]: + db = session_factory() + try: + yield db + finally: + db.close() + + app.dependency_overrides[get_db] = override_db + if current_user is not None: + app.dependency_overrides[get_current_user] = lambda: current_user + return TestClient(app), session_factory, app + + +@pytest.mark.parametrize( + ("path", "payload"), + [ + ("/api/v1/steward/plans", {"message": "报销交通费"}), + ("/api/v1/steward/plans/stream", {"message": "报销交通费"}), + ( + "/api/v1/steward/slot-decisions", + {"task_type": "reimbursement", "user_message": "报销交通费"}, + ), + ( + "/api/v1/steward/runtime-decisions", + {"user_message": "继续处理"}, + ), + ], +) +def test_steward_ai_endpoints_require_authenticated_user( + path: str, + payload: dict[str, object], +) -> None: + client, _session_factory, app = _build_client(current_user=None) + try: + response = client.post(path, json=payload) + assert response.status_code == 401 + finally: + app.dependency_overrides.clear() + + +@pytest.mark.parametrize( + ("path", "payload"), + [ + ("/api/v1/steward/plans", {"message": "报销交通费"}), + ("/api/v1/steward/plans/stream", {"message": "报销交通费"}), + ( + "/api/v1/steward/slot-decisions", + {"task_type": "reimbursement", "user_message": "报销交通费"}, + ), + ( + "/api/v1/steward/runtime-decisions", + {"user_message": "继续处理"}, + ), + ], +) +def test_steward_ai_endpoints_fail_closed_without_tenant( + path: str, + payload: dict[str, object], +) -> None: + client, _session_factory, app = _build_client( + current_user=_current_user(tenant_id=""), + ) + try: + response = client.post(path, json=payload) + assert response.status_code == 403 + assert "缺少租户归属" in response.json()["detail"] + finally: + app.dependency_overrides.clear() + + +def test_steward_plan_and_stream_use_authenticated_identity(monkeypatch) -> None: + captured_payloads = [] + + class CapturingPlanner: + @staticmethod + def _clean_text(value): + return str(value or "").strip() + + @staticmethod + def _resolve_base_date(_client_now_iso, _context_json): + return datetime(2026, 7, 16, tzinfo=UTC).date() + + @staticmethod + def _looks_like_ambiguous_travel_flow(_message, _base_date, _payload): + return False + + @staticmethod + def build_plan(payload): + captured_payloads.append(payload) + return StewardPlanResponse( + plan_id=f"plan-auth-{len(captured_payloads)}", + summary="已按可信登录身份生成计划。", + ) + + monkeypatch.setattr( + steward_endpoint, + "_build_steward_planner", + lambda _db: CapturingPlanner(), + ) + current_user = _current_user() + client, session_factory, app = _build_client(current_user=current_user) + forged_context = { + "tenant_id": "tenant-forged", + "tenantId": "tenant-forged-camel", + "user_id": "forged-user", + "userId": "forged-user-camel", + "username": "forged@example.com", + "role_codes": ["admin"], + "is_admin": True, + "auth_session_id": "forged-session", + "session_type": "steward", + } + try: + plan_response = client.post( + "/api/v1/steward/plans", + json={ + "message": "报销交通费", + "user_id": "forged-user", + "context_json": forged_context, + }, + ) + assert plan_response.status_code == 200 + + with client.stream( + "POST", + "/api/v1/steward/plans/stream", + json={ + "message": "继续报销交通费", + "user_id": "forged-user", + "context_json": forged_context, + }, + ) as stream_response: + assert stream_response.status_code == 200 + assert any(stream_response.iter_lines()) + + assert len(captured_payloads) == 2 + for captured in captured_payloads: + assert captured.user_id == current_user.username + assert captured.context_json["tenant_id"] == "tenant-a" + assert captured.context_json["user_id"] == current_user.username + assert captured.context_json["username"] == current_user.username + assert captured.context_json["role_codes"] == ["user"] + assert captured.context_json["is_admin"] is False + assert "tenantId" not in captured.context_json + assert "userId" not in captured.context_json + assert "auth_session_id" not in captured.context_json + + with session_factory() as db: + conversations = list(db.scalars(select(AgentConversation)).all()) + assert len(conversations) == 2 + assert all(item.user_id == current_user.username for item in conversations) + assert all( + item.state_json.get("tenant_id") == "tenant-a" + for item in conversations + ) + finally: + app.dependency_overrides.clear() + + +def test_steward_slot_and_runtime_ignore_forged_identity(monkeypatch) -> None: + captured_slot_payloads = [] + captured_runtime_payloads = [] + + def fake_slot_decision(payload, _runtime_chat): + captured_slot_payloads.append(payload) + return StewardSlotDecisionResponse( + next_action="ask_user", + question="请补充金额。", + ) + + def fake_runtime_decision(payload, _runtime_chat): + captured_runtime_payloads.append(payload) + return StewardRuntimeDecisionResponse(next_action="no_op") + + monkeypatch.setattr(steward_endpoint, "_decide_steward_slot", fake_slot_decision) + monkeypatch.setattr( + steward_endpoint, + "_decide_steward_runtime", + fake_runtime_decision, + ) + client, _session_factory, app = _build_client(current_user=_current_user()) + try: + slot_response = client.post( + "/api/v1/steward/slot-decisions", + json={ + "task_type": "reimbursement", + "user_message": "报销交通费", + "task_context": { + "tenant_id": "tenant-forged", + "tenantId": "tenant-forged-camel", + "username": "forged@example.com", + "is_admin": True, + }, + }, + ) + runtime_response = client.post( + "/api/v1/steward/runtime-decisions", + json={ + "user_message": "继续处理", + "context_json": { + "tenant_id": "tenant-forged", + "tenantId": "tenant-forged-camel", + "username": "forged@example.com", + }, + "runtime_state": { + "tenant_id": "tenant-forged-runtime", + "user_id": "forged-runtime-user", + }, + }, + ) + + assert slot_response.status_code == 200 + assert runtime_response.status_code == 200 + assert len(captured_slot_payloads) == 1 + assert len(captured_runtime_payloads) == 1 + slot_context = captured_slot_payloads[0].task_context + assert slot_context["tenant_id"] == "tenant-a" + assert slot_context["username"] == "trusted.user@example.com" + assert slot_context["is_admin"] is False + assert "tenantId" not in slot_context + + runtime_payload = captured_runtime_payloads[0] + assert runtime_payload.context_json["tenant_id"] == "tenant-a" + assert runtime_payload.runtime_state["tenant_id"] == "tenant-a" + assert runtime_payload.context_json["user_id"] == "trusted.user@example.com" + assert runtime_payload.runtime_state["user_id"] == "trusted.user@example.com" + assert "tenantId" not in runtime_payload.context_json + finally: + app.dependency_overrides.clear() + + +def test_steward_runtime_rejects_cross_tenant_conversation(monkeypatch) -> None: + client, session_factory, app = _build_client(current_user=_current_user()) + with session_factory() as db: + db.add( + AgentConversation( + conversation_id="conv-tenant-b", + user_id="trusted.user@example.com", + source="user_message", + state_json={ + "tenant_id": "tenant-b", + "session_type": "steward", + "steward_state": {"active_flow": "travel_reimbursement"}, + }, + ) + ) + db.commit() + + monkeypatch.setattr( + steward_endpoint, + "_decide_steward_runtime", + lambda *_args, **_kwargs: pytest.fail("越权会话不应进入决策服务"), + ) + try: + response = client.post( + "/api/v1/steward/runtime-decisions", + json={ + "user_message": "继续处理", + "context_json": {"conversation_id": "conv-tenant-b"}, + }, + ) + assert response.status_code == 403 + assert "不属于登录用户或租户" in response.json()["detail"] + finally: + app.dependency_overrides.clear() + + +def test_steward_application_candidates_are_tenant_scoped() -> None: + _client, session_factory, app = _build_client(current_user=_current_user()) + try: + with session_factory() as db: + for tenant_id, suffix in (("tenant-a", "A"), ("tenant-b", "B")): + claim = ExpenseClaim( + id=f"application-tenant-{suffix.lower()}", + tenant_id=tenant_id, + claim_no=f"AP-TENANT-{suffix}", + employee_id="employee-trusted", + employee_name="可信用户", + department_name="交付部", + expense_type="travel_application", + reason="上海客户现场部署", + location="上海", + amount=Decimal("1200.00"), + currency="CNY", + invoice_count=0, + occurred_at=datetime(2026, 7, 20, tzinfo=UTC), + submitted_at=datetime(2026, 7, 16, tzinfo=UTC), + status="approved", + approval_stage="已完成", + risk_flags_json=[], + ) + db.add(claim) + db.flush() + ExpenseCaseService(db).ensure_case_for_claim( + claim, + tenant_id=tenant_id, + ) + db.commit() + + payload = steward_endpoint._bind_authenticated_plan_request( + steward_endpoint.StewardPlanRequest( + message="7月20日去上海出差，继续发起报销", + context_json={}, + ), + _current_user(), + ) + candidates = steward_endpoint._query_required_application_gate_candidates( + db, + payload, + payload.context_json, + tenant_id="tenant-a", + ) + + assert [item["claim_no"] for item in candidates] == ["AP-TENANT-A"] + finally: + app.dependency_overrides.clear() diff --git a/server/tests/test_tenant_identity_security.py b/server/tests/test_tenant_identity_security.py new file mode 100644 index 0000000..6b8a05e --- /dev/null +++ b/server/tests/test_tenant_identity_security.py @@ -0,0 +1,196 @@ +from __future__ import annotations + +from datetime import UTC, datetime, timedelta + +import pytest +from sqlalchemy import create_engine, select +from sqlalchemy.orm import Session, sessionmaker +from sqlalchemy.pool import StaticPool + +import app.models # noqa: F401 - 注册完整 metadata +from app.core.security import hash_password +from app.db.base_class import Base +from app.models.auth_session import AuthSession +from app.models.employee import Employee +from app.models.tenant import Tenant, TenantMembership +from app.schemas.auth import LoginRequest +from app.schemas.employee import EmployeeUpdate +from app.services.auth import AuthenticatedUser, AuthService +from app.services.auth_sessions import AuthSessionService +from app.services.employee import EmployeeService + + +@pytest.fixture() +def factory() -> sessionmaker[Session]: + engine = create_engine( + "sqlite+pysqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(engine) + result = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False) + try: + yield result + finally: + Base.metadata.drop_all(engine) + engine.dispose() + + +def _seed_two_tenants(db: Session) -> tuple[Employee, Employee]: + db.add_all( + [ + Tenant( + tenant_id="tenant-a", + tenant_code="company-a", + name="企业 A", + status="active", + ), + Tenant( + tenant_id="tenant-b", + tenant_code="company-b", + name="企业 B", + status="active", + ), + ] + ) + employee_a = Employee( + tenant_id="tenant-a", + employee_no="E-SAME", + name="A 员工", + email="same@example.com", + password_hash=hash_password("secure-password"), + employment_status="在职", + ) + employee_b = Employee( + tenant_id="tenant-b", + employee_no="E-SAME", + name="B 员工", + email="same@example.com", + password_hash=hash_password("secure-password"), + employment_status="在职", + ) + db.add_all([employee_a, employee_b]) + db.flush() + db.add_all( + [ + TenantMembership( + tenant_id="tenant-a", + employee_id=employee_a.id, + status="active", + is_primary=True, + ), + TenantMembership( + tenant_id="tenant-b", + employee_id=employee_b.id, + status="active", + is_primary=True, + ), + ] + ) + db.commit() + return employee_a, employee_b + + +def test_same_email_login_is_bound_to_authenticated_membership( + factory: sessionmaker[Session], +) -> None: + with factory() as db: + employee_a, employee_b = _seed_two_tenants(db) + auth = AuthService(db) + + with pytest.raises(ValueError, match="关联多个企业"): + auth.login( + LoginRequest( + username="same@example.com", + password="secure-password", + ) + ) + + login_a = auth.login( + LoginRequest( + username="same@example.com", + password="secure-password", + tenantId="company-a", + ) + ) + login_b = auth.login( + LoginRequest( + username="same@example.com", + password="secure-password", + tenantId="tenant-b", + ) + ) + + assert login_a.user.tenantId == "tenant-a" + assert login_b.user.tenantId == "tenant-b" + sessions = list(db.scalars(select(AuthSession).order_by(AuthSession.created_at))) + assert {item.tenant_id for item in sessions} == {"tenant-a", "tenant-b"} + assert {item.employee_id for item in sessions} == {employee_a.id, employee_b.id} + + +def test_employee_service_hides_cross_tenant_resource_ids( + factory: sessionmaker[Session], +) -> None: + with factory() as db: + employee_a, employee_b = _seed_two_tenants(db) + service_a = EmployeeService(db, tenant_id="tenant-a") + + assert service_a.get_employee(employee_a.id) is not None + assert service_a.get_employee(employee_b.id) is None + with pytest.raises(LookupError, match="Employee not found"): + service_a.update_employee( + employee_b.id, + EmployeeUpdate(name="被 A 修改"), + ) + + db.refresh(employee_b) + assert employee_b.name == "B 员工" + + +def test_session_tenant_cannot_be_rebound_to_another_employee( + factory: sessionmaker[Session], +) -> None: + with factory() as db: + employee_a, _employee_b = _seed_two_tenants(db) + mismatched = AuthSession( + token_hash="mismatched-token-hash", + tenant_id="tenant-b", + principal_type="employee", + employee_id=employee_a.id, + username=employee_a.email, + metric_session_id="metric-a", + issued_at=datetime.now(UTC), + expires_at=datetime.now(UTC) + timedelta(minutes=30), + last_seen_at=datetime.now(UTC), + ) + db.add(mismatched) + db.commit() + + assert AuthService(db).get_session_user(mismatched) is None + + +def test_session_issue_rejects_missing_tenant( + factory: sessionmaker[Session], +) -> None: + with factory() as db: + user = AuthenticatedUser( + username="missing-tenant@example.com", + name="无租户用户", + role="使用者", + department="", + position="", + grade="", + employee_no="", + manager_name="", + location="", + cost_center="", + finance_owner_name="", + risk_profile={}, + role_codes=["user"], + email="missing-tenant@example.com", + avatar="无", + tenant_id="", + ) + + with pytest.raises(ValueError, match="tenant_id"): + AuthSessionService(db).issue(user, metric_session_id="metric-missing") diff --git a/server/tests/test_user_agent_application_draft_events.py b/server/tests/test_user_agent_application_draft_events.py index edebf1e..e280140 100644 --- a/server/tests/test_user_agent_application_draft_events.py +++ b/server/tests/test_user_agent_application_draft_events.py @@ -79,12 +79,14 @@ def test_ai_application_draft_update_writes_tenant_scoped_event() -> None: with session_factory() as db: owner = Employee( id="owner-1", + tenant_id="tenant-a", employee_no="E001", name="张三", email="owner@example.com", ) claim = ExpenseClaim( id="application-1", + tenant_id="tenant-a", claim_no="AP-DRAFT-001", employee_id=owner.id, employee_name=owner.name, @@ -151,9 +153,10 @@ def test_untrusted_application_path_does_not_write_learning_ledger() -> None: ) assert db.get(ExpenseClaim, claim.id) is not None - assert db.scalar( - select(BusinessEvent).where(BusinessEvent.aggregate_id == claim.id) - ) is not None + assert ( + db.scalar(select(BusinessEvent).where(BusinessEvent.aggregate_id == claim.id)) + is not None + ) assert list(db.scalars(select(AIDecision)).all()) == [] assert list(db.scalars(select(AIDecisionFeedback)).all()) == [] assert list(db.scalars(select(WorkflowOutcome)).all()) == [] @@ -166,6 +169,23 @@ def test_non_default_tenant_direct_submit_creates_same_tenant_case_link() -> Non run_id="application-direct-submit-tenant", tenant_id="tenant-direct-submit", ) + manager = Employee( + id="manager-direct-submit", + tenant_id="tenant-direct-submit", + employee_no="M-DIRECT", + name="直属领导", + email="manager-direct@example.com", + ) + owner = Employee( + id="owner-direct-submit", + tenant_id="tenant-direct-submit", + employee_no="E001", + name="张三", + email="owner@example.com", + manager=manager, + ) + db.add_all([manager, owner]) + db.commit() service = UserAgentService(db) submitted = service._create_expense_application_record( request, @@ -352,9 +372,7 @@ def test_ai_application_draft_update_deduplicates_identical_snapshot() -> None: ) events = list( - db.scalars( - select(BusinessEvent).where(BusinessEvent.aggregate_id == claim.id) - ).all() + db.scalars(select(BusinessEvent).where(BusinessEvent.aggregate_id == claim.id)).all() ) assert len(events) == 1 assert events[0].event_type == "claim_draft_updated" @@ -413,9 +431,7 @@ def test_ai_application_draft_update_keeps_distinct_snapshots_in_same_run() -> N ) events = list( - db.scalars( - select(BusinessEvent).where(BusinessEvent.aggregate_id == claim.id) - ).all() + db.scalars(select(BusinessEvent).where(BusinessEvent.aggregate_id == claim.id)).all() ) assert len(events) == 2 assert {event.event_type for event in events} == {"claim_draft_updated"} diff --git a/web/src/assets/styles/components/cfo-value-dashboard.css b/web/src/assets/styles/components/cfo-value-dashboard.css new file mode 100644 index 0000000..27787c4 --- /dev/null +++ b/web/src/assets/styles/components/cfo-value-dashboard.css @@ -0,0 +1,424 @@ +.cfo-value-dashboard { + min-width: 0; + display: grid; + gap: 16px; + padding-bottom: 16px; +} + +.value-dashboard-header { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 24px; + padding: 20px; + border-top: 3px solid var(--theme-primary); +} + +.value-eyebrow { + display: block; + margin-bottom: 5px; + color: var(--theme-primary-active); + font-size: 12px; + font-weight: 850; + letter-spacing: .1em; + text-transform: uppercase; +} + +.value-dashboard-header h2 { + margin: 0; + color: var(--ink); + font-size: 21px; + line-height: 1.3; +} + +.value-dashboard-header p, +.value-filter-head p, +.value-card-head p { + margin: 5px 0 0; + color: var(--muted); + font-size: 12px; + line-height: 1.55; +} + +.value-source-summary { + min-width: 220px; + display: grid; + grid-template-columns: auto auto; + align-items: center; + justify-content: end; + gap: 6px 12px; +} + +.value-source-summary small { + grid-column: 1; + color: var(--muted); + font-size: 12px; + text-align: right; +} + +.value-source-summary .btn { + grid-column: 2; + grid-row: 1 / span 3; + min-width: 96px; + min-height: 44px; +} + +.value-source-status { + min-height: 28px; + display: inline-flex; + align-items: center; + justify-content: center; + gap: 6px; + padding: 4px 9px; + border-radius: 999px; + background: var(--success-soft); + color: var(--success); + font-size: 12px; + font-weight: 800; +} + +.value-source-status.partial, +.value-source-status.stale { background: var(--warning-soft); color: var(--warning); } +.value-source-status.error, +.value-source-status.permission { background: var(--danger-soft); color: var(--danger); } +.value-source-status.empty, +.value-source-status.loading { background: var(--info-soft); color: var(--info); } + +.value-filter-panel { padding: 16px 18px; } + +.value-filter-head, +.value-card-head, +.value-filter-actions { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 16px; +} + +.value-filter-head h3, +.value-card-head h3 { + margin: 0; + color: var(--ink); + font-size: 15px; + line-height: 1.35; +} + +.value-filter-toggle { + min-height: 44px; + display: inline-flex; + align-items: center; + gap: 7px; + padding: 0 12px; + border: 1px solid var(--line); + border-radius: var(--radius); + background: var(--surface); + color: var(--text); + font-size: 12px; + font-weight: 750; +} + +.value-filter-toggle span { + min-width: 20px; + height: 20px; + display: grid; + place-items: center; + border-radius: 999px; + background: var(--theme-primary); + color: var(--theme-primary-contrast); + font-size: 12px; +} + +.value-filter-form { display: grid; gap: 12px; margin-top: 14px; } +.value-filter-primary, +.value-filter-advanced { + display: grid; + grid-template-columns: repeat(4, minmax(0, 1fr)); + gap: 10px; +} + +.value-filter-advanced { + grid-template-columns: repeat(5, minmax(0, 1fr)); + padding-top: 12px; + border-top: 1px dashed var(--line); +} + +.value-filter-form label, +.value-compact-select { + min-width: 0; + display: grid; + gap: 6px; + color: var(--text); + font-size: 12px; + font-weight: 750; +} + +.value-filter-form label > span em { + margin-left: 4px; + color: var(--muted); + font-size: 12px; + font-style: normal; + font-weight: 500; +} + +.value-filter-form input, +.value-filter-form select, +.value-compact-select select { + width: 100%; + min-height: 44px; + padding: 8px 10px; + border: 1px solid var(--line-strong); + border-radius: var(--radius); + background: var(--surface); + color: var(--ink); + font-size: 13px; +} + +.value-filter-actions { + justify-content: flex-end; + padding-top: 2px; +} + +.value-filter-actions .btn { min-width: 112px; } +.btn:disabled { cursor: not-allowed; opacity: .5; transform: none; box-shadow: none; } + +.value-feedback { + min-height: 44px; + display: flex; + align-items: center; + gap: 9px; + padding: 10px 14px; + border: 1px solid var(--line); + border-left-width: 4px; + border-radius: var(--radius); + background: var(--surface); + color: var(--text); + font-size: 12px; + line-height: 1.5; +} + +.value-feedback.success { border-color: var(--success-line); border-left-color: var(--success); background: var(--success-soft); color: var(--success-active); } +.value-feedback.warning, +.value-feedback.stale { border-color: var(--warning-line); border-left-color: var(--warning); background: var(--warning-soft); color: var(--warning-active); } +.value-feedback.error { border-color: var(--danger-line); border-left-color: var(--danger); background: var(--danger-soft); color: var(--danger-active); } + +.value-page-state { + min-height: 320px; + display: grid; + place-content: center; + justify-items: center; + gap: 10px; + padding: 32px; + text-align: center; +} + +.value-page-state > i { color: var(--line-strong); font-size: 40px; } +.value-page-state.loading > i { color: var(--theme-primary); } +.value-page-state.error > i, +.value-page-state.permission > i { color: var(--danger); } +.value-page-state strong { color: var(--ink); font-size: 17px; } +.value-page-state span { max-width: 560px; color: var(--muted); font-size: 13px; line-height: 1.65; } +.value-page-state .btn { margin-top: 8px; } + +.value-kpi-grid { + display: grid; + grid-template-columns: repeat(3, minmax(0, 1fr)); + gap: 14px; +} + +.value-kpi-card { + min-width: 0; + min-height: 188px; + display: flex; + flex-direction: column; + padding: 17px; + border-top: 3px solid var(--theme-primary); +} + +.value-kpi-card.available { border-color: var(--success); } +.value-kpi-card.zero { border-color: var(--info); } +.value-kpi-card.baseline-missing { border-color: var(--warning); } +.value-kpi-card.unavailable { border-color: var(--line-strong); } + +.value-kpi-head { display: flex; align-items: center; justify-content: space-between; gap: 10px; } +.value-kpi-head span { min-width: 0; color: var(--muted); font-size: 12px; font-weight: 750; } +.value-kpi-head em { + min-height: 24px; + display: inline-flex; + align-items: center; + padding: 3px 8px; + border-radius: 999px; + background: var(--theme-primary-soft); + color: var(--theme-primary-active); + font-size: 12px; + font-style: normal; + font-weight: 850; + white-space: nowrap; +} + +.value-kpi-card.zero .value-kpi-head em { background: var(--info-soft); color: var(--info); } +.value-kpi-card.baseline-missing .value-kpi-head em { background: var(--warning-soft); color: var(--warning); } +.value-kpi-card.unavailable .value-kpi-head em { background: var(--info-soft); color: var(--info); } +.value-kpi-card > strong { margin-top: 20px; color: var(--ink); font-size: clamp(22px, 2vw, 30px); line-height: 1.1; font-variant-numeric: tabular-nums; overflow-wrap: anywhere; } +.value-kpi-card > p { margin: 8px 0 0; color: var(--muted); font-size: 12px; line-height: 1.5; } +.value-kpi-card details { margin-top: auto; padding-top: 14px; } +.value-kpi-card summary { min-height: 44px; display: inline-flex; align-items: center; color: var(--theme-primary-active); font-size: 12px; font-weight: 750; cursor: pointer; } +.value-kpi-card details p, +.value-kpi-card details ul { margin: 6px 0 0; padding-left: 18px; color: var(--muted); font-size: 12px; line-height: 1.6; } + +.value-dashboard-grid { + display: grid; + grid-template-columns: repeat(12, minmax(0, 1fr)); + gap: 16px; +} + +.value-dashboard-card { min-width: 0; padding: 18px; } +.value-funnel-card { grid-column: span 5; } +.value-trend-card { grid-column: span 7; } +.value-breakdown-card { grid-column: span 7; } +.value-guardrail-card { grid-column: span 5; } +.value-quality-card { grid-column: span 12; } +.value-card-head { margin-bottom: 16px; } +.value-compact-select { min-width: 96px; } +.value-compact-select select { min-height: 44px; } + +.value-funnel-list { display: grid; gap: 12px; margin: 0; padding: 0; list-style: none; } +.value-funnel-list li { display: grid; grid-template-columns: minmax(0, 1fr) 110px; gap: 6px 12px; } +.value-funnel-list li > div { display: flex; justify-content: space-between; gap: 12px; } +.value-funnel-list span, +.value-funnel-list strong, +.value-funnel-list small { font-size: 12px; } +.value-funnel-list span { color: var(--text); font-weight: 700; } +.value-funnel-list strong { color: var(--ink); } +.value-funnel-list small { grid-column: 2; grid-row: 1 / span 2; align-self: center; color: var(--muted); text-align: right; font-variant-numeric: tabular-nums; overflow-wrap: anywhere; } +.value-funnel-list i { height: 8px; overflow: hidden; border-radius: 999px; background: var(--info-soft); } +.value-funnel-list b { display: block; height: 100%; border-radius: inherit; background: var(--theme-gradient-primary); } +.value-realization-rates { display: grid; gap: 5px; margin-top: 16px; padding: 11px; border-radius: var(--radius); background: var(--surface-soft); } +.value-realization-rates span { color: var(--muted); font-size: 12px; } +.value-realization-rates strong { color: var(--ink); } + +.value-dimension-tabs { display: flex; gap: 7px; margin-bottom: 14px; overflow-x: auto; padding-bottom: 4px; } +.value-dimension-tabs button { + min-height: 44px; + padding: 0 12px; + border: 1px solid var(--line); + border-radius: var(--radius); + background: var(--surface); + color: var(--muted); + font-size: 12px; + font-weight: 750; + white-space: nowrap; +} +.value-dimension-tabs button.active { border-color: var(--theme-primary); background: var(--theme-primary-soft); color: var(--theme-primary-active); } +.value-breakdown-list { display: grid; gap: 12px; } +.value-breakdown-row { display: grid; grid-template-columns: minmax(100px, .9fr) minmax(120px, 1fr) minmax(130px, 1.1fr); align-items: center; gap: 12px; } +.value-breakdown-row div:last-child { text-align: right; } +.value-breakdown-row strong, +.value-breakdown-row span { display: block; overflow-wrap: anywhere; } +.value-breakdown-row strong { color: var(--ink); font-size: 12px; } +.value-breakdown-row span { margin-top: 3px; color: var(--muted); font-size: 12px; } +.value-breakdown-row > i { height: 8px; overflow: hidden; border-radius: 999px; background: var(--info-soft); } +.value-breakdown-row b { display: block; height: 100%; border-radius: inherit; background: var(--success); } + +.value-guardrail-list { display: grid; gap: 10px; } +.value-guardrail-row { display: grid; grid-template-columns: 32px minmax(0, 1fr) auto; align-items: center; gap: 10px; padding: 10px; border: 1px solid var(--line); border-radius: var(--radius); background: var(--surface-soft); } +.value-guardrail-row > i { width: 32px; height: 32px; display: grid; place-items: center; border-radius: var(--radius); background: var(--success-soft); color: var(--success); font-size: 17px; } +.value-guardrail-row.attention > i { background: var(--warning-soft); color: var(--warning); } +.value-guardrail-row.unavailable > i { background: var(--info-soft); color: var(--info); } +.value-guardrail-row strong, +.value-guardrail-row span { display: block; } +.value-guardrail-row strong { color: var(--ink); font-size: 12px; } +.value-guardrail-row span { margin-top: 3px; color: var(--muted); font-size: 12px; line-height: 1.45; } +.value-guardrail-row em { color: var(--muted); font-size: 12px; font-style: normal; font-weight: 800; white-space: nowrap; } + +.value-quality-grid { display: grid; grid-template-columns: repeat(6, minmax(0, 1fr)); gap: 10px; } +.value-quality-grid > div { min-height: 76px; display: grid; place-content: center; justify-items: center; gap: 4px; padding: 9px; border: 1px solid var(--success-line); border-radius: var(--radius); background: var(--success-soft); text-align: center; } +.value-quality-grid > div.attention { border-color: var(--warning-line); background: var(--warning-soft); } +.value-quality-grid > div.unavailable { border-color: var(--info-line); background: var(--info-soft); } +.value-quality-grid strong { color: var(--ink); font-size: 20px; font-variant-numeric: tabular-nums; } +.value-quality-grid span { color: var(--muted); font-size: 12px; line-height: 1.4; } +.value-source-chip { min-height: 28px; display: inline-flex; align-items: center; padding: 3px 8px; border-radius: 999px; background: var(--info-soft); color: var(--info); font-size: 12px; font-weight: 800; white-space: nowrap; } +.value-coverage-notes { margin: 14px 0 0; padding: 12px 12px 12px 28px; border-top: 1px solid var(--line); color: var(--muted); font-size: 12px; line-height: 1.6; } +.value-inline-state { min-height: 160px; display: grid; place-content: center; color: var(--muted); font-size: 12px; text-align: center; } + +.value-opportunity-panel { min-width: 0; padding: 18px; } +.value-list-loading { color: var(--theme-primary-active); font-size: 12px; } +.value-opportunity-list { display: grid; border-top: 1px solid var(--line); } +.value-opportunity-row { min-width: 0; display: grid; grid-template-columns: minmax(220px, 1.5fr) minmax(150px, .9fr) minmax(130px, .8fr) auto; align-items: center; gap: 14px; padding: 14px 0; border-bottom: 1px solid var(--line); } +.value-opportunity-main { min-width: 0; display: flex; align-items: center; gap: 10px; } +.value-status-pill { min-height: 24px; display: inline-flex; align-items: center; flex: 0 0 auto; padding: 3px 8px; border-radius: 999px; background: var(--info-soft); color: var(--info); font-size: 12px; font-weight: 800; } +.value-status-pill.success { background: var(--success-soft); color: var(--success); } +.value-status-pill.warning { background: var(--warning-soft); color: var(--warning); } +.value-status-pill.danger { background: var(--danger-soft); color: var(--danger); } +.value-status-pill.primary { background: var(--theme-primary-soft); color: var(--theme-primary-active); } +.value-opportunity-main div, +.value-opportunity-amount, +.value-opportunity-owner { min-width: 0; } +.value-opportunity-main strong, +.value-opportunity-main small, +.value-opportunity-amount strong, +.value-opportunity-amount span, +.value-opportunity-owner strong, +.value-opportunity-owner span { display: block; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.value-opportunity-main strong, +.value-opportunity-amount strong, +.value-opportunity-owner strong { color: var(--ink); font-size: 12px; } +.value-opportunity-main small, +.value-opportunity-amount span, +.value-opportunity-owner span { margin-top: 4px; color: var(--muted); font-size: 12px; } +.value-opportunity-amount strong { font-size: 14px; font-variant-numeric: tabular-nums; } +.value-opportunity-row > .btn { min-width: 104px; min-height: 44px; font-size: 12px; } +.value-list-state { min-height: 160px; display: flex; align-items: center; justify-content: center; gap: 12px; color: var(--muted); font-size: 12px; text-align: center; } +.value-list-state.error { color: var(--danger-active); } +.value-pagination { justify-content: flex-end; margin-top: 16px; } +.value-pagination :deep(.el-pager li), +.value-pagination :deep(.btn-prev), +.value-pagination :deep(.btn-next) { min-width: 44px; height: 44px; } + +@media (max-width: 1120px) { + .value-filter-primary { grid-template-columns: repeat(2, minmax(0, 1fr)); } + .value-filter-advanced { grid-template-columns: repeat(3, minmax(0, 1fr)); } + .value-funnel-card, + .value-trend-card, + .value-breakdown-card, + .value-guardrail-card { grid-column: span 12; } + .value-quality-grid { grid-template-columns: repeat(3, minmax(0, 1fr)); } + .value-opportunity-row { grid-template-columns: minmax(220px, 1.4fr) minmax(150px, 1fr) auto; } + .value-opportunity-owner { display: none; } +} + +@media (max-width: 760px) { + .value-dashboard-header, + .value-filter-head, + .value-card-head { flex-direction: column; align-items: stretch; } + .value-source-summary { width: 100%; grid-template-columns: 1fr auto; justify-content: stretch; } + .value-source-summary small { text-align: left; } + .value-kpi-grid { grid-template-columns: 1fr; } + .value-filter-primary, + .value-filter-advanced { grid-template-columns: 1fr; } + .value-filter-actions { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); } + .value-filter-actions .btn { width: 100%; min-width: 0; } + .value-quality-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } + .value-opportunity-row { grid-template-columns: 1fr; align-items: stretch; gap: 10px; padding: 16px 0; } + .value-opportunity-owner { display: block; } + .value-opportunity-main { align-items: flex-start; } + .value-opportunity-main strong, + .value-opportunity-main small, + .value-opportunity-amount strong, + .value-opportunity-amount span, + .value-opportunity-owner strong, + .value-opportunity-owner span { white-space: normal; } + .value-opportunity-row > .btn { width: 100%; } + .value-breakdown-row { grid-template-columns: 1fr; gap: 7px; padding-bottom: 12px; border-bottom: 1px solid var(--line); } + .value-breakdown-row div:last-child { text-align: left; } + .value-funnel-list li { grid-template-columns: minmax(0, 1fr) 100px; } +} + +@media (max-width: 420px) { + .value-dashboard-header, + .value-filter-panel, + .value-dashboard-card, + .value-opportunity-panel { padding: 14px; } + .value-quality-grid { grid-template-columns: 1fr; } + .value-guardrail-row { grid-template-columns: 32px minmax(0, 1fr); } + .value-guardrail-row em { grid-column: 2; } +} diff --git a/web/src/assets/styles/components/travel-request-application-facts.css b/web/src/assets/styles/components/travel-request-application-facts.css new file mode 100644 index 0000000..cac23ac --- /dev/null +++ b/web/src/assets/styles/components/travel-request-application-facts.css @@ -0,0 +1,149 @@ +.application-detail-facts { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + margin-top: 4px; + overflow: hidden; + border: 1px solid #e2e8f0; + border-radius: 4px; + background: #fff; +} + +.application-detail-fact { + display: grid; + grid-template-columns: minmax(96px, 28%) minmax(0, 1fr); + min-height: 48px; + border-top: 1px solid #edf2f7; + border-left: 1px solid #edf2f7; +} + +.application-detail-fact:nth-child(-n + 2) { + border-top: 0; +} + +.application-detail-fact:nth-child(2n + 1) { + border-left: 0; +} + +.application-detail-fact > span, +.application-detail-fact > strong { + display: flex; + align-items: center; + min-width: 0; + padding: 11px 14px; + line-height: 1.5; +} + +.application-detail-fact > span { + background: #f8fafc; + color: #64748b; + font-size: 12px; + font-weight: 800; +} + +.application-detail-fact strong { + border-left: 1px solid #edf2f7; + color: #0f172a; + font-size: 13px; + font-weight: 750; + gap: 8px; + overflow-wrap: anywhere; +} + +.application-detail-fact.highlight > span { + background: var(--theme-primary-soft); + color: var(--theme-primary-active); +} + +.application-detail-fact.highlight strong { + background: color-mix(in srgb, var(--theme-primary-soft) 55%, #ffffff); +} + +.application-detail-fact.emphasis strong { + color: var(--theme-primary-active); + font-weight: 850; +} + +.application-detail-fact-value { + min-width: 0; + flex: 1 1 auto; + overflow-wrap: anywhere; +} + +.application-detail-edit-btn, +.application-detail-edit-confirm, +.application-detail-edit-cancel { + flex: 0 0 auto; + width: 24px; + height: 24px; + display: inline-grid; + place-items: center; + border: 1px solid transparent; + border-radius: 4px; + background: transparent; + color: #64748b; + cursor: pointer; +} + +.application-detail-edit-btn { + opacity: 0; + transition: + opacity 0.16s ease, + background 0.16s ease, + color 0.16s ease; +} + +.application-detail-fact.editable:hover .application-detail-edit-btn, +.application-detail-edit-btn:focus-visible { + opacity: 1; +} + +.application-detail-edit-btn:hover:not(:disabled), +.application-detail-edit-btn:focus-visible, +.application-detail-edit-confirm:hover:not(:disabled), +.application-detail-edit-cancel:hover:not(:disabled) { + background: rgba(var(--theme-primary-rgb, 58, 124, 165), 0.1); + color: var(--theme-primary-active); +} + +.application-detail-edit-confirm { + background: rgba(22, 163, 74, 0.1); + color: #15803d; +} + +.application-detail-edit-cancel { + background: #f1f5f9; +} + +.application-detail-edit-btn:disabled, +.application-detail-edit-confirm:disabled, +.application-detail-edit-cancel:disabled { + cursor: not-allowed; + opacity: 0.45; +} + +.application-detail-fact.editing strong { + align-items: center; +} + +.application-detail-editor-control { + flex: 1 1 auto; + min-width: 0; +} + +.application-detail-editor-select { + width: 100%; +} + +@media (max-width: 760px) { + .application-detail-facts { + grid-template-columns: 1fr; + } + + .application-detail-fact { + border-left: 0; + } + + .application-detail-fact:nth-child(2) { + border-top: 1px solid #edf2f7; + } +} diff --git a/web/src/assets/styles/views/budget-center-view.css b/web/src/assets/styles/views/budget-center-view.css index edc9256..458cb8d 100644 --- a/web/src/assets/styles/views/budget-center-view.css +++ b/web/src/assets/styles/views/budget-center-view.css @@ -17,6 +17,54 @@ overflow: hidden; } +.budget-list-head { + display: grid; + gap: 8px; +} + +.budget-list-head .budget-configuration-focus { + margin-bottom: 0; +} + +.budget-configuration-focus { + min-height: 48px; + display: flex; + align-items: flex-start; + gap: 10px; + margin-bottom: 10px; + padding: 9px 12px; + border: 1px solid rgba(var(--theme-primary-rgb), .24); + border-radius: 4px; + background: var(--theme-primary-soft); + color: var(--theme-primary-active); +} + +.budget-configuration-focus > i { + margin-top: 1px; + font-size: 18px; +} + +.budget-configuration-focus strong, +.budget-configuration-focus span { + display: block; +} + +.budget-configuration-focus strong { + font-size: 13px; +} + +.budget-configuration-focus span { + margin-top: 3px; + color: #475569; + font-size: 12px; + line-height: 1.45; +} + +.budget-detail-table tr.is-source-focus td { + background: var(--warning-soft); + box-shadow: inset 0 1px 0 rgba(245, 158, 11, .22), inset 0 -1px 0 rgba(245, 158, 11, .22); +} + .budget-scope-tabs small { min-width: 22px; height: 18px; diff --git a/web/src/assets/styles/views/travel-request-detail-view-part2.css b/web/src/assets/styles/views/travel-request-detail-view-part2.css index a691b1b..595c080 100644 --- a/web/src/assets/styles/views/travel-request-detail-view-part2.css +++ b/web/src/assets/styles/views/travel-request-detail-view-part2.css @@ -835,18 +835,6 @@ white-space: normal; } - .application-detail-facts { - grid-template-columns: 1fr; - } - - .application-detail-fact { - border-left: 0; - } - - .application-detail-fact:nth-child(2) { - border-top: 1px solid #edf2f7; - } - .detail-card { padding: 14px 16px; } diff --git a/web/src/assets/styles/views/travel-request-detail-view.css b/web/src/assets/styles/views/travel-request-detail-view.css index b09db59..8c729e6 100644 --- a/web/src/assets/styles/views/travel-request-detail-view.css +++ b/web/src/assets/styles/views/travel-request-detail-view.css @@ -590,142 +590,6 @@ white-space: pre-wrap; } -.application-detail-facts { - display: grid; - grid-template-columns: repeat(2, minmax(0, 1fr)); - margin-top: 4px; - overflow: hidden; - border: 1px solid #e2e8f0; - border-radius: 4px; - background: #fff; -} - -.application-detail-fact { - display: grid; - grid-template-columns: minmax(96px, 28%) minmax(0, 1fr); - min-height: 48px; - border-top: 1px solid #edf2f7; - border-left: 1px solid #edf2f7; -} - -.application-detail-fact:nth-child(-n + 2) { - border-top: 0; -} - -.application-detail-fact:nth-child(2n + 1) { - border-left: 0; -} - -.application-detail-fact > span, -.application-detail-fact > strong { - display: flex; - align-items: center; - min-width: 0; - padding: 11px 14px; - line-height: 1.5; -} - -.application-detail-fact > span { - background: #f8fafc; - color: #64748b; - font-size: 12px; - font-weight: 800; -} - -.application-detail-fact strong { - border-left: 1px solid #edf2f7; - color: #0f172a; - font-size: 13px; - font-weight: 750; - gap: 8px; - overflow-wrap: anywhere; -} - -.application-detail-fact.highlight > span { - background: var(--theme-primary-soft); - color: var(--theme-primary-active); -} - -.application-detail-fact.highlight strong { - background: color-mix(in srgb, var(--theme-primary-soft) 55%, #ffffff); -} - -.application-detail-fact.emphasis strong { - color: var(--theme-primary-active); - font-weight: 850; -} - -.application-detail-fact-value { - min-width: 0; - flex: 1 1 auto; - overflow-wrap: anywhere; -} - -.application-detail-edit-btn, -.application-detail-edit-confirm, -.application-detail-edit-cancel { - flex: 0 0 auto; - width: 24px; - height: 24px; - display: inline-grid; - place-items: center; - border: 1px solid transparent; - border-radius: 4px; - background: transparent; - color: #64748b; - cursor: pointer; -} - -.application-detail-edit-btn { - opacity: 0; - transition: - opacity 0.16s ease, - background 0.16s ease, - color 0.16s ease; -} - -.application-detail-fact.editable:hover .application-detail-edit-btn, -.application-detail-edit-btn:focus-visible { - opacity: 1; -} - -.application-detail-edit-btn:hover:not(:disabled), -.application-detail-edit-btn:focus-visible, -.application-detail-edit-confirm:hover:not(:disabled), -.application-detail-edit-cancel:hover:not(:disabled) { - background: rgba(var(--theme-primary-rgb, 58, 124, 165), 0.1); - color: var(--theme-primary-active); -} - -.application-detail-edit-confirm { - background: rgba(22, 163, 74, 0.1); - color: #15803d; -} - -.application-detail-edit-cancel { - background: #f1f5f9; -} - -.application-detail-edit-btn:disabled, -.application-detail-edit-confirm:disabled, -.application-detail-edit-cancel:disabled { - cursor: not-allowed; - opacity: 0.45; -} - -.application-detail-fact.editing strong { - align-items: center; -} - -.application-detail-editor-control { - flex: 1 1 auto; - min-width: 0; -} - -.application-detail-editor-select { - width: 100%; -} - .related-application-facts { margin-top: 0; } diff --git a/web/src/components/audit/AuditJsonRiskRuleDetail.vue b/web/src/components/audit/AuditJsonRiskRuleDetail.vue index f4d4e8e..8639ae7 100644 --- a/web/src/components/audit/AuditJsonRiskRuleDetail.vue +++ b/web/src/components/audit/AuditJsonRiskRuleDetail.vue @@ -110,6 +110,17 @@

+ +
@@ -132,6 +143,7 @@ diff --git a/web/src/components/audit/AuditReleaseMonitorPanel.vue b/web/src/components/audit/AuditReleaseMonitorPanel.vue new file mode 100644 index 0000000..4144e08 --- /dev/null +++ b/web/src/components/audit/AuditReleaseMonitorPanel.vue @@ -0,0 +1,206 @@ + + + + + diff --git a/web/src/components/charts/CfoValueTrendChart.vue b/web/src/components/charts/CfoValueTrendChart.vue new file mode 100644 index 0000000..f30b225 --- /dev/null +++ b/web/src/components/charts/CfoValueTrendChart.vue @@ -0,0 +1,158 @@ + + + + + diff --git a/web/src/components/commercial/CommercialAccountOverview.vue b/web/src/components/commercial/CommercialAccountOverview.vue new file mode 100644 index 0000000..643bd4e --- /dev/null +++ b/web/src/components/commercial/CommercialAccountOverview.vue @@ -0,0 +1,122 @@ + + + diff --git a/web/src/components/commercial/CommercialAdminConsole.vue b/web/src/components/commercial/CommercialAdminConsole.vue new file mode 100644 index 0000000..b7927e7 --- /dev/null +++ b/web/src/components/commercial/CommercialAdminConsole.vue @@ -0,0 +1,105 @@ + + + diff --git a/web/src/components/commercial/CommercialHistoryPanel.vue b/web/src/components/commercial/CommercialHistoryPanel.vue new file mode 100644 index 0000000..59ccec6 --- /dev/null +++ b/web/src/components/commercial/CommercialHistoryPanel.vue @@ -0,0 +1,127 @@ + + + diff --git a/web/src/components/commercial/CommercialMetricCard.vue b/web/src/components/commercial/CommercialMetricCard.vue new file mode 100644 index 0000000..958948f --- /dev/null +++ b/web/src/components/commercial/CommercialMetricCard.vue @@ -0,0 +1,56 @@ + + + diff --git a/web/src/components/commercial/CommercialMutationDialog.vue b/web/src/components/commercial/CommercialMutationDialog.vue new file mode 100644 index 0000000..0080bb7 --- /dev/null +++ b/web/src/components/commercial/CommercialMutationDialog.vue @@ -0,0 +1,245 @@ + + + diff --git a/web/src/components/commercial/CommercialPricingScenarioPanel.vue b/web/src/components/commercial/CommercialPricingScenarioPanel.vue new file mode 100644 index 0000000..cdeac02 --- /dev/null +++ b/web/src/components/commercial/CommercialPricingScenarioPanel.vue @@ -0,0 +1,278 @@ + + + diff --git a/web/src/components/commercial/CommercialValueProofPanel.vue b/web/src/components/commercial/CommercialValueProofPanel.vue new file mode 100644 index 0000000..91243b9 --- /dev/null +++ b/web/src/components/commercial/CommercialValueProofPanel.vue @@ -0,0 +1,102 @@ + + + diff --git a/web/src/components/commercial/CommercialWorkspace.vue b/web/src/components/commercial/CommercialWorkspace.vue new file mode 100644 index 0000000..8fce13d --- /dev/null +++ b/web/src/components/commercial/CommercialWorkspace.vue @@ -0,0 +1,179 @@ + + + diff --git a/web/src/components/commercial/commercial-workspace.css b/web/src/components/commercial/commercial-workspace.css new file mode 100644 index 0000000..49e4e23 --- /dev/null +++ b/web/src/components/commercial/commercial-workspace.css @@ -0,0 +1,679 @@ +.commercial-workspace { + --commercial-ink: #0f172a; + --commercial-copy: #475569; + --commercial-muted: #64748b; + --commercial-line: #dbe5ef; + --commercial-surface: rgba(255, 255, 255, 0.96); + --commercial-primary: var(--theme-primary, #2563eb); + --commercial-primary-deep: var(--theme-primary-active, #1d4ed8); + --commercial-primary-soft: var(--theme-primary-soft, #eff6ff); + --commercial-success: #047857; + --commercial-success-soft: #ecfdf5; + --commercial-warning: #b45309; + --commercial-warning-soft: #fffbeb; + --commercial-danger: #b91c1c; + --commercial-danger-soft: #fef2f2; + display: grid; + gap: 20px; + color: var(--commercial-ink); +} + +.commercial-workspace *, +.commercial-workspace *::before, +.commercial-workspace *::after, +.commercial-dialog *, +.commercial-dialog *::before, +.commercial-dialog *::after { + box-sizing: border-box; +} + +.commercial-hero, +.commercial-section, +.commercial-empty-workspace, +.commercial-safety-banner { + border: 1px solid var(--commercial-line); + border-radius: 12px; + background: var(--commercial-surface); + box-shadow: 0 12px 30px rgba(15, 23, 42, 0.06); +} + +.commercial-hero { + display: grid; + grid-template-columns: minmax(0, 1fr) minmax(300px, 390px); + gap: 24px; + padding: 28px; + background: + radial-gradient(circle at 88% 10%, rgba(var(--theme-primary-rgb, 37, 99, 235), 0.14), transparent 32%), + linear-gradient(135deg, #ffffff, #f8fbff); +} + +.commercial-eyebrow, +.commercial-section-head > div > span, +.commercial-ledger > header span, +.commercial-dialog > header span { + display: block; + margin-bottom: 7px; + color: var(--commercial-primary-deep); + font-size: 12px; + font-weight: 800; + letter-spacing: 0.08em; + text-transform: uppercase; +} + +.commercial-hero h2, +.commercial-section h3, +.commercial-ledger h4, +.commercial-dialog h3 { + margin: 0; + color: var(--commercial-ink); + line-height: 1.25; +} + +.commercial-hero h2 { font-size: clamp(24px, 3vw, 34px); } +.commercial-section h3 { font-size: 21px; } +.commercial-hero p, +.commercial-section-head p, +.commercial-ledger > header p, +.commercial-dialog > header p { + margin: 9px 0 0; + color: var(--commercial-copy); + font-size: 14px; + line-height: 1.7; +} + +.commercial-trust-row { + display: flex; + flex-wrap: wrap; + gap: 8px; + margin-top: 18px; +} + +.commercial-trust-row span { + min-height: 32px; + display: inline-flex; + align-items: center; + gap: 6px; + padding: 0 10px; + border: 1px solid rgba(var(--theme-primary-rgb, 37, 99, 235), 0.18); + border-radius: 999px; + background: rgba(255, 255, 255, 0.88); + color: var(--commercial-primary-deep); + font-size: 12px; + font-weight: 750; +} + +.commercial-tenant-picker, +.commercial-readonly-badge { + align-self: center; + padding: 18px; + border: 1px solid rgba(var(--theme-primary-rgb, 37, 99, 235), 0.18); + border-radius: 10px; + background: rgba(255, 255, 255, 0.9); +} + +.commercial-tenant-picker > label { + display: block; + margin-bottom: 8px; + color: var(--commercial-ink); + font-size: 13px; + font-weight: 800; +} + +.commercial-tenant-picker > div { display: flex; gap: 8px; } +.commercial-tenant-picker input { min-width: 0; flex: 1; } +.commercial-tenant-picker small, +.commercial-form-grid label small { + display: block; + margin-top: 6px; + color: var(--commercial-muted); + font-size: 12px; + line-height: 1.45; +} + +.commercial-readonly-badge { + display: flex; + align-items: center; + gap: 12px; +} + +.commercial-readonly-badge > i { color: var(--commercial-primary); font-size: 30px; } +.commercial-readonly-badge strong, +.commercial-readonly-badge span { display: block; } +.commercial-readonly-badge span { margin-top: 3px; color: var(--commercial-muted); font-size: 12px; } + +.commercial-section { padding: 24px; } +.commercial-section-head { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 20px; + margin-bottom: 20px; +} + +.commercial-section-head > div { max-width: 760px; } +.commercial-button, +.commercial-icon-button, +.commercial-action-button { + font: inherit; + cursor: pointer; + touch-action: manipulation; + transition: background 180ms ease, border-color 180ms ease, color 180ms ease, box-shadow 180ms ease; +} + +.commercial-button { + min-height: 44px; + display: inline-flex; + align-items: center; + justify-content: center; + gap: 7px; + padding: 0 16px; + border: 1px solid transparent; + border-radius: 7px; + font-size: 14px; + font-weight: 800; +} + +.commercial-button.primary { background: var(--commercial-primary); color: #fff; } +.commercial-button.primary:hover:not(:disabled) { background: var(--commercial-primary-deep); } +.commercial-button.secondary { border-color: var(--commercial-line); background: #fff; color: var(--commercial-ink); } +.commercial-button.secondary:hover:not(:disabled), +.commercial-button.ghost:hover:not(:disabled) { border-color: var(--commercial-primary); color: var(--commercial-primary-deep); } +.commercial-button.ghost { border-color: transparent; background: transparent; color: var(--commercial-copy); } +.commercial-button:disabled, +.commercial-icon-button:disabled, +.commercial-action-button:disabled { cursor: not-allowed; opacity: 0.48; } + +.commercial-workspace button:focus-visible, +.commercial-workspace input:focus-visible, +.commercial-workspace select:focus-visible, +.commercial-workspace textarea:focus-visible, +.commercial-dialog button:focus-visible, +.commercial-dialog input:focus-visible, +.commercial-dialog select:focus-visible, +.commercial-dialog textarea:focus-visible, +.commercial-metric-card summary:focus-visible, +.commercial-pricing-notes summary:focus-visible, +.commercial-pricing-confirmation:focus-visible, +.commercial-form-error:focus-visible, +.commercial-operation-feedback:focus-visible { + outline: 3px solid rgba(var(--theme-primary-rgb, 37, 99, 235), 0.28); + outline-offset: 2px; +} + +.commercial-workspace input, +.commercial-workspace select, +.commercial-dialog input, +.commercial-dialog select, +.commercial-dialog textarea { + width: 100%; + min-height: 44px; + padding: 9px 11px; + border: 1px solid #cbd5e1; + border-radius: 7px; + background: #fff; + color: var(--commercial-ink); + font: inherit; + font-size: 14px; +} + +.commercial-dialog textarea { min-height: 90px; resize: vertical; line-height: 1.55; } +.commercial-empty-workspace { + min-height: 280px; + display: flex; + flex-direction: column; + align-items: center; + justify-content: center; + gap: 9px; + padding: 40px; + text-align: center; +} + +.commercial-empty-workspace i { color: var(--commercial-primary); font-size: 48px; } +.commercial-empty-workspace strong { font-size: 20px; } +.commercial-empty-workspace span { max-width: 560px; color: var(--commercial-copy); line-height: 1.6; } +.commercial-safety-banner, +.commercial-state-card { + display: flex; + align-items: center; + gap: 13px; + padding: 16px 18px; +} + +.commercial-safety-banner { border-color: rgba(180, 83, 9, 0.24); background: var(--commercial-warning-soft); box-shadow: none; } +.commercial-safety-banner > i, +.commercial-state-card > i { flex: 0 0 auto; color: var(--commercial-warning); font-size: 26px; } +.commercial-safety-banner strong, +.commercial-safety-banner span, +.commercial-state-card strong, +.commercial-state-card span { display: block; } +.commercial-safety-banner span, +.commercial-state-card span { margin-top: 3px; color: var(--commercial-copy); font-size: 13px; line-height: 1.55; } + +.commercial-state-card { + min-height: 96px; + border: 1px dashed var(--commercial-line); + border-radius: 9px; + background: #f8fafc; +} + +.commercial-state-card > div { flex: 1; } +.commercial-state-card.danger { border-color: rgba(185, 28, 28, 0.25); background: var(--commercial-danger-soft); } +.commercial-state-card.danger > i { color: var(--commercial-danger); } +.commercial-state-card.warning { border-color: rgba(180, 83, 9, 0.25); background: var(--commercial-warning-soft); } +.commercial-state-card.permission { border-color: rgba(var(--theme-primary-rgb, 37, 99, 235), 0.23); background: var(--commercial-primary-soft); } +.commercial-state-card.permission > i { color: var(--commercial-primary); } + +.commercial-inline-warning, +.commercial-operation-feedback, +.commercial-quality-banner { + display: flex; + align-items: flex-start; + gap: 9px; + margin-bottom: 16px; + padding: 12px 14px; + border: 1px solid rgba(180, 83, 9, 0.24); + border-radius: 8px; + background: var(--commercial-warning-soft); + color: var(--commercial-warning); + font-size: 13px; + line-height: 1.55; +} + +.commercial-inline-warning.danger, +.commercial-operation-feedback.danger { border-color: rgba(185, 28, 28, 0.22); background: var(--commercial-danger-soft); color: var(--commercial-danger); } +.commercial-operation-feedback.success { border-color: rgba(4, 120, 87, 0.2); background: var(--commercial-success-soft); color: var(--commercial-success); } + +.commercial-account-grid { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: 14px; +} + +.commercial-account-card, +.commercial-quota-panel, +.commercial-ledger, +.commercial-admin-groups > article { + border: 1px solid var(--commercial-line); + border-radius: 10px; + background: #fff; +} + +.commercial-account-card { padding: 18px; } +.commercial-account-card > span { color: var(--commercial-muted); font-size: 12px; font-weight: 750; } +.commercial-card-title { display: flex; align-items: center; justify-content: space-between; gap: 10px; margin: 8px 0 14px; } +.commercial-card-title strong { font-size: 18px; } +.commercial-card-title em, +.commercial-admin-access { + min-height: 28px; + display: inline-flex; + align-items: center; + gap: 5px; + padding: 0 9px; + border-radius: 999px; + background: var(--commercial-primary-soft); + color: var(--commercial-primary-deep); + font-size: 12px; + font-style: normal; + font-weight: 800; +} + +.commercial-account-card dl { display: grid; gap: 8px; margin: 0; } +.commercial-account-card dl div { display: flex; justify-content: space-between; gap: 12px; padding-top: 8px; border-top: 1px solid #eef2f7; } +.commercial-account-card dt { color: var(--commercial-muted); font-size: 13px; } +.commercial-account-card dd { margin: 0; text-align: right; font-size: 13px; font-weight: 750; font-variant-numeric: tabular-nums; } + +.commercial-quota-panel { margin-top: 14px; overflow: hidden; } +.commercial-quota-panel > header { display: flex; justify-content: space-between; gap: 12px; padding: 16px 18px; border-bottom: 1px solid var(--commercial-line); } +.commercial-quota-panel h4 { margin: 0; font-size: 16px; } +.commercial-quota-panel p { margin: 4px 0 0; color: var(--commercial-muted); font-size: 12px; } +.commercial-quota-panel > header > span { align-self: center; color: var(--commercial-primary-deep); font-size: 12px; font-weight: 800; } +.commercial-table-wrap { overflow-x: auto; } +.commercial-table-wrap table { width: 100%; min-width: 820px; border-collapse: collapse; } +.commercial-table-wrap th, +.commercial-table-wrap td { padding: 12px 14px; border-bottom: 1px solid #eef2f7; text-align: left; font-size: 13px; vertical-align: middle; } +.commercial-table-wrap th { background: #f8fafc; color: var(--commercial-muted); font-size: 12px; font-weight: 800; } +.commercial-table-wrap td small { display: block; margin-top: 3px; color: var(--commercial-muted); } +.commercial-table-wrap td.numeric { font-variant-numeric: tabular-nums; } +.commercial-status-pill { display: inline-flex; padding: 4px 8px; border-radius: 999px; background: var(--commercial-primary-soft); color: var(--commercial-primary-deep); font-size: 12px; font-weight: 800; white-space: nowrap; } +.commercial-status-pill.approaching { background: var(--commercial-warning-soft); color: var(--commercial-warning); } +.commercial-status-pill.exhausted, +.commercial-status-pill.inactive { background: var(--commercial-danger-soft); color: var(--commercial-danger); } +.commercial-status-pill.unlimited { background: var(--commercial-success-soft); color: var(--commercial-success); } +.commercial-status-pill.complete, +.commercial-status-pill.feasible { background: var(--commercial-success-soft); color: var(--commercial-success); } +.commercial-status-pill.partial, +.commercial-status-pill.cost_only, +.commercial-status-pill.insufficient_value { background: var(--commercial-warning-soft); color: var(--commercial-warning); } +.commercial-status-pill.unavailable { background: var(--commercial-danger-soft); color: var(--commercial-danger); } +.commercial-table-empty { min-height: 110px; display: flex; align-items: center; justify-content: center; gap: 9px; padding: 20px; color: var(--commercial-muted); font-size: 13px; text-align: center; } +.commercial-table-empty i { font-size: 24px; } + +.commercial-range-form { display: flex; align-items: end; gap: 8px; flex-wrap: wrap; } +.commercial-range-form label { display: grid; gap: 5px; color: var(--commercial-muted); font-size: 12px; font-weight: 750; } +.commercial-range-form input { min-width: 146px; } +.commercial-quality-banner { align-items: center; border-color: var(--commercial-line); background: #f8fafc; color: var(--commercial-copy); } +.commercial-quality-banner.partial { border-color: rgba(180, 83, 9, 0.22); background: var(--commercial-warning-soft); } +.commercial-quality-banner.unavailable { border-color: rgba(185, 28, 28, 0.22); background: var(--commercial-danger-soft); } +.commercial-quality-banner strong, +.commercial-quality-banner span { display: block; } +.commercial-quality-banner span { margin-top: 3px; font-size: 12px; line-height: 1.5; } + +.commercial-ledger { margin-top: 16px; padding: 18px; } +.commercial-ledger > header { display: flex; align-items: center; justify-content: space-between; gap: 14px; margin-bottom: 14px; } +.commercial-ledger > header > i { color: var(--commercial-primary); font-size: 34px; } +.commercial-ledger.platform > header > i { color: var(--commercial-success); } +.commercial-ledger h4 { font-size: 18px; } +.commercial-metric-grid { display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 10px; } +.commercial-metric-grid.platform-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } +.commercial-metric-card { min-width: 0; padding: 16px; border: 1px solid var(--commercial-line); border-radius: 9px; background: #fff; } +.commercial-metric-card.partial { border-color: rgba(180, 83, 9, 0.24); } +.commercial-metric-card.unavailable { background: #f8fafc; } +.commercial-metric-card > header { display: flex; justify-content: space-between; gap: 8px; } +.commercial-metric-card > header span { color: var(--commercial-muted); font-size: 10px; font-weight: 800; text-transform: uppercase; } +.commercial-metric-card h4 { margin: 4px 0 0; font-size: 14px; } +.commercial-metric-card > header em { height: fit-content; padding: 3px 7px; border-radius: 999px; background: var(--commercial-primary-soft); color: var(--commercial-primary-deep); font-size: 10px; font-style: normal; font-weight: 800; white-space: nowrap; } +.commercial-metric-card.partial > header em { background: var(--commercial-warning-soft); color: var(--commercial-warning); } +.commercial-metric-card.unavailable > header em { background: #e2e8f0; color: #475569; } +.commercial-metric-values { display: grid; gap: 8px; margin-top: 15px; } +.commercial-metric-values > div { padding-bottom: 8px; border-bottom: 1px solid #eef2f7; } +.commercial-metric-values strong, +.commercial-metric-values small { display: block; } +.commercial-metric-values strong, +.commercial-unavailable-value { color: var(--commercial-ink); font-size: 20px; font-variant-numeric: tabular-nums; } +.commercial-metric-values small { margin-top: 3px; color: var(--commercial-muted); font-size: 10px; line-height: 1.4; } +.commercial-unavailable-value { display: block; margin-top: 15px; color: #64748b; } +.commercial-metric-card > p { margin: 10px 0 0; color: var(--commercial-copy); font-size: 12px; line-height: 1.55; } +.commercial-metric-card .commercial-currency-guard { color: var(--commercial-primary-deep); } +.commercial-metric-card details { margin-top: 10px; color: var(--commercial-copy); font-size: 12px; } +.commercial-metric-card summary { min-height: 32px; display: flex; align-items: center; cursor: pointer; color: var(--commercial-primary-deep); font-weight: 750; } +.commercial-metric-card ul { margin: 7px 0 0; padding-left: 18px; line-height: 1.55; } + +.commercial-pricing-access { + min-height: 32px; + display: inline-flex; + align-items: center; + gap: 6px; + padding: 0 10px; + border-radius: 999px; + font-size: 12px; + font-weight: 800; + white-space: nowrap; +} + +.commercial-pricing-access.allowed { background: var(--commercial-success-soft); color: var(--commercial-success); } +.commercial-pricing-access.denied { background: #f1f5f9; color: var(--commercial-muted); } +.commercial-operation-feedback.pricing-progress { border-color: var(--commercial-line); background: #f8fafc; color: var(--commercial-primary-deep); } +.commercial-operation-feedback > span strong { display: block; margin-bottom: 2px; } +.commercial-pricing-form { + padding: 18px; + border: 1px solid var(--commercial-line); + border-radius: 10px; + background: #fff; +} + +.commercial-pricing-form fieldset { min-width: 0; margin: 0; padding: 0; border: 0; } +.commercial-pricing-form legend { margin-bottom: 14px; color: var(--commercial-ink); font-size: 15px; font-weight: 800; } +.commercial-pricing-fields { display: grid; grid-template-columns: repeat(5, minmax(0, 1fr)); gap: 12px; } +.commercial-pricing-fields label { min-width: 0; color: #334155; font-size: 13px; font-weight: 750; } +.commercial-pricing-fields label > span { display: block; margin-bottom: 6px; } +.commercial-pricing-fields label small { display: block; margin-top: 6px; color: var(--commercial-muted); font-size: 11px; font-weight: 500; line-height: 1.45; } +.commercial-percent-input { position: relative; } +.commercial-percent-input input { padding-right: 36px; font-variant-numeric: tabular-nums; } +.commercial-percent-input > span { position: absolute; top: 50%; right: 12px; color: var(--commercial-muted); font-size: 13px; transform: translateY(-50%); } +.commercial-pricing-form-actions { display: flex; align-items: center; justify-content: space-between; gap: 16px; margin-top: 18px; } +.commercial-pricing-form-actions > span { color: var(--commercial-muted); font-size: 12px; line-height: 1.5; } + +.commercial-pricing-confirmation { + padding: 18px; + border: 1px solid rgba(var(--theme-primary-rgb, 37, 99, 235), 0.22); + border-radius: 10px; + background: var(--commercial-primary-soft); +} + +.commercial-pricing-confirmation > header { display: flex; align-items: center; gap: 12px; } +.commercial-pricing-confirmation > header > i { color: var(--commercial-primary); font-size: 30px; } +.commercial-pricing-confirmation > header strong, +.commercial-pricing-confirmation > header span { display: block; } +.commercial-pricing-confirmation > header strong { font-size: 16px; } +.commercial-pricing-confirmation > header span { margin-top: 3px; color: var(--commercial-copy); font-size: 12px; } +.commercial-pricing-confirmation dl { display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 8px; margin: 16px 0 0; } +.commercial-pricing-confirmation dl div { padding: 11px; border: 1px solid rgba(var(--theme-primary-rgb, 37, 99, 235), 0.15); border-radius: 7px; background: #fff; } +.commercial-pricing-confirmation dt { color: var(--commercial-muted); font-size: 11px; } +.commercial-pricing-confirmation dd { margin: 5px 0 0; color: var(--commercial-ink); font-size: 13px; font-weight: 800; font-variant-numeric: tabular-nums; line-height: 1.45; } +.commercial-pricing-confirmation footer { display: flex; justify-content: flex-end; gap: 8px; margin-top: 16px; } + +.commercial-pricing-empty { + min-height: 120px; + display: flex; + align-items: center; + justify-content: center; + gap: 12px; + margin-top: 16px; + padding: 20px; + border: 1px dashed var(--commercial-line); + border-radius: 9px; + background: #f8fafc; + text-align: left; +} + +.commercial-pricing-empty > i { color: var(--commercial-primary); font-size: 32px; } +.commercial-pricing-empty strong, +.commercial-pricing-empty span { display: block; } +.commercial-pricing-empty span { margin-top: 4px; color: var(--commercial-copy); font-size: 13px; line-height: 1.5; } +.commercial-pricing-result { margin-top: 18px; padding-top: 18px; border-top: 1px solid var(--commercial-line); } +.commercial-pricing-result > header { display: flex; align-items: flex-start; justify-content: space-between; gap: 12px; } +.commercial-pricing-result > header > div > span { color: var(--commercial-primary-deep); font-size: 11px; font-weight: 800; letter-spacing: 0.08em; text-transform: uppercase; } +.commercial-pricing-result h4 { margin: 4px 0 0; font-size: 18px; } +.commercial-pricing-result > header p { margin: 5px 0 0; color: var(--commercial-muted); font-size: 12px; } +.commercial-pricing-summary { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 10px; margin: 14px 0; } +.commercial-pricing-summary article { padding: 14px; border: 1px solid var(--commercial-line); border-radius: 8px; background: #f8fafc; } +.commercial-pricing-summary span, +.commercial-pricing-summary strong, +.commercial-pricing-summary small { display: block; } +.commercial-pricing-summary span { color: var(--commercial-muted); font-size: 11px; font-weight: 750; } +.commercial-pricing-summary strong { margin-top: 5px; font-size: 15px; line-height: 1.45; } +.commercial-pricing-summary small { margin-top: 4px; color: var(--commercial-copy); font-size: 11px; line-height: 1.45; } +.commercial-pricing-grid { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 12px; } +.commercial-pricing-card { min-width: 0; padding: 16px; border: 1px solid var(--commercial-line); border-radius: 9px; background: #fff; } +.commercial-pricing-card.feasible { border-color: rgba(4, 120, 87, 0.25); } +.commercial-pricing-card.insufficient_value, +.commercial-pricing-card.cost_only { border-color: rgba(180, 83, 9, 0.27); } +.commercial-pricing-card.unavailable { border-color: rgba(185, 28, 28, 0.22); background: #f8fafc; } +.commercial-pricing-card > header { display: flex; align-items: center; justify-content: space-between; gap: 10px; } +.commercial-pricing-card > header span, +.commercial-pricing-card > header strong { display: block; } +.commercial-pricing-card > header span { color: var(--commercial-muted); font-size: 10px; font-weight: 750; text-transform: uppercase; } +.commercial-pricing-card > header strong { margin-top: 2px; font-size: 20px; } +.commercial-pricing-card > header em { padding: 4px 8px; border-radius: 999px; background: var(--commercial-primary-soft); color: var(--commercial-primary-deep); font-size: 11px; font-style: normal; font-weight: 800; } +.commercial-pricing-card.insufficient_value > header em, +.commercial-pricing-card.cost_only > header em { background: var(--commercial-warning-soft); color: var(--commercial-warning); } +.commercial-pricing-card.unavailable > header em { background: var(--commercial-danger-soft); color: var(--commercial-danger); } +.commercial-pricing-card dl { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 0 12px; margin: 14px 0 0; } +.commercial-pricing-card dl div { min-width: 0; padding: 9px 0; border-top: 1px solid #eef2f7; } +.commercial-pricing-card dt { color: var(--commercial-muted); font-size: 11px; } +.commercial-pricing-card dd { margin: 4px 0 0; overflow-wrap: anywhere; color: var(--commercial-ink); font-size: 13px; font-weight: 800; font-variant-numeric: tabular-nums; } +.commercial-pricing-card .pricing-bound dd { color: var(--commercial-primary-deep); font-size: 15px; } +.commercial-pricing-card > p { margin: 12px 0 0; color: var(--commercial-copy); font-size: 12px; line-height: 1.55; } +.commercial-pricing-missing { margin-top: 12px; padding: 10px 12px; border-radius: 7px; background: var(--commercial-warning-soft); color: var(--commercial-warning); font-size: 12px; } +.commercial-pricing-missing ul { margin: 5px 0 0; padding-left: 18px; line-height: 1.5; } +.commercial-pricing-notes { margin-top: 12px; color: var(--commercial-copy); font-size: 12px; } +.commercial-pricing-notes summary { min-height: 44px; display: flex; align-items: center; cursor: pointer; color: var(--commercial-primary-deep); font-weight: 800; } +.commercial-pricing-notes ul { margin: 4px 0 0; padding: 12px 12px 12px 30px; border-radius: 8px; background: #f8fafc; line-height: 1.6; } + +.commercial-admin-access.allowed { background: var(--commercial-success-soft); color: var(--commercial-success); } +.commercial-admin-access.denied { background: #f1f5f9; color: var(--commercial-muted); } +.commercial-admin-groups { display: grid; grid-template-columns: 2fr 1fr; gap: 14px; } +.commercial-admin-groups > article { overflow: hidden; } +.commercial-admin-groups article > header { display: flex; align-items: center; gap: 10px; padding: 15px 16px; border-bottom: 1px solid var(--commercial-line); background: #f8fafc; } +.commercial-admin-groups article > header > i { color: var(--commercial-primary); font-size: 24px; } +.commercial-admin-groups article > header strong, +.commercial-admin-groups article > header span { display: block; } +.commercial-admin-groups article > header span { margin-top: 2px; color: var(--commercial-muted); font-size: 11px; } +.commercial-admin-groups article > div { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 8px; padding: 12px; } +.commercial-admin-groups article:last-child > div { grid-template-columns: 1fr; } +.commercial-action-button { min-height: 46px; display: flex; align-items: center; justify-content: space-between; gap: 8px; padding: 0 12px; border: 1px solid var(--commercial-line); border-radius: 7px; background: #fff; color: var(--commercial-ink); font-size: 13px; font-weight: 750; text-align: left; } +.commercial-action-button:hover:not(:disabled) { border-color: var(--commercial-primary); color: var(--commercial-primary-deep); } + +.commercial-history-empty { + min-height: 170px; + display: flex; + flex-direction: column; + align-items: center; + justify-content: center; + gap: 7px; + padding: 24px; + border: 1px dashed var(--commercial-line); + border-radius: 9px; + color: var(--commercial-copy); + text-align: center; +} + +.commercial-history-empty > i { color: var(--commercial-primary); font-size: 34px; } +.commercial-history-empty > strong { color: var(--commercial-ink); font-size: 16px; } +.commercial-history-empty > span { margin-bottom: 6px; font-size: 13px; line-height: 1.55; } +.commercial-history-tabs { + display: flex; + gap: 7px; + margin-bottom: 12px; + padding-bottom: 2px; + overflow-x: auto; +} + +.commercial-history-tabs button { + min-height: 44px; + flex: 0 0 auto; + display: inline-flex; + align-items: center; + gap: 7px; + padding: 0 12px; + border: 1px solid var(--commercial-line); + border-radius: 7px; + background: #fff; + color: var(--commercial-copy); + font: inherit; + font-size: 13px; + font-weight: 750; + cursor: pointer; +} + +.commercial-history-tabs button span { min-width: 20px; padding: 2px 5px; border-radius: 999px; background: #f1f5f9; font-size: 10px; text-align: center; } +.commercial-history-tabs button.active { border-color: var(--commercial-primary); background: var(--commercial-primary-soft); color: var(--commercial-primary-deep); } +.commercial-history-tabs button.active span { background: #fff; } +.commercial-history-table { border: 1px solid var(--commercial-line); border-radius: 9px; } +.commercial-history-table table { min-width: 940px; } +.commercial-history-table-empty { min-height: 150px; border: 1px dashed var(--commercial-line); border-radius: 9px; } +.commercial-mono { max-width: 210px; overflow-wrap: anywhere; color: var(--commercial-copy); font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace; font-size: 11px !important; } +.commercial-history-actions { display: flex; align-items: center; gap: 6px; white-space: nowrap; } +.commercial-history-actions button { + min-height: 44px; + padding: 0 9px; + border: 1px solid var(--commercial-line); + border-radius: 6px; + background: #fff; + color: var(--commercial-primary-deep); + font: inherit; + font-size: 11px; + font-weight: 800; + cursor: pointer; +} + +.commercial-history-actions button:hover { border-color: var(--commercial-primary); } +.commercial-history-actions > span { color: var(--commercial-muted); font-size: 11px; } + +.commercial-dialog-mask { + position: fixed; + inset: 0; + z-index: 10050; + display: grid; + place-items: center; + padding: 20px; + background: rgba(15, 23, 42, 0.54); + backdrop-filter: blur(7px); + -webkit-backdrop-filter: blur(7px); +} + +.commercial-dialog { + width: min(760px, calc(100vw - 40px)); + max-height: calc(100dvh - 40px); + display: flex; + flex-direction: column; + border: 1px solid rgba(var(--theme-primary-rgb, 37, 99, 235), 0.2); + border-radius: 12px; + background: #fff; + box-shadow: 0 30px 80px rgba(15, 23, 42, 0.28); + overflow: hidden; + color: var(--commercial-ink, #0f172a); +} + +.commercial-dialog > header { flex: 0 0 auto; display: flex; align-items: flex-start; justify-content: space-between; gap: 16px; padding: 20px 22px; border-bottom: 1px solid #e2e8f0; } +.commercial-dialog > header > div { max-width: 630px; } +.commercial-dialog h3 { font-size: 21px; } +.commercial-icon-button { width: 44px; height: 44px; flex: 0 0 auto; border: 1px solid #e2e8f0; border-radius: 7px; background: #fff; color: #475569; font-size: 20px; } +.commercial-icon-button:hover:not(:disabled) { border-color: var(--commercial-primary, #2563eb); color: var(--commercial-primary-deep, #1d4ed8); } +.commercial-dialog-form, +.commercial-confirmation { min-height: 0; overflow-y: auto; padding: 20px 22px; } +.commercial-form-grid { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 14px; } +.commercial-form-grid label { min-width: 0; color: #334155; font-size: 13px; font-weight: 750; } +.commercial-form-grid label > span { display: block; margin-bottom: 6px; } +.commercial-form-grid .commercial-form-wide { grid-column: 1 / -1; } +.commercial-check-label { min-height: 44px; align-self: end; display: flex; align-items: center; gap: 9px; padding: 0 4px; } +.commercial-check-label input { width: 20px; min-height: 20px; } +.commercial-check-label > span { margin: 0 !important; } +.commercial-form-note { display: flex; align-items: center; gap: 8px; margin: 0; padding: 12px; border-radius: 7px; background: var(--commercial-primary-soft, #eff6ff); color: var(--commercial-primary-deep, #1d4ed8); font-size: 13px; line-height: 1.55; } +.commercial-form-error { margin: 14px 0 0; padding: 11px 13px; border: 1px solid rgba(185, 28, 28, 0.22); border-radius: 7px; background: #fef2f2; color: #b91c1c; font-size: 13px; line-height: 1.5; } +.commercial-dialog footer { display: flex; justify-content: flex-end; gap: 9px; margin-top: 20px; } +.commercial-confirmation { text-align: center; } +.commercial-confirmation-icon { width: 58px; height: 58px; display: grid; place-items: center; margin: 2px auto 12px; border-radius: 50%; background: var(--commercial-primary-soft, #eff6ff); color: var(--commercial-primary, #2563eb); font-size: 30px; } +.commercial-confirmation > strong { font-size: 19px; } +.commercial-confirmation > p { margin: 8px auto 0; max-width: 580px; color: #475569; font-size: 13px; line-height: 1.6; } +.commercial-confirmation ul { margin: 18px 0 0; padding: 0; list-style: none; text-align: left; } +.commercial-confirmation li { padding: 10px 12px; border-bottom: 1px solid #eef2f7; color: #334155; font-size: 13px; line-height: 1.5; } + +@media (max-width: 1080px) { + .commercial-hero { grid-template-columns: 1fr; } + .commercial-metric-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } + .commercial-pricing-fields { grid-template-columns: repeat(3, minmax(0, 1fr)); } +} + +@media (max-width: 760px) { + .commercial-hero, + .commercial-section { padding: 18px; } + .commercial-section-head { flex-direction: column; } + .commercial-section-head > .commercial-button { width: 100%; } + .commercial-account-grid, + .commercial-admin-groups, + .commercial-metric-grid, + .commercial-metric-grid.platform-grid, + .commercial-pricing-fields, + .commercial-pricing-summary, + .commercial-pricing-grid, + .commercial-pricing-confirmation dl { grid-template-columns: 1fr; } + .commercial-range-form { width: 100%; display: grid; grid-template-columns: 1fr; } + .commercial-range-form label, + .commercial-range-form input, + .commercial-range-form button { width: 100%; } + .commercial-admin-groups article > div { grid-template-columns: 1fr; } + .commercial-pricing-form { padding: 14px; } + .commercial-pricing-form-actions, + .commercial-pricing-confirmation footer { align-items: stretch; flex-direction: column; } + .commercial-pricing-form-actions .commercial-button, + .commercial-pricing-confirmation footer .commercial-button { width: 100%; } + .commercial-pricing-result > header { flex-direction: column; } + .commercial-pricing-card dl { grid-template-columns: 1fr; } + .commercial-tenant-picker > div { flex-direction: column; } + .commercial-dialog-mask { padding: 10px; } + .commercial-dialog { width: calc(100vw - 20px); max-height: calc(100dvh - 20px); } + .commercial-dialog > header, + .commercial-dialog-form, + .commercial-confirmation { padding: 16px; } + .commercial-form-grid { grid-template-columns: 1fr; } + .commercial-form-grid .commercial-form-wide { grid-column: auto; } + .commercial-dialog footer { flex-direction: column-reverse; } + .commercial-dialog footer .commercial-button { width: 100%; } +} + +@media (prefers-reduced-motion: reduce) { + .commercial-button, + .commercial-icon-button, + .commercial-action-button { transition: none; } +} diff --git a/web/src/components/commercial/commercialFormModel.js b/web/src/components/commercial/commercialFormModel.js new file mode 100644 index 0000000..6eb2d93 --- /dev/null +++ b/web/src/components/commercial/commercialFormModel.js @@ -0,0 +1,226 @@ +import { COMMERCIAL_ACTIONS } from './commercialWorkspaceModel.js' + +let idempotencySequence = 0 + +function toLocalDateTime(value) { + const date = value instanceof Date ? value : new Date(value) + const pad = (item) => String(item).padStart(2, '0') + return [ + date.getFullYear(), + '-', + pad(date.getMonth() + 1), + '-', + pad(date.getDate()), + 'T', + pad(date.getHours()), + ':', + pad(date.getMinutes()) + ].join('') +} + +function nextMonth(value) { + const date = new Date(value) + date.setMonth(date.getMonth() + 1) + return date +} + +function createIdempotencyKey(prefix) { + idempotencySequence += 1 + const random = typeof crypto !== 'undefined' && typeof crypto.randomUUID === 'function' + ? crypto.randomUUID() + : `${Date.now()}-${idempotencySequence}` + return `${prefix}-${random}` +} + +export function createCommercialForm(kind, account = null, now = new Date()) { + const current = toLocalDateTime(now) + const monthEnd = toLocalDateTime(nextMonth(now)) + const plan = account?.plan || {} + const subscription = account?.subscription || {} + const firstQuota = account?.quotas?.[0] || {} + const entitlement = firstQuota.entitlement || {} + + if (kind === 'createPlan') { + return { + planCode: String(plan.planCode || ''), + name: '', + pricingModel: 'subscription', + billingInterval: 'annual', + currency: String(plan.currency || 'CNY'), + baseFee: '0', + includedSeats: Number(plan.includedSeats || 1), + overageEnabled: false, + effectiveFrom: current, + effectiveTo: '', + contractTermsJson: '{}' + } + } + if (kind === 'createSubscription') { + return { + subscriptionKey: '', + planId: String(plan.id || ''), + status: 'active', + startsAt: current, + endsAt: '', + currentPeriodStart: current, + currentPeriodEnd: monthEnd, + seats: Math.max(1, Number(plan.includedSeats || subscription.seats || 1)), + autoRenew: false, + externalProvider: '', + externalSubscriptionId: '', + metadataJson: '{}' + } + } + if (kind === 'upsertEntitlement') { + return { + subscriptionId: String(subscription.id || ''), + entitlementKey: '', + metricKey: '', + entitlementType: 'metered', + unit: '次', + includedQuantity: '0', + hardLimitQuantity: '', + resetInterval: 'monthly', + overagePolicy: 'block', + status: 'active', + effectiveFrom: current, + effectiveTo: '', + configJson: '{}' + } + } + if (kind === 'recordUsage') { + return { + subscriptionId: String(subscription.id || ''), + entitlementId: String(entitlement.id || ''), + eventType: 'usage', + quantity: '1', + occurredAt: current, + sourceSystem: 'x-financial-web', + idempotencyKey: createIdempotencyKey('usage'), + reversalOfEventId: '', + subjectType: '', + subjectId: '', + correlationId: '', + metadataJson: '{}' + } + } + if (kind === 'recordCost') { + return { + subscriptionId: String(subscription.id || ''), + usageEventId: '', + eventType: 'incurred', + costCategory: 'ai_inference', + quantity: '1', + unit: '次', + unitCost: '0', + originalCurrency: String(subscription.currency || plan.currency || 'CNY'), + reportingCurrency: String(subscription.currency || plan.currency || 'CNY'), + fxRate: '1', + provider: '', + sku: '', + modelName: '', + allocationKey: '', + occurredAt: current, + sourceSystem: 'x-financial-web', + idempotencyKey: createIdempotencyKey('cost'), + reversalOfCostEventId: '', + correlationId: '', + metadataJson: '{}' + } + } + + if (kind === 'transitionSubscription') { + const allowedTargets = resolveSubscriptionTransitionTargets(subscription.status) + return { + resourceId: String(subscription.id || ''), + expectedVersion: Number(subscription.version || 1), + currentStatus: String(subscription.status || ''), + targetStatus: allowedTargets[0] || '', + reason: '' + } + } + + const currentResource = kind === 'activatePlan' + ? plan + : kind === 'activateSubscription' + ? subscription + : entitlement + return { + resourceId: String(currentResource.id || ''), + expectedVersion: Number(currentResource.version || 1), + reason: '' + } +} + +export function resolveSubscriptionTransitionTargets(status) { + const allowed = { + trialing: ['past_due', 'suspended', 'canceled', 'expired'], + active: ['past_due', 'suspended', 'canceled', 'expired'], + past_due: ['suspended', 'canceled', 'expired'], + suspended: ['canceled', 'expired'], + canceled: [], + expired: [] + } + return allowed[String(status || '')] || ['past_due', 'suspended', 'canceled', 'expired'] +} + +export function commercialActionMeta(kind) { + return COMMERCIAL_ACTIONS[kind] || null +} + +export function buildCommercialConfirmationRows(kind, payload = {}) { + if (kind === 'createPlan') { + return [ + `套餐：${payload.name}（${payload.planCode}）`, + `计价：${payload.pricingModel} / ${payload.billingInterval}`, + `基础费用：${payload.currency} ${payload.baseFee}`, + '创建后为草稿，仍需单独激活才会退役同编码旧版本。' + ] + } + if (kind === 'createSubscription') { + return [ + `订阅键：${payload.subscriptionKey}`, + `套餐 ID：${payload.planId}`, + `席位数：${payload.seats}`, + '订阅快照会固定当时的价格和币种。' + ] + } + if (kind === 'upsertEntitlement') { + return [ + `权益：${payload.entitlementKey}`, + `指标：${payload.metricKey}（${payload.unit}）`, + `超额策略：${payload.overagePolicy}`, + '已有用量后，影响计费口径的字段不可被覆盖修改。' + ] + } + if (kind.startsWith('activate')) { + return [ + `资源 ID：${payload.resourceId}`, + `期望版本：v${payload.expectedVersion}`, + '若版本已变化，服务器会拒绝本次操作并要求刷新。' + ] + } + if (kind === 'transitionSubscription') { + return [ + `订阅 ID：${payload.resourceId}`, + `期望版本：v${payload.expectedVersion}`, + `目标状态：${payload.targetStatus}`, + `变更原因：${payload.reason}`, + '暂停、取消或过期会使商业能力失败关闭；恢复需单独执行重新激活。' + ] + } + if (kind === 'recordUsage') { + return [ + `权益 ID：${payload.entitlementId}`, + `事件：${payload.eventType} ${payload.quantity}`, + `幂等键：${payload.idempotencyKey}`, + '用量事件写入后不可编辑；纠错必须新增 credit、adjustment 或 reversal。' + ] + } + return [ + `成本：${payload.costCategory} · ${payload.quantity} ${payload.unit}`, + `原币：${payload.originalCurrency}，报告币：${payload.reportingCurrency}，汇率：${payload.fxRate}`, + `幂等键：${payload.idempotencyKey}`, + '成本事件写入后不可编辑；不同币种不会在界面中隐式合并。' + ] +} diff --git a/web/src/components/commercial/commercialWorkspaceModel.js b/web/src/components/commercial/commercialWorkspaceModel.js new file mode 100644 index 0000000..37d3a87 --- /dev/null +++ b/web/src/components/commercial/commercialWorkspaceModel.js @@ -0,0 +1,570 @@ +const STATUS_LABELS = { + complete: '完整', + available: '可用', + partial: '部分可用', + unavailable: '不可用', + active: '生效中', + trialing: '试用中', + past_due: '逾期待处理', + suspended: '已暂停', + canceled: '已取消', + expired: '已过期', + draft: '草稿', + retired: '已退役', + approaching: '接近配额', + exhausted: '配额耗尽', + unlimited: '不限量', + inactive: '当前不可用', + feasible: '区间可行', + insufficient_value: '价值不足', + cost_only: '仅成本可用' +} + +const PRICING_MODEL_LABELS = { + hybrid: '混合定价（基础订阅 + 封顶成功费）', + subscription: '订阅制', + pilot_collecting: '试点采集证据', + optimize_unit_economics: '先优化单位经济性' +} + +export const COMMERCIAL_ACTIONS = Object.freeze({ + createPlan: { label: '创建套餐版本', family: 'configuration', confirm: '确认创建套餐版本' }, + activatePlan: { label: '激活套餐版本', family: 'activation', confirm: '确认激活套餐版本' }, + createSubscription: { label: '创建订阅快照', family: 'configuration', confirm: '确认创建并启用订阅' }, + activateSubscription: { label: '重新激活订阅', family: 'activation', confirm: '确认重新激活订阅' }, + transitionSubscription: { label: '变更订阅状态', family: 'activation', confirm: '确认变更订阅状态' }, + upsertEntitlement: { label: '配置权益版本', family: 'configuration', confirm: '确认保存权益版本' }, + activateEntitlement: { label: '激活权益版本', family: 'activation', confirm: '确认激活权益版本' }, + recordUsage: { label: '记录用量事件', family: 'metering', confirm: '确认写入不可变用量事件' }, + recordCost: { label: '记录成本事件', family: 'metering', confirm: '确认写入不可变成本事件' } +}) + +export function statusLabel(status) { + const normalized = String(status || '').trim() + return STATUS_LABELS[normalized] || normalized || '未知' +} + +export function classifyCommercialAccountState({ loading = false, error = null, account = null } = {}) { + if (loading && !account) return 'loading' + if (error?.status === 401 || error?.status === 403) return 'permission' + if (error && !account) return 'error' + if (!account || account.dataStatus === 'unavailable') return 'unavailable' + if (account.dataStatus === 'partial') return 'partial' + return 'ready' +} + +function normalizeCurrency(value) { + const currency = String(value || '').trim().toUpperCase() + return /^[A-Z]{3}$/u.test(currency) ? currency : '' +} + +function finiteNumber(value) { + if (value === null || value === undefined || value === '') return null + const parsed = Number(value) + return Number.isFinite(parsed) ? parsed : null +} + +export function formatCurrencyAmount(value, currency, locale = 'zh-CN') { + const amount = finiteNumber(value) + const normalizedCurrency = normalizeCurrency(currency) + if (amount === null || !normalizedCurrency) return '—' + try { + return new Intl.NumberFormat(locale, { + style: 'currency', + currency: normalizedCurrency, + minimumFractionDigits: 0, + maximumFractionDigits: 4 + }).format(amount) + } catch { + return `${normalizedCurrency} ${amount.toLocaleString(locale)}` + } +} + +export function formatRatio(value, locale = 'zh-CN') { + const ratio = finiteNumber(value) + if (ratio === null) return '—' + return new Intl.NumberFormat(locale, { + style: 'percent', + minimumFractionDigits: 1, + maximumFractionDigits: 2 + }).format(ratio) +} + +export function formatMetricValues(metric = {}) { + if (metric.status === 'unavailable') return [] + return (Array.isArray(metric.values) ? metric.values : []) + .map((item) => ({ + currency: normalizeCurrency(item.currency), + amount: finiteNumber(item.amount), + displayValue: formatCurrencyAmount(item.amount, item.currency), + basis: String(item.basis || '') + })) + .filter((item) => item.currency && item.amount !== null) +} + +export function formatMetricRatios(metric = {}) { + if (metric.status === 'unavailable') return [] + return (Array.isArray(metric.ratios) ? metric.ratios : []) + .map((item) => ({ + currency: normalizeCurrency(item.currency), + ratio: finiteNumber(item.ratio), + displayValue: formatRatio(item.ratio), + numeratorLabel: formatCurrencyAmount(item.numerator, item.currency), + denominatorLabel: formatCurrencyAmount(item.denominator, item.currency) + })) + .filter((item) => item.currency && item.ratio !== null) +} + +function metricCard(metric, category) { + const normalized = metric && typeof metric === 'object' ? metric : {} + const values = formatMetricValues(normalized) + const ratios = formatMetricRatios(normalized) + const status = ['available', 'partial', 'unavailable'].includes(normalized.status) + ? normalized.status + : 'unavailable' + return { + key: String(normalized.key || category), + label: String(normalized.label || '数据不可用'), + category, + status, + statusLabel: statusLabel(status), + values, + ratios, + displayValue: ratios[0]?.displayValue || values[0]?.displayValue || '不可用', + multiCurrency: new Set([...values, ...ratios].map((item) => item.currency)).size > 1, + reason: String(normalized.reason || '后端未返回该指标的事实口径。'), + requiredInputs: Array.isArray(normalized.requiredInputs) ? normalized.requiredInputs : [], + notes: Array.isArray(normalized.notes) ? normalized.notes : [] + } +} + +export function buildCommercialValueLedgers(analytics = null) { + if (!analytics) { + return { customer: [], platform: [] } + } + return { + customer: [ + metricCard(analytics.verifiedCashSavings, 'verifiedCashSavings'), + metricCard(analytics.customerCharges, 'customerCharges'), + metricCard(analytics.customerRoi, 'customerRoi'), + metricCard(analytics.customerLaborValue, 'customerLaborValue') + ], + platform: [ + metricCard(analytics.internalCosts, 'internalCosts'), + metricCard(analytics.contributionMargin, 'contributionMargin') + ] + } +} + +export function buildQuotaRows(account = null) { + return (Array.isArray(account?.quotas) ? account.quotas : []).map((quota) => { + const entitlement = quota.entitlement || {} + const used = finiteNumber(quota.usedQuantity) + const hardRemaining = finiteNumber(quota.hardLimitRemaining) + return { + id: String(entitlement.id || ''), + key: String(entitlement.entitlementKey || entitlement.metricKey || ''), + metricKey: String(entitlement.metricKey || ''), + unit: String(entitlement.unit || '次'), + type: String(entitlement.entitlementType || ''), + status: String(quota.status || 'inactive'), + statusLabel: statusLabel(quota.status), + usedLabel: used === null ? '不可用' : used.toLocaleString('zh-CN'), + remainingLabel: quota.status === 'unlimited' + ? '不限量' + : hardRemaining === null + ? '未配置硬上限' + : hardRemaining.toLocaleString('zh-CN'), + commerciallyAllowed: quota.commerciallyAllowed === true, + reason: String(quota.reason || ''), + version: Number(entitlement.version || 0) + } + }) +} + +export function createDefaultCommercialRange(now = new Date()) { + const end = new Date(now) + const start = new Date(now) + start.setUTCDate(start.getUTCDate() - 90) + const toInputDate = (value) => value.toISOString().slice(0, 10) + return { start: toInputDate(start), end: toInputDate(end) } +} + +export function buildAnalyticsWindow(range = {}) { + const startText = String(range.start || '').trim() + const endText = String(range.end || '').trim() + if (!startText || !endText) throw new Error('请选择完整的分析开始与结束日期。') + const start = new Date(`${startText}T00:00:00`) + const end = new Date(`${endText}T23:59:59.999`) + if (!Number.isFinite(start.getTime()) || !Number.isFinite(end.getTime()) || start >= end) { + throw new Error('分析开始日期必须早于结束日期。') + } + if (end.getTime() - start.getTime() > 731 * 86_400_000) { + throw new Error('单次商业分析时间范围不能超过 731 天。') + } + return { start: start.toISOString(), end: end.toISOString(), asOf: new Date().toISOString() } +} + +function toLocalDateTimeInput(value) { + const date = value instanceof Date ? value : new Date(value) + if (!Number.isFinite(date.getTime())) return '' + const pad = (item) => String(item).padStart(2, '0') + return `${date.getFullYear()}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}T${pad(date.getHours())}:${pad(date.getMinutes())}` +} + +export function createDefaultPricingScenarioForm(range = {}, now = new Date()) { + const fallback = createDefaultCommercialRange(now) + return { + start: String(range.start || fallback.start), + end: String(range.end || fallback.end), + asOf: toLocalDateTimeInput(now), + targetContributionMarginPercent: '65', + maxVerifiedSavingsSharePercent: '25' + } +} + +function percentageRate(value, label, { allowZero = false, upperExclusive = false } = {}) { + const percent = Number(value) + const validLower = allowZero ? percent >= 0 : percent > 0 + const roundedRate = Number((percent / 100).toFixed(6)) + const validUpper = upperExclusive ? roundedRate < 0.95 : roundedRate <= 1 + if (!Number.isFinite(percent) || !validLower || !validUpper) { + const range = upperExclusive ? '大于等于 0 且小于 95' : '大于 0 且不超过 100' + throw new Error(`${label}必须${range}。`) + } + return roundedRate.toFixed(6).replace(/(?:\.0+|(?\.\d*?)0+)$/u, '$') +} + +export function buildPricingScenarioPayload(form = {}) { + const window = buildAnalyticsWindow({ start: form.start, end: form.end }) + const asOf = toExplicitTimezone(form.asOf, '证据截止时间') + if (new Date(asOf).getTime() < new Date(window.start).getTime()) { + throw new Error('证据截止时间不能早于定价分析开始时间。') + } + return { + start: window.start, + end: window.end, + asOf, + targetContributionMarginRate: percentageRate( + form.targetContributionMarginPercent, + '目标贡献毛利率', + { allowZero: true, upperExclusive: true } + ), + maxVerifiedSavingsShare: percentageRate( + form.maxVerifiedSavingsSharePercent, + '最大价值分享比例' + ) + } +} + +function pricingMoneyLabel(value, currency) { + return value === null || value === undefined + ? '不可用' + : formatCurrencyAmount(value, currency) +} + +export function buildPricingScenarioView(result = null) { + if (!result || typeof result !== 'object') return null + const rows = (Array.isArray(result.scenarios) ? result.scenarios : []).map((item, index) => { + const currency = normalizeCurrency(item.currency) + const status = ['feasible', 'insufficient_value', 'cost_only', 'unavailable'].includes(item.status) + ? item.status + : 'unavailable' + const missingInputs = [] + if (item.internalCost === null || item.internalCost === undefined) { + missingInputs.push('同币种内部成本事实') + } + if (item.verifiedCashSavings === null || item.verifiedCashSavings === undefined) { + missingInputs.push('同币种财务确认现金节省') + } + return { + key: `${currency || 'unknown'}-${index}`, + currency: currency || '未知币种', + status, + statusLabel: statusLabel(status), + internalCostLabel: pricingMoneyLabel(item.internalCost, currency), + verifiedCashSavingsLabel: pricingMoneyLabel(item.verifiedCashSavings, currency), + minimumSustainableChargeLabel: pricingMoneyLabel(item.minimumSustainableCharge, currency), + maximumValueAlignedChargeLabel: pricingMoneyLabel(item.maximumValueAlignedCharge, currency), + maximumSuccessFeeLabel: pricingMoneyLabel(item.maximumSuccessFee, currency), + customerRoiLabel: formatRatio(item.customerRoiAtMinimumCharge), + contributionMarginLabel: formatRatio(item.contributionMarginAtValueCeiling), + missingInputs, + reason: String(item.reason || '后端未返回该币种定价结论。') + } + }) + const evidenceStatus = ['complete', 'partial', 'unavailable'].includes(result.evidenceStatus) + ? result.evidenceStatus + : 'unavailable' + return { + tenantId: String(result.tenantId || ''), + start: String(result.start || ''), + end: String(result.end || ''), + asOf: String(result.asOf || ''), + targetMarginLabel: formatRatio(result.targetContributionMarginRate), + maxValueShareLabel: formatRatio(result.maxVerifiedSavingsShare), + recommendedModel: String(result.recommendedModel || 'pilot_collecting'), + recommendedModelLabel: PRICING_MODEL_LABELS[result.recommendedModel] || '继续采集证据', + evidenceStatus, + evidenceStatusLabel: statusLabel(evidenceStatus), + rows, + multiCurrency: new Set(rows.map((item) => item.currency)).size > 1, + notes: Array.isArray(result.notes) ? result.notes.map((item) => String(item)) : [] + } +} + +export function toExplicitTimezone(value, fieldLabel) { + const normalized = String(value || '').trim() + if (!normalized) throw new Error(`请填写${fieldLabel}。`) + const parsed = new Date(normalized) + if (!Number.isFinite(parsed.getTime())) throw new Error(`${fieldLabel}格式无效。`) + return parsed.toISOString() +} + +function requiredText(form, key, label) { + const value = String(form?.[key] ?? '').trim() + if (!value) throw new Error(`请填写${label}。`) + return value +} + +function positiveInteger(value, label, minimum = 1) { + const number = Number(value) + if (!Number.isInteger(number) || number < minimum) { + throw new Error(`${label}必须是不小于 ${minimum} 的整数。`) + } + return number +} + +function nonNegativeNumber(value, label, optional = false) { + if (optional && (value === '' || value === null || value === undefined)) return null + const number = Number(value) + if (!Number.isFinite(number) || number < 0) throw new Error(`${label}必须是非负数。`) + return String(value) +} + +function positiveNumber(value, label) { + const number = Number(value) + if (!Number.isFinite(number) || number <= 0) throw new Error(`${label}必须大于 0。`) + return String(value) +} + +function ensureAscendingWindow(start, end, label) { + if (end && new Date(end).getTime() <= new Date(start).getTime()) { + throw new Error(`${label}结束时间必须晚于开始时间。`) + } +} + +function parseJsonObject(value, label) { + const normalized = String(value || '').trim() + if (!normalized) return {} + let parsed + try { + parsed = JSON.parse(normalized) + } catch { + throw new Error(`${label}必须是有效 JSON。`) + } + if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) { + throw new Error(`${label}必须是 JSON 对象。`) + } + return parsed +} + +function optionalText(value) { + const normalized = String(value || '').trim() + return normalized || null +} + +function createPlanPayload(form) { + const currency = normalizeCurrency(requiredText(form, 'currency', '套餐币种')) + if (!currency) throw new Error('套餐币种必须是三位英文字母。') + const payload = { + planCode: requiredText(form, 'planCode', '套餐编码'), + name: requiredText(form, 'name', '套餐名称'), + pricingModel: requiredText(form, 'pricingModel', '计价模式'), + billingInterval: requiredText(form, 'billingInterval', '计费周期'), + currency, + baseFee: nonNegativeNumber(form.baseFee, '基础费用'), + includedSeats: positiveInteger(form.includedSeats, '包含席位', 0), + overageEnabled: Boolean(form.overageEnabled), + effectiveFrom: toExplicitTimezone(form.effectiveFrom, '套餐生效时间'), + effectiveTo: form.effectiveTo ? toExplicitTimezone(form.effectiveTo, '套餐失效时间') : null, + contractTermsJson: parseJsonObject(form.contractTermsJson, '合同条款') + } + ensureAscendingWindow(payload.effectiveFrom, payload.effectiveTo, '套餐生效窗口') + return payload +} + +function createSubscriptionPayload(form, account) { + const provider = optionalText(form.externalProvider) + const externalId = optionalText(form.externalSubscriptionId) + if (Boolean(provider) !== Boolean(externalId)) { + throw new Error('外部订阅提供商和订阅编号必须同时填写或同时留空。') + } + const payload = { + subscriptionKey: requiredText(form, 'subscriptionKey', '订阅键'), + planId: requiredText({ planId: form.planId || account?.plan?.id }, 'planId', '套餐 ID'), + status: String(form.status || 'active'), + startsAt: toExplicitTimezone(form.startsAt, '订阅开始时间'), + endsAt: form.endsAt ? toExplicitTimezone(form.endsAt, '订阅结束时间') : null, + currentPeriodStart: toExplicitTimezone(form.currentPeriodStart, '当前周期开始时间'), + currentPeriodEnd: toExplicitTimezone(form.currentPeriodEnd, '当前周期结束时间'), + seats: positiveInteger(form.seats, '席位数'), + autoRenew: Boolean(form.autoRenew), + externalProvider: provider, + externalSubscriptionId: externalId, + metadataJson: parseJsonObject(form.metadataJson, '订阅元数据') + } + ensureAscendingWindow(payload.startsAt, payload.endsAt, '订阅') + ensureAscendingWindow(payload.currentPeriodStart, payload.currentPeriodEnd, '当前订阅周期') + return payload +} + +function entitlementPayload(form, account) { + const entitlementType = requiredText(form, 'entitlementType', '权益类型') + let includedQuantity = nonNegativeNumber(form.includedQuantity, '包含量', true) + let hardLimitQuantity = nonNegativeNumber(form.hardLimitQuantity, '硬配额', true) + if (entitlementType === 'unlimited') { + includedQuantity = null + hardLimitQuantity = null + } else if (entitlementType === 'metered' && includedQuantity === null) { + throw new Error('计量权益必须填写包含量。') + } + if (entitlementType === 'feature') { + const allowed = new Set([null, '0', '1']) + if (!allowed.has(includedQuantity) || !allowed.has(hardLimitQuantity)) { + throw new Error('功能权益的包含量和硬配额只能是 0 或 1。') + } + } + if (includedQuantity !== null && hardLimitQuantity !== null && Number(hardLimitQuantity) < Number(includedQuantity)) { + throw new Error('硬配额不能小于包含量。') + } + const payload = { + subscriptionId: requiredText( + { subscriptionId: form.subscriptionId || account?.subscription?.id }, + 'subscriptionId', + '订阅 ID' + ), + entitlementKey: requiredText(form, 'entitlementKey', '权益键'), + metricKey: requiredText(form, 'metricKey', '计量指标键'), + entitlementType, + unit: requiredText(form, 'unit', '计量单位'), + includedQuantity, + hardLimitQuantity, + resetInterval: requiredText(form, 'resetInterval', '重置周期'), + overagePolicy: requiredText(form, 'overagePolicy', '超额策略'), + status: String(form.status || 'active'), + effectiveFrom: toExplicitTimezone(form.effectiveFrom, '权益生效时间'), + effectiveTo: form.effectiveTo ? toExplicitTimezone(form.effectiveTo, '权益失效时间') : null, + configJson: parseJsonObject(form.configJson, '权益配置') + } + ensureAscendingWindow(payload.effectiveFrom, payload.effectiveTo, '权益生效窗口') + return payload +} + +function activationPayload(form, account, kind) { + const defaults = { + activatePlan: account?.plan, + activateSubscription: account?.subscription, + activateEntitlement: null + } + const current = defaults[kind] + const reason = requiredText(form, 'reason', '激活原因') + if (reason.length < 2) throw new Error('激活原因至少填写 2 个字符。') + return { + resourceId: requiredText({ resourceId: form.resourceId || current?.id }, 'resourceId', '资源 ID'), + expectedVersion: positiveInteger(form.expectedVersion || current?.version, '期望版本'), + reason + } +} + +function subscriptionTransitionPayload(form, account) { + const activation = activationPayload(form, account, 'activateSubscription') + const targetStatus = requiredText(form, 'targetStatus', '目标状态') + if (!['past_due', 'suspended', 'canceled', 'expired'].includes(targetStatus)) { + throw new Error('目标订阅状态无效。') + } + const reason = requiredText(form, 'reason', '状态变更原因') + if (reason.length < 2) throw new Error('状态变更原因至少填写 2 个字符。') + return { ...activation, targetStatus, reason } +} + +function usagePayload(form, account) { + const eventType = String(form.eventType || 'usage') + const quantity = Number(requiredText(form, 'quantity', '用量数量')) + if (!Number.isFinite(quantity)) throw new Error('用量数量必须是有效数字。') + if (eventType === 'usage' && quantity <= 0) throw new Error('用量事件数量必须大于 0。') + if (eventType === 'credit' && quantity >= 0) throw new Error('抵扣事件数量必须小于 0。') + if (['adjustment', 'reversal'].includes(eventType) && quantity === 0) throw new Error('调整或冲回事件数量不能为 0。') + const reversalOfEventId = optionalText(form.reversalOfEventId) + if ((eventType === 'reversal') !== Boolean(reversalOfEventId)) { + throw new Error('只有冲回事件必须且只能填写被冲回事件 ID。') + } + const subjectType = optionalText(form.subjectType) + const subjectId = optionalText(form.subjectId) + if (Boolean(subjectType) !== Boolean(subjectId)) throw new Error('主体类型和主体 ID 必须同时填写或同时留空。') + return { + subscriptionId: requiredText( + { subscriptionId: form.subscriptionId || account?.subscription?.id }, + 'subscriptionId', + '订阅 ID' + ), + entitlementId: requiredText(form, 'entitlementId', '权益 ID'), + eventType, + quantity: String(form.quantity), + occurredAt: toExplicitTimezone(form.occurredAt, '用量发生时间'), + sourceSystem: requiredText(form, 'sourceSystem', '来源系统'), + idempotencyKey: requiredText(form, 'idempotencyKey', '幂等键'), + reversalOfEventId, + subjectType, + subjectId, + correlationId: optionalText(form.correlationId), + metadataJson: parseJsonObject(form.metadataJson, '用量元数据') + } +} + +function costPayload(form, account) { + const originalCurrency = normalizeCurrency(requiredText(form, 'originalCurrency', '原始币种')) + const reportingCurrency = normalizeCurrency(requiredText(form, 'reportingCurrency', '报告币种')) + if (!originalCurrency || !reportingCurrency) throw new Error('成本币种必须是三位英文字母。') + const subscriptionId = optionalText(form.subscriptionId || account?.subscription?.id) + const usageEventId = optionalText(form.usageEventId) + if (usageEventId && !subscriptionId) throw new Error('关联用量事件时必须填写订阅 ID。') + const eventType = String(form.eventType || 'incurred') + const reversalOfCostEventId = optionalText(form.reversalOfCostEventId) + if ((eventType === 'reversal') !== Boolean(reversalOfCostEventId)) { + throw new Error('只有冲回成本事件必须且只能填写被冲回成本事件 ID。') + } + return { + subscriptionId, + usageEventId, + eventType, + costCategory: requiredText(form, 'costCategory', '成本类别'), + quantity: positiveNumber(form.quantity, '成本数量'), + unit: requiredText(form, 'unit', '成本单位'), + unitCost: nonNegativeNumber(form.unitCost, '单位成本'), + originalCurrency, + reportingCurrency, + fxRate: positiveNumber(form.fxRate, '汇率'), + provider: optionalText(form.provider), + sku: optionalText(form.sku), + modelName: optionalText(form.modelName), + allocationKey: requiredText(form, 'allocationKey', '归集键'), + occurredAt: toExplicitTimezone(form.occurredAt, '成本发生时间'), + sourceSystem: requiredText(form, 'sourceSystem', '来源系统'), + idempotencyKey: requiredText(form, 'idempotencyKey', '幂等键'), + reversalOfCostEventId, + correlationId: optionalText(form.correlationId), + metadataJson: parseJsonObject(form.metadataJson, '成本元数据') + } +} + +export function buildCommercialMutation(kind, form = {}, account = null) { + if (!COMMERCIAL_ACTIONS[kind]) throw new Error('不支持的商业管理操作。') + if (kind === 'createPlan') return createPlanPayload(form) + if (kind === 'createSubscription') return createSubscriptionPayload(form, account) + if (kind === 'upsertEntitlement') return entitlementPayload(form, account) + if (kind === 'transitionSubscription') return subscriptionTransitionPayload(form, account) + if (kind.startsWith('activate')) return activationPayload(form, account, kind) + if (kind === 'recordUsage') return usagePayload(form, account) + return costPayload(form, account) +} diff --git a/web/src/components/dashboard/CfoValueActionDialog.vue b/web/src/components/dashboard/CfoValueActionDialog.vue new file mode 100644 index 0000000..3c6b424 --- /dev/null +++ b/web/src/components/dashboard/CfoValueActionDialog.vue @@ -0,0 +1,169 @@ + + + + + diff --git a/web/src/components/dashboard/CfoValueDashboard.vue b/web/src/components/dashboard/CfoValueDashboard.vue new file mode 100644 index 0000000..77bb0c2 --- /dev/null +++ b/web/src/components/dashboard/CfoValueDashboard.vue @@ -0,0 +1,393 @@ + + + + + diff --git a/web/src/components/dashboard/CfoValueOpportunityDrawer.vue b/web/src/components/dashboard/CfoValueOpportunityDrawer.vue new file mode 100644 index 0000000..6356ba5 --- /dev/null +++ b/web/src/components/dashboard/CfoValueOpportunityDrawer.vue @@ -0,0 +1,330 @@ + + + + + diff --git a/web/src/components/dashboard/FinancialConnectorHealthPanel.vue b/web/src/components/dashboard/FinancialConnectorHealthPanel.vue new file mode 100644 index 0000000..97dfbc2 --- /dev/null +++ b/web/src/components/dashboard/FinancialConnectorHealthPanel.vue @@ -0,0 +1,237 @@ + + + + + diff --git a/web/src/components/layout/useTopBarOverviewRange.js b/web/src/components/layout/useTopBarOverviewRange.js index 0d4e024..5a1c2f4 100644 --- a/web/src/components/layout/useTopBarOverviewRange.js +++ b/web/src/components/layout/useTopBarOverviewRange.js @@ -4,6 +4,8 @@ import { formatDateValue } from '../../utils/dateRangeDefaults.js' const OVERVIEW_DASHBOARD_OPTIONS = [ { label: '财务看板', value: 'finance' }, + { label: '经营价值看板', value: 'value' }, + { label: '商业化管理', value: 'commercial' }, { label: '风险看板', value: 'risk' }, { label: '数字员工看板', value: 'digitalEmployee' }, { label: '系统看板', value: 'system' } diff --git a/web/src/components/travel/TravelRequestApplicationFacts.vue b/web/src/components/travel/TravelRequestApplicationFacts.vue new file mode 100644 index 0000000..81c294a --- /dev/null +++ b/web/src/components/travel/TravelRequestApplicationFacts.vue @@ -0,0 +1,111 @@ + + + + + diff --git a/web/src/composables/useAppShell.js b/web/src/composables/useAppShell.js index cf17120..ea20fac 100644 --- a/web/src/composables/useAppShell.js +++ b/web/src/composables/useAppShell.js @@ -34,7 +34,7 @@ import { createCurrentYearDateRange } from '../utils/dateRangeDefaults.js' const SESSION_TYPE_EXPENSE = 'expense' const SMART_ENTRY_SOURCE_APPLICATION = 'application' const SMART_ENTRY_SOURCE_REIMBURSEMENT = 'topbar' -const DOCUMENT_DETAIL_RETURN_TARGETS = new Set(['workbench', 'conversation']) +const DOCUMENT_DETAIL_RETURN_TARGETS = new Set(['workbench', 'conversation', 'value']) function resolveDocumentDetailReturnTarget(value) { const target = String(value || '').trim() @@ -116,7 +116,8 @@ export function useAppShell() { if (detailReturnTarget.value === 'conversation') { return '返回对话' } - return detailReturnTarget.value === 'workbench' ? '返回首页' : '返回单据中心' + if (detailReturnTarget.value === 'workbench') return '返回首页' + return detailReturnTarget.value === 'value' ? '返回经营价值' : '返回单据中心' }) const detailAlerts = computed(() => ( detailMode.value @@ -585,6 +586,17 @@ export function useAppShell() { return nextQuery } + function buildValueDashboardReturnQuery() { + const nextQuery = {} + Object.entries(buildDocumentReturnQuery()).forEach(([key, value]) => { + if (['range', 'start', 'end'].includes(key) || key.startsWith('value_')) { + nextQuery[key] = value + } + }) + nextQuery.dashboard = 'value' + return nextQuery + } + function openRequestDetail(request, options = {}) { const requestId = resolveRequestDetailLookupId(request) if (!requestId) { @@ -609,6 +621,10 @@ export function useAppShell() { return router.push({ name: 'app-workbench' }) } + if (detailReturnTarget.value === 'value') { + return router.push({ name: 'app-overview', query: buildValueDashboardReturnQuery() }) + } + return router.push({ name: 'app-documents', query: buildDocumentReturnQuery() }) } diff --git a/web/src/composables/useCfoValueDashboard.js b/web/src/composables/useCfoValueDashboard.js new file mode 100644 index 0000000..9fe910d --- /dev/null +++ b/web/src/composables/useCfoValueDashboard.js @@ -0,0 +1,424 @@ +import { computed, onMounted, reactive, ref, watch } from 'vue' +import { useRoute, useRouter } from 'vue-router' + +import { + executeSavingsOpportunityAction, + executeSavingsRealizationAction, + fetchCfoValueDashboard, + fetchSavingsOpportunities, + fetchSavingsOpportunity +} from '../services/analyticsValue.js' +import { + buildBreakdownGroups, + buildDataQualityRows, + buildFunnelRows, + buildGuardrailRows, + buildOpportunityRows, + buildTrendSeries, + buildValueKpis, + classifyCfoDashboardState, + createEmptyValueFilters, + createRequestId, + isDashboardStale, + readValueFiltersFromQuery, + resolveRangeWindow, + writeValueFiltersToQuery +} from '../views/scripts/cfoValueDashboardModel.js' +import { + hasValueOpportunityQuery, + normalizeValueOpportunityId, + opportunityMatchesValueContext, + readValueOpportunityId, + shouldClearValueOpportunityError, + writeValueOpportunityToQuery +} from '../views/scripts/cfoValueSourceLinks.js' + +const PAGE_SIZE = 12 + +export function useCfoValueDashboard(options = {}) { + const route = useRoute() + const router = useRouter() + const initialFilters = readValueFiltersFromQuery(route.query) + const filterDraft = reactive({ ...initialFilters }) + const appliedFilters = ref({ ...initialFilters }) + const page = ref(readPage(route.query.value_page)) + const dashboard = ref(null) + const dashboardLoading = ref(false) + const dashboardError = ref(null) + const opportunities = ref(emptyOpportunityPage()) + const opportunitiesLoading = ref(false) + const opportunitiesError = ref(null) + const selectedOpportunity = ref(null) + const selectedOpportunityId = ref('') + const selectedOpportunityLoading = ref(false) + const selectedOpportunityError = ref(null) + const detailOpen = ref(false) + const mutationLoading = ref(false) + const mutationError = ref(null) + const mutationMessage = ref('') + const trendCurrency = ref('') + const activeBreakdownDimension = ref('department') + let dashboardRequestSeq = 0 + let opportunityRequestSeq = 0 + let detailRequestSeq = 0 + + const dateWindow = computed(() => resolveRangeWindow( + options.activeRange, + options.customRange || {} + )) + const dashboardState = computed(() => classifyCfoDashboardState({ + dashboard: dashboard.value, + error: dashboardError.value, + loading: dashboardLoading.value + })) + const dashboardStale = computed(() => isDashboardStale(dashboard.value || {})) + const kpis = computed(() => buildValueKpis(dashboard.value || {})) + const funnelRows = computed(() => buildFunnelRows(dashboard.value || {})) + const trend = computed(() => buildTrendSeries(dashboard.value || {}, trendCurrency.value)) + const breakdownGroups = computed(() => buildBreakdownGroups(dashboard.value || {})) + const activeBreakdown = computed(() => ( + breakdownGroups.value.find((group) => group.dimension === activeBreakdownDimension.value) + || breakdownGroups.value[0] + || { dimension: '', label: '暂无维度', items: [] } + )) + const guardrailRows = computed(() => buildGuardrailRows(dashboard.value || {})) + const dataQualityRows = computed(() => buildDataQualityRows(dashboard.value || {})) + const opportunityRows = computed(() => buildOpportunityRows(opportunities.value)) + const hasPartialEvidence = computed(() => ( + dashboardState.value === 'partial' + || dataQualityRows.value.some((item) => item.count > 0) + )) + + async function loadDashboard() { + const requestSeq = ++dashboardRequestSeq + dashboardLoading.value = true + dashboardError.value = null + try { + const payload = await fetchCfoValueDashboard({ + ...dateWindow.value, + asOf: new Date().toISOString(), + ...withoutStatus(appliedFilters.value) + }) + if (requestSeq !== dashboardRequestSeq) return + dashboard.value = payload + if (!trend.value.currencies.includes(trendCurrency.value)) { + trendCurrency.value = trend.value.currencies[0] || '' + } + if (!breakdownGroups.value.some((group) => group.dimension === activeBreakdownDimension.value)) { + activeBreakdownDimension.value = breakdownGroups.value[0]?.dimension || '' + } + } catch (error) { + if (requestSeq === dashboardRequestSeq) dashboardError.value = error + } finally { + if (requestSeq === dashboardRequestSeq) dashboardLoading.value = false + } + } + + async function loadOpportunities() { + const requestSeq = ++opportunityRequestSeq + opportunitiesLoading.value = true + opportunitiesError.value = null + try { + const payload = await fetchSavingsOpportunities({ + ...appliedFilters.value, + page: page.value, + pageSize: PAGE_SIZE, + createdFrom: dateWindow.value.start, + createdTo: dateWindow.value.end, + sort: 'created_desc' + }) + if (requestSeq === opportunityRequestSeq) { + opportunities.value = payload + const lastPage = Math.max(1, Number(payload.totalPages || 0)) + if (page.value > lastPage) void setPage(lastPage) + } + } catch (error) { + if (requestSeq === opportunityRequestSeq) opportunitiesError.value = error + } finally { + if (requestSeq === opportunityRequestSeq) opportunitiesLoading.value = false + } + } + + async function reloadAll() { + await Promise.allSettled([loadDashboard(), loadOpportunities()]) + } + + async function loadOpportunityDetail(opportunityId) { + const normalizedId = normalizeValueOpportunityId(opportunityId) + if (!normalizedId) { + await clearUnavailableOpportunity() + return + } + const requestSeq = ++detailRequestSeq + selectedOpportunityId.value = normalizedId + detailOpen.value = true + selectedOpportunity.value = null + selectedOpportunityError.value = null + selectedOpportunityLoading.value = true + try { + const payload = await fetchSavingsOpportunity(normalizedId) + if (requestSeq !== detailRequestSeq) return + if (!opportunityMatchesValueContext(payload, appliedFilters.value, dateWindow.value)) { + await clearUnavailableOpportunity() + return + } + selectedOpportunity.value = payload + } catch (error) { + if (requestSeq !== detailRequestSeq) return + if (shouldClearValueOpportunityError(error)) { + await clearUnavailableOpportunity() + return + } + selectedOpportunityError.value = error + } finally { + if (requestSeq === detailRequestSeq) selectedOpportunityLoading.value = false + } + } + + async function openOpportunity(opportunityId) { + const normalizedId = normalizeValueOpportunityId(opportunityId) + if (!normalizedId) { + await clearUnavailableOpportunity() + return + } + const nextQuery = writeValueOpportunityToQuery(route.query, normalizedId) + if (sameQuery(nextQuery, route.query)) { + await loadOpportunityDetail(normalizedId) + return + } + await router.push({ query: nextQuery }) + } + + async function closeOpportunity() { + resetOpportunityDetail() + const nextQuery = writeValueOpportunityToQuery(route.query) + if (!sameQuery(nextQuery, route.query)) await router.push({ query: nextQuery }) + } + + function resetOpportunityDetail() { + detailRequestSeq += 1 + detailOpen.value = false + selectedOpportunity.value = null + selectedOpportunityId.value = '' + selectedOpportunityError.value = null + selectedOpportunityLoading.value = false + } + + async function clearUnavailableOpportunity() { + resetOpportunityDetail() + const nextQuery = writeValueOpportunityToQuery(route.query) + if (!sameQuery(nextQuery, route.query)) await router.replace({ query: nextQuery }) + } + + async function syncOpportunityFromRoute() { + const opportunityId = readValueOpportunityId(route.query) + if (!opportunityId) { + resetOpportunityDetail() + if (hasValueOpportunityQuery(route.query)) await clearUnavailableOpportunity() + return + } + if ( + opportunityId === selectedOpportunityId.value + && selectedOpportunity.value + && opportunityMatchesValueContext(selectedOpportunity.value, appliedFilters.value, dateWindow.value) + ) { + detailOpen.value = true + return + } + if (opportunityId === selectedOpportunityId.value && selectedOpportunityLoading.value) return + await loadOpportunityDetail(opportunityId) + } + + async function submitAction(command = {}) { + mutationLoading.value = true + mutationError.value = null + mutationMessage.value = '' + const requestId = command.requestId || createRequestId('cfo-value') + try { + let response + if (command.kind === 'record' || command.action === 'record_realization') { + throw new Error('实际结果必须由付款事件或具备可追溯凭证的连接器写入。') + } else if (command.kind === 'realization') { + response = await executeSavingsRealizationAction(command.realizationId, { + action: command.action, + requestId, + expectedVersion: command.expectedVersion, + comment: command.comment, + ...(command.action === 'reverse' ? { reversalAmount: command.reversalAmount } : {}), + evidence: [] + }) + } else { + response = await executeSavingsOpportunityAction(command.opportunityId, { + action: command.action, + requestId, + expectedVersion: command.expectedVersion, + comment: command.comment + }) + } + mutationMessage.value = response.replayed ? '请求已处理，本次返回原执行结果。' : '操作已完成并写入价值证据链。' + selectedOpportunity.value = response.opportunity || selectedOpportunity.value + await reloadAll() + if (detailOpen.value && command.opportunityId) { + await loadOpportunityDetail(command.opportunityId) + } + return response + } catch (error) { + mutationError.value = error + throw error + } finally { + mutationLoading.value = false + } + } + + async function applyFilters() { + const nextFilters = Object.fromEntries( + Object.entries(filterDraft).map(([key, value]) => [key, String(value || '').trim()]) + ) + const nextQuery = writeValueFiltersToQuery(route.query, nextFilters) + delete nextQuery.value_page + if (sameQuery(nextQuery, route.query)) { + appliedFilters.value = nextFilters + page.value = 1 + await reloadAll() + return + } + await router.replace({ query: nextQuery }) + } + + async function resetFilters() { + Object.assign(filterDraft, createEmptyValueFilters()) + await applyFilters() + } + + async function setPage(nextPage) { + const normalized = Math.max(1, Number(nextPage || 1)) + const nextQuery = { ...route.query } + if (normalized > 1) nextQuery.value_page = String(normalized) + else delete nextQuery.value_page + if (sameQuery(nextQuery, route.query)) { + page.value = normalized + await loadOpportunities() + return + } + await router.replace({ query: nextQuery }) + } + + watch( + () => [ + valueFilterRouteSignature(route.query), + readPage(route.query.value_page), + valueOpportunityRouteSignature(route.query) + ], + ([filterSignature, nextPage, opportunitySignature], previous = []) => { + const [previousFilterSignature, previousPage] = previous + const nextFilters = readValueFiltersFromQuery(route.query) + Object.assign(filterDraft, nextFilters) + appliedFilters.value = nextFilters + page.value = nextPage + if (filterSignature !== previousFilterSignature) { + void reloadAll().then(() => syncOpportunityFromRoute()) + } else if (nextPage !== previousPage) { + void loadOpportunities() + } + if (filterSignature === previousFilterSignature && opportunitySignature !== previous[2]) { + void syncOpportunityFromRoute() + } + } + ) + + watch( + () => [options.activeRange, options.customRange?.start, options.customRange?.end], + () => { + void Promise.allSettled([loadDashboard(), setPage(1)]).then(() => syncOpportunityFromRoute()) + } + ) + + watch(() => trend.value.selectedCurrency, (currency) => { + if (currency && currency !== trendCurrency.value) trendCurrency.value = currency + }, { immediate: true }) + + onMounted(() => { + void reloadAll().then(() => syncOpportunityFromRoute()) + }) + + return { + activeBreakdown, + activeBreakdownDimension, + appliedFilters, + breakdownGroups, + dashboard, + dashboardError, + dashboardLoading, + dashboardState, + dashboardStale, + dataQualityRows, + dateWindow, + detailOpen, + filterDraft, + funnelRows, + guardrailRows, + hasPartialEvidence, + kpis, + mutationError, + mutationLoading, + mutationMessage, + opportunities, + opportunitiesError, + opportunitiesLoading, + opportunityRows, + page, + selectedOpportunity, + selectedOpportunityId, + selectedOpportunityError, + selectedOpportunityLoading, + trend, + trendCurrency, + applyFilters, + closeOpportunity, + loadDashboard, + loadOpportunities, + openOpportunity, + reloadAll, + resetFilters, + setPage, + submitAction + } +} + +function emptyOpportunityPage() { + return { items: [], total: 0, page: 1, pageSize: PAGE_SIZE, totalPages: 0, generatedAt: '' } +} + +function withoutStatus(filters = {}) { + const { status: _status, ...dashboardFilters } = filters + return dashboardFilters +} + +function readPage(value) { + const raw = Array.isArray(value) ? value[0] : value + const page = Number(raw || 1) + return Number.isInteger(page) && page > 0 ? page : 1 +} + +function valueFilterRouteSignature(query = {}) { + return JSON.stringify( + Object.fromEntries( + Object.entries(query) + .filter(([key]) => key.startsWith('value_') && !['value_page', 'value_opportunity'].includes(key)) + .sort(([left], [right]) => left.localeCompare(right)) + ) + ) +} + +function valueOpportunityRouteSignature(query = {}) { + if (!Object.hasOwn(query, 'value_opportunity')) return '' + const value = query.value_opportunity + return JSON.stringify(Array.isArray(value) ? value.map(String) : String(value || '')) +} + +function sameQuery(left = {}, right = {}) { + const normalize = (query) => Object.entries(query) + .map(([key, value]) => [key, Array.isArray(value) ? value.map(String) : String(value)]) + .sort(([leftKey], [rightKey]) => leftKey.localeCompare(rightKey)) + return JSON.stringify(normalize(left)) === JSON.stringify(normalize(right)) +} diff --git a/web/src/composables/useCommercialWorkspace.js b/web/src/composables/useCommercialWorkspace.js new file mode 100644 index 0000000..3e18812 --- /dev/null +++ b/web/src/composables/useCommercialWorkspace.js @@ -0,0 +1,293 @@ +import { computed, reactive, ref, toValue, watch } from 'vue' + +import { + activateCommercialEntitlement, + activateCommercialPlan, + activateCommercialSubscription, + buildCommercialPricingScenario, + createCommercialPlan, + createCommercialSubscription, + fetchCommercialAccount, + fetchCommercialAnalytics, + fetchCommercialCostEvents, + fetchCommercialEntitlements, + fetchCommercialPlans, + fetchCommercialSubscriptions, + fetchCommercialUsageEvents, + recordCommercialCost, + recordCommercialUsage, + transitionCommercialSubscription, + upsertCommercialEntitlement +} from '../services/commercial.js' +import { + buildAnalyticsWindow, + buildCommercialMutation, + buildCommercialValueLedgers, + buildPricingScenarioPayload, + buildPricingScenarioView, + buildQuotaRows, + classifyCommercialAccountState, + createDefaultCommercialRange +} from '../components/commercial/commercialWorkspaceModel.js' + +function resolveBoolean(value) { + return Boolean(toValue(value)) +} + +function resolveText(value) { + return String(toValue(value) || '').trim() +} + +export function useCommercialWorkspace(options = {}) { + const account = ref(null) + const accountLoading = ref(false) + const accountError = ref(null) + const analytics = ref(null) + const analyticsLoading = ref(false) + const analyticsError = ref(null) + const mutationLoading = ref(false) + const mutationError = ref(null) + const mutationMessage = ref('') + const pricingScenario = ref(null) + const pricingLoading = ref(false) + const pricingError = ref(null) + const history = reactive({ plans: [], subscriptions: [], entitlements: [], usageEvents: [], costEvents: [] }) + const historyLoading = ref(false) + const historyError = ref(null) + const historyLoaded = ref(false) + const range = reactive(createDefaultCommercialRange()) + let accountRequestSeq = 0 + let analyticsRequestSeq = 0 + let historyRequestSeq = 0 + let pricingRequestSeq = 0 + + const tenantId = computed(() => resolveText(options.tenantId)) + const platformAdmin = computed(() => resolveBoolean(options.platformAdmin)) + const active = computed(() => options.active === undefined || resolveBoolean(options.active)) + const canManage = computed(() => platformAdmin.value && Boolean(tenantId.value)) + const accountState = computed(() => classifyCommercialAccountState({ + loading: accountLoading.value, + error: accountError.value, + account: account.value + })) + const quotaRows = computed(() => buildQuotaRows(account.value)) + const ledgers = computed(() => buildCommercialValueLedgers(analytics.value)) + const pricingScenarioView = computed(() => buildPricingScenarioView(pricingScenario.value)) + + async function loadAccount() { + if (!active.value) return null + const requestSeq = ++accountRequestSeq + accountLoading.value = true + accountError.value = null + try { + const payload = await fetchCommercialAccount({ + tenantId: tenantId.value, + platformAdmin: platformAdmin.value + }) + if (requestSeq === accountRequestSeq) account.value = payload + return payload + } catch (error) { + if (requestSeq === accountRequestSeq) accountError.value = error + throw error + } finally { + if (requestSeq === accountRequestSeq) accountLoading.value = false + } + } + + async function loadAnalytics() { + if (!active.value || !platformAdmin.value || !tenantId.value) { + analyticsRequestSeq += 1 + analytics.value = null + analyticsError.value = null + analyticsLoading.value = false + return null + } + const requestSeq = ++analyticsRequestSeq + analyticsLoading.value = true + analyticsError.value = null + try { + const payload = await fetchCommercialAnalytics( + tenantId.value, + buildAnalyticsWindow(range) + ) + if (requestSeq === analyticsRequestSeq) analytics.value = payload + return payload + } catch (error) { + if (requestSeq === analyticsRequestSeq) analyticsError.value = error + throw error + } finally { + if (requestSeq === analyticsRequestSeq) analyticsLoading.value = false + } + } + + async function reload() { + const operations = [loadAccount()] + if (platformAdmin.value && tenantId.value) operations.push(loadAnalytics()) + return Promise.allSettled(operations) + } + + async function loadHistory() { + if (!active.value || !platformAdmin.value || !tenantId.value) return null + const requestSeq = ++historyRequestSeq + historyLoading.value = true + historyError.value = null + const listOptions = { limit: 100, offset: 0 } + const eventOptions = { limit: 200, offset: 0 } + const requests = [ + ['plans', fetchCommercialPlans(tenantId.value, listOptions)], + ['subscriptions', fetchCommercialSubscriptions(tenantId.value, listOptions)], + ['entitlements', fetchCommercialEntitlements(tenantId.value, { ...listOptions, limit: 200 })], + ['usageEvents', fetchCommercialUsageEvents(tenantId.value, eventOptions)], + ['costEvents', fetchCommercialCostEvents(tenantId.value, eventOptions)] + ] + const results = await Promise.allSettled(requests.map(([, request]) => request)) + if (requestSeq !== historyRequestSeq) return null + const failures = [] + results.forEach((result, index) => { + const key = requests[index][0] + if (result.status === 'fulfilled') history[key] = Array.isArray(result.value) ? result.value : [] + else { + history[key] = [] + failures.push(`${key}: ${result.reason?.message || '加载失败'}`) + } + }) + historyLoaded.value = true + historyLoading.value = false + if (failures.length) { + historyError.value = new Error(`部分商业历史加载失败：${failures.join('；')}`) + } + return history + } + + async function calculatePricingScenario(rawForm) { + if (!canManage.value) { + const error = new Error('只有平台管理员可以为显式选择的租户计算定价场景。') + error.status = 403 + pricingError.value = error + throw error + } + const requestSeq = ++pricingRequestSeq + pricingLoading.value = true + pricingError.value = null + try { + const payload = buildPricingScenarioPayload(rawForm) + const response = await buildCommercialPricingScenario(tenantId.value, payload) + if (requestSeq === pricingRequestSeq) pricingScenario.value = response + return response + } catch (error) { + if (requestSeq === pricingRequestSeq) pricingError.value = error + throw error + } finally { + if (requestSeq === pricingRequestSeq) pricingLoading.value = false + } + } + + async function executeMutation(kind, rawForm) { + if (!canManage.value) { + const error = new Error('只有平台管理员可以修改商业配置和事实台账。') + error.status = 403 + throw error + } + const payload = buildCommercialMutation(kind, rawForm, account.value) + mutationLoading.value = true + mutationError.value = null + mutationMessage.value = '' + try { + let response + if (kind === 'createPlan') { + response = await createCommercialPlan(tenantId.value, payload) + } else if (kind === 'activatePlan') { + response = await activateCommercialPlan(tenantId.value, payload.resourceId, payload.expectedVersion, payload.reason) + } else if (kind === 'createSubscription') { + response = await createCommercialSubscription(tenantId.value, payload) + } else if (kind === 'activateSubscription') { + response = await activateCommercialSubscription(tenantId.value, payload.resourceId, payload.expectedVersion, payload.reason) + } else if (kind === 'transitionSubscription') { + response = await transitionCommercialSubscription(tenantId.value, payload.resourceId, { + expectedVersion: payload.expectedVersion, + targetStatus: payload.targetStatus, + reason: payload.reason + }) + } else if (kind === 'upsertEntitlement') { + response = await upsertCommercialEntitlement(tenantId.value, payload) + } else if (kind === 'activateEntitlement') { + response = await activateCommercialEntitlement(tenantId.value, payload.resourceId, payload.expectedVersion, payload.reason) + } else if (kind === 'recordUsage') { + response = await recordCommercialUsage(tenantId.value, payload) + } else if (kind === 'recordCost') { + response = await recordCommercialCost(tenantId.value, payload) + } else { + throw new Error('不支持的商业管理操作。') + } + mutationMessage.value = response?.created === false + ? '相同幂等事件已存在，本次返回原记录。' + : '操作已完成，并已刷新商业账户与价值分析。' + await reload() + if (historyLoaded.value) await loadHistory() + return response + } catch (error) { + mutationError.value = error + throw error + } finally { + mutationLoading.value = false + } + } + + watch( + () => [tenantId.value, platformAdmin.value, active.value], + ([nextTenantId, nextPlatformAdmin, nextActive], previous = []) => { + const [previousTenantId, previousPlatformAdmin, previousActive] = previous + if (!nextActive) return + if ( + nextTenantId !== previousTenantId + || nextPlatformAdmin !== previousPlatformAdmin + || nextActive !== previousActive + ) { + pricingRequestSeq += 1 + pricingScenario.value = null + pricingLoading.value = false + pricingError.value = null + historyRequestSeq += 1 + historyLoaded.value = false + historyLoading.value = false + historyError.value = null + Object.keys(history).forEach((key) => { history[key] = [] }) + void reload() + } + }, + { immediate: options.immediate !== false } + ) + + return { + account, + accountLoading, + accountError, + accountState, + analytics, + analyticsLoading, + analyticsError, + mutationLoading, + mutationError, + mutationMessage, + pricingScenario, + pricingScenarioView, + pricingLoading, + pricingError, + history, + historyLoading, + historyError, + historyLoaded, + range, + tenantId, + platformAdmin, + canManage, + quotaRows, + ledgers, + loadAccount, + loadAnalytics, + reload, + loadHistory, + calculatePricingScenario, + executeMutation + } +} diff --git a/web/src/composables/useLoginView.js b/web/src/composables/useLoginView.js index dc48e9c..27c27a6 100644 --- a/web/src/composables/useLoginView.js +++ b/web/src/composables/useLoginView.js @@ -3,7 +3,7 @@ import { h, ref } from 'vue' export function useLoginView() { const username = ref('') const password = ref('') - const tenant = ref('远光软件股份有限公司') + const tenant = ref('default') const remember = ref(true) const showPassword = ref(false) diff --git a/web/src/composables/useOverviewView.js b/web/src/composables/useOverviewView.js index 8331c7f..8fffdcc 100644 --- a/web/src/composables/useOverviewView.js +++ b/web/src/composables/useOverviewView.js @@ -61,6 +61,8 @@ import { export function useOverviewView(options = {}) { const activeDashboardKey = computed(() => { const dashboard = String(options.dashboard || '').trim() + if (dashboard === 'commercial') return 'commercial' + if (dashboard === 'value') return 'value' if (dashboard === 'system') return 'system' if (dashboard === 'risk') return 'risk' if (dashboard === 'digitalEmployee') return 'digitalEmployee' @@ -222,6 +224,10 @@ export function useOverviewView(options = {}) { } const loadActiveDashboard = () => { + if (activeDashboardKey.value === 'commercial' || activeDashboardKey.value === 'value') { + stopRiskDashboardRealtimeRefresh() + return + } if (activeDashboardKey.value === 'system') { void loadSystemDashboard() stopRiskDashboardRealtimeRefresh() diff --git a/web/src/composables/useSystemState.js b/web/src/composables/useSystemState.js index e6e1b9f..7ec3a92 100644 --- a/web/src/composables/useSystemState.js +++ b/web/src/composables/useSystemState.js @@ -93,8 +93,9 @@ function readAuthState() { } function buildAnonymousUser() { - return { - username: '', + return { + username: '', + tenantId: '', name: '', role: '', department: '', @@ -646,10 +647,11 @@ async function handleLogin(credentials) { loginError.value = '' try { - const response = await loginByAccount({ - username: credentials.username, - password: credentials.password - }) + const response = await loginByAccount({ + username: credentials.username, + password: credentials.password, + tenantId: credentials.tenantId + }) const responseUser = normalizeStoredAuthUser(response?.user || buildAnonymousUser()) const responseRoleCodes = responseUser.roleCodes diff --git a/web/src/composables/workbenchAiMode/usePersonalWorkbenchAiMode.js b/web/src/composables/workbenchAiMode/usePersonalWorkbenchAiMode.js index ba026d0..e02d424 100644 --- a/web/src/composables/workbenchAiMode/usePersonalWorkbenchAiMode.js +++ b/web/src/composables/workbenchAiMode/usePersonalWorkbenchAiMode.js @@ -1,4 +1,4 @@ -import { computed, nextTick, onBeforeUnmount, onMounted, ref, watch } from 'vue' +import { computed, onBeforeUnmount, onMounted, ref, watch } from 'vue' import { useSystemState } from '../useSystemState.js' import { useToast } from '../useToast.js' import { useWorkbenchComposerDate } from '../useWorkbenchComposerDate.js' @@ -7,9 +7,7 @@ import { calculateTravelReimbursement, fetchExpenseClaimDetail } from '../../ser import { useApplicationPreviewEditor } from '../../views/scripts/useApplicationPreviewEditor.js' import { deleteAiWorkbenchConversation, - loadAiWorkbenchConversationHistory, - markAiWorkbenchConversationDocumentDeleted, - saveAiWorkbenchConversation + markAiWorkbenchConversationDocumentDeleted } from '../../utils/aiWorkbenchConversationStore.js' import { renderAiConversationHtml } from '../../utils/aiConversationHtmlRenderer.js' import { @@ -41,23 +39,12 @@ import { } from './workbenchAiApplicationGateModel.js' import { useWorkbenchAiCommandIntents } from './useWorkbenchAiCommandIntents.js' import { - buildRuleFallbackWorkbenchAiIntentPlan, - isLowConfidenceTravelApplicationPlan, - normalizeWorkbenchAiIntentPlan, - resolveExecutableTravelApplicationPlan, shouldRequestWorkbenchAiIntentPlan } from './workbenchAiIntentPlannerModel.js' -import { - buildInitialModelPlanningThinkingEvents, - buildModelPlanningProgressSchedule, - mergeWorkbenchAiThinkingEvents -} from './workbenchAiPlanningThinkingModel.js' +import { useWorkbenchAiConversationRuntime } from './useWorkbenchAiConversationRuntime.js' +import { useWorkbenchAiIntentExecution } from './useWorkbenchAiIntentExecution.js' const AI_SEARCH_CONVERSATION_ID = 'ai-search' -const INLINE_ANSWER_STREAM_CHUNK_SIZE = 6 -const INLINE_ANSWER_STREAM_DELAY_MS = 24 -const INLINE_AUTO_SCROLL_THRESHOLD = 96 -const INLINE_LAYOUT_SETTLE_SCROLL_DELAY_MS = 260 export function usePersonalWorkbenchAiMode(props, emit) { const { currentUser } = useSystemState() @@ -126,7 +113,47 @@ export function usePersonalWorkbenchAiMode(props, emit) { handleWorkbenchDateInputChange, removeWorkbenchDateTag, buildWorkbenchPromptText - } = useWorkbenchComposerDate({ draft: assistantDraft, focusInput: focusAiModeInput }) + } = useWorkbenchComposerDate({ draft: assistantDraft, focusInput: () => focusAiModeInput() }) + + const conversationRuntime = useWorkbenchAiConversationRuntime({ + activeConversationTitle, + applicationSubmitConfirmContext, + applicationSubmitConfirmOpen, + assistantDraft, + assistantInputRef, + attachmentOcrExpandedMessageIds, + clearAiModeFiles: () => filesFlow.clearAiModeFiles(), + clearWorkbenchDateSelection, + conversationId, + conversationMessages, + conversationScrollRef, + conversationStarted, + currentUser, + deleteDialogOpen, + emit, + inlineConversationAutoScrollPinned, + searchConversationId: AI_SEARCH_CONVERSATION_ID, + serializeRuntimeMessage, + stewardState, + thinkingCollapsedMessageIds, + thinkingExpandedMessageIds + }) + const { + activateInlineConversation, + appendInlineMessageContent, + focusAiModeInput, + handleInlineConversationScroll, + persistCurrentConversation, + refreshConversationHistory, + replaceInlineMessage, + resetInlineConversationState, + scrollInlineConversationToBottom, + scrollInlineConversationToTop, + setAssistantInputRef, + streamInlineAssistantContent, + streamOrSetInlineAssistantContent, + updateInlineMessageContent + } = conversationRuntime const aiModeActionItems = AI_MODE_ACTION_ITEMS const displayUserName = computed(() => { @@ -225,6 +252,8 @@ export function usePersonalWorkbenchAiMode(props, emit) { thinkingExpandedMessageIds }) + let intentExecution = null + const applicationFlow = useWorkbenchAiApplicationPreviewFlow({ activateInlineConversation, applicationPreviewEditor, @@ -259,7 +288,7 @@ export function usePersonalWorkbenchAiMode(props, emit) { scrollInlineConversationToBottom, sending, toast, - onApplicationActionCompleted: startModelPlannedNextTask + onApplicationActionCompleted: (remainingTasks) => intentExecution?.startModelPlannedNextTask(remainingTasks) }) const expenseFlow = useWorkbenchAiExpenseFlow({ @@ -354,6 +383,29 @@ export function usePersonalWorkbenchAiMode(props, emit) { toast }) + intentExecution = useWorkbenchAiIntentExecution({ + actionRouter, + activeConversationTitle, + activateInlineConversation, + applicationFlow, + assistantDraft, + clearAiModeFiles: filesFlow.clearAiModeFiles, + closeWorkbenchDatePicker, + conversationId, + conversationMessages, + createInlineMessage, + expenseFlow, + inlineConversationAutoScrollPinned, + persistCurrentConversation, + removeWorkbenchDateTag, + replaceInlineMessage, + resolveInlineThinkingEvents, + scrollInlineConversationToBottom, + searchConversationId: AI_SEARCH_CONVERSATION_ID, + sending, + stewardFlow + }) + const applicationPreviewEstimatePending = computed(() => ( conversationMessages.value.some((message) => applicationFlow.isApplicationPreviewEstimatePending(message)) )) @@ -378,187 +430,6 @@ export function usePersonalWorkbenchAiMode(props, emit) { } } - function focusAiModeInput() { - nextTick(() => { - assistantInputRef.value?.focus() - }) - } - - function setAssistantInputRef(element) { - assistantInputRef.value = element - } - - function isInlineConversationNearBottom() { - const el = conversationScrollRef.value - if (!el) { - return true - } - return el.scrollHeight - el.clientHeight - el.scrollTop <= INLINE_AUTO_SCROLL_THRESHOLD - } - - function handleInlineConversationScroll() { - inlineConversationAutoScrollPinned.value = isInlineConversationNearBottom() - } - - function forceInlineConversationToBottom() { - const el = conversationScrollRef.value - if (el) { - el.scrollTop = el.scrollHeight - inlineConversationAutoScrollPinned.value = true - } - } - - function scrollInlineConversationToBottom(options = {}) { - const shouldScroll = options.force !== false - nextTick(() => { - if (!shouldScroll) { - return - } - forceInlineConversationToBottom() - window.requestAnimationFrame(() => { - forceInlineConversationToBottom() - }) - window.setTimeout(() => { - if (inlineConversationAutoScrollPinned.value) { - forceInlineConversationToBottom() - } - }, INLINE_LAYOUT_SETTLE_SCROLL_DELAY_MS) - }) - } - - function scrollInlineConversationToTop() { - nextTick(() => { - const el = conversationScrollRef.value - if (el) { - inlineConversationAutoScrollPinned.value = false - el.scrollTo({ top: 0, behavior: 'smooth' }) - } - }) - } - - function updateInlineMessageContent(message, content) { - if (!message) { - return - } - message.content = String(content || '') - message.paragraphs = String(message.content || '') - .split(/\n{2,}|\n/) - .map((item) => item.trim()) - .filter(Boolean) - } - - function appendInlineMessageContent(message, delta) { - const nextDelta = String(delta || '') - if (!nextDelta) { - return - } - updateInlineMessageContent(message, `${message.content || ''}${nextDelta}`) - } - - function waitInlineAnswerStreamFrame() { - return new Promise((resolve) => { - window.setTimeout(resolve, INLINE_ANSWER_STREAM_DELAY_MS) - }) - } - - async function streamInlineAssistantContent(messageId, content) { - const targetContent = String(content || '').trim() - let streamedContent = '' - - for (let index = 0; index < targetContent.length; index += INLINE_ANSWER_STREAM_CHUNK_SIZE) { - const message = conversationMessages.value.find((item) => item.id === messageId) - if (!message || !message.pending) { - return - } - const shouldAutoScroll = inlineConversationAutoScrollPinned.value - streamedContent += targetContent.slice(index, index + INLINE_ANSWER_STREAM_CHUNK_SIZE) - updateInlineMessageContent(message, streamedContent) - scrollInlineConversationToBottom({ force: shouldAutoScroll }) - await waitInlineAnswerStreamFrame() - } - } - - async function streamOrSetInlineAssistantContent(messageId, content) { - const targetContent = String(content || '').trim() - if (//.test(targetContent)) { - const message = conversationMessages.value.find((item) => item.id === messageId) - if (message?.pending) { - updateInlineMessageContent(message, targetContent) - scrollInlineConversationToBottom({ force: inlineConversationAutoScrollPinned.value }) - } - return - } - await streamInlineAssistantContent(messageId, targetContent) - } - - function refreshConversationHistory() { - const history = loadAiWorkbenchConversationHistory(currentUser.value || {}) - emit('conversation-history-change', history) - return history - } - - function isPersistableInlineConversation() { - return Boolean( - conversationId.value && - conversationId.value !== AI_SEARCH_CONVERSATION_ID && - conversationMessages.value.length - ) - } - - function persistCurrentConversation() { - if (!isPersistableInlineConversation()) { - refreshConversationHistory() - return [] - } - - const history = saveAiWorkbenchConversation(currentUser.value || {}, { - id: conversationId.value, - conversationId: conversationId.value, - title: activeConversationTitle.value, - source: 'workbench', - sessionType: 'steward', - stewardState: stewardState.value, - messages: conversationMessages.value.map((message) => serializeRuntimeMessage(message)) - }) - emit('conversation-history-change', history) - return history - } - - function resetInlineConversationState() { - conversationStarted.value = false - conversationMessages.value = [] - conversationId.value = '' - stewardState.value = null - activeConversationTitle.value = '' - assistantDraft.value = '' - thinkingExpandedMessageIds.value = new Set() - thinkingCollapsedMessageIds.value = new Set() - attachmentOcrExpandedMessageIds.value = new Set() - deleteDialogOpen.value = false - applicationSubmitConfirmOpen.value = false - applicationSubmitConfirmContext.value = null - clearWorkbenchDateSelection() - filesFlow.clearAiModeFiles() - } - - function replaceInlineMessage(id, nextMessage) { - const index = conversationMessages.value.findIndex((item) => item.id === id) - if (index === -1) { - conversationMessages.value.push(nextMessage) - return - } - conversationMessages.value.splice(index, 1, nextMessage) - } - - function activateInlineConversation(options = {}) { - conversationStarted.value = true - if (!conversationId.value) { - conversationId.value = options.id || `inline-${Date.now()}` - } - activeConversationTitle.value = options.title || activeConversationTitle.value || '新对话' - emit('conversation-change', { id: conversationId.value, title: activeConversationTitle.value }) - } - function renderInlineConversationHtml(content) { return renderAiConversationHtml(content) } function isUnavailableDocumentDetailError(error) { @@ -648,238 +519,6 @@ export function usePersonalWorkbenchAiMode(props, emit) { return false } - function isModelPlannedReimbursementTask(modelPlan = {}) { - const tasks = Array.isArray(modelPlan?.tasks) ? modelPlan.tasks : [] - return tasks.some((task) => { - const taskType = String(task?.task_type || task?.taskType || '').trim() - const assignedAgent = String(task?.assigned_agent || task?.assignedAgent || '').trim() - return taskType === 'reimbursement' || assignedAgent === 'reimbursement_assistant' - }) - } - - function updateModelPlanningThinkingEvent(messageId, event) { - const message = conversationMessages.value.find((item) => item.id === messageId) - if (!message) { - return - } - const currentPlan = message.stewardPlan || {} - message.stewardPlan = { - ...currentPlan, - streamStatus: 'streaming', - thinkingEvents: mergeWorkbenchAiThinkingEvents(resolveInlineThinkingEvents(message), [event]) - } - persistCurrentConversation() - scrollInlineConversationToBottom({ force: inlineConversationAutoScrollPinned.value }) - } - - function startModelPlanningProgressUpdates(messageId) { - const timerIds = buildModelPlanningProgressSchedule().map(({ delayMs, event }) => ( - globalThis.setTimeout(() => { - updateModelPlanningThinkingEvent(messageId, event) - }, delayMs) - )) - return () => { - timerIds.forEach((timerId) => globalThis.clearTimeout(timerId)) - } - } - - function startModelPlanningConversation(cleanPrompt, entry = {}) { - if (conversationId.value === AI_SEARCH_CONVERSATION_ID) { - conversationId.value = '' - conversationMessages.value = [] - activeConversationTitle.value = '' - } - activateInlineConversation({ - title: entry.label || cleanPrompt.slice(0, 18) || '新对话' - }) - inlineConversationAutoScrollPinned.value = true - conversationMessages.value.push(createInlineMessage('user', cleanPrompt)) - assistantDraft.value = '' - removeWorkbenchDateTag() - closeWorkbenchDatePicker() - filesFlow.clearAiModeFiles() - const pendingMessage = createInlineMessage('assistant', '正在识别意图，准备拆解申请、报销和附件任务。', { - pending: true, - stewardPlan: { - streamStatus: 'streaming', - thinkingEvents: buildInitialModelPlanningThinkingEvents() - } - }) - conversationMessages.value.push(pendingMessage) - scrollInlineConversationToBottom() - persistCurrentConversation() - return pendingMessage - } - - function buildModelPlannedNextTaskAction(remainingTasks = []) { - const tasks = Array.isArray(remainingTasks) ? remainingTasks : [] - const nextTask = tasks[0] - if (!nextTask || typeof nextTask !== 'object') { - return null - } - const taskType = String(nextTask.task_type || nextTask.taskType || '').trim() - const assignedAgent = String(nextTask.assigned_agent || nextTask.assignedAgent || '').trim() - const isApplication = taskType === 'expense_application' || assignedAgent === 'application_assistant' - const isReimbursement = taskType === 'reimbursement' || assignedAgent === 'reimbursement_assistant' - if (!isApplication && !isReimbursement) { - return null - } - const ontologyFields = nextTask.ontology_fields || nextTask.ontologyFields || {} - const flowId = isApplication ? 'travel_application' : 'travel_reimbursement' - const taskLabel = isApplication ? '出差申请' : '费用报销' - return { - label: `继续处理${taskLabel}`, - action_type: 'steward_continue_next_task', - payload: { - steward_confirm_flow: true, - flow_id: flowId, - steward_current_task: nextTask, - expense_type: String(ontologyFields.expense_type || 'travel').trim() || 'travel', - expense_type_label: String(ontologyFields.expense_type_label || '差旅费').trim() || '差旅费', - ontology_fields: ontologyFields, - original_message: String(nextTask.summary || nextTask.title || `继续处理${taskLabel}`).trim(), - steward_remaining_tasks: tasks.slice(1) - } - } - } - - function startModelPlannedNextTask(remainingTasks = []) { - const nextTaskAction = buildModelPlannedNextTaskAction(remainingTasks) - if (!nextTaskAction) { - return - } - actionRouter.handleInlineSuggestedAction(nextTaskAction) - } - - function startModelPlannedApplicationPreview(travelApplicationRequest, plannerPendingMessage = null) { - void applicationFlow.startAiApplicationPreview( - travelApplicationRequest.expenseType, - travelApplicationRequest.expenseTypeLabel, - travelApplicationRequest.sourceText, - { - userMessage: travelApplicationRequest.sourceText, - pushUserMessage: !plannerPendingMessage, - pendingMessageId: plannerPendingMessage?.id, - ontologyFields: travelApplicationRequest.ontologyFields, - autoSubmit: travelApplicationRequest.autoSubmit, - autoSaveDraft: travelApplicationRequest.autoSaveDraft, - requestedSubmit: travelApplicationRequest.requestedSubmit, - submitRequiresConfirmation: travelApplicationRequest.submitRequiresConfirmation, - stewardRemainingTasks: travelApplicationRequest.stewardRemainingTasks, - onPreviewReadyForNextTask: startModelPlannedNextTask, - onApplicationActionCompleted: startModelPlannedNextTask - } - ) - } - - function startModelPlannedTravelApplicationConfirmation(travelApplicationRequest, plan, plannerPendingMessage) { - const confirmText = buildLowConfidenceTravelApplicationConfirmationText(travelApplicationRequest, plan) - const confirmAction = { - label: '确认发起出差申请', - description: '根据上面识别到的信息生成出差申请预览。', - icon: 'mdi mdi-check-circle-outline', - action_type: 'ai_application_confirm_intent', - payload: { - ontologyFields: travelApplicationRequest.ontologyFields, - sourceText: travelApplicationRequest.sourceText, - autoSubmit: travelApplicationRequest.autoSubmit, - autoSaveDraft: travelApplicationRequest.autoSaveDraft, - requestedSubmit: travelApplicationRequest.requestedSubmit, - submitRequiresConfirmation: travelApplicationRequest.submitRequiresConfirmation, - stewardRemainingTasks: travelApplicationRequest.stewardRemainingTasks - } - } - replaceInlineMessage(plannerPendingMessage.id, createInlineMessage('assistant', confirmText, { - id: plannerPendingMessage.id, - suggestedActions: [confirmAction], - stewardPlan: { - streamStatus: 'completed', - thinkingEvents: resolveInlineThinkingEvents(plannerPendingMessage).map((item) => ({ ...item, status: 'completed' })) - } - })) - persistCurrentConversation() - scrollInlineConversationToBottom({ force: inlineConversationAutoScrollPinned.value }) - } - - function buildLowConfidenceTravelApplicationConfirmationText(request, plan) { - const fields = request.ontologyFields || {} - const summaryParts = [] - if (fields.time_range) { - summaryParts.push(`时间：${fields.time_range}`) - } - if (fields.location) { - summaryParts.push(`地点：${fields.location}`) - } - if (fields.reason) { - summaryParts.push(`事由：${fields.reason}`) - } - if (fields.transport_mode) { - summaryParts.push(`交通：${fields.transport_mode}`) - } - const summary = summaryParts.length ? `\n\n${summaryParts.join('；')}` : '' - const confidenceNote = Number.isFinite(Number(plan?.confidence)) - ? `（模型识别置信度较低，约 ${Math.round(Number(plan.confidence) * 100)}%）` - : '（模型识别置信度较低）' - return [ - '### 需要确认：您是要发起出差申请吗？', - '', - `小财管家把这句话理解成了“发起差旅申请”${confidenceNote}，为避免误操作，先请您确认。`, - summary, - '', - '点击下方「确认发起出差申请」即可继续；如果理解有误，请补充说明您的实际需求。' - ].filter(Boolean).join('\n') - } - - async function executeModelPlannedWorkbenchIntent(cleanPrompt, entry = {}, files = []) { - let intentPlan = null - let modelPlan = null - const plannerPendingMessage = startModelPlanningConversation(cleanPrompt, entry) - const stopPlanningProgressUpdates = startModelPlanningProgressUpdates(plannerPendingMessage.id) - sending.value = true - try { - modelPlan = await stewardFlow.resolveInlineExecutionPlan(cleanPrompt, entry, files, { - pendingMessageId: plannerPendingMessage.id - }) - intentPlan = normalizeWorkbenchAiIntentPlan(modelPlan, { prompt: cleanPrompt }) - } catch (error) { - console.warn('AI mode intent planner failed, using local fallback:', error) - const rulePlan = buildRuleFallbackWorkbenchAiIntentPlan(cleanPrompt) - const ruleRequest = resolveExecutableTravelApplicationPlan(rulePlan) - if (ruleRequest) { - sending.value = false - startModelPlannedApplicationPreview(ruleRequest, plannerPendingMessage) - return - } - } finally { - stopPlanningProgressUpdates() - sending.value = false - } - - const travelApplicationRequest = resolveExecutableTravelApplicationPlan(intentPlan) - if (travelApplicationRequest) { - if (isLowConfidenceTravelApplicationPlan(intentPlan)) { - startModelPlannedTravelApplicationConfirmation(travelApplicationRequest, intentPlan, plannerPendingMessage) - return - } - startModelPlannedApplicationPreview(travelApplicationRequest, plannerPendingMessage) - return - } - - if (isModelPlannedReimbursementTask(modelPlan) || isReimbursementCreationIntent(cleanPrompt)) { - replaceInlineMessage(plannerPendingMessage.id, createInlineMessage('assistant', '已识别为报销任务，正在进入报销流程。', { - id: plannerPendingMessage.id, - stewardPlan: { - streamStatus: 'completed', - thinkingEvents: resolveInlineThinkingEvents(plannerPendingMessage).map((item) => ({ ...item, status: 'completed' })) - } - })) - void expenseFlow.startAiReimbursementAssociationGate(cleanPrompt, entry.label || cleanPrompt) - return - } - - void stewardFlow.requestInlineAssistantReply(cleanPrompt, entry, files, { pendingMessageId: plannerPendingMessage.id }) - } - async function handleAiAnswerMarkdownClick(event) { const target = event?.target const link = target?.closest?.('a[href^="#ai-open-document-detail:"], a[href^="#ai-open-application-detail:"]') @@ -923,7 +562,7 @@ export function usePersonalWorkbenchAiMode(props, emit) { } if (shouldRequestWorkbenchAiIntentPlan(cleanPrompt)) { - void executeModelPlannedWorkbenchIntent(cleanPrompt, entry, files) + void intentExecution.executeModelPlannedWorkbenchIntent(cleanPrompt, entry, files) return } diff --git a/web/src/composables/workbenchAiMode/useWorkbenchAiConversationRuntime.js b/web/src/composables/workbenchAiMode/useWorkbenchAiConversationRuntime.js new file mode 100644 index 0000000..5b736f2 --- /dev/null +++ b/web/src/composables/workbenchAiMode/useWorkbenchAiConversationRuntime.js @@ -0,0 +1,234 @@ +import { nextTick } from 'vue' +import { + loadAiWorkbenchConversationHistory, + saveAiWorkbenchConversation +} from '../../utils/aiWorkbenchConversationStore.js' + +const INLINE_ANSWER_STREAM_CHUNK_SIZE = 6 +const INLINE_ANSWER_STREAM_DELAY_MS = 24 +const INLINE_AUTO_SCROLL_THRESHOLD = 96 +const INLINE_LAYOUT_SETTLE_SCROLL_DELAY_MS = 260 + +export function useWorkbenchAiConversationRuntime(options) { + const { + activeConversationTitle, + applicationSubmitConfirmContext, + applicationSubmitConfirmOpen, + assistantDraft, + assistantInputRef, + attachmentOcrExpandedMessageIds, + clearAiModeFiles, + clearWorkbenchDateSelection, + conversationId, + conversationMessages, + conversationScrollRef, + conversationStarted, + currentUser, + deleteDialogOpen, + emit, + inlineConversationAutoScrollPinned, + searchConversationId, + serializeRuntimeMessage, + stewardState, + thinkingCollapsedMessageIds, + thinkingExpandedMessageIds + } = options + + function focusAiModeInput() { + nextTick(() => { + assistantInputRef.value?.focus() + }) + } + + function setAssistantInputRef(element) { + assistantInputRef.value = element + } + + function isInlineConversationNearBottom() { + const el = conversationScrollRef.value + if (!el) { + return true + } + return el.scrollHeight - el.clientHeight - el.scrollTop <= INLINE_AUTO_SCROLL_THRESHOLD + } + + function handleInlineConversationScroll() { + inlineConversationAutoScrollPinned.value = isInlineConversationNearBottom() + } + + function forceInlineConversationToBottom() { + const el = conversationScrollRef.value + if (el) { + el.scrollTop = el.scrollHeight + inlineConversationAutoScrollPinned.value = true + } + } + + function scrollInlineConversationToBottom(options = {}) { + const shouldScroll = options.force !== false + nextTick(() => { + if (!shouldScroll) { + return + } + forceInlineConversationToBottom() + window.requestAnimationFrame(() => { + forceInlineConversationToBottom() + }) + window.setTimeout(() => { + if (inlineConversationAutoScrollPinned.value) { + forceInlineConversationToBottom() + } + }, INLINE_LAYOUT_SETTLE_SCROLL_DELAY_MS) + }) + } + + function scrollInlineConversationToTop() { + nextTick(() => { + const el = conversationScrollRef.value + if (el) { + inlineConversationAutoScrollPinned.value = false + el.scrollTo({ top: 0, behavior: 'smooth' }) + } + }) + } + + function updateInlineMessageContent(message, content) { + if (!message) { + return + } + message.content = String(content || '') + message.paragraphs = String(message.content || '') + .split(/\n{2,}|\n/) + .map((item) => item.trim()) + .filter(Boolean) + } + + function appendInlineMessageContent(message, delta) { + const nextDelta = String(delta || '') + if (!nextDelta) { + return + } + updateInlineMessageContent(message, `${message.content || ''}${nextDelta}`) + } + + function waitInlineAnswerStreamFrame() { + return new Promise((resolve) => { + window.setTimeout(resolve, INLINE_ANSWER_STREAM_DELAY_MS) + }) + } + + async function streamInlineAssistantContent(messageId, content) { + const targetContent = String(content || '').trim() + let streamedContent = '' + + for (let index = 0; index < targetContent.length; index += INLINE_ANSWER_STREAM_CHUNK_SIZE) { + const message = conversationMessages.value.find((item) => item.id === messageId) + if (!message || !message.pending) { + return + } + const shouldAutoScroll = inlineConversationAutoScrollPinned.value + streamedContent += targetContent.slice(index, index + INLINE_ANSWER_STREAM_CHUNK_SIZE) + updateInlineMessageContent(message, streamedContent) + scrollInlineConversationToBottom({ force: shouldAutoScroll }) + await waitInlineAnswerStreamFrame() + } + } + + async function streamOrSetInlineAssistantContent(messageId, content) { + const targetContent = String(content || '').trim() + if (//.test(targetContent)) { + const message = conversationMessages.value.find((item) => item.id === messageId) + if (message?.pending) { + updateInlineMessageContent(message, targetContent) + scrollInlineConversationToBottom({ force: inlineConversationAutoScrollPinned.value }) + } + return + } + await streamInlineAssistantContent(messageId, targetContent) + } + + function refreshConversationHistory() { + const history = loadAiWorkbenchConversationHistory(currentUser.value || {}) + emit('conversation-history-change', history) + return history + } + + function isPersistableInlineConversation() { + return Boolean( + conversationId.value && + conversationId.value !== searchConversationId && + conversationMessages.value.length + ) + } + + function persistCurrentConversation() { + if (!isPersistableInlineConversation()) { + refreshConversationHistory() + return [] + } + + const history = saveAiWorkbenchConversation(currentUser.value || {}, { + id: conversationId.value, + conversationId: conversationId.value, + title: activeConversationTitle.value, + source: 'workbench', + sessionType: 'steward', + stewardState: stewardState.value, + messages: conversationMessages.value.map((message) => serializeRuntimeMessage(message)) + }) + emit('conversation-history-change', history) + return history + } + + function resetInlineConversationState() { + conversationStarted.value = false + conversationMessages.value = [] + conversationId.value = '' + stewardState.value = null + activeConversationTitle.value = '' + assistantDraft.value = '' + thinkingExpandedMessageIds.value = new Set() + thinkingCollapsedMessageIds.value = new Set() + attachmentOcrExpandedMessageIds.value = new Set() + deleteDialogOpen.value = false + applicationSubmitConfirmOpen.value = false + applicationSubmitConfirmContext.value = null + clearWorkbenchDateSelection() + clearAiModeFiles() + } + + function replaceInlineMessage(id, nextMessage) { + const index = conversationMessages.value.findIndex((item) => item.id === id) + if (index === -1) { + conversationMessages.value.push(nextMessage) + return + } + conversationMessages.value.splice(index, 1, nextMessage) + } + + function activateInlineConversation(options = {}) { + conversationStarted.value = true + if (!conversationId.value) { + conversationId.value = options.id || `inline-${Date.now()}` + } + activeConversationTitle.value = options.title || activeConversationTitle.value || '新对话' + emit('conversation-change', { id: conversationId.value, title: activeConversationTitle.value }) + } + + return { + activateInlineConversation, + appendInlineMessageContent, + focusAiModeInput, + handleInlineConversationScroll, + persistCurrentConversation, + refreshConversationHistory, + replaceInlineMessage, + resetInlineConversationState, + scrollInlineConversationToBottom, + scrollInlineConversationToTop, + setAssistantInputRef, + streamInlineAssistantContent, + streamOrSetInlineAssistantContent, + updateInlineMessageContent + } +} diff --git a/web/src/composables/workbenchAiMode/useWorkbenchAiIntentExecution.js b/web/src/composables/workbenchAiMode/useWorkbenchAiIntentExecution.js new file mode 100644 index 0000000..eff10d4 --- /dev/null +++ b/web/src/composables/workbenchAiMode/useWorkbenchAiIntentExecution.js @@ -0,0 +1,277 @@ +import { isReimbursementCreationIntent } from './workbenchAiApplicationGateModel.js' +import { + buildRuleFallbackWorkbenchAiIntentPlan, + isLowConfidenceTravelApplicationPlan, + normalizeWorkbenchAiIntentPlan, + resolveExecutableTravelApplicationPlan +} from './workbenchAiIntentPlannerModel.js' +import { + buildInitialModelPlanningThinkingEvents, + buildModelPlanningProgressSchedule, + mergeWorkbenchAiThinkingEvents +} from './workbenchAiPlanningThinkingModel.js' + +export function useWorkbenchAiIntentExecution(options) { + const { + actionRouter, + activeConversationTitle, + activateInlineConversation, + applicationFlow, + assistantDraft, + clearAiModeFiles, + closeWorkbenchDatePicker, + conversationId, + conversationMessages, + createInlineMessage, + expenseFlow, + inlineConversationAutoScrollPinned, + persistCurrentConversation, + removeWorkbenchDateTag, + replaceInlineMessage, + resolveInlineThinkingEvents, + scrollInlineConversationToBottom, + searchConversationId, + sending, + stewardFlow + } = options + + function isModelPlannedReimbursementTask(modelPlan = {}) { + const tasks = Array.isArray(modelPlan?.tasks) ? modelPlan.tasks : [] + return tasks.some((task) => { + const taskType = String(task?.task_type || task?.taskType || '').trim() + const assignedAgent = String(task?.assigned_agent || task?.assignedAgent || '').trim() + return taskType === 'reimbursement' || assignedAgent === 'reimbursement_assistant' + }) + } + + function updateModelPlanningThinkingEvent(messageId, event) { + const message = conversationMessages.value.find((item) => item.id === messageId) + if (!message) { + return + } + const currentPlan = message.stewardPlan || {} + message.stewardPlan = { + ...currentPlan, + streamStatus: 'streaming', + thinkingEvents: mergeWorkbenchAiThinkingEvents(resolveInlineThinkingEvents(message), [event]) + } + persistCurrentConversation() + scrollInlineConversationToBottom({ force: inlineConversationAutoScrollPinned.value }) + } + + function startModelPlanningProgressUpdates(messageId) { + const timerIds = buildModelPlanningProgressSchedule().map(({ delayMs, event }) => ( + globalThis.setTimeout(() => { + updateModelPlanningThinkingEvent(messageId, event) + }, delayMs) + )) + return () => { + timerIds.forEach((timerId) => globalThis.clearTimeout(timerId)) + } + } + + function startModelPlanningConversation(cleanPrompt, entry = {}) { + if (conversationId.value === searchConversationId) { + conversationId.value = '' + conversationMessages.value = [] + activeConversationTitle.value = '' + } + activateInlineConversation({ + title: entry.label || cleanPrompt.slice(0, 18) || '新对话' + }) + inlineConversationAutoScrollPinned.value = true + conversationMessages.value.push(createInlineMessage('user', cleanPrompt)) + assistantDraft.value = '' + removeWorkbenchDateTag() + closeWorkbenchDatePicker() + clearAiModeFiles() + const pendingMessage = createInlineMessage('assistant', '正在识别意图，准备拆解申请、报销和附件任务。', { + pending: true, + stewardPlan: { + streamStatus: 'streaming', + thinkingEvents: buildInitialModelPlanningThinkingEvents() + } + }) + conversationMessages.value.push(pendingMessage) + scrollInlineConversationToBottom() + persistCurrentConversation() + return pendingMessage + } + + function buildModelPlannedNextTaskAction(remainingTasks = []) { + const tasks = Array.isArray(remainingTasks) ? remainingTasks : [] + const nextTask = tasks[0] + if (!nextTask || typeof nextTask !== 'object') { + return null + } + const taskType = String(nextTask.task_type || nextTask.taskType || '').trim() + const assignedAgent = String(nextTask.assigned_agent || nextTask.assignedAgent || '').trim() + const isApplication = taskType === 'expense_application' || assignedAgent === 'application_assistant' + const isReimbursement = taskType === 'reimbursement' || assignedAgent === 'reimbursement_assistant' + if (!isApplication && !isReimbursement) { + return null + } + const ontologyFields = nextTask.ontology_fields || nextTask.ontologyFields || {} + const flowId = isApplication ? 'travel_application' : 'travel_reimbursement' + const taskLabel = isApplication ? '出差申请' : '费用报销' + return { + label: `继续处理${taskLabel}`, + action_type: 'steward_continue_next_task', + payload: { + steward_confirm_flow: true, + flow_id: flowId, + steward_current_task: nextTask, + expense_type: String(ontologyFields.expense_type || 'travel').trim() || 'travel', + expense_type_label: String(ontologyFields.expense_type_label || '差旅费').trim() || '差旅费', + ontology_fields: ontologyFields, + original_message: String(nextTask.summary || nextTask.title || `继续处理${taskLabel}`).trim(), + steward_remaining_tasks: tasks.slice(1) + } + } + } + + function startModelPlannedNextTask(remainingTasks = []) { + const nextTaskAction = buildModelPlannedNextTaskAction(remainingTasks) + if (nextTaskAction) { + actionRouter.handleInlineSuggestedAction(nextTaskAction) + } + } + + function startModelPlannedApplicationPreview(travelApplicationRequest, plannerPendingMessage = null) { + void applicationFlow.startAiApplicationPreview( + travelApplicationRequest.expenseType, + travelApplicationRequest.expenseTypeLabel, + travelApplicationRequest.sourceText, + { + userMessage: travelApplicationRequest.sourceText, + pushUserMessage: !plannerPendingMessage, + pendingMessageId: plannerPendingMessage?.id, + ontologyFields: travelApplicationRequest.ontologyFields, + autoSubmit: travelApplicationRequest.autoSubmit, + autoSaveDraft: travelApplicationRequest.autoSaveDraft, + requestedSubmit: travelApplicationRequest.requestedSubmit, + submitRequiresConfirmation: travelApplicationRequest.submitRequiresConfirmation, + stewardRemainingTasks: travelApplicationRequest.stewardRemainingTasks, + onPreviewReadyForNextTask: startModelPlannedNextTask, + onApplicationActionCompleted: startModelPlannedNextTask + } + ) + } + + function buildLowConfidenceTravelApplicationConfirmationText(request, plan) { + const fields = request.ontologyFields || {} + const summaryParts = [] + if (fields.time_range) summaryParts.push(`时间：${fields.time_range}`) + if (fields.location) summaryParts.push(`地点：${fields.location}`) + if (fields.reason) summaryParts.push(`事由：${fields.reason}`) + if (fields.transport_mode) summaryParts.push(`交通：${fields.transport_mode}`) + const summary = summaryParts.length ? `\n\n${summaryParts.join('；')}` : '' + const confidenceNote = Number.isFinite(Number(plan?.confidence)) + ? `（模型识别置信度较低，约 ${Math.round(Number(plan.confidence) * 100)}%）` + : '（模型识别置信度较低）' + return [ + '### 需要确认：您是要发起出差申请吗？', + '', + `小财管家把这句话理解成了“发起差旅申请”${confidenceNote}，为避免误操作，先请您确认。`, + summary, + '', + '点击下方「确认发起出差申请」即可继续；如果理解有误，请补充说明您的实际需求。' + ].filter(Boolean).join('\n') + } + + function startModelPlannedTravelApplicationConfirmation(travelApplicationRequest, plan, plannerPendingMessage) { + const confirmAction = { + label: '确认发起出差申请', + description: '根据上面识别到的信息生成出差申请预览。', + icon: 'mdi mdi-check-circle-outline', + action_type: 'ai_application_confirm_intent', + payload: { + ontologyFields: travelApplicationRequest.ontologyFields, + sourceText: travelApplicationRequest.sourceText, + autoSubmit: travelApplicationRequest.autoSubmit, + autoSaveDraft: travelApplicationRequest.autoSaveDraft, + requestedSubmit: travelApplicationRequest.requestedSubmit, + submitRequiresConfirmation: travelApplicationRequest.submitRequiresConfirmation, + stewardRemainingTasks: travelApplicationRequest.stewardRemainingTasks + } + } + replaceInlineMessage(plannerPendingMessage.id, createInlineMessage( + 'assistant', + buildLowConfidenceTravelApplicationConfirmationText(travelApplicationRequest, plan), + { + id: plannerPendingMessage.id, + suggestedActions: [confirmAction], + stewardPlan: { + streamStatus: 'completed', + thinkingEvents: resolveInlineThinkingEvents(plannerPendingMessage) + .map((item) => ({ ...item, status: 'completed' })) + } + } + )) + persistCurrentConversation() + scrollInlineConversationToBottom({ force: inlineConversationAutoScrollPinned.value }) + } + + async function executeModelPlannedWorkbenchIntent(cleanPrompt, entry = {}, files = []) { + let intentPlan = null + let modelPlan = null + const plannerPendingMessage = startModelPlanningConversation(cleanPrompt, entry) + const stopPlanningProgressUpdates = startModelPlanningProgressUpdates(plannerPendingMessage.id) + sending.value = true + try { + modelPlan = await stewardFlow.resolveInlineExecutionPlan(cleanPrompt, entry, files, { + pendingMessageId: plannerPendingMessage.id + }) + intentPlan = normalizeWorkbenchAiIntentPlan(modelPlan, { prompt: cleanPrompt }) + } catch (error) { + console.warn('AI mode intent planner failed, using local fallback:', error) + const ruleRequest = resolveExecutableTravelApplicationPlan( + buildRuleFallbackWorkbenchAiIntentPlan(cleanPrompt) + ) + if (ruleRequest) { + sending.value = false + startModelPlannedApplicationPreview(ruleRequest, plannerPendingMessage) + return + } + } finally { + stopPlanningProgressUpdates() + sending.value = false + } + + const travelApplicationRequest = resolveExecutableTravelApplicationPlan(intentPlan) + if (travelApplicationRequest) { + if (isLowConfidenceTravelApplicationPlan(intentPlan)) { + startModelPlannedTravelApplicationConfirmation(travelApplicationRequest, intentPlan, plannerPendingMessage) + } else { + startModelPlannedApplicationPreview(travelApplicationRequest, plannerPendingMessage) + } + return + } + + if (isModelPlannedReimbursementTask(modelPlan) || isReimbursementCreationIntent(cleanPrompt)) { + replaceInlineMessage(plannerPendingMessage.id, createInlineMessage( + 'assistant', + '已识别为报销任务，正在进入报销流程。', + { + id: plannerPendingMessage.id, + stewardPlan: { + streamStatus: 'completed', + thinkingEvents: resolveInlineThinkingEvents(plannerPendingMessage) + .map((item) => ({ ...item, status: 'completed' })) + } + } + )) + void expenseFlow.startAiReimbursementAssociationGate(cleanPrompt, entry.label || cleanPrompt) + return + } + + void stewardFlow.requestInlineAssistantReply(cleanPrompt, entry, files, { + pendingMessageId: plannerPendingMessage.id + }) + } + + return { + executeModelPlannedWorkbenchIntent, + startModelPlannedNextTask + } +} diff --git a/web/src/services/agentAssets.js b/web/src/services/agentAssets.js index 17a8af5..c23df4d 100644 --- a/web/src/services/agentAssets.js +++ b/web/src/services/agentAssets.js @@ -265,6 +265,99 @@ export function publishRiskRuleAsset(assetId, options = {}) { }) } +export function fetchAgentAssetReleaseState(assetId) { + return apiRequest(`/agent-assets/${assetId}/release`) +} + +const RELEASE_REVIEW_METRIC_KEYS = [ + 'observed_count', + 'runtime_failure_count', + 'runtime_failure_rate', + 'negative_sample_count', + 'negative_labeled_count', + 'negative_pending_label_count', + 'false_negative_count', + 'estimated_false_negative_count', + 'false_negative_upper_bound', + 'random_negative_population_count', + 'random_negative_sample_count', + 'random_negative_labeled_count', + 'recall', + 'recall_lower_bound', + 'recall_confidence_level', + 'recall_method', + 'negative_ground_truth_status' +] +const RELEASE_REVIEW_LABELS = new Set(['risk_present', 'risk_absent']) + +export function normalizeAgentAssetReleaseReviewQueue(payload = {}) { + const metrics = payload?.metrics && typeof payload.metrics === 'object' ? payload.metrics : {} + return { + asset_id: text(payload.asset_id), + release_id: text(payload.release_id), + stage: text(payload.stage), + version: text(payload.version), + pending_total: count(payload.pending_total), + telemetry_status: text(payload.telemetry_status), + reasons: Array.isArray(payload.reasons) ? payload.reasons.map(text).filter(Boolean) : [], + metrics: Object.fromEntries( + RELEASE_REVIEW_METRIC_KEYS + .filter((key) => Object.hasOwn(metrics, key)) + .map((key) => [key, metrics[key]]) + ), + alerts: (Array.isArray(payload.alerts) ? payload.alerts : []).map((alert) => ({ + code: text(alert?.code), + severity: text(alert?.severity), + message: text(alert?.message), + recommended_action: text(alert?.recommended_action) + })), + items: (Array.isArray(payload.items) ? payload.items : []).map((item) => ({ + sample_id: text(item?.sample_id), + observation_id: text(item?.observation_id), + source_document_id: text(item?.source_document_id), + rule_code: text(item?.rule_code), + business_stage: text(item?.business_stage), + prediction_blinded: item?.prediction_blinded === true, + reviewer_count: count(item?.reviewer_count), + required_reviewers: Math.max(1, count(item?.required_reviewers, 1)), + conflicted: item?.conflicted === true, + created_at: text(item?.created_at) + })) + } +} + +export function normalizeAgentAssetReleaseReviewLabel(label) { + const normalized = text(label) + if (!RELEASE_REVIEW_LABELS.has(normalized)) { + throw new TypeError('盲审结论必须是 risk_present 或 risk_absent。') + } + return normalized +} + +export async function fetchAgentAssetReleaseReviewQueue(assetId, limit = 50) { + const payload = await apiRequest( + `/agent-assets/${assetId}/release/review-queue${buildQuery({ limit })}` + ) + return normalizeAgentAssetReleaseReviewQueue(payload) +} + +export function labelAgentAssetReleaseObservation( + assetId, + observationId, + label, + options = {} +) { + const reviewLabel = normalizeAgentAssetReleaseReviewLabel(label) + return apiRequest( + `/agent-assets/${assetId}/release/review-queue/${observationId}/labels`, + { + method: 'POST', + body: JSON.stringify({ label: reviewLabel }), + headers: buildWriteHeaders(options) + } + ) +} + export function setRiskRuleAssetEnabled(assetId, enabled, options = {}) { return apiRequest(`/agent-assets/${assetId}/risk-rule-enabled`, { method: 'POST', @@ -273,6 +366,15 @@ export function setRiskRuleAssetEnabled(assetId, enabled, options = {}) { }) } +function text(value) { + return String(value || '').trim() +} + +function count(value, fallback = 0) { + const parsed = Number(value) + return Number.isInteger(parsed) && parsed >= 0 ? parsed : fallback +} + export function setRiskRuleAssetLevel(assetId, riskLevel, options = {}) { return apiRequest(`/agent-assets/${assetId}/risk-rule-level`, { method: 'POST', diff --git a/web/src/services/analyticsValue.js b/web/src/services/analyticsValue.js new file mode 100644 index 0000000..7d7d6db --- /dev/null +++ b/web/src/services/analyticsValue.js @@ -0,0 +1,139 @@ +import { apiRequest } from './api.js' + +const FILTER_QUERY_KEYS = [ + 'departmentId', + 'projectCode', + 'expenseType', + 'supplierId', + 'city', + 'ownerId', + 'sourceType', + 'valueKind' +] + +const OPPORTUNITY_QUERY_KEYS = [ + ...FILTER_QUERY_KEYS, + 'status', + 'claimId', + 'createdFrom', + 'createdTo', + 'sort', + 'page', + 'pageSize' +] + +function camelizeKey(value) { + return String(value || '').replace(/_([a-z0-9])/gu, (_, letter) => letter.toUpperCase()) +} + +export function normalizeAnalyticsValuePayload(value) { + if (Array.isArray(value)) { + return value.map((item) => normalizeAnalyticsValuePayload(item)) + } + if (!value || typeof value !== 'object' || value instanceof Date) { + return value + } + + return Object.fromEntries( + Object.entries(value).map(([key, item]) => [ + camelizeKey(key), + normalizeAnalyticsValuePayload(item) + ]) + ) +} + +function snakeCaseKey(value) { + return String(value || '').replace(/[A-Z]/gu, (letter) => `_${letter.toLowerCase()}`) +} + +function appendQueryValue(search, key, value) { + if (value === undefined || value === null || value === '') { + return + } + search.set(snakeCaseKey(key), String(value)) +} + +export function buildCfoValueSearch(options = {}) { + const search = new URLSearchParams() + appendQueryValue(search, 'start', options.start) + appendQueryValue(search, 'end', options.end) + appendQueryValue(search, 'asOf', options.asOf) + FILTER_QUERY_KEYS.forEach((key) => appendQueryValue(search, key, options[key])) + return search +} + +export function buildSavingsOpportunitySearch(options = {}) { + const search = new URLSearchParams() + OPPORTUNITY_QUERY_KEYS.forEach((key) => appendQueryValue(search, key, options[key])) + return search +} + +function buildMutationBody(payload = {}) { + if (Array.isArray(payload)) { + return payload.map((item) => buildMutationBody(item)) + } + if (!payload || typeof payload !== 'object' || payload instanceof Date) { + return payload + } + return Object.fromEntries( + Object.entries(payload).map(([key, value]) => [snakeCaseKey(key), buildMutationBody(value)]) + ) +} + +export async function fetchCfoValueDashboard(options = {}) { + const search = buildCfoValueSearch(options) + const suffix = search.size ? `?${search.toString()}` : '' + const payload = await apiRequest(`/analytics/cfo-value${suffix}`, { + timeoutMs: Number(options.timeoutMs || 10_000), + timeoutMessage: '经营价值看板加载超时，请稍后重试。' + }) + return normalizeAnalyticsValuePayload(payload) +} + +export async function fetchSavingsOpportunities(options = {}) { + const search = buildSavingsOpportunitySearch(options) + const suffix = search.size ? `?${search.toString()}` : '' + const payload = await apiRequest(`/savings/opportunities${suffix}`, { + timeoutMs: Number(options.timeoutMs || 10_000), + timeoutMessage: '节省机会台账加载超时，请稍后重试。' + }) + return normalizeAnalyticsValuePayload(payload) +} + +export async function fetchSavingsOpportunity(opportunityId, options = {}) { + const payload = await apiRequest(`/savings/opportunities/${encodeURIComponent(opportunityId)}`, { + timeoutMs: Number(options.timeoutMs || 10_000), + timeoutMessage: '节省机会证据链加载超时，请稍后重试。' + }) + return normalizeAnalyticsValuePayload(payload) +} + +export async function executeSavingsOpportunityAction(opportunityId, payload) { + const response = await apiRequest(`/savings/opportunities/${encodeURIComponent(opportunityId)}/actions`, { + method: 'POST', + body: JSON.stringify(buildMutationBody(payload)), + timeoutMs: 15_000, + timeoutMessage: '节省机会动作执行超时，可使用同一操作窗口安全重试。' + }) + return normalizeAnalyticsValuePayload(response) +} + +export async function recordSavingsRealization(opportunityId, payload) { + const response = await apiRequest(`/savings/opportunities/${encodeURIComponent(opportunityId)}/realizations`, { + method: 'POST', + body: JSON.stringify(buildMutationBody(payload)), + timeoutMs: 15_000, + timeoutMessage: '实际结果写入超时，可使用同一操作窗口安全重试。' + }) + return normalizeAnalyticsValuePayload(response) +} + +export async function executeSavingsRealizationAction(realizationId, payload) { + const response = await apiRequest(`/savings/realizations/${encodeURIComponent(realizationId)}/actions`, { + method: 'POST', + body: JSON.stringify(buildMutationBody(payload)), + timeoutMs: 15_000, + timeoutMessage: '财务确认动作执行超时，可使用同一操作窗口安全重试。' + }) + return normalizeAnalyticsValuePayload(response) +} diff --git a/web/src/services/commercial.js b/web/src/services/commercial.js new file mode 100644 index 0000000..90e1f15 --- /dev/null +++ b/web/src/services/commercial.js @@ -0,0 +1,250 @@ +import { apiRequest } from './api.js' + +const DEFAULT_TIMEOUT_MS = 12_000 +const MUTATION_TIMEOUT_MS = 15_000 +let requestSequence = 0 + +function camelizeKey(value) { + return String(value || '').replace(/_([a-z0-9])/gu, (_, letter) => letter.toUpperCase()) +} + +function snakeCaseKey(value) { + return String(value || '').replace(/[A-Z]/gu, (letter) => `_${letter.toLowerCase()}`) +} + +export function normalizeCommercialPayload(value) { + if (Array.isArray(value)) { + return value.map((item) => normalizeCommercialPayload(item)) + } + if (!value || typeof value !== 'object' || value instanceof Date) { + return value + } + return Object.fromEntries( + Object.entries(value).map(([key, item]) => [ + camelizeKey(key), + normalizeCommercialPayload(item) + ]) + ) +} + +export function serializeCommercialPayload(value) { + if (Array.isArray(value)) { + return value.map((item) => serializeCommercialPayload(item)) + } + if (!value || typeof value !== 'object' || value instanceof Date) { + return value + } + return Object.fromEntries( + Object.entries(value) + .filter(([, item]) => item !== undefined) + .map(([key, item]) => [ + snakeCaseKey(key), + serializeCommercialPayload(item) + ]) + ) +} + +function requireTenantId(tenantId) { + const normalized = String(tenantId || '').trim() + if (!normalized) { + throw new Error('请选择需要管理的租户。') + } + return normalized +} + +function tenantAdminPath(tenantId, suffix = '') { + return `/commercial/admin/tenants/${encodeURIComponent(requireTenantId(tenantId))}${suffix}` +} + +function appendQuery(search, key, value) { + if (value === undefined || value === null || value === '') { + return + } + search.set(snakeCaseKey(key), String(value)) +} + +export function buildCommercialAnalyticsSearch(options = {}) { + const search = new URLSearchParams() + appendQuery(search, 'start', options.start) + appendQuery(search, 'end', options.end) + appendQuery(search, 'asOf', options.asOf) + return search +} + +function buildCommercialListSearch(options = {}, keys = []) { + const search = new URLSearchParams() + keys.forEach((key) => appendQuery(search, key, options[key])) + appendQuery(search, 'limit', options.limit) + appendQuery(search, 'offset', options.offset) + return search +} + +function withSearch(path, search) { + return search.size ? `${path}?${search.toString()}` : path +} + +async function readCommercial(path, options = {}) { + const payload = await apiRequest(path, { + timeoutMs: Number(options.timeoutMs || DEFAULT_TIMEOUT_MS), + timeoutMessage: options.timeoutMessage || '商业账户加载超时，请稍后重试。' + }) + return normalizeCommercialPayload(payload) +} + +async function mutateCommercial(path, method, payload, timeoutMessage) { + requestSequence += 1 + const requestId = `commercial-web-${Date.now().toString(36)}-${requestSequence.toString(36)}` + const response = await apiRequest(path, { + method, + headers: { 'X-Request-Id': requestId }, + body: JSON.stringify(serializeCommercialPayload(payload)), + timeoutMs: MUTATION_TIMEOUT_MS, + timeoutMessage + }) + return normalizeCommercialPayload(response) +} + +export async function fetchCommercialAccount(options = {}) { + const tenantId = String(options.tenantId || '').trim() + const path = options.platformAdmin && tenantId + ? tenantAdminPath(tenantId, '/account') + : '/commercial/account' + const search = new URLSearchParams() + appendQuery(search, 'asOf', options.asOf) + const suffix = search.size ? `?${search.toString()}` : '' + return readCommercial(`${path}${suffix}`, options) +} + +export async function fetchCommercialAnalytics(tenantId, options = {}) { + const search = buildCommercialAnalyticsSearch(options) + const suffix = search.size ? `?${search.toString()}` : '' + return readCommercial(`${tenantAdminPath(tenantId, '/analytics')}${suffix}`, { + ...options, + timeoutMessage: '商业价值分析加载超时，请稍后重试。' + }) +} + +export function buildCommercialPricingScenario(tenantId, payload) { + return mutateCommercial( + tenantAdminPath(tenantId, '/pricing-scenarios'), + 'POST', + payload, + '定价场景计算超时，本次计算不会自动创建或修改套餐，请确认后重试。' + ) +} + +export function fetchCommercialPlans(tenantId, options = {}) { + return readCommercial(withSearch( + tenantAdminPath(tenantId, '/plans'), + buildCommercialListSearch(options, ['planStatus']) + ), options) +} + +export function fetchCommercialSubscriptions(tenantId, options = {}) { + return readCommercial(withSearch( + tenantAdminPath(tenantId, '/subscriptions'), + buildCommercialListSearch(options, ['subscriptionStatus']) + ), options) +} + +export function fetchCommercialEntitlements(tenantId, options = {}) { + return readCommercial(withSearch( + tenantAdminPath(tenantId, '/entitlements'), + buildCommercialListSearch(options, ['subscriptionId', 'entitlementStatus']) + ), options) +} + +export function fetchCommercialUsageEvents(tenantId, options = {}) { + return readCommercial(withSearch( + tenantAdminPath(tenantId, '/usage-events'), + buildCommercialListSearch(options, ['subscriptionId', 'start', 'end']) + ), options) +} + +export function fetchCommercialCostEvents(tenantId, options = {}) { + return readCommercial(withSearch( + tenantAdminPath(tenantId, '/cost-events'), + buildCommercialListSearch(options, ['subscriptionId', 'start', 'end']) + ), options) +} + +export function createCommercialPlan(tenantId, payload) { + return mutateCommercial( + tenantAdminPath(tenantId, '/plans'), + 'POST', + payload, + '套餐创建超时，请刷新账户确认结果后再重试。' + ) +} + +export function activateCommercialPlan(tenantId, planId, expectedVersion, reason) { + return mutateCommercial( + tenantAdminPath(tenantId, `/plans/${encodeURIComponent(planId)}/activate`), + 'POST', + { expectedVersion, reason }, + '套餐激活超时，请刷新版本状态后再重试。' + ) +} + +export function createCommercialSubscription(tenantId, payload) { + return mutateCommercial( + tenantAdminPath(tenantId, '/subscriptions'), + 'POST', + payload, + '订阅创建超时，请刷新账户确认结果后再重试。' + ) +} + +export function activateCommercialSubscription(tenantId, subscriptionId, expectedVersion, reason) { + return mutateCommercial( + tenantAdminPath(tenantId, `/subscriptions/${encodeURIComponent(subscriptionId)}/activate`), + 'POST', + { expectedVersion, reason }, + '订阅激活超时，请刷新版本状态后再重试。' + ) +} + +export function transitionCommercialSubscription(tenantId, subscriptionId, payload) { + return mutateCommercial( + tenantAdminPath(tenantId, `/subscriptions/${encodeURIComponent(subscriptionId)}/transition`), + 'POST', + payload, + '订阅状态变更超时，请刷新版本状态后再重试。' + ) +} + +export function upsertCommercialEntitlement(tenantId, payload) { + return mutateCommercial( + tenantAdminPath(tenantId, '/entitlements'), + 'PUT', + payload, + '权益保存超时，请刷新账户确认结果后再重试。' + ) +} + +export function activateCommercialEntitlement(tenantId, entitlementId, expectedVersion, reason) { + return mutateCommercial( + tenantAdminPath(tenantId, `/entitlements/${encodeURIComponent(entitlementId)}/activate`), + 'POST', + { expectedVersion, reason }, + '权益激活超时，请刷新版本状态后再重试。' + ) +} + +export function recordCommercialUsage(tenantId, payload) { + return mutateCommercial( + tenantAdminPath(tenantId, '/usage-events'), + 'POST', + payload, + '用量事件写入超时，可使用相同幂等键安全重试。' + ) +} + +export function recordCommercialCost(tenantId, payload) { + return mutateCommercial( + tenantAdminPath(tenantId, '/cost-events'), + 'POST', + payload, + '成本事件写入超时，可使用相同幂等键安全重试。' + ) +} diff --git a/web/src/services/financialConnectors.js b/web/src/services/financialConnectors.js new file mode 100644 index 0000000..65c55f7 --- /dev/null +++ b/web/src/services/financialConnectors.js @@ -0,0 +1,27 @@ +import { apiRequest } from './api.js' + +function clampWindowHours(value) { + const parsed = Number(value || 24) + if (!Number.isFinite(parsed)) return 24 + return Math.max(1, Math.min(720, Math.round(parsed))) +} + +export function fetchFinancialConnectorObservability(options = {}) { + const search = new URLSearchParams() + search.set('window_hours', String(clampWindowHours(options.windowHours))) + return apiRequest(`/financial-connectors/observability?${search.toString()}`, { + timeoutMs: Number(options.timeoutMs || 5000), + timeoutMessage: '财务连接器运行指标加载超时，请稍后重试。' + }) +} + +export function fetchFinancialPaymentEvidence(claimId, options = {}) { + const normalizedClaimId = String(claimId || '').trim() + if (!normalizedClaimId) { + return Promise.reject(new Error('报销单 ID 不能为空。')) + } + return apiRequest(`/financial-connectors/payment-evidence/${encodeURIComponent(normalizedClaimId)}`, { + timeoutMs: Number(options.timeoutMs || 5000), + timeoutMessage: '付款证据等级加载超时，请稍后重试。' + }) +} diff --git a/web/src/utils/aiApplicationPrecheckModel.js b/web/src/utils/aiApplicationPrecheckModel.js index 81a834e..6dfb6c0 100644 --- a/web/src/utils/aiApplicationPrecheckModel.js +++ b/web/src/utils/aiApplicationPrecheckModel.js @@ -314,7 +314,7 @@ export function buildAiApplicationPrecheckMessage(preview = {}, precheck = {}) { lines.push( '', '**后续行动建议**：', - '- 请检查本次申请时间是否填写正确。若日期填错，请直接回复正确的出发时间和返回时间，我会重新查询；', + '- 请先检查本次申请时间是否填写正确。若日期填错，请直接回复正确的出发时间和返回时间，我会重新查询；', '- 若日期无误，请先处理或关联已有申请单，避免重复申请。', '', '我会先暂停本次申请表生成，不会开放保存草稿或提交入口。' @@ -372,7 +372,7 @@ export function buildAiApplicationSubmitConflictMessage(preview = {}, precheck = lines.push( '', '**后续行动建议**：', - '- 请核对申请时间是否填写正确。若日期填错，请直接回复正确的出发时间和返回时间，我会重新查询；', + '- 请先核对申请时间是否填写正确。若日期填错，请直接回复正确的出发时间和返回时间，我会重新查询；', '- 若日期无误，请先查看或处理已有申请单，避免重复申请。', '', '我会先暂停本次提交，不会生成新的审批流。' diff --git a/web/src/utils/authUser.js b/web/src/utils/authUser.js index ea0d96f..5c96952 100644 --- a/web/src/utils/authUser.js +++ b/web/src/utils/authUser.js @@ -86,6 +86,7 @@ export function normalizeAuthUserSnapshot(payload = {}, defaults = {}) { return { username, + tenantId: pickText(payload, ['tenantId', 'tenant_id']), name, role: String(payload.role || defaults.defaultRole || ''), department, diff --git a/web/src/utils/expenseApplicationPreviewParsing.js b/web/src/utils/expenseApplicationPreviewParsing.js index e6fd5ad..f1b7648 100644 --- a/web/src/utils/expenseApplicationPreviewParsing.js +++ b/web/src/utils/expenseApplicationPreviewParsing.js @@ -1,6 +1,13 @@ import { buildMockApplicationTransportEstimate } from './expenseApplicationEstimate.js' import { getTodayDateValue } from './workbenchComposerDate.js' +export { + resolveCurrentUserDepartment, + resolveCurrentUserGrade, + resolveCurrentUserManagerName, + resolveCurrentUserPosition +} from './expenseApplicationUserProfile.js' + export const APPLICATION_PREVIEW_FIELD_DEFINITIONS = [ { key: 'applicationType', label: '申请类型' }, { key: 'applicant', label: '姓名', editable: false, required: false }, @@ -459,48 +466,6 @@ function uniqueApplicationCandidates(values) { .filter((item, index, list) => list.indexOf(item) === index) } -export function resolveCurrentUserGrade(currentUser = {}) { - return String( - currentUser.grade - || currentUser.employeeGrade - || currentUser.employee_grade - || currentUser.profileGrade - || '' - ).trim() -} - -export function resolveCurrentUserDepartment(currentUser = {}) { - return String( - currentUser.department - || currentUser.departmentName - || currentUser.department_name - || '' - ).trim() -} - -export function resolveCurrentUserPosition(currentUser = {}) { - return String( - currentUser.position - || currentUser.employeePosition - || currentUser.employee_position - || currentUser.jobTitle - || currentUser.job_title - || '' - ).trim() -} - -export function resolveCurrentUserManagerName(currentUser = {}) { - return String( - currentUser.managerName - || currentUser.manager_name - || currentUser.directManagerName - || currentUser.direct_manager_name - || currentUser.leaderName - || currentUser.leader_name - || '' - ).trim() -} - export function parseApplicationDaysValue(value) { const match = String(value || '').match(/\d+/) const days = match ? Number(match[0]) : parseChineseNumber(value) diff --git a/web/src/utils/expenseApplicationUserProfile.js b/web/src/utils/expenseApplicationUserProfile.js new file mode 100644 index 0000000..80346b5 --- /dev/null +++ b/web/src/utils/expenseApplicationUserProfile.js @@ -0,0 +1,41 @@ +export function resolveCurrentUserGrade(currentUser = {}) { + return String( + currentUser.grade + || currentUser.employeeGrade + || currentUser.employee_grade + || currentUser.profileGrade + || '' + ).trim() +} + +export function resolveCurrentUserDepartment(currentUser = {}) { + return String( + currentUser.department + || currentUser.departmentName + || currentUser.department_name + || '' + ).trim() +} + +export function resolveCurrentUserPosition(currentUser = {}) { + return String( + currentUser.position + || currentUser.employeePosition + || currentUser.employee_position + || currentUser.jobTitle + || currentUser.job_title + || '' + ).trim() +} + +export function resolveCurrentUserManagerName(currentUser = {}) { + return String( + currentUser.managerName + || currentUser.manager_name + || currentUser.directManagerName + || currentUser.direct_manager_name + || currentUser.leaderName + || currentUser.leader_name + || '' + ).trim() +} diff --git a/web/src/views/AppShellRouteView.vue b/web/src/views/AppShellRouteView.vue index 3a0de41..abc891d 100644 --- a/web/src/views/AppShellRouteView.vue +++ b/web/src/views/AppShellRouteView.vue @@ -130,6 +130,7 @@ v-if="activeView === 'overview'" :filtered-requests="filteredRequests" :dashboard="overviewDashboard" + :current-user="currentUser" :active-range="activeRange" :custom-range="customRange" @approve="handleApprove" @@ -241,6 +242,7 @@ diff --git a/web/src/views/AuditView.vue b/web/src/views/AuditView.vue index 47fffa7..fe0f90e 100644 --- a/web/src/views/AuditView.vue +++ b/web/src/views/AuditView.vue @@ -48,6 +48,14 @@ v-else-if="selectedSkill.usesJsonRiskRule" :selected-skill="selectedSkill" :risk-rule-test-passed="riskRuleTestPassed" + :release-state="releaseState" + :release-queue="releaseQueue" + :release-loading="releaseLoading" + :release-error="releaseError" + :release-action="releaseAction" + :can-review-release="canManageSelected" + @refresh-release="loadReleaseMonitor" + @label-release="submitReleaseLabel" />
- +
+ + +
@@ -229,6 +238,13 @@ back-label="返回预算中心" @back="backToList" > +
{{ item.label }} @@ -292,7 +308,11 @@ - + {{ item.name }} {{ item.amountLabel }} {{ item.usedLabel }} diff --git a/web/src/views/LoginView.vue b/web/src/views/LoginView.vue index 91ebddf..28f9932 100644 --- a/web/src/views/LoginView.vue +++ b/web/src/views/LoginView.vue @@ -77,7 +77,7 @@
@@ -339,7 +355,10 @@ import SystemAgentRatioBar from '../components/charts/SystemAgentRatioBar.vue' import SystemLoginWaveChart from '../components/charts/SystemLoginWaveChart.vue' import SystemTokenDailyWaveChart from '../components/charts/SystemTokenDailyWaveChart.vue' import SystemUserTokenPie from '../components/charts/SystemUserTokenPie.vue' +import CommercialWorkspace from '../components/commercial/CommercialWorkspace.vue' +import CfoValueDashboard from '../components/dashboard/CfoValueDashboard.vue' import DigitalEmployeeDashboard from '../components/dashboard/DigitalEmployeeDashboard.vue' +import FinancialConnectorHealthPanel from '../components/dashboard/FinancialConnectorHealthPanel.vue' import RiskObservationDashboard from '../components/dashboard/RiskObservationDashboard.vue' import TableLoadingState from '../components/shared/TableLoadingState.vue' @@ -348,6 +367,7 @@ import { useOverviewView } from '../composables/useOverviewView.js' const props = defineProps({ filteredRequests: { type: Array, required: true }, dashboard: { type: String, default: 'finance' }, + currentUser: { type: Object, default: () => ({}) }, activeRange: { type: String, default: '近10日' }, customRange: { type: Object, @@ -401,18 +421,24 @@ const { } = useOverviewView(props) const activeDashboard = computed(() => { + if (props.dashboard === 'commercial') return 'commercial' + if (props.dashboard === 'value') return 'value' if (props.dashboard === 'system') return 'system' if (props.dashboard === 'risk') return 'risk' if (props.dashboard === 'digitalEmployee') return 'digitalEmployee' return 'finance' }) const activeKpiMetrics = computed(() => { + if (activeDashboard.value === 'commercial') return [] + if (activeDashboard.value === 'value') return [] if (activeDashboard.value === 'system') return systemKpiMetrics.value if (activeDashboard.value === 'digitalEmployee') return digitalEmployeeKpiMetrics.value if (activeDashboard.value === 'risk') return riskKpiMetrics.value return kpiMetrics.value }) const activeDashboardLoading = computed(() => { + if (activeDashboard.value === 'commercial') return false + if (activeDashboard.value === 'value') return false if (activeDashboard.value === 'system') { return systemDashboardLoading.value } @@ -425,6 +451,8 @@ const activeDashboardLoading = computed(() => { return financeDashboardLoading.value }) const activeDashboardLoadingText = computed(() => { + if (activeDashboard.value === 'commercial') return '正在加载商业化管理与价值证明数据' + if (activeDashboard.value === 'value') return '正在加载经营价值看板数据' if (activeDashboard.value === 'system') return '正在加载系统看板数据' if (activeDashboard.value === 'digitalEmployee') return '正在加载数字员工看板数据' if (activeDashboard.value === 'risk') return '正在加载风险看板数据' diff --git a/web/src/views/ReceiptFolderView.vue b/web/src/views/ReceiptFolderView.vue index 4e01be2..e2fb9af 100644 --- a/web/src/views/ReceiptFolderView.vue +++ b/web/src/views/ReceiptFolderView.vue @@ -384,6 +384,10 @@ import { import { inferPreviewKindFromBlob } from '../utils/documentPreviewAssets.js' import { createReceiptDetailDashboardModel } from './scripts/receiptFolderDetailDashboard.js' import { createReceiptDetailFieldModel } from './scripts/receiptFolderDetailFields.js' +import { + formatReceiptDateTime as formatDateTime, + formatReceiptRecognitionScore as formatScore +} from './scripts/receiptFolderFormatting.js' import { createReceiptFolderListFilterModel } from './scripts/receiptFolderListFilters.js' const NEW_CLAIM_VALUE = '__new_claim__' @@ -783,19 +787,6 @@ async function openAssociationConversation() { } } -function formatScore(value) { - const score = Number(value || 0) - if (!Number.isFinite(score) || score <= 0) return '待确认' - return `${Math.round(score * 100)}%` -} - -function formatDateTime(value) { - if (!String(value ?? '').trim()) return '待确认' - const date = new Date(value) - if (Number.isNaN(date.getTime())) return '待确认' - return `${date.getFullYear()}-${String(date.getMonth() + 1).padStart(2, '0')}-${String(date.getDate()).padStart(2, '0')} ${String(date.getHours()).padStart(2, '0')}:${String(date.getMinutes()).padStart(2, '0')}` -} - diff --git a/web/src/views/TravelRequestDetailView.vue b/web/src/views/TravelRequestDetailView.vue index 736dd88..7816d8b 100644 --- a/web/src/views/TravelRequestDetailView.vue +++ b/web/src/views/TravelRequestDetailView.vue @@ -40,91 +40,20 @@
-
-
- {{ item.label }} - - - - -
-
+ selectedSkillIsRule.value && (selectedSkill.value?.reviewNote || selectedSkill.value?.reviewTimeLabel) ) @@ -674,6 +688,11 @@ export default { riskRuleEditOpen, riskRuleEditMode, riskRuleEditForm, + releaseState, + releaseQueue, + releaseLoading, + releaseError, + releaseAction, riskRuleBusinessStageOptions: RISK_RULE_BUSINESS_STAGE_OPTIONS, riskRuleExpenseCategoryOptions: RISK_RULE_EXPENSE_CATEGORY_OPTIONS, showReviewNote, @@ -730,6 +749,8 @@ export default { closePublishRiskRuleDialog, publishSelectedRiskRule, toggleSelectedRiskRuleEnabled, + loadReleaseMonitor, + submitReleaseLabel, activateSelectedRule, restoreSelectedVersion, openVersionTimeline: openVersionTimelineInState, diff --git a/web/src/views/scripts/BudgetCenterView.js b/web/src/views/scripts/BudgetCenterView.js index 8881ea2..485d67a 100644 --- a/web/src/views/scripts/BudgetCenterView.js +++ b/web/src/views/scripts/BudgetCenterView.js @@ -1,5 +1,6 @@ -import { computed, onMounted, ref, watch } from 'vue' +import { computed, nextTick, onMounted, ref, watch } from 'vue' import { ElButton } from 'element-plus/es/components/button/index.mjs' +import { useRoute } from 'vue-router' import BudgetTrendChart from '../../components/charts/BudgetTrendChart.vue' import DocumentDropdownFilter from '../../components/shared/DocumentDropdownFilter.vue' @@ -17,8 +18,13 @@ import { } from '../../utils/accessControl.js' import { BUDGET_QUARTER_OPTIONS, - BUDGET_YEAR_OPTIONS + BUDGET_YEAR_OPTIONS, + resolveBudgetExpenseTypeLabel } from '../../utils/budgetOntology.js' +import { + BUDGET_CONFIGURATION_NOTICE, + readBudgetConfigurationFocus +} from './cfoValueSourceLinks.js' import { BUDGET_PAGE_SIZE_OPTIONS, BUDGET_SCOPE_ALL, @@ -112,6 +118,7 @@ export default { ElButton }, setup(props, { emit }) { + const route = useRoute() const departments = ref(FALLBACK_DEPARTMENTS) const activeBudgetScope = ref(BUDGET_SCOPE_ALL) const budgetKeyword = ref('') @@ -126,6 +133,13 @@ export default { quarter: 'Q1', status: '全部' }) + const budgetConfigurationFocus = computed(() => readBudgetConfigurationFocus(route.query)) + const budgetConfigurationFocusSummary = computed(() => { + const focus = budgetConfigurationFocus.value + const department = focus.departmentName || focus.departmentId || '当前可见部门' + const expenseType = resolveBudgetExpenseTypeLabel(focus.expenseType, focus.expenseType) + return [department, expenseType].filter(Boolean).join(' · ') + }) const canEditBudget = computed(() => canEditBudgetCenter(props.currentUser) || isBudgetMonitorUser(props.currentUser) @@ -170,6 +184,7 @@ export default { const filteredBudgetRows = computed(() => activeScopeRows.value .filter((row) => filters.value.status === '全部' || row.statusLabel === filters.value.status) + .filter((row) => matchesBudgetConfigurationExpense(row)) .filter((row) => matchesBudgetKeyword(row, budgetKeyword.value)) ) const totalBudgetRows = computed(() => filteredBudgetRows.value.length) @@ -227,12 +242,16 @@ export default { const showEmpty = computed(() => !budgetLoading.value && !budgetError.value && visibleBudgetRows.value.length === 0) const emptyState = computed(() => ({ eyebrow: activeScopeLabel.value, - title: `暂无${activeScopeLabel.value}`, - desc: '当前筛选条件下没有匹配的预算记录。', + title: budgetConfigurationFocus.value.active ? '未找到匹配的预算配置' : `暂无${activeScopeLabel.value}`, + desc: budgetConfigurationFocus.value.active + ? `当前配置列表没有“${budgetConfigurationFocusSummary.value}”对应的预算科目；这不代表预算为零。` + : '当前筛选条件下没有匹配的预算记录。', icon: 'mdi mdi-database-search-outline', tone: 'blue', artLabel: '预算列表为空', - tips: ['可以调整年度、季度、状态或关键词后重试。'] + tips: budgetConfigurationFocus.value.active + ? ['返回经营价值看板调整维度，或等待正式预算科目数据接入。'] + : ['可以调整年度、季度、状态或关键词后重试。'] })) const pageSummary = computed(() => `共 ${totalBudgetRows.value} 条，目前第 ${currentBudgetPage.value} / ${totalBudgetPages.value} 页`) @@ -313,6 +332,50 @@ export default { selectedBudgetId.value = '' } + async function applyBudgetConfigurationFocus() { + const focus = budgetConfigurationFocus.value + if (!focus.active) return + const scopeChanged = activeBudgetScope.value !== BUDGET_SCOPE_ALL + activeBudgetScope.value = BUDGET_SCOPE_ALL + filters.value.status = '全部' + budgetPage.value = 1 + if (scopeChanged) await nextTick() + + const departmentTokens = [focus.departmentId, focus.departmentName] + .map((value) => String(value || '').trim().toLocaleLowerCase('zh-CN')) + .filter(Boolean) + const department = departments.value.find((item) => [ + item.id, item.code, item.name, item.costCenter + ].some((value) => departmentTokens.includes(String(value || '').trim().toLocaleLowerCase('zh-CN')))) + const rows = budgetRowsByScope.value[BUDGET_SCOPE_ALL] || [] + const row = department + ? rows.find((item) => ( + item.departmentCode === department.code + || item.departmentName === department.name + || (department.costCenter && item.costCenter === department.costCenter) + )) + : null + + selectedBudgetId.value = row && matchesBudgetConfigurationExpense(row) ? row.id : '' + budgetKeyword.value = row ? '' : (focus.departmentName || focus.departmentId || '') + } + + function matchesBudgetConfigurationExpense(row = {}) { + const focus = String(budgetConfigurationFocus.value.expenseType || '').trim().toLocaleLowerCase('zh-CN') + if (!budgetConfigurationFocus.value.active || !focus) return true + return (Array.isArray(row.categoryRows) ? row.categoryRows : []).some((item) => ( + [item.code, item.name].some((value) => String(value || '').trim().toLocaleLowerCase('zh-CN') === focus) + )) + } + + function isFocusedBudgetCategory(item = {}) { + const focus = String(budgetConfigurationFocus.value.expenseType || '').trim().toLocaleLowerCase('zh-CN') + if (!focus) return false + return [item.code, item.name].some((value) => ( + String(value || '').trim().toLocaleLowerCase('zh-CN') === focus + )) + } + function handleRowAction(row) { if (activeBudgetScope.value === BUDGET_SCOPE_REVIEW && canAuditBudgetDrafts.value) { openBudgetReviewAssistant(row) @@ -387,9 +450,19 @@ export default { } onMounted(() => { - void loadDepartments() + void loadDepartments().then(() => applyBudgetConfigurationFocus()) }) + watch( + () => [ + route.query.budget_view, + route.query.budget_department_id, + route.query.budget_department_name, + route.query.budget_expense_type + ], + () => void applyBudgetConfigurationFocus() + ) + watch( () => activeBudgetScope.value, () => { @@ -439,6 +512,9 @@ export default { activeBudgetFilterKey, activeBudgetScope, budgetError, + budgetConfigurationFocus, + budgetConfigurationFocusSummary, + budgetConfigurationNotice: BUDGET_CONFIGURATION_NOTICE, budgetKeyword, budgetLoading, budgetPage: currentBudgetPage, @@ -457,6 +533,7 @@ export default { filters, goToBudgetPage, handleRowAction, + isFocusedBudgetCategory, openBudgetAssistant, openBudgetDetail, openBudgetReviewAssistant, diff --git a/web/src/views/scripts/TravelRequestDetailView.js b/web/src/views/scripts/TravelRequestDetailView.js index 2f37946..f860135 100644 --- a/web/src/views/scripts/TravelRequestDetailView.js +++ b/web/src/views/scripts/TravelRequestDetailView.js @@ -4,6 +4,7 @@ import { ElInput } from 'element-plus/es/components/input/index.mjs' import EnterpriseSelect from '../../components/shared/EnterpriseSelect.vue' import ConfirmDialog from '../../components/shared/ConfirmDialog.vue' import TravelRequestApprovalDialog from '../../components/travel/TravelRequestApprovalDialog.vue' +import TravelRequestApplicationFacts from '../../components/travel/TravelRequestApplicationFacts.vue' import TravelRequestBudgetAnalysis from '../../components/travel/TravelRequestBudgetAnalysis.vue' import TravelRequestDeleteDialog from '../../components/travel/TravelRequestDeleteDialog.vue' import TravelRequestDetailHero from '../../components/travel/TravelRequestDetailHero.vue' @@ -22,6 +23,7 @@ export default { EnterpriseSelect, StageRiskAdviceCard, TravelRequestApprovalDialog, + TravelRequestApplicationFacts, TravelRequestBudgetAnalysis, TravelRequestDeleteDialog, TravelRequestDetailHero, diff --git a/web/src/views/scripts/cfoValueDashboardModel.js b/web/src/views/scripts/cfoValueDashboardModel.js new file mode 100644 index 0000000..d6a3658 --- /dev/null +++ b/web/src/views/scripts/cfoValueDashboardModel.js @@ -0,0 +1,357 @@ +const STATUS_META = { + identified: { label: '待评估', tone: 'info' }, + accepted: { label: '已接受', tone: 'primary' }, + in_progress: { label: '执行中', tone: 'warning' }, + realized: { label: '实际待确认', tone: 'warning' }, + verified: { label: '财务已确认', tone: 'success' }, + reversed: { label: '已冲回', tone: 'danger' }, + rejected: { label: '已拒绝', tone: 'danger' }, + expired: { label: '已到期', tone: 'muted' } +} + +const ACTION_META = { + accept: { label: '接受机会', tone: 'primary', kind: 'opportunity' }, + start: { label: '开始执行', tone: 'primary', kind: 'opportunity' }, + reject: { label: '拒绝', tone: 'danger', kind: 'opportunity' }, + expire: { label: '标记到期', tone: 'danger', kind: 'opportunity' }, + record_realization: { label: '登记实际结果', tone: 'warning', kind: 'evidence_required' }, + confirm: { label: '财务确认', tone: 'primary', kind: 'realization' }, + reverse: { label: '冲回', tone: 'danger', kind: 'realization' } +} + +const DIMENSION_LABELS = { + department: '部门', + project: '项目', + expense_type: '费用类型', + supplier: '供应商', + city: '城市', + owner: '负责人', + source: '来源' +} + +const DATA_QUALITY_LABELS = { + pendingConfirmationCount: '待财务确认', + businessStateOnlyCount: '仅平台业务状态', + pendingDedupeCount: '待 canonical 去重', + missingFxCount: '汇率证据缺失', + missingEvidenceCount: '证据缺失', + actualOverEstimateCount: '实际超过预计' +} + +const FILTER_KEYS = [ + 'departmentId', + 'projectCode', + 'expenseType', + 'supplierId', + 'city', + 'ownerId', + 'sourceType', + 'valueKind', + 'status' +] + +export function createEmptyValueFilters() { + return Object.fromEntries(FILTER_KEYS.map((key) => [key, ''])) +} + +export function readValueFiltersFromQuery(query = {}) { + const result = createEmptyValueFilters() + FILTER_KEYS.forEach((key) => { + const value = query[`value_${snakeCase(key)}`] + result[key] = Array.isArray(value) ? String(value[0] || '') : String(value || '') + }) + return result +} + +export function writeValueFiltersToQuery(query = {}, filters = {}) { + const result = { ...query } + FILTER_KEYS.forEach((key) => { + const queryKey = `value_${snakeCase(key)}` + const value = String(filters[key] || '').trim() + if (value) { + result[queryKey] = value + } else { + delete result[queryKey] + } + }) + return result +} + +export function resolveRangeWindow(activeRange, customRange = {}, nowValue = new Date()) { + const now = new Date(nowValue) + const end = new Date(now) + const start = new Date(now) + start.setHours(0, 0, 0, 0) + + if (activeRange === 'custom' && customRange.start && customRange.end) { + const customStart = new Date(`${customRange.start}T00:00:00`) + const customEnd = new Date(`${customRange.end}T23:59:59.999`) + return { + start: customStart.toISOString(), + end: customEnd.toISOString() + } + } + if (activeRange === '本周') { + const day = start.getDay() || 7 + start.setDate(start.getDate() - day + 1) + } else if (activeRange === '本月') { + start.setDate(1) + } else if (activeRange === '近10日') { + start.setDate(start.getDate() - 9) + } + + return { start: start.toISOString(), end: end.toISOString() } +} + +export function classifyCfoDashboardState({ dashboard, error, loading, now = new Date() } = {}) { + if (loading && !dashboard) return 'loading' + if (error?.status === 403) return 'permission' + if (error && !dashboard) return 'error' + if (!dashboard) return 'empty' + + const source = dashboard.source || {} + const opportunityCount = Number(source.opportunityCount || 0) + const realizationCount = Number(source.realizationCount || 0) + if (source.dataStatus === 'empty' && opportunityCount === 0 && realizationCount === 0) { + return 'empty' + } + if (source.dataStatus === 'partial' || ((opportunityCount || realizationCount) && !source.freshnessAt)) { + return 'partial' + } + if (isDashboardStale(dashboard, now)) return 'stale' + return 'ready' +} + +export function isDashboardStale(dashboard = {}, nowValue = new Date(), thresholdHours = 24) { + const freshness = dashboard.source?.freshnessAt + if (!freshness) return false + const timestamp = new Date(freshness).getTime() + const now = new Date(nowValue).getTime() + return Number.isFinite(timestamp) && Number.isFinite(now) && now - timestamp > thresholdHours * 3_600_000 +} + +export function buildValueKpis(dashboard = {}) { + const source = dashboard.source || {} + const cash = dashboard.kpis?.verifiedCash || {} + const labor = dashboard.kpis?.releasableLabor || {} + const safe = dashboard.kpis?.safeStraightThrough || {} + const valueKindFilter = String(dashboard.filters?.valueKind || '') + const hasLedgerRows = Number(source.opportunityCount || 0) + Number(source.realizationCount || 0) > 0 + const cashValues = Array.isArray(cash.values) ? cash.values : [] + const zeroCashLabel = resolveZeroCashLabel(dashboard) + const cashExcluded = valueKindFilter === 'labor' + const laborExcluded = valueKindFilter === 'cash' + + return [ + { + key: 'verified_cash', + label: cash.label || '财务确认净现金节省', + displayValue: cashExcluded ? '未纳入筛选' : (cashValues.length ? formatMoneyValues(cashValues) : (hasLedgerRows ? zeroCashLabel : '暂无数据')), + statusLabel: cashExcluded ? '筛选排除' : (cashValues.length ? '已核验' : (hasLedgerRows ? '真实零值' : '无台账数据')), + state: cashExcluded ? 'unavailable' : (cashValues.length ? 'available' : (hasLedgerRows ? 'zero' : 'empty')), + detail: cashExcluded + ? '当前筛选仅查看工时价值，现金 KPI 未参与本次查询。' + : cashValues.length + ? `${Number(cash.confirmedRealizationCount || 0)} 笔独立财务确认` + : (hasLedgerRows ? '当前周期尚无可计入主指标的确认结果' : '当前筛选范围暂无节省机会'), + definition: cash.definition || '' + }, + { + key: 'labor', + label: labor.label || '财务确认可释放工时价值', + displayValue: laborExcluded ? '未纳入筛选' : '待采集', + statusLabel: laborExcluded ? '筛选排除' : (labor.status === 'unavailable' ? '基线缺失' : '采集中'), + state: laborExcluded ? 'unavailable' : 'baseline-missing', + detail: laborExcluded ? '当前筛选仅查看现金节省，工时价值未参与本次查询。' : (labor.reason || '缺少经过客户确认的工时基线。'), + requiredInputs: laborExcluded ? [] : (Array.isArray(labor.requiredInputs) ? labor.requiredInputs : []) + }, + { + key: 'straight_through', + label: safe.label || '安全智能直通率', + displayValue: '待采集', + statusLabel: '审计样本不足', + state: 'baseline-missing', + detail: safe.reason || '缺少 eligibility 与事后审计结果。', + requiredInputs: Array.isArray(safe.requiredInputs) ? safe.requiredInputs : [] + } + ] +} + +export function buildFunnelRows(dashboard = {}) { + const stages = Array.isArray(dashboard.funnel?.stages) ? dashboard.funnel.stages : [] + const maxCount = Math.max(...stages.map((stage) => Number(stage.count || 0)), 1) + return stages.map((stage) => ({ + ...stage, + count: Number(stage.count || 0), + valueLabel: formatMoneyValues(stage.values), + width: `${Math.max((Number(stage.count || 0) / maxCount) * 100, stage.count ? 8 : 0)}%` + })) +} + +export function buildTrendSeries(dashboard = {}, currency = '') { + const points = Array.isArray(dashboard.trend) ? dashboard.trend : [] + const currencies = [...new Set(points.map((item) => String(item.currency || '')).filter(Boolean))] + const selectedCurrency = currencies.includes(currency) ? currency : (currencies[0] || '') + const rows = points + .filter((item) => !selectedCurrency || item.currency === selectedCurrency) + .map((item) => ({ + period: item.period || '', + currency: item.currency || '', + verifiedNet: Number(item.verifiedNet || 0), + actualPending: Number(item.actualPending || 0), + reversal: Number(item.reversal || 0) + })) + return { currencies, selectedCurrency, rows } +} + +export function buildBreakdownGroups(dashboard = {}) { + return (Array.isArray(dashboard.breakdowns) ? dashboard.breakdowns : []).map((group) => { + const items = Array.isArray(group.items) ? group.items : [] + const normalized = items.flatMap((item) => splitBreakdownItemByCurrency(item)) + const maxByCurrency = normalized.reduce((result, item) => { + result[item.currency] = Math.max(result[item.currency] || 0, item.verifiedMagnitude) + return result + }, {}) + normalized.sort((left, right) => ( + left.currency.localeCompare(right.currency) + || right.verifiedMagnitude - left.verifiedMagnitude + || left.dimensionName.localeCompare(right.dimensionName, 'zh-CN') + )) + return { + ...group, + label: DIMENSION_LABELS[group.dimension] || group.dimension, + items: normalized.map((item) => ({ + ...item, + width: `${Math.max((item.verifiedMagnitude / Math.max(maxByCurrency[item.currency] || 0, 1)) * 100, item.verifiedMagnitude ? 6 : 0)}%` + })) + } + }) +} + +export function buildGuardrailRows(dashboard = {}) { + return (Array.isArray(dashboard.guardrails) ? dashboard.guardrails : []).map((item) => ({ + ...item, + countLabel: item.count === null || item.count === undefined ? '不可用' : `${Number(item.count)} 项`, + valueLabel: formatMoneyValues(item.values), + rateLabel: item.rate === null || item.rate === undefined ? '' : formatPercent(item.rate), + statusLabel: item.status === 'ok' ? '正常' : (item.status === 'attention' ? '需关注' : '待接入') + })) +} + +export function buildDataQualityRows(dashboard = {}) { + const quality = dashboard.dataQuality || {} + return Object.entries(DATA_QUALITY_LABELS).map(([key, label]) => { + const available = Object.hasOwn(quality, key) + const count = available ? Number(quality[key] || 0) : null + return { + key, + label, + count, + displayValue: available ? String(count) : '—', + tone: available ? (count > 0 ? 'attention' : 'ok') : 'unavailable' + } + }) +} + +export function buildOpportunityRows(payload = {}) { + return (Array.isArray(payload.items) ? payload.items : []).map((item) => ({ + ...item, + statusMeta: STATUS_META[item.status] || { label: item.status || '未知', tone: 'muted' }, + estimatedLabel: formatMoney(item.estimatedNet, item.reportingCurrency || item.currency), + rangeLabel: `${formatMoney(item.estimatedLow, item.reportingCurrency || item.currency)} ~ ${formatMoney(item.estimatedHigh, item.reportingCurrency || item.currency)}`, + dueLabel: formatDateTime(item.dueAt), + updatedLabel: formatDateTime(item.updatedAt), + confidenceLabel: formatPercent(item.confidence), + ownerLabel: item.ownerName || item.ownerId || '未分配' + })) +} + +export function resolveActionMeta(action) { + return ACTION_META[action] || { label: action || '执行动作', tone: 'primary', kind: 'opportunity' } +} + +export function createRequestId(prefix = 'value') { + if (globalThis.crypto?.randomUUID) { + return `${prefix}-${globalThis.crypto.randomUUID()}` + } + return `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2, 12)}` +} + +export function formatMoneyValues(values = []) { + const rows = Array.isArray(values) ? values : [] + if (!rows.length) return '—' + return rows.map((item) => formatMoney(item.amount, item.currency)).join(' / ') +} + +export function formatMoney(amount, currency = 'CNY') { + const number = Number(amount || 0) + const normalizedCurrency = String(currency || 'CNY').toUpperCase() + try { + return new Intl.NumberFormat('zh-CN', { + style: 'currency', + currency: normalizedCurrency, + minimumFractionDigits: 0, + maximumFractionDigits: 2 + }).format(number) + } catch { + return `${normalizedCurrency} ${number.toLocaleString('zh-CN', { maximumFractionDigits: 2 })}` + } +} + +export function formatPercent(value) { + const number = Number(value) + if (!Number.isFinite(number)) return '—' + return `${(number * 100).toLocaleString('zh-CN', { maximumFractionDigits: 1 })}%` +} + +export function formatDateTime(value) { + if (!value) return '—' + const date = new Date(value) + if (Number.isNaN(date.getTime())) return String(value) + return date.toLocaleString('zh-CN', { + month: '2-digit', + day: '2-digit', + hour: '2-digit', + minute: '2-digit', + hour12: false + }) +} + +function splitBreakdownItemByCurrency(item = {}) { + const verifiedValues = Array.isArray(item.verifiedValues) ? item.verifiedValues : [] + const estimatedValues = Array.isArray(item.estimatedValues) ? item.estimatedValues : [] + const currencies = [...new Set( + [...verifiedValues, ...estimatedValues].map((value) => String(value.currency || '')).filter(Boolean) + )] + const rows = currencies.length ? currencies : [''] + return rows.map((currency) => { + const verified = verifiedValues.find((value) => value.currency === currency) + const estimated = estimatedValues.find((value) => value.currency === currency) + const verifiedAmount = Number(verified?.amount || 0) + return { + ...item, + rowKey: `${item.dimensionId || item.dimensionName}:${currency || 'no-currency'}`, + currency, + verifiedLabel: currency ? formatMoney(verifiedAmount, currency) : '—', + estimatedLabel: currency ? formatMoney(estimated?.amount || 0, currency) : '—', + verifiedMagnitude: Math.abs(verifiedAmount) + } + }) +} + +function resolveZeroCashLabel(dashboard = {}) { + const currencies = new Set() + const stages = Array.isArray(dashboard.funnel?.stages) ? dashboard.funnel.stages : [] + stages.forEach((stage) => { + const values = Array.isArray(stage.values) ? stage.values : [] + values.forEach((item) => { + if (item.currency) currencies.add(String(item.currency)) + }) + }) + if (!currencies.size) return '0（无确认币种）' + return [...currencies].map((currency) => formatMoney(0, currency)).join(' / ') +} + +function snakeCase(value) { + return String(value || '').replace(/[A-Z]/gu, (letter) => `_${letter.toLowerCase()}`) +} diff --git a/web/src/views/scripts/cfoValueSourceLinks.js b/web/src/views/scripts/cfoValueSourceLinks.js new file mode 100644 index 0000000..1cdfe94 --- /dev/null +++ b/web/src/views/scripts/cfoValueSourceLinks.js @@ -0,0 +1,242 @@ +const VALUE_OPPORTUNITY_QUERY_KEY = 'value_opportunity' +const OPPORTUNITY_ID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/iu + +const DIMENSION_LINKS = [ + { key: 'departmentId', dimensionKey: 'department_id', queryKey: 'value_department_id', label: '部门' }, + { key: 'projectCode', dimensionKey: 'project_code', queryKey: 'value_project_code', label: '项目' }, + { key: 'expenseType', dimensionKey: 'expense_type', queryKey: 'value_expense_type', label: '费用类型' }, + { key: 'supplierId', dimensionKey: 'supplier_id', queryKey: 'value_supplier_id', label: '供应商' }, + { key: 'city', dimensionKey: 'city', queryKey: 'value_city', label: '城市' } +] + +export const BUDGET_CONFIGURATION_NOTICE = '预算配置视图只用于核对预算规则、范围与阈值，不代表当前节省机会的真实预算金额。' + +export function normalizeValueOpportunityId(value) { + const raw = Array.isArray(value) ? value[0] : value + const normalized = String(raw || '').trim() + return OPPORTUNITY_ID_PATTERN.test(normalized) ? normalized : '' +} + +export function readValueOpportunityId(query = {}) { + return normalizeValueOpportunityId(query[VALUE_OPPORTUNITY_QUERY_KEY]) +} + +export function hasValueOpportunityQuery(query = {}) { + return Object.hasOwn(query, VALUE_OPPORTUNITY_QUERY_KEY) +} + +export function writeValueOpportunityToQuery(query = {}, opportunityId = '') { + const result = { ...query } + const normalized = normalizeValueOpportunityId(opportunityId) + if (normalized) result[VALUE_OPPORTUNITY_QUERY_KEY] = normalized + else delete result[VALUE_OPPORTUNITY_QUERY_KEY] + return result +} + +export function shouldClearValueOpportunityError(error) { + return [403, 404].includes(Number(error?.status || 0)) +} + +export function opportunityMatchesValueContext(opportunity, filters = {}, window = {}) { + if (!opportunity || typeof opportunity !== 'object') return false + const dimensions = opportunity.dimensionJson || {} + const comparisons = [ + [filters.departmentId, dimensions.department_id], + [filters.projectCode, dimensions.project_code], + [filters.expenseType, dimensions.expense_type], + [filters.supplierId, dimensions.supplier_id], + [filters.city, dimensions.city], + [filters.ownerId, opportunity.ownerId], + [filters.sourceType, opportunity.sourceType], + [filters.valueKind, opportunity.valueKind], + [filters.status, opportunity.status] + ] + if (comparisons.some(([expected, actual]) => expected && !sameFilterValue(expected, actual))) { + return false + } + + const createdAt = new Date(opportunity.createdAt || '').getTime() + const start = window.start ? new Date(window.start).getTime() : Number.NaN + const end = window.end ? new Date(window.end).getTime() : Number.NaN + if ((Number.isFinite(start) || Number.isFinite(end)) && !Number.isFinite(createdAt)) return false + if (Number.isFinite(start) && createdAt < start) return false + if (Number.isFinite(end) && createdAt > end) return false + return true +} + +export function buildCfoOpportunitySourceLinks(opportunity = {}, options = {}) { + const dimensions = opportunity.dimensionJson || {} + const evidence = collectEvidence(opportunity) + const riskResource = evidence.find((item) => isRiskResource(item.resourceType)) + const claimResource = evidence.find((item) => isClaimResource(item.resourceType)) + const claimId = safeRouteValue(opportunity.claimId || claimResource?.resourceId) + const expenseCaseId = safeRouteValue(opportunity.expenseCaseId) + const aiDecisionId = safeRouteValue(opportunity.aiDecisionId) + const riskObservationId = safeRouteValue( + riskResource?.resourceId || (isRiskSource(opportunity) ? opportunity.sourceId : '') + ) + const returnQuery = buildValueDashboardReturnQuery(options.dashboardQuery, { includeOpportunity: true }) + const links = [] + + if (claimId) { + const riskFocused = Boolean(riskObservationId || isRiskSource(opportunity)) + links.push({ + key: riskFocused ? 'risk-claim' : 'claim', + kind: 'document', + label: riskFocused ? '查看风险来源单据' : '查看关联单据', + description: buildClaimDescription(opportunity, expenseCaseId, riskFocused), + icon: riskFocused ? 'mdi mdi-shield-search-outline' : 'mdi mdi-file-search-outline', + to: { + name: 'app-document-detail', + params: { requestId: claimId }, + query: compactQuery({ + ...returnQuery, + returnTo: 'value', + ...(expenseCaseId ? { expense_case_id: expenseCaseId } : {}), + ...(riskFocused ? { focus: 'risk' } : {}), + ...(riskObservationId ? { risk_observation_id: riskObservationId } : {}), + ...(aiDecisionId ? { ai_decision_id: aiDecisionId } : {}) + }), + ...(riskFocused ? { hash: '#risk-observation-active-detail' } : {}) + } + }) + } + + const departmentId = safeRouteValue(dimensions.department_id) + const departmentName = safeRouteValue(dimensions.department_name) + const expenseType = safeRouteValue(dimensions.expense_type) + if (departmentId || departmentName || expenseType) { + links.push({ + key: 'budget-configuration', + kind: 'budget', + label: '查看预算配置视图', + description: BUDGET_CONFIGURATION_NOTICE, + icon: 'mdi mdi-wallet-outline', + to: { + name: 'app-budget', + query: compactQuery({ + budget_view: 'configuration', + budget_department_id: departmentId, + budget_department_name: departmentName, + budget_expense_type: expenseType + }) + } + }) + } + + links.push(...buildDimensionLinks(opportunity, options.dashboardQuery)) + return links +} + +export function readBudgetConfigurationFocus(query = {}) { + const view = readQueryValue(query.budget_view) + return { + active: view === 'configuration', + departmentId: readQueryValue(query.budget_department_id), + departmentName: readQueryValue(query.budget_department_name), + expenseType: readQueryValue(query.budget_expense_type) + } +} + +function buildDimensionLinks(opportunity, dashboardQuery = {}) { + const dimensions = opportunity.dimensionJson || {} + const baseQuery = buildValueDashboardReturnQuery(dashboardQuery, { includeOpportunity: false }) + delete baseQuery.value_page + const links = DIMENSION_LINKS.flatMap((definition) => { + const value = safeRouteValue(dimensions[definition.dimensionKey]) + if (!value) return [] + return [{ + key: `dimension-${definition.key}`, + kind: 'dimension', + label: `按${definition.label}查看 CFO 分析`, + description: `返回经营价值看板，并筛选${definition.label}“${value}”。`, + icon: 'mdi mdi-chart-box-outline', + to: { + name: 'app-overview', + query: { ...baseQuery, [definition.queryKey]: value } + } + }] + }) + + const ownerId = safeRouteValue(opportunity.ownerId) + if (ownerId) { + links.push(buildTopLevelDimensionLink('owner', '负责人', 'value_owner_id', ownerId, baseQuery)) + } + const sourceType = safeRouteValue(opportunity.sourceType) + if (sourceType) { + links.push(buildTopLevelDimensionLink('source', '来源', 'value_source_type', sourceType, baseQuery)) + } + return links +} + +function buildTopLevelDimensionLink(key, label, queryKey, value, baseQuery) { + return { + key: `dimension-${key}`, + kind: 'dimension', + label: `按${label}查看 CFO 分析`, + description: `返回经营价值看板，并筛选${label}“${value}”。`, + icon: 'mdi mdi-chart-box-outline', + to: { name: 'app-overview', query: { ...baseQuery, [queryKey]: value } } + } +} + +function buildValueDashboardReturnQuery(query = {}, { includeOpportunity } = {}) { + const result = {} + Object.entries(query || {}).forEach(([key, value]) => { + const isOverviewWindow = ['range', 'start', 'end'].includes(key) + const isValueQuery = key.startsWith('value_') + if (isOverviewWindow || (isValueQuery && (includeOpportunity || key !== VALUE_OPPORTUNITY_QUERY_KEY))) { + result[key] = value + } + }) + result.dashboard = 'value' + return result +} + +function buildClaimDescription(opportunity, expenseCaseId, riskFocused) { + const claimLabel = safeRouteValue(opportunity.claimNoSnapshot || opportunity.claimId) || '关联单据' + const caseSuffix = expenseCaseId ? `，费用案件 ${expenseCaseId}` : '' + return riskFocused + ? `打开 ${claimLabel} 的风险证据位置${caseSuffix}。` + : `打开 ${claimLabel} 核对业务来源${caseSuffix}。` +} + +function collectEvidence(opportunity = {}) { + const rows = [ + ...(Array.isArray(opportunity.evidence) ? opportunity.evidence : []), + ...(Array.isArray(opportunity.evidenceJson) ? opportunity.evidenceJson : []) + ] + return rows.map((item) => ({ + resourceType: String(item?.resourceType || item?.resource_type || '').trim(), + resourceId: String(item?.resourceId || item?.resource_id || '').trim() + })) +} + +function isRiskResource(value) { + return /risk|observation/iu.test(String(value || '')) +} + +function isClaimResource(value) { + return /^(expense_)?claim$|reimbursement|application_document/iu.test(String(value || '')) +} + +function isRiskSource(opportunity = {}) { + return /risk|observation/iu.test(`${opportunity.sourceType || ''} ${opportunity.category || ''}`) +} + +function sameFilterValue(left, right) { + return String(left || '').trim().toLocaleLowerCase('zh-CN') === String(right || '').trim().toLocaleLowerCase('zh-CN') +} + +function safeRouteValue(value) { + const normalized = String(value || '').trim() + return normalized && normalized.length <= 160 ? normalized : '' +} + +function readQueryValue(value) { + return safeRouteValue(Array.isArray(value) ? value[0] : value) +} + +function compactQuery(query = {}) { + return Object.fromEntries(Object.entries(query).filter(([, value]) => value !== '' && value !== undefined && value !== null)) +} diff --git a/web/src/views/scripts/receiptFolderFormatting.js b/web/src/views/scripts/receiptFolderFormatting.js new file mode 100644 index 0000000..664cc47 --- /dev/null +++ b/web/src/views/scripts/receiptFolderFormatting.js @@ -0,0 +1,12 @@ +export function formatReceiptRecognitionScore(value) { + const score = Number(value || 0) + if (!Number.isFinite(score) || score <= 0) return '待确认' + return `${Math.round(score * 100)}%` +} + +export function formatReceiptDateTime(value) { + if (!String(value ?? '').trim()) return '待确认' + const date = new Date(value) + if (Number.isNaN(date.getTime())) return '待确认' + return `${date.getFullYear()}-${String(date.getMonth() + 1).padStart(2, '0')}-${String(date.getDate()).padStart(2, '0')} ${String(date.getHours()).padStart(2, '0')}:${String(date.getMinutes()).padStart(2, '0')}` +} diff --git a/web/src/views/scripts/stewardPlanFields.js b/web/src/views/scripts/stewardPlanFields.js index 9e5d615..77857cd 100644 --- a/web/src/views/scripts/stewardPlanFields.js +++ b/web/src/views/scripts/stewardPlanFields.js @@ -112,3 +112,88 @@ export function formatStewardFieldDisplayValue(field, value) { const normalizedValue = String(value || '').trim() return FIELD_VALUE_DISPLAY_CONFIG[key]?.[normalizedValue] || normalizedValue } + +export function buildStewardFieldItems(fields = [], taskType = '') { + const safeFields = filterStewardBlockingMissingFields(fields, taskType) + const seen = new Set() + return safeFields + .map((field) => normalizeFieldKey(field)) + .filter((field) => { + if (!field || seen.has(field)) { + return false + } + seen.add(field) + return true + }) + .map((field) => resolveFieldDisplay(field, taskType)) +} + +export function formatStewardMissingFieldList(fields = [], taskType = '', options = {}) { + const includeHints = options.includeHints !== false + return buildStewardFieldItems(fields, taskType) + .map((item) => includeHints && item.hint ? `${item.label}（${item.hint}）` : item.label) + .join('、') +} + +export function filterStewardBlockingMissingFields(fields = [], taskType = '') { + const safeFields = Array.isArray(fields) ? fields : [] + const seen = new Set() + if (taskType !== 'expense_application') { + return safeFields + .map((field) => normalizeFieldKey(field)) + .filter((field) => { + if (!field || seen.has(field)) { + return false + } + seen.add(field) + return true + }) + } + return safeFields + .map((field) => normalizeFieldKey(field)) + .filter((field) => { + if (!field || seen.has(field) || APPLICATION_NON_BLOCKING_MISSING_FIELDS.has(field)) { + return false + } + seen.add(field) + return true + }) +} + +export function formatStewardOntologyFields(fields = {}, taskType = '') { + return Object.entries(fields || {}) + .filter(([, value]) => String(value || '').trim()) + .map(([key, value]) => { + const field = resolveFieldDisplay(key, taskType) + return `${field.label}：${formatStewardFieldDisplayValue(field.key, value)}` + }) + .join('；') +} + +function buildStewardOntologyFieldRows(fields = {}, taskType = '') { + return Object.entries(fields || {}) + .filter(([, value]) => String(value || '').trim()) + .map(([key, value]) => { + const field = resolveFieldDisplay(key, taskType) + return { + label: field.label, + value: formatStewardFieldDisplayValue(field.key, value) + } + }) +} + +function escapeMarkdownTableCell(value) { + return String(value || '').replace(/\|/g, '\\|').replace(/\n+/g, ' ').trim() +} + +export function formatStewardOntologyFieldsTable(fields = {}, taskType = '') { + const rows = buildStewardOntologyFieldRows(fields, taskType) + if (!rows.length) { + return '' + } + return [ + '| 字段 | 内容 |', + '| --- | --- |', + ...rows.map((row) => `| ${escapeMarkdownTableCell(row.label)} | ${escapeMarkdownTableCell(row.value)} |`) + ].join('\n') +} diff --git a/web/src/views/scripts/stewardPlanModel.js b/web/src/views/scripts/stewardPlanModel.js index 84644b7..9a506de 100644 --- a/web/src/views/scripts/stewardPlanModel.js +++ b/web/src/views/scripts/stewardPlanModel.js @@ -7,13 +7,21 @@ import { SESSION_TYPE_EXPENSE } from './travelReimbursementConversationModel.js' import { - APPLICATION_NON_BLOCKING_MISSING_FIELDS, FLOW_EXPENSE_TYPE_LABELS, - formatStewardFieldDisplayValue, - normalizeFieldKey, - resolveFieldDisplay + buildStewardFieldItems, + filterStewardBlockingMissingFields, + formatStewardMissingFieldList, + formatStewardOntologyFields, + formatStewardOntologyFieldsTable } from './stewardPlanFields.js' +export { + buildStewardFieldItems, + filterStewardBlockingMissingFields, + formatStewardMissingFieldList, + formatStewardOntologyFields +} + const TASK_TYPE_LABELS = { expense_application: '费用申请', reimbursement: '费用报销', @@ -198,91 +206,6 @@ export function buildStewardPlanMessageText(plan) { ].filter((line, index, lines) => line || lines[index - 1]).join('\n') } -export function buildStewardFieldItems(fields = [], taskType = '') { - const safeFields = filterStewardBlockingMissingFields(fields, taskType) - const seen = new Set() - return safeFields - .map((field) => normalizeFieldKey(field)) - .filter((field) => { - if (!field || seen.has(field)) { - return false - } - seen.add(field) - return true - }) - .map((field) => resolveFieldDisplay(field, taskType)) -} - -export function formatStewardMissingFieldList(fields = [], taskType = '', options = {}) { - const includeHints = options.includeHints !== false - return buildStewardFieldItems(fields, taskType) - .map((item) => includeHints && item.hint ? `${item.label}（${item.hint}）` : item.label) - .join('、') -} - -export function filterStewardBlockingMissingFields(fields = [], taskType = '') { - const safeFields = Array.isArray(fields) ? fields : [] - const seen = new Set() - if (taskType !== 'expense_application') { - return safeFields - .map((field) => normalizeFieldKey(field)) - .filter((field) => { - if (!field || seen.has(field)) { - return false - } - seen.add(field) - return true - }) - } - return safeFields - .map((field) => normalizeFieldKey(field)) - .filter((field) => { - if (!field || seen.has(field) || APPLICATION_NON_BLOCKING_MISSING_FIELDS.has(field)) { - return false - } - seen.add(field) - return true - }) -} - -export function formatStewardOntologyFields(fields = {}, taskType = '') { - return Object.entries(fields || {}) - .filter(([, value]) => String(value || '').trim()) - .map(([key, value]) => { - const field = resolveFieldDisplay(key, taskType) - return `${field.label}：${formatStewardFieldDisplayValue(field.key, value)}` - }) - .join('；') -} - -function buildStewardOntologyFieldRows(fields = {}, taskType = '') { - return Object.entries(fields || {}) - .filter(([, value]) => String(value || '').trim()) - .map(([key, value]) => { - const field = resolveFieldDisplay(key, taskType) - return { - label: field.label, - value: formatStewardFieldDisplayValue(field.key, value) - } - }) -} - -function escapeMarkdownTableCell(value) { - return String(value || '').replace(/\|/g, '\\|').replace(/\n+/g, ' ').trim() -} - -function formatStewardOntologyFieldsTable(fields = {}, taskType = '') { - const rows = buildStewardOntologyFieldRows(fields, taskType) - if (!rows.length) { - return '' - } - return [ - '| 字段 | 内容 |', - '| --- | --- |', - ...rows.map((row) => `| ${escapeMarkdownTableCell(row.label)} | ${escapeMarkdownTableCell(row.value)} |`) - ].join('\n') -} - function resolveCandidateFlowExpenseType(flow = {}) { const rawType = String(flow?.ontologyFields?.expense_type || flow?.ontologyFields?.expenseType || '').trim() if (rawType === '差旅' || rawType === 'travel') { diff --git a/web/src/views/scripts/useAuditReleaseMonitor.js b/web/src/views/scripts/useAuditReleaseMonitor.js new file mode 100644 index 0000000..51cafaa --- /dev/null +++ b/web/src/views/scripts/useAuditReleaseMonitor.js @@ -0,0 +1,243 @@ +import { ref, watch } from 'vue' + +import { + fetchAgentAssetReleaseReviewQueue, + fetchAgentAssetReleaseState, + labelAgentAssetReleaseObservation +} from '../../services/agentAssets.js' + +function requestId(assetId, observationId, label) { + const suffix = typeof globalThis.crypto?.randomUUID === 'function' + ? globalThis.crypto.randomUUID() + : `${Date.now()}-${Math.random().toString(16).slice(2)}` + return `release-review:${assetId}:${observationId}:${label}:${suffix}`.slice(0, 160) +} + +export function buildReleaseMonitorMetricCards(reviewQueue = {}) { + const metrics = reviewQueue?.metrics || {} + const recall = metricNumber(metrics.recall) + const recallLowerBound = metricNumber(metrics.recall_lower_bound) + const groundTruthStatus = String(metrics.negative_ground_truth_status || '').trim() + const recallReady = groundTruthStatus.startsWith('available_') + && recall !== null + && recallLowerBound !== null + const confidence = metricNumber(metrics.recall_confidence_level) + const falseNegativeCount = metricNumber(metrics.false_negative_count) + const estimatedFalseNegativeCount = metricNumber(metrics.estimated_false_negative_count) + const falseNegativeUpperBound = metricNumber(metrics.false_negative_upper_bound) + + return [ + { label: '运行样本', value: countLabel(metrics.observed_count), hint: '真实执行次数' }, + { label: '待盲审', value: countLabel(reviewQueue.pending_total), hint: '正负样本随机混排' }, + { + label: '负样本池', + value: countLabel(metrics.negative_sample_count), + hint: `已标注 ${countLabel(metrics.negative_labeled_count)}` + }, + { + label: '负样本标注进度', + value: progress(metrics.negative_labeled_count, metrics.negative_sample_count), + hint: progressHint(metrics.negative_labeled_count, metrics.negative_sample_count) + }, + { + label: '负样本积压', + value: countLabel(metrics.negative_pending_label_count), + hint: '盲审完成前不自动晋级' + }, + { + label: '随机负样本抽检', + value: progress(metrics.random_negative_labeled_count, metrics.random_negative_sample_count), + hint: randomSampleHint(metrics) + }, + { + label: '运行失败', + value: countLabel(metrics.runtime_failure_count), + hint: `失败率 ${percent(metrics.runtime_failure_rate)}` + }, + { + label: '实际漏检', + value: nullableNumber(falseNegativeCount), + hint: falseNegativeCount === null + ? '负样本真值未就绪，不按 0 展示' + : '独立盲审确认的真实漏检' + }, + { + label: '估计漏检', + value: nullableNumber(estimatedFalseNegativeCount), + hint: falseNegativeUpperBound === null + ? '保守上界不可用' + : `保守上界 ${decimal(falseNegativeUpperBound)}` + }, + { + label: '召回率', + value: recallReady ? percent(recall) : '不可用', + hint: recallReady + ? `保守下界 ${percent(recallLowerBound)}${confidence === null ? '' : ` · ${(confidence * 100).toFixed(0)}% 置信度`} · ${recallMethodLabel(metrics.recall_method)}` + : negativeGroundTruthHint(groundTruthStatus) + } + ] +} + +export function buildReleaseReviewDocumentRoute(item = {}) { + const documentId = String(item?.source_document_id || '').trim() + if (!documentId || documentId.length > 160) return null + return { name: 'app-document-detail', params: { requestId: documentId } } +} + +export function useAuditReleaseMonitor({ selectedSkill, canManageSelected, toast }) { + const releaseState = ref(null) + const releaseQueue = ref(null) + const releaseLoading = ref(false) + const releaseError = ref('') + const releaseAction = ref('') + let loadSequence = 0 + + function reset() { + releaseState.value = null + releaseQueue.value = null + releaseError.value = '' + releaseAction.value = '' + } + + async function loadReleaseMonitor() { + const assetId = String(selectedSkill.value?.id || '').trim() + const stage = String(selectedSkill.value?.configJson?.release_guard?.stage || '').trim() + if (!assetId || !selectedSkill.value?.usesJsonRiskRule || !stage || stage === 'rolled_back') { + reset() + return + } + const sequence = ++loadSequence + releaseLoading.value = true + releaseError.value = '' + try { + const [state, queue] = await Promise.all([ + fetchAgentAssetReleaseState(assetId), + fetchAgentAssetReleaseReviewQueue(assetId) + ]) + if (sequence !== loadSequence || selectedSkill.value?.id !== assetId) return + releaseState.value = state + releaseQueue.value = queue + } catch (error) { + if (sequence !== loadSequence) return + releaseError.value = error?.message || '发布遥测加载失败，请稍后重试。' + } finally { + if (sequence === loadSequence) releaseLoading.value = false + } + } + + async function submitReleaseLabel(observationId, label) { + const assetId = String(selectedSkill.value?.id || '').trim() + if (!assetId || !canManageSelected.value || releaseAction.value) return + releaseAction.value = `${observationId}:${label}` + try { + const result = await labelAgentAssetReleaseObservation(assetId, observationId, label, { + requestId: requestId(assetId, observationId, label) + }) + toast( + label === 'risk_present' + ? '已提交“存在真实风险”，发布指标已重新聚合。' + : '已提交“确认无该风险”，发布指标已重新聚合。' + ) + await loadReleaseMonitor() + return result + } catch (error) { + toast(error?.message || '发布复核提交失败，请稍后重试。') + return null + } finally { + releaseAction.value = '' + } + } + + watch( + () => [ + selectedSkill.value?.id || '', + selectedSkill.value?.configJson?.release_guard?.release_id || '', + selectedSkill.value?.configJson?.release_guard?.stage || '', + selectedSkill.value?.loading ? '1' : '0' + ], + () => { + if (selectedSkill.value?.loading) return + void loadReleaseMonitor() + }, + { immediate: true } + ) + + return { + releaseState, + releaseQueue, + releaseLoading, + releaseError, + releaseAction, + loadReleaseMonitor, + submitReleaseLabel + } +} + +function metricNumber(value) { + if (value === null || typeof value === 'undefined' || value === '') return null + const parsed = Number(value) + return Number.isFinite(parsed) ? parsed : null +} + +function countLabel(value) { + const parsed = metricNumber(value) + return parsed === null ? '不可用' : Math.max(0, parsed).toLocaleString('zh-CN') +} + +function nullableNumber(value) { + return value === null ? '不可用' : decimal(value) +} + +function decimal(value) { + const parsed = metricNumber(value) + if (parsed === null) return '不可用' + return parsed.toLocaleString('zh-CN', { maximumFractionDigits: 2 }) +} + +function percent(value) { + const parsed = metricNumber(value) + return parsed === null ? '不可用' : `${(parsed * 100).toFixed(1)}%` +} + +function progress(completed, total) { + const completedCount = metricNumber(completed) + const totalCount = metricNumber(total) + if (completedCount === null || totalCount === null || totalCount <= 0) return '不可用' + return percent(Math.min(Math.max(completedCount / totalCount, 0), 1)) +} + +function progressHint(completed, total) { + const completedCount = metricNumber(completed) + const totalCount = metricNumber(total) + if (completedCount === null || totalCount === null || totalCount <= 0) { + return '负样本证据未就绪' + } + const completedLabel = countLabel(completed) + const totalLabel = countLabel(total) + return `${completedLabel}/${totalLabel} 已完成` +} + +function randomSampleHint(metrics = {}) { + const sampleCount = metricNumber(metrics.random_negative_sample_count) + if (sampleCount === null || sampleCount <= 0) return '随机负样本证据未就绪' + const sample = countLabel(metrics.random_negative_sample_count) + const labeled = countLabel(metrics.random_negative_labeled_count) + const population = countLabel(metrics.random_negative_population_count) + if ([sample, labeled, population].includes('不可用')) return '随机负样本证据未就绪' + return `抽检 ${labeled}/${sample} · 总体 ${population}` +} + +function negativeGroundTruthHint(status) { + return { + available_stratified_random_audit: '负样本证据已就绪，但当前没有可计算分母', + collecting_candidate_labels: '正负样本标签采集中', + insufficient_random_negative_reviews: '随机负样本复核不足', + unavailable: '负样本证据未就绪' + }[status] || '负样本证据未就绪' +} + +function recallMethodLabel(method) { + return method === 'stratified_random_audit_wilson_upper_bound' + ? '分层随机盲审 Wilson 保守上界' + : '独立负样本盲审' +} diff --git a/web/tests/agent-release-monitor-panel.test.mjs b/web/tests/agent-release-monitor-panel.test.mjs new file mode 100644 index 0000000..b8cb3df --- /dev/null +++ b/web/tests/agent-release-monitor-panel.test.mjs @@ -0,0 +1,204 @@ +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import test from 'node:test' +import { fileURLToPath } from 'node:url' + +import { + labelAgentAssetReleaseObservation, + normalizeAgentAssetReleaseReviewLabel, + normalizeAgentAssetReleaseReviewQueue +} from '../src/services/agentAssets.js' +import { + buildReleaseMonitorMetricCards, + buildReleaseReviewDocumentRoute +} from '../src/views/scripts/useAuditReleaseMonitor.js' + +function source(path) { + return readFileSync(fileURLToPath(new URL(path, import.meta.url)), 'utf8') +} + +const panel = source('../src/components/audit/AuditReleaseMonitorPanel.vue') +const detail = source('../src/components/audit/AuditJsonRiskRuleDetail.vue') +const view = source('../src/views/AuditView.vue') +const service = source('../src/services/agentAssets.js') +const composable = source('../src/views/scripts/useAuditReleaseMonitor.js') + +test('盲审面板正负样本混排且不读取或暗示模型预测', () => { + assert.match(panel, /正负样本随机混排/u) + assert.match(panel, /候选与基线结论对复核人隐藏/u) + assert.match(panel, /模型结论已隐藏/u) + assert.match(panel, /存在真实风险/u) + assert.match(panel, /确认无该风险/u) + assert.doesNotMatch(panel, /candidate_hit|baseline_hit/u) + assert.doesNotMatch(service, /candidate_hit|baseline_hit/u) + assert.doesNotMatch(panel, /'confirmed'|'false_positive'/u) + assert.match(panel, /'risk_present'/u) + assert.match(panel, /'risk_absent'/u) + assert.doesNotMatch(panel, /确认命中|标记误报|待复核正例/u) + assert.match(panel, /class="review-decision"[\s\S]*class="review-decision"/u) + assert.match(panel, /:key="item\.sample_id \|\| item\.observation_id"/u) + assert.match(panel, /target="_blank"/u) + assert.match(panel, /rel="noopener noreferrer"/u) +}) + +test('复核队列只保留盲审字段并可从 source_document_id 打开单据', () => { + const queue = normalizeAgentAssetReleaseReviewQueue({ + asset_id: 'asset-1', + release_id: 'release-1', + stage: 'shadow', + version: 'v2', + pending_total: 1, + telemetry_status: 'collecting', + items: [{ + sample_id: 'sample-1', + observation_id: 'observation-1', + source_document_id: 'claim-1', + rule_code: 'RISK-001', + business_stage: 'reimbursement', + prediction_blinded: true, + reviewer_count: 1, + required_reviewers: 2, + conflicted: false, + created_at: '2026-07-17T00:00:00Z', + candidate_hit: true, + baseline_hit: false + }] + }) + + assert.deepEqual(Object.keys(queue.items[0]).sort(), [ + 'business_stage', 'conflicted', 'created_at', 'observation_id', 'prediction_blinded', + 'required_reviewers', 'reviewer_count', 'rule_code', 'sample_id', 'source_document_id' + ]) + assert.equal(queue.items[0].prediction_blinded, true) + assert.deepEqual(buildReleaseReviewDocumentRoute(queue.items[0]), { + name: 'app-document-detail', + params: { requestId: 'claim-1' } + }) + assert.equal(buildReleaseReviewDocumentRoute({ source_document_id: '' }), null) +}) + +test('负样本证据未就绪时召回和实际漏检不可用，null 不伪装为 0', () => { + const queue = normalizeAgentAssetReleaseReviewQueue({ + pending_total: 4, + metrics: { + observed_count: 20, + runtime_failure_count: 1, + runtime_failure_rate: 0.05, + negative_sample_count: 8, + negative_labeled_count: 3, + negative_pending_label_count: 5, + false_negative_count: null, + estimated_false_negative_count: null, + false_negative_upper_bound: null, + random_negative_population_count: 12, + random_negative_sample_count: 4, + random_negative_labeled_count: 1, + recall: null, + recall_lower_bound: null, + recall_confidence_level: 0.95, + recall_method: 'stratified_random_audit_wilson_upper_bound', + negative_ground_truth_status: 'insufficient_random_negative_reviews' + } + }) + const cards = cardsByLabel(buildReleaseMonitorMetricCards(queue)) + + assert.equal(queue.metrics.false_negative_count, null) + assert.equal(cards['实际漏检'].value, '不可用') + assert.match(cards['实际漏检'].hint, /不按 0 展示/u) + assert.equal(cards['估计漏检'].value, '不可用') + assert.equal(cards['召回率'].value, '不可用') + assert.equal(cards['召回率'].hint, '随机负样本复核不足') + assert.equal(cards['负样本积压'].value, '5') + assert.equal(cards['负样本标注进度'].value, '37.5%') + assert.equal(cards['随机负样本抽检'].value, '25.0%') +}) + +test('真实负样本证据显示召回点估计、保守下界和漏检上界', () => { + const cards = cardsByLabel(buildReleaseMonitorMetricCards({ + pending_total: 2, + metrics: { + observed_count: 100, + runtime_failure_count: 0, + runtime_failure_rate: 0, + negative_sample_count: 20, + negative_labeled_count: 18, + negative_pending_label_count: 2, + false_negative_count: 2, + estimated_false_negative_count: 2.5, + false_negative_upper_bound: 4.75, + random_negative_population_count: 40, + random_negative_sample_count: 10, + random_negative_labeled_count: 10, + recall: 0.8, + recall_lower_bound: 0.64, + recall_confidence_level: 0.95, + recall_method: 'stratified_random_audit_wilson_upper_bound', + negative_ground_truth_status: 'available_stratified_random_audit' + } + })) + + assert.equal(cards['实际漏检'].value, '2') + assert.equal(cards['估计漏检'].value, '2.5') + assert.equal(cards['估计漏检'].hint, '保守上界 4.75') + assert.equal(cards['负样本标注进度'].value, '90.0%') + assert.equal(cards['负样本积压'].value, '2') + assert.equal(cards['召回率'].value, '80.0%') + assert.match(cards['召回率'].hint, /保守下界 64\.0% · 95% 置信度/u) + assert.match(cards['召回率'].hint, /分层随机盲审 Wilson 保守上界/u) +}) + +test('发布复核使用独立租户鉴权 API、幂等请求号和中性结果文案', () => { + assert.match(service, /\/release\/review-queue/u) + assert.match(service, /\/release\/review-queue\/\$\{observationId\}\/labels/u) + assert.match(service, /normalizeAgentAssetReleaseReviewQueue\(payload\)/u) + assert.match(composable, /requestId:\s*requestId\(assetId, observationId, label\)/u) + assert.match(composable, /Promise\.all\(\[/u) + assert.match(composable, /fetchAgentAssetReleaseState/u) + assert.match(composable, /fetchAgentAssetReleaseReviewQueue/u) + assert.match(composable, /已提交“存在真实风险”/u) + assert.match(composable, /已提交“确认无该风险”/u) + assert.equal(normalizeAgentAssetReleaseReviewLabel('risk_present'), 'risk_present') + assert.equal(normalizeAgentAssetReleaseReviewLabel('risk_absent'), 'risk_absent') + assert.throws(() => normalizeAgentAssetReleaseReviewLabel('confirmed'), /risk_present/u) + assert.throws(() => normalizeAgentAssetReleaseReviewLabel('false_positive'), /risk_absent/u) +}) + +test('复核服务只发送 risk_present 或 risk_absent 新语义', async () => { + const originalFetch = globalThis.fetch + const bodies = [] + globalThis.fetch = async (_url, options) => { + bodies.push(JSON.parse(options.body)) + return new Response(JSON.stringify({ label: options.body.includes('risk_present') ? 'risk_present' : 'risk_absent' }), { + status: 200, + headers: { 'content-type': 'application/json' } + }) + } + + try { + await labelAgentAssetReleaseObservation('asset-1', 'observation-1', 'risk_present', { + actor: 'auditor', requestId: 'blind-review-present' + }) + await labelAgentAssetReleaseObservation('asset-1', 'observation-2', 'risk_absent', { + actor: 'auditor', requestId: 'blind-review-absent' + }) + assert.throws( + () => labelAgentAssetReleaseObservation('asset-1', 'observation-3', 'confirmed'), + /risk_present/u + ) + } finally { + globalThis.fetch = originalFetch + } + + assert.deepEqual(bodies, [{ label: 'risk_present' }, { label: 'risk_absent' }]) +}) + +test('规则详情和审计页复用独立发布监控组件', () => { + assert.match(detail, / [card.label, card])) +} diff --git a/web/tests/app-shell-financial-assistant-entry.test.mjs b/web/tests/app-shell-financial-assistant-entry.test.mjs index 10c27fd..5ece5fa 100644 --- a/web/tests/app-shell-financial-assistant-entry.test.mjs +++ b/web/tests/app-shell-financial-assistant-entry.test.mjs @@ -116,12 +116,16 @@ test('documents center uses the full request list instead of the global date-fil }) test('workbench summary merges approval inbox requests without polluting document center rows', () => { - assert.match(appShellComposable, /import \{ fetchAllApprovalExpenseClaims, fetchExpenseClaimDetail \} from '\.\.\/services\/reimbursements\.js'/) + assert.match( + appShellComposable, + /import \{\s*REIMBURSEMENT_LIST_PREVIEW_PARAMS,\s*extractExpenseClaimItems,\s*fetchApprovalExpenseClaims,\s*fetchExpenseClaimDetail\s*\} from '\.\.\/services\/reimbursements\.js'/ + ) assert.match(appShellComposable, /const workbenchApprovalRequests = ref\(\[\]\)/) assert.match(appShellComposable, /async function reloadWorkbenchApprovalRequests\(\)/) assert.match(appShellComposable, /async function reloadWorkbenchRequests\(\)/) - assert.match(appShellComposable, /fetchAllApprovalExpenseClaims\(\)/) - assert.match(appShellComposable, /payload\.map\(\(item\) => mapExpenseClaimToRequest\(item\)\)/) + assert.match(appShellComposable, /fetchApprovalExpenseClaims\(REIMBURSEMENT_LIST_PREVIEW_PARAMS\)/) + assert.match(appShellComposable, /extractExpenseClaimItems\(payload\)\.map\(\(item\) => mapExpenseClaimToRequest\(item\)\)/) + assert.doesNotMatch(appShellComposable, /fetchAllApprovalExpenseClaims\(\)/) assert.match(appShellComposable, /Promise\.all\(\[[\s\S]*reloadRequests\(\{ silent: true \}\),[\s\S]*reloadWorkbenchApprovalRequests\(\)[\s\S]*\]\)/) assert.match(appShellComposable, /if \(view === 'workbench'\) \{[\s\S]*void reloadWorkbenchRequests\(\)/) assert.match(appShellComposable, /const workbenchRequests = computed\(\(\) =>[\s\S]*mergeWorkbenchRequests\(requests\.value, workbenchApprovalRequests\.value\)/) @@ -161,7 +165,10 @@ test('document detail navigation preserves document center list query', () => { }) test('document detail refreshes claim detail instead of relying on stale list cache', () => { - assert.match(appShellComposable, /import \{ fetchAllApprovalExpenseClaims, fetchExpenseClaimDetail \} from '\.\.\/services\/reimbursements\.js'/) + assert.match( + appShellComposable, + /import \{\s*REIMBURSEMENT_LIST_PREVIEW_PARAMS,\s*extractExpenseClaimItems,\s*fetchApprovalExpenseClaims,\s*fetchExpenseClaimDetail\s*\} from '\.\.\/services\/reimbursements\.js'/ + ) assert.match(appShellComposable, /import \{ mapExpenseClaimToRequest, useRequests \} from '\.\/useRequests\.js'/) assert.match(appShellComposable, /const snapshot = normalizeRequestForUi\(selectedRequestSnapshot\.value\)[\s\S]*if \(isSameRequestIdentity\(snapshot, requestId\)\) \{[\s\S]*return snapshot/) assert.match(appShellComposable, /async function refreshSelectedRequestDetail\(requestOrId = selectedRequestSnapshot\.value\) \{[\s\S]*fetchExpenseClaimDetail\(lookupId\)[\s\S]*mapExpenseClaimToRequest\(payload\)[\s\S]*upsertRequestSnapshot\(mappedRequest\)/) diff --git a/web/tests/assistant-session-draft-delete.test.mjs b/web/tests/assistant-session-draft-delete.test.mjs index 6293aaf..685c1e1 100644 --- a/web/tests/assistant-session-draft-delete.test.mjs +++ b/web/tests/assistant-session-draft-delete.test.mjs @@ -71,10 +71,10 @@ test('claim delete flow invalidates the matching financial assistant session', ( fileURLToPath(new URL('../src/views/AppShellRouteView.vue', import.meta.url)), 'utf8' ) - const createViewScript = readFileSync( - fileURLToPath(new URL('../src/views/scripts/TravelReimbursementCreateView.js', import.meta.url)), - 'utf8' - ) + const createViewScript = [ + '../src/views/scripts/TravelReimbursementCreateView.js', + '../src/views/scripts/useTravelReimbursementCreateViewSessionCleanup.js' + ].map((path) => readFileSync(fileURLToPath(new URL(path, import.meta.url)), 'utf8')).join('\n') assert.match(appShellScript, /clearAssistantSessionSnapshotForDraftClaim/) assert.match(appShellScript, /async function handleRequestDeleted\(payload = \{\}\)/) @@ -194,8 +194,12 @@ test('saving a draft keeps the financial assistant open for continued work', () }) test('detail smart entry is scoped to the current claim instead of the latest conversation', () => { - const detailViewScript = readFileSync( - fileURLToPath(new URL('../src/views/scripts/TravelRequestDetailView.js', import.meta.url)), + const detailExpenseEditorScript = readFileSync( + fileURLToPath(new URL('../src/views/scripts/useTravelRequestDetailExpenseEditor.js', import.meta.url)), + 'utf8' + ) + const smartEntryRecognitionScript = readFileSync( + fileURLToPath(new URL('../src/views/scripts/travelRequestDetailSmartEntryRecognition.js', import.meta.url)), 'utf8' ) const appShellScript = readFileSync( @@ -206,13 +210,17 @@ test('detail smart entry is scoped to the current claim instead of the latest co fileURLToPath(new URL('../src/views/scripts/useTravelReimbursementSessionState.js', import.meta.url)), 'utf8' ) - const submitComposerScript = readFileSync( - fileURLToPath(new URL('../src/views/scripts/useTravelReimbursementSubmitComposer.js', import.meta.url)), - 'utf8' - ) + const submitComposerScript = [ + '../src/views/scripts/useTravelReimbursementSubmitComposer.js', + '../src/views/scripts/travelReimbursementSubmitApplicationPreview.js' + ].map((path) => readFileSync(fileURLToPath(new URL(path, import.meta.url)), 'utf8')).join('\n') - assert.match(detailViewScript, /restoreLatestConversation:\s*false/) - assert.match(detailViewScript, /scope:\s*claimId[\s\S]*type:\s*'claim'[\s\S]*claimId/) + assert.match(detailExpenseEditorScript, /if \(!request\.value\.claimId\) \{[\s\S]*当前草稿缺少 claimId/) + assert.match(detailExpenseEditorScript, /startSmartEntryRecognitionTask\(\{[\s\S]*claimId:\s*request\.value\.claimId/) + assert.match(detailExpenseEditorScript, /bindSmartEntryRecognitionTask\(request\.value\.claimId\)/) + assert.match(smartEntryRecognitionScript, /const normalizedClaimId = normalizeSmartEntryClaimId\(claimId\)/) + assert.match(smartEntryRecognitionScript, /smartEntryRecognitionTasks\.set\(normalizedClaimId, task\)/) + assert.doesNotMatch(detailExpenseEditorScript, /restoreLatestConversation/) assert.match(appShellScript, /function isDetailClaimScopedPayload\(payload = \{\}\)/) assert.match(appShellScript, /if \(isDetailClaimScopedPayload\(payload\)\) \{[\s\S]*return null[\s\S]*\}/) assert.match(sessionStateScript, /const shouldPersistLocalSnapshot = props\.entrySource !== 'detail'/) diff --git a/web/tests/cfo-value-dashboard.test.mjs b/web/tests/cfo-value-dashboard.test.mjs new file mode 100644 index 0000000..517c8ad --- /dev/null +++ b/web/tests/cfo-value-dashboard.test.mjs @@ -0,0 +1,426 @@ +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import test from 'node:test' +import { fileURLToPath } from 'node:url' + +import { + buildCfoValueSearch, + buildSavingsOpportunitySearch, + normalizeAnalyticsValuePayload, + recordSavingsRealization +} from '../src/services/analyticsValue.js' +import { + buildBreakdownGroups, + buildDataQualityRows, + buildValueKpis, + classifyCfoDashboardState, + formatMoneyValues, + isDashboardStale, + readValueFiltersFromQuery, + resolveRangeWindow, + writeValueFiltersToQuery +} from '../src/views/scripts/cfoValueDashboardModel.js' +import { + BUDGET_CONFIGURATION_NOTICE, + buildCfoOpportunitySourceLinks, + hasValueOpportunityQuery, + normalizeValueOpportunityId, + opportunityMatchesValueContext, + readBudgetConfigurationFocus, + readValueOpportunityId, + shouldClearValueOpportunityError, + writeValueOpportunityToQuery +} from '../src/views/scripts/cfoValueSourceLinks.js' + +const overviewView = readSource('../src/views/OverviewView.vue') +const topBarRange = readSource('../src/components/layout/useTopBarOverviewRange.js') +const appShell = readSource('../src/views/AppShellRouteView.vue') +const appShellComposable = readSource('../src/composables/useAppShell.js') +const budgetCenter = readSource('../src/views/BudgetCenterView.vue') +const budgetCenterScript = readSource('../src/views/scripts/BudgetCenterView.js') +const dashboardComponent = readSource('../src/components/dashboard/CfoValueDashboard.vue') +const dashboardComposable = readSource('../src/composables/useCfoValueDashboard.js') +const dashboardModel = readSource('../src/views/scripts/cfoValueDashboardModel.js') +const valueService = readSource('../src/services/analyticsValue.js') +const actionDialog = readSource('../src/components/dashboard/CfoValueActionDialog.vue') +const opportunityDrawer = readSource('../src/components/dashboard/CfoValueOpportunityDrawer.vue') +const trendChart = readSource('../src/components/charts/CfoValueTrendChart.vue') +const dashboardStyles = readSource('../src/assets/styles/components/cfo-value-dashboard.css') + +test('经营价值接口把后端 snake_case 契约规范为前端字段且保留币种分账', () => { + const normalized = normalizeAnalyticsValuePayload({ + source: { data_status: 'complete', opportunity_count: 2 }, + funnel: { realization_rate_by_currency: { CNY: '0.2500', USD: null } }, + data_quality: { pending_confirmation_count: 1 } + }) + + assert.equal(normalized.source.dataStatus, 'complete') + assert.equal(normalized.source.opportunityCount, 2) + assert.equal(normalized.funnel.realizationRateByCurrency.CNY, '0.2500') + assert.equal(normalized.funnel.realizationRateByCurrency.USD, null) + assert.equal(normalized.dataQuality.pendingConfirmationCount, 1) +}) + +test('经营价值与机会台账查询只发送真实筛选字段', () => { + const dashboardSearch = buildCfoValueSearch({ + start: '2026-07-01T00:00:00.000Z', + end: '2026-07-16T00:00:00.000Z', + departmentId: 'dept-1', + projectCode: '', + valueKind: 'cash' + }) + const opportunitySearch = buildSavingsOpportunitySearch({ + page: 2, + pageSize: 12, + status: 'in_progress', + ownerId: 'finance-1' + }) + + assert.equal(dashboardSearch.get('department_id'), 'dept-1') + assert.equal(dashboardSearch.get('value_kind'), 'cash') + assert.equal(dashboardSearch.has('project_code'), false) + assert.equal(opportunitySearch.get('page_size'), '12') + assert.equal(opportunitySearch.get('status'), 'in_progress') + assert.equal(opportunitySearch.get('owner_id'), 'finance-1') +}) + +test('看板明确区分 loading permission error empty partial stale 与 ready', () => { + const complete = dashboardFixture({ dataStatus: 'complete' }) + const partial = dashboardFixture({ dataStatus: 'partial' }) + const empty = dashboardFixture({ dataStatus: 'empty', opportunityCount: 0, realizationCount: 0 }) + const stale = dashboardFixture({ dataStatus: 'complete', freshnessAt: '2026-07-01T00:00:00Z' }) + + assert.equal(classifyCfoDashboardState({ loading: true }), 'loading') + assert.equal(classifyCfoDashboardState({ error: { status: 403 } }), 'permission') + assert.equal(classifyCfoDashboardState({ error: new Error('offline') }), 'error') + assert.equal(classifyCfoDashboardState({ dashboard: empty }), 'empty') + assert.equal(classifyCfoDashboardState({ dashboard: partial }), 'partial') + assert.equal(classifyCfoDashboardState({ dashboard: dashboardFixture({ dataStatus: 'complete', freshnessAt: '' }) }), 'partial') + assert.equal(classifyCfoDashboardState({ dashboard: complete, now: new Date('2026-07-16T01:00:00Z') }), 'ready') + assert.equal(classifyCfoDashboardState({ dashboard: stale, now: new Date('2026-07-16T01:00:00Z') }), 'stale') + assert.equal(isDashboardStale(stale, new Date('2026-07-16T01:00:00Z')), true) +}) + +test('数据质量缺字段显示不可用而不是伪装成零缺口', () => { + const rows = buildDataQualityRows({ dataQuality: { pendingConfirmationCount: 0 } }) + assert.deepEqual(rows[0], { + key: 'pendingConfirmationCount', + label: '待财务确认', + count: 0, + displayValue: '0', + tone: 'ok' + }) + assert.equal(rows[1].count, null) + assert.equal(rows[1].displayValue, '—') + assert.equal(rows[1].tone, 'unavailable') +}) + +test('零节省、空台账与缺失工时基线不会混为默认数字', () => { + const dashboard = dashboardFixture({ dataStatus: 'complete', opportunityCount: 2, realizationCount: 1 }) + dashboard.kpis = { + verifiedCash: { label: '财务确认净现金节省', status: 'empty', values: [], confirmedRealizationCount: 0 }, + releasableLabor: { label: '财务确认可释放工时价值', status: 'collecting', reason: '缺少工时基线', requiredInputs: ['人工分钟'] }, + safeStraightThrough: { label: '安全智能直通率', status: 'collecting', reason: '缺少审计样本', requiredInputs: ['审计结果'] } + } + dashboard.funnel = { stages: [{ key: 'estimated', values: [{ currency: 'USD', amount: '50.00' }] }] } + + const [cash, labor, straightThrough] = buildValueKpis(dashboard) + assert.equal(cash.state, 'zero') + assert.match(cash.displayValue, /US\$0|\$0/u) + assert.equal(labor.state, 'baseline-missing') + assert.equal(labor.displayValue, '待采集') + assert.equal(straightThrough.displayValue, '待采集') + assert.match( + formatMoneyValues([{ currency: 'CNY', amount: '10' }, { currency: 'USD', amount: '20' }]), + /¥10 \/ (?:US)?\$20/u + ) + + dashboard.filters = { valueKind: 'labor' } + const [excludedCash] = buildValueKpis(dashboard) + assert.equal(excludedCash.state, 'unavailable') + assert.equal(excludedCash.displayValue, '未纳入筛选') +}) + +test('价值分解按币种拆行，绝不把不同币种相加后比较', () => { + const groups = buildBreakdownGroups({ + breakdowns: [{ + dimension: 'department', + items: [{ + dimensionId: 'dept-1', + dimensionName: '财务部', + opportunityCount: 2, + verifiedValues: [ + { currency: 'CNY', amount: '100' }, + { currency: 'USD', amount: '20' } + ], + estimatedValues: [ + { currency: 'CNY', amount: '180' }, + { currency: 'USD', amount: '30' } + ] + }] + }] + }) + + assert.equal(groups[0].items.length, 2) + assert.deepEqual(groups[0].items.map((item) => item.currency), ['CNY', 'USD']) + assert.deepEqual(groups[0].items.map((item) => item.verifiedMagnitude), [100, 20]) + assert.ok(groups[0].items.every((item) => item.width === '100%')) +}) + +test('筛选条件和时间窗口可以通过 URL 恢复', () => { + const filters = readValueFiltersFromQuery({ + value_department_id: 'dept-7', + value_value_kind: 'cash', + value_status: 'verified' + }) + assert.deepEqual( + { departmentId: filters.departmentId, valueKind: filters.valueKind, status: filters.status }, + { departmentId: 'dept-7', valueKind: 'cash', status: 'verified' } + ) + + const query = writeValueFiltersToQuery({ dashboard: 'value', value_city: 'old' }, { + ...filters, + city: '' + }) + assert.equal(query.dashboard, 'value') + assert.equal(query.value_department_id, 'dept-7') + assert.equal(Object.hasOwn(query, 'value_city'), false) + + const range = resolveRangeWindow('custom', { start: '2026-07-01', end: '2026-07-16' }) + assert.match(range.start, /^2026-06-30T16:00:00\.000Z$|^2026-07-01T00:00:00\.000Z$/u) + assert.ok(new Date(range.end) > new Date(range.start)) +}) + +test('机会抽屉 URL 可恢复、关闭可清理且非法 ID 不会残留', () => { + const opportunityId = '247b5e9d-f9ee-463f-a88e-6fd41757769b' + const opened = writeValueOpportunityToQuery({ dashboard: 'value', value_city: '上海' }, opportunityId) + assert.equal(readValueOpportunityId(opened), opportunityId) + assert.equal(hasValueOpportunityQuery(opened), true) + + const closed = writeValueOpportunityToQuery(opened) + assert.equal(hasValueOpportunityQuery(closed), false) + assert.equal(closed.value_city, '上海') + + const malformed = { ...opened, value_opportunity: '../tenant-b/opportunity' } + assert.equal(normalizeValueOpportunityId(malformed.value_opportunity), '') + assert.equal(readValueOpportunityId(malformed), '') + assert.equal(hasValueOpportunityQuery(writeValueOpportunityToQuery(malformed)), false) + assert.equal(shouldClearValueOpportunityError({ status: 403 }), true) + assert.equal(shouldClearValueOpportunityError({ status: 404 }), true) + assert.equal(shouldClearValueOpportunityError({ status: 500 }), false) +}) + +test('打开机会必须仍属于当前筛选和时间窗口', () => { + const opportunity = { + departmentId: 'ignored-top-level', + dimensionJson: { + department_id: 'dept-7', + project_code: 'PROJECT-1', + expense_type: 'hotel', + supplier_id: 'supplier-1', + city: '上海' + }, + ownerId: 'finance-1', + sourceType: 'risk_observation', + valueKind: 'cash', + status: 'identified', + createdAt: '2026-07-15T08:00:00Z' + } + const filters = { + departmentId: 'dept-7', projectCode: 'PROJECT-1', expenseType: 'hotel', + supplierId: 'supplier-1', city: '上海', ownerId: 'finance-1', + sourceType: 'risk_observation', valueKind: 'cash', status: 'identified' + } + const window = { start: '2026-07-01T00:00:00Z', end: '2026-07-16T23:59:59Z' } + + assert.equal(opportunityMatchesValueContext(opportunity, filters, window), true) + assert.equal(opportunityMatchesValueContext(opportunity, { ...filters, departmentId: 'dept-other' }, window), false) + assert.equal(opportunityMatchesValueContext(opportunity, filters, { ...window, end: '2026-07-14T00:00:00Z' }), false) +}) + +test('来源动作可定位风险单据、预算配置和 CFO 维度且不伪造预算事实', () => { + const opportunityId = '247b5e9d-f9ee-463f-a88e-6fd41757769b' + const links = buildCfoOpportunitySourceLinks({ + id: opportunityId, + claimId: '8a25e85a-b7d8-41dc-8717-57bd1d458fd2', + claimNoSnapshot: 'BX-20260716-001', + expenseCaseId: '04452d7a-cdb8-4665-bb9d-99b7f8fc7f1b', + aiDecisionId: 'b100ca12-8ce3-47eb-a80d-35883410545a', + sourceType: 'risk_observation', + sourceId: 'risk-fallback', + category: 'risk_avoidance', + ownerId: 'finance-1', + dimensionJson: { + department_id: 'dept-7', + department_name: '财务部', + project_code: 'PROJECT-1', + expense_type: 'hotel', + city: '上海' + }, + evidence: [{ resourceType: 'risk_observation', resourceId: 'risk-observation-7' }] + }, { + dashboardQuery: { + dashboard: 'value', + range: '本月', + value_city: '上海', + value_opportunity: opportunityId + } + }) + + const risk = links.find((item) => item.key === 'risk-claim') + assert.equal(risk.label, '查看风险来源单据') + assert.equal(risk.to.name, 'app-document-detail') + assert.equal(risk.to.params.requestId, '8a25e85a-b7d8-41dc-8717-57bd1d458fd2') + assert.equal(risk.to.query.returnTo, 'value') + assert.equal(risk.to.query.focus, 'risk') + assert.equal(risk.to.query.risk_observation_id, 'risk-observation-7') + assert.equal(risk.to.query.ai_decision_id, 'b100ca12-8ce3-47eb-a80d-35883410545a') + assert.equal(risk.to.hash, '#risk-observation-active-detail') + + const budget = links.find((item) => item.key === 'budget-configuration') + assert.equal(budget.label, '查看预算配置视图') + assert.equal(budget.to.name, 'app-budget') + assert.equal(budget.to.query.budget_department_id, 'dept-7') + assert.equal(budget.to.query.budget_expense_type, 'hotel') + assert.equal(budget.description, BUDGET_CONFIGURATION_NOTICE) + assert.match(budget.description, /不代表当前节省机会的真实预算金额/u) + + const dimension = links.find((item) => item.key === 'dimension-expenseType') + assert.equal(dimension.to.name, 'app-overview') + assert.equal(dimension.to.query.dashboard, 'value') + assert.equal(dimension.to.query.value_expense_type, 'hotel') + assert.equal(Object.hasOwn(dimension.to.query, 'value_opportunity'), false) + + assert.deepEqual(readBudgetConfigurationFocus(budget.to.query), { + active: true, + departmentId: 'dept-7', + departmentName: '财务部', + expenseType: 'hotel' + }) +}) + +test('带可追溯凭证的实际结果递归序列化为后端 snake_case', async () => { + const originalFetch = globalThis.fetch + let requestBody = null + globalThis.fetch = async (_url, options) => { + requestBody = JSON.parse(options.body) + return new Response(JSON.stringify({ realization: {}, opportunity: {}, event: {}, replayed: false }), { + status: 200, + headers: { 'content-type': 'application/json' } + }) + } + + try { + await recordSavingsRealization('opp-1', { + requestId: 'request-12345678', + expectedVersion: 2, + comment: '平台付款完成', + actualGross: '100', + incrementalCost: '10', + realizedAt: '2026-07-16T00:00:00Z', + evidenceLevel: 'external_document', + evidence: [{ + evidenceKey: 'payment-receipt-001', + evidenceRole: 'payment_receipt', + resourceType: 'payment_receipt', + resourceId: 'receipt-001', + sourceSystem: 'erp', + contentHash: '0123456789abcdef0123456789abcdef', + occurredAt: '2026-07-16T00:00:00Z', + verificationStatus: 'unverified', + metadataJson: { connector: 'erp' } + }] + }) + } finally { + globalThis.fetch = originalFetch + } + + assert.equal(requestBody.request_id, 'request-12345678') + assert.equal(requestBody.expected_version, 2) + assert.equal(requestBody.incremental_cost, '10') + assert.equal(requestBody.evidence_level, 'external_document') + assert.equal(requestBody.evidence[0].evidence_key, 'payment-receipt-001') + assert.equal(requestBody.evidence[0].metadata_json.connector, 'erp') +}) + +test('经营价值入口、独立模块、URL 深链与完整状态 UI 已接入分析看板', () => { + assert.match(topBarRange, /label: '经营价值看板', value: 'value'/u) + assert.match(overviewView, /仅筛选下方台账<\/em>/u) + assert.match(dashboardModel, /baseline-missing/u) + assert.match(dashboardModel, /真实零值/u) + assert.match(dashboardComposable, /valueOpportunityRouteSignature/u) + assert.match(dashboardComposable, /clearUnavailableOpportunity/u) + assert.match(dashboardComposable, /router\.replace\(\{ query: nextQuery \}\)/u) + assert.match(appShellComposable, /DOCUMENT_DETAIL_RETURN_TARGETS = new Set\(\['workbench', 'conversation', 'value'\]\)/u) + assert.match(appShellComposable, /name: 'app-overview', query: buildValueDashboardReturnQuery\(\)/u) + assert.match(opportunityDrawer, /id="value-source-links-title">查看来源/u) + assert.match(opportunityDrawer, /\{\{ link\.label \}\}/u) + assert.match(budgetCenter, /预算配置视图 · \{\{ budgetConfigurationFocusSummary \}\}/u) + assert.match(budgetCenter, /isFocusedBudgetCategory\(item\)/u) + assert.match(budgetCenterScript, /readBudgetConfigurationFocus\(route\.query\)/u) + assert.match(budgetCenterScript, /\.filter\(\(row\) => matchesBudgetConfigurationExpense\(row\)\)/u) + assert.match(budgetCenterScript, /这不代表预算为零/u) +}) + +test('经营价值实现不包含 demo 或 fallback 指标数字', () => { + assert.doesNotMatch(valueService, /FALLBACK|DEMO|mock/iu) + assert.doesNotMatch(dashboardComposable, /fallback|demo|mock/iu) + assert.match(dashboardComponent, /不会展示旧的演示数字/u) +}) + +test('移动端、键盘与读屏交互具备可验证的生产语义', () => { + assert.match(dashboardStyles, /@media \(max-width: 760px\)/u) + assert.match(dashboardStyles, /min-height: 44px/u) + assert.match(dashboardComponent, /role="alert"/u) + assert.match(dashboardComponent, /aria-live="polite"/u) + assert.match(actionDialog, /