from __future__ import annotations import copy import hashlib import shutil import tempfile from contextlib import contextmanager from dataclasses import dataclass from datetime import date from decimal import Decimal from pathlib import Path from threading import Lock from typing import Any from sqlalchemy import select, text from sqlalchemy.orm import Session from app.api.deps import CurrentUserContext from app.models.expense_case import BusinessEvent, ExpenseCase, ExpenseCaseLink from app.models.financial_record import ExpenseClaim, ExpenseClaimItem from app.schemas.receipt_folder import ReceiptFolderDetailRead from app.schemas.reimbursement import ExpenseClaimItemCreate from app.services.expense_cases import ExpenseCaseService from app.services.expense_claim_attachment_storage import ExpenseClaimAttachmentStorage from app.services.expense_claim_constants import DOCUMENT_TYPE_ITEM_TYPE_MAP from app.services.expense_claims import ExpenseClaimService from app.services.expense_receipt_matcher import ( ExpenseReceiptMatchCandidate, ExpenseReceiptMatcher, extract_city_tokens, normalize_date_token, ) from app.services.receipt_folder import ReceiptFolderService _claim_locks: dict[str, Lock] = {} _claim_locks_guard = Lock() @dataclass(slots=True) class _ReceiptMutation: receipt: ReceiptFolderDetailRead item_id: str original_meta: dict[str, Any] attachment_dir: str = "" attachment_backup_dir: str = "" @dataclass(slots=True) class _AssociationDbSnapshot: tenant_id: str case_ids: set[str] link_ids: set[str] event_ids: set[str] claim_id: str receipt_ids: list[str] @dataclass(slots=True) class _ClaimMutationSnapshot: claim_id: str amount: Decimal invoice_count: int risk_flags_json: list[Any] approval_stage: str submitted_at: Any status: str items: dict[str, dict[str, Any]] class ExpenseReceiptAssociationService: """高置信票据归集编排；低置信结果保持严格零写入。""" def __init__(self, db: Session) -> None: self.db = db self.claim_service = ExpenseClaimService(db) self.case_service = ExpenseCaseService(db) self.receipt_service = ReceiptFolderService() self.matcher = ExpenseReceiptMatcher(db) self.attachment_storage = ExpenseClaimAttachmentStorage() def associate( self, *, receipts: list[ReceiptFolderDetailRead], current_user: CurrentUserContext, ) -> dict[str, Any]: match = self.matcher.match(receipts=receipts, current_user=current_user) if match.requires_confirmation or match.recommended is None: return self._build_confirmation_result(match) recommended = match.recommended claim = recommended.claim if claim is None: return self._build_confirmation_result(match) initial_claim_id = str(claim.id or "") with _claim_execution_lock( self.db, tenant_id=current_user.tenant_id, claim_id=initial_claim_id, ): locked_match = self.matcher.match(receipts=receipts, current_user=current_user) locked_recommended = locked_match.recommended locked_claim = getattr(locked_recommended, "claim", None) if locked_match.requires_confirmation or locked_claim is None: return self._build_confirmation_result(locked_match) if str(locked_claim.id or "") != initial_claim_id: locked_match.resolution = "requires_confirmation" locked_match.requires_confirmation = True locked_match.exceptions.append( "并发处理期间候选报销草稿发生变化，本次未写入，请重新确认。" ) return self._build_confirmation_result(locked_match) return self._associate_matched( receipts=receipts, current_user=current_user, match=locked_match, recommended=locked_recommended, claim=locked_claim, ) def _associate_matched( self, *, receipts: list[ReceiptFolderDetailRead], current_user: CurrentUserContext, match: Any, recommended: ExpenseReceiptMatchCandidate, claim: ExpenseClaim, ) -> dict[str, Any]: claim_snapshot = self._capture_claim_snapshot(claim) db_snapshot = self._capture_db_snapshot( tenant_id=current_user.tenant_id, claim_id=claim.id, receipt_ids=[receipt.id for receipt in receipts], ) expense_case = self._ensure_expense_case( recommended, claim=claim, tenant_id=current_user.tenant_id, ) correlation_id = self.case_service.normalize_correlation_id( "zero-entry:" + ":".join(sorted(receipt.id for receipt in receipts)) ) mutations: list[_ReceiptMutation] = [] uploaded_count = 0 skipped_count = 0 try: for receipt in receipts: if self._is_linked_to_claim(receipt, claim.id): skipped_count += 1 item_id = str((receipt.raw_meta or {}).get("linked_item_id") or "").strip() self._record_receipt_events( expense_case, receipt=receipt, claim=claim, item_id=item_id, current_user=current_user, correlation_id=correlation_id, ) continue if self._is_linked_to_other_claim(receipt, claim.id): raise ValueError("票据已归属其他报销单，自动归集已停止。") target_item = self._resolve_target_item( claim_id=claim.id, receipt=receipt, current_user=current_user, ) source_path, media_type, file_name = self.receipt_service.resolve_source( receipt.id, current_user, ) mutation = self._prepare_receipt_mutation( receipt=receipt, claim_id=claim.id, item_id=target_item.id, ) mutations.append(mutation) result = self.claim_service.upload_claim_item_attachment( claim_id=claim.id, item_id=target_item.id, filename=file_name, content=source_path.read_bytes(), media_type=media_type, current_user=current_user, source_receipt_id=receipt.id, commit=False, link_source_receipt=False, write_audit=False, refresh_pre_review=False, ) if result is None: raise ValueError("无法把票据写入目标报销草稿。") self.case_service.link_resource( expense_case, resource_type="receipt", resource_id=receipt.id, relation_type="receipt", tenant_id=current_user.tenant_id, ) self._record_receipt_events( expense_case, receipt=receipt, claim=claim, item_id=target_item.id, current_user=current_user, correlation_id=correlation_id, ) self.receipt_service.mark_receipt_linked( receipt_id=receipt.id, current_user=current_user, claim_id=claim.id, claim_no=claim.claim_no, item_id=target_item.id, ) uploaded_count += 1 self.db.commit() self.db.refresh(claim) self._discard_attachment_backups(mutations) except Exception as exc: self.db.rollback() compensation_errors: list[str] = [] try: self._restore_claim_snapshot( claim_snapshot, db_snapshot=db_snapshot, expense_case_id=expense_case.id, ) except Exception as compensation_exc: compensation_errors.append(str(compensation_exc)) for mutation in reversed(mutations): try: self.receipt_service.restore_receipt_meta( receipt_id=mutation.receipt.id, current_user=current_user, meta=mutation.original_meta, ) except Exception as compensation_exc: compensation_errors.append(str(compensation_exc)) try: self._restore_attachment_directory(mutation) except Exception as compensation_exc: compensation_errors.append(str(compensation_exc)) if compensation_errors: raise RuntimeError( f"{exc}；自动补偿未完全成功：{'；'.join(compensation_errors)}" ) from exc raise return self._build_success_result( match_candidate=recommended, expense_case=expense_case, claim=claim, uploaded_count=uploaded_count, skipped_count=skipped_count, candidates=[candidate.to_payload() for candidate in match.candidates], ) def _ensure_expense_case( self, candidate: ExpenseReceiptMatchCandidate, *, claim: ExpenseClaim, tenant_id: str, ) -> ExpenseCase: if candidate.expense_case is not None: self.case_service.link_claim( candidate.expense_case, claim, tenant_id=tenant_id, relation_type="generated_reimbursement" if candidate.application_claim else "claim", ) return candidate.expense_case if candidate.application_claim is not None: expense_case = self.case_service.ensure_case_for_claim( candidate.application_claim, tenant_id=tenant_id, relation_type="application", ) self.case_service.link_claim( expense_case, claim, tenant_id=tenant_id, relation_type="generated_reimbursement", ) return expense_case return self.case_service.ensure_case_for_claim(claim, tenant_id=tenant_id) @staticmethod def _capture_claim_snapshot(claim: ExpenseClaim) -> _ClaimMutationSnapshot: return _ClaimMutationSnapshot( claim_id=str(claim.id or ""), amount=Decimal(claim.amount or Decimal("0.00")), invoice_count=int(claim.invoice_count or 0), risk_flags_json=copy.deepcopy(list(claim.risk_flags_json or [])), approval_stage=str(claim.approval_stage or ""), submitted_at=claim.submitted_at, status=str(claim.status or ""), items={ str(item.id): { "item_date": item.item_date, "item_type": item.item_type, "item_reason": item.item_reason, "item_location": item.item_location, "item_note": item.item_note, "item_amount": Decimal(item.item_amount or Decimal("0.00")), "invoice_id": item.invoice_id, } for item in list(claim.items or []) }, ) def _capture_db_snapshot( self, *, tenant_id: str, claim_id: str, receipt_ids: list[str], ) -> _AssociationDbSnapshot: normalized_tenant = self.case_service.normalize_tenant_id(tenant_id) receipt_ids = list(dict.fromkeys(str(item or "").strip() for item in receipt_ids if item)) links = list( self.db.scalars( select(ExpenseCaseLink).where( ExpenseCaseLink.tenant_id == normalized_tenant, ( (ExpenseCaseLink.resource_type == "expense_claim") & (ExpenseCaseLink.resource_id == claim_id) ) | ( (ExpenseCaseLink.resource_type == "receipt") & ExpenseCaseLink.resource_id.in_(receipt_ids) ), ) ).all() ) events = list( self.db.scalars( select(BusinessEvent).where( BusinessEvent.tenant_id == normalized_tenant, BusinessEvent.aggregate_type == "receipt", BusinessEvent.aggregate_id.in_(receipt_ids), ) ).all() ) return _AssociationDbSnapshot( tenant_id=normalized_tenant, case_ids=set( self.db.scalars( select(ExpenseCase.id).where(ExpenseCase.tenant_id == normalized_tenant) ).all() ), link_ids={str(link.id) for link in links}, event_ids={str(event.id) for event in events}, claim_id=str(claim_id or ""), receipt_ids=receipt_ids, ) def _restore_claim_snapshot( self, snapshot: _ClaimMutationSnapshot, *, db_snapshot: _AssociationDbSnapshot, expense_case_id: str, ) -> None: claim = self.db.get(ExpenseClaim, snapshot.claim_id) if claim is None: return for item in list(claim.items or []): item_id = str(item.id or "") item_snapshot = snapshot.items.get(item_id) if item_snapshot is None: claim.items.remove(item) self.db.delete(item) continue for key, value in item_snapshot.items(): setattr(item, key, value) claim.amount = snapshot.amount claim.invoice_count = snapshot.invoice_count claim.risk_flags_json = copy.deepcopy(snapshot.risk_flags_json) claim.approval_stage = snapshot.approval_stage claim.submitted_at = snapshot.submitted_at claim.status = snapshot.status for event in list( self.db.scalars( select(BusinessEvent).where( BusinessEvent.tenant_id == db_snapshot.tenant_id, BusinessEvent.aggregate_type == "receipt", BusinessEvent.aggregate_id.in_(db_snapshot.receipt_ids), ) ).all() ): if str(event.id) not in db_snapshot.event_ids: self.db.delete(event) for link in list( self.db.scalars( select(ExpenseCaseLink).where( ExpenseCaseLink.tenant_id == db_snapshot.tenant_id, ( (ExpenseCaseLink.resource_type == "expense_claim") & (ExpenseCaseLink.resource_id == db_snapshot.claim_id) ) | ( (ExpenseCaseLink.resource_type == "receipt") & ExpenseCaseLink.resource_id.in_(db_snapshot.receipt_ids) ), ) ).all() ): if str(link.id) not in db_snapshot.link_ids: self.db.delete(link) if expense_case_id not in db_snapshot.case_ids: created_case = self.db.get(ExpenseCase, expense_case_id) if created_case is not None: self.db.delete(created_case) self.db.commit() def _prepare_receipt_mutation( self, *, receipt: ReceiptFolderDetailRead, claim_id: str, item_id: str, ) -> _ReceiptMutation: attachment_dir = self.attachment_storage.build_item_dir( claim_id, item_id, ) backup_root = "" if attachment_dir.exists(): backup_path = Path(tempfile.mkdtemp(prefix="x-financial-attachment-backup-")) shutil.copytree(attachment_dir, backup_path / "item") backup_root = str(backup_path) return _ReceiptMutation( receipt=receipt, item_id=item_id, original_meta=copy.deepcopy(dict(receipt.raw_meta or {})), attachment_dir=str(attachment_dir), attachment_backup_dir=backup_root, ) @staticmethod def _discard_attachment_backups(mutations: list[_ReceiptMutation]) -> None: for mutation in mutations: if mutation.attachment_backup_dir: shutil.rmtree(mutation.attachment_backup_dir, ignore_errors=True) @staticmethod def _restore_attachment_directory(mutation: _ReceiptMutation) -> None: attachment_dir = mutation.attachment_dir if not attachment_dir: return shutil.rmtree(attachment_dir, ignore_errors=True) backup_dir = mutation.attachment_backup_dir backup_item_dir = Path(backup_dir) / "item" if backup_dir else None if backup_item_dir is not None and backup_item_dir.exists(): shutil.copytree(backup_item_dir, attachment_dir) if backup_dir: shutil.rmtree(backup_dir, ignore_errors=True) def _record_receipt_events( self, expense_case: ExpenseCase, *, receipt: ReceiptFolderDetailRead, claim: ExpenseClaim, item_id: str, current_user: CurrentUserContext, correlation_id: str, ) -> None: self.case_service.link_resource( expense_case, resource_type="receipt", resource_id=receipt.id, relation_type="receipt", tenant_id=current_user.tenant_id, ) received_event = self.case_service.record_resource_event( expense_case, aggregate_type="receipt", aggregate_id=receipt.id, event_type="receipt_received", actor_id=current_user.username, tenant_id=current_user.tenant_id, correlation_id=correlation_id, idempotency_key=f"receipt-received:{receipt.id}", payload={ "file_name": receipt.file_name, "document_type": receipt.document_type, "scene_code": receipt.scene_code, }, ) self.case_service.record_resource_event( expense_case, aggregate_type="receipt", aggregate_id=receipt.id, event_type="attachment_associated", actor_id=current_user.username, tenant_id=current_user.tenant_id, correlation_id=correlation_id, causation_id=received_event.id, idempotency_key=f"receipt-associated:{receipt.id}:{claim.id}", payload={ "claim_id": claim.id, "claim_no": claim.claim_no, "item_id": item_id, }, ) def _resolve_target_item( self, *, claim_id: str, receipt: ReceiptFolderDetailRead, current_user: CurrentUserContext, ) -> ExpenseClaimItem: claim = self.claim_service.get_claim(claim_id, current_user) if claim is None: raise ValueError("匹配到的报销草稿不存在，请刷新后再试。") preferred_type = resolve_receipt_item_type(receipt) empty_items = [ item for item in list(claim.items or []) if not str(item.invoice_id or "").strip() and not item.is_system_generated ] for item in empty_items: if preferred_type and str(item.item_type or "").strip() == preferred_type: return item if empty_items: return empty_items[0] before_ids = {str(item.id) for item in list(claim.items or [])} created_claim = self.claim_service.create_claim_item( claim_id=claim.id, payload=build_item_payload_from_receipt(claim, receipt, preferred_type), current_user=current_user, commit=False, write_audit=False, refresh_pre_review=False, ) if created_claim is None: raise ValueError("无法创建票据归集明细，请进入详情页手动处理。") for item in list(created_claim.items or []): if str(item.id) not in before_ids and not str(item.invoice_id or "").strip(): return item raise ValueError("无法找到可归集的费用明细，请进入详情页手动处理。") @staticmethod def _is_linked_to_claim(receipt: ReceiptFolderDetailRead, claim_id: str) -> bool: return ( str(receipt.status or "").strip().lower() == "linked" and str(receipt.linked_claim_id or "").strip() == str(claim_id or "").strip() ) @staticmethod def _is_linked_to_other_claim(receipt: ReceiptFolderDetailRead, claim_id: str) -> bool: linked_claim_id = str(receipt.linked_claim_id or "").strip() return bool( str(receipt.status or "").strip().lower() == "linked" and linked_claim_id and linked_claim_id != str(claim_id or "").strip() ) @staticmethod def _build_confirmation_result(match: Any) -> dict[str, Any]: recommended = match.recommended return { "resolution": "requires_confirmation", "requires_confirmation": True, "claim_id": str(getattr(getattr(recommended, "claim", None), "id", "") or ""), "claim_no": str(getattr(getattr(recommended, "claim", None), "claim_no", "") or ""), "expense_case_id": str( getattr(getattr(recommended, "expense_case", None), "id", "") or "" ), "application_claim_id": str( getattr(getattr(recommended, "application_claim", None), "id", "") or "" ), "application_claim_no": str( getattr(getattr(recommended, "application_claim", None), "claim_no", "") or "" ), "confidence": str(getattr(recommended, "confidence", "low") or "low"), "confidence_score": float(getattr(recommended, "normalized_score", 0.0) or 0.0), "match_reasons": list(getattr(recommended, "reasons", []) or []), "exceptions": list(match.exceptions or []), "missing_fields": [], "risk_items": [], "candidates": [candidate.to_payload() for candidate in match.candidates], "uploaded_count": 0, "skipped_count": 0, "draft_payload": None, } @staticmethod def _build_success_result( *, match_candidate: ExpenseReceiptMatchCandidate, expense_case: ExpenseCase, claim: ExpenseClaim, uploaded_count: int, skipped_count: int, candidates: list[dict[str, Any]], ) -> dict[str, Any]: risk_items = [ str(flag.get("message") or flag.get("label") or "").strip() for flag in list(claim.risk_flags_json or []) if isinstance(flag, dict) and str(flag.get("severity") or "").strip().lower() in {"warning", "high", "critical"} and str(flag.get("message") or flag.get("label") or "").strip() ] application = match_candidate.application_claim return { "resolution": "auto_associated", "requires_confirmation": False, "claim_id": str(claim.id or ""), "claim_no": str(claim.claim_no or ""), "expense_case_id": str(expense_case.id or ""), "application_claim_id": str(getattr(application, "id", "") or ""), "application_claim_no": str(getattr(application, "claim_no", "") or ""), "confidence": match_candidate.confidence, "confidence_score": match_candidate.normalized_score, "match_reasons": list(match_candidate.reasons), "exceptions": [], "missing_fields": [], "risk_items": risk_items, "candidates": candidates, "uploaded_count": uploaded_count, "skipped_count": skipped_count, "draft_payload": { "draft_type": "expense", "title": f"费用草稿 {claim.claim_no}", "body": f"已自动归集 {uploaded_count} 份票据。", "confirmation_required": bool(risk_items), "claim_id": claim.id, "claim_no": claim.claim_no, "status": claim.status, "approval_stage": claim.approval_stage, "expense_type": claim.expense_type, }, } @contextmanager def _claim_execution_lock( db: Session, *, tenant_id: str, claim_id: str, ): lock_key = f"claim:{str(tenant_id or 'default').strip() or 'default'}:{claim_id}" with _claim_locks_guard: process_lock = _claim_locks.setdefault(lock_key, Lock()) process_lock.acquire() advisory_key: int | None = None try: # 初次匹配产生的只读事务可能早于互斥锁；进入锁后必须读取最新提交状态。 db.rollback() db.expire_all() if db.get_bind().dialect.name == "postgresql": advisory_key = _stable_advisory_lock_key(lock_key) db.execute( text("SELECT pg_advisory_lock(:lock_key)"), {"lock_key": advisory_key}, ) yield finally: try: if advisory_key is not None: db.rollback() db.execute( text("SELECT pg_advisory_unlock(:lock_key)"), {"lock_key": advisory_key}, ) except Exception: # 连接关闭也会释放会话级锁，解锁失败不能覆盖原始业务异常。 db.rollback() finally: process_lock.release() def _stable_advisory_lock_key(value: str) -> int: return int.from_bytes( hashlib.sha256(value.encode("utf-8")).digest()[:8], byteorder="big", signed=True, ) def resolve_receipt_item_type(receipt: ReceiptFolderDetailRead) -> str: document_type = str(receipt.document_type or "").strip() if document_type in DOCUMENT_TYPE_ITEM_TYPE_MAP: return DOCUMENT_TYPE_ITEM_TYPE_MAP[document_type] scene_code = str(receipt.scene_code or "").strip() return "travel" if scene_code == "travel" else scene_code or "other" def build_item_payload_from_receipt( claim: ExpenseClaim, receipt: ReceiptFolderDetailRead, preferred_type: str, ) -> ExpenseClaimItemCreate: item_date = resolve_receipt_item_date(receipt) or ( claim.occurred_at.date() if claim.occurred_at else None ) return ExpenseClaimItemCreate( item_date=item_date, item_type=preferred_type or str(claim.expense_type or "").strip() or "other", item_reason=str(receipt.summary or receipt.file_name or "").strip(), item_location=resolve_receipt_item_location(receipt) or str(claim.location or "").strip(), item_amount=Decimal("0.00"), ) def resolve_receipt_item_date(receipt: ReceiptFolderDetailRead) -> date | None: for value in [ *[ field.value for field in list(receipt.fields or []) if "日期" in str(field.label or "") or "时间" in str(field.label or "") ], receipt.document_date, ]: token = normalize_date_token(value) if len(token) == 10: try: return date.fromisoformat(token) except ValueError: continue return None def resolve_receipt_item_location(receipt: ReceiptFolderDetailRead) -> str: for field in list(receipt.fields or []): label = str(field.label or "") value = str(field.value or "").strip() if value and ("行程" in label or "到达" in label or "地点" in label or "城市" in label): cities = extract_city_tokens(value) return cities[-1] if cities else value[:40] cities = extract_city_tokens( "\n".join( str(value or "") for value in (receipt.file_name, receipt.summary, receipt.ocr_text) ) ) return cities[-1] if cities else ""