fix: 算力节点任务/GPU 统计纳入评测与推理占用，补充测试依赖

- compute_nodes() 的 current_running_jobs 纳入运行中的评测任务（eval_tasks）
  与已加载的推理模型（compare_tasks 持久化状态 + 内存标记兜底），
  多节点时 GPU 被评测/推理占用不再显示 0/1
- gpus() 直接按 eval_tasks / compare_tasks 派生 GPU busy/reserved 状态，
  修复 gpu_id=0 时 or -1 导致匹配失败；删除评测后 GPU 不再残留 busy
- 评测不再复用 mark_inference_loaded 内存标记，GPU 占用由任务数据驱动
- requirements.txt 补充 pytest / ruff（此前仅声明在 pyproject dev 可选依赖）

Co-Authored-By: Claude <noreply@anthropic.com>

.dockerignore

@@ -0,0 +1,11 @@
+.git
+.gitignore
+node_modules
+frontend/node_modules
+frontend/dist
+frontend/.vite
+npm-debug.log*
+docker-compose*.yml
+README.md
+design-qa.md
+docs

.gitignore

@@ -12,6 +12,8 @@ __pycache__/
 build/
 develop-eggs/
 dist/
+node_modules/
+*.tsbuildinfo
 downloads/
 eggs/
 .eggs/
@@ -37,6 +39,16 @@ MANIFEST
 pip-log.txt
 pip-delete-this-directory.txt
 
+# Runtime data and logs
+runtime/
+backend/runtime/
+backend/storage/
+logs/
+backend/logs/
+*.db
+*.sqlite
+*.sqlite3
+
 # Unit test / coverage reports
 htmlcov/
 .tox/
@@ -130,6 +142,7 @@ celerybeat.pid
 
 # Environments
 .env
+!.env.example
 .venv
 env/
 venv/
@@ -137,6 +150,16 @@ ENV/
 env.bak/
 venv.bak/
 
+# Local backend config (含数据库账号密码等敏感信息，勿提交)
+backend/config.yaml
+
+# Agent / IDE 工具产物，不应进版本库
+.codex-backups/
+.pnpm-store/
+.zcode/
+.claude/
+CLAUDE.md
+
 # Spyder project settings
 .spyderproject
 .spyproject
@@ -174,3 +197,14 @@ cython_debug/
 # PyPI configuration file
 .pypirc
 
+docker/llamafactory-latest.tar.gz
+# Compute data - 保留目录结构和 README，忽略子目录内容（日志、模型、数据集等）
+!docker/compute/data/yg-ft/logs/
+docker/compute/data/yg-ft/datasets/*
+docker/compute/data/yg-ft/models/*
+docker/compute/data/yg-ft/outputs/*
+docker/compute/data/yg-ft/logs/**
+!docker/compute/data/yg-ft/logs/compute/
+!docker/compute/data/yg-ft/logs/training/
+!docker/compute/data/yg-ft/**/.gitkeep
+!docker/compute/data/yg-ft/**/README.md

README.md

@@ -1,3 +1,231 @@
-# YG_FT
+# YG_FT 模型微调平台
 
-远光微调平台
+YG_FT 是一个面向企业治理场景的模型微调平台，覆盖用户中心、多租户、项目隔离、数据集管理、模型管理、训练任务、评测、推理、审批流、审计留存、算力调度和训练引擎适配。
+
+当前前端已有基础页面，后端与算力平台已按多人协作开发方式建立工程骨架，并开始实现正式系统主链路能力。当前代码和 SQL 均作为后续生产演进基线维护，不再以一次性演示或静态 Mock 为开发准则。
+
+## 总体架构
+
+```text
+YG_FT/
+  frontend/                 # 前端控制台
+  backend/                  # FastAPI 应用平台后端
+    app/
+      api/v1/               # 对前端暴露的 REST API
+      core/                 # 配置、日志、中间件、权限等基础能力
+      db/                   # 数据库连接、迁移、事务工具
+      modules/              # 业务模块目录
+      schemas/              # Pydantic 入参/出参模型
+      services/             # 跨模块应用服务
+      workers/              # 后台任务入口
+    requirements.txt        # 后端 Python 第三方依赖
+  compute/                  # 算力平台与训练框架适配层
+    api/                    # 内部 Compute API
+    agent/                  # 单机多 GPU 调度与进程管理
+    engines/llama_factory/  # LLaMA-Factory 适配器
+    file_gateway/           # 本地文件上传、下载、导入、产物管理
+  docs/                     # 需求、接口、数据库、开发计划和部署文档
+  docker/                   # 容器化配置
+```
+
+## 平台分层
+
+| 层级 | 职责 | 主要目录 |
+| --- | --- | --- |
+| 前端控制台 | 用户操作入口、任务看板、项目/模型/数据集/训练/审批/审计页面 | `frontend/` |
+| 应用平台后端 | 用户中心、多租户、RBAC/ABAC、项目隔离、元数据、审批流、审计、API 编排 | `backend/` |
+| 算力平台 | GPU 发现、资源锁定、训练进程管理、日志采集、产物归档、任务状态同步 | `compute/` |
+| 训练引擎 | 当前固定接入 LLaMA-Factory，预留其他训练平台适配标准 | `compute/engines/` |
+| 数据层 | PostgreSQL、Redis、本地文件存储、日志归档 | `docs/postgres-schema.sql` |
+
+## 当前开发基线
+
+- 使用 FastAPI 提供统一 API 响应结构 `{ code, message, data }`。
+- 本地运行阶段统一使用 PostgreSQL，后端启动时会在 PG 中初始化当前运行表和系统内置账号；模型、数据集、算力节点、GPU、微调任务等业务数据必须通过页面、接口或正式导入流程产生。
+- 支持登录、模型管理、数据集管理、微调任务创建/启动/停止/进度轮询。
+- 支持训练日志、loss 指标、checkpoint 和训练产物接口；真实训练执行器接入前，联调状态机必须通过显式环境变量开启。
+- 支持多算力节点、GPU、任务队列、资源副本和资源同步状态接口。
+- 前端新增 `/compute` 算力节点页面，展示节点地址、权重、标签、启用状态、GPU、队列和资源副本。
+- `compute/engines/llama_factory/adapter.py` 提供 LLaMA-Factory 参数校验、命令生成和日志解析基础能力。
+
+## 前后端一键启动
+
+首次使用前，请先按下方“后端启动”和“前端启动”说明安装依赖，并确保
+PostgreSQL 已可用。之后在项目根目录执行：
+
+```bash
+bash ./start.sh
+```
+
+脚本会同时启动前端 `http://localhost:16801` 和后端
+`http://127.0.0.1:17861`，按 `Ctrl+C` 会同时停止两个服务。脚本只负责
+启动前后端，不会自动安装依赖，也不会启动 PostgreSQL、Redis 或算力服务。
+
+仅检查依赖和端口而不启动服务：
+
+```bash
+bash ./start.sh --check
+```
+
+本地启动推荐只配置数据库主机。脚本会复用 `docker/app/.env` 中已有的
+`POSTGRES_USER`、`POSTGRES_PASSWORD` 和 `POSTGRES_DB`，端口默认使用
+PostgreSQL 标准端口 `5432`：
+
+```bash
+DATABASE_HOST='www.caoxiaozhu.com' bash ./start.sh
+```
+
+也可以在 `docker/app/.env` 中增加：
+
+```env
+DATABASE_HOST=www.caoxiaozhu.com
+```
+
+需要使用非标准端口时再设置 `DATABASE_PORT`。`DATABASE_URL` 仍可作为完整连接串
+高级覆盖项；终端环境变量优先级最高。脚本不会输出数据库密码。
+
+## 后端启动
+
+```bash
+cd backend
+python -m venv .venv
+.venv\Scripts\activate
+pip install -r requirements.txt
+uvicorn app.main:app --reload --port 17861
+```
+
+默认接口前缀为 `/modelTF`，例如：
+
+```text
+GET /modelTF/health
+POST /modelTF/login
+GET /modelTF/model-manage
+GET /modelTF/dataset-manage
+GET /modelTF/fine-tune
+GET /modelTF/compute/nodes
+```
+
+本地运行时默认 PostgreSQL 连接：
+
+```text
+DATABASE_URL=postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft
+```
+
+本地启动前需要确保 PostgreSQL 已监听 `localhost:15432`，并已创建 `yg_ft` 数据库和 `yg_ft` 用户。后端启动后会自动创建当前运行表并写入内置管理员账号，运行数据统一写入 PostgreSQL。
+
+开发阶段内置登录账号：
+
+| 角色 | 账号 | 密码 | 说明 |
+| --- | --- | --- | --- |
+| 超级管理员 | `admin` | `admin123` | 拥有当前全部页面权限 |
+| 操作员 | `operator` | `operator123` | 拥有业务操作相关页面权限 |
+
+以上账号仅用于本地开发和联调。生产环境初始化后应立即修改密码，或改为企业统一身份认证/管理员初始化流程。
+
+## 前端启动
+
+```bash
+cd frontend
+npm install
+npm run dev
+```
+
+前端开发服务默认运行在 `http://localhost:16801`，并通过 Vite proxy 将 `/modelTF` 转发到 `http://localhost:17861`。
+
+## 算力服务启动
+
+算力服务是一个 FastAPI 应用，同时承载 Compute API（模型训练/推理/GPU 管理）和 File Gateway（文件上传下载）路由。Docker 部署时对外暴露两个端口（19100 和 19101）均指向同一服务，方便应用平台分别配置 `api_base_url` 和 `file_gateway_url`。本地开发只需启动一个进程。
+
+### 方式一：Docker 启动（推荐）
+
+```bash
+cd docker/compute
+cp .env.example .env
+docker compose up -d
+```
+
+### 方式二：本地开发启动
+
+**Windows (cmd)：**
+
+```cmd
+cd /d E:\yg_ft\compute
+set PYTHONPATH=E:\yg_ft
+.\.venv\Scripts\python.exe -m uvicorn api.main:app --reload --port 19100
+```
+
+> `PYTHONPATH=E:\yg_ft` 是必需的，因为代码使用 `from compute.agent...` 绝对导入。
+
+**Linux / macOS：**
+
+```bash
+cd compute
+PYTHONPATH=.. uvicorn api.main:app --reload --port 19100
+```
+
+### 环境变量说明
+
+| 变量 | 默认值 | 说明 |
+|---|---|---|
+| `COMPUTE_MODE` | `real` | `real` / `simulator`，仅隔离联调用 simulator |
+| `COMPUTE_EXECUTION_MODE` | `real` | 训练执行模式 |
+| `COMPUTE_SERVICE_TOKEN` | `change_me` | 服务间认证 token |
+| `MODELTF_ROUTE_PREFIX` | `/modelTF` | API 路由前缀 |
+
+应用平台通过数据库 `compute_nodes` 表中的 `api_base_url` 和 `file_gateway_url` 主动轮询算力节点状态。
+
+## 日志
+
+后端日志模块位于 `backend/app/core/logging.py`，说明文档见：
+
+- `docs/backend-logging.md`
+
+默认输出：
+
+```text
+logs/backend-YYYY-MM-DD.log
+logs/error-YYYY-MM-DD.log
+```
+
+日志格式为 JSON Lines，单个文件不超过 20MB，只保留最近 10 天。
+
+## 主要文档
+
+- `docs/platform-architecture-requirements.md`：平台需求、功能模块、页面补全建议。
+- `docs/menu-functional-requirements.md`：当前菜单、二级路由、规划菜单、功能需求、接口和数据库映射。
+- `docs/backend-api-design.md`：FastAPI 接口分组、参数定义、权限说明。
+- `docs/postgres-schema.sql`：PostgreSQL 数据库脚本，包含权限、用户中心、多租户、审批、审计等模型。
+- `docs/system-development-plan.md`：多人协作开发计划，按前端、后端、DB、部署拆分。
+- `docs/team-development-plan.md`：3-4 人并行开发分工计划，按人员边界标注页面、接口、数据库和交付节奏。
+- `docs/first-version-development-plan.md`：当前系统主链路开发计划，覆盖前端、后端、DB、Compute API、GPU 和 LLaMA-Factory 适配。
+- `docs/backend-logging.md`：后端日志模块使用说明。
+- `docs/deployment-plan.md`：后期部署方案，覆盖单机算力服务器部署与应用/算力分离部署。
+- `docker/README.md`：Docker 部署入口，包含应用服务器和算力服务器两套 Compose 使用方式。
+
+## Docker 部署入口
+
+应用服务器：
+
+```bash
+cd docker/app
+cp .env.example .env
+docker compose up -d
+```
+
+算力服务器：
+
+```bash
+cd docker/compute
+cp .env.example .env
+docker compose up -d
+```
+
+两套 Compose 均采用代码外挂方式运行，镜像只包含运行时环境和第三方依赖。项目根目录不再保留 `Dockerfile` 和 `docker-compose.yml`，部署时统一进入 `docker/app` 或 `docker/compute` 目录执行。
+
+## 后续开发原则
+
+- 接口实现优先遵循 `docs/backend-api-design.md`。
+- 数据库实现优先遵循 `docs/postgres-schema.sql`，后续通过 Alembic 迁移管理变更。
+- 前端页面与后端接口、数据库表之间的映射以文档中的“对应页面/功能模块”为准。
+- 训练引擎适配必须通过 `compute/engines/` 下的标准接口，不在应用平台后端直接拼接训练命令。
+- 敏感信息不得写入日志，生产环境密钥通过环境变量或密钥管理系统注入。

backend/README.md

@@ -0,0 +1,65 @@
+# Backend Service
+
+后端工程使用 FastAPI，定位为模型微调平台的应用平台服务，负责用户中心、多租户、权限隔离、项目、数据集、模型、训练任务、审批、审计和算力平台编排。
+
+## 目录结构
+
+```text
+backend/
+  app/
+    main.py                  # FastAPI 应用入口
+    api/v1/                  # 对前端暴露的 接口路由
+    core/                    # 配置、日志、中间件、权限等基础能力
+    db/                      # 数据库连接、迁移集成、事务工具
+    modules/                 # 业务模块
+      auth/
+      tenant/
+      project/
+      model/
+      dataset/
+      data_process/
+      fine_tune/
+      eval/
+      inference/
+      approval/
+      audit/
+      compute_gateway/
+      file_gateway/
+      engine_registry/
+      retention/
+      system/
+    schemas/                 # Pydantic 入参/出参模型
+    services/                # 跨模块应用服务
+    workers/                 # 后台任务入口
+  requirements.txt           # 后端第三方依赖
+  logs/                      # 本地开发日志目录，生产环境建议挂载到独立日志盘
+```
+
+## 本地启动
+
+```bash
+cd backend
+python -m venv .venv
+.venv\Scripts\activate
+pip install -r requirements.txt
+uvicorn app.main:app --reload
+```
+
+健康检查：
+
+```text
+GET /modelTF/health
+```
+
+## 日志
+
+日志模块位于 `app/core/logging.py`，使用说明见 `../docs/backend-logging.md`。
+
+默认日志文件：
+
+```text
+logs/backend-YYYY-MM-DD.log
+logs/error-YYYY-MM-DD.log
+```
+
+文件日志为 JSON Lines 格式，单个文件不超过 20MB，只保存最近 10 天，错误日志按 `ERROR` 级别独立拆分，便于 ELK/日志平台采集。

backend/_check_sessions.py

@@ -0,0 +1,10 @@
+from app.db.platform_store import get_platform_store
+
+store = get_platform_store()
+with store.connect() as conn:
+    rows = conn.execute(
+        "SELECT id, user_id, login_at, logout_at, duration_seconds FROM sessions ORDER BY login_at DESC LIMIT 10"
+    ).fetchall()
+    print(f"sessions count: {len(rows)}")
+    for r in rows:
+        print(f"  user={r['user_id'][:25]}... login={r['login_at']} logout={r['logout_at']} dur={r['duration_seconds']}")

backend/app/__init__.py

@@ -0,0 +1 @@
+"""Application package."""

backend/app/api/__init__.py

@@ -0,0 +1 @@
+"""API package."""

backend/app/api/v1/__init__.py

@@ -0,0 +1 @@
+"""Versioned API package."""

backend/app/api/v1/endpoints/__init__.py

@@ -0,0 +1 @@
+"""API endpoint modules."""

backend/app/api/v1/endpoints/health.py

@@ -0,0 +1,18 @@
+from fastapi import APIRouter
+
+from app.core.logging import get_logger
+from app.db.platform_store import get_platform_store
+
+router = APIRouter()
+logger = get_logger(__name__)
+
+
+@router.get("/health")
+async def health_check() -> dict[str, object]:
+    logger.info("health check requested")
+    return {
+        "code": 0,
+        "message": "ok",
+        "data": get_platform_store().health_metrics(),
+    }
+

backend/app/api/v1/router.py

@@ -0,0 +1,22 @@
+from fastapi import APIRouter
+
+from app.api.v1.endpoints.data_process import router as data_process_router
+from app.api.v1.endpoints.platform import router as platform_router
+from app.api.v1.endpoints.health import router as health_router
+from app.modules.tenant.router import router as tenant_router
+from app.modules.project.router import router as project_router
+from app.modules.approval.router import router as approval_router
+from app.modules.system.router import router as system_router
+from app.modules.retention.router import router as retention_router
+from app.modules.resource.router import router as resource_router
+
+api_router = APIRouter()
+api_router.include_router(health_router, tags=["health"])
+api_router.include_router(data_process_router, tags=["data-process"])
+api_router.include_router(platform_router, tags=["platform"])
+api_router.include_router(system_router, tags=["system"])
+api_router.include_router(tenant_router, tags=["tenant"])
+api_router.include_router(project_router, tags=["project"])
+api_router.include_router(approval_router, tags=["approval"])
+api_router.include_router(retention_router, tags=["retention"])
+api_router.include_router(resource_router, tags=["resource"])

backend/app/core/__init__.py

@@ -0,0 +1 @@
+"""Core infrastructure modules."""

backend/app/core/auth.py

@@ -0,0 +1,138 @@
+"""鉴权依赖：从 Authorization header 解析当前用户，提供权限校验。"""
+from __future__ import annotations
+
+from typing import Any
+
+from fastapi import Depends, HTTPException, Query, Request, status
+
+from app.db.platform_store import get_platform_store
+
+# 无需鉴权的路径前缀（健康检查、登录等）
+PUBLIC_PATHS = ("/health", "/login", "/system-info")
+
+
+def _extract_token(request: Request) -> str | None:
+    """从 Authorization header 提取 token（格式: Bearer platform-token-{user_id}）。"""
+    auth = request.headers.get("Authorization", "")
+    token = auth.replace("Bearer ", "").strip()
+    if token.startswith("platform-token-"):
+        return token[len("platform-token-"):]
+    return None
+
+
+def get_current_user(request: Request) -> dict[str, Any]:
+    """
+    FastAPI 依赖：解析当前登录用户。
+    - 公开路径（/health, /login 等）直接放行，返回匿名用户。
+    - 无 token 或 token 无效时抛 401。
+    - admin 用户标记为超级管理员，拥有全部权限。
+    """
+    path = request.url.path
+    # 去掉路由前缀后判断
+    for prefix in PUBLIC_PATHS:
+        if path.endswith(prefix):
+            return {"id": None, "username": "anonymous", "role": "viewer", "permissions": [], "protected": False}
+
+    user_id = _extract_token(request)
+    if not user_id:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="missing or invalid token")
+
+    store = get_platform_store()
+    for u in store.users():
+        if u.get("id") == user_id:
+            return u
+    raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="user not found")
+
+
+def require_admin(current_user: dict[str, Any] = Depends(get_current_user)) -> dict[str, Any]:
+    """FastAPI 依赖：要求当前用户是管理员（role=admin 或 protected）。"""
+    if current_user.get("role") == "admin" or current_user.get("protected"):
+        return current_user
+    raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="admin permission required")
+
+
+def is_admin(user: dict[str, Any]) -> bool:
+    """判断用户是否为管理员（admin 角色或 protected 标记）。"""
+    return user.get("role") == "admin" or user.get("protected", False)
+
+
+def has_resource_access(
+    resource_type: str,
+    resource_id: str,
+    user: dict[str, Any],
+    permission: str = "read",
+) -> bool:
+    """
+    检查用户对某资源是否有指定权限。
+    - admin/protected 用户直接放行（旁路）。
+    - 其他用户检查 acls 表中是否有对应授权。
+    """
+    if user.get("role") == "admin" or user.get("protected"):
+        return True
+
+    store = get_platform_store()
+    acls = store.get_acl(resource_type, resource_id)
+    user_id = user.get("id")
+    user_role = user.get("role")
+
+    for entry in acls:
+        # 按 user 授权
+        if entry.get("principal_type") == "user" and entry.get("principal_id") == user_id:
+            if _permission_covers(entry.get("permission"), permission):
+                return True
+        # 按 role 授权
+        if entry.get("principal_type") == "role" and entry.get("principal_id") == user_role:
+            if _permission_covers(entry.get("permission"), permission):
+                return True
+    return False
+
+
+def _permission_covers(granted: str | None, required: str) -> bool:
+    """权限覆盖判断：write/execute 覆盖 read；admin 覆盖一切。"""
+    if not granted:
+        return False
+    if granted == "admin":
+        return True
+    if granted == required:
+        return True
+    # write 覆盖 read
+    if required == "read" and granted in ("write", "execute"):
+        return True
+    return False
+
+
+def filter_accessible_resource_ids(
+    resource_type: str,
+    all_ids: list[str],
+    user: dict[str, Any],
+) -> list[str]:
+    """
+    从全部资源 ID 中过滤出当前用户可访问的 ID 列表。
+    - admin 直接返回全部。
+    - 普通用户查 acls 表取交集。
+    """
+    if user.get("role") == "admin" or user.get("protected"):
+        return all_ids
+
+    if not all_ids:
+        return []
+
+    store = get_platform_store()
+    user_id = user.get("id")
+    user_role = user.get("role")
+
+    # 查询该用户在该资源类型下有 read 权限的所有 resource_id
+    with store.connect() as conn:
+        rows = conn.execute(
+            """
+            SELECT DISTINCT resource_id FROM acls
+            WHERE resource_type=? AND (
+                (principal_type='user' AND principal_id=?)
+                OR (principal_type='role' AND principal_id=?)
+            )
+            """,
+            (resource_type, user_id, user_role),
+        ).fetchall()
+
+    accessible = {r["resource_id"] for r in rows}
+    return [rid for rid in all_ids if rid in accessible]

backend/app/core/config.py

@@ -0,0 +1,71 @@
+from dataclasses import dataclass
+from functools import lru_cache
+import os
+
+try:
+    from pathlib import Path as _Path
+
+    from dotenv import load_dotenv
+
+    # 显式指定 backend 目录下的 .env，并强制覆盖已有环境变量，
+    # 确保远程数据库配置生效，不被本地默认值或残留环境变量影响。
+    _env_path = _Path(__file__).resolve().parent.parent.parent / ".env"
+    load_dotenv(dotenv_path=_env_path, override=True)
+except ImportError:
+    pass
+
+
+def _int_env(name: str, default: int) -> int:
+    raw = os.getenv(name)
+    if raw is None or raw == "":
+        return default
+    return int(raw)
+
+
+def _list_env(name: str, default: list[str]) -> list[str]:
+    raw = os.getenv(name)
+    if raw is None or raw.strip() == "":
+        return default
+    return [item.strip() for item in raw.split(",") if item.strip()]
+
+
+@dataclass(frozen=True)
+class Settings:
+    app_name: str = os.getenv("APP_NAME", "YG Fine-Tune Platform API")
+    app_env: str = os.getenv("APP_ENV", "local")
+    route_prefix: str = os.getenv("MODELTF_ROUTE_PREFIX", "/modelTF")
+    app_mode: str = os.getenv("APP_MODE", "local")
+    database_url: str = os.getenv("DATABASE_URL", "postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft")
+    cors_allow_origins: list[str] = None  # type: ignore[assignment]
+    compute_mode: str = os.getenv("COMPUTE_MODE", "real")
+    compute_status_sync_mode: str = os.getenv("COMPUTE_STATUS_SYNC_MODE", "polling")
+    compute_poll_interval_seconds: int = _int_env("COMPUTE_POLL_INTERVAL_SECONDS", 3)
+    compute_request_timeout_seconds: int = _int_env("COMPUTE_REQUEST_TIMEOUT_SECONDS", 5)
+    compute_service_token: str = os.getenv("COMPUTE_SERVICE_TOKEN", "")
+    log_level: str = os.getenv("LOG_LEVEL", "INFO")
+    log_dir: str = os.getenv("LOG_DIR", "./logs")
+    log_file_prefix: str = os.getenv("LOG_FILE_PREFIX", "backend")
+    log_error_file_prefix: str = os.getenv("LOG_ERROR_FILE_PREFIX", "error")
+    log_max_bytes: int = _int_env("LOG_MAX_BYTES", 20 * 1024 * 1024)
+    log_retention_days: int = _int_env("LOG_RETENTION_DAYS", 10)
+
+    def __post_init__(self) -> None:
+        object.__setattr__(
+            self,
+            "cors_allow_origins",
+            _list_env(
+                "CORS_ALLOW_ORIGINS",
+                [
+                    "http://localhost:16801",
+                    "http://127.0.0.1:16801",
+                    "http://localhost:17861",
+                    "http://127.0.0.1:17861",
+                ],
+            ),
+        )
+
+
+@lru_cache
+def get_settings() -> Settings:
+    return Settings()
+

backend/app/core/logging.py

@@ -0,0 +1,253 @@
+from __future__ import annotations
+
+from contextvars import ContextVar
+from datetime import date, datetime, timedelta
+import json
+import logging
+from logging import Handler, LogRecord
+from pathlib import Path
+import re
+import time
+from typing import Any
+from uuid import uuid4
+
+from fastapi import FastAPI, Request
+
+from app.core.config import Settings, get_settings
+
+request_id_var: ContextVar[str] = ContextVar("request_id", default="-")
+
+
+class RequestIdFilter(logging.Filter):
+    def filter(self, record: LogRecord) -> bool:
+        record.request_id = request_id_var.get()
+        return True
+
+
+class JsonLogFormatter(logging.Formatter):
+    """Format one JSON object per line for ELK/Filebeat collection."""
+
+    def format(self, record: LogRecord) -> str:
+        payload: dict[str, Any] = {
+            "@timestamp": datetime.fromtimestamp(record.created).astimezone().isoformat(
+                timespec="milliseconds"
+            ),
+            "level": record.levelname,
+            "logger": record.name,
+            "message": record.getMessage(),
+            "module": record.module,
+            "function": record.funcName,
+            "file": record.pathname,
+            "line": record.lineno,
+            "process": record.process,
+            "thread": record.thread,
+            "thread_name": record.threadName,
+            "request_id": getattr(record, "request_id", "-"),
+        }
+        if record.exc_info:
+            payload["exception"] = self.formatException(record.exc_info)
+        if record.stack_info:
+            payload["stack"] = self.formatStack(record.stack_info)
+        return json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
+
+
+class DateSizeRotatingFileHandler(Handler):
+    """Rotate log files by date and size while keeping date in every file name."""
+
+    def __init__(
+        self,
+        log_dir: str | Path,
+        file_prefix: str,
+        max_bytes: int,
+        retention_days: int,
+        encoding: str = "utf-8",
+    ) -> None:
+        super().__init__()
+        self.log_dir = Path(log_dir)
+        self.file_prefix = file_prefix
+        self.max_bytes = max_bytes
+        self.retention_days = retention_days
+        self.encoding = encoding
+        self._current_date: date | None = None
+        self._stream: Any | None = None
+        self._current_path: Path | None = None
+        self.log_dir.mkdir(parents=True, exist_ok=True)
+
+    def emit(self, record: LogRecord) -> None:
+        try:
+            message = self.format(record) + self.terminator
+            encoded_size = len(message.encode(self.encoding))
+            self._ensure_stream()
+            if self._should_rotate(encoded_size):
+                self._rotate_by_size()
+                self._ensure_stream(force=True)
+            self._stream.write(message)
+            self.flush()
+            self._cleanup_expired_files()
+        except Exception:
+            self.handleError(record)
+
+    @property
+    def terminator(self) -> str:
+        return "\n"
+
+    def flush(self) -> None:
+        if self._stream and not self._stream.closed:
+            self._stream.flush()
+
+    def close(self) -> None:
+        try:
+            if self._stream and not self._stream.closed:
+                self._stream.close()
+        finally:
+            self._stream = None
+            super().close()
+
+    def _dated_path(self, target_date: date) -> Path:
+        return self.log_dir / f"{self.file_prefix}-{target_date.isoformat()}.log"
+
+    def _ensure_stream(self, force: bool = False) -> None:
+        today = date.today()
+        if not force and self._stream and self._current_date == today:
+            return
+
+        if self._stream and not self._stream.closed:
+            self._stream.close()
+
+        self._current_date = today
+        self._current_path = self._dated_path(today)
+        self._stream = self._current_path.open("a", encoding=self.encoding)
+
+    def _should_rotate(self, incoming_size: int) -> bool:
+        if not self._current_path or self.max_bytes <= 0:
+            return False
+        if not self._current_path.exists():
+            return False
+        return self._current_path.stat().st_size + incoming_size > self.max_bytes
+
+    def _rotate_by_size(self) -> None:
+        if not self._current_path or not self._current_path.exists():
+            return
+
+        if self._stream and not self._stream.closed:
+            self._stream.close()
+            self._stream = None
+
+        stem = self._current_path.stem
+        suffix = self._current_path.suffix
+        index = 1
+        while True:
+            rotated_path = self.log_dir / f"{stem}.{index}{suffix}"
+            if not rotated_path.exists():
+                self._current_path.rename(rotated_path)
+                return
+            index += 1
+
+    def _cleanup_expired_files(self) -> None:
+        if self.retention_days <= 0:
+            return
+
+        cutoff = date.today() - timedelta(days=self.retention_days - 1)
+        pattern = re.compile(
+            rf"^{re.escape(self.file_prefix)}-(\d{{4}}-\d{{2}}-\d{{2}})(?:\.\d+)?\.log$"
+        )
+        for path in self.log_dir.glob(f"{self.file_prefix}-*.log"):
+            match = pattern.match(path.name)
+            if not match:
+                continue
+            file_date = datetime.strptime(match.group(1), "%Y-%m-%d").date()
+            if file_date < cutoff:
+                path.unlink(missing_ok=True)
+
+
+def configure_logging(settings: Settings | None = None) -> None:
+    settings = settings or get_settings()
+
+    root_logger = logging.getLogger()
+    root_logger.handlers.clear()
+    root_logger.setLevel(settings.log_level.upper())
+
+    console_formatter = logging.Formatter(
+        fmt=(
+            "%(asctime)s | %(levelname)s | pid=%(process)d | %(threadName)s | "
+            "request_id=%(request_id)s | %(name)s | %(pathname)s:%(lineno)d | %(message)s"
+        ),
+        datefmt="%Y-%m-%d %H:%M:%S",
+    )
+    json_formatter = JsonLogFormatter()
+    request_filter = RequestIdFilter()
+
+    console_handler = logging.StreamHandler()
+    console_handler.setFormatter(console_formatter)
+    console_handler.addFilter(request_filter)
+
+    file_handler = DateSizeRotatingFileHandler(
+        log_dir=settings.log_dir,
+        file_prefix=settings.log_file_prefix,
+        max_bytes=settings.log_max_bytes,
+        retention_days=settings.log_retention_days,
+    )
+    file_handler.setFormatter(json_formatter)
+    file_handler.addFilter(request_filter)
+
+    error_file_handler = DateSizeRotatingFileHandler(
+        log_dir=settings.log_dir,
+        file_prefix=settings.log_error_file_prefix,
+        max_bytes=settings.log_max_bytes,
+        retention_days=settings.log_retention_days,
+    )
+    error_file_handler.setLevel(logging.ERROR)
+    error_file_handler.setFormatter(json_formatter)
+    error_file_handler.addFilter(request_filter)
+
+    root_logger.addHandler(console_handler)
+    root_logger.addHandler(file_handler)
+    root_logger.addHandler(error_file_handler)
+
+    for logger_name in ("uvicorn", "uvicorn.error", "uvicorn.access"):
+        logger = logging.getLogger(logger_name)
+        logger.handlers.clear()
+        logger.propagate = True
+
+
+def get_logger(name: str) -> logging.Logger:
+    return logging.getLogger(name)
+
+
+def set_request_id(request_id: str) -> None:
+    request_id_var.set(request_id)
+
+
+def setup_request_logging(app: FastAPI) -> None:
+    logger = get_logger("app.access")
+
+    @app.middleware("http")
+    async def request_logging_middleware(request: Request, call_next):  # type: ignore[no-untyped-def]
+        request_id = request.headers.get("X-Request-ID") or str(uuid4())
+        token = request_id_var.set(request_id)
+        started_at = time.perf_counter()
+        try:
+            response = await call_next(request)
+            elapsed_ms = (time.perf_counter() - started_at) * 1000
+            logger.info(
+                "request completed method=%s path=%s status_code=%s duration_ms=%.2f client=%s",
+                request.method,
+                request.url.path,
+                response.status_code,
+                elapsed_ms,
+                request.client.host if request.client else "-",
+            )
+            response.headers["X-Request-ID"] = request_id
+            return response
+        except Exception:
+            elapsed_ms = (time.perf_counter() - started_at) * 1000
+            logger.exception(
+                "request failed method=%s path=%s duration_ms=%.2f client=%s",
+                request.method,
+                request.url.path,
+                elapsed_ms,
+                request.client.host if request.client else "-",
+            )
+            raise
+        finally:
+            request_id_var.reset(token)

backend/app/db/__init__.py

@@ -0,0 +1 @@
+"""Database infrastructure package."""

backend/app/db/session.py

@@ -0,0 +1,40 @@
+from __future__ import annotations
+
+import os
+from collections.abc import Generator
+from contextlib import contextmanager
+
+from sqlalchemy import create_engine
+from sqlalchemy.orm import Session, sessionmaker
+
+
+DATABASE_URL = os.getenv("DATABASE_URL", "postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft")
+
+engine = create_engine(
+    DATABASE_URL,
+    pool_pre_ping=True,
+    future=True,
+)
+SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, expire_on_commit=False, future=True)
+
+
+def get_db() -> Generator[Session, None, None]:
+    db = SessionLocal()
+    try:
+        yield db
+    finally:
+        db.close()
+
+
+@contextmanager
+def session_scope() -> Generator[Session, None, None]:
+    db = SessionLocal()
+    try:
+        yield db
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+    finally:
+        db.close()
+

backend/app/db/sql/001_platform_runtime.sql

@@ -0,0 +1,335 @@
+CREATE TABLE IF NOT EXISTS users (
+    id TEXT PRIMARY KEY,
+    username TEXT NOT NULL UNIQUE,
+    password_hash TEXT NOT NULL,
+    display_name TEXT NOT NULL,
+    role TEXT NOT NULL,
+    status TEXT NOT NULL,
+    permissions TEXT NOT NULL,
+    create_time TEXT NOT NULL,
+    last_login TEXT,
+    protected INTEGER NOT NULL DEFAULT 0
+);
+
+CREATE TABLE IF NOT EXISTS models (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL UNIQUE,
+    type TEXT NOT NULL,
+    purpose TEXT NOT NULL,
+    model_source TEXT NOT NULL,
+    description TEXT,
+    path TEXT,
+    api_url TEXT,
+    api_key TEXT,
+    online_model_name TEXT,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS trained_models (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL UNIQUE,
+    train_methods TEXT NOT NULL,
+    base_model_path TEXT,
+    create_time TEXT NOT NULL,
+    merged INTEGER NOT NULL DEFAULT 0,
+    merging INTEGER NOT NULL DEFAULT 0,
+    merged_path TEXT,
+    artifact_dir TEXT,
+    compute_node_id TEXT,
+    compute_node_name TEXT
+);
+
+CREATE TABLE IF NOT EXISTS model_lineage (
+    id TEXT PRIMARY KEY,
+    child_resource_type TEXT NOT NULL,
+    child_resource_id TEXT NOT NULL,
+    parent_resource_type TEXT NOT NULL,
+    parent_resource_id TEXT NOT NULL,
+    relation_type TEXT NOT NULL,
+    compute_job_id TEXT,
+    payload TEXT NOT NULL,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS model_artifacts (
+    id TEXT PRIMARY KEY,
+    model_id TEXT NOT NULL,
+    model_kind TEXT NOT NULL,
+    artifact_type TEXT NOT NULL,
+    path TEXT NOT NULL,
+    size_bytes BIGINT NOT NULL DEFAULT 0,
+    checksum_sha256 TEXT,
+    metadata TEXT NOT NULL,
+    compute_job_id TEXT,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS model_export_jobs (
+    id TEXT PRIMARY KEY,
+    trained_model_id TEXT,
+    compute_job_id TEXT NOT NULL,
+    node_id TEXT,
+    export_type TEXT NOT NULL,
+    quantization_bit INTEGER NOT NULL DEFAULT 0,
+    status TEXT NOT NULL,
+    output_dir TEXT,
+    payload TEXT NOT NULL,
+    create_time TEXT NOT NULL,
+    completed_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS datasets (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL UNIQUE,
+    type TEXT NOT NULL,
+    storage_type TEXT NOT NULL,
+    source TEXT NOT NULL,
+    task_id TEXT,
+    size TEXT,
+    count INTEGER NOT NULL DEFAULT 0,
+    description TEXT,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS dataset_files (
+    id TEXT PRIMARY KEY,
+    dataset_id TEXT NOT NULL REFERENCES datasets(id) ON DELETE CASCADE,
+    name TEXT NOT NULL,
+    size TEXT,
+    content TEXT NOT NULL,
+    active_version_id TEXT NOT NULL,
+    versions TEXT NOT NULL,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS compute_nodes (
+    id TEXT PRIMARY KEY,
+    code TEXT NOT NULL UNIQUE,
+    name TEXT NOT NULL,
+    api_base_url TEXT NOT NULL,
+    file_gateway_url TEXT NOT NULL,
+    enabled INTEGER NOT NULL DEFAULT 1,
+    scheduler_status TEXT NOT NULL,
+    scheduler_weight INTEGER NOT NULL DEFAULT 100,
+    tags TEXT NOT NULL,
+    gpu_count INTEGER NOT NULL DEFAULT 0,
+    current_running_jobs INTEGER NOT NULL DEFAULT 0,
+    max_parallel_jobs INTEGER NOT NULL DEFAULT 2,
+    data_root TEXT NOT NULL,
+    model_root TEXT NOT NULL,
+    log_root TEXT NOT NULL,
+    api_version TEXT NOT NULL DEFAULT 'v1',
+    capabilities TEXT NOT NULL DEFAULT '[]',
+    description TEXT,
+    last_health_check_at TEXT,
+    health_detail TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS gpus (
+    id TEXT PRIMARY KEY,
+    node_id TEXT NOT NULL REFERENCES compute_nodes(id) ON DELETE CASCADE,
+    gpu_index INTEGER NOT NULL,
+    uuid TEXT NOT NULL,
+    name TEXT NOT NULL,
+    memory_total_gb DOUBLE PRECISION NOT NULL,
+    power_limit_w DOUBLE PRECISION NOT NULL,
+    base_temperature INTEGER NOT NULL,
+    last_seen_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS fine_tune_tasks (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL UNIQUE,
+    payload TEXT NOT NULL,
+    status TEXT NOT NULL,
+    progress INTEGER NOT NULL DEFAULT 0,
+    process_id INTEGER,
+    create_time TEXT NOT NULL,
+    start_time TEXT,
+    completed_at TEXT,
+    compute_node_id TEXT REFERENCES compute_nodes(id) ON DELETE SET NULL,
+    gpus TEXT NOT NULL,
+    sync_job_id TEXT,
+    compute_job_id TEXT
+);
+
+CREATE TABLE IF NOT EXISTS fine_tune_metrics (
+    id TEXT PRIMARY KEY,
+    task_id TEXT NOT NULL REFERENCES fine_tune_tasks(id) ON DELETE CASCADE,
+    step INTEGER NOT NULL,
+    epoch DOUBLE PRECISION,
+    loss DOUBLE PRECISION,
+    grad_norm DOUBLE PRECISION,
+    learning_rate DOUBLE PRECISION,
+    raw TEXT NOT NULL,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS fine_tune_checkpoints (
+    id TEXT PRIMARY KEY,
+    task_id TEXT NOT NULL REFERENCES fine_tune_tasks(id) ON DELETE CASCADE,
+    step INTEGER NOT NULL,
+    name TEXT NOT NULL,
+    path TEXT NOT NULL,
+    size_bytes BIGINT NOT NULL DEFAULT 0,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS compute_jobs (
+    id TEXT PRIMARY KEY,
+    task_id TEXT REFERENCES fine_tune_tasks(id) ON DELETE SET NULL,
+    node_id TEXT REFERENCES compute_nodes(id) ON DELETE SET NULL,
+    engine TEXT NOT NULL,
+    status TEXT NOT NULL,
+    command TEXT NOT NULL,
+    output_dir TEXT,
+    log_file TEXT,
+    payload TEXT NOT NULL,
+    create_time TEXT NOT NULL,
+    update_time TEXT NOT NULL,
+    completed_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS gpu_allocations (
+    id TEXT PRIMARY KEY,
+    task_id TEXT REFERENCES fine_tune_tasks(id) ON DELETE CASCADE,
+    compute_job_id TEXT,
+    node_id TEXT REFERENCES compute_nodes(id) ON DELETE CASCADE,
+    gpu_index INTEGER NOT NULL,
+    status TEXT NOT NULL,
+    create_time TEXT NOT NULL,
+    released_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS scheduler_locks (
+    lock_key TEXT PRIMARY KEY,
+    owner TEXT NOT NULL,
+    expires_at TEXT NOT NULL,
+    create_time TEXT NOT NULL,
+    update_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS resource_replicas (
+    id TEXT PRIMARY KEY,
+    node_id TEXT NOT NULL REFERENCES compute_nodes(id) ON DELETE CASCADE,
+    resource_type TEXT NOT NULL,
+    resource_id TEXT NOT NULL,
+    local_path TEXT NOT NULL,
+    status TEXT NOT NULL,
+    sync_status TEXT NOT NULL,
+    checksum_sha256 TEXT,
+    byte_size BIGINT NOT NULL DEFAULT 0,
+    last_checked_at TEXT,
+    last_error TEXT,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS resource_sync_jobs (
+    id TEXT PRIMARY KEY,
+    target_node_id TEXT NOT NULL,
+    resources TEXT NOT NULL,
+    status TEXT NOT NULL,
+    progress INTEGER NOT NULL DEFAULT 0,
+    create_time TEXT NOT NULL,
+    completed_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS eval_tasks (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL,
+    payload TEXT NOT NULL,
+    status TEXT NOT NULL,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS eval_dimensions (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL,
+    payload TEXT NOT NULL,
+    is_active INTEGER NOT NULL DEFAULT 1,
+    is_default INTEGER NOT NULL DEFAULT 0,
+    create_time TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS compare_tasks (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL,
+    payload TEXT NOT NULL,
+    status TEXT NOT NULL,
+    create_time TEXT NOT NULL
+);
+
+CREATE INDEX IF NOT EXISTS idx_fine_tune_status ON fine_tune_tasks(status);
+CREATE INDEX IF NOT EXISTS idx_fine_tune_compute_job ON fine_tune_tasks(compute_job_id);
+CREATE INDEX IF NOT EXISTS idx_fine_tune_compute_node_status ON fine_tune_tasks(compute_node_id, status);
+CREATE INDEX IF NOT EXISTS idx_model_lineage_child ON model_lineage(child_resource_type, child_resource_id);
+CREATE INDEX IF NOT EXISTS idx_model_lineage_parent ON model_lineage(parent_resource_type, parent_resource_id);
+CREATE INDEX IF NOT EXISTS idx_model_artifacts_model ON model_artifacts(model_kind, model_id, artifact_type);
+CREATE INDEX IF NOT EXISTS idx_model_export_jobs_model ON model_export_jobs(trained_model_id, create_time DESC);
+CREATE INDEX IF NOT EXISTS idx_model_export_jobs_compute ON model_export_jobs(compute_job_id);
+CREATE INDEX IF NOT EXISTS idx_fine_tune_metrics_task_step ON fine_tune_metrics(task_id, step);
+CREATE UNIQUE INDEX IF NOT EXISTS uq_fine_tune_metrics_task_step_epoch ON fine_tune_metrics(task_id, step, epoch);
+CREATE INDEX IF NOT EXISTS idx_fine_tune_checkpoints_task_step ON fine_tune_checkpoints(task_id, step);
+CREATE UNIQUE INDEX IF NOT EXISTS uq_fine_tune_checkpoints_task_path ON fine_tune_checkpoints(task_id, path);
+CREATE INDEX IF NOT EXISTS idx_compute_jobs_task ON compute_jobs(task_id);
+CREATE INDEX IF NOT EXISTS idx_compute_jobs_node_status ON compute_jobs(node_id, status);
+CREATE INDEX IF NOT EXISTS idx_gpu_allocations_node_status ON gpu_allocations(node_id, status);
+CREATE UNIQUE INDEX IF NOT EXISTS uq_gpu_allocations_active ON gpu_allocations(node_id, gpu_index) WHERE status IN ('allocated','running');
+CREATE INDEX IF NOT EXISTS idx_scheduler_locks_expires ON scheduler_locks(expires_at);
+CREATE INDEX IF NOT EXISTS idx_dataset_files_dataset ON dataset_files(dataset_id);
+CREATE INDEX IF NOT EXISTS idx_gpus_node ON gpus(node_id);
+CREATE UNIQUE INDEX IF NOT EXISTS uq_gpus_node_index ON gpus(node_id, gpu_index);
+CREATE INDEX IF NOT EXISTS idx_replicas_resource ON resource_replicas(resource_type, resource_id);
+CREATE UNIQUE INDEX IF NOT EXISTS uq_replicas_node_resource ON resource_replicas(node_id, resource_type, resource_id);
+CREATE INDEX IF NOT EXISTS idx_sync_jobs_node_status ON resource_sync_jobs(target_node_id, status);
+CREATE INDEX IF NOT EXISTS idx_eval_tasks_status ON eval_tasks(status);
+CREATE INDEX IF NOT EXISTS idx_eval_dimensions_active ON eval_dimensions(is_active);
+CREATE INDEX IF NOT EXISTS idx_compare_tasks_status ON compare_tasks(status);
+
+-- ===================== Project / Tenant =====================
+
+CREATE TABLE IF NOT EXISTS projects (
+    id TEXT PRIMARY KEY,
+    tenant_id TEXT NOT NULL DEFAULT 'default',
+    name TEXT NOT NULL,
+    code TEXT NOT NULL,
+    description TEXT,
+    quota TEXT,
+    status TEXT NOT NULL DEFAULT 'active',
+    create_time TEXT NOT NULL,
+    create_by TEXT,
+    updated_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS project_members (
+    project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
+    user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+    role TEXT NOT NULL DEFAULT 'member',
+    create_time TEXT NOT NULL,
+    PRIMARY KEY (project_id, user_id)
+);
+
+CREATE TABLE IF NOT EXISTS roles (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL,
+    permissions TEXT NOT NULL DEFAULT '[]',
+    create_time TEXT
+);
+
+CREATE TABLE IF NOT EXISTS sessions (
+    id TEXT PRIMARY KEY,
+    user_id TEXT NOT NULL,
+    issued_at TEXT NOT NULL,
+    expires_at TEXT NOT NULL,
+    ip TEXT
+);
+
+CREATE TABLE IF NOT EXISTS acls (
+    id TEXT PRIMARY KEY,
+    resource_type TEXT NOT NULL,
+    resource_id TEXT NOT NULL,
+    principal_type TEXT NOT NULL,
+    principal_id TEXT NOT NULL,
+    permission TEXT NOT NULL,
+    create_time TEXT
+);

backend/app/db/sql/002_data_process.sql

@@ -0,0 +1,343 @@
+-- Data processing migration.
+--
+-- IMPORTANT: This file is intentionally NOT wired into application startup.
+-- Apply it explicitly in a controlled deployment, or call
+-- DataProcessStore.ensure_schema() from an administrative command.
+
+BEGIN;
+
+-- This migration targets the current runtime schema created by
+-- 001_platform_runtime.sql. Refuse the UUID/JSONB target-design schema instead
+-- of partially altering it with incompatible TEXT foreign keys.
+DO $$
+DECLARE
+    datasets_id_type TEXT;
+BEGIN
+    SELECT format_type(a.atttypid, a.atttypmod)
+      INTO datasets_id_type
+      FROM pg_attribute a
+      JOIN pg_class c ON c.oid = a.attrelid
+      JOIN pg_namespace n ON n.oid = c.relnamespace
+     WHERE n.nspname = current_schema()
+       AND c.relname = 'datasets'
+       AND a.attname = 'id'
+       AND a.attnum > 0
+       AND NOT a.attisdropped;
+    IF datasets_id_type IS NULL THEN
+        RAISE EXCEPTION '002_data_process.sql requires 001_platform_runtime.sql first';
+    END IF;
+    IF datasets_id_type <> 'text' THEN
+        RAISE EXCEPTION
+          '002_data_process.sql supports only the current TEXT runtime schema; found datasets.id type %',
+          datasets_id_type;
+    END IF;
+END $$;
+
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS source_task_id TEXT;
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS size_bytes BIGINT NOT NULL DEFAULT 0;
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS record_count BIGINT NOT NULL DEFAULT 0;
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS metadata TEXT NOT NULL DEFAULT '{}';
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS tenant_id TEXT;
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS project_id TEXT;
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS owner_id TEXT;
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS created_by TEXT;
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS created_at TIMESTAMPTZ NOT NULL DEFAULT now();
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS updated_at TIMESTAMPTZ NOT NULL DEFAULT now();
+ALTER TABLE datasets ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ;
+
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS storage_object_id TEXT;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS current_version_id TEXT;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS size_bytes BIGINT NOT NULL DEFAULT 0;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS record_count BIGINT NOT NULL DEFAULT 0;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS file_format VARCHAR(40);
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS checksum_sha256 CHAR(64);
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS version_no INTEGER NOT NULL DEFAULT 1;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS source_task_id TEXT;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS tenant_id TEXT;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS project_id TEXT;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS created_by TEXT;
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS metadata TEXT NOT NULL DEFAULT '{}';
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS created_at TIMESTAMPTZ NOT NULL DEFAULT now();
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS updated_at TIMESTAMPTZ NOT NULL DEFAULT now();
+ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ;
+
+CREATE TABLE IF NOT EXISTS data_process_tasks (
+    id TEXT PRIMARY KEY,
+    name VARCHAR(150) NOT NULL,
+    description TEXT,
+    status VARCHAR(20) NOT NULL DEFAULT 'pending'
+        CHECK (status IN ('pending', 'running', 'completed', 'failed', 'stopped')),
+    process_type VARCHAR(20) NOT NULL
+        CHECK (process_type IN ('structured', 'unstructured', 'external')),
+    source_dataset_id TEXT REFERENCES datasets(id) ON DELETE SET NULL,
+    output_dataset_id TEXT REFERENCES datasets(id) ON DELETE SET NULL,
+    config TEXT NOT NULL DEFAULT '{}',
+    progress NUMERIC(5,2) NOT NULL DEFAULT 0 CHECK (progress >= 0 AND progress <= 100),
+    input_count BIGINT NOT NULL DEFAULT 0 CHECK (input_count >= 0),
+    output_count BIGINT NOT NULL DEFAULT 0 CHECK (output_count >= 0),
+    filtered_count BIGINT NOT NULL DEFAULT 0 CHECK (filtered_count >= 0),
+    duplicate_count BIGINT NOT NULL DEFAULT 0 CHECK (duplicate_count >= 0),
+    error_count BIGINT NOT NULL DEFAULT 0 CHECK (error_count >= 0),
+    failure_reason TEXT,
+    generation_run_id TEXT,
+    results_confirmed BOOLEAN NOT NULL DEFAULT TRUE,
+    workflow_step VARCHAR(20) NOT NULL DEFAULT 'create'
+        CHECK (workflow_step IN ('create', 'model', 'upload', 'preview', 'generate', 'results')),
+    preview_status VARCHAR(20) NOT NULL DEFAULT 'idle'
+        CHECK (preview_status IN ('idle', 'queued', 'running', 'completed', 'failed', 'cancelled')),
+    preview_progress NUMERIC(5,2) NOT NULL DEFAULT 0
+        CHECK (preview_progress >= 0 AND preview_progress <= 100),
+    preview_run_id TEXT,
+    preview_failure_reason TEXT,
+    preview_total_files INTEGER NOT NULL DEFAULT 0 CHECK (preview_total_files >= 0),
+    preview_completed_files INTEGER NOT NULL DEFAULT 0 CHECK (preview_completed_files >= 0),
+    tenant_id TEXT,
+    project_id TEXT,
+    owner_id TEXT,
+    approval_status VARCHAR(30) NOT NULL DEFAULT 'not_required',
+    created_by TEXT,
+    updated_by TEXT,
+    deleted_by TEXT,
+    started_at TIMESTAMPTZ,
+    completed_at TIMESTAMPTZ,
+    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+    deleted_at TIMESTAMPTZ
+);
+
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS generation_run_id TEXT;
+-- 历史任务在引入六步确认流程前已经完成审核，默认保留为已确认；
+-- 新任务由创建接口显式写入 FALSE，并在第六步确认后转为 TRUE。
+ALTER TABLE data_process_tasks
+    ADD COLUMN IF NOT EXISTS results_confirmed BOOLEAN NOT NULL DEFAULT TRUE;
+UPDATE data_process_tasks
+SET results_confirmed=FALSE
+WHERE status <> 'completed' AND results_confirmed=TRUE;
+
+-- 先以可空列接入旧库，才能只回填历史行；随后再收紧默认值与约束。
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS workflow_step VARCHAR(20);
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS preview_status VARCHAR(20);
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS preview_progress NUMERIC(5,2);
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS preview_run_id TEXT;
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS preview_failure_reason TEXT;
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS preview_total_files INTEGER;
+ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS preview_completed_files INTEGER;
+
+CREATE TEMP TABLE data_process_workflow_backfill_ids ON COMMIT DROP AS
+SELECT id FROM data_process_tasks WHERE workflow_step IS NULL;
+
+UPDATE data_process_tasks task
+SET workflow_step = CASE
+    WHEN task.status IN ('running', 'failed', 'stopped') THEN 'generate'
+    WHEN task.status = 'completed' AND task.results_confirmed=FALSE THEN 'generate'
+    WHEN task.status = 'completed' THEN 'results'
+    ELSE 'create'
+END
+WHERE task.workflow_step IS NULL;
+UPDATE data_process_tasks
+SET preview_status='idle', preview_progress=0,
+    preview_total_files=0, preview_completed_files=0
+WHERE preview_status IS NULL OR preview_progress IS NULL
+   OR preview_total_files IS NULL OR preview_completed_files IS NULL;
+
+ALTER TABLE data_process_tasks ALTER COLUMN workflow_step SET DEFAULT 'create';
+ALTER TABLE data_process_tasks ALTER COLUMN workflow_step SET NOT NULL;
+ALTER TABLE data_process_tasks ALTER COLUMN preview_status SET DEFAULT 'idle';
+ALTER TABLE data_process_tasks ALTER COLUMN preview_status SET NOT NULL;
+ALTER TABLE data_process_tasks ALTER COLUMN preview_progress SET DEFAULT 0;
+ALTER TABLE data_process_tasks ALTER COLUMN preview_progress SET NOT NULL;
+ALTER TABLE data_process_tasks ALTER COLUMN preview_total_files SET DEFAULT 0;
+ALTER TABLE data_process_tasks ALTER COLUMN preview_total_files SET NOT NULL;
+ALTER TABLE data_process_tasks ALTER COLUMN preview_completed_files SET DEFAULT 0;
+ALTER TABLE data_process_tasks ALTER COLUMN preview_completed_files SET NOT NULL;
+
+DO $$
+BEGIN
+    IF NOT EXISTS (
+        SELECT 1 FROM pg_constraint
+        WHERE conrelid='data_process_tasks'::regclass
+          AND conname='ck_data_process_tasks_workflow_step'
+    ) THEN
+        ALTER TABLE data_process_tasks ADD CONSTRAINT ck_data_process_tasks_workflow_step
+            CHECK (workflow_step IN ('create', 'model', 'upload', 'preview', 'generate', 'results'));
+    END IF;
+    IF NOT EXISTS (
+        SELECT 1 FROM pg_constraint
+        WHERE conrelid='data_process_tasks'::regclass
+          AND conname='ck_data_process_tasks_preview_status'
+    ) THEN
+        ALTER TABLE data_process_tasks ADD CONSTRAINT ck_data_process_tasks_preview_status
+            CHECK (preview_status IN ('idle', 'queued', 'running', 'completed', 'failed', 'cancelled'));
+    END IF;
+    IF NOT EXISTS (
+        SELECT 1 FROM pg_constraint
+        WHERE conrelid='data_process_tasks'::regclass
+          AND conname='ck_data_process_tasks_preview_progress'
+    ) THEN
+        ALTER TABLE data_process_tasks ADD CONSTRAINT ck_data_process_tasks_preview_progress
+            CHECK (preview_progress >= 0 AND preview_progress <= 100);
+    END IF;
+    IF NOT EXISTS (
+        SELECT 1 FROM pg_constraint
+        WHERE conrelid='data_process_tasks'::regclass
+          AND conname='ck_data_process_tasks_preview_file_counts'
+    ) THEN
+        ALTER TABLE data_process_tasks ADD CONSTRAINT ck_data_process_tasks_preview_file_counts
+            CHECK (preview_total_files >= 0 AND preview_completed_files >= 0
+                   AND preview_completed_files <= preview_total_files);
+    END IF;
+END $$;
+
+CREATE UNIQUE INDEX IF NOT EXISTS uq_data_process_tasks_name_alive
+    ON data_process_tasks(name) WHERE deleted_at IS NULL;
+CREATE INDEX IF NOT EXISTS idx_data_process_tasks_scope_status
+    ON data_process_tasks(tenant_id, project_id, status, created_at DESC)
+    WHERE deleted_at IS NULL;
+CREATE INDEX IF NOT EXISTS idx_data_process_tasks_creator_created
+    ON data_process_tasks(created_by, created_at DESC) WHERE deleted_at IS NULL;
+
+CREATE TABLE IF NOT EXISTS data_process_source_files (
+    id TEXT PRIMARY KEY,
+    task_id TEXT NOT NULL REFERENCES data_process_tasks(id) ON DELETE CASCADE,
+    storage_object_id TEXT,
+    name TEXT NOT NULL,
+    size_bytes BIGINT NOT NULL DEFAULT 0 CHECK (size_bytes >= 0),
+    record_count BIGINT NOT NULL DEFAULT 0 CHECK (record_count >= 0),
+    file_format VARCHAR(40),
+    checksum_sha256 CHAR(64) NOT NULL,
+    version_no INTEGER NOT NULL DEFAULT 1 CHECK (version_no > 0),
+    content TEXT NOT NULL,
+    content_preview TEXT,
+    metadata TEXT NOT NULL DEFAULT '{}',
+    tenant_id TEXT,
+    project_id TEXT,
+    created_by TEXT,
+    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+    deleted_at TIMESTAMPTZ
+);
+
+CREATE INDEX IF NOT EXISTS idx_data_process_source_files_task
+    ON data_process_source_files(task_id, created_at) WHERE deleted_at IS NULL;
+CREATE UNIQUE INDEX IF NOT EXISTS uq_data_process_source_checksum_alive
+    ON data_process_source_files(task_id, checksum_sha256) WHERE deleted_at IS NULL;
+
+CREATE TABLE IF NOT EXISTS data_process_preview_items (
+    id TEXT PRIMARY KEY,
+    task_id TEXT NOT NULL REFERENCES data_process_tasks(id) ON DELETE CASCADE,
+    source_file_id TEXT REFERENCES data_process_source_files(id) ON DELETE CASCADE,
+    original_content TEXT NOT NULL DEFAULT '',
+    edited_content TEXT NOT NULL DEFAULT '',
+    source_start INTEGER CHECK (source_start IS NULL OR source_start >= 0),
+    source_end INTEGER CHECK (source_end IS NULL OR source_end >= 0),
+    source_start_line INTEGER CHECK (source_start_line IS NULL OR source_start_line > 0),
+    source_end_line INTEGER CHECK (source_end_line IS NULL OR source_end_line > 0),
+    token_count INTEGER NOT NULL DEFAULT 0 CHECK (token_count >= 0),
+    status VARCHAR(20) NOT NULL DEFAULT 'original'
+        CHECK (status IN ('original', 'modified', 'manual', 'invalid')),
+    quality_score TEXT NOT NULL DEFAULT '{}',
+    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+    CHECK (source_start IS NULL OR source_end IS NULL OR source_end >= source_start),
+    CHECK (source_start_line IS NULL OR source_end_line IS NULL OR source_end_line >= source_start_line)
+);
+
+CREATE INDEX IF NOT EXISTS idx_data_process_preview_task_file
+    ON data_process_preview_items(task_id, source_file_id, created_at);
+
+-- 子表在新库中到这里才存在；只修复本次新增 workflow_step 前的历史任务。
+UPDATE data_process_tasks task
+SET workflow_step = CASE
+    WHEN EXISTS (
+        SELECT 1 FROM data_process_preview_items preview
+        WHERE preview.task_id=task.id
+    ) THEN 'preview'
+    WHEN EXISTS (
+        SELECT 1 FROM data_process_source_files source_file
+        WHERE source_file.task_id=task.id AND source_file.deleted_at IS NULL
+    ) THEN 'upload'
+    ELSE task.workflow_step
+END
+WHERE task.id IN (SELECT id FROM data_process_workflow_backfill_ids)
+  AND task.status='pending';
+
+CREATE TABLE IF NOT EXISTS data_process_results (
+    id TEXT PRIMARY KEY,
+    task_id TEXT NOT NULL REFERENCES data_process_tasks(id) ON DELETE CASCADE,
+    preview_item_id TEXT REFERENCES data_process_preview_items(id) ON DELETE SET NULL,
+    instruction TEXT NOT NULL,
+    input TEXT NOT NULL DEFAULT '',
+    output TEXT NOT NULL,
+    original_instruction TEXT,
+    original_input TEXT,
+    original_output TEXT,
+    status VARCHAR(20) NOT NULL DEFAULT 'valid'
+        CHECK (status IN ('valid', 'modified', 'invalid')),
+    error TEXT,
+    split VARCHAR(20) CHECK (split IS NULL OR split IN ('train', 'validation', 'test')),
+    quality_score TEXT NOT NULL DEFAULT '{}',
+    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+    updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+CREATE INDEX IF NOT EXISTS idx_data_process_results_task_status
+    ON data_process_results(task_id, status, id);
+CREATE INDEX IF NOT EXISTS idx_data_process_results_task_split
+    ON data_process_results(task_id, split);
+
+CREATE TABLE IF NOT EXISTS dataset_file_versions (
+    id TEXT PRIMARY KEY,
+    dataset_file_id TEXT NOT NULL REFERENCES dataset_files(id) ON DELETE CASCADE,
+    version_no INTEGER NOT NULL CHECK (version_no > 0),
+    storage_object_id TEXT NOT NULL,
+    content_preview TEXT,
+    description TEXT,
+    base_version_id TEXT REFERENCES dataset_file_versions(id) ON DELETE SET NULL,
+    size_bytes BIGINT NOT NULL DEFAULT 0 CHECK (size_bytes >= 0),
+    record_count BIGINT NOT NULL DEFAULT 0 CHECK (record_count >= 0),
+    checksum_sha256 CHAR(64) NOT NULL,
+    source_task_id TEXT REFERENCES data_process_tasks(id) ON DELETE SET NULL,
+    metadata TEXT NOT NULL DEFAULT '{}',
+    created_by TEXT,
+    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+ALTER TABLE dataset_file_versions ADD COLUMN IF NOT EXISTS source_task_id TEXT;
+ALTER TABLE dataset_file_versions ADD COLUMN IF NOT EXISTS metadata TEXT NOT NULL DEFAULT '{}';
+CREATE UNIQUE INDEX IF NOT EXISTS uq_dataset_file_versions_no_002
+    ON dataset_file_versions(dataset_file_id, version_no);
+CREATE INDEX IF NOT EXISTS idx_dataset_file_versions_source_task_002
+    ON dataset_file_versions(source_task_id) WHERE source_task_id IS NOT NULL;
+
+CREATE TABLE IF NOT EXISTS dataset_records (
+    id TEXT PRIMARY KEY,
+    dataset_id TEXT NOT NULL REFERENCES datasets(id) ON DELETE CASCADE,
+    dataset_file_id TEXT REFERENCES dataset_files(id) ON DELETE CASCADE,
+    version_id TEXT REFERENCES dataset_file_versions(id) ON DELETE CASCADE,
+    line_no INTEGER,
+    split VARCHAR(20) CHECK (split IS NULL OR split IN ('train', 'validation', 'test')),
+    instruction TEXT,
+    input TEXT,
+    output TEXT,
+    raw TEXT NOT NULL DEFAULT '{}',
+    status VARCHAR(20) NOT NULL DEFAULT 'valid'
+        CHECK (status IN ('valid', 'modified', 'invalid')),
+    source_task_id TEXT REFERENCES data_process_tasks(id) ON DELETE SET NULL,
+    source_result_id TEXT REFERENCES data_process_results(id) ON DELETE SET NULL,
+    preview_item_id TEXT REFERENCES data_process_preview_items(id) ON DELETE SET NULL,
+    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+ALTER TABLE dataset_records ADD COLUMN IF NOT EXISTS source_task_id TEXT;
+ALTER TABLE dataset_records ADD COLUMN IF NOT EXISTS source_result_id TEXT;
+ALTER TABLE dataset_records ADD COLUMN IF NOT EXISTS preview_item_id TEXT;
+CREATE INDEX IF NOT EXISTS idx_dataset_records_dataset_002
+    ON dataset_records(dataset_id, id);
+CREATE INDEX IF NOT EXISTS idx_dataset_records_source_task_002
+    ON dataset_records(source_task_id, source_result_id);
+CREATE INDEX IF NOT EXISTS idx_datasets_source_task_002
+    ON datasets(source_task_id) WHERE source_task_id IS NOT NULL;
+CREATE INDEX IF NOT EXISTS idx_dataset_files_source_task_002
+    ON dataset_files(source_task_id) WHERE source_task_id IS NOT NULL;
+
+COMMIT;

backend/app/db/sql/002_governance.sql

@@ -0,0 +1,68 @@
+-- 平台治理：租户 / 审批 / 审计（字段以 platform_store 实际写入为准）
+CREATE TABLE IF NOT EXISTS tenants (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL,
+    code TEXT,
+    status TEXT DEFAULT 'active',
+    owner_user_id TEXT,
+    quota TEXT,
+    retention_policy_id TEXT,
+    create_time TEXT
+);
+
+CREATE TABLE IF NOT EXISTS approval_templates (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL,
+    steps TEXT,
+    create_time TEXT
+);
+
+CREATE TABLE IF NOT EXISTS approval_instances (
+    id TEXT PRIMARY KEY,
+    template_id TEXT,
+    resource_type TEXT,
+    resource_id TEXT,
+    applicant_id TEXT,
+    status TEXT DEFAULT 'pending',
+    current_step INTEGER DEFAULT 0,
+    create_time TEXT
+);
+
+CREATE TABLE IF NOT EXISTS approval_steps (
+    id TEXT PRIMARY KEY,
+    instance_id TEXT,
+    step_index INTEGER,
+    approver_id TEXT,
+    status TEXT DEFAULT 'pending',
+    comment TEXT,
+    time TEXT
+);
+
+CREATE TABLE IF NOT EXISTS audit_logs (
+    id TEXT PRIMARY KEY,
+    tenant_id TEXT,
+    project_id TEXT,
+    actor_id TEXT,
+    action TEXT,
+    target_type TEXT,
+    target_id TEXT,
+    detail TEXT,
+    client_ip TEXT,
+    time TEXT
+);
+
+CREATE INDEX IF NOT EXISTS idx_audit_tenant ON audit_logs(tenant_id);
+CREATE INDEX IF NOT EXISTS idx_audit_project ON audit_logs(project_id);
+CREATE INDEX IF NOT EXISTS idx_audit_action ON audit_logs(action);
+CREATE INDEX IF NOT EXISTS idx_audit_time ON audit_logs(time);
+
+CREATE TABLE IF NOT EXISTS retention_policies (
+    id TEXT PRIMARY KEY,
+    name TEXT NOT NULL,
+    scope TEXT,
+    rule TEXT,
+    status TEXT DEFAULT 'active',
+    create_time TEXT,
+    create_by TEXT,
+    updated_at TEXT
+);

backend/app/db/sql/003_model_path_governance.sql

@@ -0,0 +1,18 @@
+-- 003_model_path_governance
+-- 模型路径治理：增加 can_train 标识，区分本地可训练模型与 API / 远程模型。
+-- 训练预检阶段依赖该字段拦截不适合 LLaMA-Factory 本地训练的基座模型。
+
+-- 1. models 表增加 can_train（默认 0，后设搬迁为 1 的规则如下）
+ALTER TABLE models ADD COLUMN IF NOT EXISTS can_train INTEGER NOT NULL DEFAULT 0;
+
+-- 2. 将已有模型按规则推定 can_train：
+--    - path 非空 且 model_source != 'api' → 可训练 (1)
+--    - 其余 → 不可训练 (0)
+UPDATE models
+SET can_train = CASE
+    WHEN path IS NOT NULL AND path != '' AND model_source IS NOT NULL AND model_source != 'api' THEN 1
+    ELSE 0
+END;
+
+-- 3. 给 trained_models 增加 artifact_dir（训练产物目录扫描结果目录）
+ALTER TABLE trained_models ADD COLUMN IF NOT EXISTS artifact_dir TEXT;

backend/app/db/sql/003_tenant_quota.sql

@@ -0,0 +1,3 @@
+-- 租户配额与保留策略扩展（如后续治理表需补列，可在此追加）
+ALTER TABLE tenants ADD COLUMN IF NOT EXISTS gpu_quota TEXT;
+ALTER TABLE tenants ADD COLUMN IF NOT EXISTS storage_quota TEXT;

backend/app/main.py

@@ -0,0 +1,43 @@
+import asyncio
+from contextlib import suppress
+
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+
+from app.api.v1.router import api_router
+from app.core.config import get_settings
+from app.core.logging import configure_logging, setup_request_logging
+from app.workers.compute_poller import run_compute_poller
+
+
+def create_app() -> FastAPI:
+    settings = get_settings()
+    configure_logging(settings)
+
+    app = FastAPI(title=settings.app_name)
+    app.add_middleware(
+        CORSMiddleware,
+        allow_origins=settings.cors_allow_origins,
+        allow_credentials=True,
+        allow_methods=["*"],
+        allow_headers=["*"],
+    )
+    setup_request_logging(app)
+    app.include_router(api_router, prefix=settings.route_prefix)
+
+    @app.on_event("startup")
+    async def start_workers() -> None:
+        app.state.compute_poller_task = asyncio.create_task(run_compute_poller())
+
+    @app.on_event("shutdown")
+    async def stop_workers() -> None:
+        task = getattr(app.state, "compute_poller_task", None)
+        if task:
+            task.cancel()
+            with suppress(asyncio.CancelledError):
+                await task
+
+    return app
+
+
+app = create_app()

backend/app/modules/README.md

@@ -0,0 +1,15 @@
+# Backend Module Convention
+
+每个业务模块建议保持一致结构：
+
+```text
+module_name/
+  __init__.py
+  router.py       # FastAPI router
+  schemas.py      # Pydantic request/response models
+  service.py      # Business orchestration
+  repository.py   # Database access
+  permissions.py  # Optional resource permission checks
+```
+
+模块边界以 `docs/system-development-plan.md` 的页面模块开发工作包为准。

backend/app/modules/approval/__init__.py

@@ -0,0 +1 @@
+"""Approval workflow module."""

backend/app/modules/approval/router.py

@@ -0,0 +1,91 @@
+from __future__ import annotations
+
+from fastapi import APIRouter, Body
+from typing import Any
+
+from app.api.v1.endpoints.platform import ok, fail
+from app.db.platform_store import get_platform_store
+
+router = APIRouter(prefix="/approvals", tags=["approval"])
+
+
+@router.get("/templates")
+def list_templates() -> dict[str, Any]:
+    return ok(get_platform_store().approval_templates())
+
+
+@router.post("/templates")
+def create_template(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
+    if not payload.get("name"):
+        raise fail(400, "name 必填")
+    return ok(get_platform_store().create_approval_template(payload))
+
+
+@router.get("/templates/{template_id}")
+def get_template(template_id: str) -> dict[str, Any]:
+    try:
+        return ok(get_platform_store().approval_template(template_id))
+    except KeyError:
+        raise fail(404, "template not found")
+
+
+@router.put("/templates/{template_id}")
+def update_template(template_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
+    try:
+        return ok(get_platform_store().update_approval_template(template_id, payload))
+    except KeyError:
+        raise fail(404, "template not found")
+
+
+@router.delete("/templates/{template_id}")
+def delete_template(template_id: str) -> dict[str, Any]:
+    try:
+        return ok(get_platform_store().delete_approval_template(template_id))
+    except KeyError:
+        raise fail(404, "template not found")
+
+
+@router.get("")
+def list_instances(status: str | None = None) -> dict[str, Any]:
+    return ok(get_platform_store().approval_instances(status=status))
+
+
+@router.post("")
+def create_instance(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
+    for field in ("resource_type", "resource_id", "applicant_id"):
+        if not payload.get(field):
+            raise fail(400, f"{field} 必填")
+    try:
+        return ok(get_platform_store().create_approval_instance(payload))
+    except KeyError:
+        raise fail(404, "template not found")
+
+
+@router.get("/{instance_id}")
+def get_instance(instance_id: str) -> dict[str, Any]:
+    try:
+        return ok(get_platform_store().approval_instance(instance_id))
+    except KeyError:
+        raise fail(404, "instance not found")
+
+
+@router.post("/{instance_id}/steps/{step_index}/decision")
+def decide(
+    instance_id: str,
+    step_index: int,
+    payload: dict[str, Any] = Body(...),
+) -> dict[str, Any]:
+    if not payload.get("approver_id"):
+        raise fail(400, "approver_id 必填")
+    try:
+        return ok(
+            get_platform_store().decide_approval_step(
+                instance_id,
+                step_index,
+                approver_id=payload["approver_id"],
+                approved=bool(payload.get("approved", False)),
+                comment=payload.get("comment"),
+            )
+        )
+    except (KeyError, ValueError) as e:
+        raise fail(400, str(e))

backend/app/modules/audit/__init__.py

@@ -0,0 +1 @@
+"""Audit log module."""

backend/app/modules/auth/__init__.py

@@ -0,0 +1 @@
+"""Authentication and user session module."""

backend/app/modules/compute_gateway/__init__.py

@@ -0,0 +1 @@
+"""Application-side compute platform gateway module."""

backend/app/modules/compute_gateway/client.py

@@ -0,0 +1,247 @@
+from __future__ import annotations
+
+import time
+from typing import Any
+from urllib.parse import urljoin
+
+import httpx
+
+from app.core.config import get_settings
+
+
+def _join_url(base_url: str, path: str) -> str:
+    return urljoin(base_url.rstrip("/") + "/", path.lstrip("/"))
+
+
+def _unwrap_items(payload: Any) -> list[dict[str, Any]]:
+    if isinstance(payload, list):
+        return [item for item in payload if isinstance(item, dict)]
+    if isinstance(payload, dict):
+        data = payload.get("data")
+        if isinstance(data, dict) and isinstance(data.get("items"), list):
+            return [item for item in data["items"] if isinstance(item, dict)]
+        if isinstance(payload.get("items"), list):
+            return [item for item in payload["items"] if isinstance(item, dict)]
+        if isinstance(data, list):
+            return [item for item in data if isinstance(item, dict)]
+    return []
+
+
+def _unwrap_dict(payload: Any) -> dict[str, Any]:
+    if isinstance(payload, dict) and isinstance(payload.get("data"), dict):
+        return payload["data"]
+    return payload if isinstance(payload, dict) else {}
+
+
+# Inference calls are intentionally short-timeout:
+#   - load dispatch only confirms the compute node accepted the request
+#     (the actual model load now runs asynchronously on the node).
+#   - status/unload must never block the platform for long when a node is
+#     unreachable but still marked online.
+INFERENCE_LOAD_TIMEOUT = httpx.Timeout(30, connect=10)
+INFERENCE_STATUS_TIMEOUT = httpx.Timeout(30, connect=5)
+INFERENCE_UNLOAD_TIMEOUT = httpx.Timeout(30, connect=5)
+
+
+class ComputeNodeClient:
+    """Application-side client for one compute node.
+
+    The client accepts both current YG Compute API responses and common
+    wrapper shapes such as `{code,message,data}` to make future engine/node
+    adapters less brittle.
+    """
+
+    def __init__(self, api_base_url: str, token: str | None = None, timeout: float | None = None) -> None:
+        settings = get_settings()
+        self.api_base_url = api_base_url.rstrip("/")
+        self.token = token or settings.compute_service_token
+        self.timeout = timeout or settings.compute_request_timeout_seconds
+        self.route_prefix = settings.route_prefix.rstrip("/") or "/modelTF"
+
+    def headers(self) -> dict[str, str]:
+        if not self.token:
+            return {}
+        return {"X-Compute-Token": self.token}
+
+    async def test_connection(self) -> dict[str, Any]:
+        started = time.perf_counter()
+        health = await self.health()
+        gpus = await self.gpus()
+        return {
+            "success": True,
+            "latency_ms": int((time.perf_counter() - started) * 1000),
+            "health": health,
+            "gpus": gpus,
+        }
+
+    async def health(self) -> dict[str, Any]:
+        paths = [f"{self.route_prefix}/v1/compute/health", f"{self.route_prefix}/health", "/health"]
+        last_error = ""
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            for path in paths:
+                try:
+                    response = await client.get(_join_url(self.api_base_url, path))
+                    response.raise_for_status()
+                    return _unwrap_dict(response.json())
+                except Exception as exc:  # noqa: BLE001 - keep endpoint compatibility fallback broad
+                    last_error = str(exc)
+        raise RuntimeError(last_error or "compute health check failed")
+
+    async def gpus(self) -> list[dict[str, Any]]:
+        paths = [
+            f"{self.route_prefix}/compute/resources/gpus",
+            f"{self.route_prefix}/v1/compute/resources/gpus",
+            "/compute/resources/gpus",
+        ]
+        last_error = ""
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            for path in paths:
+                try:
+                    response = await client.get(_join_url(self.api_base_url, path))
+                    response.raise_for_status()
+                    return _unwrap_items(response.json())
+                except Exception as exc:  # noqa: BLE001
+                    last_error = str(exc)
+        raise RuntimeError(last_error or "compute gpu discovery failed")
+
+    async def create_job(self, payload: dict[str, Any]) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.post(_join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs"), json=payload)
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def preview_job(self, payload: dict[str, Any]) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.post(
+                _join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/preview"),
+                json=payload,
+            )
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def validate_job(self, payload: dict[str, Any]) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.post(
+                _join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/validate"),
+                json=payload,
+            )
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def check_paths(self, paths: list[dict[str, Any]]) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.post(
+                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/check-paths"),
+                json={"paths": paths},
+            )
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def list_files(
+        self,
+        root: str = "data",
+        relative_path: str = "",
+        directories_only: bool = False,
+    ) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.get(
+                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/list"),
+                params={"root": root, "relative_path": relative_path, "directories_only": directories_only},
+            )
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def get_job(self, job_id: str) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.get(_join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/{job_id}"))
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def stop_job(self, job_id: str) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.post(_join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/{job_id}/stop"))
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def job_logs(
+        self,
+        job_id: str,
+        tail_lines: int | None = None,
+        offset: int | None = None,
+        limit: int | None = None,
+    ) -> dict[str, Any]:
+        params = {
+            key: value
+            for key, value in {"tail_lines": tail_lines, "offset": offset, "limit": limit}.items()
+            if value is not None
+        }
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.get(
+                _join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/{job_id}/logs"),
+                params=params,
+            )
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def import_local_file(self, payload: dict[str, Any]) -> dict[str, Any]:
+        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
+            response = await client.post(
+                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/import-local"),
+                json=payload,
+            )
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    async def _request(
+        self,
+        method: str,
+        path: str,
+        json_data: dict[str, Any] | None = None,
+        timeout: float | None = None,
+    ) -> dict[str, Any]:
+        """Generic request method for compute API endpoints."""
+        url = _join_url(self.api_base_url, f"{self.route_prefix}{path}")
+        async with httpx.AsyncClient(timeout=timeout or 300, headers=self.headers()) as client:
+            if method.upper() == "GET":
+                response = await client.get(url)
+            else:
+                response = await client.post(url, json=json_data)
+            response.raise_for_status()
+            return _unwrap_dict(response.json())
+
+    # ── Inference helpers (short timeouts — see module constants) ──────────
+
+    async def inference_load(self, payload: dict[str, Any]) -> dict[str, Any]:
+        """Dispatch a model load. Returns as soon as the node accepts the
+        request; the node now loads asynchronously (status goes 'loading')."""
+        return await self._request("POST", "/inference/load", json_data=payload, timeout=INFERENCE_LOAD_TIMEOUT)
+
+    async def inference_status(self) -> dict[str, Any]:
+        return await self._request("GET", "/inference/status", timeout=INFERENCE_STATUS_TIMEOUT)
+
+    async def inference_unload(self) -> dict[str, Any]:
+        return await self._request("POST", "/inference/unload", json_data={}, timeout=INFERENCE_UNLOAD_TIMEOUT)
+
+    async def upload_file(
+        self,
+        filename: str,
+        content: bytes,
+        target_relative_path: str,
+        resource_type: str | None = None,
+        resource_id: str | None = None,
+    ) -> dict[str, Any]:
+        data = {
+            "target_relative_path": target_relative_path,
+            "resource_type": resource_type or "",
+            "resource_id": resource_id or "",
+        }
+        files = {"file": (filename, content)}
+        timeout = httpx.Timeout(max(self.timeout, 60), connect=self.timeout)
+        async with httpx.AsyncClient(timeout=timeout, headers=self.headers()) as client:
+            response = await client.post(
+                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/upload"),
+                data=data,
+                files=files,
+            )
+            response.raise_for_status()
+            return _unwrap_dict(response.json())

backend/app/modules/compute_gateway/sync.py

@@ -0,0 +1,191 @@
+from __future__ import annotations
+
+import json
+import time
+from typing import Any
+
+from app.db.platform_store import get_platform_store
+from app.modules.compute_gateway.client import ComputeNodeClient
+
+# starting 状态允许的最大轮询次数（约 40 * 3s ≈ 2 分钟），超过即判定节点不可达
+MAX_STARTING_ATTEMPTS = 40
+
+
+def _node_for_task(task: dict[str, Any]) -> dict[str, Any] | None:
+    return next((node for node in get_platform_store().compute_nodes() if node["id"] == task.get("compute_node_id")), None)
+
+
+def _parse_inference_load_status(task: dict[str, Any]) -> tuple[list[dict[str, Any]], dict[str, Any]]:
+    load_status = task.get("load_status") or {}
+    if isinstance(load_status, str):
+        try:
+            load_status = json.loads(load_status)
+        except (json.JSONDecodeError, TypeError):
+            load_status = {}
+    return load_status.get("loaded_models") or [], load_status
+
+
+async def reconcile_inference_loads(store: Any) -> list[dict[str, Any]]:
+    """推进处于 starting 状态的推理加载。
+
+    模型加载已改为异步派发：/model-compare/{id}/load 立即返回，这里在每次
+    轮询时查询对应计算节点的 /inference/status，把任务从 starting 推进到
+    ready/error。使用短超时，单节点不可达不会阻塞整轮轮询。
+    """
+    reconciled: list[dict[str, Any]] = []
+    now = time.time()
+    for task in store.compare_tasks():
+        items, _ = _parse_inference_load_status(task)
+        if not any(item.get("status") == "starting" for item in items):
+            continue
+        # dirty 只要处理过任一 starting 项就置位：load_attempts / last_polled_at
+        # 必须落库，否则节点不可达时计数不会累积，封顶逻辑永远触发不了
+        dirty = False
+        for item in items:
+            if item.get("status") != "starting":
+                continue
+            # 节流：同一 item 每 3s 只查询一次
+            if now - float(item.get("last_polled_at") or 0) < 3:
+                continue
+            item["last_polled_at"] = now
+            item["load_attempts"] = int(item.get("load_attempts") or 0) + 1
+            dirty = True
+            node = next((n for n in store.compute_nodes() if n["id"] == item.get("node_id")), None)
+            if not node:
+                item["status"] = "error"
+                item["error"] = "compute node deleted"
+                store.mark_inference_unloaded(item.get("node_id") or "")
+                continue
+            if not node.get("enabled") or node.get("scheduler_status") != "online":
+                item["status"] = "error"
+                item["error"] = "compute node offline"
+                store.mark_inference_unloaded(node["id"])
+                continue
+            try:
+                status = await ComputeNodeClient(node["api_base_url"]).inference_status()
+            except Exception as exc:  # noqa: BLE001 - node unreachable; keep retrying until cap
+                if int(item.get("load_attempts") or 0) >= MAX_STARTING_ATTEMPTS:
+                    item["status"] = "error"
+                    item["error"] = f"compute node unreachable: {exc}"
+                    store.mark_inference_unloaded(node["id"])
+                continue
+            node_status = status.get("status")
+            if node_status == "ready":
+                item["status"] = "ready"
+                item.pop("error", None)
+                store.mark_inference_loaded(node["id"])
+            elif node_status == "error":
+                item["status"] = "error"
+                item["error"] = status.get("error") or "model load failed on compute node"
+                store.mark_inference_unloaded(node["id"])
+            elif node_status == "idle":
+                # 节点重启导致已加载模型丢失
+                item["status"] = "error"
+                item["error"] = "model disappeared from compute node (node may have restarted)"
+                store.mark_inference_unloaded(node["id"])
+            # node_status == "loading" -> 保持 starting，下轮再查
+        if dirty:
+            if any(i.get("status") in {"ready", "running"} for i in items):
+                new_status = "loaded"
+            elif any(i.get("status") == "starting" for i in items):
+                new_status = "starting"  # 仍在加载中，保持 starting
+            else:
+                new_status = "failed"
+            store.update_compare_task(task["id"], {"status": new_status, "load_status": {"loaded_models": items}})
+            reconciled.append({"task_id": task["id"], "status": new_status})
+    return reconciled
+
+
+async def fetch_eval_result_content(client: ComputeNodeClient, node: dict[str, Any], job: dict[str, Any]) -> dict[str, Any] | None:
+    output_dir = job.get("output_dir")
+    if not output_dir:
+        return None
+    full_path = f"{str(output_dir).rstrip('/')}/eval_results.json"
+    data_root = "/data/yg-ft/"
+    if full_path.startswith(data_root):
+        full_path = full_path[len(data_root):]
+    rel_path = full_path.lstrip("/")
+    import httpx
+    url = f"{node['api_base_url'].rstrip('/')}/modelTF/compute/files/read"
+    async with httpx.AsyncClient(timeout=30, headers=client.headers()) as http:
+        response = await http.get(url, params={"path": rel_path})
+        response.raise_for_status()
+        payload = response.json()
+        return payload if isinstance(payload, dict) else None
+
+
+async def poll_compute_jobs_once() -> dict[str, Any]:
+    store = get_platform_store()
+    synced: list[dict[str, Any]] = []
+    failed: list[dict[str, str]] = []
+    for task in store.running_compute_tasks():
+        node = _node_for_task(task)
+        if not node:
+            failed.append({"task_id": task["id"], "error": "compute node not found"})
+            continue
+        try:
+            client = ComputeNodeClient(node["api_base_url"])
+            job = await client.get_job(task["compute_job_id"])
+            try:
+                logs = await client.job_logs(task["compute_job_id"], tail_lines=5000)
+                store.record_training_log_metrics(task["id"], str(logs.get("content") or ""))
+            except Exception:
+                pass
+            # P0-4: Force-fetch last log snippet when job reaches terminal state
+            if job.get("status") in {"failed", "stopped"}:
+                try:
+                    last_logs = await client.job_logs(task["compute_job_id"], tail_lines=200)
+                    job["log_snippet"] = str(last_logs.get("content") or "")[:8192]
+                except Exception:
+                    pass
+            synced.append(store.apply_compute_job(task["id"], job))
+        except Exception as exc:  # noqa: BLE001 - keep polling other jobs
+            failed.append({"task_id": task["id"], "error": str(exc)})
+    standalone_synced: list[dict[str, Any]] = []
+    for record in store.active_standalone_compute_jobs():
+        node = next((item for item in store.compute_nodes() if item["id"] == record.get("node_id")), None)
+        if not node:
+            failed.append({"job_id": record["id"], "error": "compute node not found"})
+            continue
+        try:
+            job = await ComputeNodeClient(node["api_base_url"]).get_job(record["id"])
+            standalone_synced.append(store.sync_model_merge_job(record["id"], job))
+        except Exception as exc:  # noqa: BLE001 - keep polling other jobs
+            failed.append({"job_id": record["id"], "error": str(exc)})
+
+    # ── Eval job sync ────────────────────────────────────────────────
+    eval_synced = 0
+    for eval_task in store.running_eval_tasks():
+        node = next(
+            (item for item in store.compute_nodes() if item["id"] == eval_task.get("compute_node_id")),
+            None,
+        )
+        if not node:
+            failed.append({"eval_task_id": eval_task["id"], "error": "compute node not found"})
+            continue
+        try:
+            client = ComputeNodeClient(node["api_base_url"])
+            job = await client.get_job(eval_task["compute_job_id"])
+            result_content = None
+            # Try to read eval_results.json from the job output directory
+            if job.get("status") == "completed" and job.get("output_dir"):
+                try:
+                    result_content = await fetch_eval_result_content(client, node, job)
+                except Exception:
+                    pass
+            store.apply_eval_job_result(eval_task["id"], job, result_content)
+            # 评测 GPU 占用由 eval_tasks 状态派生，无需维护推理内存标记
+            eval_synced += 1
+        except Exception as exc:  # noqa: BLE001
+            failed.append({"eval_task_id": eval_task["id"], "error": str(exc)})
+
+    # ── Inference load reconciliation ─────────────────────────────────────
+    try:
+        inference_reconciled = await reconcile_inference_loads(store)
+    except Exception as exc:  # noqa: BLE001 - keep polling alive
+        failed.append({"inference_reconcile": str(exc)})
+        inference_reconciled = []
+
+    return {"synced": len(synced) + len(standalone_synced) + eval_synced, "failed": failed,
+            "items": synced, "standalone": standalone_synced, "eval_synced": eval_synced,
+            "inference_reconciled": inference_reconciled}

backend/app/modules/data_process/__init__.py

@@ -0,0 +1 @@
+"""Data processing module."""

backend/app/modules/data_process/constants.py

@@ -0,0 +1,7 @@
+"""数据处理模块的共享限制。"""
+
+MAX_QA_PAIRS_PER_ITEM = 50
+MODEL_GENERATION_BATCH_SIZE = 10
+
+
+__all__ = ["MAX_QA_PAIRS_PER_ITEM", "MODEL_GENERATION_BATCH_SIZE"]

backend/app/modules/data_process/dataset_format.py

@@ -0,0 +1,148 @@
+"""Dataset format validation for Alpaca, ShareGPT, DPO, CPT formats.
+
+Used by the training preflight flow to validate that uploaded dataset files
+conform to the declared format before submitting to the compute node.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+
+def _load_sample(path: str | None, content: str | None = None, max_samples: int = 20) -> list[dict[str, Any]]:
+    """Load up to max_samples records from JSONL file path or raw content string."""
+    try:
+        if content is not None:
+            text = content.strip()
+        elif path:
+            with open(path, "r", encoding="utf-8") as fh:
+                text = fh.read().strip()
+        else:
+            return []
+    except Exception:
+        return []
+
+    if not text:
+        return []
+
+    lines = text.splitlines()[:max_samples]
+    records: list[dict[str, Any]] = []
+    for line in lines:
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            record = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+        if isinstance(record, dict):
+            records.append(record)
+    return records
+
+
+def _check_alpaca(records: list[dict[str, Any]]) -> list[str]:
+    """Validate Alpaca format: requires 'instruction' field."""
+    errors: list[str] = []
+    if not records:
+        errors.append("Alpaca 格式数据集无有效记录")
+        return errors
+    missing_instruction = sum(1 for r in records if not r.get("instruction"))
+    if missing_instruction:
+        errors.append(
+            f"Alpaca 格式要求每条记录包含 instruction 字段，"
+            f"前{len(records)}条中有{missing_instruction}条缺失"
+        )
+    return errors
+
+
+def _check_sharegpt(records: list[dict[str, Any]]) -> list[str]:
+    """Validate ShareGPT format: requires 'messages' (list of dicts with role/content)."""
+    errors: list[str] = []
+    if not records:
+        errors.append("ShareGPT 格式数据集无有效记录")
+        return errors
+    bad = 0
+    for r in records:
+        messages = r.get("messages")
+        if not isinstance(messages, list) or not messages:
+            bad += 1
+            continue
+        for msg in messages:
+            if not isinstance(msg, dict) or "role" not in msg or "content" not in msg:
+                bad += 1
+                break
+    if bad:
+        errors.append(
+            f"ShareGPT 格式要求每条记录包含 messages 列表，"
+            f"每条消息需有 role 和 content 字段，前{len(records)}条中有{bad}条不符合"
+        )
+    return errors
+
+
+def _check_dpo(records: list[dict[str, Any]]) -> list[str]:
+    """Validate DPO format: requires 'chosen' and 'rejected' fields."""
+    errors: list[str] = []
+    if not records:
+        errors.append("DPO 格式数据集无有效记录")
+        return errors
+    missing_chosen = sum(1 for r in records if not r.get("chosen"))
+    missing_rejected = sum(1 for r in records if not r.get("rejected"))
+    if missing_chosen:
+        errors.append(f"DPO 格式要求 chosen 字段，前{len(records)}条中有{missing_chosen}条缺失")
+    if missing_rejected:
+        errors.append(f"DPO 格式要求 rejected 字段，前{len(records)}条中有{missing_rejected}条缺失")
+    return errors
+
+
+def _check_cpt(records: list[dict[str, Any]]) -> list[str]:
+    """Validate CPT format: requires 'text' field, should NOT have instruction/output."""
+    errors: list[str] = []
+    if not records:
+        errors.append("CPT 格式数据集无有效记录")
+        return errors
+    missing_text = sum(1 for r in records if not r.get("text"))
+    has_instruction = sum(1 for r in records if r.get("instruction") or r.get("output"))
+    if missing_text:
+        errors.append(f"CPT 格式要求 text 字段，前{len(records)}条中有{missing_text}条缺失")
+    if has_instruction:
+        errors.append(
+            f"CPT 格式不应包含 instruction/output 字段（疑似 Alpaca 格式），"
+            f"前{len(records)}条中有{has_instruction}条包含此类字段"
+        )
+    return errors
+
+
+FORMAT_VALIDATORS = {
+    "alpaca": _check_alpaca,
+    "alpaca_jsonl": _check_alpaca,
+    "sharegpt": _check_sharegpt,
+    "dpo": _check_dpo,
+    "cpt": _check_cpt,
+    "pt": _check_cpt,
+}
+
+
+def validate_dataset_format(
+    dataset_format: str,
+    content: str | None = None,
+    path: str | None = None,
+    max_samples: int = 20,
+) -> list[str]:
+    """Validate dataset content against expected format.
+
+    Args:
+        dataset_format: One of 'alpaca', 'sharegpt', 'dpo', 'cpt'.
+        content: Raw file content (JSONL text). Mutually exclusive with path.
+        path: File path to read content from.
+        max_samples: Maximum records to sample for validation.
+
+    Returns:
+        List of error messages (empty if valid).
+    """
+    fmt = str(dataset_format).lower().strip()
+    validator = FORMAT_VALIDATORS.get(fmt)
+    if not validator:
+        return [f"不支持的数据集格式: {dataset_format}，支持的格式: {', '.join(sorted(FORMAT_VALIDATORS))}"]
+    records = _load_sample(path=path, content=content, max_samples=max_samples)
+    return validator(records)

backend/app/modules/data_process/document_chunking.py

@@ -0,0 +1,444 @@
+"""基于 Docling 与 LlamaIndex 的文档切分实现。"""
+
+from __future__ import annotations
+
+import os
+import re
+import threading
+import unicodedata
+from dataclasses import dataclass
+from functools import lru_cache
+from io import BytesIO
+from typing import Any, Literal
+
+import tiktoken
+from docling_core.transforms.chunker.hierarchical_chunker import ChunkingSerializerProvider
+from llama_index.core import Document
+from llama_index.core.base.embeddings.base import BaseEmbedding
+from llama_index.core.node_parser import SemanticSplitterNodeParser, SentenceSplitter
+
+from app.modules.data_process.algorithms import normalize_text
+
+ChunkMethod = Literal["layout_hybrid", "semantic", "fixed"]
+
+_PAGE_FURNITURE = re.compile(
+    r"(?m)^\s*(?:第\s*\d+\s*页\s*共\s*\d+\s*页|[-—–]?\s*\d+\s*[/／]\s*\d+\s*[-—–]?)\s*$"
+)
+_COMPACT_CHARACTER = re.compile(r"[\w\u3400-\u4dbf\u4e00-\u9fff]", re.UNICODE)
+_CONVERTER_LOCK = threading.Lock()
+
+
+@dataclass(frozen=True, slots=True)
+class DocumentChunk:
+    """切片正文及其在原文件中的可追溯信息。"""
+
+    original_content: str
+    contextualized_content: str
+    source_start: int | None
+    source_end: int | None
+    source_start_line: int | None
+    source_end_line: int | None
+    token_count: int
+    heading_path: tuple[str, ...] = ()
+    source_pages: tuple[int, ...] = ()
+    doc_item_refs: tuple[str, ...] = ()
+    source_bboxes: tuple[dict[str, Any], ...] = ()
+
+
+def _sentence_chunks(text: str) -> list[str]:
+    """提供稳定的中英文句界，避免 LlamaIndex 默认分词器下载额外资源。"""
+
+    boundary = re.compile(
+        r".*?(?:\n\s*\n|[。！？!?；;](?:[\"'”’）】》]*)|\.(?:\s+|$)|$)",
+        re.DOTALL,
+    )
+    return [part for part in boundary.findall(text) if part]
+
+
+@lru_cache(maxsize=1)
+def _tokenizer() -> tiktoken.Encoding:
+    return tiktoken.get_encoding("cl100k_base")
+
+
+def _text_chunks(
+    text: str,
+    *,
+    chunk_size: int,
+    chunk_overlap: int,
+) -> list[DocumentChunk]:
+    normalized = normalize_text(text)
+    if not normalized:
+        return []
+    splitter = SentenceSplitter(
+        chunk_size=chunk_size,
+        chunk_overlap=chunk_overlap,
+        tokenizer=_tokenizer().encode,
+        chunking_tokenizer_fn=_sentence_chunks,
+        include_metadata=False,
+        include_prev_next_rel=False,
+    )
+    nodes = splitter.get_nodes_from_documents([Document(text=normalized)])
+    return _nodes_to_chunks(nodes, normalized)
+
+
+def chunk_fixed_text(
+    text: str,
+    *,
+    chunk_size: int,
+    chunk_overlap: int,
+) -> list[DocumentChunk]:
+    """使用 LlamaIndex SentenceSplitter 按句界控制固定 Token 长度。"""
+
+    return _text_chunks(text, chunk_size=chunk_size, chunk_overlap=chunk_overlap)
+
+
+@lru_cache(maxsize=1)
+def _semantic_embedding_model() -> BaseEmbedding:
+    # 模型可在部署环境覆盖；默认模型体积较小且适合中英文语义边界判断。
+    from llama_index.embeddings.huggingface import HuggingFaceEmbedding
+
+    return HuggingFaceEmbedding(
+        model_name=os.getenv("DATA_PROCESS_EMBEDDING_MODEL", "BAAI/bge-small-zh-v1.5"),
+        device=os.getenv("DATA_PROCESS_EMBEDDING_DEVICE", "cpu"),
+        trust_remote_code=False,
+    )
+
+
+def chunk_semantic_text(
+    text: str,
+    *,
+    chunk_size: int,
+    chunk_overlap: int,
+    breakpoint_percentile_threshold: int,
+    embed_model: BaseEmbedding | None = None,
+) -> list[DocumentChunk]:
+    """使用 LlamaIndex SemanticSplitter 识别主题跳变，再限制最大长度。"""
+
+    normalized = normalize_text(text)
+    if not normalized:
+        return []
+    splitter = SemanticSplitterNodeParser.from_defaults(
+        embed_model=embed_model or _semantic_embedding_model(),
+        breakpoint_percentile_threshold=breakpoint_percentile_threshold,
+        buffer_size=1,
+        sentence_splitter=_sentence_chunks,
+        include_metadata=False,
+        include_prev_next_rel=False,
+    )
+    semantic_nodes = splitter.get_nodes_from_documents([Document(text=normalized)])
+    result: list[DocumentChunk] = []
+    search_from = 0
+    for node in semantic_nodes:
+        content = node.get_content().strip()
+        if not content:
+            continue
+        start = _locate_text(normalized, content, search_from)
+        if start is None:
+            start = _locate_text(normalized, content, 0)
+        if start is None:
+            continue
+        if len(_tokenizer().encode(content)) <= chunk_size:
+            result.append(_make_text_chunk(normalized, start, start + len(content)))
+        else:
+            for child in _text_chunks(
+                content,
+                chunk_size=chunk_size,
+                chunk_overlap=chunk_overlap,
+            ):
+                if child.source_start is None or child.source_end is None:
+                    continue
+                result.append(
+                    _make_text_chunk(
+                        normalized,
+                        start + child.source_start,
+                        start + child.source_end,
+                    )
+                )
+        search_from = start + len(content)
+    return result
+
+
+def _nodes_to_chunks(nodes: list[Any], source_text: str) -> list[DocumentChunk]:
+    chunks: list[DocumentChunk] = []
+    search_from = 0
+    for node in nodes:
+        content = node.get_content().strip()
+        if not content:
+            continue
+        raw_start = getattr(node, "start_char_idx", None)
+        raw_end = getattr(node, "end_char_idx", None)
+        if (
+            isinstance(raw_start, int)
+            and isinstance(raw_end, int)
+            and source_text[raw_start:raw_end].strip() == content
+        ):
+            start = raw_start + len(source_text[raw_start:raw_end]) - len(source_text[raw_start:raw_end].lstrip())
+        else:
+            start = _locate_text(source_text, content, search_from)
+            if start is None:
+                start = _locate_text(source_text, content, 0)
+        if start is None:
+            continue
+        end = start + len(content)
+        chunks.append(_make_text_chunk(source_text, start, end))
+        search_from = max(search_from, end)
+    return chunks
+
+
+def _locate_text(source: str, content: str, start: int) -> int | None:
+    position = source.find(content, start)
+    return position if position >= 0 else None
+
+
+def _make_text_chunk(source: str, start: int, end: int) -> DocumentChunk:
+    content = source[start:end]
+    return DocumentChunk(
+        original_content=content,
+        contextualized_content=content,
+        source_start=start,
+        source_end=end,
+        source_start_line=source.count("\n", 0, start) + 1,
+        source_end_line=source.count("\n", 0, max(start, end - 1)) + 1,
+        token_count=len(_tokenizer().encode(content)),
+    )
+
+
+@lru_cache(maxsize=1)
+def _document_converter():
+    from docling.document_converter import DocumentConverter
+
+    return DocumentConverter()
+
+
+class _MarkdownSerializerProvider(ChunkingSerializerProvider):
+    def get_serializer(self, doc: Any):
+        from docling_core.transforms.chunker.hierarchical_chunker import ChunkingDocSerializer
+        from docling_core.transforms.serializer.markdown import (
+            MarkdownParams,
+            MarkdownTableSerializer,
+        )
+        from docling_core.types.doc import DocItemLabel
+
+        excluded = {
+            DocItemLabel.DOCUMENT_INDEX,
+            DocItemLabel.PAGE_HEADER,
+            DocItemLabel.PAGE_FOOTER,
+        }
+        return ChunkingDocSerializer(
+            doc=doc,
+            table_serializer=MarkdownTableSerializer(),
+            params=MarkdownParams(
+                labels=set(DocItemLabel) - excluded,
+                compact_tables=True,
+                image_placeholder="",
+                escape_html=False,
+                escape_underscores=False,
+            ),
+        )
+
+
+def _clean_layout_text(value: str) -> str:
+    return normalize_text(_PAGE_FURNITURE.sub("", value)).strip()
+
+
+def _compact_with_offsets(value: str) -> tuple[str, list[int]]:
+    compact: list[str] = []
+    offsets: list[int] = []
+    for index, character in enumerate(unicodedata.normalize("NFKC", value)):
+        if _COMPACT_CHARACTER.fullmatch(character):
+            compact.append(character.casefold())
+            offsets.append(index)
+    return "".join(compact), offsets
+
+
+def _project_layout_span(
+    source_text: str,
+    content: str,
+    *,
+    compact_source: str,
+    source_offsets: list[int],
+    compact_start: int,
+) -> tuple[int | None, int | None, int]:
+    compact_content, _ = _compact_with_offsets(content)
+    if len(compact_content) < 4:
+        return None, None, compact_start
+    position = compact_source.find(compact_content, compact_start)
+    if position < 0:
+        position = compact_source.find(compact_content)
+    if position < 0:
+        return None, None, compact_start
+    start = source_offsets[position]
+    end = source_offsets[position + len(compact_content) - 1] + 1
+    while start > 0 and source_text[start - 1] not in "\r\n":
+        start -= 1
+    while end < len(source_text) and source_text[end] not in "\r\n":
+        end += 1
+    return start, end, position + len(compact_content)
+
+
+def chunk_layout_document(
+    raw: bytes,
+    *,
+    filename: str,
+    source_text: str,
+    chunk_size: int,
+) -> list[DocumentChunk]:
+    """使用 Docling HybridChunker 按版面层级、列表与表格边界切分。"""
+
+    from docling.chunking import HybridChunker
+    from docling.datamodel.base_models import DocumentStream
+    from docling.exceptions import BaseError as DoclingError
+    from docling_core.transforms.chunker.tokenizer.openai import OpenAITokenizer
+    from docling_core.types.doc import DocItemLabel
+
+    try:
+        with _CONVERTER_LOCK:
+            conversion = _document_converter().convert(
+                DocumentStream(name=filename, stream=BytesIO(raw))
+            )
+    except DoclingError as exc:
+        raise ValueError(f"文档版面解析失败: {exc}") from exc
+    chunker = HybridChunker(
+        tokenizer=OpenAITokenizer(tokenizer=_tokenizer(), max_tokens=chunk_size),
+        serializer_provider=_MarkdownSerializerProvider(),
+        merge_peers=True,
+        repeat_table_header=True,
+    )
+    compact_source, source_offsets = _compact_with_offsets(source_text)
+    compact_start = 0
+    result: list[DocumentChunk] = []
+    excluded = {
+        DocItemLabel.DOCUMENT_INDEX,
+        DocItemLabel.PAGE_HEADER,
+        DocItemLabel.PAGE_FOOTER,
+    }
+    for raw_chunk in chunker.chunk(conversion.document):
+        doc_items = tuple(raw_chunk.meta.doc_items or ())
+        if doc_items and all(item.label in excluded for item in doc_items):
+            continue
+        content = _clean_layout_text(raw_chunk.text)
+        if not content:
+            continue
+        contextualized = _clean_layout_text(chunker.contextualize(raw_chunk)) or content
+        start, end, compact_start = _project_layout_span(
+            source_text,
+            content,
+            compact_source=compact_source,
+            source_offsets=source_offsets,
+            compact_start=compact_start,
+        )
+        original = source_text[start:end] if start is not None and end is not None else content
+        pages: set[int] = set()
+        refs: list[str] = []
+        bboxes: list[dict[str, Any]] = []
+        for item in doc_items:
+            refs.append(str(item.self_ref))
+            for provenance in item.prov or ():
+                pages.add(int(provenance.page_no))
+                bbox = provenance.bbox
+                bboxes.append(
+                    {
+                        "page": int(provenance.page_no),
+                        "left": float(bbox.l),
+                        "top": float(bbox.t),
+                        "right": float(bbox.r),
+                        "bottom": float(bbox.b),
+                        "origin": str(bbox.coord_origin.value),
+                    }
+                )
+        result.append(
+            DocumentChunk(
+                original_content=original,
+                contextualized_content=contextualized,
+                source_start=start,
+                source_end=end,
+                source_start_line=(source_text.count("\n", 0, start) + 1 if start is not None else None),
+                source_end_line=(
+                    source_text.count("\n", 0, max(start or 0, (end or 1) - 1)) + 1
+                    if end is not None
+                    else None
+                ),
+                token_count=len(_tokenizer().encode(contextualized)),
+                heading_path=tuple(str(item) for item in (raw_chunk.meta.headings or ())),
+                source_pages=tuple(sorted(pages)),
+                doc_item_refs=tuple(refs),
+                source_bboxes=tuple(bboxes),
+            )
+        )
+    return result
+
+
+def merge_short_chunks(
+    chunks: list[DocumentChunk],
+    *,
+    source_text: str,
+    min_token_count: int,
+    max_token_count: int,
+) -> list[DocumentChunk]:
+    """在不突破长度上限的前提下，把过短块并入相邻内容。"""
+
+    result: list[DocumentChunk] = []
+    index = 0
+    while index < len(chunks):
+        current = chunks[index]
+        if current.token_count >= min_token_count:
+            result.append(current)
+            index += 1
+            continue
+        if index + 1 < len(chunks):
+            combined = _combine_chunks(current, chunks[index + 1], source_text)
+            if combined.token_count <= max_token_count:
+                result.append(combined)
+                index += 2
+                continue
+        if result:
+            combined = _combine_chunks(result[-1], current, source_text)
+            if combined.token_count <= max_token_count:
+                result[-1] = combined
+                index += 1
+                continue
+        result.append(current)
+        index += 1
+    return result
+
+
+def _combine_chunks(
+    left: DocumentChunk,
+    right: DocumentChunk,
+    source_text: str,
+) -> DocumentChunk:
+    contextualized = "\n\n".join(
+        part for part in (left.contextualized_content, right.contextualized_content) if part
+    )
+    start = left.source_start
+    end = right.source_end
+    has_contiguous_source = (
+        start is not None
+        and left.source_end is not None
+        and right.source_start is not None
+        and end is not None
+        and left.source_end <= right.source_start
+    )
+    original = (
+        source_text[start:end]
+        if has_contiguous_source and start is not None and end is not None
+        else "\n\n".join(
+            part for part in (left.original_content, right.original_content) if part
+        )
+    )
+    if not has_contiguous_source:
+        start = None
+        end = None
+    return DocumentChunk(
+        original_content=original,
+        contextualized_content=contextualized,
+        source_start=start,
+        source_end=end,
+        source_start_line=left.source_start_line if start is not None else None,
+        source_end_line=right.source_end_line if end is not None else None,
+        token_count=len(_tokenizer().encode(contextualized)),
+        heading_path=left.heading_path or right.heading_path,
+        source_pages=tuple(sorted(set(left.source_pages) | set(right.source_pages))),
+        doc_item_refs=left.doc_item_refs + right.doc_item_refs,
+        source_bboxes=left.source_bboxes + right.source_bboxes,
+    )

backend/app/modules/data_process/generation.py

@@ -0,0 +1,562 @@
+"""数据处理任务的大模型生成适配器。"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+import re
+from collections.abc import Callable, Iterable, Mapping
+from typing import Any
+from urllib.parse import urlsplit, urlunsplit
+
+import httpx
+
+from app.modules.data_process.algorithms import normalize_text, stable_split_assignments
+from app.modules.data_process.constants import (
+    MAX_QA_PAIRS_PER_ITEM,
+    MODEL_GENERATION_BATCH_SIZE,
+)
+
+
+class ModelGenerationError(ValueError):
+    """模型配置、响应或调用失败。"""
+
+
+class _TerminalModelGenerationError(ModelGenerationError):
+    """使用相同参数重试也无法恢复的模型响应错误。"""
+
+
+OUTPUT_TYPE_STANDARD = "standard"
+OUTPUT_TYPE_REASONING = "reasoning"
+SUPPORTED_OUTPUT_TYPES = {OUTPUT_TYPE_STANDARD, OUTPUT_TYPE_REASONING}
+REASONING_DETAIL_NORMAL = "normal"
+REASONING_DETAIL_DETAILED = "detailed"
+SUPPORTED_REASONING_DETAILS = {
+    REASONING_DETAIL_NORMAL,
+    REASONING_DETAIL_DETAILED,
+}
+MINIMAX_M3_API_HOSTS = {"api.minimax.io", "api.minimaxi.com"}
+MINIMAX_M3_MIN_COMPLETION_TOKENS = 4096
+logger = logging.getLogger(__name__)
+
+
+def _is_retryable_generation_error(exc: Exception) -> bool:
+    if isinstance(exc, _TerminalModelGenerationError):
+        return False
+    if isinstance(exc, httpx.HTTPStatusError):
+        status_code = exc.response.status_code
+        return status_code in {408, 425, 429} or status_code >= 500
+    if isinstance(exc, httpx.RequestError):
+        return True
+    return isinstance(exc, (json.JSONDecodeError, ModelGenerationError))
+
+
+def _is_official_minimax_m3(endpoint: str, model_name: str) -> bool:
+    host = (urlsplit(endpoint).hostname or "").casefold()
+    return host in MINIMAX_M3_API_HOSTS and model_name.casefold() == "minimax-m3"
+
+
+def chat_completions_url(value: str) -> str:
+    """把域名、基础 URL 或完整地址统一为 chat completions 地址。"""
+
+    raw = (value or "").strip()
+    if not raw:
+        raise ModelGenerationError("generation model api_url is required")
+    if "://" not in raw:
+        raw = f"https://{raw}"
+    parsed = urlsplit(raw)
+    if parsed.scheme not in {"http", "https"} or not parsed.hostname:
+        raise ModelGenerationError("generation model api_url must be an HTTP(S) host or URL")
+    if parsed.username or parsed.password:
+        raise ModelGenerationError("generation model api_url must not contain credentials")
+
+    path = parsed.path.rstrip("/")
+    if path.endswith("/chat/completions"):
+        target_path = path
+    elif path.endswith("/v1"):
+        target_path = f"{path}/chat/completions"
+    elif not path:
+        target_path = "/v1/chat/completions"
+    else:
+        target_path = f"{path}/v1/chat/completions"
+    return urlunsplit((parsed.scheme, parsed.netloc, target_path, "", ""))
+
+
+def _response_choice(payload: Mapping[str, Any]) -> Mapping[str, Any]:
+    try:
+        choice = payload["choices"][0]
+    except (KeyError, IndexError, TypeError) as exc:
+        raise ModelGenerationError("模型响应缺少 choices[0]") from exc
+    if not isinstance(choice, Mapping):
+        raise ModelGenerationError("模型响应 choices[0] 不是对象")
+    return choice
+
+
+def _response_finish_reason(payload: Mapping[str, Any]) -> str:
+    try:
+        return str(_response_choice(payload).get("finish_reason") or "").strip().lower()
+    except ModelGenerationError:
+        return ""
+
+
+def _response_content_length(payload: Mapping[str, Any]) -> int:
+    try:
+        message = _response_choice(payload).get("message")
+        if not isinstance(message, Mapping):
+            return 0
+        content = message.get("content")
+        if isinstance(content, str):
+            return len(content)
+        if isinstance(content, list):
+            return sum(
+                len(str(item.get("text") or ""))
+                for item in content
+                if isinstance(item, Mapping)
+            )
+    except ModelGenerationError:
+        pass
+    return 0
+
+
+def _raise_for_terminal_response(payload: Mapping[str, Any]) -> Mapping[str, Any]:
+    choice = _response_choice(payload)
+    base_response = payload.get("base_resp")
+    status_code: Any = None
+    status_message = ""
+    if isinstance(base_response, Mapping):
+        status_code = base_response.get("status_code")
+        status_message = re.sub(
+            r"\s+", " ", str(base_response.get("status_msg") or "")
+        ).strip()[:200]
+
+    if bool(payload.get("input_sensitive")) or status_code in {1026, "1026"}:
+        raise _TerminalModelGenerationError(
+            f"模型输入触发内容安全拦截（code={status_code or 1026}）"
+        )
+    if bool(payload.get("output_sensitive")) or status_code in {1027, "1027"}:
+        raise _TerminalModelGenerationError(
+            f"模型输出触发内容安全拦截（code={status_code or 1027}）"
+        )
+
+    finish_reason = str(choice.get("finish_reason") or "").strip().lower()
+    if finish_reason == "length":
+        raise _TerminalModelGenerationError(
+            "模型输出因达到 Token 上限被截断（finish_reason=length），"
+            "请提高最大输出长度后重试"
+        )
+    if finish_reason == "content_filter":
+        raise _TerminalModelGenerationError(
+            "模型输出被内容安全策略拦截（finish_reason=content_filter）"
+        )
+    if finish_reason in {"tool_calls", "function_call"}:
+        raise _TerminalModelGenerationError(
+            f"模型返回了当前生成任务不支持的工具调用（finish_reason={finish_reason}）"
+        )
+    if status_code not in {None, "", 0, "0"}:
+        detail = f"：{status_message}" if status_message else ""
+        raise _TerminalModelGenerationError(
+            f"模型服务返回业务错误（code={status_code}）{detail}"
+        )
+    return choice
+
+
+def _message_content(payload: Mapping[str, Any]) -> str:
+    choice = _raise_for_terminal_response(payload)
+    message = choice.get("message")
+    if not isinstance(message, Mapping):
+        raise ModelGenerationError("模型响应缺少 choices[0].message")
+    content = message.get("content")
+    if isinstance(content, str):
+        result = content
+    elif isinstance(content, list):
+        parts = [
+            str(item.get("text") or "")
+            for item in content
+            if isinstance(item, Mapping) and item.get("type") in {None, "text", "output_text"}
+        ]
+        result = "".join(parts)
+    elif content is None:
+        result = ""
+    else:
+        raise ModelGenerationError("模型响应 content 必须是文本")
+    if not result.strip():
+        raise ModelGenerationError("模型返回的最终内容为空，未生成可解析的 JSON")
+    return result
+
+
+def _json_documents(content: str) -> list[Any]:
+    decoder = json.JSONDecoder()
+    documents: list[Any] = []
+    cursor = 0
+    while cursor < len(content):
+        match = re.search(r"[\[{]", content[cursor:])
+        if not match:
+            break
+        start = cursor + match.start()
+        try:
+            value, end = decoder.raw_decode(content[start:])
+        except json.JSONDecodeError:
+            cursor = start + 1
+            continue
+        if isinstance(value, (Mapping, list)):
+            documents.append(value)
+        cursor = start + max(end, 1)
+    return documents
+
+
+def _json_payload(content: str) -> Any:
+    # 只移除模型在 JSON 之前自行输出的思考过程，不能破坏 JSON 字段中的训练内容。
+    cleaned = content.strip()
+    if re.match(r"^\s*<think>", cleaned, flags=re.IGNORECASE) and not re.match(
+        r"^\s*<think>[\s\S]*?</think>", cleaned, flags=re.IGNORECASE
+    ):
+        raise ModelGenerationError("模型思考内容未闭合，响应可能已被截断")
+    cleaned = re.sub(
+        r"^\s*(?:<think>[\s\S]*?</think>\s*)+",
+        "",
+        cleaned,
+        count=1,
+        flags=re.IGNORECASE,
+    ).strip()
+    fenced = re.fullmatch(r"```(?:json)?\s*([\s\S]*?)\s*```", cleaned, flags=re.IGNORECASE)
+    if fenced:
+        cleaned = fenced.group(1).strip()
+    try:
+        return json.loads(cleaned)
+    except json.JSONDecodeError as direct_error:
+        documents = _json_documents(cleaned)
+        if len(documents) == 1:
+            return documents[0]
+        if len(documents) > 1:
+            raise ModelGenerationError("模型响应包含多个 JSON 对象，无法确定应使用哪一个")
+        raise ModelGenerationError(
+            "模型响应中没有找到唯一且完整的 JSON 对象"
+            f"（第 {direct_error.lineno} 行，第 {direct_error.colno} 列）"
+        ) from direct_error
+
+
+def _result_items(payload: Any) -> list[Mapping[str, Any]]:
+    if isinstance(payload, list):
+        values = payload
+    elif isinstance(payload, Mapping):
+        nested = next(
+            (
+                payload[key]
+                for key in ("items", "results", "data", "records")
+                if isinstance(payload.get(key), list)
+            ),
+            None,
+        )
+        values = nested if isinstance(nested, list) else [payload]
+    else:
+        raise ModelGenerationError("model JSON must be an object or array")
+    items = [item for item in values if isinstance(item, Mapping)]
+    if not items:
+        raise ModelGenerationError("model JSON does not contain result objects")
+    return items
+
+
+def _prompt_messages(
+    prompt: str,
+    content: str,
+    count: int,
+    *,
+    start_index: int,
+    total_count: int,
+    output_type: str,
+    reasoning_detail: str,
+) -> list[dict[str, str]]:
+    end_index = start_index + count - 1
+    if output_type == OUTPUT_TYPE_REASONING:
+        schema = '{"items":[{"instruction":"...","input":"...","reasoning":"...","answer":"..."}]}'
+        detail_rule = (
+            "推理详细程度为“详细”：完整展开问题条件、来源依据、中间计算或推导，"
+            "并在得出答案前核对结论；每一步都必须能从来源内容中验证。"
+            if reasoning_detail == REASONING_DETAIL_DETAILED
+            else
+            "推理详细程度为“普通”：只保留得出答案所需的关键依据和必要步骤，"
+            "避免冗长复述、套话和无依据扩展。"
+        )
+        output_rule = (
+            "你正在生成用于训练推理模型的思维链数据，而不是普通问答数据。"
+            "instruction、reasoning 和 answer 均不得为空；reasoning 必须是基于来源内容、"
+            f"可核对的推理过程，answer 只写最终答案。{detail_rule}"
+            "这是思维链输出模式，即使其他提示语要求省略分析，也不得省略 reasoning。"
+            "不要自行添加 <think> 标签，系统会在保存时统一组装。"
+        )
+    else:
+        schema = '{"items":[{"instruction":"...","input":"...","output":"..."}]}'
+        output_rule = (
+            "你正在生成标准监督微调问答数据。instruction 和 output 不得为空；"
+            "output 只写最终答案，禁止输出分析、推理过程或 <think> 标签。"
+        )
+    schema_instruction = (
+        f"必须只返回 JSON 对象，格式为 {schema}；items 必须包含 {count} 条。"
+        f"这是总计 {total_count} 条中的第 {start_index}-{end_index} 条，"
+        "各条必须使用不同的提问角度和表述，避免重复。"
+        f"{output_rule}不要输出 Markdown 代码围栏或 JSON 之外的说明。"
+    )
+    base_prompt = normalize_text(prompt) or "请根据来源内容生成可用于监督微调的问答数据。"
+    if "{{ content }}" in base_prompt:
+        user_prompt = base_prompt.replace("{{ content }}", content)
+        return [
+            {"role": "system", "content": schema_instruction},
+            {"role": "user", "content": user_prompt},
+        ]
+    return [
+        {"role": "system", "content": f"{base_prompt}\n{schema_instruction}"},
+        {"role": "user", "content": f"来源内容：\n{content}"},
+    ]
+
+
+def generate_model_records(
+    preview_items: Iterable[Mapping[str, Any]],
+    *,
+    model: Mapping[str, Any],
+    config: Mapping[str, Any],
+    task_id: str,
+    split: Mapping[str, int],
+    qa_pairs_per_item: int,
+    client: httpx.Client | None = None,
+    on_progress: Callable[[int, int], None] | None = None,
+) -> list[dict[str, Any]]:
+    """调用 OpenAI 兼容接口，将预览切片生成标准训练记录。
+
+    每个切片按安全批次调用模型；失败批次会产生一条可人工修复的
+    invalid 结果，已经成功的批次不会丢失。
+    """
+
+    if not 1 <= qa_pairs_per_item <= MAX_QA_PAIRS_PER_ITEM:
+        raise ModelGenerationError(f"qa_pairs_per_item must be in [1, {MAX_QA_PAIRS_PER_ITEM}]")
+    output_type = str(config.get("output_type") or OUTPUT_TYPE_STANDARD).strip().lower()
+    if output_type not in SUPPORTED_OUTPUT_TYPES:
+        raise ModelGenerationError(f"output_type must be one of {sorted(SUPPORTED_OUTPUT_TYPES)}")
+    reasoning_detail = str(
+        config.get("reasoning_detail") or REASONING_DETAIL_NORMAL
+    ).strip().lower()
+    if reasoning_detail not in SUPPORTED_REASONING_DETAILS:
+        raise ModelGenerationError(
+            f"reasoning_detail must be one of {sorted(SUPPORTED_REASONING_DETAILS)}"
+        )
+    endpoint = chat_completions_url(str(model.get("api_url") or ""))
+    model_name = str(model.get("online_model_name") or model.get("name") or "").strip()
+    if not model_name:
+        raise ModelGenerationError("generation model name is required")
+    is_minimax_m3 = _is_official_minimax_m3(endpoint, model_name)
+
+    temperature = float(config.get("temperature", 0.7))
+    max_tokens = int(config.get("max_tokens", 1024))
+    timeout = max(1.0, min(120.0, float(config.get("request_timeout_seconds", 60))))
+    retries = max(0, min(5, int(config.get("generation_retries", 2))))
+    headers = {"Content-Type": "application/json"}
+    api_key = str(model.get("api_key") or "").strip()
+    if api_key:
+        headers["Authorization"] = f"Bearer {api_key}"
+
+    owns_client = client is None
+    http_client = client or httpx.Client(timeout=timeout)
+    results: list[dict[str, Any]] = []
+    try:
+        preview_list = list(preview_items)
+        total_items = len(preview_list)
+        for item_index, item in enumerate(preview_list):
+            preview_id = str(item.get("id") or f"preview-{item_index + 1}")
+            content = normalize_text(
+                str(item.get("edited_content") or item.get("original_content") or "")
+            )
+            for batch_offset in range(0, qa_pairs_per_item, MODEL_GENERATION_BATCH_SIZE):
+                batch_count = min(
+                    MODEL_GENERATION_BATCH_SIZE,
+                    qa_pairs_per_item - batch_offset,
+                )
+                batch_start = batch_offset + 1
+                batch_end = batch_offset + batch_count
+                request_payload: dict[str, Any] = {
+                    "model": model_name,
+                    "messages": _prompt_messages(
+                        str(config.get("generation_prompt") or ""),
+                        content,
+                        batch_count,
+                        start_index=batch_start,
+                        total_count=qa_pairs_per_item,
+                        output_type=output_type,
+                        reasoning_detail=reasoning_detail,
+                    ),
+                    "temperature": temperature,
+                }
+                if is_minimax_m3:
+                    request_payload.update(
+                        reasoning_split=True,
+                        max_completion_tokens=max(
+                            max_tokens,
+                            MINIMAX_M3_MIN_COMPLETION_TOKENS,
+                        ),
+                    )
+                else:
+                    request_payload["max_tokens"] = max_tokens
+                if bool(config.get("json_mode", False)) and not is_minimax_m3:
+                    request_payload["response_format"] = {"type": "json_object"}
+
+                last_error: Exception | None = None
+                generated_items: list[Mapping[str, Any]] | None = None
+                for _ in range(retries + 1):
+                    try:
+                        response = http_client.post(
+                            endpoint,
+                            headers=headers,
+                            json=request_payload,
+                        )
+                        response.raise_for_status()
+                        body = response.json()
+                        if not isinstance(body, Mapping):
+                            raise ModelGenerationError("model response body must be a JSON object")
+                        try:
+                            candidate_items = _result_items(
+                                _json_payload(_message_content(body))
+                            )
+                        except ModelGenerationError as exc:
+                            logger.warning(
+                                "data process model response rejected task_id=%s model=%s "
+                                "finish_reason=%s response_chars=%s input_sensitive=%s "
+                                "output_sensitive=%s reason=%s",
+                                task_id,
+                                model_name,
+                                _response_finish_reason(body) or "missing",
+                                _response_content_length(body),
+                                bool(body.get("input_sensitive")),
+                                bool(body.get("output_sensitive")),
+                                str(exc),
+                            )
+                            raise
+                        if len(candidate_items) < batch_count:
+                            raise ModelGenerationError(
+                                "model response contains fewer result objects than requested: "
+                                f"expected {batch_count}, got {len(candidate_items)}"
+                            )
+                        generated_items = candidate_items
+                        break
+                    except (
+                        httpx.HTTPError,
+                        json.JSONDecodeError,
+                        ModelGenerationError,
+                    ) as exc:
+                        last_error = exc
+                        if not _is_retryable_generation_error(exc):
+                            break
+
+                if generated_items is None:
+                    error_message = str(last_error or "model generation failed")[:2000]
+                    failure_instruction = (
+                        f"模型生成失败，请人工补充（第 {batch_start}-{batch_end} 条）"
+                    )
+                    result_id = (
+                        "result_"
+                        f"{hashlib.sha256(f'{preview_id}:error:{batch_start}'.encode()).hexdigest()[:16]}"
+                    )
+                    results.append(
+                        {
+                            "id": result_id,
+                            "preview_item_id": preview_id,
+                            "instruction": failure_instruction,
+                            "input": content,
+                            "output": "",
+                            "original_instruction": failure_instruction,
+                            "original_input": content,
+                            "original_output": "",
+                            "status": "invalid",
+                            "error": error_message,
+                            "split": "train",
+                        }
+                    )
+                    continue
+
+                for batch_index, value in enumerate(generated_items[:batch_count]):
+                    variant_index = batch_offset + batch_index
+                    instruction = normalize_text(
+                        str(value.get("instruction") or value.get("question") or "")
+                    )
+                    input_text = normalize_text(
+                        str(value.get("input") or value.get("context") or "")
+                    )
+                    if output_type == OUTPUT_TYPE_REASONING:
+                        reasoning = normalize_text(
+                            re.sub(
+                                r"</?think>",
+                                "",
+                                str(value.get("reasoning") or value.get("analysis") or ""),
+                                flags=re.IGNORECASE,
+                            )
+                        )
+                        answer = normalize_text(
+                            re.sub(
+                                r"</?think>",
+                                "",
+                                str(
+                                    value.get("answer")
+                                    or value.get("final_answer")
+                                    or value.get("output")
+                                    or ""
+                                ),
+                                flags=re.IGNORECASE,
+                            )
+                        )
+                        output = (
+                            f"<think>\n{reasoning}\n</think>\n{answer}"
+                            if reasoning and answer
+                            else answer or (f"<think>\n{reasoning}\n</think>" if reasoning else "")
+                        )
+                        valid = bool(instruction and reasoning and answer)
+                        missing_error = "model result is missing instruction, reasoning or answer"
+                    else:
+                        output = normalize_text(
+                            str(
+                                value.get("output")
+                                or value.get("answer")
+                                or value.get("response")
+                                or ""
+                            )
+                        )
+                        output = normalize_text(
+                            re.sub(
+                                r"<think>[\s\S]*?(?:</think>|$)",
+                                "",
+                                output,
+                                flags=re.IGNORECASE,
+                            )
+                        )
+                        valid = bool(instruction and output)
+                        missing_error = "model result is missing instruction or output"
+                    raw_id = f"{preview_id}:{variant_index + 1}:{instruction}:{output}"
+                    result_id = f"result_{hashlib.sha256(raw_id.encode()).hexdigest()[:16]}"
+                    results.append(
+                        {
+                            "id": result_id,
+                            "preview_item_id": preview_id,
+                            "instruction": instruction,
+                            "input": input_text,
+                            "output": output,
+                            "original_instruction": instruction,
+                            "original_input": input_text,
+                            "original_output": output,
+                            "status": "valid" if valid else "invalid",
+                            "error": (None if valid else missing_error),
+                            "split": "train",
+                        }
+                    )
+            if on_progress:
+                on_progress(item_index + 1, total_items)
+    finally:
+        if owns_client:
+            http_client.close()
+    assignments = stable_split_assignments(
+        [str(result["id"]) for result in results],
+        split,
+        seed=task_id,
+    )
+    for result, assignment in zip(results, assignments, strict=True):
+        result["split"] = assignment
+    return results
+
+
+__all__ = ["ModelGenerationError", "chat_completions_url", "generate_model_records"]

backend/app/modules/data_process/office_preview.py

@@ -0,0 +1,308 @@
+"""Word 与 Excel 原文件的安全、受限预览模型。
+
+预览只返回浏览器绘制所需的结构化数据，不返回或执行 Office 包中的活动内容。
+DOCX 的字符偏移与上传时的正文抽取规则保持一致，供前端定位当前切片。
+"""
+
+from __future__ import annotations
+
+import io
+import re
+from typing import Any
+
+from docx import Document
+from docx.oxml.table import CT_Tbl
+from docx.oxml.text.paragraph import CT_P
+from docx.table import Table
+from docx.text.paragraph import Paragraph
+from openpyxl import load_workbook
+
+from app.modules.data_process.algorithms import (
+    _MAX_WORKBOOK_COLUMNS,
+    _MAX_WORKBOOK_HEADER_SCAN_ROWS,
+    _infer_xlsx_header_region,
+    _normalize_spreadsheet_value,
+    _rewrite_xlsx_workbook_relationships,
+    _validate_office_archive,
+    _xlsx_sheet_merge_ranges,
+    normalize_text,
+)
+
+MAX_DOCX_PREVIEW_BLOCKS = 2_000
+MAX_XLSX_PREVIEW_ROWS = 200
+
+
+def _docx_alignment(paragraph: Paragraph) -> str:
+    value = paragraph.alignment
+    return {
+        0: "left",
+        1: "center",
+        2: "right",
+        3: "justify",
+        4: "distribute",
+        5: "justify",
+        7: "justify",
+        8: "distribute",
+        9: "distribute",
+    }.get(int(value) if value is not None else -1, "left")
+
+
+def _docx_heading_level(paragraph: Paragraph) -> int | None:
+    style = paragraph.style
+    if style is None:
+        return None
+    style_name = str(style.name or "")
+    style_id = str(style.style_id or "")
+    match = re.search(r"(?:heading|标题)\s*([1-6])", f"{style_name} {style_id}", re.IGNORECASE)
+    return int(match.group(1)) if match else None
+
+
+def build_docx_preview(raw: bytes) -> dict[str, Any]:
+    """把 DOCX 转为保留标题、段落和表格顺序的浏览器预览模型。"""
+
+    _validate_office_archive(raw, "docx")
+    try:
+        document = Document(io.BytesIO(raw))
+    except Exception as exc:
+        raise ValueError(f"invalid DOCX file: {exc}") from exc
+
+    blocks: list[dict[str, Any]] = []
+    source_cursor = 0
+    has_source_content = False
+    rendered_blocks = 0
+    truncated = False
+
+    def source_range(value: str) -> tuple[str, int, int] | None:
+        nonlocal source_cursor, has_source_content
+        text = normalize_text(value)
+        if not text:
+            return None
+        if has_source_content:
+            source_cursor += 2
+        start = source_cursor
+        source_cursor += len(text)
+        has_source_content = True
+        return text, start, source_cursor
+
+    for child in document.element.body.iterchildren():
+        if rendered_blocks >= MAX_DOCX_PREVIEW_BLOCKS:
+            truncated = True
+            break
+
+        if isinstance(child, CT_P):
+            paragraph = Paragraph(child, document)
+            located = source_range(paragraph.text)
+            if located is None:
+                continue
+            text, start, end = located
+            style_name = str(paragraph.style.name or "") if paragraph.style else ""
+            blocks.append(
+                {
+                    "type": "paragraph",
+                    "text": text,
+                    "style": style_name,
+                    "heading_level": _docx_heading_level(paragraph),
+                    "alignment": _docx_alignment(paragraph),
+                    "is_list": "list" in style_name.casefold() or "列表" in style_name,
+                    "source_start": start,
+                    "source_end": end,
+                }
+            )
+            rendered_blocks += 1
+            continue
+
+        if not isinstance(child, CT_Tbl):
+            continue
+        table = Table(child, document)
+        preview_rows: list[dict[str, Any]] = []
+        for row in table.rows:
+            if rendered_blocks >= MAX_DOCX_PREVIEW_BLOCKS:
+                truncated = True
+                break
+            cell_values = [normalize_text(cell.text) for cell in row.cells]
+            located = source_range("\t".join(cell_values))
+            if located is None:
+                continue
+            _, start, end = located
+            preview_rows.append(
+                {
+                    "cells": cell_values,
+                    "source_start": start,
+                    "source_end": end,
+                }
+            )
+            rendered_blocks += 1
+        if preview_rows:
+            blocks.append({"type": "table", "rows": preview_rows})
+        if truncated:
+            break
+
+    return {
+        "format": "docx",
+        "blocks": blocks,
+        "truncated": truncated,
+    }
+
+
+def build_xlsx_preview(
+    raw: bytes,
+    *,
+    sheet_index: int = 0,
+    offset: int = 0,
+    limit: int = 100,
+) -> dict[str, Any]:
+    """按工作表分页返回 XLSX 的表头和记录网格。"""
+
+    if sheet_index < 0 or offset < 0:
+        raise ValueError("sheet_index and offset must be non-negative")
+    if limit < 1 or limit > MAX_XLSX_PREVIEW_ROWS:
+        raise ValueError(
+            f"XLSX preview limit must be in [1, {MAX_XLSX_PREVIEW_ROWS}]"
+        )
+
+    _validate_office_archive(raw, "xlsx")
+    merged_by_sheet, normalized_targets = _xlsx_sheet_merge_ranges(raw)
+    workbook_raw = (
+        _rewrite_xlsx_workbook_relationships(raw, normalized_targets)
+        if normalized_targets
+        else raw
+    )
+    try:
+        workbook = load_workbook(
+            io.BytesIO(workbook_raw),
+            read_only=True,
+            data_only=True,
+            keep_links=False,
+        )
+    except Exception as exc:
+        raise ValueError(f"invalid XLSX file: {exc}") from exc
+
+    try:
+        sheets = [
+            {
+                "index": index,
+                "name": worksheet.title,
+                "state": worksheet.sheet_state,
+            }
+            for index, worksheet in enumerate(workbook.worksheets)
+        ]
+        if not sheets:
+            raise ValueError("XLSX workbook contains no worksheets")
+        if sheet_index >= len(sheets):
+            raise ValueError("XLSX worksheet index is out of range")
+
+        worksheet = workbook.worksheets[sheet_index]
+        reset_dimensions = getattr(worksheet, "reset_dimensions", None)
+        if callable(reset_dimensions):
+            reset_dimensions()
+        row_iterator = enumerate(worksheet.iter_rows(values_only=True), start=1)
+        buffered_rows: dict[int, tuple[Any, ...]] = {}
+
+        def normalized_values(row: tuple[Any, ...]) -> list[Any]:
+            values = list(row)
+            while values and values[-1] in {None, ""}:
+                values.pop()
+            if len(values) > _MAX_WORKBOOK_COLUMNS:
+                raise ValueError(
+                    f"XLSX worksheet {worksheet.title!r} exceeds "
+                    f"{_MAX_WORKBOOK_COLUMNS} columns"
+                )
+            return values
+
+        for row_number, row in row_iterator:
+            values = normalized_values(row)
+            if not values or all(value in {None, ""} for value in values):
+                continue
+            buffered_rows[row_number] = tuple(values)
+            if len(buffered_rows) >= _MAX_WORKBOOK_HEADER_SCAN_ROWS:
+                break
+
+        if not buffered_rows:
+            return {
+                "format": "xlsx",
+                "sheets": sheets,
+                "active_sheet": {
+                    "index": sheet_index,
+                    "name": worksheet.title,
+                    "columns": [],
+                    "rows": [],
+                    "offset": offset,
+                    "limit": limit,
+                    "has_more": False,
+                },
+            }
+
+        _, header_end_row, headers = _infer_xlsx_header_region(
+            worksheet.title,
+            buffered_rows,
+            merged_by_sheet.get(worksheet.title, ()),
+        )
+        preview_rows: list[dict[str, Any]] = []
+        record_index = 0
+        has_more = False
+
+        def append_row(row_number: int, values: tuple[Any, ...] | list[Any]) -> bool:
+            nonlocal record_index, has_more
+            row_values = list(values)
+            if len(row_values) > len(headers):
+                raise ValueError(
+                    f"XLSX worksheet {worksheet.title!r} has a row wider than its header"
+                )
+            row_values.extend([None] * (len(headers) - len(row_values)))
+            record = {
+                header: _normalize_spreadsheet_value(value)
+                for header, value in zip(headers, row_values, strict=True)
+            }
+            if not any(value not in {"", None} for value in record.values()):
+                return False
+            current_index = record_index
+            record_index += 1
+            if current_index < offset:
+                return False
+            if len(preview_rows) >= limit:
+                has_more = True
+                return True
+            preview_rows.append(
+                {
+                    "row_number": row_number,
+                    "record_index": current_index,
+                    "values": [record[header] for header in headers],
+                    "record": record,
+                }
+            )
+            return False
+
+        for row_number, values in buffered_rows.items():
+            if row_number > header_end_row and append_row(row_number, values):
+                break
+        else:
+            for row_number, row in row_iterator:
+                values = normalized_values(row)
+                if not values or all(value in {None, ""} for value in values):
+                    continue
+                if append_row(row_number, values):
+                    break
+
+        return {
+            "format": "xlsx",
+            "sheets": sheets,
+            "active_sheet": {
+                "index": sheet_index,
+                "name": worksheet.title,
+                "columns": headers,
+                "rows": preview_rows,
+                "offset": offset,
+                "limit": limit,
+                "has_more": has_more,
+            },
+        }
+    finally:
+        workbook.close()
+
+
+__all__ = [
+    "MAX_DOCX_PREVIEW_BLOCKS",
+    "MAX_XLSX_PREVIEW_ROWS",
+    "build_docx_preview",
+    "build_xlsx_preview",
+]

backend/app/modules/data_process/schema_cli.py

@@ -0,0 +1,76 @@
+"""数据处理运行表的显式检查与安装命令。"""
+
+from __future__ import annotations
+
+import argparse
+from urllib.parse import urlsplit
+
+from app.modules.data_process.store import DataProcessStore
+
+REQUIRED_TASK_COLUMNS = (
+    "generation_run_id",
+    "results_confirmed",
+    "workflow_step",
+    "preview_status",
+    "preview_progress",
+    "preview_run_id",
+    "preview_failure_reason",
+    "preview_total_files",
+    "preview_completed_files",
+)
+
+
+def _target_label(database_url: str) -> str:
+    parsed = urlsplit(database_url)
+    database = parsed.path.strip("/") or "(unknown)"
+    return f"{parsed.hostname or '(unknown)'}:{parsed.port or 5432}/{database}"
+
+
+def _schema_ready(store: DataProcessStore) -> bool:
+    with store.connect() as conn:
+        row = conn.execute(
+            """
+            SELECT COUNT(*) = %s AS ready
+            FROM information_schema.columns
+            WHERE table_schema=current_schema()
+              AND table_name='data_process_tasks'
+              AND column_name = ANY(%s)
+            """,
+            (len(REQUIRED_TASK_COLUMNS), list(REQUIRED_TASK_COLUMNS)),
+        ).fetchone()
+    return bool(row and row["ready"])
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description="检查或显式安装数据处理运行表（不会由应用启动自动执行）"
+    )
+    action = parser.add_mutually_exclusive_group(required=True)
+    action.add_argument("--check", action="store_true", help="只读检查迁移是否已安装")
+    action.add_argument("--apply", action="store_true", help="执行 002 数据处理迁移")
+    parser.add_argument(
+        "--yes",
+        action="store_true",
+        help="确认允许修改 DATABASE_URL 指向的数据库；与 --apply 同时使用",
+    )
+    args = parser.parse_args()
+
+    store = DataProcessStore()
+    target = _target_label(store.database_url)
+    if args.check:
+        ready = _schema_ready(store)
+        print(f"数据处理 schema：{'已安装' if ready else '未安装'}；目标：{target}")
+        return 0 if ready else 1
+    if not args.yes:
+        parser.error("--apply 必须同时提供 --yes，确认修改目标数据库")
+
+    print(f"正在安装数据处理 schema；目标：{target}")
+    store.ensure_schema()
+    if not _schema_ready(store):
+        raise RuntimeError("迁移执行后仍未检测到 generation_run_id")
+    print("数据处理 schema 安装完成")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

backend/app/modules/data_process/storage.py

@@ -0,0 +1,560 @@
+"""数据处理原始源文件的受控本地对象存储。"""
+
+from __future__ import annotations
+
+import os
+import re
+import stat
+import unicodedata
+import uuid
+from dataclasses import dataclass
+from functools import lru_cache
+from pathlib import Path, PurePosixPath
+from typing import Iterable, Iterator
+from urllib.parse import quote, unquote, urlsplit
+
+
+class DataProcessStorageError(ValueError):
+    """本地对象引用或文件系统状态不安全。"""
+
+
+@dataclass(frozen=True, slots=True)
+class StagedSourceObject:
+    """尚未发布的原始文件；绝对路径仅在存储模块内部流转。"""
+
+    reference: str
+    _temporary_path: Path
+    _relative_path: PurePosixPath
+
+
+def _default_storage_root() -> Path:
+    return Path(__file__).resolve().parents[3] / "storage" / "data-process"
+
+
+def _configured_storage_root() -> Path:
+    configured = os.getenv("DATA_PROCESS_STORAGE_DIR", "").strip()
+    if not configured:
+        return _default_storage_root()
+    path = Path(configured).expanduser()
+    # 相对配置固定以 backend 目录为基准，
+    # 避免从不同 cwd 启动时写入不同位置。
+    return path if path.is_absolute() else Path(__file__).resolve().parents[3] / path
+
+
+def _safe_component(value: str, label: str) -> str:
+    if not value or value in {".", ".."} or len(value) > 128:
+        raise DataProcessStorageError(f"invalid {label}")
+    if not value[0].isalnum() or any(
+        not (character.isalnum() or character in {"-", "_", "."})
+        for character in value
+    ):
+        raise DataProcessStorageError(f"invalid {label}")
+    return value
+
+
+def _safe_basename(value: str) -> str:
+    if not value or len(value.encode("utf-8")) > 255:
+        raise DataProcessStorageError("invalid source file name")
+    if value != Path(value).name or "/" in value or "\\" in value or "\x00" in value:
+        raise DataProcessStorageError("invalid source file name")
+    if value in {".", ".."} or any(
+        unicodedata.category(character).startswith("C") for character in value
+    ):
+        raise DataProcessStorageError("invalid source file name")
+    return value
+
+
+class LocalDataProcessStorage:
+    """只允许访问配置根目录下的版本化原始文件。"""
+
+    def __init__(self, root: str | os.PathLike[str] | Path | None = None) -> None:
+        configured = Path(root) if root is not None else _configured_storage_root()
+        configured = configured.expanduser()
+        if configured.exists() and configured.is_symlink():
+            raise DataProcessStorageError("data process storage root must not be a symlink")
+        configured.mkdir(parents=True, exist_ok=True, mode=0o700)
+        self._root = configured.resolve(strict=True)
+        # StagedSourceObject 本身是普通 dataclass，不能只依赖其中的路径字段判断
+        # 来源；只接受由当前存储实例实际签发的对象，
+        # 避免调用方伪造暂存路径。
+        self._issued_staged_objects: dict[Path, StagedSourceObject] = {}
+        self._ensure_directory(self._root / ".staging")
+
+    @property
+    def root(self) -> Path:
+        """仅供运维和测试检查；API 响应不得序列化该属性。"""
+
+        return self._root
+
+    def new_batch_id(self) -> str:
+        return f"batch-{uuid.uuid4().hex}"
+
+    def stage_bytes(
+        self,
+        *,
+        batch_id: str,
+        task_id: str,
+        source_file_id: str,
+        version: int,
+        name: str,
+        content: bytes,
+    ) -> StagedSourceObject:
+        batch_id = _safe_component(batch_id, "batch id")
+        task_id = _safe_component(task_id, "task id")
+        source_file_id = _safe_component(source_file_id, "source file id")
+        if isinstance(version, bool) or not isinstance(version, int) or version < 1:
+            raise DataProcessStorageError("invalid source file version")
+        basename = _safe_basename(name)
+        if not isinstance(content, bytes):
+            raise TypeError("content must be bytes")
+
+        batch_directory = self._ensure_directory(self._root / ".staging" / batch_id)
+        temporary_path = batch_directory / f"{source_file_id}-{uuid.uuid4().hex}.tmp"
+        flags = os.O_CREAT | os.O_EXCL | os.O_WRONLY
+        if hasattr(os, "O_NOFOLLOW"):
+            flags |= os.O_NOFOLLOW
+        descriptor = os.open(temporary_path, flags, 0o600)
+        try:
+            with os.fdopen(descriptor, "wb", closefd=True) as stream:
+                stream.write(content)
+                stream.flush()
+                os.fsync(stream.fileno())
+        except Exception:
+            temporary_path.unlink(missing_ok=True)
+            raise
+
+        relative_path = PurePosixPath(
+            task_id,
+            source_file_id,
+            f"v{version}",
+            basename,
+        )
+        reference = (
+            "local://data-process/"
+            f"{task_id}/{source_file_id}/v{version}/{quote(basename, safe='')}"
+        )
+        staged = StagedSourceObject(reference, temporary_path, relative_path)
+        self._issued_staged_objects[temporary_path] = staged
+        return staged
+
+    def stage_copy(
+        self,
+        *,
+        batch_id: str,
+        source_reference: str,
+        expected_source_task_id: str,
+        expected_source_file_id: str,
+        task_id: str,
+        source_file_id: str,
+        version: int,
+        name: str,
+    ) -> StagedSourceObject:
+        """为不可变源对象创建独立目录项，不把大文件重新读入内存。"""
+
+        batch_id = _safe_component(batch_id, "batch id")
+        task_id = _safe_component(task_id, "task id")
+        source_file_id = _safe_component(source_file_id, "source file id")
+        if isinstance(version, bool) or not isinstance(version, int) or version < 1:
+            raise DataProcessStorageError("invalid source file version")
+        basename = _safe_basename(name)
+        source_relative = self._relative_from_reference(source_reference)
+        if source_relative is None:
+            raise DataProcessStorageError("original source object is not available")
+        self._assert_expected_owner(
+            source_relative,
+            expected_task_id=expected_source_task_id,
+            expected_source_file_id=expected_source_file_id,
+        )
+        descriptor, source_info = self._open_read_descriptor(source_relative)
+        os.close(descriptor)
+
+        batch_directory = self._ensure_directory(self._root / ".staging" / batch_id)
+        temporary_path = batch_directory / f"{source_file_id}-{uuid.uuid4().hex}.tmp"
+        source_path = self._path_for_relative(source_relative)
+        try:
+            os.link(source_path, temporary_path, follow_symlinks=False)
+            copy_info = temporary_path.lstat()
+            if (
+                not stat.S_ISREG(copy_info.st_mode)
+                or source_info.st_dev != copy_info.st_dev
+                or source_info.st_ino != copy_info.st_ino
+            ):
+                raise DataProcessStorageError("source storage object changed while copying")
+        except Exception:
+            temporary_path.unlink(missing_ok=True)
+            raise
+
+        relative_path = PurePosixPath(
+            task_id,
+            source_file_id,
+            f"v{version}",
+            basename,
+        )
+        reference = (
+            "local://data-process/"
+            f"{task_id}/{source_file_id}/v{version}/{quote(basename, safe='')}"
+        )
+        staged = StagedSourceObject(reference, temporary_path, relative_path)
+        self._issued_staged_objects[temporary_path] = staged
+        return staged
+
+    def publish(self, objects: Iterable[StagedSourceObject]) -> None:
+        staged = list(objects)
+        published: list[StagedSourceObject] = []
+        try:
+            seen_temporary_paths: set[Path] = set()
+            for item in staged:
+                self._validate_staged_object(item, require_file=True)
+                if item._temporary_path in seen_temporary_paths:
+                    raise DataProcessStorageError("duplicate staged source object")
+                seen_temporary_paths.add(item._temporary_path)
+            for item in staged:
+                final_path = self._path_for_relative(item._relative_path)
+                self._ensure_directory(final_path.parent)
+                if final_path.exists() or final_path.is_symlink():
+                    raise DataProcessStorageError("source storage object already exists")
+                os.link(item._temporary_path, final_path, follow_symlinks=False)
+                published.append(item)
+                item._temporary_path.unlink()
+                self._fsync_directory(final_path.parent)
+        except Exception:
+            for item in reversed(published):
+                try:
+                    self.delete(item.reference)
+                except Exception:
+                    # 回滚必须尽量处理其余对象，并保留真正的发布异常。
+                    pass
+            for item in staged:
+                try:
+                    self.discard([item])
+                except Exception:
+                    pass
+            raise
+        self.discard(staged)
+
+    def discard(self, objects: Iterable[StagedSourceObject]) -> None:
+        staged = list(objects)
+        for item in staged:
+            self._validate_staged_object(item, require_file=False)
+
+        batch_directories: set[Path] = set()
+        first_error: Exception | None = None
+        for item in staged:
+            temporary_path = item._temporary_path
+            try:
+                temporary_path.unlink(missing_ok=True)
+            except Exception as exc:
+                if first_error is None:
+                    first_error = exc
+            else:
+                self._issued_staged_objects.pop(temporary_path, None)
+            batch_directories.add(temporary_path.parent)
+        for directory in batch_directories:
+            self._remove_empty_directory(directory)
+        if first_error is not None:
+            raise first_error
+
+    def read(self, reference: str) -> bytes | None:
+        """读取 local 引用；旧 ``db://`` 对象返回 ``None`` 由数据库正文兜底。"""
+
+        relative_path = self._relative_from_reference(reference)
+        if relative_path is None:
+            return None
+        descriptor, _ = self._open_read_descriptor(relative_path)
+        with os.fdopen(descriptor, "rb", closefd=True) as stream:
+            return stream.read()
+
+    def file_size(
+        self,
+        reference: str,
+        *,
+        expected_task_id: str,
+        expected_source_file_id: str,
+    ) -> int | None:
+        """返回受控 local 对象大小；旧 ``db://`` 对象没有原始文件。"""
+
+        relative_path = self._relative_from_reference(reference)
+        if relative_path is None:
+            return None
+        self._assert_expected_owner(
+            relative_path,
+            expected_task_id=expected_task_id,
+            expected_source_file_id=expected_source_file_id,
+        )
+        descriptor, info = self._open_read_descriptor(relative_path)
+        os.close(descriptor)
+        return info.st_size
+
+    def iter_bytes(
+        self,
+        reference: str,
+        *,
+        expected_task_id: str,
+        expected_source_file_id: str,
+        expected_size: int,
+        start: int = 0,
+        length: int | None = None,
+        chunk_size: int = 256 * 1024,
+    ) -> Iterator[bytes]:
+        """按范围流式读取原始文件，避免 PDF 预览把大文件整体载入内存。"""
+
+        relative_path = self._relative_from_reference(reference)
+        if relative_path is None:
+            raise DataProcessStorageError("original source object is not available")
+        self._assert_expected_owner(
+            relative_path,
+            expected_task_id=expected_task_id,
+            expected_source_file_id=expected_source_file_id,
+        )
+        if start < 0 or expected_size < 0 or chunk_size < 1:
+            raise DataProcessStorageError("invalid source byte range")
+        descriptor, info = self._open_read_descriptor(relative_path)
+        if info.st_size != expected_size:
+            os.close(descriptor)
+            raise DataProcessStorageError("source object size does not match metadata")
+        remaining = expected_size - start if length is None else length
+        if remaining < 0 or start + remaining > expected_size:
+            os.close(descriptor)
+            raise DataProcessStorageError("invalid source byte range")
+        with os.fdopen(descriptor, "rb", closefd=True) as stream:
+            stream.seek(start)
+            while remaining:
+                chunk = stream.read(min(chunk_size, remaining))
+                if not chunk:
+                    raise DataProcessStorageError("source object ended unexpectedly")
+                remaining -= len(chunk)
+                yield chunk
+
+    def validate_owner(
+        self,
+        reference: str,
+        *,
+        expected_task_id: str,
+        expected_source_file_id: str,
+    ) -> bool:
+        """校验 local 引用归属；旧 ``db://`` 引用无需文件系统处理。"""
+
+        relative_path = self._relative_from_reference(reference)
+        if relative_path is None:
+            return False
+        self._assert_expected_owner(
+            relative_path,
+            expected_task_id=expected_task_id,
+            expected_source_file_id=expected_source_file_id,
+        )
+        return True
+
+    def _open_read_descriptor(
+        self,
+        relative_path: PurePosixPath,
+    ) -> tuple[int, os.stat_result]:
+        path = self._path_for_relative(relative_path)
+        self._assert_controlled_parent(path)
+        try:
+            before_open = path.lstat()
+        except FileNotFoundError as exc:
+            raise DataProcessStorageError("source storage object does not exist") from exc
+        if stat.S_ISLNK(before_open.st_mode) or not stat.S_ISREG(before_open.st_mode):
+            raise DataProcessStorageError("source storage object is not a regular file")
+        flags = os.O_RDONLY
+        if hasattr(os, "O_NOFOLLOW"):
+            flags |= os.O_NOFOLLOW
+        descriptor = os.open(path, flags)
+        after_open = os.fstat(descriptor)
+        if (
+            not stat.S_ISREG(after_open.st_mode)
+            or before_open.st_dev != after_open.st_dev
+            or before_open.st_ino != after_open.st_ino
+        ):
+            os.close(descriptor)
+            raise DataProcessStorageError("source storage object changed while opening")
+        return descriptor, after_open
+
+    def delete(
+        self,
+        reference: str,
+        *,
+        expected_task_id: str | None = None,
+        expected_source_file_id: str | None = None,
+    ) -> bool:
+        """删除受控 local 对象；旧 ``db://`` 引用保持不变。"""
+
+        relative_path = self._relative_from_reference(reference)
+        if relative_path is None:
+            return False
+        if (expected_task_id is None) != (expected_source_file_id is None):
+            raise DataProcessStorageError("both expected storage owner fields are required")
+        if expected_task_id is not None and expected_source_file_id is not None:
+            self._assert_expected_owner(
+                relative_path,
+                expected_task_id=expected_task_id,
+                expected_source_file_id=expected_source_file_id,
+            )
+        path = self._path_for_relative(relative_path)
+        self._assert_controlled_parent(path)
+        try:
+            info = path.lstat()
+        except FileNotFoundError:
+            return False
+        if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
+            raise DataProcessStorageError("refusing to delete a non-regular storage object")
+        path.unlink()
+        self._fsync_directory(path.parent)
+        for directory in (path.parent, path.parent.parent, path.parent.parent.parent):
+            self._remove_empty_directory(directory)
+        return True
+
+    @staticmethod
+    def _assert_expected_owner(
+        relative_path: PurePosixPath,
+        *,
+        expected_task_id: str,
+        expected_source_file_id: str,
+    ) -> None:
+        task_id = _safe_component(expected_task_id, "expected task id")
+        source_file_id = _safe_component(
+            expected_source_file_id,
+            "expected source file id",
+        )
+        if relative_path.parts[:2] != (task_id, source_file_id):
+            raise DataProcessStorageError("source storage object owner mismatch")
+
+    def _relative_from_reference(self, reference: str) -> PurePosixPath | None:
+        if reference.startswith("db://"):
+            return None
+        parsed = urlsplit(reference)
+        if parsed.scheme != "local" or parsed.netloc != "data-process":
+            raise DataProcessStorageError("unsupported source storage reference")
+        if parsed.query or parsed.fragment or "\\" in parsed.path:
+            raise DataProcessStorageError("unsafe source storage reference")
+        raw_parts = parsed.path.lstrip("/").split("/")
+        if len(raw_parts) != 4:
+            raise DataProcessStorageError("unsafe source storage reference")
+        if any(re.search(r"%(?![0-9A-Fa-f]{2})", part) for part in raw_parts):
+            raise DataProcessStorageError("unsafe source storage reference")
+        try:
+            decoded = [unquote(part, encoding="utf-8", errors="strict") for part in raw_parts]
+        except UnicodeDecodeError as exc:
+            raise DataProcessStorageError("unsafe source storage reference") from exc
+        if any("/" in part or "\\" in part for part in decoded):
+            raise DataProcessStorageError("unsafe source storage reference")
+        canonical_parts = [
+            quote(decoded[0], safe="-_."),
+            quote(decoded[1], safe="-_."),
+            quote(decoded[2], safe="-_."),
+            quote(decoded[3], safe=""),
+        ]
+        if canonical_parts != raw_parts:
+            raise DataProcessStorageError("source storage reference is not canonical")
+        task_id = _safe_component(decoded[0], "task id")
+        source_file_id = _safe_component(decoded[1], "source file id")
+        version_component = decoded[2]
+        if not version_component.startswith("v") or not version_component[1:].isdigit():
+            raise DataProcessStorageError("invalid source file version")
+        version = int(version_component[1:])
+        if version < 1:
+            raise DataProcessStorageError("invalid source file version")
+        basename = _safe_basename(decoded[3])
+        return PurePosixPath(task_id, source_file_id, f"v{version}", basename)
+
+    def _path_for_relative(self, relative_path: PurePosixPath) -> Path:
+        if relative_path.is_absolute() or any(
+            part in {"", ".", ".."} for part in relative_path.parts
+        ):
+            raise DataProcessStorageError("storage path escapes the configured root")
+        path = self._root.joinpath(*relative_path.parts)
+        self._assert_controlled_parent(path)
+        return path
+
+    def _validate_staged_object(
+        self,
+        item: StagedSourceObject,
+        *,
+        require_file: bool,
+    ) -> None:
+        if not isinstance(item, StagedSourceObject):
+            raise DataProcessStorageError("invalid staged source object")
+        if self._issued_staged_objects.get(item._temporary_path) is not item:
+            raise DataProcessStorageError("staged source object was not issued by this storage")
+        expected_relative = self._relative_from_reference(item.reference)
+        if expected_relative is None or expected_relative != item._relative_path:
+            raise DataProcessStorageError("staged source object reference mismatch")
+        staging_root = self._root / ".staging"
+        try:
+            relative_temporary = item._temporary_path.relative_to(staging_root)
+        except ValueError as exc:
+            raise DataProcessStorageError("staged source object escapes staging") from exc
+        if len(relative_temporary.parts) != 2:
+            raise DataProcessStorageError("invalid staged source object path")
+        _safe_component(relative_temporary.parts[0], "batch id")
+        _safe_basename(relative_temporary.parts[1])
+        self._assert_controlled_parent(item._temporary_path)
+        try:
+            info = item._temporary_path.lstat()
+        except FileNotFoundError:
+            if require_file:
+                raise DataProcessStorageError("staged source object does not exist") from None
+            return
+        if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
+            raise DataProcessStorageError("staged source object is not a regular file")
+
+    def _ensure_directory(self, directory: Path) -> Path:
+        try:
+            relative = directory.relative_to(self._root)
+        except ValueError as exc:
+            raise DataProcessStorageError("storage path escapes the configured root") from exc
+        current = self._root
+        for component in relative.parts:
+            current = current / component
+            try:
+                current.mkdir(mode=0o700)
+            except FileExistsError:
+                pass
+            info = current.lstat()
+            if stat.S_ISLNK(info.st_mode) or not stat.S_ISDIR(info.st_mode):
+                raise DataProcessStorageError("storage path contains a symlink or non-directory")
+        return directory
+
+    def _assert_controlled_parent(self, path: Path) -> None:
+        try:
+            relative_parent = path.parent.relative_to(self._root)
+        except ValueError as exc:
+            raise DataProcessStorageError("storage path escapes the configured root") from exc
+        current = self._root
+        for component in relative_parent.parts:
+            current = current / component
+            if not current.exists():
+                continue
+            info = current.lstat()
+            if stat.S_ISLNK(info.st_mode) or not stat.S_ISDIR(info.st_mode):
+                raise DataProcessStorageError("storage path contains a symlink or non-directory")
+
+    @staticmethod
+    def _fsync_directory(directory: Path) -> None:
+        descriptor = os.open(directory, os.O_RDONLY)
+        try:
+            os.fsync(descriptor)
+        finally:
+            os.close(descriptor)
+
+    def _remove_empty_directory(self, directory: Path) -> None:
+        if directory in {self._root, self._root / ".staging"}:
+            return
+        self._assert_controlled_parent(directory / "placeholder")
+        try:
+            directory.rmdir()
+        except (FileNotFoundError, OSError):
+            return
+
+
+@lru_cache
+def get_data_process_storage() -> LocalDataProcessStorage:
+    return LocalDataProcessStorage()
+
+
+__all__ = [
+    "DataProcessStorageError",
+    "LocalDataProcessStorage",
+    "StagedSourceObject",
+    "get_data_process_storage",
+]

backend/app/modules/dataset/__init__.py

@@ -0,0 +1 @@
+"""Dataset management module."""

backend/app/modules/engine_registry/__init__.py

@@ -0,0 +1 @@
+"""Training engine registry module."""

backend/app/modules/eval/__init__.py

@@ -0,0 +1 @@
+"""Evaluation module."""

backend/app/modules/file_gateway/__init__.py

@@ -0,0 +1 @@
+"""Application-side file gateway module."""

backend/app/modules/fine_tune/__init__.py

@@ -0,0 +1 @@
+"""Fine-tuning task module."""

backend/app/modules/inference/__init__.py

@@ -0,0 +1 @@
+"""Inference and compare module."""

backend/app/modules/model/__init__.py

@@ -0,0 +1 @@
+"""Model registry module."""

backend/app/modules/project/__init__.py

@@ -0,0 +1 @@
+"""Project workspace and member module."""

backend/app/modules/project/router.py

@@ -0,0 +1,244 @@
+from __future__ import annotations
+
+from fastapi import APIRouter, Body, Depends, Request
+from typing import Any
+
+from app.api.v1.endpoints.platform import ok, fail
+from app.core.auth import filter_accessible_resource_ids, get_current_user, has_resource_access, is_admin
+from app.db.platform_store import get_platform_store
+
+router = APIRouter(prefix="/projects", tags=["project"])
+
+
+def _actor(request: Request) -> str | None:
+    auth = request.headers.get("Authorization", "")
+    token = auth.replace("Bearer ", "").strip()
+    return token or None
+
+
+def _require_no_pending_approval(resource_type: str, resource_id: str) -> None:
+    """第 4 周：写操作审批拦截——存在待审批实例时拒绝执行。"""
+    store = get_platform_store()
+    pending = [
+        i for i in store.approval_instances(status="pending")
+        if i["resource_type"] == resource_type and i["resource_id"] == resource_id
+    ]
+    if pending:
+        raise fail(409, "存在待审批的变更，请先完成审批")
+
+
+def _require_approval_or_admin(
+    resource_type: str,
+    resource_id: str,
+    current_user: dict[str, Any],
+    action_desc: str = "",
+) -> dict[str, Any] | None:
+    """高风险操作审批旁路：admin 直接放行，普通用户创建审批实例（code=202）。"""
+    if is_admin(current_user):
+        return None
+    store = get_platform_store()
+    instance = store.create_approval_instance({
+        "resource_type": resource_type,
+        "resource_id": resource_id,
+        "applicant_id": current_user.get("id"),
+        "template_id": None,
+    })
+    return {
+        "code": 202,
+        "message": f"操作已提交审批，等待管理员批准：{action_desc}",
+        "data": {"approval_required": True, "approval_id": instance["id"]},
+    }
+
+
+@router.get("")
+def list_projects(
+    tenant_id: str = "default",
+    status: str | None = None,
+    keyword: str | None = None,
+    current_user: dict = Depends(get_current_user),
+) -> dict[str, Any]:
+    store = get_platform_store()
+    projects = store.projects(tenant_id=tenant_id, status=status, keyword=keyword)
+    # #1 ACL 过滤：admin 直接放行，普通用户只能看到自己被授权的项目
+    accessible_ids = set(
+        filter_accessible_resource_ids("project", [p["id"] for p in projects], current_user)
+    )
+    filtered = [p for p in projects if p["id"] in accessible_ids]
+    return ok(filtered)
+
+
+@router.post("")
+def create_project(payload: dict[str, Any] = Body(...), request: Request = None) -> dict[str, Any]:
+    store = get_platform_store()
+    proj = store.create_project(payload)
+    store.record_audit(
+        action="project.create",
+        actor_id=_actor(request) if request else None,
+        target_type="project",
+        target_id=proj["id"],
+        tenant_id=proj.get("tenant_id"),
+        detail=f"name={proj.get('name')}",
+    )
+    return ok(proj)
+
+
+@router.get("/{project_id}")
+def get_project(project_id: str, current_user: dict = Depends(get_current_user)) -> dict[str, Any]:
+    # #2 访问控制：普通用户无 read 权限则拒绝
+    if not has_resource_access("project", project_id, current_user, "read"):
+        raise fail(403, "no permission to access this project")
+    try:
+        return ok(get_platform_store().project(project_id))
+    except KeyError:
+        raise fail(404, "project not found")
+
+
+@router.put("/{project_id}")
+def update_project(
+    project_id: str,
+    payload: dict[str, Any] = Body(...),
+    request: Request = None,
+    current_user: dict = Depends(get_current_user),
+) -> dict[str, Any]:
+    if not has_resource_access("project", project_id, current_user, "write"):
+        raise fail(403, "no permission to update this project")
+    store = get_platform_store()
+    try:
+        proj = store.update_project(project_id, payload)
+    except KeyError:
+        raise fail(404, "project not found")
+    store.record_audit(
+        action="project.update",
+        actor_id=_actor(request) if request else None,
+        target_type="project",
+        target_id=project_id,
+        tenant_id=proj.get("tenant_id"),
+        detail=f"fields={','.join(payload.keys())}",
+    )
+    return ok(proj)
+
+
+@router.post("/{project_id}/archive")
+def archive_project(
+    project_id: str,
+    request: Request = None,
+    current_user: dict = Depends(get_current_user),
+) -> dict[str, Any]:
+    _require_no_pending_approval("project", project_id)
+    pending = _require_approval_or_admin("project", project_id, current_user, f"归档项目 {project_id}")
+    if pending:
+        return pending
+    store = get_platform_store()
+    try:
+        proj = store.archive_project(project_id)
+    except KeyError:
+        raise fail(404, "project not found")
+    store.record_audit(
+        action="project.archive",
+        actor_id=_actor(request) if request else None,
+        target_type="project",
+        target_id=project_id,
+        tenant_id=proj.get("tenant_id"),
+    )
+    return ok(proj)
+
+
+@router.delete("/{project_id}")
+def delete_project(
+    project_id: str,
+    request: Request = None,
+    current_user: dict = Depends(get_current_user),
+) -> dict[str, Any]:
+    _require_no_pending_approval("project", project_id)
+    pending = _require_approval_or_admin("project", project_id, current_user, f"删除项目 {project_id}")
+    if pending:
+        return pending
+    store = get_platform_store()
+    store.delete_project(project_id)
+    store.record_audit(
+        action="project.delete",
+        actor_id=_actor(request) if request else None,
+        target_type="project",
+        target_id=project_id,
+    )
+    return ok(None)
+
+
+@router.get("/{project_id}/members")
+def list_members(project_id: str, current_user: dict = Depends(get_current_user)) -> dict[str, Any]:
+    if not has_resource_access("project", project_id, current_user, "read"):
+        raise fail(403, "no permission to access this project")
+    try:
+        return ok(get_platform_store().project_members(project_id))
+    except KeyError:
+        raise fail(404, "project not found")
+
+
+@router.post("/{project_id}/members")
+def add_member(
+    project_id: str,
+    payload: dict[str, Any] = Body(...),
+    request: Request = None,
+    current_user: dict = Depends(get_current_user),
+) -> dict[str, Any]:
+    if not has_resource_access("project", project_id, current_user, "write"):
+        raise fail(403, "no permission to manage members of this project")
+    store = get_platform_store()
+    try:
+        member = store.add_project_member(project_id, payload)
+    except KeyError:
+        raise fail(404, "project not found")
+    store.record_audit(
+        action="project.member.add",
+        actor_id=_actor(request) if request else None,
+        target_type="project.member",
+        target_id=project_id,
+        detail=f"user_id={payload.get('user_id')},role={payload.get('role')}",
+    )
+    return ok(member)
+
+
+@router.put("/{project_id}/members/{user_id}")
+def update_member(
+    project_id: str,
+    user_id: str,
+    payload: dict[str, Any] = Body(...),
+    request: Request = None,
+    current_user: dict = Depends(get_current_user),
+) -> dict[str, Any]:
+    if not has_resource_access("project", project_id, current_user, "write"):
+        raise fail(403, "no permission to manage members of this project")
+    store = get_platform_store()
+    try:
+        member = store.update_project_member_role(project_id, user_id, payload)
+    except KeyError:
+        raise fail(404, "project or member not found")
+    store.record_audit(
+        action="project.member.update",
+        actor_id=_actor(request) if request else None,
+        target_type="project.member",
+        target_id=project_id,
+        detail=f"user_id={user_id},role={payload.get('role')}",
+    )
+    return ok(member)
+
+
+@router.delete("/{project_id}/members/{user_id}")
+def remove_member(
+    project_id: str,
+    user_id: str,
+    request: Request = None,
+    current_user: dict = Depends(get_current_user),
+) -> dict[str, Any]:
+    if not has_resource_access("project", project_id, current_user, "write"):
+        raise fail(403, "no permission to manage members of this project")
+    store = get_platform_store()
+    store.remove_project_member(project_id, user_id)
+    store.record_audit(
+        action="project.member.remove",
+        actor_id=_actor(request) if request else None,
+        target_type="project.member",
+        target_id=project_id,
+        detail=f"user_id={user_id}",
+    )
+    return ok(None)

backend/app/modules/resource/__init__.py

@@ -0,0 +1 @@
+"""Resource access control list (ACL) module."""

backend/app/modules/resource/router.py

@@ -0,0 +1,41 @@
+from __future__ import annotations
+
+from fastapi import APIRouter, Body, Request
+from typing import Any
+
+from app.api.v1.endpoints.platform import ok, fail
+from app.db.platform_store import get_platform_store
+
+router = APIRouter(prefix="/resources", tags=["resource"])
+
+
+def _actor(request: Request) -> str | None:
+    auth = request.headers.get("Authorization", "")
+    token = auth.replace("Bearer ", "").strip()
+    return token or None
+
+
+@router.get("/{resource_type}/{resource_id}/acl")
+def get_acl(resource_type: str, resource_id: str) -> dict[str, Any]:
+    """查询资源 ACL，返回按主体分组的权限列表。"""
+    return ok(get_platform_store().resource_acl(resource_type, resource_id))
+
+
+@router.put("/{resource_type}/{resource_id}/acl")
+def set_acl(
+    resource_type: str,
+    resource_id: str,
+    payload: dict[str, Any] = Body(...),
+    request: Request = None,
+) -> dict[str, Any]:
+    """设置资源 ACL，body: { entries: [{ subject_type, subject_id, permissions: [] }] }"""
+    entries = payload.get("entries") or []
+    result = get_platform_store().set_resource_acl(resource_type, resource_id, entries)
+    get_platform_store().record_audit(
+        action="resource.acl.set",
+        actor_id=_actor(request) if request else None,
+        target_type=resource_type,
+        target_id=resource_id,
+        detail=f"entries={len(entries)}",
+    )
+    return ok(result)

backend/app/modules/retention/__init__.py

@@ -0,0 +1 @@
+"""Retention policy and cleanup module."""

backend/app/modules/retention/router.py

@@ -0,0 +1,75 @@
+from __future__ import annotations
+
+from fastapi import APIRouter, Body, Request
+from typing import Any
+
+from app.api.v1.endpoints.platform import ok, fail
+from app.db.platform_store import get_platform_store
+
+router = APIRouter(prefix="/retention-policies", tags=["retention"])
+
+
+def _actor(request: Request) -> str | None:
+    auth = request.headers.get("Authorization", "")
+    token = auth.replace("Bearer ", "").strip()
+    return token or None
+
+
+@router.get("")
+def list_policies() -> dict[str, Any]:
+    return ok(get_platform_store().retention_policies())
+
+
+@router.post("")
+def create_policy(payload: dict[str, Any] = Body(...), request: Request = None) -> dict[str, Any]:
+    if not payload.get("name"):
+        raise fail(400, "name 必填")
+    policy = get_platform_store().create_retention_policy(payload)
+    get_platform_store().record_audit(
+        action="retention.create",
+        actor_id=_actor(request) if request else None,
+        target_type="retention_policy",
+        target_id=policy["id"],
+        detail=f"name={policy.get('name')}",
+    )
+    return ok(policy)
+
+
+@router.get("/{policy_id}")
+def get_policy(policy_id: str) -> dict[str, Any]:
+    try:
+        return ok(get_platform_store().retention_policy(policy_id))
+    except KeyError:
+        raise fail(404, "retention policy not found")
+
+
+@router.put("/{policy_id}")
+def update_policy(
+    policy_id: str, payload: dict[str, Any] = Body(...), request: Request = None
+) -> dict[str, Any]:
+    store = get_platform_store()
+    try:
+        policy = store.update_retention_policy(policy_id, payload)
+    except KeyError:
+        raise fail(404, "retention policy not found")
+    store.record_audit(
+        action="retention.update",
+        actor_id=_actor(request) if request else None,
+        target_type="retention_policy",
+        target_id=policy_id,
+        detail=f"fields={','.join(payload.keys())}",
+    )
+    return ok(policy)
+
+
+@router.delete("/{policy_id}")
+def delete_policy(policy_id: str, request: Request = None) -> dict[str, Any]:
+    store = get_platform_store()
+    store.delete_retention_policy(policy_id)
+    store.record_audit(
+        action="retention.delete",
+        actor_id=_actor(request) if request else None,
+        target_type="retention_policy",
+        target_id=policy_id,
+    )
+    return ok({"deleted": policy_id})

backend/app/modules/system/__init__.py

@@ -0,0 +1 @@
+"""System health, metrics and logs module."""

backend/app/modules/system/router.py

@@ -0,0 +1,115 @@
+from __future__ import annotations
+
+from fastapi import APIRouter, Body, Query, Request
+from fastapi.responses import StreamingResponse
+
+from app.db.platform_store import ALL_PERMISSIONS, get_platform_store
+
+
+router = APIRouter(prefix="/system", tags=["system"])
+
+
+@router.post("/audit/visit")
+def record_visit(payload: dict = Body(...), request: Request = None) -> dict:
+    """记录用户访问业务模块的行为，用于看板用户操作分布统计。"""
+    action = str(payload.get("action") or payload.get("module") or "").strip()
+    if not action:
+        return {"code": 0, "message": "ok", "data": {"recorded": False}}
+    actor_id = ""
+    if request is not None:
+        auth = request.headers.get("Authorization", "")
+        token = auth.replace("Bearer ", "").strip()
+        if token.startswith("platform-token-"):
+            actor_id = token[len("platform-token-"):]
+    get_platform_store().record_audit(
+        action=action,
+        actor_id=actor_id or None,
+        target_type="module",
+        target_id=action,
+        detail=str(payload.get("detail") or ""),
+    )
+    return {"code": 0, "message": "ok", "data": {"recorded": True}}
+
+
+@router.get("/permissions/codes")
+def permission_codes() -> dict:
+    """返回平台权限码清单（权限码接口）。"""
+    return {"code": 0, "message": "ok", "data": {"codes": ALL_PERMISSIONS}}
+
+
+@router.get("/permissions")
+def permissions_overview() -> dict:
+    """返回权限码清单与角色定义。"""
+    store = get_platform_store()
+    return {
+        "code": 0,
+        "message": "ok",
+        "data": {"codes": ALL_PERMISSIONS, "roles": store.roles()},
+    }
+
+
+@router.get("/audit-logs")
+def audit_logs(
+    tenant_id: str | None = Query(default=None, description="租户 ID"),
+    project_id: str | None = Query(default=None, description="项目 ID"),
+    actor_id: str | None = Query(default=None, description="操作人 ID"),
+    action: str | None = Query(default=None, description="动作类型"),
+    target_type: str | None = Query(default=None, description="目标类型"),
+    start_time: str | None = Query(default=None, description="ISO8601 起始时间"),
+    end_time: str | None = Query(default=None, description="ISO8601 结束时间"),
+    limit: int = Query(default=50, ge=1, le=200),
+    offset: int = Query(default=0, ge=0),
+) -> dict:
+    """审计日志查询：按租户/项目/操作人/动作/目标类型/时间范围分页过滤。"""
+    store = get_platform_store()
+    result = store.audit_logs(
+        tenant_id=tenant_id,
+        project_id=project_id,
+        actor_id=actor_id,
+        action=action,
+        target_type=target_type,
+        start_time=start_time,
+        end_time=end_time,
+        limit=limit,
+        offset=offset,
+    )
+    return {"code": 0, "message": "ok", "data": result}
+
+
+@router.get("/audit-logs/export")
+def audit_logs_export(
+    tenant_id: str | None = Query(default=None, description="租户 ID"),
+    project_id: str | None = Query(default=None, description="项目 ID"),
+    actor_id: str | None = Query(default=None, description="操作人 ID"),
+    action: str | None = Query(default=None, description="动作类型"),
+    target_type: str | None = Query(default=None, description="目标类型"),
+    start_time: str | None = Query(default=None, description="ISO8601 起始时间"),
+    end_time: str | None = Query(default=None, description="ISO8601 结束时间"),
+) -> StreamingResponse:
+    """审计日志导出：返回 CSV 流，与应用查询相同的过滤条件。"""
+    store = get_platform_store()
+    result = store.audit_logs(
+        tenant_id=tenant_id,
+        project_id=project_id,
+        actor_id=actor_id,
+        action=action,
+        target_type=target_type,
+        start_time=start_time,
+        end_time=end_time,
+        limit=10000,
+        offset=0,
+    )
+    items = result["items"]
+    columns = ["time", "tenant_id", "project_id", "actor_id", "action", "target_type", "target_id", "detail", "client_ip"]
+    header = ",".join(columns) + "\n"
+
+    def iter_rows():
+        yield header
+        for row in items:
+            yield ",".join(f'"{str(row.get(c, "") or "")}"' for c in columns) + "\n"
+
+    return StreamingResponse(
+        iter_rows(),
+        media_type="text/csv",
+        headers={"Content-Disposition": "attachment; filename=audit_logs.csv"},
+    )

backend/app/modules/tenant/__init__.py

@@ -0,0 +1 @@
+"""Tenant management module."""

backend/app/modules/tenant/router.py

@@ -0,0 +1,116 @@
+from __future__ import annotations
+
+from fastapi import APIRouter, Body, Request
+from typing import Any
+
+from app.api.v1.endpoints.platform import ok, fail
+from app.db.platform_store import get_platform_store
+
+router = APIRouter(prefix="/tenants", tags=["tenant"])
+
+
+def _actor(request: Request) -> str | None:
+    auth = request.headers.get("Authorization", "")
+    token = auth.replace("Bearer ", "").strip()
+    return token or None
+
+
+@router.get("")
+def list_tenants() -> dict[str, Any]:
+    return ok(get_platform_store().tenants())
+
+
+@router.post("")
+def create_tenant(payload: dict[str, Any] = Body(...), request: Request = None) -> dict[str, Any]:
+    store = get_platform_store()
+    try:
+        tenant = store.create_tenant(payload)
+    except KeyError as e:
+        raise fail(400, f"missing field: {e}")
+    store.record_audit(
+        action="tenant.create",
+        actor_id=_actor(request) if request else None,
+        target_type="tenant",
+        target_id=tenant["id"],
+        tenant_id=tenant["id"],
+        detail=f"name={tenant.get('name')}",
+    )
+    return ok(tenant)
+
+
+@router.get("/{tenant_id}")
+def get_tenant(tenant_id: str) -> dict[str, Any]:
+    try:
+        return ok(get_platform_store().tenant(tenant_id))
+    except KeyError:
+        raise fail(404, "tenant not found")
+
+
+@router.put("/{tenant_id}")
+def update_tenant(tenant_id: str, payload: dict[str, Any] = Body(...), request: Request = None) -> dict[str, Any]:
+    store = get_platform_store()
+    try:
+        tenant = store.update_tenant(tenant_id, payload)
+    except KeyError:
+        raise fail(404, "tenant not found")
+    store.record_audit(
+        action="tenant.update",
+        actor_id=_actor(request) if request else None,
+        target_type="tenant",
+        target_id=tenant_id,
+        tenant_id=tenant_id,
+        detail=f"fields={','.join(payload.keys())}",
+    )
+    return ok(tenant)
+
+
+@router.put("/{tenant_id}/quota")
+def set_quota(tenant_id: str, payload: dict[str, Any] = Body(...), request: Request = None) -> dict[str, Any]:
+    store = get_platform_store()
+    try:
+        tenant = store.set_tenant_quota(tenant_id, payload.get("quota", {}))
+    except KeyError:
+        raise fail(404, "tenant not found")
+    store.record_audit(
+        action="tenant.quota.set",
+        actor_id=_actor(request) if request else None,
+        target_type="tenant",
+        target_id=tenant_id,
+        tenant_id=tenant_id,
+    )
+    return ok(tenant)
+
+
+@router.put("/{tenant_id}/retention-policy")
+def set_retention(tenant_id: str, payload: dict[str, Any] = Body(...), request: Request = None) -> dict[str, Any]:
+    store = get_platform_store()
+    try:
+        tenant = store.set_tenant_retention(tenant_id, payload.get("retention_policy_id"))
+    except KeyError:
+        raise fail(404, "tenant not found")
+    store.record_audit(
+        action="tenant.retention.set",
+        actor_id=_actor(request) if request else None,
+        target_type="tenant",
+        target_id=tenant_id,
+        tenant_id=tenant_id,
+    )
+    return ok(tenant)
+
+
+@router.delete("/{tenant_id}")
+def delete_tenant(tenant_id: str, request: Request = None) -> dict[str, Any]:
+    store = get_platform_store()
+    try:
+        tenant = store.delete_tenant(tenant_id)
+    except KeyError:
+        raise fail(404, "tenant not found")
+    store.record_audit(
+        action="tenant.delete",
+        actor_id=_actor(request) if request else None,
+        target_type="tenant",
+        target_id=tenant_id,
+        tenant_id=tenant_id,
+        detail=f"name={tenant.get('name')}",
+    )
+    return ok(tenant)

backend/app/schemas/__init__.py

@@ -0,0 +1 @@
+"""Shared schemas package."""

backend/app/schemas/data_process.py

@@ -0,0 +1,386 @@
+from __future__ import annotations
+
+from enum import StrEnum
+from typing import Any, Literal
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+from app.modules.data_process.constants import MAX_QA_PAIRS_PER_ITEM
+
+
+def _config_value(config: dict[str, Any], snake_name: str, camel_name: str, default: Any) -> Any:
+    if snake_name in config:
+        return config[snake_name]
+    return config.get(camel_name, default)
+
+
+def _validate_process_config(config: dict[str, Any]) -> None:
+    chunk_method = _config_value(config, "chunk_method", "chunkMethod", "layout_hybrid")
+    if not isinstance(chunk_method, str) or chunk_method not in {
+        "layout_hybrid",
+        "semantic",
+        "fixed",
+    }:
+        raise ValueError("chunk_method must be one of: layout_hybrid, semantic, fixed")
+
+    semantic_percentile = _config_value(
+        config,
+        "semantic_breakpoint_percentile",
+        "semanticBreakpointPercentile",
+        95,
+    )
+    if (
+        isinstance(semantic_percentile, bool)
+        or not isinstance(semantic_percentile, int)
+        or not 1 <= semantic_percentile <= 99
+    ):
+        raise ValueError("semantic_breakpoint_percentile must be an integer in [1, 99]")
+
+    split = _config_value(config, "dataset_split", "datasetSplit", None)
+    if split is not None:
+        if not isinstance(split, dict) or set(split) != {"train", "validation", "test"}:
+            raise ValueError("dataset_split must contain train, validation and test")
+        values = list(split.values())
+        if any(isinstance(value, bool) or not isinstance(value, int) for value in values):
+            raise ValueError("dataset_split values must be integers")
+        if any(value < 0 or value > 100 for value in values) or sum(values) != 100:
+            raise ValueError("dataset_split values must be in [0, 100] and total 100")
+
+    chunk_fields = {
+        "chunk_size",
+        "chunkSize",
+        "chunk_overlap",
+        "chunkOverlap",
+        "min_chunk_size",
+        "minChunkSize",
+    }
+    if chunk_fields.intersection(config):
+        chunk_size = _config_value(config, "chunk_size", "chunkSize", 800)
+        overlap = _config_value(config, "chunk_overlap", "chunkOverlap", 100)
+        minimum = _config_value(config, "min_chunk_size", "minChunkSize", 100)
+        if any(
+            isinstance(value, bool) or not isinstance(value, int)
+            for value in (chunk_size, overlap, minimum)
+        ):
+            raise ValueError("chunk_size, chunk_overlap and min_chunk_size must be integers")
+        if not 16 <= chunk_size <= 32_768:
+            raise ValueError("chunk_size must be in [16, 32768]")
+        if overlap < 0 or overlap >= chunk_size:
+            raise ValueError("chunk_overlap must be in [0, chunk_size)")
+        if minimum <= 0 or minimum > chunk_size or overlap + minimum > chunk_size:
+            raise ValueError("min_chunk_size and chunk_overlap exceed chunk_size")
+
+    temperature = _config_value(config, "temperature", "temperature", None)
+    if temperature is not None:
+        if isinstance(temperature, bool) or not isinstance(temperature, (int, float)):
+            raise ValueError("temperature must be a number")
+        if not 0 <= float(temperature) <= 2:
+            raise ValueError("temperature must be in [0, 2]")
+
+    max_tokens = _config_value(config, "max_tokens", "maxTokens", None)
+    if max_tokens is not None:
+        if isinstance(max_tokens, bool) or not isinstance(max_tokens, int):
+            raise ValueError("max_tokens must be an integer")
+        if not 1 <= max_tokens <= 32_768:
+            raise ValueError("max_tokens must be in [1, 32768]")
+
+    for snake_name, camel_name in (
+        ("qa_pairs_per_row", "qaPairsPerRow"),
+        ("qa_pairs_per_chunk", "qaPairsPerChunk"),
+    ):
+        pairs = _config_value(config, snake_name, camel_name, None)
+        if pairs is None:
+            continue
+        if (
+            isinstance(pairs, bool)
+            or not isinstance(pairs, int)
+            or not 1 <= pairs <= MAX_QA_PAIRS_PER_ITEM
+        ):
+            raise ValueError(
+                f"{snake_name} must be an integer in [1, {MAX_QA_PAIRS_PER_ITEM}]"
+            )
+
+
+class DataProcessStatus(StrEnum):
+    pending = "pending"
+    running = "running"
+    completed = "completed"
+    failed = "failed"
+    stopped = "stopped"
+
+
+class DataProcessWorkflowStep(StrEnum):
+    create = "create"
+    model = "model"
+    upload = "upload"
+    preview = "preview"
+    generate = "generate"
+    results = "results"
+
+
+class DataProcessPreviewStatus(StrEnum):
+    idle = "idle"
+    queued = "queued"
+    running = "running"
+    completed = "completed"
+    failed = "failed"
+    cancelled = "cancelled"
+
+
+class ProcessType(StrEnum):
+    structured = "structured"
+    unstructured = "unstructured"
+    external = "external"
+
+
+class DataProcessTaskCreate(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    name: str = Field(min_length=1, max_length=150)
+    description: str = ""
+    process_type: ProcessType
+    source_dataset_id: str | None = None
+    config: dict[str, Any] = Field(default_factory=dict)
+
+    @field_validator("name")
+    @classmethod
+    def normalize_name(cls, value: str) -> str:
+        value = value.strip()
+        if not value:
+            raise ValueError("task name cannot be empty")
+        return value
+
+    @model_validator(mode="after")
+    def validate_config(self) -> "DataProcessTaskCreate":
+        _validate_process_config(self.config)
+        return self
+
+
+class DataProcessTaskUpdate(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    name: str | None = Field(default=None, min_length=1, max_length=150)
+    description: str | None = None
+    process_type: ProcessType | None = None
+    source_dataset_id: str | None = None
+    config: dict[str, Any] | None = None
+
+    @field_validator("name")
+    @classmethod
+    def normalize_name(cls, value: str | None) -> str | None:
+        if value is None:
+            return None
+        value = value.strip()
+        if not value:
+            raise ValueError("task name cannot be empty")
+        return value
+
+    @model_validator(mode="after")
+    def validate_config(self) -> "DataProcessTaskUpdate":
+        if self.config is not None:
+            _validate_process_config(self.config)
+        return self
+
+
+class DataProcessWorkflowStepUpdate(BaseModel):
+    """仅保存创建向导位置，不修改配置或使下游产物失效。"""
+
+    model_config = ConfigDict(extra="forbid")
+
+    workflow_step: DataProcessWorkflowStep
+
+
+class DataProcessRegenerateRequest(BaseModel):
+    """以一份完整配置准备任务重新生成。
+
+    ``expected_updated_at`` 用于防止详情页的旧快照覆盖其他人刚刚
+    保存的配置。重新生成不允许改变处理类型，避免旧源文件在新解析
+    规则下被静默误用。
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    name: str = Field(min_length=1, max_length=150)
+    description: str
+    process_type: ProcessType
+    config: dict[str, Any]
+    expected_updated_at: str = Field(min_length=1)
+
+    @field_validator("name")
+    @classmethod
+    def normalize_name(cls, value: str) -> str:
+        value = value.strip()
+        if not value:
+            raise ValueError("task name cannot be empty")
+        return value
+
+    @model_validator(mode="after")
+    def validate_config(self) -> "DataProcessRegenerateRequest":
+        _validate_process_config(self.config)
+        return self
+
+
+class DataProcessRepeatRequest(BaseModel):
+    """按已确认任务的完整快照创建一批独立的新生成结果。"""
+
+    model_config = ConfigDict(extra="forbid")
+
+    expected_updated_at: str = Field(min_length=1)
+    request_id: str = Field(
+        min_length=8,
+        max_length=80,
+        pattern=r"^[A-Za-z0-9_-]+$",
+    )
+
+
+class PreviewBuildRequest(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    replace_existing: Literal[True] = True
+    source_file_ids: list[str] | None = None
+    source_file_id: str | None = None
+
+    @model_validator(mode="after")
+    def validate_source_file_selection(self) -> "PreviewBuildRequest":
+        if self.source_file_ids is not None and self.source_file_id is not None:
+            raise ValueError("source_file_id and source_file_ids cannot be used together")
+        values = self.source_file_ids
+        if values is None and self.source_file_id is not None:
+            values = [self.source_file_id]
+        if values is None:
+            return self
+        normalized = list(dict.fromkeys(str(value).strip() for value in values))
+        if not normalized or any(not value for value in normalized):
+            raise ValueError("at least one non-empty source file id is required")
+        self.source_file_ids = normalized
+        self.source_file_id = None
+        return self
+
+
+class PreviewItemCreate(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    source_file_id: str | None = None
+    original_content: str = ""
+    edited_content: str = ""
+    source_start: int | None = Field(default=None, ge=0)
+    source_end: int | None = Field(default=None, ge=0)
+    source_start_line: int | None = Field(default=None, ge=1)
+    source_end_line: int | None = Field(default=None, ge=1)
+
+    @model_validator(mode="after")
+    def validate_ranges(self) -> "PreviewItemCreate":
+        if self.source_start is not None and self.source_end is not None:
+            if self.source_end < self.source_start:
+                raise ValueError("source_end must be greater than or equal to source_start")
+        if self.source_start_line is not None and self.source_end_line is not None:
+            if self.source_end_line < self.source_start_line:
+                raise ValueError(
+                    "source_end_line must be greater than or equal to source_start_line"
+                )
+        return self
+
+
+class PreviewItemUpdate(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    edited_content: str
+    expected_updated_at: str | None = None
+
+
+class GenerateRequest(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    replace_existing: Literal[True] = True
+
+
+class ExternalSourceRequest(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    type: str = Field(min_length=1, max_length=30)
+    url: str = Field(min_length=1, max_length=2048)
+    auth_mode: Literal["none", "basic"] = "none"
+    username: str | None = Field(default=None, max_length=150)
+    password: str | None = Field(default=None, max_length=500)
+    limit: int = Field(default=1000, ge=1, le=100_000)
+
+
+class ExternalPullRequest(ExternalSourceRequest):
+    query: str | None = Field(default=None, max_length=20_000)
+    file_name: str = Field(default="external-data.jsonl", min_length=1, max_length=255)
+
+    @field_validator("file_name")
+    @classmethod
+    def validate_file_name(cls, value: str) -> str:
+        name = value.strip()
+        if not name.lower().endswith((".jsonl", ".ndjson")):
+            raise ValueError("external pull file_name must end with .jsonl or .ndjson")
+        return name
+
+
+class ResultUpdate(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    instruction: str | None = None
+    input: str | None = None
+    output: str | None = None
+    expected_updated_at: str | None = None
+
+
+class ResultRegenerateRequest(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    expected_updated_at: str = Field(min_length=1, max_length=100)
+
+
+class ResultBatchRegenerateItem(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    result_id: str = Field(min_length=1, max_length=100)
+    expected_updated_at: str = Field(min_length=1, max_length=100)
+
+
+class ResultBatchRegenerateRequest(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    items: list[ResultBatchRegenerateItem] = Field(min_length=1, max_length=100)
+
+    @model_validator(mode="after")
+    def validate_unique_results(self) -> "ResultBatchRegenerateRequest":
+        result_ids = [item.result_id for item in self.items]
+        if len(result_ids) != len(set(result_ids)):
+            raise ValueError("result_id values must be unique")
+        return self
+
+
+class DatasetSplit(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    train: int = Field(default=80, ge=0, le=100)
+    validation: int = Field(default=10, ge=0, le=100)
+    test: int = Field(default=10, ge=0, le=100)
+
+    @model_validator(mode="after")
+    def validate_total(self) -> "DatasetSplit":
+        if self.train + self.validation + self.test != 100:
+            raise ValueError("dataset split must total 100")
+        return self
+
+
+class PublishRequest(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+    dataset_name: str = Field(min_length=1, max_length=150)
+    dataset_type: Literal["train", "test", "eval", "val", "other"] = "train"
+    storage_type: Literal["local"] = "local"
+    split: DatasetSplit = Field(default_factory=DatasetSplit)
+    format: Literal["alpaca_jsonl", "jsonl"] = "alpaca_jsonl"
+    description: str = ""
+
+    @field_validator("dataset_name")
+    @classmethod
+    def normalize_dataset_name(cls, value: str) -> str:
+        value = value.strip()
+        if not value:
+            raise ValueError("dataset name cannot be empty")
+        return value

backend/app/services/__init__.py

@@ -0,0 +1 @@
+"""Cross-module services package."""

backend/app/workers/__init__.py

@@ -0,0 +1 @@
+"""Background workers package."""

backend/app/workers/compute_poller.py

@@ -0,0 +1,31 @@
+from __future__ import annotations
+
+import asyncio
+
+from app.core.config import get_settings
+from app.core.logging import get_logger
+from app.modules.compute_gateway.sync import poll_compute_jobs_once
+
+
+logger = get_logger(__name__)
+
+
+async def run_compute_poller() -> None:
+    settings = get_settings()
+    if settings.compute_mode == "simulator" or settings.compute_status_sync_mode != "polling":
+        logger.info("compute poller disabled", extra={"compute_mode": settings.compute_mode})
+        return
+
+    interval = max(3, settings.compute_poll_interval_seconds)
+    logger.info("compute poller started", extra={"interval_seconds": interval})
+    while True:
+        try:
+            result = await poll_compute_jobs_once()
+            if result["synced"] or result["failed"]:
+                logger.info("compute jobs polled", extra={"result": result})
+        except asyncio.CancelledError:
+            logger.info("compute poller stopped")
+            raise
+        except Exception as exc:  # noqa: BLE001 - keep background polling alive
+            logger.exception("compute poller failed", extra={"error": str(exc)})
+        await asyncio.sleep(interval)

backend/pyproject.toml

@@ -0,0 +1,41 @@
+[project]
+name = "yg-ft-backend"
+version = "0.1.0"
+description = "Backend service for the model fine-tuning platform"
+requires-python = ">=3.12"
+dependencies = [
+  "fastapi>=0.111.0",
+  "uvicorn[standard]>=0.30.0",
+  "python-multipart>=0.0.9",
+  "pydantic>=2.7.0",
+  "sqlalchemy>=2.0.30",
+  "psycopg[binary]>=3.2.1",
+  "psycopg-pool>=3.2.1",
+  "alembic>=1.13.1",
+  "redis>=5.0.4",
+  "httpx>=0.27.0",
+  "PyJWT>=2.8.0",
+  "passlib[bcrypt]>=1.7.4",
+  "python-dotenv>=1.0.1",
+  "pypdf[crypto]>=5.0.0",
+  "python-docx>=1.1.2",
+  "openpyxl>=3.1.5",
+  "python-pptx>=1.0.2",
+  "llama-index-core==0.14.23",
+  "llama-index-embeddings-huggingface==0.6.1",
+  "docling==2.115.0",
+  "tiktoken>=0.7.0",
+]
+
+[project.optional-dependencies]
+dev = [
+  "pytest>=8.2.0",
+  "ruff>=0.5.0",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py312"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

backend/requirements.txt

@@ -0,0 +1,25 @@
+fastapi>=0.111.0
+uvicorn[standard]>=0.30.0
+python-multipart>=0.0.9
+pydantic>=2.7.0
+sqlalchemy>=2.0.30
+psycopg[binary]>=3.2.1
+psycopg-pool>=3.2.1
+alembic>=1.13.1
+redis>=5.0.4
+httpx>=0.27.0
+PyJWT>=2.8.0
+passlib[bcrypt]>=1.7.4
+python-dotenv>=1.0.1
+pypdf[crypto]>=5.0.0
+python-docx>=1.1.2
+openpyxl>=3.1.5
+python-pptx>=1.0.2
+llama-index-core==0.14.23
+llama-index-embeddings-huggingface==0.6.1
+docling==2.115.0
+tiktoken>=0.7.0
+
+# 测试与代码检查
+pytest>=8.2.0
+ruff>=0.5.0

backend/tests/test_compare_inference_async.py

@@ -0,0 +1,276 @@
+"""
+模型推理异步加载改造的单元测试。
+
+覆盖：
+- model_compare_load：异步派发，立即返回 starting + 节点信息（不等待加载完成）
+- model_compare_delete：先删记录，卸载失败也不阻塞删除
+- reconcile_inference_loads：starting -> ready/error/idle/不可达的状态迁移与封顶
+- _unload_from_compute_node：任务感知，只命中记录中的节点
+"""
+
+from __future__ import annotations
+
+import asyncio
+from types import SimpleNamespace
+from typing import Any
+
+from app.api.v1.endpoints.platform import model_compare_delete, model_compare_load
+import app.api.v1.endpoints.platform as platform
+from app.modules.compute_gateway.client import ComputeNodeClient
+from app.modules.compute_gateway.sync import MAX_STARTING_ATTEMPTS, reconcile_inference_loads
+
+
+class FakeInferenceStore:
+    """内存 store，仅实现推理加载/对账用到的接口。"""
+
+    def __init__(self, tasks: list[dict[str, Any]] | None = None, nodes: list[dict[str, Any]] | None = None) -> None:
+        self._tasks: dict[str, dict[str, Any]] = {t["id"]: dict(t) for t in (tasks or [])}
+        self._nodes = nodes or []
+        self._inference_nodes: set[str] = set()
+
+    def compare_task(self, task_id: str) -> dict[str, Any]:
+        if task_id not in self._tasks:
+            raise KeyError(task_id)
+        return dict(self._tasks[task_id])
+
+    def compare_tasks(self) -> list[dict[str, Any]]:
+        return [dict(t) for t in self._tasks.values()]
+
+    def update_compare_task(self, task_id: str, payload: dict[str, Any]) -> dict[str, Any]:
+        current = self._tasks[task_id]
+        merged = {**current, **payload, "id": task_id}
+        self._tasks[task_id] = merged
+        return dict(merged)
+
+    def delete_compare_task(self, task_id: str) -> None:
+        self._tasks.pop(task_id, None)
+
+    def compute_nodes(self) -> list[dict[str, Any]]:
+        return [dict(n) for n in self._nodes]
+
+    def model(self, model_id: str) -> dict[str, Any]:
+        raise KeyError(model_id)
+
+    def trained_models(self) -> list[dict[str, Any]]:
+        return []
+
+    def mark_inference_loaded(self, node_id: str) -> None:
+        self._inference_nodes.add(node_id)
+
+    def mark_inference_unloaded(self, node_id: str) -> None:
+        self._inference_nodes.discard(node_id)
+
+    def is_inference_loaded(self, node_id: str) -> bool:
+        return node_id in self._inference_nodes
+
+
+def _node(node_id: str, code: str = "") -> dict[str, Any]:
+    return {
+        "id": node_id,
+        "code": code or node_id,
+        "name": code or node_id,
+        "api_base_url": f"http://{code or node_id}:19100",
+        "enabled": True,
+        "scheduler_status": "online",
+    }
+
+
+def _task(task_id: str, *, node_id: str | None = None, load_status: dict[str, Any] | None = None) -> dict[str, Any]:
+    return {
+        "id": task_id,
+        "name": f"task-{task_id}",
+        "status": "pending",
+        "models": [
+            {"model_id": "m_1", "model_name": "qwen", "model_path": "/models/qwen", "node_id": node_id}
+        ],
+        "load_status": load_status or {"loaded_models": []},
+    }
+
+
+async def _fake_inference_load(self, payload: dict[str, Any]) -> dict[str, Any]:
+    return {"loaded": False, "status": "loading", "request_id": "req-1"}
+
+
+async def _fake_inference_unload(self) -> dict[str, Any]:
+    return {"unloaded": True, "status": "idle"}
+
+
+def _patch_store(monkeypatch, store: FakeInferenceStore) -> None:
+    monkeypatch.setattr(platform, "get_platform_store", lambda: store)
+    monkeypatch.setattr(platform, "get_settings", lambda: SimpleNamespace(compute_mode="real"))
+
+
+def test_select_eval_node_prefers_model_node(monkeypatch) -> None:
+    from app.api.v1.endpoints.platform import _select_eval_node
+
+    store = FakeInferenceStore(nodes=[_node("n1"), _node("n2")])
+    # 指定模型所在节点时优先返回该节点
+    assert _select_eval_node(store, "n2")["id"] == "n2"
+    # 无指定节点时回退到第一个在线节点
+    assert _select_eval_node(store, None)["id"] == "n1"
+
+
+def test_select_eval_node_returns_none_when_model_node_offline(monkeypatch) -> None:
+    from app.api.v1.endpoints.platform import _select_eval_node
+
+    nodes = [_node("n1"), _node("n2")]
+    nodes[1]["enabled"] = False
+    store = FakeInferenceStore(nodes=nodes)
+    # 模型所在节点不可用 → 明确失败，不派发到其它节点
+    assert _select_eval_node(store, "n2") is None
+    # 无指定节点时仍回退第一个在线节点
+    assert _select_eval_node(store, None)["id"] == "n1"
+
+
+def test_model_compare_load_dispatches_and_returns_starting(monkeypatch) -> None:
+    store = FakeInferenceStore(tasks=[_task("t1", node_id="n1")], nodes=[_node("n1")])
+    _patch_store(monkeypatch, store)
+    monkeypatch.setattr(ComputeNodeClient, "inference_load", _fake_inference_load)
+
+    result = asyncio.run(model_compare_load("t1"))
+    assert result["code"] == 0
+    updated = result["data"]
+    assert updated["status"] == "starting"
+    items = updated["load_status"]["loaded_models"]
+    assert items[0]["status"] == "starting"
+    assert items[0]["node_id"] == "n1"
+    assert "n1" in store._inference_nodes
+
+
+def test_model_compare_load_marks_error_when_all_nodes_fail(monkeypatch) -> None:
+    store = FakeInferenceStore(tasks=[_task("t1", node_id="n1")], nodes=[_node("n1")])
+    _patch_store(monkeypatch, store)
+
+    async def _raise(self, payload: dict[str, Any]) -> dict[str, Any]:
+        raise RuntimeError("conn refused")
+
+    monkeypatch.setattr(ComputeNodeClient, "inference_load", _raise)
+
+    result = asyncio.run(model_compare_load("t1"))
+    updated = result["data"]
+    assert updated["status"] == "failed"
+    assert updated["load_status"]["loaded_models"][0]["status"] == "error"
+    assert "conn refused" in updated["load_status"]["loaded_models"][0]["error"]
+
+
+def test_model_compare_delete_removes_record_even_if_unload_raises(monkeypatch) -> None:
+    task = _task(
+        "t1",
+        node_id="n1",
+        load_status={"loaded_models": [{"model_id": "m_1", "status": "ready", "node_id": "n1"}]},
+    )
+    store = FakeInferenceStore(tasks=[task], nodes=[_node("n1")])
+    _patch_store(monkeypatch, store)
+
+    async def _raise(self) -> dict[str, Any]:
+        raise RuntimeError("unload boom")
+
+    monkeypatch.setattr(ComputeNodeClient, "inference_unload", _raise)
+
+    result = asyncio.run(model_compare_delete("t1"))
+    assert result["data"] == {"deleted": "t1"}
+    assert "t1" not in store._tasks
+    # finally 中仍清掉了节点标记
+    assert "n1" not in store._inference_nodes
+
+
+def test_unload_from_compute_node_only_hits_recorded_node(monkeypatch) -> None:
+    task = _task(
+        "t1",
+        load_status={"loaded_models": [{"model_id": "m_1", "status": "ready", "node_id": "n1"}]},
+    )
+    store = FakeInferenceStore(tasks=[task], nodes=[_node("n1"), _node("n2")])
+    _patch_store(monkeypatch, store)
+    monkeypatch.setattr(ComputeNodeClient, "inference_unload", _fake_inference_unload)
+
+    from app.api.v1.endpoints.platform import _unload_from_compute_node
+
+    result = asyncio.run(_unload_from_compute_node(store, task=task))
+    assert result["unloaded"] is True
+    # 只命中任务记录中的节点 n1，n2 未被卸载
+    assert [r["node_id"] for r in result["nodes"]] == ["n1"]
+    assert "n1" not in store._inference_nodes
+
+
+async def _status_ready(self) -> dict[str, Any]:
+    return {"loaded": True, "status": "ready", "model_name": "qwen"}
+
+
+def test_reconcile_transitions_starting_to_ready(monkeypatch) -> None:
+    task = _task(
+        "t1",
+        node_id="n1",
+        load_status={"loaded_models": [{"model_id": "m_1", "status": "starting", "node_id": "n1"}]},
+    )
+    store = FakeInferenceStore(tasks=[task], nodes=[_node("n1")])
+    monkeypatch.setattr(ComputeNodeClient, "inference_status", _status_ready)
+
+    reconciled = asyncio.run(reconcile_inference_loads(store))
+    assert reconciled == [{"task_id": "t1", "status": "loaded"}]
+    updated = store._tasks["t1"]
+    assert updated["status"] == "loaded"
+    assert updated["load_status"]["loaded_models"][0]["status"] == "ready"
+    assert "n1" in store._inference_nodes
+
+
+def test_reconcile_transitions_to_error_and_failed(monkeypatch) -> None:
+    async def _status_error(self) -> dict[str, Any]:
+        return {"loaded": False, "status": "error", "error": "CUDA out of memory"}
+
+    task = _task(
+        "t1",
+        node_id="n1",
+        load_status={"loaded_models": [{"model_id": "m_1", "status": "starting", "node_id": "n1"}]},
+    )
+    store = FakeInferenceStore(tasks=[task], nodes=[_node("n1")])
+    monkeypatch.setattr(ComputeNodeClient, "inference_status", _status_error)
+
+    reconciled = asyncio.run(reconcile_inference_loads(store))
+    assert reconciled == [{"task_id": "t1", "status": "failed"}]
+    item = store._tasks["t1"]["load_status"]["loaded_models"][0]
+    assert item["status"] == "error"
+    assert "CUDA out of memory" in item["error"]
+    assert "n1" not in store._inference_nodes
+
+
+def test_reconcile_idle_marks_model_disappeared(monkeypatch) -> None:
+    async def _status_idle(self) -> dict[str, Any]:
+        return {"loaded": False, "status": "idle"}
+
+    task = _task(
+        "t1",
+        node_id="n1",
+        load_status={"loaded_models": [{"model_id": "m_1", "status": "starting", "node_id": "n1"}]},
+    )
+    store = FakeInferenceStore(tasks=[task], nodes=[_node("n1")])
+    monkeypatch.setattr(ComputeNodeClient, "inference_status", _status_idle)
+
+    asyncio.run(reconcile_inference_loads(store))
+    item = store._tasks["t1"]["load_status"]["loaded_models"][0]
+    assert item["status"] == "error"
+    assert "disappeared" in item["error"]
+    assert store._tasks["t1"]["status"] == "failed"
+
+
+def test_reconcile_unreachable_node_flips_to_error_after_cap(monkeypatch) -> None:
+    async def _raise(self) -> dict[str, Any]:
+        raise RuntimeError("conn refused")
+
+    task = _task(
+        "t1",
+        node_id="n1",
+        load_status={"loaded_models": [{"model_id": "m_1", "status": "starting", "node_id": "n1"}]},
+    )
+    store = FakeInferenceStore(tasks=[task], nodes=[_node("n1")])
+    monkeypatch.setattr(ComputeNodeClient, "inference_status", _raise)
+
+    # 每次轮询前重置节流时间戳，逐次推进 load_attempts 到封顶
+    for _ in range(MAX_STARTING_ATTEMPTS):
+        item = store._tasks["t1"]["load_status"]["loaded_models"][0]
+        item["last_polled_at"] = 0
+        asyncio.run(reconcile_inference_loads(store))
+
+    item = store._tasks["t1"]["load_status"]["loaded_models"][0]
+    assert item["status"] == "error"
+    assert "unreachable" in item["error"]
+    assert store._tasks["t1"]["status"] == "failed"

backend/tests/test_data_process_generation.py

@@ -0,0 +1,897 @@
+from __future__ import annotations
+
+import json
+
+import httpx
+import pytest
+
+from app.modules.data_process.generation import (
+    ModelGenerationError,
+    chat_completions_url,
+    generate_model_records,
+)
+
+
+def test_chat_completions_url_accepts_host_base_and_complete_url() -> None:
+    assert chat_completions_url("www.caoxiaozhu.com") == (
+        "https://www.caoxiaozhu.com/v1/chat/completions"
+    )
+    assert chat_completions_url("https://model.example/v1") == (
+        "https://model.example/v1/chat/completions"
+    )
+    complete = "https://model.example/openai/v1/chat/completions"
+    assert chat_completions_url(complete) == complete
+
+
+def test_generate_model_records_uses_prompt_auth_and_stable_split() -> None:
+    requests: list[httpx.Request] = []
+    progress_updates: list[tuple[int, int]] = []
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        requests.append(request)
+        payload = json.loads(request.content)
+        assert payload["model"] == "qwen-plus"
+        assert payload["response_format"] == {"type": "json_object"}
+        assert "客户反馈页面加载慢" in payload["messages"][1]["content"]
+        assert "你正在生成标准监督微调问答数据" in payload["messages"][0]["content"]
+        assert "禁止输出分析、推理过程" in payload["messages"][0]["content"]
+        return httpx.Response(
+            200,
+            json={
+                "choices": [
+                    {
+                        "message": {
+                            "content": json.dumps(
+                                {
+                                    "items": [
+                                        {
+                                            "instruction": "请生成简洁客服回复",
+                                            "input": "客户反馈页面加载慢",
+                                            "output": "已收到反馈，我们正在排查。",
+                                        }
+                                    ]
+                                },
+                                ensure_ascii=False,
+                            )
+                        }
+                    }
+                ]
+            },
+        )
+
+    client = httpx.Client(transport=httpx.MockTransport(handler))
+    records = generate_model_records(
+        [{"id": "preview-1", "edited_content": "客户反馈页面加载慢"}],
+        model={
+            "name": "Qwen",
+            "online_model_name": "qwen-plus",
+            "api_url": "model.example",
+            "api_key": "test-secret",
+        },
+        config={
+            "generation_prompt": "请处理：{{ content }}",
+            "json_mode": True,
+            "temperature": 0.2,
+            "max_tokens": 512,
+        },
+        task_id="task-1",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=client,
+        on_progress=lambda processed, total: progress_updates.append((processed, total)),
+    )
+
+    assert len(records) == 1
+    assert records[0]["status"] == "valid"
+    assert records[0]["split"] == "train"
+    assert requests[0].headers["Authorization"] == "Bearer test-secret"
+    assert progress_updates == [(1, 1)]
+
+
+def test_minimax_m3_uses_split_reasoning_and_completion_token_budget() -> None:
+    requests: list[dict[str, object]] = []
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        payload = json.loads(request.content)
+        requests.append(payload)
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "finish_reason": "stop",
+                    "message": {
+                        "reasoning_content": "模型内部思考不应混入业务 JSON",
+                        "content": json.dumps({
+                            "items": [{
+                                "instruction": "申请编号有什么作用？",
+                                "reasoning": "来源说明它用于标识报销申请。",
+                                "answer": "它用于唯一标识一笔报销申请。",
+                            }],
+                        }, ensure_ascii=False),
+                    },
+                }],
+                "output_sensitive": False,
+                "base_resp": {"status_code": 0, "status_msg": ""},
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-minimax", "edited_content": "申请编号用于标识报销申请。"}],
+        model={
+            "name": "MiniMax",
+            "online_model_name": "MiniMax-M3",
+            "api_url": "https://api.minimaxi.com/v1",
+        },
+        config={
+            "output_type": "reasoning",
+            "json_mode": True,
+            "max_tokens": 1024,
+            "generation_retries": 0,
+        },
+        task_id="task-minimax",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert records[0]["status"] == "valid"
+    assert len(requests) == 1
+    assert requests[0]["reasoning_split"] is True
+    assert requests[0]["max_completion_tokens"] >= 4096
+    assert "max_tokens" not in requests[0]
+    assert "response_format" not in requests[0]
+
+
+def test_minimax_m3_keeps_larger_configured_completion_budget() -> None:
+    requests: list[dict[str, object]] = []
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        requests.append(json.loads(request.content))
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "message": {
+                        "content": json.dumps({
+                            "items": [{
+                                "instruction": "问题",
+                                "output": "这是满足测试要求的完整答案。",
+                            }],
+                        }, ensure_ascii=False),
+                    },
+                }],
+            },
+        )
+
+    generate_model_records(
+        [{"id": "preview-minimax-budget", "edited_content": "来源正文"}],
+        model={
+            "online_model_name": "MiniMax-M3",
+            "api_url": "https://api.minimax.io/v1",
+        },
+        config={"max_tokens": 8192, "generation_retries": 0},
+        task_id="task-minimax-budget",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert requests[0]["max_completion_tokens"] == 8192
+
+
+def test_minimax_m3_name_on_custom_proxy_keeps_generic_openai_parameters() -> None:
+    requests: list[dict[str, object]] = []
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        requests.append(json.loads(request.content))
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "message": {
+                        "content": json.dumps({
+                            "items": [{
+                                "instruction": "问题",
+                                "output": "这是代理服务返回的完整答案。",
+                            }],
+                        }, ensure_ascii=False),
+                    },
+                }],
+            },
+        )
+
+    generate_model_records(
+        [{"id": "preview-minimax-proxy", "edited_content": "来源正文"}],
+        model={
+            "online_model_name": "MiniMax-M3",
+            "api_url": "https://model-proxy.example/v1",
+        },
+        config={"max_tokens": 1024, "json_mode": True, "generation_retries": 0},
+        task_id="task-minimax-proxy",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert requests[0]["max_tokens"] == 1024
+    assert requests[0]["response_format"] == {"type": "json_object"}
+    assert "reasoning_split" not in requests[0]
+    assert "max_completion_tokens" not in requests[0]
+
+
+def test_generate_model_records_extracts_json_surrounded_by_model_explanation() -> None:
+    content = "模型结果如下：\n```json\n" + json.dumps(
+        {
+            "items": [{
+                "instruction": "字段有什么作用？",
+                "output": "该字段用于唯一标识记录。",
+            }],
+        },
+        ensure_ascii=False,
+    ) + "\n```\n生成完毕。"
+    client = httpx.Client(
+        transport=httpx.MockTransport(
+            lambda _: httpx.Response(
+                200,
+                json={"choices": [{"message": {"content": content}}]},
+            )
+        )
+    )
+
+    records = generate_model_records(
+        [{"id": "preview-explanation", "edited_content": "字段用于唯一标识记录。"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 0},
+        task_id="task-explanation",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=client,
+    )
+
+    assert records[0]["status"] == "valid"
+    assert records[0]["output"] == "该字段用于唯一标识记录。"
+
+
+def test_generate_model_records_reports_token_truncation_instead_of_json_error() -> None:
+    client = httpx.Client(
+        transport=httpx.MockTransport(
+            lambda _: httpx.Response(
+                200,
+                json={
+                    "choices": [{
+                        "finish_reason": "length",
+                        "message": {"content": ""},
+                    }],
+                    "output_sensitive": False,
+                },
+            )
+        )
+    )
+
+    records = generate_model_records(
+        [{"id": "preview-truncated", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 0},
+        task_id="task-truncated",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=client,
+    )
+
+    assert records[0]["status"] == "invalid"
+    assert "Token" in records[0]["error"]
+    assert "截断" in records[0]["error"]
+
+
+def test_token_truncation_is_not_retried_even_when_json_looks_complete() -> None:
+    request_count = 0
+    content = json.dumps({
+        "items": [{
+            "instruction": "问题",
+            "output": "表面完整但服务端已声明截断。",
+        }],
+    }, ensure_ascii=False)
+
+    def handler(_: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "finish_reason": "length",
+                    "message": {"content": content},
+                }],
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-length", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 5},
+        task_id="task-length",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert request_count == 1
+    assert records[0]["status"] == "invalid"
+    assert "finish_reason=length" in records[0]["error"]
+
+
+def test_sensitive_model_response_is_not_retried_or_saved() -> None:
+    request_count = 0
+
+    def handler(_: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "finish_reason": "stop",
+                    "message": {"content": "{}"},
+                }],
+                "output_sensitive": True,
+                "base_resp": {"status_code": 1027, "status_msg": "output sensitive"},
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-sensitive", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 5},
+        task_id="task-sensitive",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert request_count == 1
+    assert records[0]["status"] == "invalid"
+    assert "安全拦截" in records[0]["error"]
+    assert "1027" in records[0]["error"]
+
+
+def test_empty_model_content_can_retry_then_succeed() -> None:
+    request_count = 0
+
+    def handler(_: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        if request_count == 1:
+            return httpx.Response(
+                200,
+                json={"choices": [{"finish_reason": "stop", "message": {"content": ""}}]},
+            )
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "finish_reason": "stop",
+                    "message": {
+                        "content": json.dumps({
+                            "items": [{
+                                "instruction": "问题",
+                                "output": "第二次请求返回了完整答案。",
+                            }],
+                        }, ensure_ascii=False),
+                    },
+                }],
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-empty-retry", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 1},
+        task_id="task-empty-retry",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert request_count == 2
+    assert records[0]["status"] == "valid"
+
+
+def test_multiple_top_level_json_documents_are_rejected_as_ambiguous() -> None:
+    first = json.dumps({
+        "items": [{"instruction": "问题一", "output": "答案一"}],
+    }, ensure_ascii=False)
+    second = json.dumps({
+        "items": [{"instruction": "问题二", "output": "答案二"}],
+    }, ensure_ascii=False)
+    client = httpx.Client(
+        transport=httpx.MockTransport(
+            lambda _: httpx.Response(
+                200,
+                json={"choices": [{"message": {"content": f"{first}\n{second}"}}]},
+            )
+        )
+    )
+
+    records = generate_model_records(
+        [{"id": "preview-ambiguous", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 0},
+        task_id="task-ambiguous",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=client,
+    )
+
+    assert records[0]["status"] == "invalid"
+    assert "多个 JSON" in records[0]["error"]
+
+
+def test_generate_model_records_builds_reasoning_output_with_think_tags() -> None:
+    def handler(request: httpx.Request) -> httpx.Response:
+        payload = json.loads(request.content)
+        system_prompt = payload["messages"][0]["content"]
+        assert '"reasoning":"...","answer":"..."' in system_prompt
+        assert "你正在生成用于训练推理模型的思维链数据" in system_prompt
+        assert "推理详细程度为“普通”" in system_prompt
+        assert "系统会在保存时统一组装" in system_prompt
+        content = "<think>模型接口自己的分析</think>" + json.dumps(
+            {
+                "items": [
+                    {
+                        "instruction": "计算两项费用合计",
+                        "input": "交通费 30 元，餐费 20 元",
+                        "reasoning": "先识别两项费用，再计算 30 + 20。",
+                        "answer": "合计 50 元。",
+                    }
+                ]
+            },
+            ensure_ascii=False,
+        )
+        return httpx.Response(
+            200,
+            json={"choices": [{"message": {"content": content}}]},
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-reasoning", "edited_content": "交通费 30 元，餐费 20 元"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"output_type": "reasoning"},
+        task_id="task-reasoning",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert records[0]["status"] == "valid"
+    assert records[0]["output"] == (
+        "<think>\n先识别两项费用,再计算 30 + 20。\n</think>\n合计 50 元。"
+    )
+
+
+def test_generate_model_records_uses_detailed_reasoning_instruction() -> None:
+    def handler(request: httpx.Request) -> httpx.Response:
+        system_prompt = json.loads(request.content)["messages"][0]["content"]
+        assert "推理详细程度为“详细”" in system_prompt
+        assert "完整展开问题条件、来源依据、中间计算或推导" in system_prompt
+        return httpx.Response(
+            200,
+            json={
+                "choices": [
+                    {
+                        "message": {
+                            "content": json.dumps(
+                                {
+                                    "items": [
+                                        {
+                                            "instruction": "计算报销总额",
+                                            "reasoning": "条件为交通费 30 元和餐费 20 元。分别核对后相加，30 + 20 = 50。",
+                                            "answer": "报销总额为 50 元。",
+                                        }
+                                    ]
+                                },
+                                ensure_ascii=False,
+                            )
+                        }
+                    }
+                ]
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-detailed", "edited_content": "交通费 30 元，餐费 20 元"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"output_type": "reasoning", "reasoning_detail": "detailed"},
+        task_id="task-detailed",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert records[0]["status"] == "valid"
+    assert "分别核对后相加" in records[0]["output"]
+
+
+def test_generate_model_records_marks_reasoning_without_reasoning_field_invalid() -> None:
+    response = {
+        "choices": [
+            {
+                "message": {
+                    "content": json.dumps(
+                        {
+                            "items": [
+                                {
+                                    "instruction": "问题",
+                                    "answer": "只有最终答案",
+                                }
+                            ]
+                        },
+                        ensure_ascii=False,
+                    )
+                }
+            }
+        ]
+    }
+    client = httpx.Client(
+        transport=httpx.MockTransport(lambda _: httpx.Response(200, json=response))
+    )
+
+    records = generate_model_records(
+        [{"id": "preview-missing-reasoning", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"output_type": "reasoning"},
+        task_id="task-missing-reasoning",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=client,
+    )
+
+    assert records[0]["status"] == "invalid"
+    assert records[0]["output"] == "只有最终答案"
+    assert "reasoning" in records[0]["error"]
+
+
+def test_standard_output_removes_model_think_block() -> None:
+    content = json.dumps(
+        {
+            "items": [
+                {
+                    "instruction": "问题",
+                    "output": "<think>不应保存的分析</think>最终答案",
+                }
+            ]
+        },
+        ensure_ascii=False,
+    )
+    client = httpx.Client(
+        transport=httpx.MockTransport(
+            lambda _: httpx.Response(
+                200,
+                json={"choices": [{"message": {"content": content}}]},
+            )
+        )
+    )
+
+    records = generate_model_records(
+        [{"id": "preview-standard", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"output_type": "standard"},
+        task_id="task-standard",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=client,
+    )
+
+    assert records[0]["output"] == "最终答案"
+
+
+def test_generate_model_records_keeps_partial_failure_for_manual_repair() -> None:
+    client = httpx.Client(
+        transport=httpx.MockTransport(
+            lambda _: httpx.Response(200, json={"choices": [{"message": {"content": "not-json"}}]})
+        )
+    )
+    records = generate_model_records(
+        [{"id": "preview-1", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 1},
+        task_id="task-1",
+        split={"train": 80, "validation": 10, "test": 10},
+        qa_pairs_per_item=1,
+        client=client,
+    )
+
+    assert len(records) == 1
+    assert records[0]["status"] == "invalid"
+    assert records[0]["error"]
+
+
+def test_generate_model_records_batches_fifty_results_with_unique_ids() -> None:
+    requests: list[httpx.Request] = []
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        requests.append(request)
+        batch_start = (len(requests) - 1) * 10 + 1
+        batch_end = batch_start + 9
+        payload = json.loads(request.content)
+        system_prompt = payload["messages"][0]["content"]
+        assert "items 必须包含 10 条" in system_prompt
+        assert f"第 {batch_start}-{batch_end} 条" in system_prompt
+        return httpx.Response(
+            200,
+            json={
+                "choices": [
+                    {
+                        "message": {
+                            "content": json.dumps(
+                                {
+                                    "items": [
+                                        {
+                                            "instruction": "同一问题",
+                                            "input": "来源正文",
+                                            "output": "同一答案",
+                                        }
+                                        for _ in range(batch_start, batch_end + 1)
+                                    ]
+                                },
+                                ensure_ascii=False,
+                            )
+                        }
+                    }
+                ]
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-50", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={},
+        task_id="task-50",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=50,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert len(requests) == 5
+    assert len(records) == 50
+    assert len({record["id"] for record in records}) == 50
+    assert {record["instruction"] for record in records} == {"同一问题"}
+    assert all(record["status"] == "valid" for record in records)
+
+
+def test_generate_model_records_preserves_successful_batches_when_one_fails() -> None:
+    request_count = 0
+
+    def handler(_: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        if request_count == 2:
+            return httpx.Response(500)
+        return httpx.Response(
+            200,
+            json={
+                "choices": [
+                    {
+                        "message": {
+                            "content": json.dumps(
+                                {
+                                    "items": [
+                                        {
+                                            "instruction": f"问题 {index}",
+                                            "output": f"答案 {index}",
+                                        }
+                                        for index in range(1, 11)
+                                    ]
+                                },
+                                ensure_ascii=False,
+                            )
+                        }
+                    }
+                ]
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-partial", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 0},
+        task_id="task-partial",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=20,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert len(records) == 11
+    assert sum(record["status"] == "valid" for record in records) == 10
+    failed = next(record for record in records if record["status"] == "invalid")
+    assert "第 11-20 条" in failed["instruction"]
+    assert len({record["id"] for record in records}) == len(records)
+
+
+def test_generate_model_records_retries_short_batch_then_marks_it_invalid() -> None:
+    request_count = 0
+
+    def handler(_: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        return httpx.Response(
+            200,
+            json={
+                "choices": [
+                    {
+                        "message": {
+                            "content": json.dumps(
+                                {
+                                    "items": [
+                                        {
+                                            "instruction": "只有一条",
+                                            "output": "不足本批要求数量",
+                                        }
+                                    ]
+                                },
+                                ensure_ascii=False,
+                            )
+                        }
+                    }
+                ]
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-short", "edited_content": "来源正文"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 1},
+        task_id="task-short",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=10,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert request_count == 2
+    assert len(records) == 1
+    assert records[0]["status"] == "invalid"
+    assert "expected 10, got 1" in records[0]["error"]
+
+
+def test_generate_model_records_does_not_retry_non_retryable_http_errors() -> None:
+    request_count = 0
+
+    def handler(_: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        return httpx.Response(401, json={"error": {"message": "unauthorized"}})
+
+    records = generate_model_records(
+        [{"id": "preview-auth", "edited_content": "来源内容"}],
+        model={
+            "api_url": "https://model.example/v1",
+            "online_model_name": "test-model",
+            "api_key": "invalid",
+        },
+        config={"generation_retries": 5},
+        task_id="task-auth",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert request_count == 1
+    assert records[0]["status"] == "invalid"
+    assert "401" in records[0]["error"]
+
+
+@pytest.mark.parametrize("status_code", [408, 425, 429, 500])
+def test_generate_model_records_retries_retryable_http_statuses(
+    status_code: int,
+) -> None:
+    request_count = 0
+
+    def handler(_: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        if request_count == 1:
+            return httpx.Response(status_code)
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "message": {
+                        "content": json.dumps({
+                            "items": [{
+                                "instruction": "来源内容是什么？",
+                                "output": "这是用于验证可重试错误的来源内容。",
+                            }],
+                        }, ensure_ascii=False),
+                    },
+                }],
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-retryable", "edited_content": "来源内容"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 1},
+        task_id="task-retryable",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert request_count == 2
+    assert records[0]["status"] == "valid"
+
+
+def test_generate_model_records_retries_transient_network_errors() -> None:
+    request_count = 0
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        nonlocal request_count
+        request_count += 1
+        if request_count == 1:
+            raise httpx.ConnectError("temporary connection failure", request=request)
+        return httpx.Response(
+            200,
+            json={
+                "choices": [{
+                    "message": {
+                        "content": json.dumps({
+                            "items": [{
+                                "instruction": "网络恢复了吗？",
+                                "output": "临时连接错误后，第二次模型请求已经成功。",
+                            }],
+                        }, ensure_ascii=False),
+                    },
+                }],
+            },
+        )
+
+    records = generate_model_records(
+        [{"id": "preview-network", "edited_content": "网络重试来源"}],
+        model={"name": "model", "api_url": "https://model.example/v1"},
+        config={"generation_retries": 1},
+        task_id="task-network",
+        split={"train": 100, "validation": 0, "test": 0},
+        qa_pairs_per_item=1,
+        client=httpx.Client(transport=httpx.MockTransport(handler)),
+    )
+
+    assert request_count == 2
+    assert records[0]["status"] == "valid"
+
+
+@pytest.mark.parametrize("qa_pairs_per_item", [0, 51])
+def test_generate_model_records_rejects_out_of_range_count(
+    qa_pairs_per_item: int,
+) -> None:
+    with pytest.raises(ModelGenerationError, match=r"\[1, 50\]"):
+        generate_model_records(
+            [],
+            model={"name": "model", "api_url": "https://model.example/v1"},
+            config={},
+            task_id="task-invalid",
+            split={"train": 100, "validation": 0, "test": 0},
+            qa_pairs_per_item=qa_pairs_per_item,
+        )
+
+
+def test_generate_model_records_rejects_unknown_output_type() -> None:
+    with pytest.raises(ModelGenerationError, match="output_type"):
+        generate_model_records(
+            [],
+            model={"name": "model", "api_url": "https://model.example/v1"},
+            config={"output_type": "unknown"},
+            task_id="task-invalid-output",
+            split={"train": 100, "validation": 0, "test": 0},
+            qa_pairs_per_item=1,
+        )
+
+
+def test_generate_model_records_rejects_unknown_reasoning_detail() -> None:
+    with pytest.raises(ModelGenerationError, match="reasoning_detail"):
+        generate_model_records(
+            [],
+            model={"name": "model", "api_url": "https://model.example/v1"},
+            config={"output_type": "reasoning", "reasoning_detail": "verbose"},
+            task_id="task-invalid-reasoning-detail",
+            split={"train": 100, "validation": 0, "test": 0},
+            qa_pairs_per_item=1,
+        )

backend/tests/test_data_process_migration.py

@@ -0,0 +1,61 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+from app.modules.data_process.schema_cli import REQUIRED_TASK_COLUMNS, _target_label
+
+
+def test_runtime_migration_fails_fast_on_incompatible_schema() -> None:
+    sql_path = (
+        Path(__file__).resolve().parents[1]
+        / "app"
+        / "db"
+        / "sql"
+        / "002_data_process.sql"
+    )
+    sql = sql_path.read_text(encoding="utf-8")
+
+    assert "requires 001_platform_runtime.sql first" in sql
+    assert "supports only the current TEXT runtime schema" in sql
+    assert "generation_run_id" in sql
+    assert "results_confirmed BOOLEAN NOT NULL DEFAULT TRUE" in sql
+    assert "WHERE status <> 'completed' AND results_confirmed=TRUE" in sql
+    assert "ADD COLUMN IF NOT EXISTS workflow_step VARCHAR(20)" in sql
+    assert "ADD COLUMN IF NOT EXISTS preview_status VARCHAR(20)" in sql
+    assert "ADD COLUMN IF NOT EXISTS preview_progress NUMERIC(5,2)" in sql
+    assert "ADD COLUMN IF NOT EXISTS preview_run_id TEXT" in sql
+    assert "ADD COLUMN IF NOT EXISTS preview_failure_reason TEXT" in sql
+    assert "ADD COLUMN IF NOT EXISTS preview_total_files INTEGER" in sql
+    assert "ADD COLUMN IF NOT EXISTS preview_completed_files INTEGER" in sql
+    assert "data_process_workflow_backfill_ids" in sql
+    assert "ck_data_process_tasks_workflow_step" in sql
+    assert "ck_data_process_tasks_preview_status" in sql
+    assert "ck_data_process_tasks_preview_progress" in sql
+    assert "ck_data_process_tasks_preview_file_counts" in sql
+    for value in ("create", "model", "upload", "preview", "generate", "results"):
+        assert f"'{value}'" in sql
+    for value in ("idle", "queued", "running", "completed", "failed", "cancelled"):
+        assert f"'{value}'" in sql
+    assert "CREATE TABLE IF NOT EXISTS data_process_results" in sql
+    assert sql.count("BEGIN;") == 1
+    assert sql.rstrip().endswith("COMMIT;")
+
+
+def test_schema_cli_target_label_never_contains_credentials() -> None:
+    label = _target_label("postgresql://secret-user:secret-password@db.example:5433/yg_ft")
+    assert label == "db.example:5433/yg_ft"
+    assert "secret" not in label
+
+
+def test_schema_check_requires_current_runtime_columns() -> None:
+    assert REQUIRED_TASK_COLUMNS == (
+        "generation_run_id",
+        "results_confirmed",
+        "workflow_step",
+        "preview_status",
+        "preview_progress",
+        "preview_run_id",
+        "preview_failure_reason",
+        "preview_total_files",
+        "preview_completed_files",
+    )

backend/tests/test_data_process_storage.py

@@ -0,0 +1,276 @@
+from __future__ import annotations
+
+from pathlib import Path, PurePosixPath
+
+import pytest
+
+from app.modules.data_process import storage as storage_module
+from app.modules.data_process.storage import (
+    DataProcessStorageError,
+    LocalDataProcessStorage,
+    StagedSourceObject,
+)
+
+
+def _stage(
+    storage: LocalDataProcessStorage,
+    *,
+    batch_id: str = "batch-main",
+    task_id: str = "task-1",
+    source_file_id: str = "source-1",
+    version: int = 1,
+    name: str = "source.txt",
+    content: bytes = b"payload",
+) -> StagedSourceObject:
+    return storage.stage_bytes(
+        batch_id=batch_id,
+        task_id=task_id,
+        source_file_id=source_file_id,
+        version=version,
+        name=name,
+        content=content,
+    )
+
+
+def _create_symlink(link: Path, target: Path, *, target_is_directory: bool = False) -> None:
+    try:
+        link.symlink_to(target, target_is_directory=target_is_directory)
+    except (NotImplementedError, OSError) as exc:
+        pytest.skip(f"当前平台不支持创建测试所需的符号链接: {exc}")
+
+
+def _assert_staging_empty(storage: LocalDataProcessStorage) -> None:
+    assert list((storage.root / ".staging").iterdir()) == []
+
+
+def test_stage_publish_read_delete_roundtrip_with_unicode_filename(tmp_path: Path) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+    content = "第一行\n第二行，100% 完成".encode()
+
+    staged = _stage(
+        storage,
+        name="中文 数据 100%.csv",
+        content=content,
+    )
+
+    assert "%20" in staged.reference
+    assert "%25" in staged.reference
+    storage.publish([staged])
+
+    assert storage.read(staged.reference) == content
+    assert storage.delete(staged.reference) is True
+    assert storage.delete(staged.reference) is False
+    _assert_staging_empty(storage)
+
+
+def test_stage_copy_creates_an_independently_deletable_source_object(
+    tmp_path: Path,
+) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+    original = _stage(storage, content=b"immutable source")
+    storage.publish([original])
+
+    copied = storage.stage_copy(
+        batch_id="batch-copy",
+        source_reference=original.reference,
+        expected_source_task_id="task-1",
+        expected_source_file_id="source-1",
+        task_id="task-2",
+        source_file_id="source-2",
+        version=1,
+        name="source.txt",
+    )
+    storage.publish([copied])
+
+    assert storage.read(copied.reference) == b"immutable source"
+    assert storage.delete(
+        original.reference,
+        expected_task_id="task-1",
+        expected_source_file_id="source-1",
+    ) is True
+    assert storage.read(copied.reference) == b"immutable source"
+    assert storage.delete(
+        copied.reference,
+        expected_task_id="task-2",
+        expected_source_file_id="source-2",
+    ) is True
+    _assert_staging_empty(storage)
+
+
+def test_db_reference_is_left_to_database_storage(tmp_path: Path) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+
+    assert storage.read("db://source-files/source-1") is None
+    assert storage.delete("db://source-files/source-1") is False
+
+
+def test_owned_source_can_be_streamed_by_byte_range(tmp_path: Path) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+    content = b"0123456789abcdef"
+    staged = _stage(storage, content=content)
+    storage.publish([staged])
+
+    assert storage.file_size(
+        staged.reference,
+        expected_task_id="task-1",
+        expected_source_file_id="source-1",
+    ) == len(content)
+    assert b"".join(storage.iter_bytes(
+        staged.reference,
+        expected_task_id="task-1",
+        expected_source_file_id="source-1",
+        expected_size=len(content),
+        start=4,
+        length=6,
+        chunk_size=2,
+    )) == b"456789"
+
+    with pytest.raises(DataProcessStorageError, match="owner mismatch"):
+        storage.file_size(
+            staged.reference,
+            expected_task_id="another-task",
+            expected_source_file_id="source-1",
+        )
+    with pytest.raises(DataProcessStorageError, match="does not match metadata"):
+        b"".join(storage.iter_bytes(
+            staged.reference,
+            expected_task_id="task-1",
+            expected_source_file_id="source-1",
+            expected_size=len(content) + 1,
+        ))
+
+
+@pytest.mark.parametrize(
+    "reference",
+    [
+        "local://data-process/../source-1/v1/file.txt",
+        "local://data-process/task-1/source-1/v1/file%2Fname.txt",
+        "local://data-process/task-1/source-1/v1/file.txt?download=1",
+        "local://data-process/task-1/source-1/v1/file.txt#fragment",
+        "https://data-process/task-1/source-1/v1/file.txt",
+    ],
+    ids=[
+        "parent-traversal",
+        "percent-encoded-slash",
+        "query",
+        "fragment",
+        "wrong-scheme",
+    ],
+)
+def test_unsafe_references_are_rejected(tmp_path: Path, reference: str) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+
+    with pytest.raises(DataProcessStorageError):
+        storage.read(reference)
+    with pytest.raises(DataProcessStorageError):
+        storage.delete(reference)
+
+
+def test_publish_rejects_intermediate_directory_symlink(tmp_path: Path) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+    outside = tmp_path / "outside"
+    outside.mkdir()
+    staged = _stage(storage, task_id="linked-task")
+    _create_symlink(
+        storage.root / "linked-task",
+        outside,
+        target_is_directory=True,
+    )
+
+    with pytest.raises(DataProcessStorageError, match="symlink|non-directory"):
+        storage.publish([staged])
+
+    assert list(outside.iterdir()) == []
+    _assert_staging_empty(storage)
+
+
+def test_target_symlink_is_never_followed_or_deleted(tmp_path: Path) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+    staged = _stage(storage, task_id="task-link", source_file_id="source-link")
+    outside_file = tmp_path / "outside.txt"
+    outside_file.write_bytes(b"outside sentinel")
+    final_path = storage.root.joinpath(*staged._relative_path.parts)
+    final_path.parent.mkdir(parents=True)
+    _create_symlink(final_path, outside_file)
+
+    with pytest.raises(DataProcessStorageError, match="already exists"):
+        storage.publish([staged])
+    with pytest.raises(DataProcessStorageError, match="regular file"):
+        storage.read(staged.reference)
+    with pytest.raises(DataProcessStorageError, match="non-regular"):
+        storage.delete(staged.reference)
+
+    assert final_path.is_symlink()
+    assert outside_file.read_bytes() == b"outside sentinel"
+    _assert_staging_empty(storage)
+
+
+def test_publish_rolls_back_first_object_when_second_target_collides(tmp_path: Path) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+    existing = _stage(
+        storage,
+        batch_id="batch-existing",
+        source_file_id="source-existing",
+        content=b"existing content",
+    )
+    storage.publish([existing])
+
+    first = _stage(
+        storage,
+        batch_id="batch-new",
+        source_file_id="source-new",
+        content=b"must be rolled back",
+    )
+    colliding_second = _stage(
+        storage,
+        batch_id="batch-new",
+        source_file_id="source-existing",
+        content=b"must not replace existing content",
+    )
+
+    with pytest.raises(DataProcessStorageError, match="already exists"):
+        storage.publish([first, colliding_second])
+
+    with pytest.raises(DataProcessStorageError, match="does not exist"):
+        storage.read(first.reference)
+    assert storage.read(existing.reference) == b"existing content"
+    _assert_staging_empty(storage)
+
+
+def test_publish_rejects_manually_forged_staged_object(tmp_path: Path) -> None:
+    storage = LocalDataProcessStorage(tmp_path / "storage")
+    temporary_path = storage.root / ".staging" / "batch-forged" / "forged.tmp"
+    temporary_path.parent.mkdir()
+    temporary_path.write_bytes(b"forged content")
+    relative_path = PurePosixPath("task-forged", "source-forged", "v1", "forged.txt")
+    forged = StagedSourceObject(
+        reference="local://data-process/task-forged/source-forged/v1/forged.txt",
+        _temporary_path=temporary_path,
+        _relative_path=relative_path,
+    )
+
+    with pytest.raises(DataProcessStorageError, match="was not issued"):
+        storage.publish([forged])
+    with pytest.raises(DataProcessStorageError, match="was not issued"):
+        storage.discard([forged])
+
+    assert temporary_path.read_bytes() == b"forged content"
+    assert not storage.root.joinpath(*relative_path.parts).exists()
+
+
+def test_relative_storage_configuration_is_anchored_to_backend_root(
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    relative_configuration = Path("relative-storage") / tmp_path.name
+    backend_root = Path(storage_module.__file__).resolve().parents[3]
+    monkeypatch.chdir(tmp_path)
+    monkeypatch.setenv("DATA_PROCESS_STORAGE_DIR", str(relative_configuration))
+    storage_module.get_data_process_storage.cache_clear()
+
+    try:
+        configured_root = storage_module._configured_storage_root()
+        assert configured_root == backend_root / relative_configuration
+        assert not configured_root.exists()
+    finally:
+        storage_module.get_data_process_storage.cache_clear()

backend/tests/test_document_chunking.py

@@ -0,0 +1,85 @@
+from __future__ import annotations
+
+from llama_index.core.embeddings import MockEmbedding
+
+from app.modules.data_process.document_chunking import (
+    DocumentChunk,
+    _compact_with_offsets,
+    _project_layout_span,
+    chunk_fixed_text,
+    chunk_semantic_text,
+    merge_short_chunks,
+)
+
+
+def test_fixed_splitter_preserves_offsets_and_token_limit() -> None:
+    text = "第一段说明苹果。第二段说明香蕉。\n第三段说明数据库。第四段说明索引。"
+    chunks = chunk_fixed_text(text, chunk_size=20, chunk_overlap=0)
+
+    assert len(chunks) > 1
+    assert all(chunk.source_start is not None for chunk in chunks)
+    assert all(chunk.source_end is not None for chunk in chunks)
+    assert all(
+        chunk.original_content == text[chunk.source_start : chunk.source_end]
+        for chunk in chunks
+        if chunk.source_start is not None and chunk.source_end is not None
+    )
+    assert all(chunk.token_count <= 20 for chunk in chunks)
+
+
+def test_semantic_splitter_uses_llamaindex_and_reapplies_maximum_size() -> None:
+    text = "第一段讨论水果。第二段继续讨论香蕉。第三段讨论数据库。第四段讨论索引。"
+    chunks = chunk_semantic_text(
+        text,
+        chunk_size=30,
+        chunk_overlap=0,
+        breakpoint_percentile_threshold=95,
+        embed_model=MockEmbedding(embed_dim=8),
+    )
+
+    assert len(chunks) >= 2
+    assert all(chunk.token_count <= 30 for chunk in chunks)
+    assert "".join(chunk.original_content for chunk in chunks) == text
+
+
+def test_layout_projection_ignores_layout_whitespace_but_keeps_source_lines() -> None:
+    source = "标题\n第一条 这是正文。\n第二条 后续正文。"
+    compact_source, offsets = _compact_with_offsets(source)
+    start, end, cursor = _project_layout_span(
+        source,
+        "第一条\n这是正文。",
+        compact_source=compact_source,
+        source_offsets=offsets,
+        compact_start=0,
+    )
+
+    assert source[start:end] == "第一条 这是正文。"
+    assert cursor > 0
+
+
+def test_short_layout_chunk_merges_with_neighbor_and_keeps_page_provenance() -> None:
+    source = "短标题\n这是一段足够长的正文内容，用于测试相邻切片合并。"
+    chunks = [
+        DocumentChunk("短标题", "短标题", 0, 3, 1, 1, 2, source_pages=(1,)),
+        DocumentChunk(
+            "这是一段足够长的正文内容，用于测试相邻切片合并。",
+            "这是一段足够长的正文内容，用于测试相邻切片合并。",
+            4,
+            len(source),
+            2,
+            2,
+            20,
+            source_pages=(1, 2),
+        ),
+    ]
+
+    merged = merge_short_chunks(
+        chunks,
+        source_text=source,
+        min_token_count=10,
+        max_token_count=100,
+    )
+
+    assert len(merged) == 1
+    assert merged[0].original_content == source
+    assert merged[0].source_pages == (1, 2)

backend/tests/test_governance.py

@@ -0,0 +1,774 @@
+"""
+平台治理功能集成测试 —— 覆盖第 1-4 周交付内容。
+
+测试策略：
+- 在导入 app 模块前 mock psycopg / psycopg_pool，避免依赖真实数据库驱动
+- 使用 FastAPI TestClient 对真实路由栈发起请求
+- 通过 mock.get_platform_store 替换为内存 FakeStore
+- 每周交付内容对应一组 test class，方便分阶段验收
+
+覆盖范围：
+  第 1 周 — 登录、当前用户、用户列表、权限码、日志查询
+  第 2 周 — 租户、项目、项目成员、资源 ACL
+  第 3 周 — 审批实例、审批模板、审计日志查询和导出
+  第 4 周 — 写操作审计、审批拦截、权限校验
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+import types
+from contextlib import contextmanager
+from typing import Any, Iterator
+from unittest.mock import MagicMock, patch
+
+import pytest
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+
+# ============================================================
+# 在导入 app 之前 mock psycopg / psycopg_pool
+# ============================================================
+
+_psycopg_mock = types.ModuleType("psycopg")
+_psycopg_mock.PgConn = type("PgConn", (), {})
+_psycopg_mock.PostgresConnectionPool = MagicMock()
+_psycopg_mock.connection = MagicMock()
+sys.modules.setdefault("psycopg", _psycopg_mock)
+
+_psycopg_pool_mock = types.ModuleType("psycopg_pool")
+_psycopg_pool_mock.ConnectionPool = MagicMock()
+sys.modules.setdefault("psycopg_pool", _psycopg_pool_mock)
+
+# 现在安全导入 app 模块
+from app.api.v1.endpoints.platform import ok, fail  # noqa: E402
+from app.modules.tenant.router import router as tenant_router  # noqa: E402
+from app.modules.project.router import router as project_router  # noqa: E402
+from app.modules.approval.router import router as approval_router  # noqa: E402
+from app.modules.system.router import router as system_router  # noqa: E402
+from app.modules.retention.router import router as retention_router  # noqa: E402
+from app.modules.resource.router import router as resource_router  # noqa: E402
+from app.api.v1.endpoints.platform import router as platform_router  # noqa: E402
+
+PREFIX = "/modelTF"
+ADMIN_TOKEN = "platform-token-u_admin"
+OP_TOKEN = "platform-token-u_op"
+
+
+# ============================================================
+# FakePlatformStore —— 内存实现，模拟 PlatformStore 全部治理接口
+# ============================================================
+
+class FakePlatformStore:
+    """平台治理测试专用内存 store，确保测试不连接真实数据库。"""
+
+    def __init__(self) -> None:
+        self._users: list[dict[str, Any]] = [
+            {
+                "id": "u_admin",
+                "username": "admin",
+                "display_name": "Admin",
+                "role": "admin",
+                "status": "active",
+                "permissions": [
+                    "dashboard", "fine-tune", "model-eval", "model-inference",
+                    "model-manage", "dataset", "data-process", "data-convert",
+                    "compute", "hardware", "logs", "user-settings",
+                ],
+                "last_login": "2026-08-01T10:00:00Z",
+                "protected": True,
+            },
+            {
+                "id": "u_op",
+                "username": "operator",
+                "display_name": "Operator",
+                "role": "operator",
+                "status": "active",
+                "permissions": ["dashboard", "fine-tune"],
+                "last_login": "2026-08-01T11:00:00Z",
+                "protected": False,
+            },
+        ]
+        self._tenants: dict[str, dict[str, Any]] = {}
+        self._projects: dict[str, dict[str, Any]] = {}
+        self._members: dict[str, list[dict[str, Any]]] = {}
+        self._acl: dict[str, list[dict[str, Any]]] = {}
+        self._audit_logs: list[dict[str, Any]] = []
+        self._approval_templates: dict[str, dict[str, Any]] = {}
+        self._approval_instances: dict[str, dict[str, Any]] = {}
+        self._retention_policies: dict[str, dict[str, Any]] = {}
+        self._models: list[dict[str, Any]] = []
+        self._datasets: list[dict[str, Any]] = []
+        self._tasks: list[dict[str, Any]] = []
+        self._compute_nodes: list[dict[str, Any]] = []
+        self._gpus: list[dict[str, Any]] = []
+        self._sessions: list[dict[str, Any]] = []
+        self._seq = 0
+
+    @contextmanager
+    def connect(self) -> Iterator[Any]:
+        class FakeConn:
+            def execute(self, *a, **kw):
+                return []
+
+            def commit(self):
+                pass
+
+            def rollback(self):
+                pass
+
+            def close(self):
+                pass
+
+        yield FakeConn()
+
+    # ---- helpers ----
+
+    def _next_id(self, prefix: str) -> str:
+        self._seq += 1
+        return f"{prefix}_{self._seq}"
+
+    # ==================== 第1周：登录 / 用户 / 权限码 / 日志 ====================
+
+    def login(self, username: str, password: str) -> dict[str, Any] | None:
+        for u in self._users:
+            if u["username"] == username and u["status"] == "active":
+                if password in ("admin123", "operator123", "test123"):
+                    return dict(u)
+        return None
+
+    def users(self) -> list[dict[str, Any]]:
+        return [dict(u) for u in self._users]
+
+    def create_user(self, payload: dict[str, Any]) -> dict[str, Any]:
+        u = {"id": self._next_id("u"), "protected": False, **payload}
+        self._users.append(u)
+        return u
+
+    def update_user(self, user_id: str, payload: dict[str, Any]) -> dict[str, Any]:
+        for u in self._users:
+            if u["id"] == user_id:
+                u.update(payload)
+                return u
+        raise KeyError(user_id)
+
+    def delete_user(self, user_id: str) -> None:
+        self._users = [u for u in self._users if u["id"] != user_id]
+
+    def roles(self) -> list[dict[str, Any]]:
+        return [
+            {"name": "admin", "display_name": "管理员"},
+            {"name": "operator", "display_name": "操作员"},
+            {"name": "viewer", "display_name": "访客"},
+        ]
+
+    def log_files(self, date: str | None = None) -> list[dict[str, Any]]:
+        return [{"name": "backend-2026-08-01.log", "size": "1 KB", "date": "2026-08-01"}]
+
+    def log_content(self, file: str) -> dict[str, Any]:
+        return {"file": file, "content": "[INFO] test line", "size": "1 KB"}
+
+    def training_log_files(self) -> list[dict[str, Any]]:
+        return [{"task_id": "ft_001", "name": "ft_001.log", "size": "2 KB"}]
+
+    def training_log_content(self, file: str) -> dict[str, Any]:
+        return {"file": file, "content": "epoch 0 loss 1.0", "size": "2 KB"}
+
+    # ==================== 第2周：租户 / 项目 / 成员 / ACL ====================
+
+    def tenants(self) -> list[dict[str, Any]]:
+        return list(self._tenants.values())
+
+    def tenant(self, tenant_id: str) -> dict[str, Any]:
+        if tenant_id not in self._tenants:
+            raise KeyError(tenant_id)
+        return dict(self._tenants[tenant_id])
+
+    def create_tenant(self, payload: dict[str, Any]) -> dict[str, Any]:
+        tid = self._next_id("tnt")
+        t = {"id": tid, "status": "active", "quota": "{}", "retention_policy_id": None,
+             "create_time": "2026-08-01T00:00:00Z", **payload}
+        self._tenants[tid] = t
+        return dict(t)
+
+    def update_tenant(self, tenant_id: str, payload: dict[str, Any]) -> dict[str, Any]:
+        self._tenants[tenant_id].update(payload)
+        return dict(self._tenants[tenant_id])
+
+    def set_tenant_quota(self, tenant_id: str, quota: dict[str, Any]) -> dict[str, Any]:
+        self._tenants[tenant_id]["quota"] = json.dumps(quota)
+        return dict(self._tenants[tenant_id])
+
+    def set_tenant_retention(self, tenant_id: str, retention_policy_id: str | None) -> dict[str, Any]:
+        self._tenants[tenant_id]["retention_policy_id"] = retention_policy_id
+        return dict(self._tenants[tenant_id])
+
+    def projects(self, *, tenant_id: str = "default", status: str | None = None, keyword: str | None = None) -> list[dict[str, Any]]:
+        result = []
+        for p in self._projects.values():
+            if p.get("tenant_id") != tenant_id:
+                continue
+            if status and p.get("status") != status:
+                continue
+            if keyword and keyword.lower() not in p.get("name", "").lower():
+                continue
+            result.append(dict(p))
+        return result
+
+    def project(self, project_id: str) -> dict[str, Any]:
+        if project_id not in self._projects:
+            raise KeyError(project_id)
+        return dict(self._projects[project_id])
+
+    def create_project(self, payload: dict[str, Any]) -> dict[str, Any]:
+        pid = self._next_id("prj")
+        p = {"id": pid, "status": "active", "quota": "{}", "create_time": "2026-08-01T00:00:00Z", **payload}
+        self._projects[pid] = p
+        self._members[pid] = []
+        return dict(p)
+
+    def update_project(self, project_id: str, payload: dict[str, Any]) -> dict[str, Any]:
+        self._projects[project_id].update(payload)
+        return dict(self._projects[project_id])
+
+    def archive_project(self, project_id: str) -> dict[str, Any]:
+        self._projects[project_id]["status"] = "archived"
+        return dict(self._projects[project_id])
+
+    def delete_project(self, project_id: str) -> None:
+        self._projects.pop(project_id, None)
+        self._members.pop(project_id, None)
+
+    def project_members(self, project_id: str) -> list[dict[str, Any]]:
+        return [dict(m) for m in self._members.get(project_id, [])]
+
+    def add_project_member(self, project_id: str, payload: dict[str, Any]) -> dict[str, Any]:
+        m = {"joined_at": "2026-08-01T00:00:00Z", **payload}
+        self._members.setdefault(project_id, []).append(m)
+        return m
+
+    def update_project_member_role(self, project_id: str, user_id: str, role: str) -> dict[str, Any]:
+        for m in self._members.get(project_id, []):
+            if m["user_id"] == user_id:
+                m["role"] = role
+                return m
+        raise KeyError(user_id)
+
+    def remove_project_member(self, project_id: str, user_id: str) -> None:
+        self._members[project_id] = [m for m in self._members.get(project_id, []) if m["user_id"] != user_id]
+
+    # ---- ACL ----
+
+    def get_acl(self, resource_type: str, resource_id: str) -> list[dict[str, Any]]:
+        key = f"{resource_type}:{resource_id}"
+        return [dict(a) for a in self._acl.get(key, [])]
+
+    def set_acl(self, resource_type: str, resource_id: str, entries: list[dict[str, Any]]) -> list[dict[str, Any]]:
+        key = f"{resource_type}:{resource_id}"
+        self._acl[key] = [dict(e) for e in entries]
+        return self.get_acl(resource_type, resource_id)
+
+    def resource_acl(self, resource_type: str, resource_id: str) -> list[dict[str, Any]]:
+        rows = self.get_acl(resource_type, resource_id)
+        grouped: dict[str, dict[str, Any]] = {}
+        for r in rows:
+            k = f"{r.get('principal_type')}:{r.get('principal_id')}"
+            bucket = grouped.setdefault(k, {
+                "subject_type": r.get("principal_type"),
+                "subject_id": r.get("principal_id"),
+                "permissions": [],
+            })
+            perm = r.get("permission")
+            if perm and perm not in bucket["permissions"]:
+                bucket["permissions"].append(perm)
+        return list(grouped.values())
+
+    def set_resource_acl(self, resource_type: str, resource_id: str, entries: list[dict[str, Any]]) -> list[dict[str, Any]]:
+        flat: list[dict[str, Any]] = []
+        for e in entries:
+            for perm in e.get("permissions") or []:
+                flat.append({
+                    "principal_type": e.get("subject_type"),
+                    "principal_id": e.get("subject_id"),
+                    "permission": perm,
+                })
+        self.set_acl(resource_type, resource_id, flat)
+        return self.resource_acl(resource_type, resource_id)
+
+    # ==================== 第3周：审批 / 审计 / 留存 ====================
+
+    def approval_templates(self) -> list[dict[str, Any]]:
+        return list(self._approval_templates.values())
+
+    def create_approval_template(self, payload: dict[str, Any]) -> dict[str, Any]:
+        tid = payload.get("id") or self._next_id("tpl")
+        t = {"id": tid, "steps": [], "create_time": "2026-08-01T00:00:00Z", **payload}
+        self._approval_templates[tid] = t
+        return dict(t)
+
+    def approval_instances(self, *, status: str | None = None) -> list[dict[str, Any]]:
+        result = []
+        for i in self._approval_instances.values():
+            if status and i.get("status") != status:
+                continue
+            result.append(dict(i))
+        return result
+
+    def approval_instance(self, instance_id: str) -> dict[str, Any]:
+        if instance_id not in self._approval_instances:
+            raise KeyError(instance_id)
+        return dict(self._approval_instances[instance_id])
+
+    def create_approval_instance(self, payload: dict[str, Any]) -> dict[str, Any]:
+        iid = self._next_id("appr")
+        inst = {
+            "id": iid,
+            "status": "pending",
+            "current_step": 0,
+            "steps": [],
+            "create_time": "2026-08-01T00:00:00Z",
+            **payload,
+        }
+        self._approval_instances[iid] = inst
+        return dict(inst)
+
+    def decide_approval_step(self, instance_id: str, step_index: int, *, approver_id: str, approved: bool, comment: str | None = None) -> dict[str, Any]:
+        inst = self._approval_instances[instance_id]
+        inst["status"] = "approved" if approved else "rejected"
+        inst["current_step"] = step_index + 1
+        return dict(inst)
+
+    def audit_logs(self, **kw) -> dict[str, Any]:
+        items = [dict(l) for l in self._audit_logs]
+        for filter_key in ("tenant_id", "project_id", "actor_id", "action", "target_type"):
+            val = kw.get(filter_key)
+            if val:
+                items = [l for l in items if l.get(filter_key) == val]
+        limit = kw.get("limit", 50)
+        offset = kw.get("offset", 0)
+        total = len(items)
+        items = items[offset:offset + limit]
+        return {"items": items, "total": total}
+
+    def record_audit(self, **kw) -> None:
+        log = {"id": self._next_id("log"), "time": "2026-08-01T12:00:00Z", **kw}
+        self._audit_logs.append(log)
+
+    # ---- 留存策略 ----
+
+    def retention_policies(self) -> list[dict[str, Any]]:
+        return list(self._retention_policies.values())
+
+    def retention_policy(self, policy_id: str) -> dict[str, Any]:
+        if policy_id not in self._retention_policies:
+            raise KeyError(policy_id)
+        return dict(self._retention_policies[policy_id])
+
+    def create_retention_policy(self, payload: dict[str, Any]) -> dict[str, Any]:
+        pid = payload.get("id") or self._next_id("rpol")
+        p = {"id": pid, "status": "active", "create_time": "2026-08-01T00:00:00Z", **payload}
+        self._retention_policies[pid] = p
+        return dict(p)
+
+    def update_retention_policy(self, policy_id: str, payload: dict[str, Any]) -> dict[str, Any]:
+        self._retention_policies[policy_id].update(payload)
+        return dict(self._retention_policies[policy_id])
+
+    def delete_retention_policy(self, policy_id: str) -> None:
+        self._retention_policies.pop(policy_id, None)
+
+    # ---- dashboard & other stubs ----
+
+    def login_duration_rank(self, limit: int = 8, days: int = 30) -> list[dict[str, Any]]:
+        return [{"user": "admin", "role": "admin", "duration": 10.0}]
+
+    def models(self) -> list[dict[str, Any]]:
+        return self._models
+
+    def datasets(self) -> list[dict[str, Any]]:
+        return self._datasets
+
+    def tasks(self) -> list[dict[str, Any]]:
+        return self._tasks
+
+    def compute_nodes(self) -> list[dict[str, Any]]:
+        return self._compute_nodes
+
+    def gpus(self) -> list[dict[str, Any]]:
+        return self._gpus
+
+    def system_info(self) -> dict[str, Any]:
+        return {"cpu": {}, "memory": {}}
+
+
+# ============================================================
+# 测试 fixtures
+# ============================================================
+
+@pytest.fixture(scope="module")
+def fake_store() -> FakePlatformStore:
+    return FakePlatformStore()
+
+
+def _build_client(store: FakePlatformStore) -> TestClient:
+    """构建 TestClient，patch 所有治理模块的 get_platform_store。"""
+    app = FastAPI()
+    app.include_router(platform_router, prefix=PREFIX)
+    app.include_router(system_router, prefix=PREFIX)
+    app.include_router(tenant_router, prefix=PREFIX)
+    app.include_router(project_router, prefix=PREFIX)
+    app.include_router(approval_router, prefix=PREFIX)
+    app.include_router(retention_router, prefix=PREFIX)
+    app.include_router(resource_router, prefix=PREFIX)
+
+    patches = [
+        patch("app.db.platform_store.get_platform_store", return_value=store),
+        patch("app.core.auth.get_platform_store", return_value=store),
+        patch("app.api.v1.endpoints.platform.get_platform_store", return_value=store),
+        patch("app.modules.system.router.get_platform_store", return_value=store),
+        patch("app.modules.tenant.router.get_platform_store", return_value=store),
+        patch("app.modules.project.router.get_platform_store", return_value=store),
+        patch("app.modules.approval.router.get_platform_store", return_value=store),
+        patch("app.modules.retention.router.get_platform_store", return_value=store),
+        patch("app.modules.resource.router.get_platform_store", return_value=store),
+    ]
+    for p in patches:
+        p.start()
+    client = TestClient(app, raise_server_exceptions=False)
+    client._fake_store = store  # type: ignore[attr-defined]
+    return client
+
+
+@pytest.fixture(scope="module")
+def client(fake_store: FakePlatformStore) -> TestClient:
+    c = _build_client(fake_store)
+    yield c
+
+
+def _admin_headers() -> dict[str, str]:
+    return {"Authorization": f"Bearer {ADMIN_TOKEN}"}
+
+
+def _op_headers() -> dict[str, str]:
+    return {"Authorization": f"Bearer {OP_TOKEN}"}
+
+
+# ============================================================
+# 第 1 周测试：登录、当前用户、用户列表、权限码、日志查询
+# ============================================================
+
+class TestWeek1AuthUserPermissionsLogs:
+    """第 1 周：登录、当前用户、用户列表、权限码、日志查询接口。"""
+
+    def test_login_success(self, client: TestClient):
+        resp = client.post(f"{PREFIX}/login", json={"username": "admin", "password": "admin123"})
+        assert resp.status_code == 200
+        data = resp.json()["data"]
+        assert data["token"] == ADMIN_TOKEN
+        assert data["user"]["username"] == "admin"
+
+    def test_login_invalid(self, client: TestClient):
+        resp = client.post(f"{PREFIX}/login", json={"username": "admin", "password": "wrong"})
+        assert resp.status_code == 401
+
+    def test_me_with_valid_token(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/me", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["username"] == "admin"
+
+    def test_me_without_token(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/me")
+        assert resp.status_code == 401
+
+    def test_users_list(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/users", headers=_admin_headers())
+        assert resp.status_code == 200
+        users = resp.json()["data"]
+        assert len(users) >= 2
+        assert any(u["username"] == "admin" for u in users)
+
+    def test_create_user(self, client: TestClient):
+        resp = client.post(
+            f"{PREFIX}/users",
+            json={"username": "tester", "display_name": "Tester", "role": "viewer", "password": "test123"},
+            headers=_admin_headers(),
+        )
+        assert resp.status_code == 200
+        assert resp.json()["data"]["username"] == "tester"
+
+    def test_permission_codes(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/system/permissions/codes")
+        assert resp.status_code == 200
+        codes = resp.json()["data"]["codes"]
+        assert "dashboard" in codes
+        assert "user-settings" in codes
+
+    def test_permissions_overview(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/system/permissions")
+        assert resp.status_code == 200
+        data = resp.json()["data"]
+        assert "codes" in data
+        assert "roles" in data
+
+    def test_log_files(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/log-files", headers=_admin_headers())
+        assert resp.status_code == 200
+        files = resp.json()["data"]
+        assert len(files) >= 1
+
+    def test_log_content(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/log-content", params={"file": "backend.log"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        assert "content" in resp.json()["data"]
+
+    def test_training_log_files(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/training-log-files", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert len(resp.json()["data"]) >= 1
+
+    def test_training_log_content(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/training-log-content", params={"file": "ft_001.log"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        assert "content" in resp.json()["data"]
+
+
+# ============================================================
+# 第 2 周测试：租户、项目、项目成员、资源 ACL
+# ============================================================
+
+class TestWeek2TenantProjectACL:
+    """第 2 周：租户、项目、项目成员、资源 ACL。"""
+
+    def test_tenant_crud(self, client: TestClient):
+        # 创建
+        resp = client.post(f"{PREFIX}/tenants", json={"name": "Tenant-A", "code": "ta"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        tid = resp.json()["data"]["id"]
+        # 查列表
+        resp = client.get(f"{PREFIX}/tenants", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert any(t["id"] == tid for t in resp.json()["data"])
+        # 查详情
+        resp = client.get(f"{PREFIX}/tenants/{tid}", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["name"] == "Tenant-A"
+        # 更新
+        resp = client.put(f"{PREFIX}/tenants/{tid}", json={"name": "Tenant-A2"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["name"] == "Tenant-A2"
+
+    def test_tenant_quota(self, client: TestClient):
+        resp = client.post(f"{PREFIX}/tenants", json={"name": "Q-Tenant", "code": "qt"}, headers=_admin_headers())
+        tid = resp.json()["data"]["id"]
+        resp = client.put(f"{PREFIX}/tenants/{tid}/quota", json={"quota": {"gpu": 4}}, headers=_admin_headers())
+        assert resp.status_code == 200
+
+    def test_tenant_retention(self, client: TestClient):
+        resp = client.post(f"{PREFIX}/tenants", json={"name": "R-Tenant", "code": "rt"}, headers=_admin_headers())
+        tid = resp.json()["data"]["id"]
+        resp = client.put(f"{PREFIX}/tenants/{tid}/retention-policy", json={"retention_policy_id": "rpol_1"}, headers=_admin_headers())
+        assert resp.status_code == 200
+
+    def test_project_crud(self, client: TestClient):
+        # 创建项目
+        resp = client.post(f"{PREFIX}/projects", json={"name": "Proj-1", "code": "p1", "tenant_id": "default"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        pid = resp.json()["data"]["id"]
+        # 查列表
+        resp = client.get(f"{PREFIX}/projects", params={"tenant_id": "default"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        assert any(p["id"] == pid for p in resp.json()["data"])
+        # 查详情
+        resp = client.get(f"{PREFIX}/projects/{pid}", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["name"] == "Proj-1"
+        # 更新
+        resp = client.put(f"{PREFIX}/projects/{pid}", json={"description": "updated"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        # 归档
+        resp = client.post(f"{PREFIX}/projects/{pid}/archive", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["status"] == "archived"
+
+    def test_project_members(self, client: TestClient):
+        resp = client.post(f"{PREFIX}/projects", json={"name": "Proj-M", "code": "pm", "tenant_id": "default"}, headers=_admin_headers())
+        pid = resp.json()["data"]["id"]
+        # 加成员
+        resp = client.post(f"{PREFIX}/projects/{pid}/members", json={"user_id": "u_op", "role": "developer"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        # 列成员
+        resp = client.get(f"{PREFIX}/projects/{pid}/members", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert len(resp.json()["data"]) >= 1
+        # 改角色
+        resp = client.put(f"{PREFIX}/projects/{pid}/members/u_op", json={"role": "maintainer"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        # 删成员
+        resp = client.delete(f"{PREFIX}/projects/{pid}/members/u_op", headers=_admin_headers())
+        assert resp.status_code == 200
+
+    def test_resource_acl(self, client: TestClient):
+        # 设置 ACL
+        resp = client.put(
+            f"{PREFIX}/resources/model/m001/acl",
+            json={"entries": [{"subject_type": "user", "subject_id": "u_op", "permissions": ["read", "write"]}]},
+            headers=_admin_headers(),
+        )
+        assert resp.status_code == 200
+        result = resp.json()["data"]
+        assert len(result) == 1
+        assert set(result[0]["permissions"]) == {"read", "write"}
+        # 查询 ACL
+        resp = client.get(f"{PREFIX}/resources/model/m001/acl", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert len(resp.json()["data"]) == 1
+
+
+# ============================================================
+# 第 3 周测试：审批实例、审批模板、审计日志查询和导出
+# ============================================================
+
+class TestWeek3ApprovalAudit:
+    """第 3 周：审批实例、审批模板、审计日志查询和导出。"""
+
+    def test_approval_template_crud(self, client: TestClient):
+        # 创建模板
+        resp = client.post(f"{PREFIX}/approvals/templates", json={"name": "delete-approval", "steps": [{"approver_id": "u_admin", "status": "pending"}]}, headers=_admin_headers())
+        assert resp.status_code == 200
+        tpl_id = resp.json()["data"]["id"]
+        # 查列表
+        resp = client.get(f"{PREFIX}/approvals/templates", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert any(t["id"] == tpl_id for t in resp.json()["data"])
+
+    def test_approval_instance_flow(self, client: TestClient):
+        # 创建审批实例
+        resp = client.post(f"{PREFIX}/approvals", json={
+            "resource_type": "dataset", "resource_id": "ds_001",
+            "applicant_id": "u_op",
+        }, headers=_admin_headers())
+        assert resp.status_code == 200
+        iid = resp.json()["data"]["id"]
+        # 查详情
+        resp = client.get(f"{PREFIX}/approvals/{iid}", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["status"] == "pending"
+        # 审批决策
+        resp = client.post(f"{PREFIX}/approvals/{iid}/steps/0/decision", json={
+            "approver_id": "u_admin", "approved": True, "comment": "ok",
+        }, headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["status"] == "approved"
+
+    def test_approval_instance_reject(self, client: TestClient):
+        resp = client.post(f"{PREFIX}/approvals", json={
+            "resource_type": "model", "resource_id": "m_002",
+            "applicant_id": "u_op",
+        }, headers=_admin_headers())
+        iid = resp.json()["data"]["id"]
+        resp = client.post(f"{PREFIX}/approvals/{iid}/steps/0/decision", json={
+            "approver_id": "u_admin", "approved": False, "comment": "no",
+        }, headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["status"] == "rejected"
+
+    def test_approval_missing_field(self, client: TestClient):
+        resp = client.post(f"{PREFIX}/approvals", json={"resource_type": "dataset"}, headers=_admin_headers())
+        assert resp.status_code == 400
+
+    def test_audit_logs_query(self, client: TestClient):
+        # 通过 API 写操作触发审计
+        client.post(f"{PREFIX}/tenants", json={"name": "Audit-Tenant", "code": "at"}, headers=_admin_headers())
+        # 查询
+        resp = client.get(f"{PREFIX}/system/audit-logs", params={"limit": 50}, headers=_admin_headers())
+        assert resp.status_code == 200
+        data = resp.json()["data"]
+        assert "items" in data
+        assert "total" in data
+        assert data["total"] >= 1
+
+    def test_audit_logs_filter_by_action(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/system/audit-logs", params={"action": "tenant.create"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        items = resp.json()["data"]["items"]
+        assert all(i.get("action") == "tenant.create" for i in items)
+
+    def test_audit_logs_export_csv(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/system/audit-logs/export", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert "text/csv" in resp.headers.get("content-type", "")
+        # CSV 首行是表头
+        lines = resp.text.strip().split("\n")
+        assert "time" in lines[0]
+
+
+# ============================================================
+# 第 4 周测试：写操作审计、审批拦截、权限校验
+# ============================================================
+
+class TestWeek4AuditInterceptPermission:
+    """第 4 周：写操作审计、审批拦截、权限校验。"""
+
+    def test_write_operation_produces_audit(self, client: TestClient, fake_store: FakePlatformStore):
+        # 清空审计日志便于断言
+        fake_store._audit_logs.clear()
+        # 创建租户 → 应产生 tenant.create 审计
+        client.post(f"{PREFIX}/tenants", json={"name": "W-Tenant", "code": "wt"}, headers=_admin_headers())
+        assert any(l["action"] == "tenant.create" for l in fake_store._audit_logs)
+        # 创建项目 → 应产生 project.create 审计
+        client.post(f"{PREFIX}/projects", json={"name": "W-Proj", "code": "wp", "tenant_id": "default"}, headers=_admin_headers())
+        assert any(l["action"] == "project.create" for l in fake_store._audit_logs)
+        # 设置 ACL → 应产生 resource.acl.set 审计
+        client.put(f"{PREFIX}/resources/model/w001/acl", json={"entries": []}, headers=_admin_headers())
+        assert any(l["action"] == "resource.acl.set" for l in fake_store._audit_logs)
+
+    def test_approval_intercept_on_project_archive(self, client: TestClient, fake_store: FakePlatformStore):
+        # 创建项目
+        resp = client.post(f"{PREFIX}/projects", json={"name": "I-Proj", "code": "ip", "tenant_id": "default"}, headers=_admin_headers())
+        pid = resp.json()["data"]["id"]
+        # 无待审批 → 可归档
+        resp = client.post(f"{PREFIX}/projects/{pid}/archive", headers=_admin_headers())
+        assert resp.status_code == 200
+
+    def test_approval_intercept_blocks_when_pending(self, client: TestClient, fake_store: FakePlatformStore):
+        # 创建项目
+        resp = client.post(f"{PREFIX}/projects", json={"name": "B-Proj", "code": "bp", "tenant_id": "default"}, headers=_admin_headers())
+        pid = resp.json()["data"]["id"]
+        # 注入一条待审批实例
+        fake_store.create_approval_instance({
+            "resource_type": "project",
+            "resource_id": pid,
+            "applicant_id": "u_op",
+        })
+        # 有待审批 → 归档应被拒绝
+        resp = client.post(f"{PREFIX}/projects/{pid}/archive", headers=_admin_headers())
+        assert resp.status_code == 409
+
+    def test_retention_policy_crud_with_audit(self, client: TestClient, fake_store: FakePlatformStore):
+        fake_store._audit_logs.clear()
+        # 创建
+        resp = client.post(f"{PREFIX}/retention-policies", json={"name": "30d-keep", "scope": "tenant"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        rpid = resp.json()["data"]["id"]
+        assert any(l["action"] == "retention.create" for l in fake_store._audit_logs)
+        # 查列表
+        resp = client.get(f"{PREFIX}/retention-policies", headers=_admin_headers())
+        assert resp.status_code == 200
+        assert any(p["id"] == rpid for p in resp.json()["data"])
+        # 更新
+        resp = client.put(f"{PREFIX}/retention-policies/{rpid}", json={"status": "inactive"}, headers=_admin_headers())
+        assert resp.status_code == 200
+        assert resp.json()["data"]["status"] == "inactive"
+        # 删除
+        resp = client.delete(f"{PREFIX}/retention-policies/{rpid}", headers=_admin_headers())
+        assert resp.status_code == 200
+
+    def test_login_duration_rank_in_dashboard(self, client: TestClient):
+        resp = client.get(f"{PREFIX}/dashboard/stats", headers=_admin_headers())
+        assert resp.status_code == 200
+        data = resp.json()["data"]
+        assert "login_duration_rank" in data
+        assert "recent_login_users" in data
+        assert "service_status" in data
+        assert "training_7d" in data

backend/tests/test_platform_dataset_metadata.py

@@ -0,0 +1,103 @@
+from __future__ import annotations
+
+from contextlib import contextmanager
+from typing import Any, Iterator
+
+from app.db.platform_store import PlatformStore, dataset_file_version_summary, parse_size_bytes
+
+
+class _DatasetCursor:
+    def __init__(self, rows: list[dict[str, Any]]) -> None:
+        self.rows = rows
+
+    def fetchall(self) -> list[dict[str, Any]]:
+        return self.rows
+
+
+class _DatasetConnection:
+    def __init__(self) -> None:
+        self.queries: list[str] = []
+
+    def execute(self, sql: str, params: tuple[Any, ...] | None = None) -> _DatasetCursor:
+        self.queries.append(sql)
+        if "FROM datasets dataset" in sql:
+            return _DatasetCursor(
+                [
+                    {
+                        "id": "dataset-train",
+                        "name": "cash-数据集-训练集",
+                        "type": "train",
+                        "storage_type": "local",
+                        "source": "task",
+                        "task_id": "task-cash",
+                        "source_task_id": "task-cash",
+                        "task_name": "cash",
+                        "size": "0 B",
+                        "size_bytes": 0,
+                        "metadata": "{}",
+                    }
+                ]
+            )
+        if "FROM dataset_files" in sql or "FROM dataset_records" in sql:
+            return _DatasetCursor([])
+        raise AssertionError(f"unexpected query: {sql}")
+
+
+def test_parse_size_bytes_supports_legacy_units() -> None:
+    assert parse_size_bytes("21563 B") == 21563
+    assert parse_size_bytes("1.5 KB") == 1536
+    assert parse_size_bytes("2 MB") == 2 * 1024**2
+    assert parse_size_bytes(4096) == 4096
+    assert parse_size_bytes("unknown") == 0
+
+
+def test_dataset_file_version_summary_uses_active_version_metadata() -> None:
+    summary = dataset_file_version_summary(
+        {
+            "active_version_id": "file-1-v3",
+            "current_version_id": "file-1-v1",
+            "version_no": 1,
+            "versions": (
+                '[{"id":"file-1-v1","version":1},'
+                '{"id":"file-1-v3","version_no":3}]'
+            ),
+        }
+    )
+
+    assert summary == {
+        "active_version_id": "file-1-v3",
+        "current_version_id": "file-1-v3",
+        "current_version_no": 3,
+        "version_count": 2,
+    }
+
+
+def test_dataset_file_version_summary_uses_normalized_version_number_as_fallback() -> None:
+    summary = dataset_file_version_summary(
+        {
+            "active_version_id": "",
+            "current_version_id": None,
+            "version_no": 1,
+            "versions": "[]",
+        }
+    )
+
+    assert summary["current_version_no"] == 1
+    assert summary["version_count"] == 0
+
+
+def test_dataset_list_exposes_source_task_name() -> None:
+    store = PlatformStore.__new__(PlatformStore)
+    conn = _DatasetConnection()
+
+    @contextmanager
+    def connect() -> Iterator[_DatasetConnection]:
+        yield conn
+
+    store.connect = connect  # type: ignore[method-assign]
+
+    [dataset] = store.datasets()
+
+    assert dataset["task_name"] == "cash"
+    assert dataset["name"] == "cash-数据集-训练集"
+    assert any("task.name AS task_name" in query for query in conn.queries)

compute/README.md

@@ -0,0 +1,81 @@
+# Compute Platform
+
+算力平台与应用平台分开部署，本目录用于后续实现单机多 GPU 调度、文件网关和训练引擎适配。
+
+## 目录结构
+
+```text
+compute/
+  api/                 # 只允许应用平台访问的内部 Compute API
+  agent/               # 单机 Agent，负责 GPU、进程、工作区管理
+  engines/
+    llama_factory/     # LLaMA-Factory 训练引擎适配器
+  file_gateway/        # 本地磁盘上传、下载、预览、离线导入
+  tests/
+```
+
+## 开发职责
+
+- GPU 发现、状态上报、锁定和释放。
+- 本地磁盘工作区管理。
+- 创建、停止、查询训练/评测/推理/合并任务。
+- LLaMA-Factory 命令生成、日志解析、产物收集。
+- 分片上传、短时下载、离线导入。
+- 通过服务间 token 接受应用平台调用。
+
+## 运行模式
+
+- 默认 `COMPUTE_EXECUTION_MODE=real`，Compute API 会通过 `compute.agent.process_manager.ProcessManager` 启动真实 `llamafactory-cli train` 子进程，并将日志写入 `TRAINING_LOG_ROOT`。
+- 真实模式下 GPU 发现优先使用宿主机 `nvidia-smi`。如果部署环境暂时无法调用 `nvidia-smi`，可通过 `COMPUTE_GPU_COUNT`、`COMPUTE_GPU_NAME`、`COMPUTE_GPU_MEMORY_GB`、`COMPUTE_GPU_POWER_LIMIT_W` 声明兼容 GPU 清单，便于应用侧先完成节点登记和联调。
+- 仅隔离联调时可设置 `COMPUTE_EXECUTION_MODE=simulator`，启用内存状态机和合成 GPU/日志数据。该模式不得作为生产运行路径。
+- 服务间鉴权默认开启：设置 `COMPUTE_AUTH_ENABLED=true` 和一致的 `COMPUTE_SERVICE_TOKEN`，应用侧会通过 `X-Compute-Token` 调用 Compute API。
+- 真实训练作业会登记到 `TRAINING_LOG_ROOT/compute-jobs.json`。Compute API 重启后会恢复作业索引，继续提供状态、停止和日志查询。
+- 同一算力节点内按 GPU ID 做轻量锁定；已有运行中作业占用的 GPU 不允许再次提交，避免同机多 GPU 场景下误复用。
+
+真实执行前提：
+
+- 镜像或宿主机环境中 `llamafactory-cli` 可执行。
+- `LLAMA_FACTORY_HOME` 指向 LLaMA-Factory 工作目录。
+- 基座模型路径和数据集名称/目录已经在算力服务器本地可访问。
+- 应用侧训练任务中的 GPU、模型、数据集配置能映射到当前节点本地路径。
+
+## 应用侧接入
+
+应用平台通过“算力节点”页面维护每台 GPU 服务器的 `Compute API` 和 `File Gateway` 地址。点击连接测试时，Backend API 会主动调用：
+
+```text
+GET /modelTF/v1/compute/health
+GET /modelTF/compute/resources/gpus
+```
+
+连接成功后，应用侧会同步节点健康信息、能力标签和 GPU 清单到 PostgreSQL。多节点阶段仍按“每台算力服务器 = 单机多 GPU 节点”管理，每台服务器都部署 Compute API、Agent、File Gateway 契约和 LLaMA-Factory。
+
+训练闭环：
+
+```text
+Frontend 创建/启动训练
+  -> Backend API 选择 compute_nodes 节点
+  -> Backend API POST /modelTF/compute/jobs 到目标 Compute API
+  -> Compute API 启动 llamafactory-cli 子进程
+  -> Backend Worker 定时 GET /modelTF/compute/jobs/{id}
+  -> Backend API 同步 fine_tune_tasks 状态、进度、PID、日志路径和产物索引
+```
+
+## 当前接口能力
+
+日志接口：
+
+```text
+GET /modelTF/compute/jobs/{job_id}/logs?tail_lines=200
+GET /modelTF/compute/jobs/{job_id}/logs?offset=0&limit=500
+```
+
+返回 `content`、`metrics`、`total_lines`、`offset`、`limit`、`has_more`、`next_offset`，用于前端增量刷新和日志平台采集。
+
+文件导入：
+
+```text
+POST /modelTF/compute/files/import-local
+```
+
+该接口用于应用侧调度前把算力服务器本地可访问的模型/数据集路径导入到 `YG_FT_DATA_ROOT` 内部。目标路径会校验不能逃逸出 `YG_FT_DATA_ROOT`，源路径必须已存在于算力服务器本地或挂载目录。

compute/__init__.py

@@ -0,0 +1 @@
+"""Compute platform package."""

compute/agent/__init__.py

@@ -0,0 +1 @@
+"""Compute agent package."""

compute/agent/process_manager.py

@@ -0,0 +1,281 @@
+from __future__ import annotations
+
+import os
+import json
+import contextlib
+import hashlib
+import signal
+import subprocess
+import time
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any
+
+
+TERMINAL_STATUSES = {"completed", "failed", "stopped"}
+
+
+@dataclass
+class ManagedProcess:
+    id: str
+    name: str
+    command: list[str]
+    work_dir: str
+    log_path: Path
+    output_dir: str
+    gpus: list[int]
+    process: subprocess.Popen[Any] | None
+    created_at: float
+    pid: int | None = None
+    status: str = "running"
+    progress: int = 5
+    artifacts: list[dict[str, Any]] = field(default_factory=list)
+
+
+class ProcessManager:
+    def __init__(self, log_root: str) -> None:
+        self.log_root = Path(log_root)
+        self.log_root.mkdir(parents=True, exist_ok=True)
+        self.registry_path = self.log_root / "compute-jobs.json"
+        self.jobs: dict[str, ManagedProcess] = {}
+        self._load_registry()
+
+    def create_job(self, payload: dict[str, Any], command: list[str], work_dir: str) -> dict[str, Any]:
+        job_id = str(payload.get("id") or f"job_{int(time.time() * 1000)}")
+        if job_id in self.jobs and self.jobs[job_id].status not in TERMINAL_STATUSES:
+            raise ValueError(f"job {job_id} is already running")
+
+        output_dir = str(payload.get("output_dir") or f"/data/yg-ft/outputs/{payload.get('name', job_id)}")
+        Path(output_dir).mkdir(parents=True, exist_ok=True)
+        log_path = self.log_root / f"{job_id}.log"
+        env = os.environ.copy()
+        gpus = [int(item) for item in payload.get("gpus") or []]
+        locked = self.locked_gpus()
+        conflict = sorted(set(gpus).intersection(locked))
+        if conflict:
+            raise ValueError(f"gpu already locked: {conflict}")
+        if gpus:
+            env["CUDA_VISIBLE_DEVICES"] = ",".join(str(item) for item in gpus)
+        env.update({str(k): str(v) for k, v in payload.get("env", {}).items()})
+
+        cwd = work_dir if Path(work_dir).exists() else None
+        with log_path.open("ab") as log_file:
+            log_file.write(f"[INFO] starting job_id={job_id} command={' '.join(command)}\n".encode("utf-8"))
+            process = subprocess.Popen(
+                command,
+                cwd=cwd,
+                env=env,
+                stdout=log_file,
+                stderr=subprocess.STDOUT,
+            )
+
+        managed = ManagedProcess(
+            id=job_id,
+            name=str(payload.get("name") or job_id),
+            command=command,
+            work_dir=work_dir,
+            log_path=log_path,
+            output_dir=output_dir,
+            gpus=gpus,
+            process=process,
+            created_at=time.time(),
+            pid=process.pid,
+            progress=10,
+        )
+        self.jobs[job_id] = managed
+        data = self.serialize(managed)
+        self._save_registry()
+        return data
+
+    def get_job(self, job_id: str) -> dict[str, Any] | None:
+        job = self.jobs.get(job_id)
+        if not job:
+            return None
+        return self.serialize(job)
+
+    def list_jobs(self) -> list[dict[str, Any]]:
+        return [self.serialize(job) for job in self.jobs.values()]
+
+    def stop_job(self, job_id: str) -> dict[str, Any] | None:
+        job = self.jobs.get(job_id)
+        if not job:
+            return None
+        if job.status not in TERMINAL_STATUSES:
+            try:
+                if job.process is not None and os.name == "nt":
+                    job.process.terminate()
+                elif job.pid is not None:
+                    os.kill(job.pid, signal.SIGTERM)
+                if job.process is not None:
+                    job.process.wait(timeout=10)
+            except Exception:
+                if job.process is not None:
+                    job.process.kill()
+                elif job.pid is not None:
+                    with contextlib.suppress(Exception):
+                        os.kill(job.pid, signal.SIGKILL)
+            job.status = "stopped"
+            job.progress = min(job.progress, 99)
+        data = self.serialize(job)
+        self._save_registry()
+        return data
+
+    def logs(self, job_id: str) -> str:
+        job = self.jobs.get(job_id)
+        if not job or not job.log_path.exists():
+            return ""
+        return job.log_path.read_text(encoding="utf-8", errors="replace")
+
+    def serialize(self, job: ManagedProcess) -> dict[str, Any]:
+        code = job.process.poll() if job.process is not None else None
+        checkpoints = self._collect_checkpoints(job.output_dir)
+        if job.status not in TERMINAL_STATUSES:
+            if job.process is None and job.pid is not None and not self._pid_alive(job.pid):
+                job.status = "failed"
+                job.progress = min(job.progress, 99)
+                code = -1
+            elif code is None:
+                job.status = "running"
+                elapsed = max(0, int(time.time() - job.created_at))
+                job.progress = min(95, max(job.progress, 10 + elapsed // 6))
+            elif code == 0:
+                job.status = "completed"
+                job.progress = 100
+                job.artifacts = self._collect_artifacts(job.output_dir)
+            else:
+                job.status = "failed"
+                job.progress = min(job.progress, 99)
+        self._save_registry()
+        return {
+            "id": job.id,
+            "name": job.name,
+            "status": job.status,
+            "progress": job.progress,
+            "pid": job.pid,
+            "gpus": job.gpus,
+            "created_at": job.created_at,
+            "command": job.command,
+            "work_dir": job.work_dir,
+            "output_dir": job.output_dir,
+            "log_file": str(job.log_path),
+            "artifacts": job.artifacts,
+            "checkpoints": checkpoints,
+            "return_code": code,
+        }
+
+    def locked_gpus(self) -> set[int]:
+        locked: set[int] = set()
+        for job in self.jobs.values():
+            status = self.serialize(job)["status"]
+            if status in {"queued", "running"}:
+                locked.update(job.gpus)
+        return locked
+
+    def _collect_artifacts(self, output_dir: str) -> list[dict[str, Any]]:
+        root = Path(output_dir)
+        if not root.exists():
+            return []
+        artifacts: list[dict[str, Any]] = []
+        for path in root.rglob("*"):
+            if path.is_file():
+                digest = hashlib.sha256()
+                with path.open("rb") as handle:
+                    for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+                        digest.update(chunk)
+                size = path.stat().st_size
+                artifacts.append(
+                    {
+                        "path": str(path),
+                        "name": path.name,
+                        "size": size,
+                        "size_bytes": size,
+                        "checksum_sha256": digest.hexdigest(),
+                    }
+                )
+        return artifacts[:200]
+
+    def _collect_checkpoints(self, output_dir: str) -> list[dict[str, Any]]:
+        root = Path(output_dir)
+        if not root.exists():
+            return []
+        checkpoints: list[dict[str, Any]] = []
+        for path in root.glob("checkpoint-*"):
+            if not path.is_dir():
+                continue
+            step = 0
+            try:
+                step = int(path.name.rsplit("-", 1)[-1])
+            except ValueError:
+                step = 0
+            size_bytes = sum(item.stat().st_size for item in path.rglob("*") if item.is_file())
+            checkpoints.append(
+                {
+                    "step": step,
+                    "name": path.name,
+                    "path": str(path),
+                    "size_bytes": size_bytes,
+                    "create_time": path.stat().st_mtime,
+                }
+            )
+        return sorted(checkpoints, key=lambda item: (int(item.get("step") or 0), str(item.get("name") or "")))
+
+    def _save_registry(self) -> None:
+        items = []
+        for job in self.jobs.values():
+            items.append(
+                {
+                    "id": job.id,
+                    "name": job.name,
+                    "command": job.command,
+                    "work_dir": job.work_dir,
+                    "log_path": str(job.log_path),
+                    "output_dir": job.output_dir,
+                    "gpus": job.gpus,
+                    "pid": job.pid,
+                    "created_at": job.created_at,
+                    "status": job.status,
+                    "progress": job.progress,
+                    "artifacts": job.artifacts,
+                }
+            )
+        self.registry_path.write_text(json.dumps(items, ensure_ascii=False, indent=2), encoding="utf-8")
+
+    def _load_registry(self) -> None:
+        if not self.registry_path.exists():
+            return
+        try:
+            items = json.loads(self.registry_path.read_text(encoding="utf-8"))
+        except json.JSONDecodeError:
+            return
+        for item in items if isinstance(items, list) else []:
+            if not isinstance(item, dict):
+                continue
+            pid = item.get("pid")
+            status = item.get("status", "failed")
+            if status not in TERMINAL_STATUSES and pid and not self._pid_alive(int(pid)):
+                status = "failed"
+            job = ManagedProcess(
+                id=str(item["id"]),
+                name=str(item.get("name") or item["id"]),
+                command=[str(part) for part in item.get("command") or []],
+                work_dir=str(item.get("work_dir") or ""),
+                log_path=Path(item.get("log_path") or self.log_root / f"{item['id']}.log"),
+                output_dir=str(item.get("output_dir") or ""),
+                gpus=[int(gpu) for gpu in item.get("gpus") or []],
+                process=None,
+                pid=int(pid) if pid else None,
+                created_at=float(item.get("created_at") or time.time()),
+                status=status,
+                progress=int(item.get("progress") or 0),
+                artifacts=item.get("artifacts") or [],
+            )
+            self.jobs[job.id] = job
+
+    def _pid_alive(self, pid: int) -> bool:
+        if pid <= 0:
+            return False
+        try:
+            os.kill(pid, 0)
+            return True
+        except OSError:
+            return False

compute/api/__init__.py

@@ -0,0 +1 @@
+"""Compute API package."""

compute/api/main.py

@@ -0,0 +1,871 @@
+from __future__ import annotations
+
+import asyncio
+import json
+import os
+import math
+import hashlib
+import shutil
+import subprocess
+import time
+from pathlib import Path
+from typing import Any
+
+from fastapi import FastAPI, File, Form, HTTPException, Query, Request, UploadFile
+from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
+
+from compute.agent.process_manager import ProcessManager
+from compute.engines.llama_factory.adapter import build_command, parse_log_line, prepare_runtime_files
+from compute.engines.llama_factory.inference import get_inference_session
+
+
+def create_app() -> FastAPI:
+    app = FastAPI(title="YG Fine-Tune Compute API")
+    jobs: dict[str, dict[str, Any]] = {}
+    route_prefix = os.getenv("MODELTF_ROUTE_PREFIX", "/modelTF").rstrip("/") or "/modelTF"
+    process_manager = ProcessManager(os.getenv("TRAINING_LOG_ROOT", "/opt/yg-ft/logs/training"))
+
+    @app.middleware("http")
+    async def compute_token_auth(request: Request, call_next):
+        token = os.getenv("COMPUTE_SERVICE_TOKEN", "")
+        auth_enabled = os.getenv("COMPUTE_AUTH_ENABLED", "true").lower() == "true"
+        public_paths = {f"{route_prefix}/health", "/health"}
+        if auth_enabled and token and request.url.path not in public_paths:
+            header_token = request.headers.get("x-compute-token", "")
+            auth_header = request.headers.get("authorization", "")
+            bearer_token = auth_header.removeprefix("Bearer ").strip() if auth_header.startswith("Bearer ") else ""
+            if header_token != token and bearer_token != token:
+                return JSONResponse({"detail": "invalid compute service token"}, status_code=401)
+        return await call_next(request)
+
+    def now() -> float:
+        return time.time()
+
+    def host_id() -> str:
+        return os.getenv("COMPUTE_HOST_ID", "gpu-node-01")
+
+    def execution_mode() -> str:
+        return os.getenv("COMPUTE_EXECUTION_MODE", os.getenv("COMPUTE_MODE", "real")).lower()
+
+    def _int_env(name: str, default: int) -> int:
+        raw = os.getenv(name)
+        if raw is None or raw == "":
+            return default
+        return int(raw)
+
+    def _float_env(name: str, default: float) -> float:
+        raw = os.getenv(name)
+        if raw is None or raw == "":
+            return default
+        return float(raw)
+
+    def _path_inside(root: Path, candidate: Path) -> bool:
+        try:
+            candidate.resolve().relative_to(root.resolve())
+            return True
+        except ValueError:
+            return False
+
+    def _llama_factory_version() -> str:
+        for command in (["llamafactory-cli", "version"], ["llamafactory-cli", "--version"]):
+            try:
+                result = subprocess.run(command, capture_output=True, text=True, timeout=5)
+            except Exception:
+                continue
+            output = (result.stdout or result.stderr).strip()
+            if result.returncode == 0 and output:
+                return output.splitlines()[0][:120]
+        return ""
+
+    def torch_cuda_status() -> dict[str, Any]:
+        try:
+            import torch  # type: ignore[import-not-found]
+        except Exception as exc:  # noqa: BLE001 - keep health endpoint resilient
+            return {
+                "available": False,
+                "device_count": 0,
+                "torch_version": "",
+                "torch_cuda_version": "",
+                "error": f"torch import failed: {exc}",
+            }
+        try:
+            available = bool(torch.cuda.is_available())
+            device_count = int(torch.cuda.device_count())
+            devices = []
+            for index in range(device_count):
+                props = torch.cuda.get_device_properties(index)
+                devices.append(
+                    {
+                        "index": index,
+                        "name": props.name,
+                        "memory_total_gb": round(props.total_memory / 1024 / 1024 / 1024, 2),
+                    }
+                )
+            return {
+                "available": available,
+                "device_count": device_count,
+                "torch_version": str(torch.__version__),
+                "torch_cuda_version": str(torch.version.cuda or ""),
+                "devices": devices,
+                "error": "" if available else "torch cuda is not available",
+            }
+        except Exception as exc:  # noqa: BLE001 - expose CUDA initialization failures
+            return {
+                "available": False,
+                "device_count": 0,
+                "torch_version": str(getattr(torch, "__version__", "")),
+                "torch_cuda_version": str(getattr(torch.version, "cuda", "") or ""),
+                "devices": [],
+                "error": str(exc),
+            }
+
+    def _slice_log_content(
+        content: str,
+        tail_lines: int | None = None,
+        offset: int | None = None,
+        limit: int | None = None,
+    ) -> dict[str, Any]:
+        lines = content.splitlines()
+        total = len(lines)
+        if offset is not None or limit is not None:
+            start = max(0, offset or 0)
+            end = start + limit if limit else total
+            selected = lines[start:end]
+        else:
+            tail = tail_lines or 200
+            start = max(0, total - tail)
+            selected = lines[start:]
+        next_offset = start + len(selected)
+        return {
+            "content": "\n".join(selected),
+            "total_lines": total,
+            "offset": start,
+            "limit": len(selected),
+            "has_more": next_offset < total,
+            "next_offset": next_offset if next_offset < total else None,
+        }
+
+    def _safe_float(value: Any, default: float = 0) -> float:
+        try:
+            return float(str(value).replace("[N/A]", "").strip() or default)
+        except (TypeError, ValueError):
+            return default
+
+    def job_status(job: dict[str, Any]) -> dict[str, Any]:
+        if execution_mode() != "simulator":
+            return job
+        elapsed = max(0, int(now() - job["created_at"]))
+        if job["status"] not in {"stopped", "failed", "completed"}:
+            if elapsed < 5:
+                job["status"] = "queued"
+                job["progress"] = 12 + elapsed * 3
+            elif elapsed < 60:
+                job["status"] = "running"
+                job["progress"] = min(96, 25 + int((elapsed - 5) / 55 * 70))
+            else:
+                job["status"] = "completed"
+                job["progress"] = 100
+        job["logs"] = generate_logs(job)
+        return job
+
+    def generate_logs(job: dict[str, Any]) -> str:
+        progress = int(job.get("progress", 0) or 0)
+        points = max(1, min(80, progress))
+        lines = [
+            f"[INFO] compute_host_id={host_id()} job_id={job['id']} engine=llama_factory",
+            f"[INFO] command={' '.join(job['command'])}",
+        ]
+        for step in range(1, points + 1):
+            if step % 4 != 0 and step != points:
+                continue
+            loss = max(0.11, 2.5 * math.exp(-step / 40))
+            grad_norm = 0.4 + (step % 5) * 0.04
+            lr = 0.0002 * max(0.05, 1 - step / 100)
+            epoch = round(step / points * 3, 4)
+            lines.append(
+                "{"
+                f"'loss': {loss:.4f}, 'grad_norm': {grad_norm:.4f}, "
+                f"'learning_rate': {lr:.8f}, 'epoch': {epoch:.4f}"
+                "}"
+            )
+        if job.get("status") == "completed":
+            lines.extend(
+                [
+                    "***** train metrics *****",
+                    "epoch = 3",
+                    "train_loss = 0.1181",
+                    "train_runtime = 1m 0s",
+                    "***** train metrics end *****",
+                ]
+            )
+        return "\n".join(lines)
+
+    def real_gpu_resources() -> list[dict[str, Any]]:
+        query = (
+            "index,uuid,name,memory.total,memory.used,utilization.gpu,"
+            "temperature.gpu,power.draw,power.limit"
+        )
+        try:
+            result = subprocess.run(
+                ["nvidia-smi", f"--query-gpu={query}", "--format=csv,noheader,nounits"],
+                check=True,
+                capture_output=True,
+                text=True,
+                timeout=5,
+            )
+        except Exception:
+            return fallback_gpu_resources()
+
+        items: list[dict[str, Any]] = []
+        for line in result.stdout.splitlines():
+            parts = [part.strip() for part in line.split(",")]
+            if len(parts) < 9:
+                continue
+            idx, uuid, name, mem_total, mem_used, util, temp, power, power_limit = parts[:9]
+            total_gb = round(_safe_float(mem_total) / 1024, 2)
+            used_gb = round(_safe_float(mem_used) / 1024, 2)
+            memory_percent = round(used_gb / total_gb * 100, 1) if total_gb else 0
+            gpu_percent = int(_safe_float(util))
+            items.append(
+                {
+                    "id": int(idx),
+                    "gpu_index": int(idx),
+                    "uuid": uuid,
+                    "name": name,
+                    "status": "busy" if gpu_percent >= 5 or used_gb > 1 else "idle",
+                    "gpu_percent": gpu_percent,
+                    "memory_used_gb": used_gb,
+                    "memory_total_gb": total_gb,
+                    "memory_percent": memory_percent,
+                    "temperature": int(_safe_float(temp)),
+                    "power_w": round(_safe_float(power), 1),
+                    "power_limit_w": round(_safe_float(power_limit), 1),
+                    "processes": [],
+                }
+            )
+        return items
+
+    def fallback_gpu_resources() -> list[dict[str, Any]]:
+        count = _int_env("COMPUTE_GPU_COUNT", 0)
+        if count <= 0:
+            return []
+        name = os.getenv("COMPUTE_GPU_NAME", "Configured GPU")
+        memory_total = _float_env("COMPUTE_GPU_MEMORY_GB", 80.0)
+        power_limit = _float_env("COMPUTE_GPU_POWER_LIMIT_W", 300.0)
+        return [
+            {
+                "id": idx,
+                "gpu_index": idx,
+                "uuid": f"GPU-{host_id().upper()}-{idx}",
+                "name": name,
+                "status": "idle",
+                "gpu_percent": 0,
+                "memory_used_gb": 0,
+                "memory_total_gb": memory_total,
+                "memory_percent": 0,
+                "temperature": _int_env("COMPUTE_GPU_BASE_TEMPERATURE", 35),
+                "power_w": 0,
+                "power_limit_w": power_limit,
+                "processes": [],
+            }
+            for idx in range(count)
+        ]
+
+    def gpu_resources() -> list[dict[str, Any]]:
+        if execution_mode() != "simulator":
+            return real_gpu_resources()
+        active_jobs = [job_status(job) for job in jobs.values() if job["status"] in {"queued", "running"}]
+        gpus: list[dict[str, Any]] = []
+        for idx in range(4):
+            task = next((job for job in active_jobs if idx in job.get("gpus", [])), None)
+            busy = task is not None and task["status"] == "running"
+            reserved = task is not None and task["status"] == "queued"
+            gpus.append(
+                {
+                    "id": idx,
+                    "uuid": f"GPU-{host_id().upper()}-{idx}",
+                    "name": os.getenv("COMPUTE_GPU_NAME", "NVIDIA A800-SXM4-80GB"),
+                    "status": "busy" if busy else "reserved" if reserved else "idle",
+                    "gpu_percent": 88 if busy else 25 if reserved else 4,
+                    "memory_used_gb": 58 if busy else 12 if reserved else 2,
+                    "memory_total_gb": 80,
+                    "temperature": 61 if busy else 45 if reserved else 36,
+                    "power_w": 215 if busy else 80 if reserved else 25,
+                    "power_limit_w": 300,
+                    "processes": [
+                        {
+                            "pid": task["pid"],
+                            "name": "llamafactory-cli",
+                            "task_name": task["name"],
+                            "memory_used_gb": 58 if busy else 12,
+                        }
+                    ]
+                    if task
+                    else [],
+                }
+            )
+        return gpus
+
+    def _validate_training_accelerator(payload: dict[str, Any]) -> tuple[list[str], list[str], dict[str, Any]]:
+        errors: list[str] = []
+        warnings: list[str] = []
+        if str(payload.get("engine") or payload.get("training_engine") or "llama_factory") == "smoke":
+            return errors, warnings, {}
+        requested_gpus = [int(item) for item in payload.get("gpus") or []]
+        if not requested_gpus:
+            warnings.append("no gpu selected; training will run on CPU")
+            return errors, warnings, {}
+        cuda = torch_cuda_status()
+        if not cuda.get("available"):
+            errors.append(f"torch cuda unavailable on compute node: {cuda.get('error') or 'unknown error'}")
+        device_count = int(cuda.get("device_count") or 0)
+        if device_count and max(requested_gpus) >= device_count:
+            errors.append(f"requested gpu index out of torch device range: requested={requested_gpus}, device_count={device_count}")
+        min_memory_gb = _float_env("MIN_TRAINING_GPU_MEMORY_GB", 4.0)
+        gpus = {int(item["gpu_index"]): item for item in gpu_resources() if "gpu_index" in item}
+        for gpu_index in requested_gpus:
+            gpu = gpus.get(gpu_index)
+            if not gpu:
+                errors.append(f"requested gpu not found by nvidia-smi: {gpu_index}")
+                continue
+            memory_total = float(gpu.get("memory_total_gb") or 0)
+            if memory_total and memory_total < min_memory_gb:
+                errors.append(
+                    f"gpu {gpu_index} memory too small: {memory_total}GB < required {min_memory_gb}GB"
+                )
+        return errors, warnings, cuda
+
+    def _check_path_item(item: dict[str, Any]) -> dict[str, Any]:
+        path = Path(str(item.get("path") or ""))
+        exists = path.exists()
+        expected_type = str(item.get("type") or "any")
+        ok = exists
+        if exists and expected_type == "dir":
+            ok = path.is_dir()
+        if exists and expected_type == "file":
+            ok = path.is_file()
+        return {
+            "name": item.get("name") or "",
+            "path": str(path),
+            "type": expected_type,
+            "required": bool(item.get("required", True)),
+            "exists": exists,
+            "is_dir": path.is_dir() if exists else False,
+            "is_file": path.is_file() if exists else False,
+            "byte_size": sum(child.stat().st_size for child in path.rglob("*") if child.is_file()) if exists and path.is_dir() else path.stat().st_size if exists and path.is_file() else 0,
+            "ok": ok or not item.get("required", True),
+        }
+
+    def _job_preview(payload: dict[str, Any], check_paths: bool) -> dict[str, Any]:
+        warnings: list[str] = []
+        runtime_files: list[dict[str, str]] = []
+        command_payload = {**payload, "require_dataset_files": check_paths}
+        if check_paths:
+            try:
+                runtime_files = prepare_runtime_files(command_payload)
+            except OSError as exc:
+                return {
+                    "valid": False,
+                    "errors": [f"prepare runtime files failed: {exc}"],
+                    "warnings": warnings,
+                    "engine": str(payload.get("engine") or payload.get("training_engine") or "llama_factory"),
+                    "command": [],
+                    "command_text": "",
+                    "work_dir": os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"),
+                    "env": {},
+                    "runtime_files": [],
+                    "path_checks": [],
+                }
+        try:
+            command = build_command(command_payload, os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
+        except ValueError as exc:
+            return {
+                "valid": False,
+                "errors": [part.strip() for part in str(exc).split(";") if part.strip()],
+                "warnings": warnings,
+                "engine": str(payload.get("engine") or payload.get("training_engine") or "llama_factory"),
+                "command": [],
+                "command_text": "",
+                "work_dir": os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"),
+                "env": {},
+                "runtime_files": runtime_files,
+                "path_checks": [],
+            }
+
+        errors: list[str] = []
+        engine = str(payload.get("engine") or payload.get("training_engine") or "llama_factory")
+        path_checks: list[dict[str, Any]] = []
+        accelerator: dict[str, Any] = {}
+        if check_paths and engine != "smoke":
+            path_checks = [
+                _check_path_item(
+                    {
+                        "name": "model_name_or_path",
+                        "path": payload.get("model_name_or_path") or payload.get("base_model") or payload.get("base_model_path") or "",
+                        "type": "any",
+                        "required": True,
+                    }
+                )
+            ]
+            if engine in {"merge", "export", "llama_factory_export"} and payload.get("adapter_name_or_path"):
+                path_checks.append(
+                    _check_path_item(
+                        {
+                            "name": "adapter_name_or_path",
+                            "path": payload.get("adapter_name_or_path"),
+                            "type": "any",
+                            "required": True,
+                        }
+                    )
+                )
+            if payload.get("dataset_dir"):
+                path_checks.append(
+                    _check_path_item(
+                        {
+                            "name": "dataset_dir",
+                            "path": payload.get("dataset_dir"),
+                            "type": "dir",
+                            "required": True,
+                        }
+                    )
+                )
+            output_dir = Path(str(payload.get("output_dir") or "/data/yg-ft/outputs/training-job"))
+            path_checks.append(
+                _check_path_item(
+                    {
+                        "name": "output_parent",
+                        "path": str(output_dir.parent),
+                        "type": "dir",
+                        "required": False,
+                    }
+                )
+            )
+            errors.extend(
+                [f"{item['name']} path not available: {item['path']}" for item in path_checks if not item["ok"] and item["required"]]
+            )
+            if shutil.which(command.command[0]) is None:
+                errors.append(f"training command not found: {command.command[0]}")
+            if not Path(command.work_dir).exists():
+                errors.append(f"llama_factory_home not found: {command.work_dir}")
+            if engine not in {"merge", "export", "llama_factory_export"}:
+                accelerator_errors, accelerator_warnings, accelerator = _validate_training_accelerator(payload)
+                errors.extend(accelerator_errors)
+                warnings.extend(accelerator_warnings)
+        elif engine == "eval":
+            # Eval engine: validate model path and dataset path
+            if not payload.get("model_name_or_path"):
+                errors.append("model_name_or_path is required for eval")
+            else:
+                path_checks.append(_check_path_item({
+                    "name": "model_name_or_path",
+                    "path": payload.get("model_name_or_path", ""),
+                    "type": "any",
+                    "required": True,
+                }))
+            if payload.get("dataset_path"):
+                path_checks.append(_check_path_item({
+                    "name": "dataset_path",
+                    "path": payload.get("dataset_path", ""),
+                    "type": "file",
+                    "required": True,
+                }))
+            else:
+                errors.append("dataset_path is required for eval")
+            if shutil.which("python") is None:
+                errors.append("python runtime not found")
+
+        elif engine == "smoke":
+            warnings.append("smoke engine skips model and dataset path checks")
+
+        return {
+            "valid": not errors,
+            "errors": errors,
+            "warnings": warnings,
+            "engine": engine,
+            "command": command.command,
+            "command_text": " ".join(command.command),
+            "work_dir": command.work_dir,
+            "env": command.env,
+            "runtime_files": runtime_files,
+            "accelerator": accelerator,
+            "path_checks": path_checks,
+        }
+
+    @app.get(f"{route_prefix}/health")
+    async def health_check() -> dict[str, str]:
+        return {
+            "status": "ok",
+            "compute_host_id": os.getenv("COMPUTE_HOST_ID", "unknown"),
+        }
+
+    @app.get("/health")
+    async def health_check_root() -> dict[str, str]:
+        return await health_check()
+
+    @app.get(f"{route_prefix}/v1/compute/health")
+    async def compute_health_check() -> dict[str, Any]:
+        data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
+        dataset_root = Path(os.getenv("YG_FT_DATASET_ROOT", str(data_root / "datasets")))
+        output_root = Path(os.getenv("YG_FT_OUTPUT_ROOT", str(data_root / "outputs")))
+        llama_factory_home = Path(os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
+        gpu_items = gpu_resources()
+        torch_cuda = torch_cuda_status()
+        return {
+            "status": "ok",
+            "api_version": "v1",
+            "compute_host_id": os.getenv("COMPUTE_HOST_ID", "unknown"),
+            "app_callback_enabled": os.getenv("ENABLE_APP_CALLBACK", "false").lower() == "true",
+            "data_root": str(data_root),
+            "data_root_exists": data_root.exists(),
+            "model_root": os.getenv("YG_FT_MODEL_ROOT", str(data_root / "models")),
+            "dataset_root": str(dataset_root),
+            "dataset_root_exists": dataset_root.exists(),
+            "output_root": str(output_root),
+            "output_root_exists": output_root.exists(),
+            "log_root": os.getenv("TRAINING_LOG_ROOT", "/opt/yg-ft/logs/training"),
+            "llama_factory_home": str(llama_factory_home),
+            "llama_factory_home_exists": llama_factory_home.exists(),
+            "llama_factory_version": os.getenv("LLAMA_FACTORY_VERSION", ""),
+            "execution_mode": execution_mode(),
+            "gpu_count": _int_env("COMPUTE_GPU_COUNT", 0),
+            "nvidia_gpu_count": len(gpu_items),
+            "torch_cuda": torch_cuda,
+            "gpu_discovery_endpoint": f"{route_prefix}/compute/resources/gpus",
+            "capabilities": ["gpu_discovery", "torch_cuda_diagnostics", "llama_factory", "file_gateway", "job_polling"],
+        }
+
+    @app.get(f"{route_prefix}/v1/compute/jobs")
+    async def list_jobs_alias() -> dict[str, list[dict[str, Any]]]:
+        items = process_manager.list_jobs() if execution_mode() != "simulator" else [job_status(job) for job in jobs.values()]
+        return {"items": items}
+
+    @app.get(f"{route_prefix}/compute/resources/gpus")
+    async def list_gpus() -> dict[str, Any]:
+        return {"items": gpu_resources(), "compute_host_id": host_id()}
+
+    @app.get(f"{route_prefix}/v1/compute/resources/gpus")
+    async def list_gpus_v1() -> dict[str, Any]:
+        return {"items": gpu_resources(), "compute_host_id": host_id()}
+
+    @app.post(f"{route_prefix}/compute/jobs/preview")
+    async def preview_job(payload: dict[str, Any]) -> dict[str, Any]:
+        return _job_preview(payload, check_paths=False)
+
+    @app.post(f"{route_prefix}/compute/jobs/validate")
+    async def validate_job(payload: dict[str, Any]) -> dict[str, Any]:
+        return _job_preview(payload, check_paths=True)
+
+    @app.post(f"{route_prefix}/v1/compute/jobs/preview")
+    async def preview_job_v1(payload: dict[str, Any]) -> dict[str, Any]:
+        return await preview_job(payload)
+
+    @app.post(f"{route_prefix}/v1/compute/jobs/validate")
+    async def validate_job_v1(payload: dict[str, Any]) -> dict[str, Any]:
+        return await validate_job(payload)
+
+    @app.post(f"{route_prefix}/compute/files/check-paths")
+    async def check_paths(payload: dict[str, Any]) -> dict[str, Any]:
+        items = [_check_path_item(item) for item in payload.get("paths", []) if isinstance(item, dict)]
+        return {"valid": all(item["ok"] for item in items), "items": items}
+
+    @app.get(f"{route_prefix}/compute/files/list")
+    async def list_files(
+        root: str = Query(default="data"),
+        relative_path: str = Query(default=""),
+        directories_only: bool = Query(default=False),
+    ) -> dict[str, Any]:
+        roots = {
+            "data": Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft")),
+            "models": Path(os.getenv("YG_FT_MODEL_ROOT", os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft") + "/models")),
+            "datasets": Path(os.getenv("YG_FT_DATASET_ROOT", os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft") + "/datasets")),
+            "outputs": Path(os.getenv("YG_FT_OUTPUT_ROOT", os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft") + "/outputs")),
+        }
+        base = roots.get(root)
+        if base is None:
+            raise HTTPException(status_code=400, detail="invalid root")
+        target = (base / relative_path.lstrip("/\\")).resolve()
+        if not _path_inside(base, target):
+            raise HTTPException(status_code=400, detail="path must stay inside selected root")
+        if not target.exists():
+            return {"root": root, "base_path": str(base), "relative_path": relative_path, "items": []}
+        items = []
+        for child in sorted(target.iterdir(), key=lambda path: (not path.is_dir(), path.name.lower())):
+            if directories_only and not child.is_dir():
+                continue
+            items.append(
+                {
+                    "name": child.name,
+                    "path": str(child),
+                    "relative_path": str(child.relative_to(base)).replace("\\", "/"),
+                    "type": "directory" if child.is_dir() else "file",
+                    "byte_size": child.stat().st_size if child.is_file() else 0,
+                }
+            )
+        return {"root": root, "base_path": str(base), "relative_path": relative_path, "items": items}
+
+    @app.post(f"{route_prefix}/compute/jobs")
+    async def create_job(payload: dict[str, Any]) -> dict[str, Any]:
+        payload = {**payload, "require_dataset_files": True}
+        try:
+            prepare_runtime_files(payload)
+        except OSError as exc:
+            raise HTTPException(status_code=400, detail=f"prepare runtime files failed: {exc}")
+        try:
+            command = build_command(payload, os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
+        except ValueError as exc:
+            raise HTTPException(status_code=400, detail=str(exc))
+        job_id = str(payload.get("id") or f"job_{int(now() * 1000)}")
+        if execution_mode() != "simulator":
+            try:
+                return process_manager.create_job({**payload, "id": job_id}, command.command, command.work_dir)
+            except FileNotFoundError as exc:
+                raise HTTPException(status_code=500, detail=f"training command not found: {exc.filename}")
+            except ValueError as exc:
+                raise HTTPException(status_code=409, detail=str(exc))
+        job = {
+            "id": job_id,
+            "name": payload.get("name", job_id),
+            "status": "queued",
+            "progress": 10,
+            "pid": int(52000 + now() % 10000),
+            "gpus": payload.get("gpus") or [0],
+            "created_at": now(),
+            "command": command.command,
+            "work_dir": command.work_dir,
+            "artifacts": [],
+            "logs": "",
+        }
+        jobs[job_id] = job
+        return job_status(job)
+
+    @app.get(f"{route_prefix}/compute/jobs")
+    async def list_jobs() -> dict[str, Any]:
+        items = process_manager.list_jobs() if execution_mode() != "simulator" else [job_status(job) for job in jobs.values()]
+        return {"items": items}
+
+    @app.get(f"{route_prefix}/compute/jobs/{{job_id}}")
+    async def get_job(job_id: str) -> dict[str, Any]:
+        job = jobs.get(job_id)
+        if execution_mode() != "simulator":
+            job = process_manager.get_job(job_id)
+            if not job:
+                raise HTTPException(status_code=404, detail="job not found")
+            return job
+        job = jobs.get(job_id)
+        if not job:
+            raise HTTPException(status_code=404, detail="job not found")
+        return job_status(job)
+
+    @app.post(f"{route_prefix}/compute/jobs/{{job_id}}/stop")
+    async def stop_job(job_id: str) -> dict[str, Any]:
+        if execution_mode() != "simulator":
+            job = process_manager.stop_job(job_id)
+            if not job:
+                raise HTTPException(status_code=404, detail="job not found")
+            return job
+        job = jobs.get(job_id)
+        if not job:
+            raise HTTPException(status_code=404, detail="job not found")
+        job["status"] = "stopped"
+        job["progress"] = min(job.get("progress", 0), 99)
+        return job
+
+    @app.get(f"{route_prefix}/compute/jobs/{{job_id}}/logs")
+    async def job_logs(
+        job_id: str,
+        tail_lines: int | None = Query(default=200, ge=1, le=5000),
+        offset: int | None = Query(default=None, ge=0),
+        limit: int | None = Query(default=None, ge=1, le=5000),
+    ) -> dict[str, Any]:
+        if execution_mode() != "simulator":
+            job = process_manager.get_job(job_id)
+            if not job:
+                raise HTTPException(status_code=404, detail="job not found")
+            content = process_manager.logs(job_id)
+        else:
+            job = jobs.get(job_id)
+            if not job:
+                raise HTTPException(status_code=404, detail="job not found")
+            job = job_status(job)
+            content = job["logs"]
+        window = _slice_log_content(content, tail_lines, offset, limit)
+        metrics = [parse_log_line(line) for line in window["content"].splitlines()]
+        return {"job_id": job_id, **window, "metrics": [m for m in metrics if m]}
+
+    # ── Inference Endpoints ───────────────────────────────────────────
+
+    @app.post(f"{route_prefix}/inference/load")
+    async def inference_load(payload: dict[str, Any]) -> dict[str, Any]:
+        """Load a model for inference using LLaMA-Factory ChatModel.
+
+        Expected payload:
+            model_name_or_path: str (required)
+            adapter_name_or_path: str (optional, for LoRA adapters)
+            template: str (default: "qwen")
+            infer_backend: str (default: "huggingface")
+            infer_dtype: str (default: "auto")
+        """
+        session = get_inference_session()
+        result = session.load(
+            model_name_or_path=payload.get("model_name_or_path", ""),
+            adapter_name_or_path=payload.get("adapter_name_or_path", ""),
+            template=payload.get("template", "qwen"),
+            infer_backend=payload.get("infer_backend", "huggingface"),
+            infer_dtype=payload.get("infer_dtype", "auto"),
+        )
+        return result
+
+    @app.post(f"{route_prefix}/inference/unload")
+    async def inference_unload() -> dict[str, Any]:
+        """Unload the currently loaded model and free GPU memory."""
+        # Teardown (gc.collect + cuda.empty_cache) can take a while; run it off
+        # the event loop so /health and /inference/status stay responsive.
+        return await asyncio.to_thread(get_inference_session().unload)
+
+    @app.get(f"{route_prefix}/inference/status")
+    async def inference_status() -> dict[str, Any]:
+        """Get the current inference session status."""
+        return get_inference_session().info()
+
+    @app.post(f"{route_prefix}/inference/chat")
+    async def inference_chat(payload: dict[str, Any]) -> dict[str, Any]:
+        """Chat with the loaded model (non-streaming).
+
+        Expected payload:
+            messages: list[dict] (OpenAI format)
+            temperature: float (default 0.95)
+            top_p: float (default 0.7)
+            max_new_tokens: int (default 1024)
+        """
+        messages = payload.get("messages") or []
+        if not messages:
+            raise HTTPException(status_code=400, detail="messages is required")
+        # Generation is long-running; run it in a thread so the event loop keeps
+        # serving /inference/status and /health during inference.
+        result = await asyncio.to_thread(
+            get_inference_session().chat,
+            messages=messages,
+            temperature=float(payload.get("temperature", 0.95)),
+            top_p=float(payload.get("top_p", 0.7)),
+            max_new_tokens=int(payload.get("max_new_tokens", 1024)),
+            do_sample=bool(payload.get("do_sample", True)),
+        )
+        if result.get("error"):
+            raise HTTPException(status_code=500, detail=result["error"])
+        return {"response": result["response"]}
+
+    @app.post(f"{route_prefix}/inference/chat/stream")
+    async def inference_chat_stream(payload: dict[str, Any]) -> StreamingResponse:
+        """Chat with streaming response (Server-Sent Events)."""
+        messages = payload.get("messages") or []
+        if not messages:
+            raise HTTPException(status_code=400, detail="messages is required")
+
+        def generate():
+            session = get_inference_session()
+            for chunk in session.chat_stream(
+                messages=messages,
+                temperature=float(payload.get("temperature", 0.95)),
+                top_p=float(payload.get("top_p", 0.7)),
+                max_new_tokens=int(payload.get("max_new_tokens", 1024)),
+                do_sample=bool(payload.get("do_sample", True)),
+            ):
+                yield chunk
+
+        return StreamingResponse(generate(), media_type="text/event-stream")
+
+    @app.post(f"{route_prefix}/compute/files/upload")
+    async def upload_file(
+        file: UploadFile | None = File(default=None),
+        target_relative_path: str | None = Form(default=None),
+        resource_type: str | None = Form(default=None),
+        resource_id: str | None = Form(default=None),
+    ) -> dict[str, Any]:
+        file_id = f"file_{int(now() * 1000)}"
+        data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
+        data_root.mkdir(parents=True, exist_ok=True)
+        filename = Path(file.filename if file else file_id).name
+        if target_relative_path:
+            target = (data_root / target_relative_path.lstrip("/\\")).resolve()
+            if not _path_inside(data_root, target):
+                raise HTTPException(status_code=400, detail="target path must stay inside YG_FT_DATA_ROOT")
+        else:
+            target = data_root / "uploads" / f"{file_id}_{filename}"
+        if file:
+            target.parent.mkdir(parents=True, exist_ok=True)
+            with target.open("wb") as output:
+                while chunk := await file.read(1024 * 1024):
+                    output.write(chunk)
+        else:
+            target.parent.mkdir(parents=True, exist_ok=True)
+            target.write_text("", encoding="utf-8")
+        return {
+            "id": file_id,
+            "resource_type": resource_type,
+            "resource_id": resource_id,
+            "status": "available",
+            "local_path": str(target),
+            "byte_size": target.stat().st_size,
+            "checksum_sha256": hashlib.sha256(target.read_bytes()).hexdigest() if target.is_file() else "",
+        }
+
+    @app.post(f"{route_prefix}/compute/files/import-local")
+    async def import_local_file(payload: dict[str, Any]) -> dict[str, Any]:
+        source = Path(str(payload.get("source_path") or ""))
+        if not source.exists():
+            raise HTTPException(status_code=404, detail="source path not found")
+        data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
+        data_root.mkdir(parents=True, exist_ok=True)
+        relative = str(payload.get("target_relative_path") or f"imports/{source.name}").lstrip("/\\")
+        target = (data_root / relative).resolve()
+        if not _path_inside(data_root, target):
+            raise HTTPException(status_code=400, detail="target path must stay inside YG_FT_DATA_ROOT")
+        target.parent.mkdir(parents=True, exist_ok=True)
+        if source.is_dir():
+            if target.exists():
+                shutil.rmtree(target)
+            shutil.copytree(source, target)
+            byte_size = sum(path.stat().st_size for path in target.rglob("*") if path.is_file())
+            checksum = ""
+        else:
+            shutil.copy2(source, target)
+            byte_size = target.stat().st_size
+            checksum = hashlib.sha256(target.read_bytes()).hexdigest()
+        return {
+            "id": str(payload.get("id") or f"file_{int(now() * 1000)}"),
+            "resource_type": payload.get("resource_type"),
+            "resource_id": payload.get("resource_id"),
+            "status": "available",
+            "local_path": str(target),
+            "byte_size": byte_size,
+            "checksum_sha256": checksum,
+        }
+
+    @app.get(f"{route_prefix}/compute/files/read")
+    async def read_file(path: str = Query(...)) -> JSONResponse:
+        """Read a text file from within YG_FT_DATA_ROOT.  Used by the backend
+        to fetch eval results and other job outputs."""
+        data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
+        target = (data_root / path.lstrip("/\\")).resolve()
+        if not _path_inside(data_root, target):
+            raise HTTPException(status_code=400, detail="path must stay inside YG_FT_DATA_ROOT")
+        if not target.is_file():
+            raise HTTPException(status_code=404, detail="file not found")
+        try:
+            content = target.read_text(encoding="utf-8")
+            return JSONResponse(json.loads(content) if content.strip().startswith("{") else {"content": content})
+        except Exception as exc:
+            raise HTTPException(status_code=500, detail=str(exc))
+
+    @app.get(f"{route_prefix}/compute/files/{{file_id}}/download")
+    async def download_file(file_id: str) -> FileResponse:
+        upload_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft")) / "uploads"
+        matches = list(upload_root.glob(f"{file_id}_*"))
+        if not matches:
+            raise HTTPException(status_code=404, detail="file not found")
+        return FileResponse(matches[0])
+
+    return app
+
+
+app = create_app()

compute/engines/__init__.py

@@ -0,0 +1 @@
+"""Training engine adapters package."""

compute/engines/llama_factory/__init__.py

@@ -0,0 +1 @@
+"""LLaMA-Factory engine adapter package."""

compute/engines/llama_factory/adapter.py

@@ -0,0 +1,324 @@
+from __future__ import annotations
+
+import json
+import re
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+
+@dataclass(frozen=True)
+class LlamaFactoryCommand:
+    command: list[str]
+    work_dir: str
+    env: dict[str, str]
+
+
+def _load_dataset_preview(path: Path) -> list[dict[str, Any]]:
+    if not path.exists():
+        return []
+    text = path.read_text(encoding="utf-8", errors="replace").strip()
+    if not text:
+        return []
+    if path.suffix.lower() == ".jsonl":
+        items: list[dict[str, Any]] = []
+        for line in text.splitlines()[:20]:
+            line = line.strip()
+            if not line:
+                continue
+            value = json.loads(line)
+            if isinstance(value, dict):
+                items.append(value)
+        return items
+    value = json.loads(text)
+    if isinstance(value, list):
+        return [item for item in value[:20] if isinstance(item, dict)]
+    if isinstance(value, dict):
+        return [value]
+    return []
+
+
+def _validate_dataset_columns(config: dict[str, Any]) -> list[str]:
+    dataset_dir = config.get("dataset_dir")
+    dataset_info = config.get("dataset_info")
+    if not dataset_dir or not isinstance(dataset_info, dict):
+        return []
+    root = Path(str(dataset_dir))
+    errors: list[str] = []
+    for dataset_key, item in dataset_info.items():
+        if not isinstance(item, dict):
+            continue
+        file_name = item.get("file_name")
+        file_names = file_name if isinstance(file_name, list) else [file_name]
+        columns = item.get("columns") if isinstance(item.get("columns"), dict) else {}
+        required_columns = [str(value) for value in columns.values() if value]
+        for name in file_names:
+            if not name:
+                continue
+            path = root / str(name).lstrip("/\\")
+            if not path.exists():
+                continue
+            try:
+                preview_rows = _load_dataset_preview(path)
+            except Exception as exc:  # noqa: BLE001 - expose malformed data as validation error
+                errors.append(f"dataset file parse failed: {path}: {exc}")
+                continue
+            if not preview_rows:
+                errors.append(f"dataset file has no valid object records: {path}")
+                continue
+            available = set().union(*(row.keys() for row in preview_rows))
+            missing = [column for column in required_columns if column not in available]
+            if missing:
+                errors.append(
+                    f"dataset columns missing in {path.name} for {dataset_key}: {', '.join(sorted(set(missing)))}"
+                )
+    return errors
+
+
+def validate_config(config: dict[str, Any]) -> list[str]:
+    errors: list[str] = []
+    if not config.get("base_model") and not config.get("model_name_or_path"):
+        errors.append("base_model or model_name_or_path is required")
+    if not config.get("dataset") and not config.get("dataset_dir"):
+        errors.append("dataset or dataset_dir is required")
+    try:
+        learning_rate = float(config.get("learning_rate", 0.0002))
+    except (TypeError, ValueError):
+        learning_rate = 0
+    if learning_rate <= 0:
+        errors.append("learning_rate must be greater than zero")
+    try:
+        epochs = int(config.get("n_epochs", config.get("num_train_epochs", 1)))
+    except (TypeError, ValueError):
+        epochs = 0
+    if epochs <= 0:
+        errors.append("n_epochs must be greater than zero")
+    dataset_dir = config.get("dataset_dir")
+    dataset_info = config.get("dataset_info")
+    if config.get("require_dataset_files") and dataset_dir and isinstance(dataset_info, dict):
+        root = Path(str(dataset_dir))
+        for dataset_key, item in dataset_info.items():
+            if not isinstance(item, dict):
+                errors.append(f"dataset_info entry must be object: {dataset_key}")
+                continue
+            file_name = item.get("file_name")
+            file_names = file_name if isinstance(file_name, list) else [file_name]
+            for name in file_names:
+                if not name:
+                    errors.append(f"dataset_info file_name is required: {dataset_key}")
+                    continue
+                path = root / str(name).lstrip("/\\")
+                if not path.exists():
+                    errors.append(f"dataset file not found: {path}")
+        errors.extend(_validate_dataset_columns(config))
+    return errors
+
+
+def _optional_arg(config: dict[str, Any], command: list[str], option: str, *keys: str) -> None:
+    for key in keys:
+        value = config.get(key)
+        if value is not None and value != "":
+            command.extend([option, str(value)])
+            return
+
+
+def _optional_bool_arg(config: dict[str, Any], command: list[str], option: str, *keys: str) -> None:
+    for key in keys:
+        value = config.get(key)
+        if value is True or str(value).lower() == "true":
+            command.extend([option, "true"])
+            return
+
+
+def _normalize_stage(config: dict[str, Any]) -> str:
+    raw = str(config.get("stage") or config.get("train_type") or "sft").strip().lower()
+    return {
+        "sft": "sft",
+        "dpo": "dpo",
+        "cpt": "pt",
+        "pt": "pt",
+        "pretrain": "pt",
+        "rm": "rm",
+        "ppo": "ppo",
+        "kto": "kto",
+    }.get(raw, raw or "sft")
+
+
+def prepare_runtime_files(config: dict[str, Any]) -> list[dict[str, str]]:
+    dataset_dir = config.get("dataset_dir")
+    dataset_info = config.get("dataset_info")
+    if not dataset_dir or not isinstance(dataset_info, dict):
+        return []
+    root = Path(str(dataset_dir))
+    root.mkdir(parents=True, exist_ok=True)
+    path = root / "dataset_info.json"
+    existing: dict[str, Any] = {}
+    if path.exists():
+        try:
+            loaded = json.loads(path.read_text(encoding="utf-8"))
+            existing = loaded if isinstance(loaded, dict) else {}
+        except json.JSONDecodeError:
+            existing = {}
+    existing.update(dataset_info)
+    path.write_text(json.dumps(existing, ensure_ascii=False, indent=2), encoding="utf-8")
+    return [{"name": "dataset_info", "path": str(path)}]
+
+
+def build_command(config: dict[str, Any], llama_factory_home: str = "/app/LLaMA-Factory") -> LlamaFactoryCommand:
+    engine = str(config.get("engine") or config.get("training_engine") or "llama_factory")
+    if engine in {"merge", "export", "llama_factory_export"}:
+        model_path = config.get("base_model") or config.get("model_name_or_path") or config.get("base_model_path")
+        adapter_path = config.get("adapter_name_or_path") or config.get("adapter_path") or config.get("lora_path")
+        output_dir = config.get("output_dir") or config.get("export_dir")
+        errors: list[str] = []
+        if not model_path:
+            errors.append("base_model or model_name_or_path is required")
+        if not adapter_path and engine == "merge":
+            errors.append("adapter_name_or_path or adapter_path is required")
+        if not output_dir:
+            errors.append("output_dir or export_dir is required")
+        if errors:
+            raise ValueError("; ".join(errors))
+        command = [
+            "llamafactory-cli",
+            "export",
+            "--model_name_or_path",
+            str(model_path),
+            "--template",
+            str(config.get("template", "qwen")),
+            "--finetuning_type",
+            str(config.get("train_method", config.get("finetuning_type", "lora"))),
+            "--export_dir",
+            str(output_dir),
+            "--export_size",
+            str(config.get("export_size", 2)),
+            "--export_device",
+            str(config.get("export_device", "cpu")),
+            "--export_legacy_format",
+            str(config.get("export_legacy_format", False)).lower(),
+        ]
+        if adapter_path:
+            command.extend(["--adapter_name_or_path", str(adapter_path)])
+        quantization_bit = int(config.get("export_quantization_bit", config.get("quantization_bit", 0)) or 0)
+        if quantization_bit in {4, 8}:
+            command.extend(["--quantization_bit", str(quantization_bit)])
+        return LlamaFactoryCommand(command=command, work_dir=str(Path(llama_factory_home)), env={})
+
+    if engine == "eval":
+        output_dir = config.get("output_dir") or f"/data/yg-ft/outputs/{config.get('name', 'eval-job')}"
+        eval_config_path = str(Path(output_dir) / "eval_config.json")
+        eval_config = {
+            "model_name_or_path": config.get("model_name_or_path", ""),
+            "adapter_name_or_path": config.get("adapter_name_or_path", ""),
+            "template": config.get("template", "qwen"),
+            "dataset_path": config.get("dataset_path", ""),
+            "output_dir": output_dir,
+            "basic_metrics": config.get("basic_metrics", {}),
+            "dimension": config.get("dimension", {}),
+            "temperature": config.get("temperature", 0.1),
+            "top_p": config.get("top_p", 0.95),
+            "max_new_tokens": config.get("max_new_tokens", 512),
+            "infer_backend": config.get("infer_backend", "huggingface"),
+            "infer_dtype": config.get("infer_dtype", "auto"),
+        }
+        Path(output_dir).mkdir(parents=True, exist_ok=True)
+        Path(eval_config_path).write_text(json.dumps(eval_config, ensure_ascii=False, indent=2), encoding="utf-8")
+        return LlamaFactoryCommand(
+            command=["python", "-u", "-m", "compute.engines.llama_factory.eval_runner", "--config", eval_config_path],
+            work_dir="/app",
+            env={},
+        )
+
+    errors = validate_config(config)
+    if errors:
+        raise ValueError("; ".join(errors))
+
+    if engine == "smoke":
+        output_dir = config.get("output_dir") or f"/data/yg-ft/outputs/{config.get('name', 'training-smoke')}"
+        script = (
+            "import json, os, time; "
+            f"out={str(output_dir)!r}; "
+            "os.makedirs(out, exist_ok=True); "
+            "print('[INFO] smoke training started', flush=True); "
+            "\nfor step in range(1, 7):\n"
+            "    loss=round(1.8/(step+1), 4)\n"
+            "    lr=round(0.0002*(1-step/10), 8)\n"
+            "    print({'loss': loss, 'grad_norm': round(0.4 + step*0.03, 4), 'learning_rate': lr, 'epoch': round(step/6, 4)}, flush=True)\n"
+            "    time.sleep(0.4)\n"
+            "\nopen(os.path.join(out, 'adapter_config.json'), 'w', encoding='utf-8').write(json.dumps({'engine':'smoke','status':'completed'})); "
+            "print('***** train metrics *****', flush=True); "
+            "print('train_loss = 0.12', flush=True); "
+            "print('***** train metrics end *****', flush=True)"
+        )
+        return LlamaFactoryCommand(command=["python", "-u", "-c", script], work_dir="/app", env={})
+
+    model_path = config.get("base_model") or config.get("model_name_or_path")
+    dataset = config.get("dataset") or config.get("dataset_name")
+    dataset_dir = config.get("dataset_dir")
+    output_dir = config.get("output_dir") or f"/data/yg-ft/outputs/{config.get('name', 'training-job')}"
+    command = [
+        "llamafactory-cli",
+        "train",
+        "--stage",
+        _normalize_stage(config),
+        "--do_train",
+        "true",
+        "--model_name_or_path",
+        str(model_path),
+        "--dataset",
+        str(dataset or "default"),
+        "--template",
+        str(config.get("template", "qwen")),
+        "--finetuning_type",
+        str(config.get("train_method", config.get("finetuning_type", "lora"))),
+        "--output_dir",
+        str(output_dir),
+        "--per_device_train_batch_size",
+        str(config.get("batch_size", 2)),
+        "--learning_rate",
+        str(config.get("learning_rate", 0.0002)),
+        "--num_train_epochs",
+        str(config.get("n_epochs", 3)),
+        "--save_steps",
+        str(config.get("save_steps", 50)),
+        "--logging_steps",
+        str(config.get("logging_steps", 10)),
+        "--overwrite_output_dir",
+        "true",
+        "--plot_loss",
+        "true",
+    ]
+    if dataset_dir:
+        command.extend(["--dataset_dir", str(dataset_dir)])
+    eval_dataset = config.get("eval_dataset")
+    if eval_dataset:
+        command.extend(["--eval_dataset", str(eval_dataset), "--do_eval", "true"])
+    _optional_arg(config, command, "--cutoff_len", "max_length", "cutoff_len")
+    _optional_arg(config, command, "--lr_scheduler_type", "lr_scheduler_type")
+    _optional_arg(config, command, "--warmup_ratio", "warmup_ratio")
+    _optional_arg(config, command, "--weight_decay", "weight_decay")
+    _optional_arg(config, command, "--lora_rank", "lora_rank", "rank")
+    _optional_arg(config, command, "--lora_alpha", "lora_alpha")
+    _optional_arg(config, command, "--lora_dropout", "lora_dropout")
+    _optional_arg(config, command, "--gradient_accumulation_steps", "gradient_accumulation_steps")
+    if not eval_dataset:
+        _optional_arg(config, command, "--val_size", "val_size")
+    _optional_arg(config, command, "--max_samples", "max_samples")
+    _optional_arg(config, command, "--preprocessing_num_workers", "preprocessing_num_workers")
+    _optional_bool_arg(config, command, "--fp16", "fp16")
+    _optional_bool_arg(config, command, "--bf16", "bf16")
+    quantization_bit = int(config.get("quantization_bit", 0) or 0)
+    if quantization_bit in {4, 8}:
+        command.extend(["--quantization_bit", str(quantization_bit)])
+    return LlamaFactoryCommand(command=command, work_dir=str(Path(llama_factory_home)), env={})
+
+
+def parse_log_line(line: str) -> dict[str, float] | None:
+    if "loss" not in line or "learning_rate" not in line:
+        return None
+    result: dict[str, float] = {}
+    for key in ["loss", "grad_norm", "learning_rate", "epoch"]:
+        match = re.search(rf"['\"]?{key}['\"]?\s*:\s*([-+]?\d+(?:\.\d+)?(?:[eE][-+]?\d+)?)", line)
+        if match:
+            result[key] = float(match.group(1))
+    return result or None

compute/engines/llama_factory/eval_runner.py

@@ -0,0 +1,485 @@
+"""
+Evaluation runner — executes model evaluation as a subprocess job.
+
+Usage:
+    python -m compute.engines.llama_factory.eval_runner --config <config_json_path>
+
+The config JSON is written by the compute API before spawning this subprocess.
+Results are written to ``output_dir/eval_results.json`` and progress is printed
+to stdout (captured as job logs).
+"""
+
+from __future__ import annotations
+
+import json
+import math
+import re
+import sys
+import time
+from difflib import SequenceMatcher
+from pathlib import Path
+from typing import Any
+
+
+def _load_dataset(path: str) -> list[dict[str, Any]]:
+    """Load a JSON or JSONL dataset file.
+
+    Supports common field names used across the platform:
+    * ``instruction`` + ``input`` + ``output``  (Alpaca-style)
+    * ``question`` + ``answer``
+    * ``messages``  (ShareGPT-style – the last assistant message is treated as reference)
+    """
+    file_path = Path(path)
+    text = file_path.read_text(encoding="utf-8", errors="replace").strip()
+    if not text:
+        return []
+    if file_path.suffix.lower() == ".json":
+        value = json.loads(text)
+        if isinstance(value, list):
+            return [item for item in value if isinstance(item, dict)]
+        return [value] if isinstance(value, dict) else []
+
+    samples: list[dict[str, Any]] = []
+    for line in text.splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            obj = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+        if isinstance(obj, dict):
+            samples.append(obj)
+    return samples
+
+
+def _sample_question(sample: dict[str, Any]) -> str:
+    """Extract the user-facing question / instruction from a sample."""
+    if sample.get("instruction"):
+        text = sample["instruction"]
+        if sample.get("input"):
+            text += "\n" + sample["input"]
+        return text
+    if sample.get("question"):
+        return sample["question"]
+    # ShareGPT-style: use the last user message as question
+    messages = sample.get("messages") or []
+    user_msgs = [m["content"] for m in messages if m.get("role") == "user"]
+    return user_msgs[-1] if user_msgs else ""
+
+
+def _sample_reference(sample: dict[str, Any]) -> str:
+    """Extract the reference answer from a sample."""
+    if sample.get("output"):
+        return sample["output"]
+    if sample.get("answer"):
+        return sample["answer"]
+    messages = sample.get("messages") or []
+    assistant_msgs = [m["content"] for m in messages if m.get("role") == "assistant"]
+    return assistant_msgs[-1] if assistant_msgs else ""
+
+
+# ---------------------------------------------------------------------------
+# Basic metrics
+# ---------------------------------------------------------------------------
+
+def _compute_bleu(references: list[str], predictions: list[str], ngram: int = 4) -> dict[str, Any]:
+    """Compute BLEU score via sacrebleu (corpus-level)."""
+    try:
+        from sacrebleu.metrics import BLEU
+    except ImportError:
+        return {"enabled": False, "error": "sacrebleu not installed", "score": 0}
+    bleu = BLEU(max_ngram_order=ngram)
+    # sacrebleu expects list-of-strings; we have one reference per prediction
+    score = bleu.corpus_score(predictions, [references])
+    return {
+        "enabled": True,
+        "score": round(score.score, 2),
+        "bleu": round(score.score, 2),
+    }
+
+
+def _compute_rouge(references: list[str], predictions: list[str], methods: list[str] | None = None) -> dict[str, Any]:
+    """Compute ROUGE scores via rouge-score."""
+    try:
+        from rouge_score import rouge_scorer
+    except ImportError:
+        return {"enabled": False, "error": "rouge-score not installed", "score": 0}
+    methods = methods or ["rouge1", "rouge2", "rougeL"]
+    # Normalize: map "rouge_1"/"rouge1" → "rouge1", "rouge_l"/"rougeL" → "rougeL"
+    _rouge_aliases = {"rouge_1": "rouge1", "rouge_2": "rouge2", "rouge_l": "rougeL"}
+    methods = [_rouge_aliases.get(m, m.replace("_", "")) for m in methods]
+    scorer = rouge_scorer.RougeScorer(methods, use_stemmer=True)
+    totals: dict[str, float] = {}
+    n = max(len(predictions), 1)
+    for ref, pred in zip(references, predictions):
+        result = scorer.score(ref, pred)
+        for key in methods:
+            totals[key] = totals.get(key, 0) + result[key].fmeasure
+    avg = {k: round(v / n, 4) for k, v in totals.items()}
+    return {"enabled": True, "score": round(avg.get("rougeL", avg.get("rouge1", 0)) * 100, 2), **avg}
+
+
+def _compute_cosine(references: list[str], predictions: list[str]) -> dict[str, Any]:
+    """Compute average cosine similarity via sklearn."""
+    try:
+        from sklearn.feature_extraction.text import TfidfVectorizer
+        from sklearn.metrics.pairwise import cosine_similarity
+    except ImportError:
+        return {"enabled": False, "error": "scikit-learn not installed", "score": 0}
+    try:
+        vectorizer = TfidfVectorizer()
+        tfidf = vectorizer.fit_transform(references + predictions)
+        n = len(references)
+        ref_vec = tfidf[:n]
+        pred_vec = tfidf[n:]
+        sims = cosine_similarity(ref_vec, pred_vec).diagonal()
+        return {"enabled": True, "score": round(float(sims.mean()) * 100, 2)}
+    except ValueError:
+        return {"enabled": True, "score": 0, "error": "insufficient text for vectorization"}
+
+
+def _normalize_text(value: str) -> str:
+    return re.sub(r"\s+", " ", str(value or "").strip().lower())
+
+
+def _compute_exact_match(references: list[str], predictions: list[str]) -> dict[str, Any]:
+    total = len(predictions)
+    if not total:
+        return {"enabled": True, "score": 0, "matched": 0, "total": 0}
+    matched = sum(
+        1
+        for ref, pred in zip(references, predictions)
+        if _normalize_text(ref) == _normalize_text(pred)
+    )
+    return {"enabled": True, "score": round(matched / total * 100, 2), "matched": matched, "total": total}
+
+
+def _compute_text_similarity(references: list[str], predictions: list[str]) -> dict[str, Any]:
+    if not predictions:
+        return {"enabled": True, "score": 0}
+    scores = [
+        SequenceMatcher(None, _normalize_text(ref), _normalize_text(pred)).ratio()
+        for ref, pred in zip(references, predictions)
+    ]
+    return {"enabled": True, "score": round(sum(scores) / max(len(scores), 1) * 100, 2)}
+
+
+# ---------------------------------------------------------------------------
+# LLM Judge
+# ---------------------------------------------------------------------------
+
+def _judge_sample(
+    question: str,
+    reference: str,
+    prediction: str,
+    config: dict[str, Any],
+) -> dict[str, Any]:
+    """Call an OpenAI-compatible LLM to judge a single sample.
+
+    Returns a dict with keys:
+        score, max_score, passed, judgement, evaluation_reason, error_type
+    """
+    api_url = (config.get("api_url") or "").strip().rstrip("/")
+    api_key = (config.get("api_key") or "").strip()
+    eval_model = (config.get("eval_model") or "").strip()
+    # 优先使用模型记录里配置的真实 API 模型名（如 deepseek-chat），
+    # 否则回退到平台内部模型名
+    api_model = (config.get("api_model") or "").strip() or eval_model
+    eval_prompt = (config.get("eval_prompt") or "").strip()
+    score_min = float(config.get("score_min", 0))
+    score_max = float(config.get("score_max", 5))
+    pass_threshold = float(config.get("pass_threshold", 3))
+
+    if not api_url or not eval_model:
+        return {"score": 0, "max_score": score_max, "passed": False, "judgement": "未配置",
+                "evaluation_reason": "未配置评测模型", "error_type": "其他"}
+
+    system_msg = (
+        eval_prompt
+        or "你是一个专业的评测专家。请根据参考答-案对被测模型的输出进行评分。"
+    )
+    user_msg = (
+        f"## 问题\n{question}\n\n"
+        f"## 参考答案\n{reference}\n\n"
+        f"## 模型输出\n{prediction}\n\n"
+        f"请给出 {score_min}-{score_max} 分的评分，并说明理由。"
+    )
+
+    try:
+        import urllib.request
+        import urllib.error
+
+        body = json.dumps({
+            "model": api_model,
+            "messages": [
+                {"role": "system", "content": system_msg},
+                {"role": "user", "content": user_msg},
+            ],
+            "temperature": 0.3,
+            "max_tokens": 512,
+        }).encode("utf-8")
+
+        req = urllib.request.Request(
+            f"{api_url}/v1/chat/completions",
+            data=body,
+            headers={
+                "Content-Type": "application/json",
+                "Authorization": f"Bearer {api_key}",
+            },
+        )
+        resp = urllib.request.urlopen(req, timeout=120)
+        data = json.loads(resp.read().decode("utf-8"))
+        reply = data["choices"][0]["message"]["content"]
+    except Exception as exc:
+        return {"score": 0, "max_score": score_max, "passed": False,
+                "judgement": "错误", "evaluation_reason": f"评测模型调用失败: {exc}",
+                "error_type": "其他"}
+
+    # Parse score from reply — look for patterns like "4分" or "Score: 4"
+    score = 0
+    import re
+    score_patterns = [
+        r'(?:得分|分数|评分|score)[^\d]*(\d+(?:\.\d+)?)',
+        r'(\d+(?:\.\d+)?)\s*分',
+        r'(\d+(?:\.\d+)?)\s*/\s*\d+',
+    ]
+    for pat in score_patterns:
+        m = re.search(pat, reply, re.IGNORECASE)
+        if m:
+            try:
+                score = float(m.group(1))
+            except ValueError:
+                continue
+            break
+    score = max(score_min, min(score_max, score))
+    passed = score >= pass_threshold
+
+    # Determine judgement label
+    if score >= pass_threshold + 1:
+        judgement = "正确"
+    elif score >= pass_threshold:
+        judgement = "部分正确"
+    else:
+        judgement = "错误"
+
+    # Guess error type from reply
+    reply_lower = reply.lower()
+    if any(w in reply_lower for w in ["幻觉", "hallucination", "编造"]):
+        error_type = "幻觉"
+    elif any(w in reply_lower for w in ["不完整", "incomplete", "遗漏"]):
+        error_type = "不完整"
+    elif any(w in reply_lower for w in ["格式", "format"]):
+        error_type = "格式偏差"
+    elif any(w in reply_lower for w in ["混淆", "confusion", "错误"]):
+        error_type = "混淆"
+    else:
+        error_type = "其他"
+
+    return {
+        "score": score,
+        "max_score": score_max,
+        "passed": passed,
+        "judgement": judgement,
+        "evaluation_reason": reply[:2000],
+        "error_type": error_type,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Main entry point
+# ---------------------------------------------------------------------------
+
+def run_eval(config: dict[str, Any]) -> dict[str, Any]:
+    """Execute a full evaluation run.  Returns the result dict (also written to file)."""
+    model_path = config["model_name_or_path"]
+    adapter_path = config.get("adapter_name_or_path", "")
+    template = config.get("template", "qwen")
+    dataset_path = config["dataset_path"]
+    output_dir = Path(config["output_dir"])
+    output_dir.mkdir(parents=True, exist_ok=True)
+    basic_cfg = config.get("basic_metrics", {})
+    dimension_cfg = config.get("dimension", {}) or {}
+    output_precision = int(basic_cfg.get("output_precision", 2))
+
+    # ---- 1. Load dataset ----
+    print(f"[eval] loading dataset: {dataset_path}")
+    raw_samples = _load_dataset(dataset_path)
+    print(f"[eval] loaded {len(raw_samples)} samples")
+
+    # ---- 2. Load model ----
+    print(f"[eval] loading model: {model_path}")
+    from compute.engines.llama_factory.inference import InferenceSession
+    session = InferenceSession()
+    session.load(
+        model_name_or_path=model_path,
+        adapter_name_or_path=adapter_path,
+        template=template,
+        infer_backend=config.get("infer_backend", "huggingface"),
+        infer_dtype=config.get("infer_dtype", "auto"),
+    )
+    # load() 为异步加载（立即返回 loading），必须等待后台线程完成后再进行推理
+    load_result = session.wait_until_loaded(timeout=float(config.get("load_timeout", 1800)))
+    if not load_result.get("loaded"):
+        raise RuntimeError(f"model load failed: {load_result.get('error', 'unknown')}")
+    print(f"[eval] model loaded OK")
+
+    # ---- 3. Run inference on each sample ----
+    samples: list[dict[str, Any]] = []
+    predictions: list[str] = []
+    references: list[str] = []
+    questions: list[str] = []
+
+    total = len(raw_samples)
+    judge_enabled = bool(dimension_cfg.get("eval_model") and dimension_cfg.get("api_url"))
+    print(f"[eval] starting inference on {total} samples, judge={'enabled' if judge_enabled else 'disabled'}")
+
+    for idx, raw in enumerate(raw_samples, start=1):
+        question = _sample_question(raw)
+        reference = _sample_reference(raw)
+        if not question:
+            print(f"[eval] sample {idx}/{total}: skipped (no question)")
+            continue
+
+        # Inference
+        chat_msgs = [{"role": "user", "content": question}]
+        result = session.chat(
+            chat_msgs,
+            temperature=float(config.get("temperature", 0.1)),
+            top_p=float(config.get("top_p", 0.95)),
+            max_new_tokens=int(config.get("max_new_tokens", 512)),
+            do_sample=False,
+        )
+        prediction = result.get("response", "") if not result.get("error") else f"[ERROR] {result['error']}"
+
+        predictions.append(prediction)
+        references.append(reference)
+        questions.append(question)
+
+        # LLM Judge
+        judge_result: dict[str, Any] = {}
+        if judge_enabled:
+            judge_result = _judge_sample(question, reference, prediction, dimension_cfg)
+
+        samples.append({
+            "index": idx,
+            "input": question,
+            "reference_answer": reference,
+            "model_output": prediction,
+            "score": judge_result.get("score"),
+            "max_score": judge_result.get("max_score", dimension_cfg.get("score_max", 5)),
+            "passed": judge_result.get("passed"),
+            "judgement": judge_result.get("judgement"),
+            "evaluation_reason": judge_result.get("evaluation_reason", ""),
+            "error_type": judge_result.get("error_type"),
+            "dimension_scores": [
+                {"name": "judge_score", "score": judge_result.get("score", 0),
+                 "max_score": judge_result.get("max_score", dimension_cfg.get("score_max", 5))},
+            ] if judge_result else [],
+            "status": "completed",
+        })
+
+        progress_pct = int(idx / max(total, 1) * 100)
+        print(f"[eval] sample {idx}/{total} ({progress_pct}%) done")
+
+    # ---- 4. Compute basic metrics ----
+    print(f"[eval] computing basic metrics on {len(predictions)} predictions")
+    metrics_result: dict[str, Any] = {}
+
+    bleu_cfg = basic_cfg.get("bleu", {})
+    if bleu_cfg.get("enabled"):
+        metrics_result["bleu"] = _compute_bleu(references, predictions, int(bleu_cfg.get("ngram", 4)))
+
+    rouge_cfg = basic_cfg.get("rouge", {})
+    if rouge_cfg.get("enabled"):
+        metrics_result["rouge"] = _compute_rouge(references, predictions, rouge_cfg.get("methods"))
+
+    cosine_cfg = basic_cfg.get("cosine", {})
+    if cosine_cfg.get("enabled"):
+        metrics_result["cosine"] = _compute_cosine(references, predictions)
+    metrics_result["exact_match"] = _compute_exact_match(references, predictions)
+    metrics_result["text_similarity"] = _compute_text_similarity(references, predictions)
+
+    # ---- 5. Summarise ----
+    completed = len(samples)
+    if judge_enabled:
+        scored = [s for s in samples if s.get("score") is not None]
+        passed_count = len([s for s in scored if s.get("passed")])
+        avg_score = round(sum(s["score"] for s in scored) / max(len(scored), 1), output_precision)
+        max_score = dimension_cfg.get("score_max", 5)
+        overall_score = round(avg_score / max_score * 100, output_precision)
+        overall_score_max = 100
+        dimension_summary = [{
+            "name": "综合评分",
+            "score": overall_score,
+            "max_score": 100,
+            "pass_rate": round(passed_count / max(completed, 1) * 100, 1),
+        }]
+        overall_evaluation = f"评测完成：{completed} 样本，{passed_count} 通过，平均 {avg_score}/{max_score} 分"
+    else:
+        passed_count = 0
+        enabled_scores = [
+            float(item.get("score") or 0)
+            for item in metrics_result.values()
+            if isinstance(item, dict) and item.get("enabled", True) and item.get("score") is not None
+        ]
+        overall_score = round(sum(enabled_scores) / len(enabled_scores), output_precision) if enabled_scores else 0
+        overall_score_max = 100
+        dimension_summary = [
+            {
+                "name": name,
+                "score": float(item.get("score") or 0),
+                "max_score": 100,
+                "pass_rate": float(item.get("score") or 0),
+            }
+            for name, item in metrics_result.items()
+            if isinstance(item, dict) and item.get("enabled", True) and item.get("score") is not None
+        ]
+        overall_evaluation = f"评测完成：{completed} 样本（未配置 LLM 评委）"
+
+    result = {
+        "overall_score": overall_score,
+        "overall_score_max": overall_score_max,
+        "overall_evaluation": overall_evaluation,
+        "improvement_suggestions": [],
+        "dimension_summary": dimension_summary,
+        "samples": samples,
+        "sample_count": total,
+        "completed_count": completed,
+        "passed_count": passed_count,
+        "basic_metrics": metrics_result,
+    }
+
+    # ---- 6. Write results ----
+    result_path = output_dir / "eval_results.json"
+    result_path.write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
+    print(f"[eval] results written to {result_path}")
+    return result
+
+
+def main() -> None:
+    import argparse
+    parser = argparse.ArgumentParser(description="YG-FT Evaluation Runner")
+    parser.add_argument("--config", required=True, help="Path to eval config JSON file")
+    args = parser.parse_args()
+
+    config_path = Path(args.config)
+    if not config_path.exists():
+        print(f"FATAL: config file not found: {args.config}", file=sys.stderr)
+        sys.exit(1)
+
+    config = json.loads(config_path.read_text(encoding="utf-8"))
+    start = time.time()
+    try:
+        run_eval(config)
+        elapsed = time.time() - start
+        print(f"[eval] DONE in {elapsed:.1f}s")
+    except Exception as exc:
+        print(f"[eval] FAILED: {exc}", file=sys.stderr)
+        import traceback
+        traceback.print_exc()
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

compute/engines/llama_factory/inference.py

@@ -0,0 +1,272 @@
+from __future__ import annotations
+
+import threading
+import time
+import uuid
+from typing import Any, Iterator
+
+
+class InferenceSession:
+    """Manages a loaded model for inference with LLaMA-Factory ChatModel.
+
+    Model loading is asynchronous: ``load()`` spawns a background daemon thread
+    and returns immediately with ``status == "loading"``. ``info()`` (served by
+    ``/inference/status``) is always responsive, so the platform backend can
+    poll loading progress without being blocked by a minutes-long model load —
+    which previously froze the whole compute node event loop.
+
+    State machine: idle -> loading -> ready | error, ready -> idle (unload),
+    loading -> idle (cancelled). Long operations (ChatModel build, teardown,
+    generation) never run while holding ``_state_lock``; they either run in the
+    worker thread or under ``_chat_lock`` only.
+    """
+
+    def __init__(self) -> None:
+        self._state_lock = threading.Lock()  # brief state transitions only
+        self._chat_lock = threading.Lock()  # serialize chat/teardown
+        self._status: str = "idle"
+        self._error: str = ""
+        self._request_id: str = ""
+        self._load_args: dict[str, Any] = {}
+        self._teardown_old = False  # load-while-ready: unload old before loading new
+        self._cancel_requested = False  # unload-while-loading: tear down after load finishes
+        self._load_thread: threading.Thread | None = None
+        self._model: Any = None
+        self._tokenizer: Any = None
+        self._generating_args: dict[str, Any] = {}
+        self._model_name: str = ""
+        self._adapter_path: str = ""
+        self._loaded_at: float = 0.0
+
+    @property
+    def status(self) -> str:
+        with self._state_lock:
+            return self._status
+
+    def info(self) -> dict[str, Any]:
+        with self._state_lock:
+            return {
+                "loaded": self._status == "ready",
+                "status": self._status,
+                "model_name": self._model_name,
+                "adapter_path": self._adapter_path,
+                "loaded_at": self._loaded_at,
+                "request_id": self._request_id,
+                "error": self._error,
+            }
+
+    def wait_until_loaded(self, timeout: float | None = None) -> dict[str, Any]:
+        """Wait for an in-flight async load to finish and return its outcome.
+
+        供同步消费方（如 eval_runner 子进程）使用：``load()`` 立即返回 loading 后，
+        调用本方法等待后台加载线程完成，拿到最终的 loaded/error 结果。
+        若在 timeout 秒内仍未加载完成，返回 ``status == "loading"`` 并附上超时提示。
+        """
+        with self._state_lock:
+            thread = self._load_thread
+        if thread is not None and thread.is_alive():
+            thread.join(timeout=timeout)
+        with self._state_lock:
+            loaded = self._status == "ready"
+            status = self._status
+            error = self._error
+        if not loaded and status == "loading":
+            error = error or f"model load timed out after {timeout or 'N/A'}s"
+        return {
+            "loaded": loaded,
+            "status": status,
+            "model_name": self._model_name,
+            "adapter_path": self._adapter_path,
+            "error": error,
+        }
+
+    def load(
+        self,
+        model_name_or_path,
+        adapter_name_or_path="",
+        template="qwen",
+        infer_backend="huggingface",
+        infer_dtype="auto",
+        **kwargs,
+    ) -> dict[str, Any]:
+        with self._state_lock:
+            if self._status == "loading":
+                # A model is already loading — dedupe, reuse the same request id.
+                return {"loaded": False, "status": "loading", "request_id": self._request_id}
+            self._teardown_old = self._status == "ready"
+            self._status = "loading"
+            self._error = ""
+            self._request_id = uuid.uuid4().hex[:12]
+            self._cancel_requested = False
+            self._load_args = {
+                "model_name_or_path": model_name_or_path,
+                "template": template,
+                "infer_backend": infer_backend,
+                "infer_dtype": infer_dtype,
+            }
+            if adapter_name_or_path:
+                self._load_args["adapter_name_or_path"] = adapter_name_or_path
+            self._load_args.update(kwargs)
+            self._model_name = model_name_or_path
+            self._adapter_path = adapter_name_or_path
+            self._load_thread = threading.Thread(target=self._load_worker, daemon=True)
+            self._load_thread.start()
+            return {"loaded": False, "status": "loading", "request_id": self._request_id}
+
+    def _load_worker(self) -> None:
+        """Build the ChatModel off the state lock so info() never blocks."""
+        model = None
+        tokenizer = None
+        generating_args: dict[str, Any] = {}
+        error = ""
+        try:
+            if self._teardown_old:
+                self._release_model()
+            from llamafactory.chat import ChatModel
+            from llamafactory.hparams import get_infer_args
+
+            args = dict(self._load_args)
+            infer_result = get_infer_args(args)
+            model = ChatModel(args)
+            tokenizer = getattr(model, "tokenizer", None) or model.engine.tokenizer
+            generating_args = infer_result[-1]
+            if hasattr(generating_args, "__dataclass_fields__"):
+                generating_args = {
+                    k: v for k, v in vars(generating_args).items() if not k.startswith("_")
+                }
+            else:
+                generating_args = dict(generating_args)
+        except Exception as exc:  # noqa: BLE001 - surface load failure via status
+            error = str(exc)
+        with self._state_lock:
+            if error:
+                self._model = None
+                self._tokenizer = None
+                self._status = "error"
+                self._error = error
+                return
+            if self._cancel_requested:
+                # Unload was requested while loading — drop the fresh model.
+                model = None
+                tokenizer = None
+                self._model = None
+                self._tokenizer = None
+                self._status = "idle"
+                return
+            self._model = model
+            self._tokenizer = tokenizer
+            self._generating_args = generating_args
+            self._loaded_at = time.time()
+            self._status = "ready"
+
+    def _release_model(self) -> None:
+        with self._chat_lock:
+            with self._state_lock:
+                self._status = "unloading"
+                model = self._model
+                self._model = None
+                self._tokenizer = None
+            if model is not None:
+                try:
+                    del model
+                except Exception:  # noqa: BLE001 - best-effort teardown
+                    pass
+                # 强制释放 PyTorch CUDA 缓存，真正归还 GPU 显存
+                try:
+                    import gc
+
+                    gc.collect()
+                    import torch
+
+                    if torch.cuda.is_available():
+                        torch.cuda.empty_cache()
+                        torch.cuda.synchronize()
+                except Exception:  # noqa: BLE001 - teardown must not raise
+                    pass
+            with self._state_lock:
+                self._status = "idle"
+                self._model_name = ""
+                self._adapter_path = ""
+                self._loaded_at = 0.0
+                self._error = ""
+
+    def unload(self) -> dict[str, Any]:
+        with self._state_lock:
+            if self._status == "loading":
+                # Ask the worker to tear down right after the load finishes.
+                self._cancel_requested = True
+                return {"unloaded": False, "status": "cancelling", "request_id": self._request_id}
+            was_ready = self._status == "ready"
+        if was_ready:
+            self._release_model()
+        else:
+            with self._state_lock:
+                self._model = None
+                self._tokenizer = None
+                self._status = "idle"
+                self._model_name = ""
+                self._adapter_path = ""
+                self._loaded_at = 0.0
+                self._error = ""
+        return {"unloaded": True, "status": "idle"}
+
+    def chat(self, messages, temperature=0.95, top_p=0.7, max_new_tokens=1024, do_sample=True, **kwargs) -> dict[str, Any]:
+        with self._chat_lock:
+            with self._state_lock:
+                if self._status == "loading":
+                    return {
+                        "error": f"model is still loading (request_id={self._request_id}); please retry",
+                        "response": "",
+                    }
+                if self._status == "error":
+                    return {"error": f"model load failed: {self._error}", "response": ""}
+                if self._status != "ready" or self._model is None:
+                    return {"error": "model not loaded", "response": ""}
+            try:
+                generate_kwargs = {
+                    "temperature": temperature,
+                    "top_p": top_p,
+                    "max_new_tokens": max_new_tokens,
+                    "do_sample": do_sample,
+                }
+                generate_kwargs.update(kwargs)
+                system = next((m["content"] for m in messages if m["role"] == "system"), None)
+                user_messages = [m for m in messages if m["role"] != "system"]
+                responses = []
+                for response in self._model.stream_chat(user_messages, system=system, **generate_kwargs):
+                    responses.append(response)
+                full_response = "".join(str(r) for r in responses)
+                return {"response": full_response}
+            except Exception as exc:  # noqa: BLE001 - return generation error to caller
+                return {"error": str(exc), "response": ""}
+
+    def chat_stream(self, messages, **kwargs) -> Iterator[str]:
+        with self._chat_lock:
+            with self._state_lock:
+                if self._status == "loading":
+                    yield 'data: {"error": "model is still loading; please retry"}\n\n'
+                    return
+                if self._status == "error":
+                    yield 'data: {"error": "model load failed: ' + str(self._error) + '"}\n\n'
+                    return
+                if self._status != "ready" or self._model is None:
+                    yield 'data: {"error": "model not loaded"}\n\n'
+                    return
+            try:
+                generate_kwargs = {**kwargs}
+                system = next((m["content"] for m in messages if m["role"] == "system"), None)
+                user_messages = [m for m in messages if m["role"] != "system"]
+                for new_text in self._model.stream_chat(user_messages, system=system, **generate_kwargs):
+                    yield new_text
+            except Exception as exc:  # noqa: BLE001 - stream error as SSE event
+                yield 'data: {"error": "' + str(exc) + '"}\n\n'
+
+
+_inference_session = None
+
+
+def get_inference_session() -> InferenceSession:
+    global _inference_session
+    if _inference_session is None:
+        _inference_session = InferenceSession()
+    return _inference_session

compute/file_gateway/__init__.py

@@ -0,0 +1 @@
+"""Local file gateway package."""

compute/requirements.txt

@@ -0,0 +1,12 @@
+fastapi>=0.111.0
+uvicorn[standard]>=0.30.0
+python-multipart>=0.0.9
+pydantic>=2.7.0
+python-dotenv>=1.0.1
+httpx>=0.27.0
+# 模型评测指标
+sacrebleu>=2.4.0
+rouge-score>=0.1.2
+scikit-learn>=1.3.0
+# LLaMA-Factory 训练引擎
+llamafactory

compute/tests/__init__.py

@@ -0,0 +1 @@
+"""Compute platform tests package."""

compute/tests/test_inference_session.py

@@ -0,0 +1,146 @@
+from __future__ import annotations
+
+import sys
+import time
+import types
+from typing import Any
+
+import pytest
+
+from compute.engines.llama_factory.inference import InferenceSession
+
+# 模拟模型加载耗时，用于验证 load() 立即返回、info() 不阻塞
+LOAD_DELAY = 0.2
+
+
+class FakeChatModel:
+    def __init__(self, args: dict[str, Any]) -> None:
+        time.sleep(LOAD_DELAY)
+        self.tokenizer = object()
+        self.engine = types.SimpleNamespace(tokenizer=object())
+        self._output = "hello from model"
+
+    def stream_chat(self, *args, **kwargs):
+        for _ in range(1):
+            yield self._output
+
+
+class FailingChatModel:
+    def __init__(self, args: dict[str, Any]) -> None:
+        time.sleep(LOAD_DELAY)
+        raise RuntimeError("boom: fake load failure")
+
+
+def _get_infer_args(args: dict[str, Any]) -> list[Any]:
+    # 最后一个元素为 generating_args，worker 会转成 dict
+    return [None, None, {"temperature": 0.7}]
+
+
+def _install_llamafactory(monkeypatch, chat_model: type) -> None:
+    llmf = types.ModuleType("llamafactory")
+    chat_mod = types.ModuleType("llamafactory.chat")
+    hparams_mod = types.ModuleType("llamafactory.hparams")
+    chat_mod.ChatModel = chat_model
+    hparams_mod.get_infer_args = _get_infer_args
+    llmf.chat = chat_mod
+    llmf.hparams = hparams_mod
+    monkeypatch.setitem(sys.modules, "llamafactory", llmf)
+    monkeypatch.setitem(sys.modules, "llamafactory.chat", chat_mod)
+    monkeypatch.setitem(sys.modules, "llamafactory.hparams", hparams_mod)
+
+
+@pytest.fixture
+def stub_llamafactory(monkeypatch) -> None:
+    _install_llamafactory(monkeypatch, FakeChatModel)
+
+
+@pytest.fixture
+def stub_failing_llamafactory(monkeypatch) -> None:
+    _install_llamafactory(monkeypatch, FailingChatModel)
+
+
+def _wait_for_status(session: InferenceSession, status: str, timeout: float = 3.0) -> bool:
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        if session.info()["status"] == status:
+            return True
+        time.sleep(0.02)
+    return False
+
+
+def test_load_returns_immediately_then_ready(stub_llamafactory) -> None:
+    session = InferenceSession()
+    started = time.time()
+    result = session.load("/models/qwen")
+    assert result["status"] == "loading"
+    assert result["loaded"] is False
+    assert result["request_id"]
+    # 在慢加载完成前就返回，且 info() 加载期间可响应
+    assert time.time() - started < LOAD_DELAY
+    assert session.info()["status"] == "loading"
+    assert _wait_for_status(session, "ready")
+    info = session.info()
+    assert info["loaded"] is True
+    assert info["status"] == "ready"
+    assert info["model_name"] == "/models/qwen"
+
+
+def test_second_load_while_loading_deduped(stub_llamafactory) -> None:
+    session = InferenceSession()
+    r1 = session.load("/models/a")
+    r2 = session.load("/models/b")
+    assert r2["status"] == "loading"
+    assert r2["request_id"] == r1["request_id"]
+    assert _wait_for_status(session, "ready")
+    assert session.info()["status"] == "ready"
+
+
+def test_load_error_surfaces_in_status(stub_failing_llamafactory) -> None:
+    session = InferenceSession()
+    session.load("/models/bad")
+    assert _wait_for_status(session, "error")
+    assert "boom" in session.info()["error"]
+
+
+def test_unload_while_loading_cancels(stub_llamafactory) -> None:
+    session = InferenceSession()
+    session.load("/models/qwen")
+    result = session.unload()
+    assert result["status"] == "cancelling"
+    assert _wait_for_status(session, "idle")
+
+
+def test_chat_while_loading_returns_loading_error(stub_llamafactory) -> None:
+    session = InferenceSession()
+    session.load("/models/qwen")
+    out = session.chat([{"role": "user", "content": "hi"}])
+    assert "still loading" in (out.get("error") or "")
+    assert _wait_for_status(session, "ready")
+    out = session.chat([{"role": "user", "content": "hi"}])
+    assert out.get("response") == "hello from model"
+
+
+def test_chat_stream_while_loading_yields_error(stub_llamafactory) -> None:
+    session = InferenceSession()
+    session.load("/models/qwen")
+    chunks = list(session.chat_stream([{"role": "user", "content": "hi"}]))
+    assert any("still loading" in c for c in chunks)
+
+
+def test_wait_until_loaded_blocks_until_ready(stub_llamafactory) -> None:
+    session = InferenceSession()
+    result = session.load("/models/qwen")
+    assert result["status"] == "loading"
+    # 同步等待后台加载线程完成
+    outcome = session.wait_until_loaded(timeout=3.0)
+    assert outcome["loaded"] is True
+    assert outcome["status"] == "ready"
+
+
+def test_wait_until_loaded_reports_load_error(stub_failing_llamafactory) -> None:
+    session = InferenceSession()
+    session.load("/models/bad")
+    outcome = session.wait_until_loaded(timeout=3.0)
+    assert outcome["loaded"] is False
+    assert outcome["status"] == "error"
+    assert "boom" in outcome["error"]

compute/tests/test_llama_factory_adapter.py

@@ -0,0 +1,23 @@
+from __future__ import annotations
+
+from compute.engines.llama_factory.adapter import build_command
+
+
+def test_build_command_uses_explicit_validation_dataset_without_resplitting() -> None:
+    result = build_command(
+        {
+            "base_model": "/models/qwen",
+            "dataset": "ygft_dataset_train",
+            "eval_dataset": "ygft_dataset_validation",
+            "dataset_dir": "/datasets/example",
+            "output_dir": "/outputs/example",
+            "val_size": 0.1,
+        }
+    )
+
+    assert result.command[result.command.index("--dataset") + 1] == "ygft_dataset_train"
+    assert result.command[result.command.index("--eval_dataset") + 1] == (
+        "ygft_dataset_validation"
+    )
+    assert "--do_eval" in result.command
+    assert "--val_size" not in result.command

design-qa.md

@@ -158,3 +158,323 @@ The `type-check` script now uses project-reference mode (`vue-tsc -b --noEmit`)
 Design-QA final result: passed
 
 final result: passed
+
+---
+
+# Service Dashboard Design QA
+
+## Evidence
+
+- Source visual truth: `docs/superpowers/specs/assets/service-dashboard-approved-1440.png`
+- Browser-rendered implementation: `docs/superpowers/specs/assets/service-dashboard-implementation-1440.png`
+- Viewport: 1440 × 1024
+- State: authenticated `admin` user on `/dashboard`; service dashboard navigation active; 7-day chart visible; four training tasks visible.
+
+## Full-view comparison
+
+The implementation preserves the approved composition: the product shell stays intact, the service dashboard is the active navigation item, the platform-health summary spans the top, the grouped training chart occupies the wide middle track, service health occupies the narrow track, and the training-task table spans the bottom. The three bar series, dual axes, dates, values, service counts, task names, statuses, progress, accuracy, and actions match the selected mock.
+
+A separate focused crop was not required because both source and implementation evidence are full-resolution desktop captures at a readable scale; the chart labels, axis units, service rows, and every task-table column are legible in the full-view comparison.
+
+## Required fidelity surfaces
+
+- Fonts and typography: the existing Inter/system/PingFang stack is retained. Heading, section title, metric, table header, and muted-copy weights and sizes match the selected direction.
+- Spacing and layout rhythm: 24px page padding, 14px section gaps, 10px panel radii, light separators, and the wide-chart/narrow-status grid preserve the selected hierarchy. The implementation uses the repository's 240px sidebar and 60px header exactly.
+- Colors and visual tokens: white page canvas, `#f3f5f8` shell, indigo `#4f46e5`, green `#10b981`, amber `#f59e0b`, red `#ef4444`, and slate text are aligned with the source and current product tokens.
+- Image and icon fidelity: the page contains no decorative raster imagery. The supplied product logo is preserved and existing Font Awesome icons are used consistently; no emoji, handcrafted SVG, placeholder image, or CSS illustration was introduced.
+- Copy and content: dashboard title, health summary, chart legend and units, service states, task names, task status, model names, progress, accuracy, timestamps, and action labels match the approved design.
+
+## Interaction and runtime checks
+
+- Login with the existing `admin` credentials navigated to `/dashboard`, confirming the requested default entry behavior.
+- ECharts rendered one canvas; hovering 07/10 exposed the tooltip values: training count 18, GPU count 7, and average accuracy 91%.
+- “查看全部任务” navigated to `/fine-tune` and browser back restored `/dashboard`.
+- The first “查看详情” action navigated to `/training-log/103942` and browser back restored `/dashboard`.
+- Browser console errors: none.
+- `npm run test:default-dashboard`: passed.
+- `npm run test:dashboard`: passed.
+- `npx vite build`: passed.
+
+## Comparison history
+
+### Iteration 1 — passed
+
+No actionable P0/P1/P2 differences remain. The only intentional product constraint is that the sidebar active background follows the repository's current neutral active token instead of the slightly bluer tint produced by ImageGen; location, contrast, label, and active-state clarity remain equivalent.
+
+## Validation gap
+
+`npm run type-check` remains blocked by pre-existing TypeScript errors in the mock adapter, dataset mock typing, data-process list, evaluation tabs, and fine-tune views. No reported error points to `DashboardView.vue`, the ECharts registration, router defaults, login redirect, or dashboard regression scripts. The direct Vite production build succeeds.
+
+Design-QA final result: passed
+
+final result: passed
+
+## Compact dashboard revision
+
+- User annotation reference: `/var/folders/nk/yks07zp14wb4rv3jqq0pt_4h0000gn/T/codex-clipboard-43f63327-063f-47da-9e96-31b53cebc49d.png`
+- Revised implementation screenshot: `docs/superpowers/specs/assets/service-dashboard-compact-1440.png`
+- Viewport: 1440 × 1024
+- State: authenticated dashboard, compact layout, redundant title/action row removed.
+
+### Iteration 2 — passed
+
+The annotated header row containing the duplicate “服务看板” title, subtitle, and “查看告警” action was removed entirely. Section gaps, overview height, health icon, metric type, chart height, service rows, task heading, and task rows were reduced by roughly 10%–15%. The result preserves chart labels, dual-axis readability, service-state text, task progress, accuracy, and all task actions while bringing the primary content closer to the top of the page.
+
+- ECharts tooltip remains functional after the height reduction and reports all three 07/10 series values.
+- The revised page contains no browser console errors.
+- The full-resolution comparison makes the removed annotation target and the compact replacement legible; no focused crop is necessary.
+- `npm run test:dashboard`, `npm run test:default-dashboard`, and `npx vite build` pass.
+
+Design-QA final result: passed
+
+final result: passed
+
+## One-screen dashboard revision
+
+- User annotation reference: `/var/folders/nk/yks07zp14wb4rv3jqq0pt_4h0000gn/T/codex-clipboard-3c166ffc-7243-4e57-94fc-48949599c4f1.png`
+- Revised implementation screenshot: `docs/superpowers/specs/assets/service-dashboard-one-screen-1440x768.png`
+- Viewport: 1440 × 768
+- State: authenticated dashboard with the desktop low-height compact rules active.
+
+### Iteration 3 — passed
+
+The platform-status block was reduced again, including its container padding, inner gap, health icon, status copy, metric labels, and metric values. The chart, service rows, task rows, and page-canvas padding now use a dedicated `max-height: 900px` desktop mode. The dashboard page canvas is constrained to the available application viewport so the outer content area does not introduce a vertical scrollbar.
+
+Browser measurements at 1440 × 768:
+
+- document overflow: false
+- layout-content overflow: false
+- page-canvas overflow: false
+- dashboard overflow: false
+- task section bottom: 653px within the 768px viewport
+- ECharts tooltip: passed with all three series present
+- browser console errors: none
+- `npm run test:dashboard`, `npm run test:default-dashboard`, and `npx vite build`: passed
+
+Design-QA final result: passed
+
+final result: passed
+
+## Flexible middle-region revision
+
+- User annotation reference: `/var/folders/nk/yks07zp14wb4rv3jqq0pt_4h0000gn/T/codex-clipboard-daf2bbae-baea-418d-9962-7d0e1a1c219b.png`
+- Revised implementation screenshot: `docs/superpowers/specs/assets/service-dashboard-flex-middle-1440x900.png`
+- Viewport: 1440 × 900
+- State: authenticated dashboard with flexible middle-region growth.
+
+### Iteration 4 — passed
+
+The previous fixed-height middle row caused unused white space beneath the task table on taller screens. The dashboard now reserves compact intrinsic height for the platform summary and task table while allowing the chart/service row to consume all remaining viewport height. The ECharts canvas grows with that row, and the service-state rows distribute across the matching height.
+
+Browser measurements:
+
+- at 1440 × 768, chart height: 283px; no document, layout, canvas, or dashboard overflow
+- at 1440 × 900, chart height: 415px; no document, layout, canvas, or dashboard overflow
+- task table bottom at 1440 × 900: 868px within the 900px viewport
+- ECharts tooltip: passed with all three series present
+- browser console errors: none
+- `npm run test:dashboard`, `npm run test:default-dashboard`, and `npx vite build`: passed
+
+Design-QA final result: passed
+
+final result: passed
+
+## Narrow service-status revision
+
+- User request: make the right-hand service-status panel slightly narrower.
+- Revised implementation screenshot: `docs/superpowers/specs/assets/service-dashboard-narrow-service-1440x900.png`
+- Viewports: 1440 × 900 and 1440 × 768
+- State: authenticated dashboard with the service-status column reduced to approximately 30% of the middle row.
+
+### Iteration 5 — passed
+
+The middle grid now allocates `1.9fr` to the training chart and `0.82fr` to service status, with a 300px minimum width for the service panel. This gives the chart more horizontal space while keeping all service names, status badges, and instance counts fully visible.
+
+Browser measurements:
+
+- at 1440 × 900, chart width: 799px; service width: 345px; service share: 30.1%
+- at 1440 × 768, chart width: 799px; service width: 345px
+- clipped service cells: none at both tested viewports
+- horizontal and vertical document overflow: none at both tested viewports
+- browser console errors: none
+- `npm run test:dashboard`, `npm run test:default-dashboard`, and `npx vite build`: passed
+
+Design-QA final result: passed
+
+final result: passed
+
+## Taller training-task revision
+
+- User request: increase the training-task region slightly and shorten the middle chart/service region.
+- Source visual truth: `docs/superpowers/specs/assets/service-dashboard-narrow-service-1440x900.png` plus the current user annotation.
+- Intended viewports: 1440 × 900 and 1440 × 768.
+- State: authenticated dashboard with larger task heading and table rows.
+
+### Iteration 6 — blocked
+
+The task section now uses a taller heading and table rows in both standard and low-height desktop modes. Because the middle row is the only flexible region, the additional task height is taken directly from the chart/service row while preserving the one-screen layout contract in code.
+
+Verification evidence:
+
+- `npm run test:dashboard`: passed
+- `npm run test:default-dashboard`: passed
+- `npx vite build`: passed
+- browser-rendered comparison: blocked because the in-app browser rejected the local preview URL under its URL security policy
+- implementation screenshot: unavailable for this iteration
+
+Design-QA final result: blocked
+
+final result: blocked
+
+---
+
+# Failed Result Regeneration Design QA
+
+## Evidence
+
+- Source visual truth: `/var/folders/nk/yks07zp14wb4rv3jqq0pt_4h0000gn/T/codex-clipboard-0c2e6f90-aaac-4817-9084-bc31496aed0a.png`.
+- Implementation route: `http://localhost:16801/data-process/:id/workflow`, step 6.
+- Browser evidence: `/Users/caoxiaozhu/.codex/visualizations/2026/07/27/019fa277-626b-76a1-af02-48948d66c7ec/data-process-empty-list.png`.
+- Viewport: 1280 × 720.
+- Target state: a completed task on step 6 with a selected invalid result.
+- Available browser state: authenticated data-process list with zero tasks; no representative invalid result could be opened without creating or mutating local business data.
+
+## Static and automated evidence
+
+- Invalid results render a primary plain `重新生成` button in the right side of the result header.
+- Valid results keep the existing `恢复生成结果` action.
+- While regeneration is running, the selected row shows a spinner, the button shows loading, editors are disabled, and the footer confirmation action is disabled.
+- A successful response replaces only the selected row in place; a failed second generation leaves the original invalid row untouched.
+- `regression-data-process-wizard.mjs`: passed.
+- `regression-data-process-detail.mjs`: passed.
+- `regression-data-process-list.mjs`: passed.
+- `vue-tsc -b --noEmit`: passed.
+- Relevant backend tests: 97 passed, with one third-party deprecation warning.
+
+## Required fidelity surfaces
+
+- Typography and hierarchy: the action uses the existing Element Plus small-button hierarchy and remains secondary to result content.
+- Spacing and layout rhythm: the action is placed in the existing flex header's right action slot, matching the source location.
+- Colors and tokens: the action uses the existing product primary color and existing Font Awesome refresh icon.
+- Copy and content: the visible label is exactly `重新生成`; only server-saved invalid results expose it.
+- Interaction: optimistic concurrency protects against overwriting a newer edit, and persistence happens only after the replacement passes generation and quality validation.
+
+## Findings
+
+- [P2] Representative rendered comparison unavailable
+  Location: step 6 result header with an invalid result selected.
+  Evidence: the authenticated local database currently contains zero data-processing tasks, so the target state cannot be reached without creating test business data or invoking the configured model.
+  Impact: automated and static checks prove the contract and placement, but cannot prove pixel-level spacing against the supplied screenshot.
+  Fix: open any existing failed result after one is available, capture the same 1280 × 720 state, and compare it side by side with the source screenshot.
+
+## Comparison history
+
+### Iteration 1 — blocked
+
+- Source screenshot was inspected and the implementation was aligned to its existing right-header action slot.
+- Local login and list navigation succeeded.
+- The target result state was unavailable because the local task list was empty.
+
+final result: blocked
+
+---
+
+# Dataset Version Actions Design QA
+
+## Evidence
+
+- Source visual truth: `/Users/caoxiaozhu/.codex/generated_images/019f5e2e-3bee-77f0-b285-89ef139db56c/exec-48a163b6-d4c9-4646-9199-135957c6e72e.png`
+- Historical-version implementation: `/Users/caoxiaozhu/.codex/visualizations/2026/07/14/019f5e2e-3bee-77f0-b285-89ef139db56c/dataset-version-build/implementation-historical-version-final.png`
+- Delete-confirm implementation: `/Users/caoxiaozhu/.codex/visualizations/2026/07/14/019f5e2e-3bee-77f0-b285-89ef139db56c/dataset-version-build/implementation-delete-confirm.png`
+- Full-view comparison: `/Users/caoxiaozhu/.codex/visualizations/2026/07/14/019f5e2e-3bee-77f0-b285-89ef139db56c/dataset-version-build/comparison-full.png`
+- Focused version-control comparison: `/Users/caoxiaozhu/.codex/visualizations/2026/07/14/019f5e2e-3bee-77f0-b285-89ef139db56c/dataset-version-build/comparison-version-controls-final.png`
+- Viewport: 1316 × 768 browser window; source crop normalized for the focused comparison.
+- State: authenticated dataset detail, V3 current, V2 selected as a read-only historical version.
+
+## Full-view comparison
+
+The existing product shell, dataset summary, version selector, read-only alert, file toolbar, and sample table remain unchanged. The former standalone primary action has been replaced by one compact rounded-square overflow button at the far right of the version-control row, matching the selected hierarchy.
+
+## Focused comparison and required fidelity surfaces
+
+- Fonts and typography: existing system/PingFang stack, 13px labels, 12px metadata, and Element Plus menu text are preserved.
+- Spacing and layout rhythm: the 40px overflow trigger aligns with the version selector and leaves the central status copy flexible; the 168px menu provides 40px action rows.
+- Colors and visual tokens: the existing indigo primary token is used for the activate icon; the delete item and confirmation action use the danger token.
+- Image and icon fidelity: no new raster assets are needed. Existing Font Awesome ellipsis, check-circle, and trash icons match the repository's icon system.
+- Copy and content: the menu contains exactly “设为当前版本” and “删除版本”, separated visually; the confirmation names V2 and explains that current V3 is unaffected.
+
+## Interaction checks
+
+- Created V2 and V3 through the real edit-and-save flow, then switched from current V3 to historical V2.
+- Historical records became read-only and the overflow trigger appeared; current V3 showed no history-operation trigger.
+- Opening the trigger exposed exactly two accessible menu items: “设为当前版本” and “删除版本”.
+- Choosing “删除版本” opened the danger confirmation dialog; cancelling returned focus without deleting data.
+- Actual deletion behavior, protected-version rejection, optimistic-lock handling, and non-reused version numbers are covered by `test:dataset-preview`.
+
+## Comparison history
+
+### Iteration 1 — blocked
+
+- [P2] The overflow trigger was circular while the selected mock used a small rounded square.
+- [P2] The menu did not explicitly lock its target width or primary-action icon color.
+
+Fixes:
+
+- Replaced the circular trigger with a 40px square and 10px radius.
+- Set the menu minimum width to 168px, action height to 40px, and the activate icon to the product primary color.
+
+### Iteration 2 — passed
+
+No actionable P0/P1/P2 differences remain. The desktop capture API does not retain the transient popup layer in screenshots, so the open-menu labels were additionally verified through the accessibility tree; exact popup shadow rendering remains a non-blocking P3 capture gap.
+
+final result: passed
+
+---
+
+# Login Page Responsive Design QA
+
+## Evidence
+
+- Source visual truth: `/Users/caoxiaozhu/.codex/generated_images/019f5f3f-f7e1-7e41-9605-ea307d9f09e6/exec-d376c603-0c47-45ad-86fd-3e99c1a95ec7.png`
+- Implementation route: `http://localhost:6801/login`
+- Implementation screenshot: unavailable because the in-app browser runtime could not initialize in this session.
+- Intended desktop viewport: 1536 × 1024.
+- Intended laptop viewports: 1366 × 768 and 1280 × 720.
+- State: unauthenticated login page, default username and password populated.
+
+## Static and automated evidence
+
+- Added a 1440px width breakpoint that shifts the split from 58/42 to 55/45 and caps the form at 440px.
+- Added a short-screen breakpoint for heights up to 820px that reduces title, form, input, footer, and panel spacing without hiding the left visual.
+- Kept the single-column fallback at 900px and below.
+- `regression-login-layout.mjs`: passed.
+- `regression-default-dashboard.mjs`: passed.
+- `vue-tsc -b --noEmit`: passed.
+- Vite dev transform for `LoginView.vue`: HTTP 200.
+- Production build: blocked by the pre-existing missing route component `UserPermissionView.vue`, outside the login-page change.
+
+## Required fidelity surfaces
+
+- Fonts and typography: code uses the existing product font stack with laptop-specific display-size reductions; visual comparison remains unavailable.
+- Spacing and layout rhythm: dedicated width and height media queries are present; rendered measurements remain unavailable.
+- Colors and visual tokens: existing indigo tokens and the selected dark-purple visual asset are preserved.
+- Image quality and asset fidelity: the generated `login-hero-flow.png` is used directly; the official `logo.png` is reused for the brand lockup.
+- Copy and content: platform title, supporting copy, form labels, actions, and footer match the selected design.
+
+## Findings
+
+- [P2] Browser-rendered laptop comparison unavailable
+  Location: login page at 1366 × 768 and 1280 × 720.
+  Evidence: the in-app browser runtime failed during initialization, so no implementation screenshot or side-by-side comparison could be captured.
+  Impact: static checks cannot prove that all visible spacing and crop details match the selected design at laptop sizes.
+  Fix: capture both laptop viewports in a working in-app browser session, compare them with the source, and resolve any remaining P0/P1/P2 differences.
+
+## Comparison history
+
+### Iteration 1 — blocked
+
+- User reported that the initial implementation was optimized for large displays and did not compose well on laptop screens.
+- Added explicit laptop-width and short-screen layout rules and passed targeted regression/type checks.
+- Post-fix visual evidence remains unavailable because browser capture is blocked.
+
+final result: blocked

docker/README.md

@@ -0,0 +1,299 @@
+# Docker 部署说明
+
+本目录按应用服务器和算力服务器拆分 Dockerfile 与 Docker Compose 文件。Compose 文件不包含 `build:`，不会在 `docker compose up` 时自动构建业务镜像。所有业务镜像需要先通过手动 `docker build` 构建，再由 Compose 启动。
+
+## 基础镜像
+
+| 镜像 | 用途 |
+| --- | --- |
+| `python:3.12-slim` | 应用后端基础镜像，后端运行环境要求 Python 3.12 及以上 |
+| `nginx:1.27-alpine` | 前端静态资源与 `/modelTF` 反向代理运行镜像 |
+| `hiyouga/llamafactory:latest` | 算力服务基础镜像，基于 LLaMA-Factory 官方镜像扩展 Compute API |
+| `postgres:16-alpine` | 开发阶段内置 PostgreSQL |
+| `redis:7-alpine` | 开发阶段内置 Redis |
+
+一键拉取基础镜像：
+
+```bash
+docker pull python:3.12-slim && \
+docker pull nginx:1.27-alpine && \
+docker pull hiyouga/llamafactory:latest && \
+docker pull postgres:16-alpine && \
+docker pull redis:7-alpine
+```
+
+Windows PowerShell：
+
+```powershell
+$images = @(
+  "python:3.12-slim",
+  "nginx:1.27-alpine",
+  "hiyouga/llamafactory:latest",
+  "postgres:16-alpine",
+  "redis:7-alpine"
+)
+$images | ForEach-Object { docker pull $_ }
+```
+
+如果部署环境不能访问外网，需要提前在可联网环境执行上述拉取命令，再用 `docker save` / `docker load` 导出导入。
+
+## 业务镜像
+
+| 镜像 | Dockerfile | 构建命令 |
+| --- | --- | --- |
+| `yg-ft-backend-api:latest` | `docker/app/Dockerfile.backend` | `docker build -f docker/app/Dockerfile.backend -t yg-ft-backend-api:latest .` |
+| `yg-ft-frontend-runtime:latest` | `docker/app/Dockerfile.frontend` | `docker build -f docker/app/Dockerfile.frontend -t yg-ft-frontend-runtime:latest .` |
+| `yg-ft-compute-api:latest` | `docker/compute/Dockerfile.compute` | `docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .` |
+
+## 对外端口
+
+所有宿主机对外端口统一使用 5 位端口。容器内部端口保持镜像默认端口，便于容器内服务和健康检查稳定。
+
+| 服务 | 宿主机对外端口 | 容器内部端口 | 说明 |
+| --- | --- | --- | --- |
+| 前端 Nginx | `16801` | `80` | 前端页面入口 |
+| 后端 API | `17861` | `8000` | FastAPI 服务 |
+| PostgreSQL | `15432` | `5432` | 开发阶段内置数据库 |
+| Redis | `16379` | `6379` | 开发阶段内置缓存 |
+| Compute API | `19100` | `9100` | 算力服务器 API |
+| File Gateway | `19101` | `9100` | 当前由 Compute API 暴露文件网关契约，后续可拆为独立服务 |
+
+注意：`8000` 是后端容器内部端口，不作为宿主机对外访问端口。宿主机或浏览器应访问 `http://<app-server-ip>:17861/modelTF/health`；前端 Nginx 容器在 Docker 网络内部访问 `http://backend-api:8000/modelTF/...`。
+
+对应配置文件：
+
+- `docker/app/.env.example`
+  - `FRONTEND_PORT=16801`
+  - `BACKEND_API_PORT=17861`
+  - `POSTGRES_PORT=15432`
+  - `REDIS_PORT=16379`
+- `docker/compute/.env.example`
+  - `COMPUTE_API_PORT=19100`
+  - `FILE_GATEWAY_PORT=19101`
+
+## 运行模式
+
+- 应用侧默认 `COMPUTE_MODE=real`，任务状态必须由真实算力同步逻辑更新。
+- 算力侧默认 `COMPUTE_EXECUTION_MODE=real`，真实执行器未完成前不会伪造训练作业。
+- 仅隔离联调时可显式设置 `COMPUTE_MODE=simulator` 或 `COMPUTE_EXECUTION_MODE=simulator`，该模式不得用于测试环境、生产环境或生产升级基线。
+
+## 应用服务器部署
+
+应用服务器包含前端 Nginx、Backend API、PostgreSQL、Redis。
+
+当前 Compose 内置 PostgreSQL 使用 `backend/app/db/sql/001_platform_runtime.sql` 初始化运行库。`docs/postgres-schema.sql` 是完整目标架构设计，不应直接挂载为当前运行库初始化脚本，否则会与当前后端代码的运行表结构不兼容。
+
+首次部署：
+
+```bash
+cd <repo-root>
+
+# 1. 使用当前 Windows/宿主机 npm 构建前端静态产物
+cd frontend
+npm ci
+npm run build
+cd ..
+
+# 2. 手动构建业务镜像
+docker build -f docker/app/Dockerfile.backend -t yg-ft-backend-api:latest .
+docker build -f docker/app/Dockerfile.frontend -t yg-ft-frontend-runtime:latest .
+
+# 3. 启动应用服务
+cd docker/app
+cp .env.example .env
+docker compose up -d
+```
+
+后端镜像构建过程中会执行依赖导入自检，确认 `fastapi`、`uvicorn`、`psycopg`、`sqlalchemy`、`redis` 等运行依赖已安装。构建后也可以手动检查：
+
+```bash
+docker run --rm yg-ft-backend-api:latest python -c "import psycopg; print(psycopg.__version__)"
+```
+
+默认访问地址：
+
+```text
+http://<app-server-ip>:16801
+```
+
+应用侧代码和数据外挂：
+
+```text
+../../backend -> /app
+../../frontend/dist -> /usr/share/nginx/html
+../../runtime/app/logs/backend -> /opt/yg-ft/logs/backend
+../../runtime/app/data -> /data/yg-ft
+```
+
+前端容器启动前必须确保 `../../frontend/dist/index.html` 已存在。若前端 Nginx 日志出现 `directory index of "/usr/share/nginx/html/" is forbidden` 或 `rewrite or internal redirection cycle while internally redirecting to "/index.html"`，通常表示当前执行 `docker compose` 的项目目录下没有构建好的 `frontend/dist`，或挂载路径不是同一份代码目录。
+
+```bash
+# 在执行 docker compose 的同一份代码目录中检查
+cd <repo-root>/frontend
+npm run build
+test -f dist/index.html && ls -lh dist/index.html
+
+cd ../docker/app
+docker compose up -d --force-recreate frontend
+docker compose logs --tail=80 frontend
+```
+
+如果使用 Windows npm 构建、WSL 中运行 Docker Compose，需要确认 Windows 路径和 WSL 路径指向同一份仓库。例如在 `D:\...\YG_FT\frontend` 构建不会自动生成 `/mnt/d/wuyongtao/Code/YG_FT/frontend/dist` 下的产物，除非二者本就是同一个目录。
+
+如果使用企业统一 PostgreSQL/Redis，修改 `docker/app/.env`：
+
+如果前端 Nginx 日志出现 `open() "/usr/share/nginx/html/modelTF/login" failed` 或 `open() "/usr/share/nginx/html/login" failed`，说明当前容器没有加载项目的 Nginx 代理配置，`/modelTF/*` 被当成静态文件查找。处理方式：
+
+```bash
+cd <repo-root>/docker/app
+docker compose up -d --force-recreate frontend
+docker compose exec frontend nginx -T | grep -n "location.*modelTF" -A12
+```
+
+正常配置中应存在 `location ^~ /modelTF/`，并代理到 `BACKEND_PROXY_PASS`，默认是 `http://backend-api:8000`。
+
+```env
+DATABASE_URL=postgresql+psycopg://<user>:<password>@<postgres-host>:15432/<db>
+REDIS_URL=redis://<redis-host>:16379/0
+USE_BUILTIN_POSTGRES=false
+USE_BUILTIN_REDIS=false
+```
+
+生产环境如完全使用外部基础设施，可以删除或注释 Compose 中的 `postgres`、`redis` 服务及 `backend-api.depends_on` 中对应依赖。
+
+## 算力服务器部署
+
+算力服务器包含 Compute API、后续 Compute Agent、File Gateway、GPU runtime、本地训练数据目录和 LLaMA-Factory。`Dockerfile.compute` 基于 LLaMA-Factory 官方镜像：
+
+```dockerfile
+FROM hiyouga/llamafactory:latest
+```
+
+部署前需要安装：
+
+- NVIDIA Driver
+- NVIDIA Container Toolkit
+- Docker Engine 和 Docker Compose Plugin
+- 本地训练数据目录，默认 `/data/yg-ft`
+
+首次部署：
+
+```bash
+cd <repo-root>
+
+# 手动构建算力业务镜像
+docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .
+
+# 启动算力服务
+cd docker/compute
+cp .env.example .env
+docker compose up -d
+```
+
+健康检查：
+
+```text
+GET http://<compute-server-ip>:19100/modelTF/health
+GET http://<compute-server-ip>:19100/modelTF/v1/compute/health
+```
+
+算力侧代码和数据外挂：
+
+```text
+../../compute -> /app/compute
+${YG_FT_DATA_ROOT_HOST} -> /data/yg-ft
+${YG_FT_MODEL_ROOT_HOST} -> /data/yg-ft/models
+${YG_FT_DATASET_ROOT_HOST} -> /data/yg-ft/datasets
+${YG_FT_OUTPUT_ROOT_HOST} -> /data/yg-ft/outputs
+${COMPUTE_LOG_ROOT_HOST} -> /opt/yg-ft/logs/compute
+${TRAINING_LOG_ROOT_HOST} -> /opt/yg-ft/logs/training
+```
+
+算力服务器启动前必须先在宿主机创建持久化目录，基座模型、训练数据、训练产物和训练日志都应落在宿主机磁盘上，不能只写入容器层。推荐默认目录：
+
+```bash
+cd <repo-root>/docker/compute
+mkdir -p data/yg-ft/models \
+  data/yg-ft/datasets \
+  data/yg-ft/outputs \
+  data/yg-ft/logs/compute \
+  data/yg-ft/logs/training
+```
+
+默认 `docker/compute/.env.example` 使用 `./data/yg-ft`，该相对路径以 `docker/compute/docker-compose.yml` 所在目录为基准，因此实际宿主机目录是 `<repo-root>/docker/compute/data/yg-ft`。如企业环境模型盘、数据盘、产物盘分盘挂载，可在 `docker/compute/.env` 中分别调整 `YG_FT_MODEL_ROOT_HOST`、`YG_FT_DATASET_ROOT_HOST`、`YG_FT_OUTPUT_ROOT_HOST`、`COMPUTE_LOG_ROOT_HOST`、`TRAINING_LOG_ROOT_HOST`，容器内路径建议保持 `/data/yg-ft/models`、`/data/yg-ft/datasets`、`/data/yg-ft/outputs`，避免训练参数和节点配置复杂化。
+
+页面上传数据集时，文件先进入 Backend API，再由 Backend API 调用目标算力节点的 `POST /modelTF/compute/files/upload`，写入容器内 `/data/yg-ft/datasets/{dataset_id}/`。在默认开发配置下，宿主机可在 `<repo-root>/docker/compute/data/yg-ft/datasets/{dataset_id}/` 看到对应文件。仅创建 bind mount 不会自动让应用侧上传文件出现在算力目录，必须通过这条 File Gateway 链路同步。
+
+## 应用与算力分离部署
+
+应用服务器只需要主动访问算力服务器，不要求算力服务器回调应用服务器。
+
+在 `docker/app/.env` 中配置：
+
+```env
+COMPUTE_API_BASE_URL=http://<compute-server-ip>:19100
+FILE_GATEWAY_BASE_URL=http://<compute-server-ip>:19101
+COMPUTE_SERVICE_TOKEN=change_me
+COMPUTE_STATUS_SYNC_MODE=polling
+COMPUTE_POLL_INTERVAL_SECONDS=3
+COMPUTE_POLL_BATCH_SIZE=100
+```
+
+交互链路：
+
+```text
+Frontend
+  -> Backend API
+    -> Compute API
+      -> Compute Agent / LLaMA-Factory
+      -> 本地数据目录 / 模型目录 / 训练产物
+    <- Backend Worker 定时轮询 Compute API
+```
+
+算力服务默认开启服务间鉴权。`docker/compute/.env` 中保持 `COMPUTE_AUTH_ENABLED=true`，并确保 `COMPUTE_SERVICE_TOKEN` 与 `docker/app/.env` 一致；健康检查路径仍可用于容器探活。
+
+## 多算力节点部署
+
+多算力节点仍按“单机多 GPU 节点”部署。每台 GPU 服务器都独立部署一套 `docker/compute`：
+
+```text
+gpu-node-01: docker/compute + /data/yg-ft + 19100/19101
+gpu-node-02: docker/compute + /data/yg-ft + 19100/19101
+gpu-node-03: docker/compute + /data/yg-ft + 19100/19101
+```
+
+节点之间默认不互访。应用平台主动访问每个节点的 Compute API/File Gateway，并通过 `compute_nodes`、`resource_replicas`、`resource_sync_jobs` 统一调度和同步。
+
+节点地址、权重、标签、启用状态和本地路径在前端“算力节点”页面动态维护。新增或编辑节点后，点击“测试”会由 Backend API 主动访问该节点的 `GET /modelTF/v1/compute/health` 和 `GET /modelTF/compute/resources/gpus`，并把健康信息与 GPU 清单同步到 PostgreSQL。
+
+## 常用命令
+
+重新构建应用镜像：
+
+```bash
+docker build -f docker/app/Dockerfile.backend -t yg-ft-backend-api:latest .
+docker build -f docker/app/Dockerfile.frontend -t yg-ft-frontend-runtime:latest .
+```
+
+重新构建算力镜像：
+
+```bash
+docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .
+```
+
+启动服务：
+
+```bash
+cd docker/app
+docker compose up -d
+
+cd ../compute
+docker compose up -d
+```
+
+查看服务：
+
+```bash
+docker compose ps
+docker compose logs -f
+```

docker/app/.env

@@ -0,0 +1,46 @@
+APP_ENV=prod
+APP_NAME=YG Fine-Tune Platform API
+MODELTF_ROUTE_PREFIX=/modelTF
+CORS_ALLOW_ORIGINS=http://localhost:16801,http://127.0.0.1:16801
+
+FRONTEND_IMAGE=yg-ft-frontend-runtime:latest
+BACKEND_API_IMAGE=yg-ft-backend-api:latest
+
+# Five-digit host ports exposed outside the application server.
+FRONTEND_PORT=16801
+BACKEND_API_PORT=17861
+POSTGRES_PORT=15432
+REDIS_PORT=16379
+
+POSTGRES_DB=yg_ft
+POSTGRES_USER=root
+POSTGRES_PASSWORD=8811614287327Leo
+DATABASE_URL=postgresql+psycopg://root:8811614287327Leo@www.caoxiaozhu.com:5432/yg_ft
+
+REDIS_URL=redis://redis:6379/0
+
+# PostgreSQL uses the shared external database. The local postgres service is disabled in docker-compose.yml.
+# Redis still uses the built-in service during current development.
+USE_BUILTIN_POSTGRES=false
+USE_BUILTIN_REDIS=true
+
+LOG_LEVEL=INFO
+LOG_DIR=/opt/yg-ft/logs/backend
+LOG_FILE_PREFIX=backend
+LOG_ERROR_FILE_PREFIX=error
+LOG_MAX_BYTES=20971520
+LOG_RETENTION_DAYS=10
+
+BACKEND_PROXY_PASS=http://backend-api:8000
+
+# Split deployment: set these to the compute server address, for example http://10.10.20.31:19100.
+COMPUTE_API_BASE_URL=http://compute-api:9100
+COMPUTE_SERVICE_TOKEN=change_me
+FILE_GATEWAY_BASE_URL=http://compute-api:9101
+
+# The application side polls Compute API for job state to avoid opening reverse network access.
+COMPUTE_MODE=real
+COMPUTE_STATUS_SYNC_MODE=polling
+COMPUTE_POLL_INTERVAL_SECONDS=10
+COMPUTE_POLL_BATCH_SIZE=100
+COMPUTE_REQUEST_TIMEOUT_SECONDS=5

docker/app/Dockerfile.backend

@@ -0,0 +1,21 @@
+FROM python:3.12-slim
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PIP_NO_CACHE_DIR=1
+
+WORKDIR /app
+
+COPY backend/requirements.txt /tmp/requirements.txt
+RUN pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple \
+    && pip install -r /tmp/requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple \
+    && rm -f /tmp/requirements.txt
+
+RUN python -c "import fastapi, uvicorn, psycopg, psycopg_pool, sqlalchemy, redis, jwt, passlib, httpx, alembic; print('backend dependency check ok')"
+
+RUN mkdir -p /opt/yg-ft/logs/backend /data/yg-ft \
+    && chmod -R 0775 /opt/yg-ft /data/yg-ft
+
+EXPOSE 8000
+
+CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

docker/app/Dockerfile.frontend

@@ -0,0 +1,9 @@
+
+FROM nginx:1.27-alpine
+
+RUN mkdir -p /usr/share/nginx/html
+
+EXPOSE 80
+
+HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
+  CMD test -f /usr/share/nginx/html/index.html && wget -qO- http://127.0.0.1/index.html >/dev/null || exit 1

docker/app/docker-compose.yml

@@ -0,0 +1,126 @@
+services:
+  frontend:
+    image: ${FRONTEND_IMAGE:-yg-ft-frontend-runtime:latest}
+    container_name: yg-ft-frontend
+    depends_on:
+      backend-api:
+        condition: service_started
+    ports:
+      - "${FRONTEND_PORT:-16801}:80"
+    environment:
+      BACKEND_PROXY_PASS: ${BACKEND_PROXY_PASS:-http://backend-api:8000}
+    volumes:
+      - ../../frontend/dist:/usr/share/nginx/html:ro
+      - ../nginx.conf.template:/etc/nginx/templates/default.conf.template:ro
+    command:
+      - /bin/sh
+      - -c
+      - |
+        if [ ! -f /usr/share/nginx/html/index.html ]; then
+          echo "frontend dist is missing: build frontend first and ensure ../../frontend/dist is mounted";
+          ls -la /usr/share/nginx/html;
+          exit 1;
+        fi;
+        envsubst '$$BACKEND_PROXY_PASS' < /etc/nginx/templates/default.conf.template > /etc/nginx/conf.d/default.conf;
+        nginx -t;
+        nginx -g 'daemon off;'
+    networks:
+      - yg-ft-app
+    restart: unless-stopped
+
+  backend-api:
+    image: ${BACKEND_API_IMAGE:-yg-ft-backend-api:latest}
+    container_name: yg-ft-backend-api
+    depends_on:
+      redis:
+        condition: service_healthy
+    expose:
+      - "8000"
+    ports:
+      - "${BACKEND_API_PORT:-17861}:8000"
+    environment:
+      APP_ENV: ${APP_ENV:-prod}
+      APP_NAME: ${APP_NAME:-YG Fine-Tune Platform API}
+      MODELTF_ROUTE_PREFIX: ${MODELTF_ROUTE_PREFIX:-/modelTF}
+      CORS_ALLOW_ORIGINS: ${CORS_ALLOW_ORIGINS:-http://localhost:16801,http://127.0.0.1:16801}
+      DATABASE_URL: ${DATABASE_URL:-postgresql+psycopg://root:8811614287327Leo@www.caoxiaozhu.com:5432/yg_ft}
+      REDIS_URL: ${REDIS_URL:-redis://redis:6379/0}
+      USE_BUILTIN_POSTGRES: ${USE_BUILTIN_POSTGRES:-false}
+      USE_BUILTIN_REDIS: ${USE_BUILTIN_REDIS:-true}
+      LOG_LEVEL: ${LOG_LEVEL:-INFO}
+      LOG_DIR: ${LOG_DIR:-/opt/yg-ft/logs/backend}
+      LOG_FILE_PREFIX: ${LOG_FILE_PREFIX:-backend}
+      LOG_ERROR_FILE_PREFIX: ${LOG_ERROR_FILE_PREFIX:-error}
+      LOG_MAX_BYTES: ${LOG_MAX_BYTES:-20971520}
+      LOG_RETENTION_DAYS: ${LOG_RETENTION_DAYS:-10}
+      COMPUTE_API_BASE_URL: ${COMPUTE_API_BASE_URL:-http://compute-api:9100}
+      COMPUTE_SERVICE_TOKEN: ${COMPUTE_SERVICE_TOKEN:-change_me}
+      FILE_GATEWAY_BASE_URL: ${FILE_GATEWAY_BASE_URL:-http://compute-api:9101}
+      COMPUTE_MODE: ${COMPUTE_MODE:-real}
+      COMPUTE_STATUS_SYNC_MODE: ${COMPUTE_STATUS_SYNC_MODE:-polling}
+      COMPUTE_POLL_INTERVAL_SECONDS: ${COMPUTE_POLL_INTERVAL_SECONDS:-3}
+      COMPUTE_POLL_BATCH_SIZE: ${COMPUTE_POLL_BATCH_SIZE:-100}
+      COMPUTE_REQUEST_TIMEOUT_SECONDS: ${COMPUTE_REQUEST_TIMEOUT_SECONDS:-5}
+      PYTHONPATH: /app
+    volumes:
+      - ../../backend:/app:ro
+      - ../../runtime/app/logs/backend:/opt/yg-ft/logs/backend
+      - ../../runtime/app/data:/data/yg-ft
+    networks:
+      - yg-ft-app
+    healthcheck:
+      test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/modelTF/health', timeout=3).read()\""]
+      interval: 30s
+      timeout: 5s
+      retries: 3
+      start_period: 20s
+    restart: unless-stopped
+
+  # PostgreSQL uses the shared external database configured by DATABASE_URL in docker/app/.env.
+  # Keep this local service commented out unless development needs an isolated database again.
+  # postgres:
+  #   image: postgres:16-alpine
+  #   container_name: yg-ft-postgres
+  #   environment:
+  #     POSTGRES_DB: ${POSTGRES_DB:-yg_ft}
+  #     POSTGRES_USER: ${POSTGRES_USER:-yg_ft}
+  #     POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-change_me}
+  #     PGDATA: /var/lib/postgresql/data/pgdata
+  #   volumes:
+  #     - postgres_data:/var/lib/postgresql/data
+  #     - ../../backend/app/db/sql/001_platform_runtime.sql:/docker-entrypoint-initdb.d/001-platform-runtime.sql:ro
+  #   ports:
+  #     - "${POSTGRES_PORT:-15432}:5432"
+  #   networks:
+  #     - yg-ft-app
+  #   healthcheck:
+  #     test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
+  #     interval: 10s
+  #     timeout: 5s
+  #     retries: 5
+  #   restart: unless-stopped
+
+  redis:
+    image: redis:7-alpine
+    container_name: yg-ft-redis
+    command: ["redis-server", "--appendonly", "yes"]
+    volumes:
+      - redis_data:/data
+    ports:
+      - "${REDIS_PORT:-16379}:6379"
+    networks:
+      - yg-ft-app
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 10s
+      timeout: 3s
+      retries: 5
+    restart: unless-stopped
+
+networks:
+  yg-ft-app:
+    name: yg-ft-app
+
+volumes:
+  # postgres_data:
+  redis_data: