feat: 增加 YAML 配置与一键启动脚本

Reviewed-on: #7

.gitignore

@@ -44,7 +44,6 @@ pip-delete-this-directory.txt
 # Runtime data and logs
 runtime/
 backend/runtime/
-backend/storage/
 logs/
 backend/logs/
 *.db
@@ -144,7 +143,6 @@ celerybeat.pid
 
 # Environments
 .env
-!.env.example
 .venv
 env/
 venv/
@@ -152,14 +150,6 @@ ENV/
 env.bak/
 venv.bak/
 
-# Local backend config (含数据库账号密码等敏感信息，勿提交)
-backend/config.yaml
-
-# Agent / IDE 工具产物，不应进版本库
-.codex-backups/
-.pnpm-store/
-.zcode/
-
 # Spyder project settings
 .spyderproject
 .spyproject
@@ -197,14 +187,5 @@ cython_debug/
 # PyPI configuration file
 .pypirc
 
-docker/llamafactory-latest.tar.gz
-# Compute data - 保留目录结构和 README，忽略子目录内容（日志、模型、数据集等）
-!docker/compute/data/yg-ft/logs/
-docker/compute/data/yg-ft/datasets/*
-docker/compute/data/yg-ft/models/*
-docker/compute/data/yg-ft/outputs/*
-docker/compute/data/yg-ft/logs/**
-!docker/compute/data/yg-ft/logs/compute/
-!docker/compute/data/yg-ft/logs/training/
-!docker/compute/data/yg-ft/**/.gitkeep
-!docker/compute/data/yg-ft/**/README.md
+# Local backend configuration may contain database credentials.
+backend/config.yaml

README.md

@@ -48,48 +48,36 @@ YG_FT/
 - 前端新增 `/compute` 算力节点页面，展示节点地址、权重、标签、启用状态、GPU、队列和资源副本。
 - `compute/engines/llama_factory/adapter.py` 提供 LLaMA-Factory 参数校验、命令生成和日志解析基础能力。
 
-## 前后端一键启动
-
-首次使用前，请先按下方“后端启动”和“前端启动”说明安装依赖，并确保
-PostgreSQL 已可用。之后在项目根目录执行：
+## 一键启动前端和后端
 
 ```bash
-bash ./start.sh
+./scripts/start-dev.sh --setup  # 首次运行，初始化本地依赖
+./scripts/start-dev.sh          # 后续直接启动
+./scripts/start-dev.sh --check  # 仅检查依赖和 PostgreSQL 连接
 ```
 
-脚本会同时启动前端 `http://localhost:16801` 和后端
-`http://127.0.0.1:17861`，按 `Ctrl+C` 会同时停止两个服务。脚本只负责
-启动前后端，不会自动安装依赖，也不会启动 PostgreSQL、Redis 或算力服务。
+首次运行前可将 `backend/config.example.yaml` 复制为 `backend/config.yaml` 并填写
+本地数据库凭据；`config.yaml` 已加入 Git 忽略。脚本会检查 PostgreSQL 和两个服务
+的健康状态，并在退出时同时停止前端和后端。也可以按下面步骤分别启动服务。
 
-仅检查依赖和端口而不启动服务：
+一键启动使用的前后端端口也统一配置在同一个 YAML 中：
 
-```bash
-bash ./start.sh --check
+```yaml
+server:
+  frontend_port: 16801
+  backend_port: 17861
 ```
 
-本地启动推荐只配置数据库主机。脚本会复用 `docker/app/.env` 中已有的
-`POSTGRES_USER`、`POSTGRES_PASSWORD` 和 `POSTGRES_DB`，端口默认使用
-PostgreSQL 标准端口 `5432`：
-
-```bash
-DATABASE_HOST='www.caoxiaozhu.com' bash ./start.sh
-```
-
-也可以在 `docker/app/.env` 中增加：
-
-```env
-DATABASE_HOST=www.caoxiaozhu.com
-```
-
-需要使用非标准端口时再设置 `DATABASE_PORT`。`DATABASE_URL` 仍可作为完整连接串
-高级覆盖项；终端环境变量优先级最高。脚本不会输出数据库密码。
+端口被占用时修改这两项后重新启动即可。未显式配置 `cors_allow_origins` 时，后端会
+根据 `frontend_port` 自动允许本机前端来源。环境变量 `FRONTEND_PORT` 和
+`BACKEND_PORT` 可以临时覆盖 YAML。
 
 ## 后端启动
 
 ```bash
 cd backend
-python -m venv .venv
-.venv\Scripts\activate
+/opt/miniconda3/bin/python3 -m venv .venv
+source .venv/bin/activate
 pip install -r requirements.txt
 uvicorn app.main:app --reload --port 17861
 ```
@@ -105,13 +93,22 @@ GET /modelTF/fine-tune
 GET /modelTF/compute/nodes
 ```
 
-本地运行时默认 PostgreSQL 连接：
+本地运行时默认通过 `backend/config.yaml` 连接 PostgreSQL：
 
-```text
-DATABASE_URL=postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft
+```yaml
+server:
+  frontend_port: 16801
+  backend_port: 17861
+
+database:
+  url: postgresql+psycopg://localhost:5432/yg_ft
+  username: yg_ft
+  password: "change_me"
 ```
 
-本地启动前需要确保 PostgreSQL 已监听 `localhost:15432`，并已创建 `yg_ft` 数据库和 `yg_ft` 用户。后端启动后会自动创建当前运行表并写入内置管理员账号，运行数据统一写入 PostgreSQL。
+本地启动前需要确保 PostgreSQL 已监听 YAML 配置的地址（默认 `localhost:5432`），
+并已创建 `yg_ft` 数据库和 `yg_ft` 用户。后端启动后会自动创建当前运行表并写入
+内置管理员账号，运行数据统一写入 PostgreSQL。
 
 开发阶段内置登录账号：
 
@@ -130,7 +127,9 @@ npm install
 npm run dev
 ```
 
-前端开发服务默认运行在 `http://localhost:16801`，并通过 Vite proxy 将 `/modelTF` 转发到 `http://localhost:17861`。
+通过一键脚本启动时，前端端口和 Vite proxy 的后端端口均来自
+`backend/config.yaml`。单独运行 `npm run dev` 时，默认使用前端 `16801`、后端
+`17861`，也可以通过 `FRONTEND_PORT` 和 `BACKEND_PORT` 覆盖。
 
 ## 算力服务启动
 

backend/.env.example

@@ -0,0 +1,18 @@
+# 默认自动读取 backend/config.yaml。设置此变量可切换到其他 YAML 文件。
+BACKEND_CONFIG_FILE=./config.yaml
+
+# 下列环境变量按需启用，并优先于 YAML 中的配置。
+# DATABASE_BASE_URL=postgresql+psycopg://localhost:5432/yg_ft
+# DATABASE_USERNAME=yg_ft
+# DATABASE_PASSWORD=change_me
+# DATABASE_URL=postgresql+psycopg://yg_ft:change_me@localhost:5432/yg_ft
+# DATABASE_URL 是完整连接串覆盖项，优先级高于上面三个分离项。
+# MODELTF_ROUTE_PREFIX=/modelTF
+# FRONTEND_PORT=16801
+# BACKEND_PORT=17861
+# CORS_ALLOW_ORIGINS=http://localhost:16801,http://127.0.0.1:16801
+# LOG_LEVEL=INFO
+# LOG_DIR=./logs
+# COMPUTE_MODE=real
+# COMPUTE_STATUS_SYNC_MODE=polling
+# COMPUTE_POLL_INTERVAL_SECONDS=3

backend/README.md

@@ -32,17 +32,65 @@ backend/
     services/                # 跨模块应用服务
     workers/                 # 后台任务入口
   requirements.txt           # 后端第三方依赖
+  config.example.yaml        # 可提交的脱敏配置模板
+  config.yaml                # 本地配置，已忽略，环境变量可覆盖
   logs/                      # 本地开发日志目录，生产环境建议挂载到独立日志盘
 ```
 
+## 配置
+
+先复制本地配置并填写数据库凭据：
+
+```bash
+cp config.example.yaml config.yaml
+```
+
+后端默认读取 `config.yaml`，配置优先级为：
+
+```text
+环境变量 > BACKEND_CONFIG_FILE 指定的 YAML > backend/config.yaml > 代码默认值
+```
+
+前后端端口、数据库地址、跨域来源、日志路径和计算状态轮询参数均可在 YAML 中
+配置。敏感信息或不同环境的差异建议通过环境变量覆盖，不要写入仓库。YAML 中的
+相对日志路径以该 YAML 文件所在目录为基准。使用其他配置文件时：
+
+```bash
+BACKEND_CONFIG_FILE=./config.prod.yaml uvicorn app.main:app --reload --port 17861
+```
+
+本机直连 PostgreSQL 时，数据库配置拆分为地址、用户名和密码：
+
+```yaml
+server:
+  frontend_port: 16801
+  backend_port: 17861
+
+database:
+  url: postgresql+psycopg://localhost:5432/yg_ft
+  username: yg_ft
+  password: "change_me"
+```
+
+服务启动时会安全拼装完整连接串；用户名或密码中的特殊字符会自动编码。Docker
+和生产环境仍可用完整的 `DATABASE_URL` 环境变量覆盖以上三项。
+
+这里的 `5432` 是宿主机 PostgreSQL 默认端口；如果数据库容器映射到 `15432`，
+将 YAML 中的端口改为 `15432` 即可。
+
+`server.frontend_port` 和 `server.backend_port` 由根目录的
+`scripts/start-dev.sh` 读取，并分别传给 Vite 和 Uvicorn。未配置
+`app.cors_allow_origins` 时，本机 CORS 来源会跟随前端端口。直接手工执行
+`uvicorn` 时仍需通过 `--port` 指定监听端口。
+
 ## 本地启动
 
 ```bash
 cd backend
-python -m venv .venv
-.venv\Scripts\activate
+/opt/miniconda3/bin/python3 -m venv .venv
+source .venv/bin/activate
 pip install -r requirements.txt
-uvicorn app.main:app --reload
+uvicorn app.main:app --reload --port 17861
 ```
 
 健康检查：

backend/app/api/v1/router.py

@@ -1,10 +1,9 @@
 from fastapi import APIRouter
 
-from app.api.v1.endpoints.data_process import router as data_process_router
 from app.api.v1.endpoints.platform import router as platform_router
 from app.api.v1.endpoints.health import router as health_router
 
 api_router = APIRouter()
 api_router.include_router(health_router, tags=["health"])
-api_router.include_router(data_process_router, tags=["data-process"])
 api_router.include_router(platform_router, tags=["platform"])
+

backend/app/core/config.py

@@ -1,59 +1,326 @@
-from dataclasses import dataclass
+from __future__ import annotations
+
+from dataclasses import dataclass, field
 from functools import lru_cache
 import os
+from pathlib import Path
+from typing import Any
+from urllib.parse import quote, urlsplit, urlunsplit
+
+import yaml
 
 
-def _int_env(name: str, default: int) -> int:
-    raw = os.getenv(name)
-    if raw is None or raw == "":
+BACKEND_ROOT = Path(__file__).resolve().parents[2]
+DEFAULT_CONFIG_PATH = BACKEND_ROOT / "config.yaml"
+DEFAULT_DATABASE_BASE_URL = "postgresql+psycopg://localhost:5432/yg_ft"
+DEFAULT_DATABASE_USERNAME = "yg_ft"
+DEFAULT_DATABASE_PASSWORD = "change_me"
+
+
+class ConfigurationError(ValueError):
+    """Raised when the backend configuration cannot be parsed or validated."""
+
+
+def _resolve_config_path(config_path: str | Path | None = None) -> tuple[Path, bool]:
+    configured_path = config_path or os.getenv("BACKEND_CONFIG_FILE")
+    is_explicit = configured_path is not None
+    path = Path(configured_path).expanduser() if configured_path else DEFAULT_CONFIG_PATH
+    if not path.is_absolute():
+        path = BACKEND_ROOT / path
+    return path.resolve(), is_explicit
+
+
+def _load_yaml(config_path: str | Path | None = None) -> tuple[dict[str, Any], Path]:
+    path, is_explicit = _resolve_config_path(config_path)
+    if not path.exists():
+        if is_explicit:
+            raise ConfigurationError(f"Backend config file does not exist: {path}")
+        return {}, path
+
+    try:
+        loaded = yaml.safe_load(path.read_text(encoding="utf-8"))
+    except yaml.YAMLError as exc:
+        raise ConfigurationError(f"Invalid YAML in backend config file {path}: {exc}") from exc
+
+    if loaded is None:
+        return {}, path
+    if not isinstance(loaded, dict):
+        raise ConfigurationError(f"Backend config root must be a mapping: {path}")
+    return loaded, path
+
+
+def _yaml_value(config: dict[str, Any], section: str, key: str, default: Any) -> Any:
+    section_value = config.get(section, {})
+    if section_value is None:
         return default
-    return int(raw)
+    if not isinstance(section_value, dict):
+        raise ConfigurationError(f"Config section '{section}' must be a mapping")
+    return section_value.get(key, default)
 
 
-def _list_env(name: str, default: list[str]) -> list[str]:
-    raw = os.getenv(name)
-    if raw is None or raw.strip() == "":
-        return default
-    return [item.strip() for item in raw.split(",") if item.strip()]
+def _string_setting(
+    env_name: str,
+    config: dict[str, Any],
+    section: str,
+    key: str,
+    default: str,
+) -> str:
+    env_value = os.getenv(env_name)
+    if env_value is not None:
+        return env_value
+    value = _yaml_value(config, section, key, default)
+    return default if value is None else str(value)
+
+
+def _int_setting(
+    env_name: str,
+    config: dict[str, Any],
+    section: str,
+    key: str,
+    default: int,
+) -> int:
+    env_value = os.getenv(env_name)
+    value = (
+        env_value
+        if env_value not in (None, "")
+        else _yaml_value(config, section, key, default)
+    )
+    if isinstance(value, (bool, float)):
+        raise ConfigurationError(f"Config value {section}.{key} must be an integer")
+    try:
+        return int(value)
+    except (TypeError, ValueError) as exc:
+        raise ConfigurationError(f"Config value {section}.{key} must be an integer") from exc
+
+
+def _port_setting(
+    env_name: str,
+    config: dict[str, Any],
+    key: str,
+    default: int,
+) -> int:
+    port = _int_setting(env_name, config, "server", key, default)
+    if not 1 <= port <= 65535:
+        raise ConfigurationError(
+            f"Config value server.{key} must be between 1 and 65535"
+        )
+    return port
+
+
+def _list_setting(
+    env_name: str,
+    config: dict[str, Any],
+    section: str,
+    key: str,
+    default: list[str],
+) -> list[str]:
+    env_value = os.getenv(env_name)
+    value = (
+        env_value
+        if env_value and env_value.strip()
+        else _yaml_value(config, section, key, default)
+    )
+    if isinstance(value, str):
+        return [item.strip() for item in value.split(",") if item.strip()]
+    if isinstance(value, list):
+        return [str(item).strip() for item in value if str(item).strip()]
+    raise ConfigurationError(
+        f"Config value {section}.{key} must be a list or comma-separated string"
+    )
+
+
+def _path_setting(
+    env_name: str,
+    config: dict[str, Any],
+    section: str,
+    key: str,
+    default: str,
+    config_dir: Path,
+) -> str:
+    env_value = os.getenv(env_name)
+    if env_value is not None:
+        return env_value
+    value = Path(str(_yaml_value(config, section, key, default))).expanduser()
+    if not value.is_absolute():
+        value = config_dir / value
+    return str(value.resolve())
+
+
+def _build_database_url(base_url: str, username: str, password: str) -> str:
+    try:
+        parsed = urlsplit(base_url)
+        host = parsed.hostname
+        port = parsed.port
+    except ValueError as exc:
+        raise ConfigurationError("Config value database.url is not a valid URL") from exc
+
+    if not parsed.scheme or not host:
+        raise ConfigurationError(
+            "Config value database.url must include a scheme and host"
+        )
+    if parsed.scheme not in {"postgresql", "postgresql+psycopg"}:
+        raise ConfigurationError(
+            "Config value database.url must use postgresql or postgresql+psycopg"
+        )
+    if parsed.path in {"", "/"}:
+        raise ConfigurationError("Config value database.url must include a database name")
+    if not username:
+        raise ConfigurationError("Config value database.username cannot be empty")
+    if not password:
+        raise ConfigurationError("Config value database.password cannot be empty")
+
+    formatted_host = f"[{host}]" if ":" in host and not host.startswith("[") else host
+    host_and_port = f"{formatted_host}:{port}" if port is not None else formatted_host
+    credentials = f"{quote(username, safe='')}:{quote(password, safe='')}"
+    parts = (
+        parsed.scheme,
+        f"{credentials}@{host_and_port}",
+        parsed.path,
+        parsed.query,
+        parsed.fragment,
+    )
+    return urlunsplit(parts)
+
+
+def _database_url(config: dict[str, Any]) -> str:
+    # 保留原有完整连接串环境变量，便于 Docker 和生产环境注入密钥。
+    complete_url_override = os.getenv("DATABASE_URL")
+    if complete_url_override is not None:
+        return complete_url_override
+
+    base_url = _string_setting(
+        "DATABASE_BASE_URL",
+        config,
+        "database",
+        "url",
+        DEFAULT_DATABASE_BASE_URL,
+    )
+    username = _string_setting(
+        "DATABASE_USERNAME",
+        config,
+        "database",
+        "username",
+        DEFAULT_DATABASE_USERNAME,
+    )
+    password = _string_setting(
+        "DATABASE_PASSWORD",
+        config,
+        "database",
+        "password",
+        DEFAULT_DATABASE_PASSWORD,
+    )
+    return _build_database_url(base_url, username, password)
 
 
 @dataclass(frozen=True)
 class Settings:
-    app_name: str = os.getenv("APP_NAME", "YG Fine-Tune Platform API")
-    app_env: str = os.getenv("APP_ENV", "local")
-    route_prefix: str = os.getenv("MODELTF_ROUTE_PREFIX", "/modelTF")
-    app_mode: str = os.getenv("APP_MODE", "local")
-    database_url: str = os.getenv("DATABASE_URL", "postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft")
-    cors_allow_origins: list[str] = None  # type: ignore[assignment]
-    compute_mode: str = os.getenv("COMPUTE_MODE", "real")
-    compute_status_sync_mode: str = os.getenv("COMPUTE_STATUS_SYNC_MODE", "polling")
-    compute_poll_interval_seconds: int = _int_env("COMPUTE_POLL_INTERVAL_SECONDS", 3)
-    compute_request_timeout_seconds: int = _int_env("COMPUTE_REQUEST_TIMEOUT_SECONDS", 5)
-    compute_service_token: str = os.getenv("COMPUTE_SERVICE_TOKEN", "")
-    log_level: str = os.getenv("LOG_LEVEL", "INFO")
-    log_dir: str = os.getenv("LOG_DIR", "./logs")
-    log_file_prefix: str = os.getenv("LOG_FILE_PREFIX", "backend")
-    log_error_file_prefix: str = os.getenv("LOG_ERROR_FILE_PREFIX", "error")
-    log_max_bytes: int = _int_env("LOG_MAX_BYTES", 20 * 1024 * 1024)
-    log_retention_days: int = _int_env("LOG_RETENTION_DAYS", 10)
-
-    def __post_init__(self) -> None:
-        object.__setattr__(
-            self,
-            "cors_allow_origins",
-            _list_env(
-                "CORS_ALLOW_ORIGINS",
-                [
+    app_name: str = "YG Fine-Tune Platform API"
+    app_env: str = "local"
+    route_prefix: str = "/modelTF"
+    app_mode: str = "local"
+    frontend_port: int = 16801
+    backend_port: int = 17861
+    database_url: str = field(
+        default="postgresql+psycopg://yg_ft:change_me@localhost:5432/yg_ft",
+        repr=False,
+    )
+    cors_allow_origins: list[str] = field(
+        default_factory=lambda: [
             "http://localhost:16801",
             "http://127.0.0.1:16801",
-                    "http://localhost:17861",
-                    "http://127.0.0.1:17861",
-                ],
+        ]
+    )
+    compute_mode: str = "real"
+    compute_status_sync_mode: str = "polling"
+    compute_poll_interval_seconds: int = 3
+    log_level: str = "INFO"
+    log_dir: str = "./logs"
+    log_file_prefix: str = "backend"
+    log_error_file_prefix: str = "error"
+    log_max_bytes: int = 20 * 1024 * 1024
+    log_retention_days: int = 10
+
+
+def load_settings(config_path: str | Path | None = None) -> Settings:
+    config, resolved_config_path = _load_yaml(config_path)
+    frontend_port = _port_setting(
+        "FRONTEND_PORT", config, "frontend_port", Settings.frontend_port
+    )
+    backend_port = _port_setting(
+        "BACKEND_PORT", config, "backend_port", Settings.backend_port
+    )
+    if frontend_port == backend_port:
+        raise ConfigurationError(
+            "Config values server.frontend_port and server.backend_port must be different"
+        )
+    default_cors = [
+        f"http://localhost:{frontend_port}",
+        f"http://127.0.0.1:{frontend_port}",
+    ]
+    return Settings(
+        app_name=_string_setting("APP_NAME", config, "app", "name", Settings.app_name),
+        app_env=_string_setting("APP_ENV", config, "app", "env", Settings.app_env),
+        route_prefix=_string_setting(
+            "MODELTF_ROUTE_PREFIX", config, "app", "route_prefix", Settings.route_prefix
+        ),
+        app_mode=_string_setting("APP_MODE", config, "app", "mode", Settings.app_mode),
+        frontend_port=frontend_port,
+        backend_port=backend_port,
+        database_url=_database_url(config),
+        cors_allow_origins=_list_setting(
+            "CORS_ALLOW_ORIGINS", config, "app", "cors_allow_origins", default_cors
+        ),
+        compute_mode=_string_setting(
+            "COMPUTE_MODE", config, "compute", "mode", Settings.compute_mode
+        ),
+        compute_status_sync_mode=_string_setting(
+            "COMPUTE_STATUS_SYNC_MODE",
+            config,
+            "compute",
+            "status_sync_mode",
+            Settings.compute_status_sync_mode,
+        ),
+        compute_poll_interval_seconds=_int_setting(
+            "COMPUTE_POLL_INTERVAL_SECONDS",
+            config,
+            "compute",
+            "poll_interval_seconds",
+            Settings.compute_poll_interval_seconds,
+        ),
+        log_level=_string_setting(
+            "LOG_LEVEL", config, "logging", "level", Settings.log_level
+        ),
+        log_dir=_path_setting(
+            "LOG_DIR",
+            config,
+            "logging",
+            "directory",
+            Settings.log_dir,
+            resolved_config_path.parent,
+        ),
+        log_file_prefix=_string_setting(
+            "LOG_FILE_PREFIX", config, "logging", "file_prefix", Settings.log_file_prefix
+        ),
+        log_error_file_prefix=_string_setting(
+            "LOG_ERROR_FILE_PREFIX",
+            config,
+            "logging",
+            "error_file_prefix",
+            Settings.log_error_file_prefix,
+        ),
+        log_max_bytes=_int_setting(
+            "LOG_MAX_BYTES", config, "logging", "max_bytes", Settings.log_max_bytes
+        ),
+        log_retention_days=_int_setting(
+            "LOG_RETENTION_DAYS",
+            config,
+            "logging",
+            "retention_days",
+            Settings.log_retention_days,
         ),
     )
 
 
 @lru_cache
 def get_settings() -> Settings:
-    return Settings()
-
+    return load_settings()

backend/app/db/session.py

@@ -1,14 +1,15 @@
 from __future__ import annotations
 
-import os
 from collections.abc import Generator
 from contextlib import contextmanager
 
 from sqlalchemy import create_engine
 from sqlalchemy.orm import Session, sessionmaker
 
+from app.core.config import get_settings
 
-DATABASE_URL = os.getenv("DATABASE_URL", "postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft")
+
+DATABASE_URL = get_settings().database_url
 
 engine = create_engine(
     DATABASE_URL,
@@ -37,4 +38,3 @@ def session_scope() -> Generator[Session, None, None]:
         raise
     finally:
         db.close()
-

backend/app/db/sql/001_platform_runtime.sql

@@ -36,45 +36,6 @@ CREATE TABLE IF NOT EXISTS trained_models (
     merged_path TEXT
 );
 
-CREATE TABLE IF NOT EXISTS model_lineage (
-    id TEXT PRIMARY KEY,
-    child_resource_type TEXT NOT NULL,
-    child_resource_id TEXT NOT NULL,
-    parent_resource_type TEXT NOT NULL,
-    parent_resource_id TEXT NOT NULL,
-    relation_type TEXT NOT NULL,
-    compute_job_id TEXT,
-    payload TEXT NOT NULL,
-    create_time TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS model_artifacts (
-    id TEXT PRIMARY KEY,
-    model_id TEXT NOT NULL,
-    model_kind TEXT NOT NULL,
-    artifact_type TEXT NOT NULL,
-    path TEXT NOT NULL,
-    size_bytes BIGINT NOT NULL DEFAULT 0,
-    checksum_sha256 TEXT,
-    metadata TEXT NOT NULL,
-    compute_job_id TEXT,
-    create_time TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS model_export_jobs (
-    id TEXT PRIMARY KEY,
-    trained_model_id TEXT,
-    compute_job_id TEXT NOT NULL,
-    node_id TEXT,
-    export_type TEXT NOT NULL,
-    quantization_bit INTEGER NOT NULL DEFAULT 0,
-    status TEXT NOT NULL,
-    output_dir TEXT,
-    payload TEXT NOT NULL,
-    create_time TEXT NOT NULL,
-    completed_at TEXT
-);
-
 CREATE TABLE IF NOT EXISTS datasets (
     id TEXT PRIMARY KEY,
     name TEXT NOT NULL UNIQUE,
@@ -115,9 +76,6 @@ CREATE TABLE IF NOT EXISTS compute_nodes (
     data_root TEXT NOT NULL,
     model_root TEXT NOT NULL,
     log_root TEXT NOT NULL,
-    api_version TEXT NOT NULL DEFAULT 'v1',
-    capabilities TEXT NOT NULL DEFAULT '[]',
-    description TEXT,
     last_health_check_at TEXT,
     health_detail TEXT NOT NULL
 );
@@ -130,8 +88,7 @@ CREATE TABLE IF NOT EXISTS gpus (
     name TEXT NOT NULL,
     memory_total_gb DOUBLE PRECISION NOT NULL,
     power_limit_w DOUBLE PRECISION NOT NULL,
-    base_temperature INTEGER NOT NULL,
-    last_seen_at TEXT
+    base_temperature INTEGER NOT NULL
 );
 
 CREATE TABLE IF NOT EXISTS fine_tune_tasks (
@@ -146,64 +103,7 @@ CREATE TABLE IF NOT EXISTS fine_tune_tasks (
     completed_at TEXT,
     compute_node_id TEXT REFERENCES compute_nodes(id) ON DELETE SET NULL,
     gpus TEXT NOT NULL,
-    sync_job_id TEXT,
-    compute_job_id TEXT
-);
-
-CREATE TABLE IF NOT EXISTS fine_tune_metrics (
-    id TEXT PRIMARY KEY,
-    task_id TEXT NOT NULL REFERENCES fine_tune_tasks(id) ON DELETE CASCADE,
-    step INTEGER NOT NULL,
-    epoch DOUBLE PRECISION,
-    loss DOUBLE PRECISION,
-    grad_norm DOUBLE PRECISION,
-    learning_rate DOUBLE PRECISION,
-    raw TEXT NOT NULL,
-    create_time TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS fine_tune_checkpoints (
-    id TEXT PRIMARY KEY,
-    task_id TEXT NOT NULL REFERENCES fine_tune_tasks(id) ON DELETE CASCADE,
-    step INTEGER NOT NULL,
-    name TEXT NOT NULL,
-    path TEXT NOT NULL,
-    size_bytes BIGINT NOT NULL DEFAULT 0,
-    create_time TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS compute_jobs (
-    id TEXT PRIMARY KEY,
-    task_id TEXT REFERENCES fine_tune_tasks(id) ON DELETE SET NULL,
-    node_id TEXT REFERENCES compute_nodes(id) ON DELETE SET NULL,
-    engine TEXT NOT NULL,
-    status TEXT NOT NULL,
-    command TEXT NOT NULL,
-    output_dir TEXT,
-    log_file TEXT,
-    payload TEXT NOT NULL,
-    create_time TEXT NOT NULL,
-    update_time TEXT NOT NULL,
-    completed_at TEXT
-);
-
-CREATE TABLE IF NOT EXISTS gpu_allocations (
-    id TEXT PRIMARY KEY,
-    task_id TEXT REFERENCES fine_tune_tasks(id) ON DELETE CASCADE,
-    compute_job_id TEXT,
-    node_id TEXT REFERENCES compute_nodes(id) ON DELETE CASCADE,
-    gpu_index INTEGER NOT NULL,
-    status TEXT NOT NULL,
-    create_time TEXT NOT NULL,
-    released_at TEXT
-);
-
-CREATE TABLE IF NOT EXISTS scheduler_locks (
-    lock_key TEXT PRIMARY KEY,
-    owner TEXT NOT NULL,
-    expires_at TEXT NOT NULL,
-    create_time TEXT NOT NULL,
-    update_time TEXT NOT NULL
+    sync_job_id TEXT
 );
 
 CREATE TABLE IF NOT EXISTS resource_replicas (
@@ -214,10 +114,6 @@ CREATE TABLE IF NOT EXISTS resource_replicas (
     local_path TEXT NOT NULL,
     status TEXT NOT NULL,
     sync_status TEXT NOT NULL,
-    checksum_sha256 TEXT,
-    byte_size BIGINT NOT NULL DEFAULT 0,
-    last_checked_at TEXT,
-    last_error TEXT,
     create_time TEXT NOT NULL
 );
 
@@ -231,54 +127,7 @@ CREATE TABLE IF NOT EXISTS resource_sync_jobs (
     completed_at TEXT
 );
 
-CREATE TABLE IF NOT EXISTS eval_tasks (
-    id TEXT PRIMARY KEY,
-    name TEXT NOT NULL,
-    payload TEXT NOT NULL,
-    status TEXT NOT NULL,
-    create_time TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS eval_dimensions (
-    id TEXT PRIMARY KEY,
-    name TEXT NOT NULL,
-    payload TEXT NOT NULL,
-    is_active INTEGER NOT NULL DEFAULT 1,
-    is_default INTEGER NOT NULL DEFAULT 0,
-    create_time TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS compare_tasks (
-    id TEXT PRIMARY KEY,
-    name TEXT NOT NULL,
-    payload TEXT NOT NULL,
-    status TEXT NOT NULL,
-    create_time TEXT NOT NULL
-);
-
 CREATE INDEX IF NOT EXISTS idx_fine_tune_status ON fine_tune_tasks(status);
-CREATE INDEX IF NOT EXISTS idx_fine_tune_compute_job ON fine_tune_tasks(compute_job_id);
-CREATE INDEX IF NOT EXISTS idx_fine_tune_compute_node_status ON fine_tune_tasks(compute_node_id, status);
-CREATE INDEX IF NOT EXISTS idx_model_lineage_child ON model_lineage(child_resource_type, child_resource_id);
-CREATE INDEX IF NOT EXISTS idx_model_lineage_parent ON model_lineage(parent_resource_type, parent_resource_id);
-CREATE INDEX IF NOT EXISTS idx_model_artifacts_model ON model_artifacts(model_kind, model_id, artifact_type);
-CREATE INDEX IF NOT EXISTS idx_model_export_jobs_model ON model_export_jobs(trained_model_id, create_time DESC);
-CREATE INDEX IF NOT EXISTS idx_model_export_jobs_compute ON model_export_jobs(compute_job_id);
-CREATE INDEX IF NOT EXISTS idx_fine_tune_metrics_task_step ON fine_tune_metrics(task_id, step);
-CREATE UNIQUE INDEX IF NOT EXISTS uq_fine_tune_metrics_task_step_epoch ON fine_tune_metrics(task_id, step, epoch);
-CREATE INDEX IF NOT EXISTS idx_fine_tune_checkpoints_task_step ON fine_tune_checkpoints(task_id, step);
-CREATE UNIQUE INDEX IF NOT EXISTS uq_fine_tune_checkpoints_task_path ON fine_tune_checkpoints(task_id, path);
-CREATE INDEX IF NOT EXISTS idx_compute_jobs_task ON compute_jobs(task_id);
-CREATE INDEX IF NOT EXISTS idx_compute_jobs_node_status ON compute_jobs(node_id, status);
-CREATE INDEX IF NOT EXISTS idx_gpu_allocations_node_status ON gpu_allocations(node_id, status);
-CREATE UNIQUE INDEX IF NOT EXISTS uq_gpu_allocations_active ON gpu_allocations(node_id, gpu_index) WHERE status IN ('allocated','running');
-CREATE INDEX IF NOT EXISTS idx_scheduler_locks_expires ON scheduler_locks(expires_at);
 CREATE INDEX IF NOT EXISTS idx_dataset_files_dataset ON dataset_files(dataset_id);
 CREATE INDEX IF NOT EXISTS idx_gpus_node ON gpus(node_id);
-CREATE UNIQUE INDEX IF NOT EXISTS uq_gpus_node_index ON gpus(node_id, gpu_index);
 CREATE INDEX IF NOT EXISTS idx_replicas_resource ON resource_replicas(resource_type, resource_id);
-CREATE UNIQUE INDEX IF NOT EXISTS uq_replicas_node_resource ON resource_replicas(node_id, resource_type, resource_id);
-CREATE INDEX IF NOT EXISTS idx_sync_jobs_node_status ON resource_sync_jobs(target_node_id, status);
-CREATE INDEX IF NOT EXISTS idx_eval_tasks_status ON eval_tasks(status);
-CREATE INDEX IF NOT EXISTS idx_eval_dimensions_active ON eval_dimensions(is_active);
-CREATE INDEX IF NOT EXISTS idx_compare_tasks_status ON compare_tasks(status);

backend/app/db/sql/002_data_process.sql

@@ -1,235 +0,0 @@
--- Data processing migration.
---
--- IMPORTANT: This file is intentionally NOT wired into application startup.
--- Apply it explicitly in a controlled deployment, or call
--- DataProcessStore.ensure_schema() from an administrative command.
-
-BEGIN;
-
--- This migration targets the current runtime schema created by
--- 001_platform_runtime.sql. Refuse the UUID/JSONB target-design schema instead
--- of partially altering it with incompatible TEXT foreign keys.
-DO $$
-DECLARE
-    datasets_id_type TEXT;
-BEGIN
-    SELECT format_type(a.atttypid, a.atttypmod)
-      INTO datasets_id_type
-      FROM pg_attribute a
-      JOIN pg_class c ON c.oid = a.attrelid
-      JOIN pg_namespace n ON n.oid = c.relnamespace
-     WHERE n.nspname = current_schema()
-       AND c.relname = 'datasets'
-       AND a.attname = 'id'
-       AND a.attnum > 0
-       AND NOT a.attisdropped;
-    IF datasets_id_type IS NULL THEN
-        RAISE EXCEPTION '002_data_process.sql requires 001_platform_runtime.sql first';
-    END IF;
-    IF datasets_id_type <> 'text' THEN
-        RAISE EXCEPTION
-          '002_data_process.sql supports only the current TEXT runtime schema; found datasets.id type %',
-          datasets_id_type;
-    END IF;
-END $$;
-
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS source_task_id TEXT;
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS size_bytes BIGINT NOT NULL DEFAULT 0;
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS record_count BIGINT NOT NULL DEFAULT 0;
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS metadata TEXT NOT NULL DEFAULT '{}';
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS tenant_id TEXT;
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS project_id TEXT;
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS owner_id TEXT;
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS created_by TEXT;
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS created_at TIMESTAMPTZ NOT NULL DEFAULT now();
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS updated_at TIMESTAMPTZ NOT NULL DEFAULT now();
-ALTER TABLE datasets ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ;
-
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS storage_object_id TEXT;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS current_version_id TEXT;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS size_bytes BIGINT NOT NULL DEFAULT 0;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS record_count BIGINT NOT NULL DEFAULT 0;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS file_format VARCHAR(40);
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS checksum_sha256 CHAR(64);
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS version_no INTEGER NOT NULL DEFAULT 1;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS source_task_id TEXT;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS tenant_id TEXT;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS project_id TEXT;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS created_by TEXT;
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS metadata TEXT NOT NULL DEFAULT '{}';
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS created_at TIMESTAMPTZ NOT NULL DEFAULT now();
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS updated_at TIMESTAMPTZ NOT NULL DEFAULT now();
-ALTER TABLE dataset_files ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ;
-
-CREATE TABLE IF NOT EXISTS data_process_tasks (
-    id TEXT PRIMARY KEY,
-    name VARCHAR(150) NOT NULL,
-    description TEXT,
-    status VARCHAR(20) NOT NULL DEFAULT 'pending'
-        CHECK (status IN ('pending', 'running', 'completed', 'failed', 'stopped')),
-    process_type VARCHAR(20) NOT NULL
-        CHECK (process_type IN ('structured', 'unstructured', 'external')),
-    source_dataset_id TEXT REFERENCES datasets(id) ON DELETE SET NULL,
-    output_dataset_id TEXT REFERENCES datasets(id) ON DELETE SET NULL,
-    config TEXT NOT NULL DEFAULT '{}',
-    progress NUMERIC(5,2) NOT NULL DEFAULT 0 CHECK (progress >= 0 AND progress <= 100),
-    input_count BIGINT NOT NULL DEFAULT 0 CHECK (input_count >= 0),
-    output_count BIGINT NOT NULL DEFAULT 0 CHECK (output_count >= 0),
-    filtered_count BIGINT NOT NULL DEFAULT 0 CHECK (filtered_count >= 0),
-    duplicate_count BIGINT NOT NULL DEFAULT 0 CHECK (duplicate_count >= 0),
-    error_count BIGINT NOT NULL DEFAULT 0 CHECK (error_count >= 0),
-    failure_reason TEXT,
-    generation_run_id TEXT,
-    tenant_id TEXT,
-    project_id TEXT,
-    owner_id TEXT,
-    approval_status VARCHAR(30) NOT NULL DEFAULT 'not_required',
-    created_by TEXT,
-    updated_by TEXT,
-    deleted_by TEXT,
-    started_at TIMESTAMPTZ,
-    completed_at TIMESTAMPTZ,
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    deleted_at TIMESTAMPTZ
-);
-
-ALTER TABLE data_process_tasks ADD COLUMN IF NOT EXISTS generation_run_id TEXT;
-
-CREATE UNIQUE INDEX IF NOT EXISTS uq_data_process_tasks_name_alive
-    ON data_process_tasks(name) WHERE deleted_at IS NULL;
-CREATE INDEX IF NOT EXISTS idx_data_process_tasks_scope_status
-    ON data_process_tasks(tenant_id, project_id, status, created_at DESC)
-    WHERE deleted_at IS NULL;
-CREATE INDEX IF NOT EXISTS idx_data_process_tasks_creator_created
-    ON data_process_tasks(created_by, created_at DESC) WHERE deleted_at IS NULL;
-
-CREATE TABLE IF NOT EXISTS data_process_source_files (
-    id TEXT PRIMARY KEY,
-    task_id TEXT NOT NULL REFERENCES data_process_tasks(id) ON DELETE CASCADE,
-    storage_object_id TEXT,
-    name TEXT NOT NULL,
-    size_bytes BIGINT NOT NULL DEFAULT 0 CHECK (size_bytes >= 0),
-    record_count BIGINT NOT NULL DEFAULT 0 CHECK (record_count >= 0),
-    file_format VARCHAR(40),
-    checksum_sha256 CHAR(64) NOT NULL,
-    version_no INTEGER NOT NULL DEFAULT 1 CHECK (version_no > 0),
-    content TEXT NOT NULL,
-    content_preview TEXT,
-    metadata TEXT NOT NULL DEFAULT '{}',
-    tenant_id TEXT,
-    project_id TEXT,
-    created_by TEXT,
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    deleted_at TIMESTAMPTZ
-);
-
-CREATE INDEX IF NOT EXISTS idx_data_process_source_files_task
-    ON data_process_source_files(task_id, created_at) WHERE deleted_at IS NULL;
-CREATE UNIQUE INDEX IF NOT EXISTS uq_data_process_source_checksum_alive
-    ON data_process_source_files(task_id, checksum_sha256) WHERE deleted_at IS NULL;
-
-CREATE TABLE IF NOT EXISTS data_process_preview_items (
-    id TEXT PRIMARY KEY,
-    task_id TEXT NOT NULL REFERENCES data_process_tasks(id) ON DELETE CASCADE,
-    source_file_id TEXT REFERENCES data_process_source_files(id) ON DELETE CASCADE,
-    original_content TEXT NOT NULL DEFAULT '',
-    edited_content TEXT NOT NULL DEFAULT '',
-    source_start INTEGER CHECK (source_start IS NULL OR source_start >= 0),
-    source_end INTEGER CHECK (source_end IS NULL OR source_end >= 0),
-    source_start_line INTEGER CHECK (source_start_line IS NULL OR source_start_line > 0),
-    source_end_line INTEGER CHECK (source_end_line IS NULL OR source_end_line > 0),
-    token_count INTEGER NOT NULL DEFAULT 0 CHECK (token_count >= 0),
-    status VARCHAR(20) NOT NULL DEFAULT 'original'
-        CHECK (status IN ('original', 'modified', 'manual', 'invalid')),
-    quality_score TEXT NOT NULL DEFAULT '{}',
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    CHECK (source_start IS NULL OR source_end IS NULL OR source_end >= source_start),
-    CHECK (source_start_line IS NULL OR source_end_line IS NULL OR source_end_line >= source_start_line)
-);
-
-CREATE INDEX IF NOT EXISTS idx_data_process_preview_task_file
-    ON data_process_preview_items(task_id, source_file_id, created_at);
-
-CREATE TABLE IF NOT EXISTS data_process_results (
-    id TEXT PRIMARY KEY,
-    task_id TEXT NOT NULL REFERENCES data_process_tasks(id) ON DELETE CASCADE,
-    preview_item_id TEXT REFERENCES data_process_preview_items(id) ON DELETE SET NULL,
-    instruction TEXT NOT NULL,
-    input TEXT NOT NULL DEFAULT '',
-    output TEXT NOT NULL,
-    original_instruction TEXT,
-    original_input TEXT,
-    original_output TEXT,
-    status VARCHAR(20) NOT NULL DEFAULT 'valid'
-        CHECK (status IN ('valid', 'modified', 'invalid')),
-    error TEXT,
-    split VARCHAR(20) CHECK (split IS NULL OR split IN ('train', 'validation', 'test')),
-    quality_score TEXT NOT NULL DEFAULT '{}',
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
-);
-
-CREATE INDEX IF NOT EXISTS idx_data_process_results_task_status
-    ON data_process_results(task_id, status, id);
-CREATE INDEX IF NOT EXISTS idx_data_process_results_task_split
-    ON data_process_results(task_id, split);
-
-CREATE TABLE IF NOT EXISTS dataset_file_versions (
-    id TEXT PRIMARY KEY,
-    dataset_file_id TEXT NOT NULL REFERENCES dataset_files(id) ON DELETE CASCADE,
-    version_no INTEGER NOT NULL CHECK (version_no > 0),
-    storage_object_id TEXT NOT NULL,
-    content_preview TEXT,
-    description TEXT,
-    base_version_id TEXT REFERENCES dataset_file_versions(id) ON DELETE SET NULL,
-    size_bytes BIGINT NOT NULL DEFAULT 0 CHECK (size_bytes >= 0),
-    record_count BIGINT NOT NULL DEFAULT 0 CHECK (record_count >= 0),
-    checksum_sha256 CHAR(64) NOT NULL,
-    source_task_id TEXT REFERENCES data_process_tasks(id) ON DELETE SET NULL,
-    metadata TEXT NOT NULL DEFAULT '{}',
-    created_by TEXT,
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
-);
-
-ALTER TABLE dataset_file_versions ADD COLUMN IF NOT EXISTS source_task_id TEXT;
-ALTER TABLE dataset_file_versions ADD COLUMN IF NOT EXISTS metadata TEXT NOT NULL DEFAULT '{}';
-CREATE UNIQUE INDEX IF NOT EXISTS uq_dataset_file_versions_no_002
-    ON dataset_file_versions(dataset_file_id, version_no);
-CREATE INDEX IF NOT EXISTS idx_dataset_file_versions_source_task_002
-    ON dataset_file_versions(source_task_id) WHERE source_task_id IS NOT NULL;
-
-CREATE TABLE IF NOT EXISTS dataset_records (
-    id TEXT PRIMARY KEY,
-    dataset_id TEXT NOT NULL REFERENCES datasets(id) ON DELETE CASCADE,
-    dataset_file_id TEXT REFERENCES dataset_files(id) ON DELETE CASCADE,
-    version_id TEXT REFERENCES dataset_file_versions(id) ON DELETE CASCADE,
-    line_no INTEGER,
-    split VARCHAR(20) CHECK (split IS NULL OR split IN ('train', 'validation', 'test')),
-    instruction TEXT,
-    input TEXT,
-    output TEXT,
-    raw TEXT NOT NULL DEFAULT '{}',
-    status VARCHAR(20) NOT NULL DEFAULT 'valid'
-        CHECK (status IN ('valid', 'modified', 'invalid')),
-    source_task_id TEXT REFERENCES data_process_tasks(id) ON DELETE SET NULL,
-    source_result_id TEXT REFERENCES data_process_results(id) ON DELETE SET NULL,
-    preview_item_id TEXT REFERENCES data_process_preview_items(id) ON DELETE SET NULL,
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
-);
-
-ALTER TABLE dataset_records ADD COLUMN IF NOT EXISTS source_task_id TEXT;
-ALTER TABLE dataset_records ADD COLUMN IF NOT EXISTS source_result_id TEXT;
-ALTER TABLE dataset_records ADD COLUMN IF NOT EXISTS preview_item_id TEXT;
-CREATE INDEX IF NOT EXISTS idx_dataset_records_dataset_002
-    ON dataset_records(dataset_id, id);
-CREATE INDEX IF NOT EXISTS idx_dataset_records_source_task_002
-    ON dataset_records(source_task_id, source_result_id);
-CREATE INDEX IF NOT EXISTS idx_datasets_source_task_002
-    ON datasets(source_task_id) WHERE source_task_id IS NOT NULL;
-CREATE INDEX IF NOT EXISTS idx_dataset_files_source_task_002
-    ON dataset_files(source_task_id) WHERE source_task_id IS NOT NULL;
-
-COMMIT;

backend/app/main.py

@@ -1,13 +1,9 @@
-import asyncio
-from contextlib import suppress
-
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
 
 from app.api.v1.router import api_router
 from app.core.config import get_settings
 from app.core.logging import configure_logging, setup_request_logging
-from app.workers.compute_poller import run_compute_poller
 
 
 def create_app() -> FastAPI:
@@ -24,19 +20,6 @@ def create_app() -> FastAPI:
     )
     setup_request_logging(app)
     app.include_router(api_router, prefix=settings.route_prefix)
-
-    @app.on_event("startup")
-    async def start_workers() -> None:
-        app.state.compute_poller_task = asyncio.create_task(run_compute_poller())
-
-    @app.on_event("shutdown")
-    async def stop_workers() -> None:
-        task = getattr(app.state, "compute_poller_task", None)
-        if task:
-            task.cancel()
-            with suppress(asyncio.CancelledError):
-                await task
-
     return app
 
 

backend/app/modules/compute_gateway/client.py

@@ -1,206 +0,0 @@
-from __future__ import annotations
-
-import time
-from typing import Any
-from urllib.parse import urljoin
-
-import httpx
-
-from app.core.config import get_settings
-
-
-def _join_url(base_url: str, path: str) -> str:
-    return urljoin(base_url.rstrip("/") + "/", path.lstrip("/"))
-
-
-def _unwrap_items(payload: Any) -> list[dict[str, Any]]:
-    if isinstance(payload, list):
-        return [item for item in payload if isinstance(item, dict)]
-    if isinstance(payload, dict):
-        data = payload.get("data")
-        if isinstance(data, dict) and isinstance(data.get("items"), list):
-            return [item for item in data["items"] if isinstance(item, dict)]
-        if isinstance(payload.get("items"), list):
-            return [item for item in payload["items"] if isinstance(item, dict)]
-        if isinstance(data, list):
-            return [item for item in data if isinstance(item, dict)]
-    return []
-
-
-def _unwrap_dict(payload: Any) -> dict[str, Any]:
-    if isinstance(payload, dict) and isinstance(payload.get("data"), dict):
-        return payload["data"]
-    return payload if isinstance(payload, dict) else {}
-
-
-class ComputeNodeClient:
-    """Application-side client for one compute node.
-
-    The client accepts both current YG Compute API responses and common
-    wrapper shapes such as `{code,message,data}` to make future engine/node
-    adapters less brittle.
-    """
-
-    def __init__(self, api_base_url: str, token: str | None = None, timeout: float | None = None) -> None:
-        settings = get_settings()
-        self.api_base_url = api_base_url.rstrip("/")
-        self.token = token or settings.compute_service_token
-        self.timeout = timeout or settings.compute_request_timeout_seconds
-        self.route_prefix = settings.route_prefix.rstrip("/") or "/modelTF"
-
-    def headers(self) -> dict[str, str]:
-        if not self.token:
-            return {}
-        return {"X-Compute-Token": self.token}
-
-    async def test_connection(self) -> dict[str, Any]:
-        started = time.perf_counter()
-        health = await self.health()
-        gpus = await self.gpus()
-        return {
-            "success": True,
-            "latency_ms": int((time.perf_counter() - started) * 1000),
-            "health": health,
-            "gpus": gpus,
-        }
-
-    async def health(self) -> dict[str, Any]:
-        paths = [f"{self.route_prefix}/v1/compute/health", f"{self.route_prefix}/health", "/health"]
-        last_error = ""
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            for path in paths:
-                try:
-                    response = await client.get(_join_url(self.api_base_url, path))
-                    response.raise_for_status()
-                    return _unwrap_dict(response.json())
-                except Exception as exc:  # noqa: BLE001 - keep endpoint compatibility fallback broad
-                    last_error = str(exc)
-        raise RuntimeError(last_error or "compute health check failed")
-
-    async def gpus(self) -> list[dict[str, Any]]:
-        paths = [
-            f"{self.route_prefix}/compute/resources/gpus",
-            f"{self.route_prefix}/v1/compute/resources/gpus",
-            "/compute/resources/gpus",
-        ]
-        last_error = ""
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            for path in paths:
-                try:
-                    response = await client.get(_join_url(self.api_base_url, path))
-                    response.raise_for_status()
-                    return _unwrap_items(response.json())
-                except Exception as exc:  # noqa: BLE001
-                    last_error = str(exc)
-        raise RuntimeError(last_error or "compute gpu discovery failed")
-
-    async def create_job(self, payload: dict[str, Any]) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.post(_join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs"), json=payload)
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def preview_job(self, payload: dict[str, Any]) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.post(
-                _join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/preview"),
-                json=payload,
-            )
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def validate_job(self, payload: dict[str, Any]) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.post(
-                _join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/validate"),
-                json=payload,
-            )
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def check_paths(self, paths: list[dict[str, Any]]) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.post(
-                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/check-paths"),
-                json={"paths": paths},
-            )
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def list_files(
-        self,
-        root: str = "data",
-        relative_path: str = "",
-        directories_only: bool = False,
-    ) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.get(
-                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/list"),
-                params={"root": root, "relative_path": relative_path, "directories_only": directories_only},
-            )
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def get_job(self, job_id: str) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.get(_join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/{job_id}"))
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def stop_job(self, job_id: str) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.post(_join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/{job_id}/stop"))
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def job_logs(
-        self,
-        job_id: str,
-        tail_lines: int | None = None,
-        offset: int | None = None,
-        limit: int | None = None,
-    ) -> dict[str, Any]:
-        params = {
-            key: value
-            for key, value in {"tail_lines": tail_lines, "offset": offset, "limit": limit}.items()
-            if value is not None
-        }
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.get(
-                _join_url(self.api_base_url, f"{self.route_prefix}/compute/jobs/{job_id}/logs"),
-                params=params,
-            )
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def import_local_file(self, payload: dict[str, Any]) -> dict[str, Any]:
-        async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers()) as client:
-            response = await client.post(
-                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/import-local"),
-                json=payload,
-            )
-            response.raise_for_status()
-            return _unwrap_dict(response.json())
-
-    async def upload_file(
-        self,
-        filename: str,
-        content: bytes,
-        target_relative_path: str,
-        resource_type: str | None = None,
-        resource_id: str | None = None,
-    ) -> dict[str, Any]:
-        data = {
-            "target_relative_path": target_relative_path,
-            "resource_type": resource_type or "",
-            "resource_id": resource_id or "",
-        }
-        files = {"file": (filename, content)}
-        async with httpx.AsyncClient(timeout=max(self.timeout, 60), headers=self.headers()) as client:
-            response = await client.post(
-                _join_url(self.api_base_url, f"{self.route_prefix}/compute/files/upload"),
-                data=data,
-                files=files,
-            )
-            response.raise_for_status()
-            return _unwrap_dict(response.json())

backend/app/modules/compute_gateway/sync.py

@@ -1,44 +0,0 @@
-from __future__ import annotations
-
-from typing import Any
-
-from app.db.platform_store import get_platform_store
-from app.modules.compute_gateway.client import ComputeNodeClient
-
-
-def _node_for_task(task: dict[str, Any]) -> dict[str, Any] | None:
-    return next((node for node in get_platform_store().compute_nodes() if node["id"] == task.get("compute_node_id")), None)
-
-
-async def poll_compute_jobs_once() -> dict[str, Any]:
-    store = get_platform_store()
-    synced: list[dict[str, Any]] = []
-    failed: list[dict[str, str]] = []
-    for task in store.running_compute_tasks():
-        node = _node_for_task(task)
-        if not node:
-            failed.append({"task_id": task["id"], "error": "compute node not found"})
-            continue
-        try:
-            client = ComputeNodeClient(node["api_base_url"])
-            job = await client.get_job(task["compute_job_id"])
-            try:
-                logs = await client.job_logs(task["compute_job_id"], tail_lines=5000)
-                store.record_training_log_metrics(task["id"], str(logs.get("content") or ""))
-            except Exception:
-                pass
-            synced.append(store.apply_compute_job(task["id"], job))
-        except Exception as exc:  # noqa: BLE001 - keep polling other jobs
-            failed.append({"task_id": task["id"], "error": str(exc)})
-    standalone_synced: list[dict[str, Any]] = []
-    for record in store.active_standalone_compute_jobs():
-        node = next((item for item in store.compute_nodes() if item["id"] == record.get("node_id")), None)
-        if not node:
-            failed.append({"job_id": record["id"], "error": "compute node not found"})
-            continue
-        try:
-            job = await ComputeNodeClient(node["api_base_url"]).get_job(record["id"])
-            standalone_synced.append(store.sync_model_merge_job(record["id"], job))
-        except Exception as exc:  # noqa: BLE001 - keep polling other jobs
-            failed.append({"job_id": record["id"], "error": str(exc)})
-    return {"synced": len(synced) + len(standalone_synced), "failed": failed, "items": synced, "standalone": standalone_synced}

backend/app/modules/data_process/generation.py

@@ -1,250 +0,0 @@
-"""数据处理任务的大模型生成适配器。"""
-
-from __future__ import annotations
-
-import hashlib
-import json
-import re
-from collections.abc import Callable, Iterable, Mapping
-from typing import Any
-from urllib.parse import urlsplit, urlunsplit
-
-import httpx
-
-from app.modules.data_process.algorithms import normalize_text, stable_split
-
-
-class ModelGenerationError(ValueError):
-    """模型配置、响应或调用失败。"""
-
-
-def chat_completions_url(value: str) -> str:
-    """把域名、基础 URL 或完整地址统一为 chat completions 地址。"""
-
-    raw = (value or "").strip()
-    if not raw:
-        raise ModelGenerationError("generation model api_url is required")
-    if "://" not in raw:
-        raw = f"https://{raw}"
-    parsed = urlsplit(raw)
-    if parsed.scheme not in {"http", "https"} or not parsed.hostname:
-        raise ModelGenerationError("generation model api_url must be an HTTP(S) host or URL")
-    if parsed.username or parsed.password:
-        raise ModelGenerationError("generation model api_url must not contain credentials")
-
-    path = parsed.path.rstrip("/")
-    if path.endswith("/chat/completions"):
-        target_path = path
-    elif path.endswith("/v1"):
-        target_path = f"{path}/chat/completions"
-    elif not path:
-        target_path = "/v1/chat/completions"
-    else:
-        target_path = f"{path}/v1/chat/completions"
-    return urlunsplit((parsed.scheme, parsed.netloc, target_path, "", ""))
-
-
-def _message_content(payload: Mapping[str, Any]) -> str:
-    try:
-        content = payload["choices"][0]["message"]["content"]
-    except (KeyError, IndexError, TypeError) as exc:
-        raise ModelGenerationError("model response does not contain choices[0].message.content") from exc
-    if isinstance(content, str):
-        return content
-    if isinstance(content, list):
-        parts = [
-            str(item.get("text") or "")
-            for item in content
-            if isinstance(item, Mapping) and item.get("type") in {None, "text", "output_text"}
-        ]
-        if parts:
-            return "".join(parts)
-    raise ModelGenerationError("model response content must be text")
-
-
-def _json_payload(content: str) -> Any:
-    cleaned = re.sub(r"<think>[\s\S]*?</think>", "", content, flags=re.IGNORECASE).strip()
-    fenced = re.fullmatch(r"```(?:json)?\s*([\s\S]*?)\s*```", cleaned, flags=re.IGNORECASE)
-    if fenced:
-        cleaned = fenced.group(1).strip()
-    try:
-        return json.loads(cleaned)
-    except json.JSONDecodeError as exc:
-        raise ModelGenerationError(
-            f"model response is not valid JSON at line {exc.lineno}, column {exc.colno}"
-        ) from exc
-
-
-def _result_items(payload: Any) -> list[Mapping[str, Any]]:
-    if isinstance(payload, list):
-        values = payload
-    elif isinstance(payload, Mapping):
-        nested = next(
-            (
-                payload[key]
-                for key in ("items", "results", "data", "records")
-                if isinstance(payload.get(key), list)
-            ),
-            None,
-        )
-        values = nested if isinstance(nested, list) else [payload]
-    else:
-        raise ModelGenerationError("model JSON must be an object or array")
-    items = [item for item in values if isinstance(item, Mapping)]
-    if not items:
-        raise ModelGenerationError("model JSON does not contain result objects")
-    return items
-
-
-def _prompt_messages(prompt: str, content: str, count: int) -> list[dict[str, str]]:
-    schema_instruction = (
-        f"必须只返回 JSON 对象，格式为 {{\"items\":[{{\"instruction\":\"...\","
-        f"\"input\":\"...\",\"output\":\"...\"}}]}}；items 必须包含 {count} 条。"
-        "instruction 和 output 不得为空，不要输出 Markdown 代码围栏或分析过程。"
-    )
-    base_prompt = (
-        normalize_text(prompt)
-        or "请根据来源内容生成可用于监督微调的问答数据。"
-    )
-    if "{{ content }}" in base_prompt:
-        user_prompt = base_prompt.replace("{{ content }}", content)
-        return [
-            {"role": "system", "content": schema_instruction},
-            {"role": "user", "content": user_prompt},
-        ]
-    return [
-        {"role": "system", "content": f"{base_prompt}\n{schema_instruction}"},
-        {"role": "user", "content": f"来源内容：\n{content}"},
-    ]
-
-
-def generate_model_records(
-    preview_items: Iterable[Mapping[str, Any]],
-    *,
-    model: Mapping[str, Any],
-    config: Mapping[str, Any],
-    task_id: str,
-    split: Mapping[str, int],
-    qa_pairs_per_item: int,
-    client: httpx.Client | None = None,
-    on_progress: Callable[[int, int], None] | None = None,
-) -> list[dict[str, Any]]:
-    """调用 OpenAI 兼容接口，将预览切片生成标准训练记录。
-
-    单条调用失败会产生可人工修复的 invalid 结果，不会丢弃整批任务。
-    """
-
-    if not 1 <= qa_pairs_per_item <= 5:
-        raise ModelGenerationError("qa_pairs_per_item must be in [1, 5]")
-    endpoint = chat_completions_url(str(model.get("api_url") or ""))
-    model_name = str(model.get("online_model_name") or model.get("name") or "").strip()
-    if not model_name:
-        raise ModelGenerationError("generation model name is required")
-
-    temperature = float(config.get("temperature", 0.7))
-    max_tokens = int(config.get("max_tokens", 1024))
-    timeout = max(1.0, min(120.0, float(config.get("request_timeout_seconds", 60))))
-    retries = max(0, min(5, int(config.get("generation_retries", 2))))
-    headers = {"Content-Type": "application/json"}
-    api_key = str(model.get("api_key") or "").strip()
-    if api_key:
-        headers["Authorization"] = f"Bearer {api_key}"
-
-    owns_client = client is None
-    http_client = client or httpx.Client(timeout=timeout)
-    results: list[dict[str, Any]] = []
-    try:
-        preview_list = list(preview_items)
-        total_items = len(preview_list)
-        for item_index, item in enumerate(preview_list):
-            preview_id = str(item.get("id") or f"preview-{item_index + 1}")
-            content = normalize_text(
-                str(item.get("edited_content") or item.get("original_content") or "")
-            )
-            request_payload: dict[str, Any] = {
-                "model": model_name,
-                "messages": _prompt_messages(
-                    str(config.get("generation_prompt") or ""),
-                    content,
-                    qa_pairs_per_item,
-                ),
-                "temperature": temperature,
-                "max_tokens": max_tokens,
-            }
-            if bool(config.get("json_mode", False)):
-                request_payload["response_format"] = {"type": "json_object"}
-
-            last_error: Exception | None = None
-            generated_items: list[Mapping[str, Any]] | None = None
-            for _ in range(retries + 1):
-                try:
-                    response = http_client.post(endpoint, headers=headers, json=request_payload)
-                    response.raise_for_status()
-                    body = response.json()
-                    if not isinstance(body, Mapping):
-                        raise ModelGenerationError("model response body must be a JSON object")
-                    generated_items = _result_items(_json_payload(_message_content(body)))
-                    break
-                except (httpx.HTTPError, json.JSONDecodeError, ModelGenerationError) as exc:
-                    last_error = exc
-
-            if generated_items is None:
-                error_message = str(last_error or "model generation failed")[:2000]
-                result_id = f"result_{hashlib.sha256(f'{preview_id}:error'.encode()).hexdigest()[:16]}"
-                results.append(
-                    {
-                        "id": result_id,
-                        "preview_item_id": preview_id,
-                        "instruction": "模型生成失败，请人工补充",
-                        "input": content,
-                        "output": "",
-                        "original_instruction": "模型生成失败，请人工补充",
-                        "original_input": content,
-                        "original_output": "",
-                        "status": "invalid",
-                        "error": error_message,
-                        "split": stable_split(result_id, split, seed=task_id),
-                    }
-                )
-                if on_progress:
-                    on_progress(item_index + 1, total_items)
-                continue
-
-            for variant_index, value in enumerate(generated_items[:qa_pairs_per_item]):
-                instruction = normalize_text(str(value.get("instruction") or value.get("question") or ""))
-                input_text = normalize_text(str(value.get("input") or value.get("context") or ""))
-                output = normalize_text(
-                    str(
-                        value.get("output")
-                        or value.get("answer")
-                        or value.get("response")
-                        or ""
-                    )
-                )
-                raw_id = f"{preview_id}:{variant_index + 1}:{instruction}:{output}"
-                result_id = f"result_{hashlib.sha256(raw_id.encode()).hexdigest()[:16]}"
-                valid = bool(instruction and output)
-                results.append(
-                    {
-                        "id": result_id,
-                        "preview_item_id": preview_id,
-                        "instruction": instruction,
-                        "input": input_text,
-                        "output": output,
-                        "original_instruction": instruction,
-                        "original_input": input_text,
-                        "original_output": output,
-                        "status": "valid" if valid else "invalid",
-                        "error": None if valid else "model result is missing instruction or output",
-                        "split": stable_split(result_id, split, seed=task_id),
-                    }
-                )
-            if on_progress:
-                on_progress(item_index + 1, total_items)
-    finally:
-        if owns_client:
-            http_client.close()
-    return results
-
-
-__all__ = ["ModelGenerationError", "chat_completions_url", "generate_model_records"]

backend/app/modules/data_process/schema_cli.py

@@ -1,65 +0,0 @@
-"""数据处理运行表的显式检查与安装命令。"""
-
-from __future__ import annotations
-
-import argparse
-from urllib.parse import urlsplit
-
-from app.modules.data_process.store import DataProcessStore
-
-
-def _target_label(database_url: str) -> str:
-    parsed = urlsplit(database_url)
-    database = parsed.path.strip("/") or "(unknown)"
-    return f"{parsed.hostname or '(unknown)'}:{parsed.port or 5432}/{database}"
-
-
-def _schema_ready(store: DataProcessStore) -> bool:
-    with store.connect() as conn:
-        row = conn.execute(
-            """
-            SELECT EXISTS (
-                SELECT 1
-                FROM information_schema.columns
-                WHERE table_schema=current_schema()
-                  AND table_name='data_process_tasks'
-                  AND column_name='generation_run_id'
-            ) AS ready
-            """
-        ).fetchone()
-    return bool(row and row["ready"])
-
-
-def main() -> int:
-    parser = argparse.ArgumentParser(
-        description="检查或显式安装数据处理运行表（不会由应用启动自动执行）"
-    )
-    action = parser.add_mutually_exclusive_group(required=True)
-    action.add_argument("--check", action="store_true", help="只读检查迁移是否已安装")
-    action.add_argument("--apply", action="store_true", help="执行 002 数据处理迁移")
-    parser.add_argument(
-        "--yes",
-        action="store_true",
-        help="确认允许修改 DATABASE_URL 指向的数据库；与 --apply 同时使用",
-    )
-    args = parser.parse_args()
-
-    store = DataProcessStore()
-    target = _target_label(store.database_url)
-    if args.check:
-        ready = _schema_ready(store)
-        print(f"数据处理 schema：{'已安装' if ready else '未安装'}；目标：{target}")
-        return 0 if ready else 1
-    if not args.yes:
-        parser.error("--apply 必须同时提供 --yes，确认修改目标数据库")
-
-    print(f"正在安装数据处理 schema；目标：{target}")
-    store.ensure_schema()
-    if not _schema_ready(store):
-        raise RuntimeError("迁移执行后仍未检测到 generation_run_id")
-    print("数据处理 schema 安装完成")
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

backend/app/modules/data_process/storage.py

@@ -1,499 +0,0 @@
-"""数据处理原始源文件的受控本地对象存储。"""
-
-from __future__ import annotations
-
-import os
-import re
-import stat
-import unicodedata
-import uuid
-from dataclasses import dataclass
-from functools import lru_cache
-from pathlib import Path, PurePosixPath
-from typing import Iterable, Iterator
-from urllib.parse import quote, unquote, urlsplit
-
-
-class DataProcessStorageError(ValueError):
-    """本地对象引用或文件系统状态不安全。"""
-
-
-@dataclass(frozen=True, slots=True)
-class StagedSourceObject:
-    """尚未发布的原始文件；绝对路径仅在存储模块内部流转。"""
-
-    reference: str
-    _temporary_path: Path
-    _relative_path: PurePosixPath
-
-
-def _default_storage_root() -> Path:
-    return Path(__file__).resolve().parents[3] / "storage" / "data-process"
-
-
-def _configured_storage_root() -> Path:
-    configured = os.getenv("DATA_PROCESS_STORAGE_DIR", "").strip()
-    if not configured:
-        return _default_storage_root()
-    path = Path(configured).expanduser()
-    # 相对配置固定以 backend 目录为基准，
-    # 避免从不同 cwd 启动时写入不同位置。
-    return path if path.is_absolute() else Path(__file__).resolve().parents[3] / path
-
-
-def _safe_component(value: str, label: str) -> str:
-    if not value or value in {".", ".."} or len(value) > 128:
-        raise DataProcessStorageError(f"invalid {label}")
-    if not value[0].isalnum() or any(
-        not (character.isalnum() or character in {"-", "_", "."})
-        for character in value
-    ):
-        raise DataProcessStorageError(f"invalid {label}")
-    return value
-
-
-def _safe_basename(value: str) -> str:
-    if not value or len(value.encode("utf-8")) > 255:
-        raise DataProcessStorageError("invalid source file name")
-    if value != Path(value).name or "/" in value or "\\" in value or "\x00" in value:
-        raise DataProcessStorageError("invalid source file name")
-    if value in {".", ".."} or any(
-        unicodedata.category(character).startswith("C") for character in value
-    ):
-        raise DataProcessStorageError("invalid source file name")
-    return value
-
-
-class LocalDataProcessStorage:
-    """只允许访问配置根目录下的版本化原始文件。"""
-
-    def __init__(self, root: str | os.PathLike[str] | Path | None = None) -> None:
-        configured = Path(root) if root is not None else _configured_storage_root()
-        configured = configured.expanduser()
-        if configured.exists() and configured.is_symlink():
-            raise DataProcessStorageError("data process storage root must not be a symlink")
-        configured.mkdir(parents=True, exist_ok=True, mode=0o700)
-        self._root = configured.resolve(strict=True)
-        # StagedSourceObject 本身是普通 dataclass，不能只依赖其中的路径字段判断
-        # 来源；只接受由当前存储实例实际签发的对象，
-        # 避免调用方伪造暂存路径。
-        self._issued_staged_objects: dict[Path, StagedSourceObject] = {}
-        self._ensure_directory(self._root / ".staging")
-
-    @property
-    def root(self) -> Path:
-        """仅供运维和测试检查；API 响应不得序列化该属性。"""
-
-        return self._root
-
-    def new_batch_id(self) -> str:
-        return f"batch-{uuid.uuid4().hex}"
-
-    def stage_bytes(
-        self,
-        *,
-        batch_id: str,
-        task_id: str,
-        source_file_id: str,
-        version: int,
-        name: str,
-        content: bytes,
-    ) -> StagedSourceObject:
-        batch_id = _safe_component(batch_id, "batch id")
-        task_id = _safe_component(task_id, "task id")
-        source_file_id = _safe_component(source_file_id, "source file id")
-        if isinstance(version, bool) or not isinstance(version, int) or version < 1:
-            raise DataProcessStorageError("invalid source file version")
-        basename = _safe_basename(name)
-        if not isinstance(content, bytes):
-            raise TypeError("content must be bytes")
-
-        batch_directory = self._ensure_directory(self._root / ".staging" / batch_id)
-        temporary_path = batch_directory / f"{source_file_id}-{uuid.uuid4().hex}.tmp"
-        flags = os.O_CREAT | os.O_EXCL | os.O_WRONLY
-        if hasattr(os, "O_NOFOLLOW"):
-            flags |= os.O_NOFOLLOW
-        descriptor = os.open(temporary_path, flags, 0o600)
-        try:
-            with os.fdopen(descriptor, "wb", closefd=True) as stream:
-                stream.write(content)
-                stream.flush()
-                os.fsync(stream.fileno())
-        except Exception:
-            temporary_path.unlink(missing_ok=True)
-            raise
-
-        relative_path = PurePosixPath(
-            task_id,
-            source_file_id,
-            f"v{version}",
-            basename,
-        )
-        reference = (
-            "local://data-process/"
-            f"{task_id}/{source_file_id}/v{version}/{quote(basename, safe='')}"
-        )
-        staged = StagedSourceObject(reference, temporary_path, relative_path)
-        self._issued_staged_objects[temporary_path] = staged
-        return staged
-
-    def publish(self, objects: Iterable[StagedSourceObject]) -> None:
-        staged = list(objects)
-        published: list[StagedSourceObject] = []
-        try:
-            seen_temporary_paths: set[Path] = set()
-            for item in staged:
-                self._validate_staged_object(item, require_file=True)
-                if item._temporary_path in seen_temporary_paths:
-                    raise DataProcessStorageError("duplicate staged source object")
-                seen_temporary_paths.add(item._temporary_path)
-            for item in staged:
-                final_path = self._path_for_relative(item._relative_path)
-                self._ensure_directory(final_path.parent)
-                if final_path.exists() or final_path.is_symlink():
-                    raise DataProcessStorageError("source storage object already exists")
-                os.link(item._temporary_path, final_path, follow_symlinks=False)
-                published.append(item)
-                item._temporary_path.unlink()
-                self._fsync_directory(final_path.parent)
-        except Exception:
-            for item in reversed(published):
-                try:
-                    self.delete(item.reference)
-                except Exception:
-                    # 回滚必须尽量处理其余对象，并保留真正的发布异常。
-                    pass
-            for item in staged:
-                try:
-                    self.discard([item])
-                except Exception:
-                    pass
-            raise
-        self.discard(staged)
-
-    def discard(self, objects: Iterable[StagedSourceObject]) -> None:
-        staged = list(objects)
-        for item in staged:
-            self._validate_staged_object(item, require_file=False)
-
-        batch_directories: set[Path] = set()
-        first_error: Exception | None = None
-        for item in staged:
-            temporary_path = item._temporary_path
-            try:
-                temporary_path.unlink(missing_ok=True)
-            except Exception as exc:
-                if first_error is None:
-                    first_error = exc
-            else:
-                self._issued_staged_objects.pop(temporary_path, None)
-            batch_directories.add(temporary_path.parent)
-        for directory in batch_directories:
-            self._remove_empty_directory(directory)
-        if first_error is not None:
-            raise first_error
-
-    def read(self, reference: str) -> bytes | None:
-        """读取 local 引用；旧 ``db://`` 对象返回 ``None`` 由数据库正文兜底。"""
-
-        relative_path = self._relative_from_reference(reference)
-        if relative_path is None:
-            return None
-        descriptor, _ = self._open_read_descriptor(relative_path)
-        with os.fdopen(descriptor, "rb", closefd=True) as stream:
-            return stream.read()
-
-    def file_size(
-        self,
-        reference: str,
-        *,
-        expected_task_id: str,
-        expected_source_file_id: str,
-    ) -> int | None:
-        """返回受控 local 对象大小；旧 ``db://`` 对象没有原始文件。"""
-
-        relative_path = self._relative_from_reference(reference)
-        if relative_path is None:
-            return None
-        self._assert_expected_owner(
-            relative_path,
-            expected_task_id=expected_task_id,
-            expected_source_file_id=expected_source_file_id,
-        )
-        descriptor, info = self._open_read_descriptor(relative_path)
-        os.close(descriptor)
-        return info.st_size
-
-    def iter_bytes(
-        self,
-        reference: str,
-        *,
-        expected_task_id: str,
-        expected_source_file_id: str,
-        expected_size: int,
-        start: int = 0,
-        length: int | None = None,
-        chunk_size: int = 256 * 1024,
-    ) -> Iterator[bytes]:
-        """按范围流式读取原始文件，避免 PDF 预览把大文件整体载入内存。"""
-
-        relative_path = self._relative_from_reference(reference)
-        if relative_path is None:
-            raise DataProcessStorageError("original source object is not available")
-        self._assert_expected_owner(
-            relative_path,
-            expected_task_id=expected_task_id,
-            expected_source_file_id=expected_source_file_id,
-        )
-        if start < 0 or expected_size < 0 or chunk_size < 1:
-            raise DataProcessStorageError("invalid source byte range")
-        descriptor, info = self._open_read_descriptor(relative_path)
-        if info.st_size != expected_size:
-            os.close(descriptor)
-            raise DataProcessStorageError("source object size does not match metadata")
-        remaining = expected_size - start if length is None else length
-        if remaining < 0 or start + remaining > expected_size:
-            os.close(descriptor)
-            raise DataProcessStorageError("invalid source byte range")
-        with os.fdopen(descriptor, "rb", closefd=True) as stream:
-            stream.seek(start)
-            while remaining:
-                chunk = stream.read(min(chunk_size, remaining))
-                if not chunk:
-                    raise DataProcessStorageError("source object ended unexpectedly")
-                remaining -= len(chunk)
-                yield chunk
-
-    def validate_owner(
-        self,
-        reference: str,
-        *,
-        expected_task_id: str,
-        expected_source_file_id: str,
-    ) -> bool:
-        """校验 local 引用归属；旧 ``db://`` 引用无需文件系统处理。"""
-
-        relative_path = self._relative_from_reference(reference)
-        if relative_path is None:
-            return False
-        self._assert_expected_owner(
-            relative_path,
-            expected_task_id=expected_task_id,
-            expected_source_file_id=expected_source_file_id,
-        )
-        return True
-
-    def _open_read_descriptor(
-        self,
-        relative_path: PurePosixPath,
-    ) -> tuple[int, os.stat_result]:
-        path = self._path_for_relative(relative_path)
-        self._assert_controlled_parent(path)
-        try:
-            before_open = path.lstat()
-        except FileNotFoundError as exc:
-            raise DataProcessStorageError("source storage object does not exist") from exc
-        if stat.S_ISLNK(before_open.st_mode) or not stat.S_ISREG(before_open.st_mode):
-            raise DataProcessStorageError("source storage object is not a regular file")
-        flags = os.O_RDONLY
-        if hasattr(os, "O_NOFOLLOW"):
-            flags |= os.O_NOFOLLOW
-        descriptor = os.open(path, flags)
-        after_open = os.fstat(descriptor)
-        if (
-            not stat.S_ISREG(after_open.st_mode)
-            or before_open.st_dev != after_open.st_dev
-            or before_open.st_ino != after_open.st_ino
-        ):
-            os.close(descriptor)
-            raise DataProcessStorageError("source storage object changed while opening")
-        return descriptor, after_open
-
-    def delete(
-        self,
-        reference: str,
-        *,
-        expected_task_id: str | None = None,
-        expected_source_file_id: str | None = None,
-    ) -> bool:
-        """删除受控 local 对象；旧 ``db://`` 引用保持不变。"""
-
-        relative_path = self._relative_from_reference(reference)
-        if relative_path is None:
-            return False
-        if (expected_task_id is None) != (expected_source_file_id is None):
-            raise DataProcessStorageError("both expected storage owner fields are required")
-        if expected_task_id is not None and expected_source_file_id is not None:
-            self._assert_expected_owner(
-                relative_path,
-                expected_task_id=expected_task_id,
-                expected_source_file_id=expected_source_file_id,
-            )
-        path = self._path_for_relative(relative_path)
-        self._assert_controlled_parent(path)
-        try:
-            info = path.lstat()
-        except FileNotFoundError:
-            return False
-        if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
-            raise DataProcessStorageError("refusing to delete a non-regular storage object")
-        path.unlink()
-        self._fsync_directory(path.parent)
-        for directory in (path.parent, path.parent.parent, path.parent.parent.parent):
-            self._remove_empty_directory(directory)
-        return True
-
-    @staticmethod
-    def _assert_expected_owner(
-        relative_path: PurePosixPath,
-        *,
-        expected_task_id: str,
-        expected_source_file_id: str,
-    ) -> None:
-        task_id = _safe_component(expected_task_id, "expected task id")
-        source_file_id = _safe_component(
-            expected_source_file_id,
-            "expected source file id",
-        )
-        if relative_path.parts[:2] != (task_id, source_file_id):
-            raise DataProcessStorageError("source storage object owner mismatch")
-
-    def _relative_from_reference(self, reference: str) -> PurePosixPath | None:
-        if reference.startswith("db://"):
-            return None
-        parsed = urlsplit(reference)
-        if parsed.scheme != "local" or parsed.netloc != "data-process":
-            raise DataProcessStorageError("unsupported source storage reference")
-        if parsed.query or parsed.fragment or "\\" in parsed.path:
-            raise DataProcessStorageError("unsafe source storage reference")
-        raw_parts = parsed.path.lstrip("/").split("/")
-        if len(raw_parts) != 4:
-            raise DataProcessStorageError("unsafe source storage reference")
-        if any(re.search(r"%(?![0-9A-Fa-f]{2})", part) for part in raw_parts):
-            raise DataProcessStorageError("unsafe source storage reference")
-        try:
-            decoded = [unquote(part, encoding="utf-8", errors="strict") for part in raw_parts]
-        except UnicodeDecodeError as exc:
-            raise DataProcessStorageError("unsafe source storage reference") from exc
-        if any("/" in part or "\\" in part for part in decoded):
-            raise DataProcessStorageError("unsafe source storage reference")
-        canonical_parts = [
-            quote(decoded[0], safe="-_."),
-            quote(decoded[1], safe="-_."),
-            quote(decoded[2], safe="-_."),
-            quote(decoded[3], safe=""),
-        ]
-        if canonical_parts != raw_parts:
-            raise DataProcessStorageError("source storage reference is not canonical")
-        task_id = _safe_component(decoded[0], "task id")
-        source_file_id = _safe_component(decoded[1], "source file id")
-        version_component = decoded[2]
-        if not version_component.startswith("v") or not version_component[1:].isdigit():
-            raise DataProcessStorageError("invalid source file version")
-        version = int(version_component[1:])
-        if version < 1:
-            raise DataProcessStorageError("invalid source file version")
-        basename = _safe_basename(decoded[3])
-        return PurePosixPath(task_id, source_file_id, f"v{version}", basename)
-
-    def _path_for_relative(self, relative_path: PurePosixPath) -> Path:
-        if relative_path.is_absolute() or any(
-            part in {"", ".", ".."} for part in relative_path.parts
-        ):
-            raise DataProcessStorageError("storage path escapes the configured root")
-        path = self._root.joinpath(*relative_path.parts)
-        self._assert_controlled_parent(path)
-        return path
-
-    def _validate_staged_object(
-        self,
-        item: StagedSourceObject,
-        *,
-        require_file: bool,
-    ) -> None:
-        if not isinstance(item, StagedSourceObject):
-            raise DataProcessStorageError("invalid staged source object")
-        if self._issued_staged_objects.get(item._temporary_path) is not item:
-            raise DataProcessStorageError("staged source object was not issued by this storage")
-        expected_relative = self._relative_from_reference(item.reference)
-        if expected_relative is None or expected_relative != item._relative_path:
-            raise DataProcessStorageError("staged source object reference mismatch")
-        staging_root = self._root / ".staging"
-        try:
-            relative_temporary = item._temporary_path.relative_to(staging_root)
-        except ValueError as exc:
-            raise DataProcessStorageError("staged source object escapes staging") from exc
-        if len(relative_temporary.parts) != 2:
-            raise DataProcessStorageError("invalid staged source object path")
-        _safe_component(relative_temporary.parts[0], "batch id")
-        _safe_basename(relative_temporary.parts[1])
-        self._assert_controlled_parent(item._temporary_path)
-        try:
-            info = item._temporary_path.lstat()
-        except FileNotFoundError:
-            if require_file:
-                raise DataProcessStorageError("staged source object does not exist") from None
-            return
-        if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
-            raise DataProcessStorageError("staged source object is not a regular file")
-
-    def _ensure_directory(self, directory: Path) -> Path:
-        try:
-            relative = directory.relative_to(self._root)
-        except ValueError as exc:
-            raise DataProcessStorageError("storage path escapes the configured root") from exc
-        current = self._root
-        for component in relative.parts:
-            current = current / component
-            try:
-                current.mkdir(mode=0o700)
-            except FileExistsError:
-                pass
-            info = current.lstat()
-            if stat.S_ISLNK(info.st_mode) or not stat.S_ISDIR(info.st_mode):
-                raise DataProcessStorageError("storage path contains a symlink or non-directory")
-        return directory
-
-    def _assert_controlled_parent(self, path: Path) -> None:
-        try:
-            relative_parent = path.parent.relative_to(self._root)
-        except ValueError as exc:
-            raise DataProcessStorageError("storage path escapes the configured root") from exc
-        current = self._root
-        for component in relative_parent.parts:
-            current = current / component
-            if not current.exists():
-                continue
-            info = current.lstat()
-            if stat.S_ISLNK(info.st_mode) or not stat.S_ISDIR(info.st_mode):
-                raise DataProcessStorageError("storage path contains a symlink or non-directory")
-
-    @staticmethod
-    def _fsync_directory(directory: Path) -> None:
-        descriptor = os.open(directory, os.O_RDONLY)
-        try:
-            os.fsync(descriptor)
-        finally:
-            os.close(descriptor)
-
-    def _remove_empty_directory(self, directory: Path) -> None:
-        if directory in {self._root, self._root / ".staging"}:
-            return
-        self._assert_controlled_parent(directory / "placeholder")
-        try:
-            directory.rmdir()
-        except (FileNotFoundError, OSError):
-            return
-
-
-@lru_cache
-def get_data_process_storage() -> LocalDataProcessStorage:
-    return LocalDataProcessStorage()
-
-
-__all__ = [
-    "DataProcessStorageError",
-    "LocalDataProcessStorage",
-    "StagedSourceObject",
-    "get_data_process_storage",
-]

backend/app/schemas/data_process.py

@@ -1,280 +0,0 @@
-from __future__ import annotations
-
-from enum import StrEnum
-from typing import Any, Literal
-
-from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
-
-
-def _config_value(config: dict[str, Any], snake_name: str, camel_name: str, default: Any) -> Any:
-    if snake_name in config:
-        return config[snake_name]
-    return config.get(camel_name, default)
-
-
-def _validate_process_config(config: dict[str, Any]) -> None:
-    chunk_method = _config_value(config, "chunk_method", "chunkMethod", "structure")
-    if not isinstance(chunk_method, str) or chunk_method not in {
-        "structure",
-        "fixed",
-        "custom",
-    }:
-        raise ValueError("chunk_method must be one of: structure, fixed, custom")
-    custom_delimiter = _config_value(
-        config,
-        "custom_delimiter",
-        "customDelimiter",
-        "",
-    )
-    if chunk_method == "custom" and (
-        not isinstance(custom_delimiter, str) or not custom_delimiter
-    ):
-        raise ValueError("custom_delimiter is required for custom chunking")
-
-    split = _config_value(config, "dataset_split", "datasetSplit", None)
-    if split is not None:
-        if not isinstance(split, dict) or set(split) != {"train", "validation", "test"}:
-            raise ValueError("dataset_split must contain train, validation and test")
-        values = list(split.values())
-        if any(isinstance(value, bool) or not isinstance(value, int) for value in values):
-            raise ValueError("dataset_split values must be integers")
-        if any(value < 0 or value > 100 for value in values) or sum(values) != 100:
-            raise ValueError("dataset_split values must be in [0, 100] and total 100")
-
-    chunk_fields = {
-        "chunk_size",
-        "chunkSize",
-        "chunk_overlap",
-        "chunkOverlap",
-        "min_chunk_size",
-        "minChunkSize",
-    }
-    if chunk_fields.intersection(config):
-        chunk_size = _config_value(config, "chunk_size", "chunkSize", 800)
-        overlap = _config_value(config, "chunk_overlap", "chunkOverlap", 100)
-        minimum = _config_value(config, "min_chunk_size", "minChunkSize", 100)
-        if any(
-            isinstance(value, bool) or not isinstance(value, int)
-            for value in (chunk_size, overlap, minimum)
-        ):
-            raise ValueError("chunk_size, chunk_overlap and min_chunk_size must be integers")
-        if not 16 <= chunk_size <= 32_768:
-            raise ValueError("chunk_size must be in [16, 32768]")
-        if overlap < 0 or overlap >= chunk_size:
-            raise ValueError("chunk_overlap must be in [0, chunk_size)")
-        if minimum <= 0 or minimum > chunk_size or overlap + minimum > chunk_size:
-            raise ValueError("min_chunk_size and chunk_overlap exceed chunk_size")
-
-    temperature = _config_value(config, "temperature", "temperature", None)
-    if temperature is not None:
-        if isinstance(temperature, bool) or not isinstance(temperature, (int, float)):
-            raise ValueError("temperature must be a number")
-        if not 0 <= float(temperature) <= 2:
-            raise ValueError("temperature must be in [0, 2]")
-
-    max_tokens = _config_value(config, "max_tokens", "maxTokens", None)
-    if max_tokens is not None:
-        if isinstance(max_tokens, bool) or not isinstance(max_tokens, int):
-            raise ValueError("max_tokens must be an integer")
-        if not 1 <= max_tokens <= 32_768:
-            raise ValueError("max_tokens must be in [1, 32768]")
-
-    for snake_name, camel_name in (
-        ("qa_pairs_per_row", "qaPairsPerRow"),
-        ("qa_pairs_per_chunk", "qaPairsPerChunk"),
-    ):
-        pairs = _config_value(config, snake_name, camel_name, None)
-        if pairs is None:
-            continue
-        if isinstance(pairs, bool) or not isinstance(pairs, int) or not 1 <= pairs <= 5:
-            raise ValueError(f"{snake_name} must be an integer in [1, 5]")
-
-
-class DataProcessStatus(StrEnum):
-    pending = "pending"
-    running = "running"
-    completed = "completed"
-    failed = "failed"
-    stopped = "stopped"
-
-
-class ProcessType(StrEnum):
-    structured = "structured"
-    unstructured = "unstructured"
-    external = "external"
-
-
-class DataProcessTaskCreate(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    name: str = Field(min_length=1, max_length=150)
-    description: str = ""
-    process_type: ProcessType
-    source_dataset_id: str | None = None
-    config: dict[str, Any] = Field(default_factory=dict)
-
-    @field_validator("name")
-    @classmethod
-    def normalize_name(cls, value: str) -> str:
-        value = value.strip()
-        if not value:
-            raise ValueError("task name cannot be empty")
-        return value
-
-    @model_validator(mode="after")
-    def validate_config(self) -> "DataProcessTaskCreate":
-        _validate_process_config(self.config)
-        return self
-
-
-class DataProcessTaskUpdate(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    name: str | None = Field(default=None, min_length=1, max_length=150)
-    description: str | None = None
-    process_type: ProcessType | None = None
-    source_dataset_id: str | None = None
-    config: dict[str, Any] | None = None
-
-    @field_validator("name")
-    @classmethod
-    def normalize_name(cls, value: str | None) -> str | None:
-        if value is None:
-            return None
-        value = value.strip()
-        if not value:
-            raise ValueError("task name cannot be empty")
-        return value
-
-    @model_validator(mode="after")
-    def validate_config(self) -> "DataProcessTaskUpdate":
-        if self.config is not None:
-            _validate_process_config(self.config)
-        return self
-
-
-class PreviewBuildRequest(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    replace_existing: Literal[True] = True
-    source_file_ids: list[str] | None = None
-    source_file_id: str | None = None
-
-    @model_validator(mode="after")
-    def validate_source_file_selection(self) -> "PreviewBuildRequest":
-        if self.source_file_ids is not None and self.source_file_id is not None:
-            raise ValueError("source_file_id and source_file_ids cannot be used together")
-        values = self.source_file_ids
-        if values is None and self.source_file_id is not None:
-            values = [self.source_file_id]
-        if values is None:
-            return self
-        normalized = list(dict.fromkeys(str(value).strip() for value in values))
-        if not normalized or any(not value for value in normalized):
-            raise ValueError("at least one non-empty source file id is required")
-        self.source_file_ids = normalized
-        self.source_file_id = None
-        return self
-
-
-class PreviewItemCreate(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    source_file_id: str | None = None
-    original_content: str = ""
-    edited_content: str = ""
-    source_start: int | None = Field(default=None, ge=0)
-    source_end: int | None = Field(default=None, ge=0)
-    source_start_line: int | None = Field(default=None, ge=1)
-    source_end_line: int | None = Field(default=None, ge=1)
-
-    @model_validator(mode="after")
-    def validate_ranges(self) -> "PreviewItemCreate":
-        if self.source_start is not None and self.source_end is not None:
-            if self.source_end < self.source_start:
-                raise ValueError("source_end must be greater than or equal to source_start")
-        if self.source_start_line is not None and self.source_end_line is not None:
-            if self.source_end_line < self.source_start_line:
-                raise ValueError(
-                    "source_end_line must be greater than or equal to source_start_line"
-                )
-        return self
-
-
-class PreviewItemUpdate(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    edited_content: str
-    expected_updated_at: str | None = None
-
-
-class GenerateRequest(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    replace_existing: Literal[True] = True
-
-
-class ExternalSourceRequest(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    type: str = Field(min_length=1, max_length=30)
-    url: str = Field(min_length=1, max_length=2048)
-    auth_mode: Literal["none", "basic"] = "none"
-    username: str | None = Field(default=None, max_length=150)
-    password: str | None = Field(default=None, max_length=500)
-    limit: int = Field(default=1000, ge=1, le=100_000)
-
-
-class ExternalPullRequest(ExternalSourceRequest):
-    query: str | None = Field(default=None, max_length=20_000)
-    file_name: str = Field(default="external-data.jsonl", min_length=1, max_length=255)
-
-    @field_validator("file_name")
-    @classmethod
-    def validate_file_name(cls, value: str) -> str:
-        name = value.strip()
-        if not name.lower().endswith((".jsonl", ".ndjson")):
-            raise ValueError("external pull file_name must end with .jsonl or .ndjson")
-        return name
-
-
-class ResultUpdate(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    instruction: str | None = None
-    input: str | None = None
-    output: str | None = None
-    expected_updated_at: str | None = None
-
-
-class DatasetSplit(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    train: int = Field(default=80, ge=0, le=100)
-    validation: int = Field(default=10, ge=0, le=100)
-    test: int = Field(default=10, ge=0, le=100)
-
-    @model_validator(mode="after")
-    def validate_total(self) -> "DatasetSplit":
-        if self.train + self.validation + self.test != 100:
-            raise ValueError("dataset split must total 100")
-        return self
-
-
-class PublishRequest(BaseModel):
-    model_config = ConfigDict(extra="forbid")
-
-    dataset_name: str = Field(min_length=1, max_length=150)
-    dataset_type: Literal["train", "test", "eval", "val", "other"] = "train"
-    storage_type: Literal["local"] = "local"
-    split: DatasetSplit = Field(default_factory=DatasetSplit)
-    format: Literal["alpaca_jsonl", "jsonl"] = "alpaca_jsonl"
-    description: str = ""
-
-    @field_validator("dataset_name")
-    @classmethod
-    def normalize_dataset_name(cls, value: str) -> str:
-        value = value.strip()
-        if not value:
-            raise ValueError("dataset name cannot be empty")
-        return value

backend/app/workers/compute_poller.py

@@ -1,31 +0,0 @@
-from __future__ import annotations
-
-import asyncio
-
-from app.core.config import get_settings
-from app.core.logging import get_logger
-from app.modules.compute_gateway.sync import poll_compute_jobs_once
-
-
-logger = get_logger(__name__)
-
-
-async def run_compute_poller() -> None:
-    settings = get_settings()
-    if settings.compute_mode == "simulator" or settings.compute_status_sync_mode != "polling":
-        logger.info("compute poller disabled", extra={"compute_mode": settings.compute_mode})
-        return
-
-    interval = max(3, settings.compute_poll_interval_seconds)
-    logger.info("compute poller started", extra={"interval_seconds": interval})
-    while True:
-        try:
-            result = await poll_compute_jobs_once()
-            if result["synced"] or result["failed"]:
-                logger.info("compute jobs polled", extra={"result": result})
-        except asyncio.CancelledError:
-            logger.info("compute poller stopped")
-            raise
-        except Exception as exc:  # noqa: BLE001 - keep background polling alive
-            logger.exception("compute poller failed", extra={"error": str(exc)})
-        await asyncio.sleep(interval)

backend/config.example.yaml

@@ -0,0 +1,31 @@
+# 后端配置模板。复制为 config.yaml 后填写本地数据库凭据。
+# config.yaml 已加入 Git 忽略，不会提交真实密码。
+
+app:
+  name: YG Fine-Tune Platform API
+  env: local
+  mode: local
+  route_prefix: /modelTF
+
+server:
+  # 一键启动脚本统一读取这里的端口；端口被占用时只需修改这两项。
+  frontend_port: 16801
+  backend_port: 17861
+
+database:
+  url: postgresql+psycopg://localhost:5432/yg_ft
+  username: yg_ft
+  password: "change_me"
+
+compute:
+  mode: real
+  status_sync_mode: polling
+  poll_interval_seconds: 3
+
+logging:
+  level: INFO
+  directory: ./logs
+  file_prefix: backend
+  error_file_prefix: error
+  max_bytes: 20971520
+  retention_days: 10

backend/pyproject.toml

@@ -16,11 +16,7 @@ dependencies = [
   "PyJWT>=2.8.0",
   "passlib[bcrypt]>=1.7.4",
   "python-dotenv>=1.0.1",
-  "pypdf[crypto]>=5.0.0",
-  "python-docx>=1.1.2",
-  "openpyxl>=3.1.5",
-  "python-pptx>=1.0.2",
-  "llama-index-core==0.14.23",
+  "PyYAML>=6.0.2",
 ]
 
 [project.optional-dependencies]

backend/requirements.txt

@@ -10,8 +10,4 @@ httpx>=0.27.0
 PyJWT>=2.8.0
 passlib[bcrypt]>=1.7.4
 python-dotenv>=1.0.1
-pypdf[crypto]>=5.0.0
-python-docx>=1.1.2
-openpyxl>=3.1.5
-python-pptx>=1.0.2
-llama-index-core==0.14.23
+PyYAML>=6.0.2

backend/tests/test_config.py

@@ -0,0 +1,196 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+
+from app.core.config import ConfigurationError, load_settings
+
+
+CONFIG_ENV_NAMES = (
+    "APP_NAME",
+    "APP_ENV",
+    "APP_MODE",
+    "MODELTF_ROUTE_PREFIX",
+    "FRONTEND_PORT",
+    "BACKEND_PORT",
+    "DATABASE_URL",
+    "DATABASE_BASE_URL",
+    "DATABASE_USERNAME",
+    "DATABASE_PASSWORD",
+    "CORS_ALLOW_ORIGINS",
+    "COMPUTE_MODE",
+    "COMPUTE_STATUS_SYNC_MODE",
+    "COMPUTE_POLL_INTERVAL_SECONDS",
+    "LOG_LEVEL",
+    "LOG_DIR",
+    "LOG_FILE_PREFIX",
+    "LOG_ERROR_FILE_PREFIX",
+    "LOG_MAX_BYTES",
+    "LOG_RETENTION_DAYS",
+)
+
+
+def clear_config_env(monkeypatch: pytest.MonkeyPatch) -> None:
+    for env_name in CONFIG_ENV_NAMES:
+        monkeypatch.delenv(env_name, raising=False)
+
+
+def write_config(path: Path) -> None:
+    path.write_text(
+        """
+app:
+  name: YAML API
+  route_prefix: /yaml-api
+  cors_allow_origins:
+    - http://yaml.example
+server:
+  frontend_port: 18001
+  backend_port: 18002
+database:
+  url: postgresql+psycopg://db:5432/yaml
+  username: yaml-user
+  password: yaml-secret
+compute:
+  mode: simulated
+  poll_interval_seconds: 9
+logging:
+  directory: ./yaml-logs
+  max_bytes: 1024
+  retention_days: 2
+""".strip(),
+        encoding="utf-8",
+    )
+
+
+def test_load_settings_from_yaml(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    clear_config_env(monkeypatch)
+    config_path = tmp_path / "config.yaml"
+    write_config(config_path)
+
+    settings = load_settings(config_path)
+
+    assert settings.app_name == "YAML API"
+    assert settings.route_prefix == "/yaml-api"
+    assert settings.frontend_port == 18001
+    assert settings.backend_port == 18002
+    assert settings.database_url == "postgresql+psycopg://yaml-user:yaml-secret@db:5432/yaml"
+    assert settings.cors_allow_origins == ["http://yaml.example"]
+    assert settings.compute_mode == "simulated"
+    assert settings.compute_poll_interval_seconds == 9
+    assert settings.log_dir == str(tmp_path / "yaml-logs")
+    assert settings.log_max_bytes == 1024
+    assert settings.log_retention_days == 2
+
+
+def test_environment_overrides_yaml(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    clear_config_env(monkeypatch)
+    config_path = tmp_path / "config.yaml"
+    write_config(config_path)
+    monkeypatch.setenv("DATABASE_URL", "postgresql+psycopg://env:secret@db:5432/env")
+    monkeypatch.setenv("CORS_ALLOW_ORIGINS", "http://one.example,http://two.example")
+    monkeypatch.setenv("COMPUTE_POLL_INTERVAL_SECONDS", "15")
+    monkeypatch.setenv("FRONTEND_PORT", "19001")
+    monkeypatch.setenv("BACKEND_PORT", "19002")
+
+    settings = load_settings(config_path)
+
+    assert settings.database_url == "postgresql+psycopg://env:secret@db:5432/env"
+    assert settings.cors_allow_origins == ["http://one.example", "http://two.example"]
+    assert settings.compute_poll_interval_seconds == 15
+    assert settings.frontend_port == 19001
+    assert settings.backend_port == 19002
+
+
+def test_separate_database_credentials_are_encoded(
+    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    clear_config_env(monkeypatch)
+    config_path = tmp_path / "config.yaml"
+    write_config(config_path)
+    monkeypatch.setenv("DATABASE_USERNAME", "user@example.com")
+    monkeypatch.setenv("DATABASE_PASSWORD", "secret:/?#[]@")
+
+    settings = load_settings(config_path)
+
+    assert settings.database_url == (
+        "postgresql+psycopg://user%40example.com:secret%3A%2F%3F%23%5B%5D%40@db:5432/yaml"
+    )
+
+
+def test_explicit_missing_config_is_rejected(tmp_path: Path) -> None:
+    with pytest.raises(ConfigurationError, match="does not exist"):
+        load_settings(tmp_path / "missing.yaml")
+
+
+def test_default_cors_follows_frontend_port(
+    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    clear_config_env(monkeypatch)
+    config_path = tmp_path / "config.yaml"
+    config_path.write_text(
+        "server:\n  frontend_port: 28001\n  backend_port: 28002\n",
+        encoding="utf-8",
+    )
+
+    settings = load_settings(config_path)
+
+    assert settings.cors_allow_origins == [
+        "http://localhost:28001",
+        "http://127.0.0.1:28001",
+    ]
+
+
+@pytest.mark.parametrize("frontend_port", [0, 65536, "invalid", True])
+def test_invalid_server_port_is_rejected(
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+    frontend_port: object,
+) -> None:
+    clear_config_env(monkeypatch)
+    config_path = tmp_path / "config.yaml"
+    config_path.write_text(
+        "server:\n"
+        f"  frontend_port: {str(frontend_port).lower()}\n"
+        "  backend_port: 28002\n",
+        encoding="utf-8",
+    )
+
+    with pytest.raises(ConfigurationError, match="server.frontend_port"):
+        load_settings(config_path)
+
+
+def test_frontend_and_backend_ports_must_differ(
+    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    clear_config_env(monkeypatch)
+    config_path = tmp_path / "config.yaml"
+    config_path.write_text(
+        "server:\n  frontend_port: 28001\n  backend_port: 28001\n",
+        encoding="utf-8",
+    )
+
+    with pytest.raises(ConfigurationError, match="must be different"):
+        load_settings(config_path)
+
+
+@pytest.mark.parametrize(
+    ("database_base_url", "message"),
+    [
+        ("mysql://localhost:3306/yg_ft", "must use postgresql"),
+        ("postgresql+psycopg://localhost:5432", "must include a database name"),
+    ],
+)
+def test_invalid_database_url_is_rejected(
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+    database_base_url: str,
+    message: str,
+) -> None:
+    clear_config_env(monkeypatch)
+    config_path = tmp_path / "config.yaml"
+    write_config(config_path)
+    monkeypatch.setenv("DATABASE_BASE_URL", database_base_url)
+
+    with pytest.raises(ConfigurationError, match=message):
+        load_settings(config_path)

backend/tests/test_data_process_algorithms.py

@@ -1,895 +0,0 @@
-from __future__ import annotations
-
-import io
-import json
-import xml.etree.ElementTree as ET
-import zipfile
-from datetime import datetime
-
-import pytest
-from docx import Document
-from openpyxl import Workbook
-from pptx import Presentation
-from pptx.util import Inches
-from pypdf import PdfWriter
-
-from app.modules.data_process.algorithms import (
-    PdfPageText,
-    chunk_unstructured,
-    content_quality_flags,
-    desensitize_pii,
-    desensitize_structured_record,
-    detect_document_structure,
-    detect_pdf_document_noise,
-    detect_text_format,
-    estimate_token_count,
-    extract_pdf_page_texts,
-    extract_structured_records,
-    generate_standard_records,
-    is_near_duplicate,
-    merge_short_blocks,
-    normalize_text,
-    parse_text_content,
-    preprocess_structured_records,
-    record_fingerprint,
-    remove_document_noise,
-    score_quality,
-    stable_split,
-)
-
-
-def _pdf_page_texts(*texts: str) -> tuple[PdfPageText, ...]:
-    pages: list[PdfPageText] = []
-    offset = 0
-    for page_number, text in enumerate(texts, start=1):
-        normalized = normalize_text(text)
-        if pages:
-            offset += 2
-        start = offset
-        offset += len(normalized)
-        pages.append(
-            PdfPageText(
-                page_number=page_number,
-                text=normalized,
-                source_start=start,
-                source_end=offset,
-            )
-        )
-    return tuple(pages)
-
-
-def _minimal_pdf(text: str = "Hello PDF") -> bytes:
-    stream = f"BT /F1 12 Tf 72 720 Td ({text}) Tj ET".encode("ascii")
-    objects = [
-        b"<< /Type /Catalog /Pages 2 0 R >>",
-        b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
-        (
-            b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] "
-            b"/Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R >>"
-        ),
-        b"<< /Length " + str(len(stream)).encode("ascii") + b" >>\nstream\n"
-        + stream
-        + b"\nendstream",
-        b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
-    ]
-    result = bytearray(b"%PDF-1.4\n%\xe2\xe3\xcf\xd3\n")
-    offsets = [0]
-    for object_number, value in enumerate(objects, start=1):
-        offsets.append(len(result))
-        result.extend(f"{object_number} 0 obj\n".encode("ascii"))
-        result.extend(value)
-        result.extend(b"\nendobj\n")
-    xref_offset = len(result)
-    result.extend(f"xref\n0 {len(objects) + 1}\n".encode("ascii"))
-    result.extend(b"0000000000 65535 f \n")
-    for offset in offsets[1:]:
-        result.extend(f"{offset:010d} 00000 n \n".encode("ascii"))
-    result.extend(
-        (
-            f"trailer\n<< /Size {len(objects) + 1} /Root 1 0 R >>\n"
-            f"startxref\n{xref_offset}\n%%EOF\n"
-        ).encode("ascii")
-    )
-    return bytes(result)
-
-
-def _aes_encrypted_pdf(*, user_password: str) -> bytes:
-    writer = PdfWriter(clone_from=io.BytesIO(_minimal_pdf()))
-    writer.encrypt(
-        user_password=user_password,
-        owner_password="owner-secret",
-        algorithm="AES-256",
-    )
-    output = io.BytesIO()
-    writer.write(output)
-    return output.getvalue()
-
-
-def _docx_bytes() -> bytes:
-    document = Document()
-    document.add_heading("服务说明", level=1)
-    document.add_paragraph("这是 DOCX 正文。")
-    table = document.add_table(rows=1, cols=2)
-    table.cell(0, 0).text = "字段"
-    table.cell(0, 1).text = "内容"
-    output = io.BytesIO()
-    document.save(output)
-    return output.getvalue()
-
-
-def _xlsx_bytes() -> bytes:
-    workbook = Workbook()
-    worksheet = workbook.active
-    worksheet.title = "数据"
-    worksheet.append(["name", "score", "created_at"])
-    worksheet.append(["Alice", 95, datetime(2026, 7, 23, 10, 30)])
-    worksheet.append(["Bob", 88, datetime(2026, 7, 24, 9, 0)])
-    output = io.BytesIO()
-    workbook.save(output)
-    workbook.close()
-    return output.getvalue()
-
-
-def _xlsx_with_worksheet_relationship(
-    raw: bytes,
-    target: str,
-    *,
-    target_mode: str | None = None,
-) -> bytes:
-    member_name = "xl/_rels/workbook.xml.rels"
-    source = io.BytesIO(raw)
-    output = io.BytesIO()
-    with zipfile.ZipFile(source) as original, zipfile.ZipFile(output, "w") as rewritten:
-        for member in original.infolist():
-            content = original.read(member.filename)
-            if member.filename == member_name:
-                root = ET.fromstring(content)
-                worksheet_relationship = next(
-                    element
-                    for element in root
-                    if element.attrib.get("Type", "").endswith("/worksheet")
-                )
-                worksheet_relationship.set("Target", target)
-                if target_mode is None:
-                    worksheet_relationship.attrib.pop("TargetMode", None)
-                else:
-                    worksheet_relationship.set("TargetMode", target_mode)
-                content = ET.tostring(root, encoding="utf-8", xml_declaration=True)
-            rewritten.writestr(member, content)
-    return output.getvalue()
-
-
-def _pptx_bytes() -> bytes:
-    presentation = Presentation()
-    slide = presentation.slides.add_slide(presentation.slide_layouts[6])
-    text_box = slide.shapes.add_textbox(Inches(1), Inches(1), Inches(6), Inches(1))
-    text_box.text = "PPTX 页面正文"
-    output = io.BytesIO()
-    presentation.save(output)
-    return output.getvalue()
-
-
-def test_parse_utf8_json_jsonl_csv_markdown_and_txt() -> None:
-    parsed_json = parse_text_content(
-        b'\xef\xbb\xbf{"data":[{"name":"\xe5\xbc\xa0\xe4\xb8\x89"}]}',
-        filename="records.json",
-    )
-    assert parsed_json.format == "json"
-    assert parsed_json.records == ({"name": "张三"},)
-
-    parsed_jsonl = parse_text_content('{"id":1}\n\n{"id":2}\n', filename="records.jsonl")
-    assert parsed_jsonl.format == "jsonl"
-    assert parsed_jsonl.records == ({"id": 1}, {"id": 2})
-
-    parsed_csv = parse_text_content("name,answer\r\nAlice,yes\r\nBob,no", filename="records.csv")
-    assert parsed_csv.format == "csv"
-    assert parsed_csv.text == "name,answer\nAlice,yes\nBob,no"
-    assert parsed_csv.records[1] == {"name": "Bob", "answer": "no"}
-
-    parsed_markdown = parse_text_content("# 标题\n\n正文", filename="README.md")
-    assert parsed_markdown.format == "markdown"
-    assert parsed_markdown.records == ()
-
-    parsed_txt = parse_text_content("普通文本", filename="note.txt")
-    assert parsed_txt.format == "txt"
-    assert parsed_txt.text == "普通文本"
-
-
-def test_parse_pdf_docx_xlsx_and_pptx() -> None:
-    parsed_pdf = parse_text_content(_minimal_pdf(), filename="manual.pdf")
-    assert parsed_pdf.format == "pdf"
-    assert "Hello PDF" in parsed_pdf.text
-    assert parsed_pdf.records == ()
-
-    pdf_pages = extract_pdf_page_texts(_minimal_pdf())
-    assert len(pdf_pages) == 1
-    assert pdf_pages[0].page_number == 1
-    assert pdf_pages[0].text == "Hello PDF"
-    assert pdf_pages[0].source_start == 0
-    assert pdf_pages[0].source_end == len(parsed_pdf.text)
-
-    parsed_docx = parse_text_content(_docx_bytes(), filename="manual.docx")
-    assert parsed_docx.format == "docx"
-    assert "服务说明" in parsed_docx.text
-    assert "这是 DOCX 正文。" in parsed_docx.text
-    assert "字段\t内容" in parsed_docx.text
-    assert parsed_docx.records == ()
-
-    parsed_xlsx = parse_text_content(_xlsx_bytes(), filename="records.xlsx")
-    assert parsed_xlsx.format == "xlsx"
-    assert parsed_xlsx.records == (
-        {"name": "Alice", "score": 95, "created_at": "2026-07-23T10:30:00"},
-        {"name": "Bob", "score": 88, "created_at": "2026-07-24T09:00:00"},
-    )
-    assert json.loads(parsed_xlsx.text.splitlines()[0]) == parsed_xlsx.records[0]
-
-    parsed_pptx = parse_text_content(_pptx_bytes(), filename="slides.pptx")
-    assert parsed_pptx.format == "pptx"
-    assert parsed_pptx.text == "PPTX 页面正文"
-    assert parsed_pptx.records == ()
-
-
-def test_pdf_document_noise_removes_headers_page_numbers_and_toc_safely() -> None:
-    pages = _pdf_page_texts(
-        """
-        远光制度文件 文件编码 2024
-        秘密等级 商密【中】
-        第 1 页 共 5 页
-        正文第一页，关于适用范围的说明。
-        业务提示保留
-        第一页补充说明甲
-        第一页补充说明乙
-        第一页补充说明丙
-        """,
-        """
-        远光制度文件 文件编码 2024
-        秘密等级 商密【中】
-        第 2 页 共 5 页
-        目 录
-        第一章 总则........3
-        第二章 报销申请........4
-        第三章 附则........5
-        """,
-        """
-        远光制度文件 文件编码 2024
-        秘密等级 商密【中】
-        第 3 页 共 5 页
-        1.1 管理要求........6
-        1.2 审批职责 7
-        1.3 费用标准........8
-        1.4 例外处理........9
-        """,
-        """
-        远光制度文件 文件编码 2024
-        秘密等级 商密【中】
-        第 4 页 共 5 页
-        正文中可以说“请参见第 3 页说明”，不应误删。
-        第 99 页 共 100 页
-        系统可用率........99.9%
-        业务提示保留
-        第四页补充说明甲
-        第四页补充说明乙
-        第四页补充说明丙
-        """,
-        """
-        远光制度文件 文件编码 2024
-        秘密等级 商密【中】
-        第 5 页 共 5 页
-        本办法自发布之日起施行。
-        业务提示保留
-        第五页补充说明甲
-        第五页补充说明乙
-        第五页补充说明丙
-        """,
-    )
-    source = "\n\n".join(page.text for page in pages)
-
-    spans = detect_pdf_document_noise(pages)
-    cleaned = remove_document_noise(source, spans)
-
-    assert {span.kind for span in spans} == {
-        "page_number",
-        "repeated_margin",
-        "table_of_contents",
-    }
-    assert "远光制度文件" not in cleaned
-    assert "商密【中】" not in cleaned
-    assert "第 1 页 共 5 页" not in cleaned
-    assert "第一章 总则" not in cleaned
-    assert "1.2 审批职责 7" not in cleaned
-    assert "请参见第 3 页说明" in cleaned
-    assert "第 99 页 共 100 页" in cleaned
-    assert "系统可用率........99.9%" in cleaned
-    assert cleaned.count("业务提示保留") == 3
-
-
-def test_pdf_document_noise_does_not_infer_repeated_margins_for_short_documents() -> None:
-    pages = _pdf_page_texts(
-        "公司内部文件\n正文 A",
-        "公司内部文件\n正文 B",
-    )
-
-    spans = detect_pdf_document_noise(pages)
-
-    assert not any(span.kind == "repeated_margin" for span in spans)
-
-
-def test_xlsx_merged_multilevel_headers_are_flattened_without_losing_columns() -> None:
-    workbook = Workbook()
-    worksheet = workbook.active
-    worksheet.merge_cells("A1:A2")
-    worksheet.merge_cells("B1:C1")
-    worksheet["A1"] = "地区"
-    worksheet["B1"] = "销售"
-    worksheet["B2"] = "Q1"
-    worksheet["C2"] = "Q2"
-    worksheet.append(["华东", 100, 120])
-    output = io.BytesIO()
-    workbook.save(output)
-    workbook.close()
-
-    parsed = parse_text_content(output.getvalue(), filename="sales.xlsx")
-    assert parsed.records == ({"地区": "华东", "销售.Q1": 100, "销售.Q2": 120},)
-
-
-def test_xlsx_header_inference_skips_more_than_eight_merged_report_titles() -> None:
-    workbook = Workbook()
-    worksheet = workbook.active
-    for row_number in range(1, 13):
-        worksheet.merge_cells(
-            start_row=row_number,
-            start_column=1,
-            end_row=row_number,
-            end_column=4,
-        )
-        worksheet.cell(row_number, 1, f"报表说明 {row_number}")
-    worksheet.append(["姓名", "部门", "得分", "日期"])
-    worksheet.append(["张三", "研发", 95, "2026-07-23"])
-    output = io.BytesIO()
-    workbook.save(output)
-    workbook.close()
-
-    parsed = parse_text_content(output.getvalue(), filename="report.xlsx")
-    assert parsed.records == (
-        {"姓名": "张三", "部门": "研发", "得分": 95, "日期": "2026-07-23"},
-    )
-
-
-def test_xlsx_header_inference_ignores_continuous_body_merges() -> None:
-    workbook = Workbook()
-    worksheet = workbook.active
-    worksheet.append(["类别", "名称", "数量"])
-    worksheet.append(["水果", "苹果", 10])
-    worksheet.append([None, "香蕉", 12])
-    worksheet.append(["蔬菜", "白菜", 8])
-    worksheet.append([None, "萝卜", 9])
-    worksheet.merge_cells("A2:A3")
-    worksheet.merge_cells("A4:A5")
-    output = io.BytesIO()
-    workbook.save(output)
-    workbook.close()
-
-    parsed = parse_text_content(output.getvalue(), filename="inventory.xlsx")
-    assert parsed.records == (
-        {"类别": "水果", "名称": "苹果", "数量": 10},
-        {"类别": "", "名称": "香蕉", "数量": 12},
-        {"类别": "蔬菜", "名称": "白菜", "数量": 8},
-        {"类别": "", "名称": "萝卜", "数量": 9},
-    )
-
-
-def test_xlsx_header_inference_supports_title_and_two_header_levels() -> None:
-    workbook = Workbook()
-    worksheet = workbook.active
-    worksheet.merge_cells("A1:C1")
-    worksheet["A1"] = "区域销售报表"
-    worksheet["A2"] = "统计日期"
-    worksheet["B2"] = "2026-07-23"
-    worksheet.merge_cells("A4:A5")
-    worksheet.merge_cells("B4:C4")
-    worksheet["A4"] = "地区"
-    worksheet["B4"] = "销售"
-    worksheet["B5"] = "Q1"
-    worksheet["C5"] = "Q2"
-    worksheet.append(["华南", 88, 92])
-    output = io.BytesIO()
-    workbook.save(output)
-    workbook.close()
-
-    parsed = parse_text_content(output.getvalue(), filename="two-level.xlsx")
-    assert parsed.records == (
-        {"地区": "华南", "销售.Q1": 88, "销售.Q2": 92},
-    )
-
-
-def test_xlsx_header_inference_supports_title_and_three_header_levels() -> None:
-    workbook = Workbook()
-    worksheet = workbook.active
-    worksheet.merge_cells("A1:D1")
-    worksheet["A1"] = "年度销售分析报告"
-    worksheet["A2"] = "统计日期"
-    worksheet["B2"] = "2026-07-23"
-    worksheet.merge_cells("A4:A6")
-    worksheet.merge_cells("B4:D4")
-    worksheet.merge_cells("B5:C5")
-    worksheet.merge_cells("D5:D6")
-    worksheet["A4"] = "地区"
-    worksheet["B4"] = "销售"
-    worksheet["B5"] = "国内"
-    worksheet["D5"] = "海外"
-    worksheet["B6"] = "Q1"
-    worksheet["C6"] = "Q2"
-    worksheet.append(["华东", 100, 120, 80])
-    output = io.BytesIO()
-    workbook.save(output)
-    workbook.close()
-
-    parsed = parse_text_content(output.getvalue(), filename="three-level.xlsx")
-    assert parsed.records == (
-        {
-            "地区": "华东",
-            "销售.国内.Q1": 100,
-            "销售.国内.Q2": 120,
-            "销售.海外": 80,
-        },
-    )
-
-
-def test_xlsx_header_inference_keeps_an_ordinary_single_header_row() -> None:
-    parsed = parse_text_content(_xlsx_bytes(), filename="ordinary.xlsx")
-    assert tuple(parsed.records[0]) == ("name", "score", "created_at")
-    assert len(parsed.records) == 2
-
-
-@pytest.mark.parametrize(
-    "target",
-    [
-        "./worksheets/../worksheets/sheet1.xml",
-        "./worksheets/%2e%2e/worksheets/sheet1.xml",
-        "../xl/worksheets/sheet1.xml",
-        "/xl/worksheets/./sheet1.xml",
-    ],
-)
-def test_xlsx_worksheet_relationship_allows_safe_dot_segments(target: str) -> None:
-    raw = _xlsx_with_worksheet_relationship(_xlsx_bytes(), target)
-    parsed = parse_text_content(raw, filename="records.xlsx")
-    assert parsed.records[0]["name"] == "Alice"
-
-
-@pytest.mark.parametrize(
-    "target",
-    [
-        "../../outside.xml",
-        "worksheets\\sheet1.xml",
-        "%2e%2e/%2e%2e/outside.xml",
-        "%252e%252e/%252e%252e/outside.xml",
-        "https://example.com/sheet1.xml",
-    ],
-)
-def test_xlsx_worksheet_relationship_rejects_path_traversal(target: str) -> None:
-    raw = _xlsx_with_worksheet_relationship(_xlsx_bytes(), target)
-    with pytest.raises(ValueError, match="unsafe worksheet path"):
-        parse_text_content(raw, filename="unsafe.xlsx")
-
-
-def test_xlsx_worksheet_relationship_rejects_external_and_missing_targets() -> None:
-    external = _xlsx_with_worksheet_relationship(
-        _xlsx_bytes(),
-        "https://example.com/sheet1.xml",
-        target_mode="External",
-    )
-    with pytest.raises(ValueError, match="external relationship"):
-        parse_text_content(external, filename="external.xlsx")
-
-    missing = _xlsx_with_worksheet_relationship(
-        _xlsx_bytes(),
-        "worksheets/missing.xml",
-    )
-    with pytest.raises(ValueError, match="target does not exist"):
-        parse_text_content(missing, filename="missing.xlsx")
-
-
-@pytest.mark.parametrize(
-    ("filename", "replacement"),
-    [
-        ("legacy.doc", ".docx"),
-        ("legacy.xls", ".xlsx"),
-        ("legacy.ppt", ".pptx"),
-    ],
-)
-def test_legacy_office_formats_require_conversion(filename: str, replacement: str) -> None:
-    with pytest.raises(ValueError, match=rf"convert the file to \{replacement}"):
-        parse_text_content(b"legacy", filename=filename)
-
-
-def test_office_zip_bomb_and_invalid_pdf_are_rejected_before_parsing() -> None:
-    archive = io.BytesIO()
-    with zipfile.ZipFile(archive, "w", compression=zipfile.ZIP_DEFLATED) as package:
-        package.writestr("[Content_Types].xml", "<Types/>")
-        package.writestr("word/document.xml", b"A" * (2 * 1024 * 1024))
-    with pytest.raises(ValueError, match="unsafe compression ratio"):
-        parse_text_content(archive.getvalue(), filename="unsafe.docx")
-
-    active_xml = io.BytesIO()
-    with zipfile.ZipFile(active_xml, "w") as package:
-        package.writestr("[Content_Types].xml", "<Types/>")
-        package.writestr(
-            "word/document.xml",
-            '<!DOCTYPE document [<!ENTITY xxe SYSTEM "file:///etc/passwd">]><document/>',
-        )
-    with pytest.raises(ValueError, match="unsupported active XML"):
-        parse_text_content(active_xml.getvalue(), filename="active.docx")
-
-    with pytest.raises(ValueError, match="missing PDF header"):
-        parse_text_content(b"not a pdf", filename="broken.pdf")
-
-    blank_pdf = io.BytesIO()
-    blank_writer = PdfWriter()
-    blank_writer.add_blank_page(width=612, height=792)
-    blank_writer.write(blank_pdf)
-    with pytest.raises(ValueError, match="scanned PDF requires OCR"):
-        parse_text_content(blank_pdf.getvalue(), filename="scanned.pdf")
-
-    aes_pdf_without_open_password = parse_text_content(
-        _aes_encrypted_pdf(user_password=""),
-        filename="aes-no-password.pdf",
-    )
-    assert "Hello PDF" in aes_pdf_without_open_password.text
-
-    with pytest.raises(ValueError, match="password-protected PDF files are not supported"):
-        parse_text_content(
-            _aes_encrypted_pdf(user_password="secret"),
-            filename="aes-password.pdf",
-        )
-
-
-def test_invalid_utf8_and_malformed_structured_content_fail_loudly() -> None:
-    with pytest.raises(ValueError, match="not valid UTF-8"):
-        parse_text_content(b"\xff\xfe", filename="broken.txt")
-    with pytest.raises(ValueError, match="invalid JSONL at line 2"):
-        extract_structured_records('{"id":1}\nnot-json', "jsonl")
-    with pytest.raises(ValueError, match="more fields"):
-        extract_structured_records("a,b\n1,2,3", "csv")
-
-
-def test_detect_format_from_content_and_normalize() -> None:
-    assert detect_text_format(text='{"id":1}\n{"id":2}') == "jsonl"
-    assert detect_text_format(text="# Heading\ntext") == "markdown"
-    assert detect_text_format(text="a,b\n1,2") == "csv"
-    assert normalize_text("\ufeffＡＢＣ  \r\n第二\x00行\u200b\t \r\n") == "ABC\n第二行"
-
-
-def test_extract_json_scalar_and_nested_values_are_stable() -> None:
-    assert extract_structured_records("[1, true, null]", "json") == [
-        {"value": 1},
-        {"value": True},
-        {"value": None},
-    ]
-    result = extract_structured_records(
-        json.dumps({"items": [{"text": "  内容  "}], "ignored": 1}, ensure_ascii=False),
-        "json",
-    )
-    assert result == [{"text": "内容"}]
-
-
-def test_desensitize_pii_returns_masked_text_and_counts() -> None:
-    source = "邮箱 a.user+tag@example.com，手机 +86 13800138000，身份证 11010519491231002X。"
-    masked, counts = desensitize_pii(source)
-    assert masked == "邮箱 [EMAIL]，手机 [PHONE]，身份证 [ID_CARD]。"
-    assert counts == {"email": 1, "phone": 1, "id_card": 1, "total": 3}
-
-
-def test_every_structured_preprocess_option_has_independent_behavior() -> None:
-    clean_source = [
-        {"id": "1", "name": "有效", "empty_column": ""},
-        {"id": "", "name": "缺少关键字段", "empty_column": ""},
-        {"id": "2", "name": "有效", "empty_column": ""},
-    ]
-    assert preprocess_structured_records(clean_source, []) == clean_source
-    assert preprocess_structured_records(clean_source, ["clean_invalid"]) == [
-        {"id": "1", "name": "有效"},
-        {"id": "2", "name": "有效"},
-    ]
-
-    nested = [{"id": 1, "profile": {"name": "张三", "level": 2}}]
-    assert "profile" in preprocess_structured_records(nested, [])[0]
-    assert preprocess_structured_records(nested, ["detect_structure"])[0] == {
-        "id": 1,
-        "profile.name": "张三",
-        "profile.level": 2,
-    }
-
-    duplicates = [
-        {"customer_id": "C-1", "value": "first"},
-        {"customer_id": "C-1", "value": "updated"},
-        {"customer_id": "", "value": "blank-one"},
-        {"customer_id": "", "value": "blank-two"},
-    ]
-    assert len(preprocess_structured_records(duplicates, [])) == 4
-    deduplicated = preprocess_structured_records(duplicates, ["deduplicate"])
-    assert [record["value"] for record in deduplicated] == [
-        "first",
-        "blank-one",
-        "blank-two",
-    ]
-
-    unnormalized = [{" User Name ": "ＡＢＣ\r\n第二行"}]
-    assert preprocess_structured_records(unnormalized, []) == unnormalized
-    assert preprocess_structured_records(unnormalized, ["normalize_format"]) == [
-        {"user_name": "ABC\n第二行"}
-    ]
-
-    anomaly_source = [
-        {"id": 10_000 + index, "amount": amount, "text": "正常内容"}
-        for index, amount in enumerate((10, 10, 11, 11, 12, 12, 13, 1000))
-    ]
-    assert len(preprocess_structured_records(anomaly_source, [])) == 8
-    filtered = preprocess_structured_records(anomaly_source, ["filter_anomaly"])
-    assert len(filtered) == 7
-    assert all(record["amount"] != 1000 for record in filtered)
-    assert max(record["id"] for record in filtered) > 10_000
-
-    sensitive = [{"姓名": "张三", "phone": "13800138000", "email": "a@b.com"}]
-    assert preprocess_structured_records(sensitive, []) == sensitive
-    masked = preprocess_structured_records(sensitive, ["desensitize"])[0]
-    assert masked == {"姓名": "[NAME]", "phone": "[PHONE]", "email": "[EMAIL]"}
-
-
-def test_structured_desensitization_counts_and_document_helpers() -> None:
-    masked, counts = desensitize_structured_record(
-        {"联系人姓名": "李四", "说明": "邮箱 user@example.com，手机 13900139000"}
-    )
-    assert masked == {
-        "联系人姓名": "[NAME]",
-        "说明": "邮箱 [EMAIL]，手机 [PHONE]",
-    }
-    assert counts == {"email": 1, "phone": 1, "id_card": 0, "name": 1, "total": 3}
-
-    structure = detect_document_structure(
-        "# 第一章\n正文\n\n## 细节\n- 项目一\n- 项目二\n\n```python\nprint(1)\n```"
-    )
-    assert [heading.title for heading in structure.headings] == ["第一章", "细节"]
-    assert structure.list_block_count == 1
-    assert structure.code_block_count == 1
-    assert merge_short_blocks(["短一", "短二", "这是一段足够长的正文内容"], min_token_count=4)
-    assert "mojibake" in content_quality_flags("正常文字锟斤拷内容", min_chars=0, min_tokens=0)
-    assert is_near_duplicate(
-        "alpha beta gamma delta epsilon zeta eta theta iota kappa lambda mu nu xi omicron",
-        "alpha beta gamma, delta epsilon zeta eta theta iota kappa lambda mu nu xi omicron",
-        similarity_threshold=0.92,
-        max_hamming_distance=2,
-    )
-
-
-@pytest.mark.parametrize("method", ["structure", "fixed", "custom"])
-def test_chunk_methods_preserve_offsets_and_always_advance(method: str) -> None:
-    text = "# 第一章\n" + "甲。" * 18 + "\n# 第二章\n" + "乙。" * 18
-    kwargs = {"custom_delimiter": "\\n"} if method == "custom" else {}
-    chunks = chunk_unstructured(
-        text,
-        method=method,  # type: ignore[arg-type]
-        chunk_size=12,
-        chunk_overlap=2,
-        min_chunk_size=4,
-        **kwargs,
-    )
-    assert len(chunks) > 1
-    assert all(chunk.content == normalize_text(text)[chunk.start : chunk.end] for chunk in chunks)
-    assert all(chunk.end > chunk.start for chunk in chunks)
-    assert all(left.start < right.start for left, right in zip(chunks, chunks[1:]))
-    assert all(chunk.start_line <= chunk.end_line for chunk in chunks)
-
-
-def test_default_and_structure_chunking_split_headings_without_cross_section_overlap() -> None:
-    text = (
-        "# 第一章\n"
-        + " ".join(f"alpha{i}" for i in range(18))
-        + "\n# 第二章\n"
-        + " ".join(f"beta{i}" for i in range(18))
-    )
-    normalized = normalize_text(text)
-    second_chapter_start = normalized.index("# 第二章")
-    kwargs = {"chunk_size": 10, "chunk_overlap": 3, "min_chunk_size": 4}
-
-    default_chunks = chunk_unstructured(text, **kwargs)
-    structure_chunks = chunk_unstructured(text, method="structure", **kwargs)
-
-    assert default_chunks == structure_chunks
-    assert len(structure_chunks) > 2
-    assert all(
-        chunk.content == normalized[chunk.start : chunk.end] for chunk in structure_chunks
-    )
-    assert all(
-        not (chunk.start < second_chapter_start < chunk.end) for chunk in structure_chunks
-    )
-    second_chapter_chunks = [
-        chunk for chunk in structure_chunks if chunk.start >= second_chapter_start
-    ]
-    assert second_chapter_chunks[0].start == second_chapter_start
-    assert second_chapter_chunks[0].content.startswith("# 第二章")
-
-
-def test_fixed_chunk_offsets_and_actual_token_overlap_are_exact() -> None:
-    text = "  ".join(f"token{i}" for i in range(30))
-    normalized = normalize_text(text)
-    chunks = chunk_unstructured(
-        text,
-        method="fixed",
-        chunk_size=10,
-        chunk_overlap=3,
-        min_chunk_size=4,
-    )
-    assert len(chunks) > 2
-    assert all(chunk.content == normalized[chunk.start : chunk.end] for chunk in chunks)
-    assert all(chunk.token_count == estimate_token_count(chunk.content) for chunk in chunks)
-    assert all(chunk.token_count == 10 for chunk in chunks[:-1])
-    for left, right in zip(chunks, chunks[1:]):
-        overlap_text = normalized[right.start : left.end]
-        assert right.start < left.end
-        assert estimate_token_count(overlap_text) == 3
-        assert left.content.endswith(overlap_text)
-        assert right.content.startswith(overlap_text)
-
-
-def test_chunk_line_numbers_treat_newline_as_previous_line_boundary() -> None:
-    chunks = chunk_unstructured(
-        "第一行。\n第二行。\n第三行。",
-        method="custom",
-        chunk_size=8,
-        chunk_overlap=0,
-        min_chunk_size=2,
-        custom_delimiter="\\n",
-    )
-    assert chunks[0].content.endswith("\n")
-    assert chunks[0].start_line == 1
-    assert chunks[0].end_line == 1
-    assert chunks[1].start_line == 2
-
-
-def test_custom_delimiter_is_preserved_as_the_chunk_boundary() -> None:
-    custom_chunks = chunk_unstructured(
-        "a b c d <CUT> e f g h i j",
-        method="custom",
-        chunk_size=8,
-        chunk_overlap=0,
-        min_chunk_size=2,
-        custom_delimiter="<CUT>",
-    )
-    assert custom_chunks[0].content.endswith("<CUT>")
-
-
-@pytest.mark.parametrize(
-    ("field", "block"),
-    [
-        (
-            "preserve_code_blocks",
-            "```python\n" + "\n".join(f"value_{i} = {i}" for i in range(30)) + "\n```",
-        ),
-        (
-            "preserve_tables",
-            "| 字段 | 说明 |\n| --- | --- |\n"
-            + "\n".join(f"| field_{i} | value_{i} |" for i in range(30)),
-        ),
-        (
-            "preserve_lists",
-            "\n".join(f"- 第 {i} 项需要完整保留" for i in range(30)),
-        ),
-    ],
-)
-def test_markdown_protected_blocks_are_not_split(field: str, block: str) -> None:
-    text = "前言。" * 15 + "\n" + block + "\n" + "结尾。" * 40
-    unprotected = chunk_unstructured(
-        text,
-        method="fixed",
-        chunk_size=40,
-        chunk_overlap=0,
-        min_chunk_size=10,
-    )
-    chunks = chunk_unstructured(
-        text,
-        method="fixed",
-        chunk_size=40,
-        chunk_overlap=0,
-        min_chunk_size=10,
-        **{field: True},
-    )
-    assert all(block not in chunk.content for chunk in unprotected)
-    assert any(block in chunk.content for chunk in chunks)
-
-
-@pytest.mark.parametrize(
-    ("kwargs", "message"),
-    [
-        ({"chunk_size": 0}, "chunk_size"),
-        ({"chunk_size": 10, "chunk_overlap": 10}, "chunk_overlap"),
-        ({"chunk_size": 10, "chunk_overlap": 0, "min_chunk_size": 11}, "min_chunk_size"),
-        (
-            {"chunk_size": 10, "chunk_overlap": 5, "min_chunk_size": 6},
-            "cannot exceed",
-        ),
-        ({"method": "custom", "custom_delimiter": ""}, "custom_delimiter"),
-        ({"method": "semantic"}, "unsupported chunk method"),
-        ({"method": "heading"}, "unsupported chunk method"),
-    ],
-)
-def test_chunk_configuration_validation(kwargs: dict[str, object], message: str) -> None:
-    with pytest.raises(ValueError, match=message):
-        chunk_unstructured("some text", **kwargs)  # type: ignore[arg-type]
-
-
-def test_quality_scoring_covers_all_dimensions_and_duplicates() -> None:
-    valid = {
-        "instruction": "如何修改收货地址？",
-        "input": "订单尚未发货",
-        "output": "可以在订单详情页申请修改收货地址。",
-    }
-    source = "订单尚未发货时，可以在订单详情页申请修改收货地址。"
-    first_score = score_quality(valid, min_output_length=10, source_content=source)
-    assert first_score.is_valid
-    assert first_score.completeness == 100
-    assert first_score.length == 100
-    assert first_score.readability >= 90
-    assert first_score.relevance >= 70
-    assert first_score.duplicate == 100
-
-    duplicate_score = score_quality(valid, known_fingerprints={first_score.fingerprint})
-    assert duplicate_score.duplicate == 0
-    assert "duplicate_record" in duplicate_score.flags
-
-    unrelated_score = score_quality(
-        valid,
-        min_output_length=10,
-        source_content="量子计算使用量子比特处理信息。",
-    )
-    assert unrelated_score.relevance < first_score.relevance
-    assert "low_source_relevance" in unrelated_score.flags
-
-    invalid_score = score_quality({"instruction": "", "output": "短"}, min_output_length=10)
-    assert not invalid_score.is_valid
-    assert {"missing_instruction", "output_too_short"}.issubset(invalid_score.flags)
-    assert record_fingerprint(valid) == record_fingerprint(dict(reversed(list(valid.items()))))
-
-
-def test_stable_split_is_reproducible_and_validates_ratios() -> None:
-    first = stable_split("record-42", seed="task-1")
-    assert stable_split("record-42", seed="task-1") == first
-    assert first in {"train", "validation", "test"}
-    assert stable_split("record-42", {"train": 100, "validation": 0, "test": 0}) == "train"
-    with pytest.raises(ValueError, match="sum to 100"):
-        stable_split("record", {"train": 80, "validation": 10, "test": 9})
-
-
-def test_generate_standard_records_supports_json_qa_and_stable_variants() -> None:
-    previews = [
-        {
-            "id": "preview-json",
-            "edited_content": json.dumps(
-                {"instruction": "问题", "input": "上下文", "output": "答案"},
-                ensure_ascii=False,
-            ),
-        },
-        {"id": "preview-qa", "editedContent": "问：如何操作？\n答：按步骤操作。"},
-    ]
-    records = generate_standard_records(
-        previews,
-        qa_pairs_per_item=2,
-        semantic_enrichment=True,
-        split={"train": 100, "validation": 0, "test": 0},
-        split_seed="task-1",
-    )
-    assert len(records) == 4
-    assert records[0]["instruction"] == "问题"
-    assert records[0]["input"] == "上下文"
-    assert records[0]["output"] == "答案"
-    assert records[1]["instruction"].endswith("问题")
-    assert records[2]["instruction"] == "如何操作?"
-    assert records[2]["output"] == "按步骤操作。"
-    assert all(record["status"] == "valid" for record in records)
-    assert all(record["split"] == "train" for record in records)
-    assert records == generate_standard_records(
-        previews,
-        qa_pairs_per_item=2,
-        semantic_enrichment=True,
-        split={"train": 100, "validation": 0, "test": 0},
-        split_seed="task-1",
-    )

backend/tests/test_data_process_generation.py

@@ -1,102 +0,0 @@
-from __future__ import annotations
-
-import json
-
-import httpx
-
-from app.modules.data_process.generation import chat_completions_url, generate_model_records
-
-
-def test_chat_completions_url_accepts_host_base_and_complete_url() -> None:
-    assert chat_completions_url("www.caoxiaozhu.com") == (
-        "https://www.caoxiaozhu.com/v1/chat/completions"
-    )
-    assert chat_completions_url("https://model.example/v1") == (
-        "https://model.example/v1/chat/completions"
-    )
-    complete = "https://model.example/openai/v1/chat/completions"
-    assert chat_completions_url(complete) == complete
-
-
-def test_generate_model_records_uses_prompt_auth_and_stable_split() -> None:
-    requests: list[httpx.Request] = []
-    progress_updates: list[tuple[int, int]] = []
-
-    def handler(request: httpx.Request) -> httpx.Response:
-        requests.append(request)
-        payload = json.loads(request.content)
-        assert payload["model"] == "qwen-plus"
-        assert payload["response_format"] == {"type": "json_object"}
-        assert "客户反馈页面加载慢" in payload["messages"][1]["content"]
-        return httpx.Response(
-            200,
-            json={
-                "choices": [
-                    {
-                        "message": {
-                            "content": json.dumps(
-                                {
-                                    "items": [
-                                        {
-                                            "instruction": "请生成简洁客服回复",
-                                            "input": "客户反馈页面加载慢",
-                                            "output": "已收到反馈，我们正在排查。",
-                                        }
-                                    ]
-                                },
-                                ensure_ascii=False,
-                            )
-                        }
-                    }
-                ]
-            },
-        )
-
-    client = httpx.Client(transport=httpx.MockTransport(handler))
-    records = generate_model_records(
-        [{"id": "preview-1", "edited_content": "客户反馈页面加载慢"}],
-        model={
-            "name": "Qwen",
-            "online_model_name": "qwen-plus",
-            "api_url": "model.example",
-            "api_key": "test-secret",
-        },
-        config={
-            "generation_prompt": "请处理：{{ content }}",
-            "json_mode": True,
-            "temperature": 0.2,
-            "max_tokens": 512,
-        },
-        task_id="task-1",
-        split={"train": 100, "validation": 0, "test": 0},
-        qa_pairs_per_item=1,
-        client=client,
-        on_progress=lambda processed, total: progress_updates.append((processed, total)),
-    )
-
-    assert len(records) == 1
-    assert records[0]["status"] == "valid"
-    assert records[0]["split"] == "train"
-    assert requests[0].headers["Authorization"] == "Bearer test-secret"
-    assert progress_updates == [(1, 1)]
-
-
-def test_generate_model_records_keeps_partial_failure_for_manual_repair() -> None:
-    client = httpx.Client(
-        transport=httpx.MockTransport(
-            lambda _: httpx.Response(200, json={"choices": [{"message": {"content": "not-json"}}]})
-        )
-    )
-    records = generate_model_records(
-        [{"id": "preview-1", "edited_content": "来源正文"}],
-        model={"name": "model", "api_url": "https://model.example/v1"},
-        config={"generation_retries": 1},
-        task_id="task-1",
-        split={"train": 80, "validation": 10, "test": 10},
-        qa_pairs_per_item=1,
-        client=client,
-    )
-
-    assert len(records) == 1
-    assert records[0]["status"] == "invalid"
-    assert records[0]["error"]

backend/tests/test_data_process_migration.py

@@ -1,29 +0,0 @@
-from __future__ import annotations
-
-from pathlib import Path
-
-from app.modules.data_process.schema_cli import _target_label
-
-
-def test_runtime_migration_fails_fast_on_incompatible_schema() -> None:
-    sql_path = (
-        Path(__file__).resolve().parents[1]
-        / "app"
-        / "db"
-        / "sql"
-        / "002_data_process.sql"
-    )
-    sql = sql_path.read_text(encoding="utf-8")
-
-    assert "requires 001_platform_runtime.sql first" in sql
-    assert "supports only the current TEXT runtime schema" in sql
-    assert "generation_run_id" in sql
-    assert "CREATE TABLE IF NOT EXISTS data_process_results" in sql
-    assert sql.count("BEGIN;") == 1
-    assert sql.rstrip().endswith("COMMIT;")
-
-
-def test_schema_cli_target_label_never_contains_credentials() -> None:
-    label = _target_label("postgresql://secret-user:secret-password@db.example:5433/yg_ft")
-    assert label == "db.example:5433/yg_ft"
-    assert "secret" not in label

backend/tests/test_data_process_storage.py

@@ -1,242 +0,0 @@
-from __future__ import annotations
-
-from pathlib import Path, PurePosixPath
-
-import pytest
-
-from app.modules.data_process import storage as storage_module
-from app.modules.data_process.storage import (
-    DataProcessStorageError,
-    LocalDataProcessStorage,
-    StagedSourceObject,
-)
-
-
-def _stage(
-    storage: LocalDataProcessStorage,
-    *,
-    batch_id: str = "batch-main",
-    task_id: str = "task-1",
-    source_file_id: str = "source-1",
-    version: int = 1,
-    name: str = "source.txt",
-    content: bytes = b"payload",
-) -> StagedSourceObject:
-    return storage.stage_bytes(
-        batch_id=batch_id,
-        task_id=task_id,
-        source_file_id=source_file_id,
-        version=version,
-        name=name,
-        content=content,
-    )
-
-
-def _create_symlink(link: Path, target: Path, *, target_is_directory: bool = False) -> None:
-    try:
-        link.symlink_to(target, target_is_directory=target_is_directory)
-    except (NotImplementedError, OSError) as exc:
-        pytest.skip(f"当前平台不支持创建测试所需的符号链接: {exc}")
-
-
-def _assert_staging_empty(storage: LocalDataProcessStorage) -> None:
-    assert list((storage.root / ".staging").iterdir()) == []
-
-
-def test_stage_publish_read_delete_roundtrip_with_unicode_filename(tmp_path: Path) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-    content = "第一行\n第二行，100% 完成".encode()
-
-    staged = _stage(
-        storage,
-        name="中文 数据 100%.csv",
-        content=content,
-    )
-
-    assert "%20" in staged.reference
-    assert "%25" in staged.reference
-    storage.publish([staged])
-
-    assert storage.read(staged.reference) == content
-    assert storage.delete(staged.reference) is True
-    assert storage.delete(staged.reference) is False
-    _assert_staging_empty(storage)
-
-
-def test_db_reference_is_left_to_database_storage(tmp_path: Path) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-
-    assert storage.read("db://source-files/source-1") is None
-    assert storage.delete("db://source-files/source-1") is False
-
-
-def test_owned_source_can_be_streamed_by_byte_range(tmp_path: Path) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-    content = b"0123456789abcdef"
-    staged = _stage(storage, content=content)
-    storage.publish([staged])
-
-    assert storage.file_size(
-        staged.reference,
-        expected_task_id="task-1",
-        expected_source_file_id="source-1",
-    ) == len(content)
-    assert b"".join(storage.iter_bytes(
-        staged.reference,
-        expected_task_id="task-1",
-        expected_source_file_id="source-1",
-        expected_size=len(content),
-        start=4,
-        length=6,
-        chunk_size=2,
-    )) == b"456789"
-
-    with pytest.raises(DataProcessStorageError, match="owner mismatch"):
-        storage.file_size(
-            staged.reference,
-            expected_task_id="another-task",
-            expected_source_file_id="source-1",
-        )
-    with pytest.raises(DataProcessStorageError, match="does not match metadata"):
-        b"".join(storage.iter_bytes(
-            staged.reference,
-            expected_task_id="task-1",
-            expected_source_file_id="source-1",
-            expected_size=len(content) + 1,
-        ))
-
-
-@pytest.mark.parametrize(
-    "reference",
-    [
-        "local://data-process/../source-1/v1/file.txt",
-        "local://data-process/task-1/source-1/v1/file%2Fname.txt",
-        "local://data-process/task-1/source-1/v1/file.txt?download=1",
-        "local://data-process/task-1/source-1/v1/file.txt#fragment",
-        "https://data-process/task-1/source-1/v1/file.txt",
-    ],
-    ids=[
-        "parent-traversal",
-        "percent-encoded-slash",
-        "query",
-        "fragment",
-        "wrong-scheme",
-    ],
-)
-def test_unsafe_references_are_rejected(tmp_path: Path, reference: str) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-
-    with pytest.raises(DataProcessStorageError):
-        storage.read(reference)
-    with pytest.raises(DataProcessStorageError):
-        storage.delete(reference)
-
-
-def test_publish_rejects_intermediate_directory_symlink(tmp_path: Path) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-    outside = tmp_path / "outside"
-    outside.mkdir()
-    staged = _stage(storage, task_id="linked-task")
-    _create_symlink(
-        storage.root / "linked-task",
-        outside,
-        target_is_directory=True,
-    )
-
-    with pytest.raises(DataProcessStorageError, match="symlink|non-directory"):
-        storage.publish([staged])
-
-    assert list(outside.iterdir()) == []
-    _assert_staging_empty(storage)
-
-
-def test_target_symlink_is_never_followed_or_deleted(tmp_path: Path) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-    staged = _stage(storage, task_id="task-link", source_file_id="source-link")
-    outside_file = tmp_path / "outside.txt"
-    outside_file.write_bytes(b"outside sentinel")
-    final_path = storage.root.joinpath(*staged._relative_path.parts)
-    final_path.parent.mkdir(parents=True)
-    _create_symlink(final_path, outside_file)
-
-    with pytest.raises(DataProcessStorageError, match="already exists"):
-        storage.publish([staged])
-    with pytest.raises(DataProcessStorageError, match="regular file"):
-        storage.read(staged.reference)
-    with pytest.raises(DataProcessStorageError, match="non-regular"):
-        storage.delete(staged.reference)
-
-    assert final_path.is_symlink()
-    assert outside_file.read_bytes() == b"outside sentinel"
-    _assert_staging_empty(storage)
-
-
-def test_publish_rolls_back_first_object_when_second_target_collides(tmp_path: Path) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-    existing = _stage(
-        storage,
-        batch_id="batch-existing",
-        source_file_id="source-existing",
-        content=b"existing content",
-    )
-    storage.publish([existing])
-
-    first = _stage(
-        storage,
-        batch_id="batch-new",
-        source_file_id="source-new",
-        content=b"must be rolled back",
-    )
-    colliding_second = _stage(
-        storage,
-        batch_id="batch-new",
-        source_file_id="source-existing",
-        content=b"must not replace existing content",
-    )
-
-    with pytest.raises(DataProcessStorageError, match="already exists"):
-        storage.publish([first, colliding_second])
-
-    with pytest.raises(DataProcessStorageError, match="does not exist"):
-        storage.read(first.reference)
-    assert storage.read(existing.reference) == b"existing content"
-    _assert_staging_empty(storage)
-
-
-def test_publish_rejects_manually_forged_staged_object(tmp_path: Path) -> None:
-    storage = LocalDataProcessStorage(tmp_path / "storage")
-    temporary_path = storage.root / ".staging" / "batch-forged" / "forged.tmp"
-    temporary_path.parent.mkdir()
-    temporary_path.write_bytes(b"forged content")
-    relative_path = PurePosixPath("task-forged", "source-forged", "v1", "forged.txt")
-    forged = StagedSourceObject(
-        reference="local://data-process/task-forged/source-forged/v1/forged.txt",
-        _temporary_path=temporary_path,
-        _relative_path=relative_path,
-    )
-
-    with pytest.raises(DataProcessStorageError, match="was not issued"):
-        storage.publish([forged])
-    with pytest.raises(DataProcessStorageError, match="was not issued"):
-        storage.discard([forged])
-
-    assert temporary_path.read_bytes() == b"forged content"
-    assert not storage.root.joinpath(*relative_path.parts).exists()
-
-
-def test_relative_storage_configuration_is_anchored_to_backend_root(
-    tmp_path: Path,
-    monkeypatch: pytest.MonkeyPatch,
-) -> None:
-    relative_configuration = Path("relative-storage") / tmp_path.name
-    backend_root = Path(storage_module.__file__).resolve().parents[3]
-    monkeypatch.chdir(tmp_path)
-    monkeypatch.setenv("DATA_PROCESS_STORAGE_DIR", str(relative_configuration))
-    storage_module.get_data_process_storage.cache_clear()
-
-    try:
-        configured_root = storage_module._configured_storage_root()
-        assert configured_root == backend_root / relative_configuration
-        assert not configured_root.exists()
-    finally:
-        storage_module.get_data_process_storage.cache_clear()

backend/tests/test_data_process_store.py

@@ -1,54 +0,0 @@
-from __future__ import annotations
-
-import pytest
-
-from app.modules.data_process.store import (
-    DataProcessStoreError,
-    _source_storage_descriptor,
-)
-
-
-def test_source_storage_descriptor_accepts_owned_local_and_legacy_db_references() -> None:
-    task_id = "dpt_task"
-    source_file_id = "dpsf_source"
-    local_reference = (
-        f"local://data-process/{task_id}/{source_file_id}/v1/source%20100%25.csv"
-    )
-
-    reference, metadata = _source_storage_descriptor(
-        {
-            "storage_object_id": local_reference,
-            "metadata": {"storage_backend": "spoofed", "content_type": "text/csv"},
-        },
-        task_id,
-        source_file_id,
-    )
-    assert reference == local_reference
-    assert metadata == {"storage_backend": "local", "content_type": "text/csv"}
-
-    legacy_reference, legacy_metadata = _source_storage_descriptor(
-        {"metadata": {"legacy": True}},
-        task_id,
-        source_file_id,
-    )
-    assert legacy_reference == f"db://data-process/{task_id}/{source_file_id}/v1"
-    assert legacy_metadata == {"storage_backend": "database", "legacy": True}
-
-
-@pytest.mark.parametrize(
-    "reference",
-    [
-        "local://data-process/dpt_other/dpsf_source/v1/source.txt",
-        "db://data-process/dpt_task/dpsf_other/v1",
-        "/var/tmp/source.txt",
-    ],
-)
-def test_source_storage_descriptor_rejects_unowned_or_unsupported_references(
-    reference: str,
-) -> None:
-    with pytest.raises(DataProcessStoreError):
-        _source_storage_descriptor(
-            {"storage_object_id": reference},
-            "dpt_task",
-            "dpsf_source",
-        )

compute/README.md

@@ -25,57 +25,5 @@ compute/
 
 ## 运行模式
 
-- 默认 `COMPUTE_EXECUTION_MODE=real`，Compute API 会通过 `compute.agent.process_manager.ProcessManager` 启动真实 `llamafactory-cli train` 子进程，并将日志写入 `TRAINING_LOG_ROOT`。
-- 真实模式下 GPU 发现优先使用宿主机 `nvidia-smi`。如果部署环境暂时无法调用 `nvidia-smi`，可通过 `COMPUTE_GPU_COUNT`、`COMPUTE_GPU_NAME`、`COMPUTE_GPU_MEMORY_GB`、`COMPUTE_GPU_POWER_LIMIT_W` 声明兼容 GPU 清单，便于应用侧先完成节点登记和联调。
+- 默认 `COMPUTE_EXECUTION_MODE=real`，Compute API 只暴露健康检查和接口契约；真实训练执行器完成前，创建作业会返回未实现错误。
 - 仅隔离联调时可设置 `COMPUTE_EXECUTION_MODE=simulator`，启用内存状态机和合成 GPU/日志数据。该模式不得作为生产运行路径。
-- 服务间鉴权默认开启：设置 `COMPUTE_AUTH_ENABLED=true` 和一致的 `COMPUTE_SERVICE_TOKEN`，应用侧会通过 `X-Compute-Token` 调用 Compute API。
-- 真实训练作业会登记到 `TRAINING_LOG_ROOT/compute-jobs.json`。Compute API 重启后会恢复作业索引，继续提供状态、停止和日志查询。
-- 同一算力节点内按 GPU ID 做轻量锁定；已有运行中作业占用的 GPU 不允许再次提交，避免同机多 GPU 场景下误复用。
-
-真实执行前提：
-
-- 镜像或宿主机环境中 `llamafactory-cli` 可执行。
-- `LLAMA_FACTORY_HOME` 指向 LLaMA-Factory 工作目录。
-- 基座模型路径和数据集名称/目录已经在算力服务器本地可访问。
-- 应用侧训练任务中的 GPU、模型、数据集配置能映射到当前节点本地路径。
-
-## 应用侧接入
-
-应用平台通过“算力节点”页面维护每台 GPU 服务器的 `Compute API` 和 `File Gateway` 地址。点击连接测试时，Backend API 会主动调用：
-
-```text
-GET /modelTF/v1/compute/health
-GET /modelTF/compute/resources/gpus
-```
-
-连接成功后，应用侧会同步节点健康信息、能力标签和 GPU 清单到 PostgreSQL。多节点阶段仍按“每台算力服务器 = 单机多 GPU 节点”管理，每台服务器都部署 Compute API、Agent、File Gateway 契约和 LLaMA-Factory。
-
-训练闭环：
-
-```text
-Frontend 创建/启动训练
-  -> Backend API 选择 compute_nodes 节点
-  -> Backend API POST /modelTF/compute/jobs 到目标 Compute API
-  -> Compute API 启动 llamafactory-cli 子进程
-  -> Backend Worker 定时 GET /modelTF/compute/jobs/{id}
-  -> Backend API 同步 fine_tune_tasks 状态、进度、PID、日志路径和产物索引
-```
-
-## 当前接口能力
-
-日志接口：
-
-```text
-GET /modelTF/compute/jobs/{job_id}/logs?tail_lines=200
-GET /modelTF/compute/jobs/{job_id}/logs?offset=0&limit=500
-```
-
-返回 `content`、`metrics`、`total_lines`、`offset`、`limit`、`has_more`、`next_offset`，用于前端增量刷新和日志平台采集。
-
-文件导入：
-
-```text
-POST /modelTF/compute/files/import-local
-```
-
-该接口用于应用侧调度前把算力服务器本地可访问的模型/数据集路径导入到 `YG_FT_DATA_ROOT` 内部。目标路径会校验不能逃逸出 `YG_FT_DATA_ROOT`，源路径必须已存在于算力服务器本地或挂载目录。

compute/agent/process_manager.py

@@ -1,281 +0,0 @@
-from __future__ import annotations
-
-import os
-import json
-import contextlib
-import hashlib
-import signal
-import subprocess
-import time
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Any
-
-
-TERMINAL_STATUSES = {"completed", "failed", "stopped"}
-
-
-@dataclass
-class ManagedProcess:
-    id: str
-    name: str
-    command: list[str]
-    work_dir: str
-    log_path: Path
-    output_dir: str
-    gpus: list[int]
-    process: subprocess.Popen[Any] | None
-    created_at: float
-    pid: int | None = None
-    status: str = "running"
-    progress: int = 5
-    artifacts: list[dict[str, Any]] = field(default_factory=list)
-
-
-class ProcessManager:
-    def __init__(self, log_root: str) -> None:
-        self.log_root = Path(log_root)
-        self.log_root.mkdir(parents=True, exist_ok=True)
-        self.registry_path = self.log_root / "compute-jobs.json"
-        self.jobs: dict[str, ManagedProcess] = {}
-        self._load_registry()
-
-    def create_job(self, payload: dict[str, Any], command: list[str], work_dir: str) -> dict[str, Any]:
-        job_id = str(payload.get("id") or f"job_{int(time.time() * 1000)}")
-        if job_id in self.jobs and self.jobs[job_id].status not in TERMINAL_STATUSES:
-            raise ValueError(f"job {job_id} is already running")
-
-        output_dir = str(payload.get("output_dir") or f"/data/yg-ft/outputs/{payload.get('name', job_id)}")
-        Path(output_dir).mkdir(parents=True, exist_ok=True)
-        log_path = self.log_root / f"{job_id}.log"
-        env = os.environ.copy()
-        gpus = [int(item) for item in payload.get("gpus") or []]
-        locked = self.locked_gpus()
-        conflict = sorted(set(gpus).intersection(locked))
-        if conflict:
-            raise ValueError(f"gpu already locked: {conflict}")
-        if gpus:
-            env["CUDA_VISIBLE_DEVICES"] = ",".join(str(item) for item in gpus)
-        env.update({str(k): str(v) for k, v in payload.get("env", {}).items()})
-
-        cwd = work_dir if Path(work_dir).exists() else None
-        with log_path.open("ab") as log_file:
-            log_file.write(f"[INFO] starting job_id={job_id} command={' '.join(command)}\n".encode("utf-8"))
-            process = subprocess.Popen(
-                command,
-                cwd=cwd,
-                env=env,
-                stdout=log_file,
-                stderr=subprocess.STDOUT,
-            )
-
-        managed = ManagedProcess(
-            id=job_id,
-            name=str(payload.get("name") or job_id),
-            command=command,
-            work_dir=work_dir,
-            log_path=log_path,
-            output_dir=output_dir,
-            gpus=gpus,
-            process=process,
-            created_at=time.time(),
-            pid=process.pid,
-            progress=10,
-        )
-        self.jobs[job_id] = managed
-        data = self.serialize(managed)
-        self._save_registry()
-        return data
-
-    def get_job(self, job_id: str) -> dict[str, Any] | None:
-        job = self.jobs.get(job_id)
-        if not job:
-            return None
-        return self.serialize(job)
-
-    def list_jobs(self) -> list[dict[str, Any]]:
-        return [self.serialize(job) for job in self.jobs.values()]
-
-    def stop_job(self, job_id: str) -> dict[str, Any] | None:
-        job = self.jobs.get(job_id)
-        if not job:
-            return None
-        if job.status not in TERMINAL_STATUSES:
-            try:
-                if job.process is not None and os.name == "nt":
-                    job.process.terminate()
-                elif job.pid is not None:
-                    os.kill(job.pid, signal.SIGTERM)
-                if job.process is not None:
-                    job.process.wait(timeout=10)
-            except Exception:
-                if job.process is not None:
-                    job.process.kill()
-                elif job.pid is not None:
-                    with contextlib.suppress(Exception):
-                        os.kill(job.pid, signal.SIGKILL)
-            job.status = "stopped"
-            job.progress = min(job.progress, 99)
-        data = self.serialize(job)
-        self._save_registry()
-        return data
-
-    def logs(self, job_id: str) -> str:
-        job = self.jobs.get(job_id)
-        if not job or not job.log_path.exists():
-            return ""
-        return job.log_path.read_text(encoding="utf-8", errors="replace")
-
-    def serialize(self, job: ManagedProcess) -> dict[str, Any]:
-        code = job.process.poll() if job.process is not None else None
-        checkpoints = self._collect_checkpoints(job.output_dir)
-        if job.status not in TERMINAL_STATUSES:
-            if job.process is None and job.pid is not None and not self._pid_alive(job.pid):
-                job.status = "failed"
-                job.progress = min(job.progress, 99)
-                code = -1
-            elif code is None:
-                job.status = "running"
-                elapsed = max(0, int(time.time() - job.created_at))
-                job.progress = min(95, max(job.progress, 10 + elapsed // 6))
-            elif code == 0:
-                job.status = "completed"
-                job.progress = 100
-                job.artifacts = self._collect_artifacts(job.output_dir)
-            else:
-                job.status = "failed"
-                job.progress = min(job.progress, 99)
-        self._save_registry()
-        return {
-            "id": job.id,
-            "name": job.name,
-            "status": job.status,
-            "progress": job.progress,
-            "pid": job.pid,
-            "gpus": job.gpus,
-            "created_at": job.created_at,
-            "command": job.command,
-            "work_dir": job.work_dir,
-            "output_dir": job.output_dir,
-            "log_file": str(job.log_path),
-            "artifacts": job.artifacts,
-            "checkpoints": checkpoints,
-            "return_code": code,
-        }
-
-    def locked_gpus(self) -> set[int]:
-        locked: set[int] = set()
-        for job in self.jobs.values():
-            status = self.serialize(job)["status"]
-            if status in {"queued", "running"}:
-                locked.update(job.gpus)
-        return locked
-
-    def _collect_artifacts(self, output_dir: str) -> list[dict[str, Any]]:
-        root = Path(output_dir)
-        if not root.exists():
-            return []
-        artifacts: list[dict[str, Any]] = []
-        for path in root.rglob("*"):
-            if path.is_file():
-                digest = hashlib.sha256()
-                with path.open("rb") as handle:
-                    for chunk in iter(lambda: handle.read(1024 * 1024), b""):
-                        digest.update(chunk)
-                size = path.stat().st_size
-                artifacts.append(
-                    {
-                        "path": str(path),
-                        "name": path.name,
-                        "size": size,
-                        "size_bytes": size,
-                        "checksum_sha256": digest.hexdigest(),
-                    }
-                )
-        return artifacts[:200]
-
-    def _collect_checkpoints(self, output_dir: str) -> list[dict[str, Any]]:
-        root = Path(output_dir)
-        if not root.exists():
-            return []
-        checkpoints: list[dict[str, Any]] = []
-        for path in root.glob("checkpoint-*"):
-            if not path.is_dir():
-                continue
-            step = 0
-            try:
-                step = int(path.name.rsplit("-", 1)[-1])
-            except ValueError:
-                step = 0
-            size_bytes = sum(item.stat().st_size for item in path.rglob("*") if item.is_file())
-            checkpoints.append(
-                {
-                    "step": step,
-                    "name": path.name,
-                    "path": str(path),
-                    "size_bytes": size_bytes,
-                    "create_time": path.stat().st_mtime,
-                }
-            )
-        return sorted(checkpoints, key=lambda item: (int(item.get("step") or 0), str(item.get("name") or "")))
-
-    def _save_registry(self) -> None:
-        items = []
-        for job in self.jobs.values():
-            items.append(
-                {
-                    "id": job.id,
-                    "name": job.name,
-                    "command": job.command,
-                    "work_dir": job.work_dir,
-                    "log_path": str(job.log_path),
-                    "output_dir": job.output_dir,
-                    "gpus": job.gpus,
-                    "pid": job.pid,
-                    "created_at": job.created_at,
-                    "status": job.status,
-                    "progress": job.progress,
-                    "artifacts": job.artifacts,
-                }
-            )
-        self.registry_path.write_text(json.dumps(items, ensure_ascii=False, indent=2), encoding="utf-8")
-
-    def _load_registry(self) -> None:
-        if not self.registry_path.exists():
-            return
-        try:
-            items = json.loads(self.registry_path.read_text(encoding="utf-8"))
-        except json.JSONDecodeError:
-            return
-        for item in items if isinstance(items, list) else []:
-            if not isinstance(item, dict):
-                continue
-            pid = item.get("pid")
-            status = item.get("status", "failed")
-            if status not in TERMINAL_STATUSES and pid and not self._pid_alive(int(pid)):
-                status = "failed"
-            job = ManagedProcess(
-                id=str(item["id"]),
-                name=str(item.get("name") or item["id"]),
-                command=[str(part) for part in item.get("command") or []],
-                work_dir=str(item.get("work_dir") or ""),
-                log_path=Path(item.get("log_path") or self.log_root / f"{item['id']}.log"),
-                output_dir=str(item.get("output_dir") or ""),
-                gpus=[int(gpu) for gpu in item.get("gpus") or []],
-                process=None,
-                pid=int(pid) if pid else None,
-                created_at=float(item.get("created_at") or time.time()),
-                status=status,
-                progress=int(item.get("progress") or 0),
-                artifacts=item.get("artifacts") or [],
-            )
-            self.jobs[job.id] = job
-
-    def _pid_alive(self, pid: int) -> bool:
-        if pid <= 0:
-            return False
-        try:
-            os.kill(pid, 0)
-            return True
-        except OSError:
-            return False

compute/api/main.py

@@ -2,38 +2,19 @@ from __future__ import annotations
 
 import os
 import math
-import hashlib
-import shutil
-import subprocess
 import time
 from pathlib import Path
 from typing import Any
 
-from fastapi import FastAPI, File, Form, HTTPException, Query, Request, UploadFile
-from fastapi.responses import FileResponse, JSONResponse
+from fastapi import FastAPI, HTTPException
 
-from compute.agent.process_manager import ProcessManager
-from compute.engines.llama_factory.adapter import build_command, parse_log_line, prepare_runtime_files
+from compute.engines.llama_factory.adapter import build_command, parse_log_line
 
 
 def create_app() -> FastAPI:
     app = FastAPI(title="YG Fine-Tune Compute API")
     jobs: dict[str, dict[str, Any]] = {}
     route_prefix = os.getenv("MODELTF_ROUTE_PREFIX", "/modelTF").rstrip("/") or "/modelTF"
-    process_manager = ProcessManager(os.getenv("TRAINING_LOG_ROOT", "/opt/yg-ft/logs/training"))
-
-    @app.middleware("http")
-    async def compute_token_auth(request: Request, call_next):
-        token = os.getenv("COMPUTE_SERVICE_TOKEN", "")
-        auth_enabled = os.getenv("COMPUTE_AUTH_ENABLED", "true").lower() == "true"
-        public_paths = {f"{route_prefix}/health", "/health"}
-        if auth_enabled and token and request.url.path not in public_paths:
-            header_token = request.headers.get("x-compute-token", "")
-            auth_header = request.headers.get("authorization", "")
-            bearer_token = auth_header.removeprefix("Bearer ").strip() if auth_header.startswith("Bearer ") else ""
-            if header_token != token and bearer_token != token:
-                return JSONResponse({"detail": "invalid compute service token"}, status_code=401)
-        return await call_next(request)
 
     def now() -> float:
         return time.time()
@@ -44,110 +25,6 @@ def create_app() -> FastAPI:
     def execution_mode() -> str:
         return os.getenv("COMPUTE_EXECUTION_MODE", os.getenv("COMPUTE_MODE", "real")).lower()
 
-    def _int_env(name: str, default: int) -> int:
-        raw = os.getenv(name)
-        if raw is None or raw == "":
-            return default
-        return int(raw)
-
-    def _float_env(name: str, default: float) -> float:
-        raw = os.getenv(name)
-        if raw is None or raw == "":
-            return default
-        return float(raw)
-
-    def _path_inside(root: Path, candidate: Path) -> bool:
-        try:
-            candidate.resolve().relative_to(root.resolve())
-            return True
-        except ValueError:
-            return False
-
-    def _llama_factory_version() -> str:
-        for command in (["llamafactory-cli", "version"], ["llamafactory-cli", "--version"]):
-            try:
-                result = subprocess.run(command, capture_output=True, text=True, timeout=5)
-            except Exception:
-                continue
-            output = (result.stdout or result.stderr).strip()
-            if result.returncode == 0 and output:
-                return output.splitlines()[0][:120]
-        return ""
-
-    def torch_cuda_status() -> dict[str, Any]:
-        try:
-            import torch  # type: ignore[import-not-found]
-        except Exception as exc:  # noqa: BLE001 - keep health endpoint resilient
-            return {
-                "available": False,
-                "device_count": 0,
-                "torch_version": "",
-                "torch_cuda_version": "",
-                "error": f"torch import failed: {exc}",
-            }
-        try:
-            available = bool(torch.cuda.is_available())
-            device_count = int(torch.cuda.device_count())
-            devices = []
-            for index in range(device_count):
-                props = torch.cuda.get_device_properties(index)
-                devices.append(
-                    {
-                        "index": index,
-                        "name": props.name,
-                        "memory_total_gb": round(props.total_memory / 1024 / 1024 / 1024, 2),
-                    }
-                )
-            return {
-                "available": available,
-                "device_count": device_count,
-                "torch_version": str(torch.__version__),
-                "torch_cuda_version": str(torch.version.cuda or ""),
-                "devices": devices,
-                "error": "" if available else "torch cuda is not available",
-            }
-        except Exception as exc:  # noqa: BLE001 - expose CUDA initialization failures
-            return {
-                "available": False,
-                "device_count": 0,
-                "torch_version": str(getattr(torch, "__version__", "")),
-                "torch_cuda_version": str(getattr(torch.version, "cuda", "") or ""),
-                "devices": [],
-                "error": str(exc),
-            }
-
-    def _slice_log_content(
-        content: str,
-        tail_lines: int | None = None,
-        offset: int | None = None,
-        limit: int | None = None,
-    ) -> dict[str, Any]:
-        lines = content.splitlines()
-        total = len(lines)
-        if offset is not None or limit is not None:
-            start = max(0, offset or 0)
-            end = start + limit if limit else total
-            selected = lines[start:end]
-        else:
-            tail = tail_lines or 200
-            start = max(0, total - tail)
-            selected = lines[start:]
-        next_offset = start + len(selected)
-        return {
-            "content": "\n".join(selected),
-            "total_lines": total,
-            "offset": start,
-            "limit": len(selected),
-            "has_more": next_offset < total,
-            "next_offset": next_offset if next_offset < total else None,
-        }
-
-    def _safe_float(value: Any, default: float = 0) -> float:
-        try:
-            return float(str(value).replace("[N/A]", "").strip() or default)
-        except (TypeError, ValueError):
-            return default
-
     def job_status(job: dict[str, Any]) -> dict[str, Any]:
         if execution_mode() != "simulator":
             return job
@@ -197,80 +74,9 @@ def create_app() -> FastAPI:
             )
         return "\n".join(lines)
 
-    def real_gpu_resources() -> list[dict[str, Any]]:
-        query = (
-            "index,uuid,name,memory.total,memory.used,utilization.gpu,"
-            "temperature.gpu,power.draw,power.limit"
-        )
-        try:
-            result = subprocess.run(
-                ["nvidia-smi", f"--query-gpu={query}", "--format=csv,noheader,nounits"],
-                check=True,
-                capture_output=True,
-                text=True,
-                timeout=5,
-            )
-        except Exception:
-            return fallback_gpu_resources()
-
-        items: list[dict[str, Any]] = []
-        for line in result.stdout.splitlines():
-            parts = [part.strip() for part in line.split(",")]
-            if len(parts) < 9:
-                continue
-            idx, uuid, name, mem_total, mem_used, util, temp, power, power_limit = parts[:9]
-            total_gb = round(_safe_float(mem_total) / 1024, 2)
-            used_gb = round(_safe_float(mem_used) / 1024, 2)
-            memory_percent = round(used_gb / total_gb * 100, 1) if total_gb else 0
-            gpu_percent = int(_safe_float(util))
-            items.append(
-                {
-                    "id": int(idx),
-                    "gpu_index": int(idx),
-                    "uuid": uuid,
-                    "name": name,
-                    "status": "busy" if gpu_percent >= 5 or used_gb > 1 else "idle",
-                    "gpu_percent": gpu_percent,
-                    "memory_used_gb": used_gb,
-                    "memory_total_gb": total_gb,
-                    "memory_percent": memory_percent,
-                    "temperature": int(_safe_float(temp)),
-                    "power_w": round(_safe_float(power), 1),
-                    "power_limit_w": round(_safe_float(power_limit), 1),
-                    "processes": [],
-                }
-            )
-        return items
-
-    def fallback_gpu_resources() -> list[dict[str, Any]]:
-        count = _int_env("COMPUTE_GPU_COUNT", 0)
-        if count <= 0:
-            return []
-        name = os.getenv("COMPUTE_GPU_NAME", "Configured GPU")
-        memory_total = _float_env("COMPUTE_GPU_MEMORY_GB", 80.0)
-        power_limit = _float_env("COMPUTE_GPU_POWER_LIMIT_W", 300.0)
-        return [
-            {
-                "id": idx,
-                "gpu_index": idx,
-                "uuid": f"GPU-{host_id().upper()}-{idx}",
-                "name": name,
-                "status": "idle",
-                "gpu_percent": 0,
-                "memory_used_gb": 0,
-                "memory_total_gb": memory_total,
-                "memory_percent": 0,
-                "temperature": _int_env("COMPUTE_GPU_BASE_TEMPERATURE", 35),
-                "power_w": 0,
-                "power_limit_w": power_limit,
-                "processes": [],
-            }
-            for idx in range(count)
-        ]
-
     def gpu_resources() -> list[dict[str, Any]]:
         if execution_mode() != "simulator":
-            return real_gpu_resources()
+            return []
         active_jobs = [job_status(job) for job in jobs.values() if job["status"] in {"queued", "running"}]
         gpus: list[dict[str, Any]] = []
         for idx in range(4):
@@ -303,168 +109,6 @@ def create_app() -> FastAPI:
             )
         return gpus
 
-    def _validate_training_accelerator(payload: dict[str, Any]) -> tuple[list[str], list[str], dict[str, Any]]:
-        errors: list[str] = []
-        warnings: list[str] = []
-        if str(payload.get("engine") or payload.get("training_engine") or "llama_factory") == "smoke":
-            return errors, warnings, {}
-        requested_gpus = [int(item) for item in payload.get("gpus") or []]
-        if not requested_gpus:
-            warnings.append("no gpu selected; training will run on CPU")
-            return errors, warnings, {}
-        cuda = torch_cuda_status()
-        if not cuda.get("available"):
-            errors.append(f"torch cuda unavailable on compute node: {cuda.get('error') or 'unknown error'}")
-        device_count = int(cuda.get("device_count") or 0)
-        if device_count and max(requested_gpus) >= device_count:
-            errors.append(f"requested gpu index out of torch device range: requested={requested_gpus}, device_count={device_count}")
-        min_memory_gb = _float_env("MIN_TRAINING_GPU_MEMORY_GB", 4.0)
-        gpus = {int(item["gpu_index"]): item for item in gpu_resources() if "gpu_index" in item}
-        for gpu_index in requested_gpus:
-            gpu = gpus.get(gpu_index)
-            if not gpu:
-                errors.append(f"requested gpu not found by nvidia-smi: {gpu_index}")
-                continue
-            memory_total = float(gpu.get("memory_total_gb") or 0)
-            if memory_total and memory_total < min_memory_gb:
-                errors.append(
-                    f"gpu {gpu_index} memory too small: {memory_total}GB < required {min_memory_gb}GB"
-                )
-        return errors, warnings, cuda
-
-    def _check_path_item(item: dict[str, Any]) -> dict[str, Any]:
-        path = Path(str(item.get("path") or ""))
-        exists = path.exists()
-        expected_type = str(item.get("type") or "any")
-        ok = exists
-        if exists and expected_type == "dir":
-            ok = path.is_dir()
-        if exists and expected_type == "file":
-            ok = path.is_file()
-        return {
-            "name": item.get("name") or "",
-            "path": str(path),
-            "type": expected_type,
-            "required": bool(item.get("required", True)),
-            "exists": exists,
-            "is_dir": path.is_dir() if exists else False,
-            "is_file": path.is_file() if exists else False,
-            "byte_size": sum(child.stat().st_size for child in path.rglob("*") if child.is_file()) if exists and path.is_dir() else path.stat().st_size if exists and path.is_file() else 0,
-            "ok": ok or not item.get("required", True),
-        }
-
-    def _job_preview(payload: dict[str, Any], check_paths: bool) -> dict[str, Any]:
-        warnings: list[str] = []
-        runtime_files: list[dict[str, str]] = []
-        command_payload = {**payload, "require_dataset_files": check_paths}
-        if check_paths:
-            try:
-                runtime_files = prepare_runtime_files(command_payload)
-            except OSError as exc:
-                return {
-                    "valid": False,
-                    "errors": [f"prepare runtime files failed: {exc}"],
-                    "warnings": warnings,
-                    "engine": str(payload.get("engine") or payload.get("training_engine") or "llama_factory"),
-                    "command": [],
-                    "command_text": "",
-                    "work_dir": os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"),
-                    "env": {},
-                    "runtime_files": [],
-                    "path_checks": [],
-                }
-        try:
-            command = build_command(command_payload, os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
-        except ValueError as exc:
-            return {
-                "valid": False,
-                "errors": [part.strip() for part in str(exc).split(";") if part.strip()],
-                "warnings": warnings,
-                "engine": str(payload.get("engine") or payload.get("training_engine") or "llama_factory"),
-                "command": [],
-                "command_text": "",
-                "work_dir": os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"),
-                "env": {},
-                "runtime_files": runtime_files,
-                "path_checks": [],
-            }
-
-        errors: list[str] = []
-        engine = str(payload.get("engine") or payload.get("training_engine") or "llama_factory")
-        path_checks: list[dict[str, Any]] = []
-        accelerator: dict[str, Any] = {}
-        if check_paths and engine != "smoke":
-            path_checks = [
-                _check_path_item(
-                    {
-                        "name": "model_name_or_path",
-                        "path": payload.get("model_name_or_path") or payload.get("base_model") or payload.get("base_model_path") or "",
-                        "type": "any",
-                        "required": True,
-                    }
-                )
-            ]
-            if engine in {"merge", "export", "llama_factory_export"} and payload.get("adapter_name_or_path"):
-                path_checks.append(
-                    _check_path_item(
-                        {
-                            "name": "adapter_name_or_path",
-                            "path": payload.get("adapter_name_or_path"),
-                            "type": "any",
-                            "required": True,
-                        }
-                    )
-                )
-            if payload.get("dataset_dir"):
-                path_checks.append(
-                    _check_path_item(
-                        {
-                            "name": "dataset_dir",
-                            "path": payload.get("dataset_dir"),
-                            "type": "dir",
-                            "required": True,
-                        }
-                    )
-                )
-            output_dir = Path(str(payload.get("output_dir") or "/data/yg-ft/outputs/training-job"))
-            path_checks.append(
-                _check_path_item(
-                    {
-                        "name": "output_parent",
-                        "path": str(output_dir.parent),
-                        "type": "dir",
-                        "required": False,
-                    }
-                )
-            )
-            errors.extend(
-                [f"{item['name']} path not available: {item['path']}" for item in path_checks if not item["ok"] and item["required"]]
-            )
-            if shutil.which(command.command[0]) is None:
-                errors.append(f"training command not found: {command.command[0]}")
-            if not Path(command.work_dir).exists():
-                errors.append(f"llama_factory_home not found: {command.work_dir}")
-            if engine not in {"merge", "export", "llama_factory_export"}:
-                accelerator_errors, accelerator_warnings, accelerator = _validate_training_accelerator(payload)
-                errors.extend(accelerator_errors)
-                warnings.extend(accelerator_warnings)
-        elif engine == "smoke":
-            warnings.append("smoke engine skips model and dataset path checks")
-
-        return {
-            "valid": not errors,
-            "errors": errors,
-            "warnings": warnings,
-            "engine": engine,
-            "command": command.command,
-            "command_text": " ".join(command.command),
-            "work_dir": command.work_dir,
-            "env": command.env,
-            "runtime_files": runtime_files,
-            "accelerator": accelerator,
-            "path_checks": path_checks,
-        }
-
     @app.get(f"{route_prefix}/health")
     async def health_check() -> dict[str, str]:
         return {
@@ -472,130 +116,41 @@ def create_app() -> FastAPI:
             "compute_host_id": os.getenv("COMPUTE_HOST_ID", "unknown"),
         }
 
-    @app.get("/health")
-    async def health_check_root() -> dict[str, str]:
-        return await health_check()
-
     @app.get(f"{route_prefix}/v1/compute/health")
-    async def compute_health_check() -> dict[str, Any]:
+    async def compute_health_check() -> dict[str, str | bool]:
         data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
-        dataset_root = Path(os.getenv("YG_FT_DATASET_ROOT", str(data_root / "datasets")))
-        output_root = Path(os.getenv("YG_FT_OUTPUT_ROOT", str(data_root / "outputs")))
         llama_factory_home = Path(os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
-        gpu_items = gpu_resources()
-        torch_cuda = torch_cuda_status()
         return {
             "status": "ok",
-            "api_version": "v1",
             "compute_host_id": os.getenv("COMPUTE_HOST_ID", "unknown"),
             "app_callback_enabled": os.getenv("ENABLE_APP_CALLBACK", "false").lower() == "true",
             "data_root": str(data_root),
             "data_root_exists": data_root.exists(),
-            "model_root": os.getenv("YG_FT_MODEL_ROOT", str(data_root / "models")),
-            "dataset_root": str(dataset_root),
-            "dataset_root_exists": dataset_root.exists(),
-            "output_root": str(output_root),
-            "output_root_exists": output_root.exists(),
-            "log_root": os.getenv("TRAINING_LOG_ROOT", "/opt/yg-ft/logs/training"),
             "llama_factory_home": str(llama_factory_home),
             "llama_factory_home_exists": llama_factory_home.exists(),
-            "llama_factory_version": os.getenv("LLAMA_FACTORY_VERSION", ""),
             "execution_mode": execution_mode(),
-            "gpu_count": _int_env("COMPUTE_GPU_COUNT", 0),
-            "nvidia_gpu_count": len(gpu_items),
-            "torch_cuda": torch_cuda,
-            "gpu_discovery_endpoint": f"{route_prefix}/compute/resources/gpus",
-            "capabilities": ["gpu_discovery", "torch_cuda_diagnostics", "llama_factory", "file_gateway", "job_polling"],
         }
 
     @app.get(f"{route_prefix}/v1/compute/jobs")
     async def list_jobs_alias() -> dict[str, list[dict[str, Any]]]:
-        items = process_manager.list_jobs() if execution_mode() != "simulator" else [job_status(job) for job in jobs.values()]
-        return {"items": items}
+        return {"items": [job_status(job) for job in jobs.values()]}
 
     @app.get(f"{route_prefix}/compute/resources/gpus")
     async def list_gpus() -> dict[str, Any]:
         return {"items": gpu_resources(), "compute_host_id": host_id()}
 
-    @app.get(f"{route_prefix}/v1/compute/resources/gpus")
-    async def list_gpus_v1() -> dict[str, Any]:
-        return {"items": gpu_resources(), "compute_host_id": host_id()}
-
-    @app.post(f"{route_prefix}/compute/jobs/preview")
-    async def preview_job(payload: dict[str, Any]) -> dict[str, Any]:
-        return _job_preview(payload, check_paths=False)
-
-    @app.post(f"{route_prefix}/compute/jobs/validate")
-    async def validate_job(payload: dict[str, Any]) -> dict[str, Any]:
-        return _job_preview(payload, check_paths=True)
-
-    @app.post(f"{route_prefix}/v1/compute/jobs/preview")
-    async def preview_job_v1(payload: dict[str, Any]) -> dict[str, Any]:
-        return await preview_job(payload)
-
-    @app.post(f"{route_prefix}/v1/compute/jobs/validate")
-    async def validate_job_v1(payload: dict[str, Any]) -> dict[str, Any]:
-        return await validate_job(payload)
-
-    @app.post(f"{route_prefix}/compute/files/check-paths")
-    async def check_paths(payload: dict[str, Any]) -> dict[str, Any]:
-        items = [_check_path_item(item) for item in payload.get("paths", []) if isinstance(item, dict)]
-        return {"valid": all(item["ok"] for item in items), "items": items}
-
-    @app.get(f"{route_prefix}/compute/files/list")
-    async def list_files(
-        root: str = Query(default="data"),
-        relative_path: str = Query(default=""),
-        directories_only: bool = Query(default=False),
-    ) -> dict[str, Any]:
-        roots = {
-            "data": Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft")),
-            "models": Path(os.getenv("YG_FT_MODEL_ROOT", os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft") + "/models")),
-            "datasets": Path(os.getenv("YG_FT_DATASET_ROOT", os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft") + "/datasets")),
-            "outputs": Path(os.getenv("YG_FT_OUTPUT_ROOT", os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft") + "/outputs")),
-        }
-        base = roots.get(root)
-        if base is None:
-            raise HTTPException(status_code=400, detail="invalid root")
-        target = (base / relative_path.lstrip("/\\")).resolve()
-        if not _path_inside(base, target):
-            raise HTTPException(status_code=400, detail="path must stay inside selected root")
-        if not target.exists():
-            return {"root": root, "base_path": str(base), "relative_path": relative_path, "items": []}
-        items = []
-        for child in sorted(target.iterdir(), key=lambda path: (not path.is_dir(), path.name.lower())):
-            if directories_only and not child.is_dir():
-                continue
-            items.append(
-                {
-                    "name": child.name,
-                    "path": str(child),
-                    "relative_path": str(child.relative_to(base)).replace("\\", "/"),
-                    "type": "directory" if child.is_dir() else "file",
-                    "byte_size": child.stat().st_size if child.is_file() else 0,
-                }
-            )
-        return {"root": root, "base_path": str(base), "relative_path": relative_path, "items": items}
-
     @app.post(f"{route_prefix}/compute/jobs")
     async def create_job(payload: dict[str, Any]) -> dict[str, Any]:
-        payload = {**payload, "require_dataset_files": True}
-        try:
-            prepare_runtime_files(payload)
-        except OSError as exc:
-            raise HTTPException(status_code=400, detail=f"prepare runtime files failed: {exc}")
         try:
             command = build_command(payload, os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
         except ValueError as exc:
             raise HTTPException(status_code=400, detail=str(exc))
-        job_id = str(payload.get("id") or f"job_{int(now() * 1000)}")
         if execution_mode() != "simulator":
-            try:
-                return process_manager.create_job({**payload, "id": job_id}, command.command, command.work_dir)
-            except FileNotFoundError as exc:
-                raise HTTPException(status_code=500, detail=f"training command not found: {exc.filename}")
-            except ValueError as exc:
-                raise HTTPException(status_code=409, detail=str(exc))
+            raise HTTPException(
+                status_code=501,
+                detail="real compute executor is not implemented yet; set COMPUTE_EXECUTION_MODE=simulator only for isolated development",
+            )
+        job_id = str(payload.get("id") or f"job_{int(now() * 1000)}")
         job = {
             "id": job_id,
             "name": payload.get("name", job_id),
@@ -614,29 +169,17 @@ def create_app() -> FastAPI:
 
     @app.get(f"{route_prefix}/compute/jobs")
     async def list_jobs() -> dict[str, Any]:
-        items = process_manager.list_jobs() if execution_mode() != "simulator" else [job_status(job) for job in jobs.values()]
-        return {"items": items}
+        return {"items": [job_status(job) for job in jobs.values()]}
 
     @app.get(f"{route_prefix}/compute/jobs/{{job_id}}")
     async def get_job(job_id: str) -> dict[str, Any]:
         job = jobs.get(job_id)
-        if execution_mode() != "simulator":
-            job = process_manager.get_job(job_id)
-            if not job:
-                raise HTTPException(status_code=404, detail="job not found")
-            return job
-        job = jobs.get(job_id)
         if not job:
             raise HTTPException(status_code=404, detail="job not found")
         return job_status(job)
 
     @app.post(f"{route_prefix}/compute/jobs/{{job_id}}/stop")
     async def stop_job(job_id: str) -> dict[str, Any]:
-        if execution_mode() != "simulator":
-            job = process_manager.stop_job(job_id)
-            if not job:
-                raise HTTPException(status_code=404, detail="job not found")
-            return job
         job = jobs.get(job_id)
         if not job:
             raise HTTPException(status_code=404, detail="job not found")
@@ -645,101 +188,22 @@ def create_app() -> FastAPI:
         return job
 
     @app.get(f"{route_prefix}/compute/jobs/{{job_id}}/logs")
-    async def job_logs(
-        job_id: str,
-        tail_lines: int | None = Query(default=200, ge=1, le=5000),
-        offset: int | None = Query(default=None, ge=0),
-        limit: int | None = Query(default=None, ge=1, le=5000),
-    ) -> dict[str, Any]:
-        if execution_mode() != "simulator":
-            job = process_manager.get_job(job_id)
-            if not job:
-                raise HTTPException(status_code=404, detail="job not found")
-            content = process_manager.logs(job_id)
-        else:
+    async def job_logs(job_id: str) -> dict[str, Any]:
         job = jobs.get(job_id)
         if not job:
             raise HTTPException(status_code=404, detail="job not found")
         job = job_status(job)
-            content = job["logs"]
-        window = _slice_log_content(content, tail_lines, offset, limit)
-        metrics = [parse_log_line(line) for line in window["content"].splitlines()]
-        return {"job_id": job_id, **window, "metrics": [m for m in metrics if m]}
+        metrics = [parse_log_line(line) for line in job["logs"].splitlines()]
+        return {"job_id": job_id, "content": job["logs"], "metrics": [m for m in metrics if m]}
 
     @app.post(f"{route_prefix}/compute/files/upload")
-    async def upload_file(
-        file: UploadFile | None = File(default=None),
-        target_relative_path: str | None = Form(default=None),
-        resource_type: str | None = Form(default=None),
-        resource_id: str | None = Form(default=None),
-    ) -> dict[str, Any]:
-        file_id = f"file_{int(now() * 1000)}"
-        data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
-        data_root.mkdir(parents=True, exist_ok=True)
-        filename = Path(file.filename if file else file_id).name
-        if target_relative_path:
-            target = (data_root / target_relative_path.lstrip("/\\")).resolve()
-            if not _path_inside(data_root, target):
-                raise HTTPException(status_code=400, detail="target path must stay inside YG_FT_DATA_ROOT")
-        else:
-            target = data_root / "uploads" / f"{file_id}_{filename}"
-        if file:
-            target.parent.mkdir(parents=True, exist_ok=True)
-            with target.open("wb") as output:
-                while chunk := await file.read(1024 * 1024):
-                    output.write(chunk)
-        else:
-            target.parent.mkdir(parents=True, exist_ok=True)
-            target.write_text("", encoding="utf-8")
-        return {
-            "id": file_id,
-            "resource_type": resource_type,
-            "resource_id": resource_id,
-            "status": "available",
-            "local_path": str(target),
-            "byte_size": target.stat().st_size,
-            "checksum_sha256": hashlib.sha256(target.read_bytes()).hexdigest() if target.is_file() else "",
-        }
-
-    @app.post(f"{route_prefix}/compute/files/import-local")
-    async def import_local_file(payload: dict[str, Any]) -> dict[str, Any]:
-        source = Path(str(payload.get("source_path") or ""))
-        if not source.exists():
-            raise HTTPException(status_code=404, detail="source path not found")
-        data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
-        data_root.mkdir(parents=True, exist_ok=True)
-        relative = str(payload.get("target_relative_path") or f"imports/{source.name}").lstrip("/\\")
-        target = (data_root / relative).resolve()
-        if not _path_inside(data_root, target):
-            raise HTTPException(status_code=400, detail="target path must stay inside YG_FT_DATA_ROOT")
-        target.parent.mkdir(parents=True, exist_ok=True)
-        if source.is_dir():
-            if target.exists():
-                shutil.rmtree(target)
-            shutil.copytree(source, target)
-            byte_size = sum(path.stat().st_size for path in target.rglob("*") if path.is_file())
-            checksum = ""
-        else:
-            shutil.copy2(source, target)
-            byte_size = target.stat().st_size
-            checksum = hashlib.sha256(target.read_bytes()).hexdigest()
-        return {
-            "id": str(payload.get("id") or f"file_{int(now() * 1000)}"),
-            "resource_type": payload.get("resource_type"),
-            "resource_id": payload.get("resource_id"),
-            "status": "available",
-            "local_path": str(target),
-            "byte_size": byte_size,
-            "checksum_sha256": checksum,
-        }
+    async def upload_file(payload: dict[str, Any]) -> dict[str, Any]:
+        file_id = str(payload.get("id") or f"file_{int(now() * 1000)}")
+        return {"id": file_id, "status": "available", "local_path": f"/data/yg-ft/uploads/{file_id}"}
 
     @app.get(f"{route_prefix}/compute/files/{{file_id}}/download")
-    async def download_file(file_id: str) -> FileResponse:
-        upload_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft")) / "uploads"
-        matches = list(upload_root.glob(f"{file_id}_*"))
-        if not matches:
-            raise HTTPException(status_code=404, detail="file not found")
-        return FileResponse(matches[0])
+    async def download_file(file_id: str) -> dict[str, Any]:
+        return {"id": file_id, "status": "ready", "download_url": f"{route_prefix}/compute/files/{file_id}/download"}
 
     return app
 

compute/engines/llama_factory/adapter.py

@@ -1,6 +1,5 @@
 from __future__ import annotations
 
-import json
 import re
 from dataclasses import dataclass
 from pathlib import Path
@@ -14,234 +13,40 @@ class LlamaFactoryCommand:
     env: dict[str, str]
 
 
-def _load_dataset_preview(path: Path) -> list[dict[str, Any]]:
-    if not path.exists():
-        return []
-    text = path.read_text(encoding="utf-8", errors="replace").strip()
-    if not text:
-        return []
-    if path.suffix.lower() == ".jsonl":
-        items: list[dict[str, Any]] = []
-        for line in text.splitlines()[:20]:
-            line = line.strip()
-            if not line:
-                continue
-            value = json.loads(line)
-            if isinstance(value, dict):
-                items.append(value)
-        return items
-    value = json.loads(text)
-    if isinstance(value, list):
-        return [item for item in value[:20] if isinstance(item, dict)]
-    if isinstance(value, dict):
-        return [value]
-    return []
-
-
-def _validate_dataset_columns(config: dict[str, Any]) -> list[str]:
-    dataset_dir = config.get("dataset_dir")
-    dataset_info = config.get("dataset_info")
-    if not dataset_dir or not isinstance(dataset_info, dict):
-        return []
-    root = Path(str(dataset_dir))
-    errors: list[str] = []
-    for dataset_key, item in dataset_info.items():
-        if not isinstance(item, dict):
-            continue
-        file_name = item.get("file_name")
-        file_names = file_name if isinstance(file_name, list) else [file_name]
-        columns = item.get("columns") if isinstance(item.get("columns"), dict) else {}
-        required_columns = [str(value) for value in columns.values() if value]
-        for name in file_names:
-            if not name:
-                continue
-            path = root / str(name).lstrip("/\\")
-            if not path.exists():
-                continue
-            try:
-                preview_rows = _load_dataset_preview(path)
-            except Exception as exc:  # noqa: BLE001 - expose malformed data as validation error
-                errors.append(f"dataset file parse failed: {path}: {exc}")
-                continue
-            if not preview_rows:
-                errors.append(f"dataset file has no valid object records: {path}")
-                continue
-            available = set().union(*(row.keys() for row in preview_rows))
-            missing = [column for column in required_columns if column not in available]
-            if missing:
-                errors.append(
-                    f"dataset columns missing in {path.name} for {dataset_key}: {', '.join(sorted(set(missing)))}"
-                )
-    return errors
-
-
 def validate_config(config: dict[str, Any]) -> list[str]:
     errors: list[str] = []
     if not config.get("base_model") and not config.get("model_name_or_path"):
         errors.append("base_model or model_name_or_path is required")
     if not config.get("dataset") and not config.get("dataset_dir"):
         errors.append("dataset or dataset_dir is required")
-    try:
     learning_rate = float(config.get("learning_rate", 0.0002))
-    except (TypeError, ValueError):
-        learning_rate = 0
     if learning_rate <= 0:
         errors.append("learning_rate must be greater than zero")
-    try:
     epochs = int(config.get("n_epochs", config.get("num_train_epochs", 1)))
-    except (TypeError, ValueError):
-        epochs = 0
     if epochs <= 0:
         errors.append("n_epochs must be greater than zero")
-    dataset_dir = config.get("dataset_dir")
-    dataset_info = config.get("dataset_info")
-    if config.get("require_dataset_files") and dataset_dir and isinstance(dataset_info, dict):
-        root = Path(str(dataset_dir))
-        for dataset_key, item in dataset_info.items():
-            if not isinstance(item, dict):
-                errors.append(f"dataset_info entry must be object: {dataset_key}")
-                continue
-            file_name = item.get("file_name")
-            file_names = file_name if isinstance(file_name, list) else [file_name]
-            for name in file_names:
-                if not name:
-                    errors.append(f"dataset_info file_name is required: {dataset_key}")
-                    continue
-                path = root / str(name).lstrip("/\\")
-                if not path.exists():
-                    errors.append(f"dataset file not found: {path}")
-        errors.extend(_validate_dataset_columns(config))
     return errors
 
 
-def _optional_arg(config: dict[str, Any], command: list[str], option: str, *keys: str) -> None:
-    for key in keys:
-        value = config.get(key)
-        if value is not None and value != "":
-            command.extend([option, str(value)])
-            return
-
-
-def _optional_bool_arg(config: dict[str, Any], command: list[str], option: str, *keys: str) -> None:
-    for key in keys:
-        value = config.get(key)
-        if value is True or str(value).lower() == "true":
-            command.extend([option, "true"])
-            return
-
-
-def _normalize_stage(config: dict[str, Any]) -> str:
-    raw = str(config.get("stage") or config.get("train_type") or "sft").strip().lower()
-    return {
-        "sft": "sft",
-        "dpo": "dpo",
-        "cpt": "pt",
-        "pt": "pt",
-        "pretrain": "pt",
-        "rm": "rm",
-        "ppo": "ppo",
-        "kto": "kto",
-    }.get(raw, raw or "sft")
-
-
-def prepare_runtime_files(config: dict[str, Any]) -> list[dict[str, str]]:
-    dataset_dir = config.get("dataset_dir")
-    dataset_info = config.get("dataset_info")
-    if not dataset_dir or not isinstance(dataset_info, dict):
-        return []
-    root = Path(str(dataset_dir))
-    root.mkdir(parents=True, exist_ok=True)
-    path = root / "dataset_info.json"
-    existing: dict[str, Any] = {}
-    if path.exists():
-        try:
-            loaded = json.loads(path.read_text(encoding="utf-8"))
-            existing = loaded if isinstance(loaded, dict) else {}
-        except json.JSONDecodeError:
-            existing = {}
-    existing.update(dataset_info)
-    path.write_text(json.dumps(existing, ensure_ascii=False, indent=2), encoding="utf-8")
-    return [{"name": "dataset_info", "path": str(path)}]
-
-
 def build_command(config: dict[str, Any], llama_factory_home: str = "/app/LLaMA-Factory") -> LlamaFactoryCommand:
-    engine = str(config.get("engine") or config.get("training_engine") or "llama_factory")
-    if engine in {"merge", "export", "llama_factory_export"}:
-        model_path = config.get("base_model") or config.get("model_name_or_path") or config.get("base_model_path")
-        adapter_path = config.get("adapter_name_or_path") or config.get("adapter_path") or config.get("lora_path")
-        output_dir = config.get("output_dir") or config.get("export_dir")
-        errors: list[str] = []
-        if not model_path:
-            errors.append("base_model or model_name_or_path is required")
-        if not adapter_path and engine == "merge":
-            errors.append("adapter_name_or_path or adapter_path is required")
-        if not output_dir:
-            errors.append("output_dir or export_dir is required")
-        if errors:
-            raise ValueError("; ".join(errors))
-        command = [
-            "llamafactory-cli",
-            "export",
-            "--model_name_or_path",
-            str(model_path),
-            "--template",
-            str(config.get("template", "qwen")),
-            "--finetuning_type",
-            str(config.get("train_method", config.get("finetuning_type", "lora"))),
-            "--export_dir",
-            str(output_dir),
-            "--export_size",
-            str(config.get("export_size", 2)),
-            "--export_device",
-            str(config.get("export_device", "cpu")),
-            "--export_legacy_format",
-            str(config.get("export_legacy_format", False)).lower(),
-        ]
-        if adapter_path:
-            command.extend(["--adapter_name_or_path", str(adapter_path)])
-        quantization_bit = int(config.get("export_quantization_bit", config.get("quantization_bit", 0)) or 0)
-        if quantization_bit in {4, 8}:
-            command.extend(["--quantization_bit", str(quantization_bit)])
-        return LlamaFactoryCommand(command=command, work_dir=str(Path(llama_factory_home)), env={})
-
     errors = validate_config(config)
     if errors:
         raise ValueError("; ".join(errors))
 
-    if engine == "smoke":
-        output_dir = config.get("output_dir") or f"/data/yg-ft/outputs/{config.get('name', 'training-smoke')}"
-        script = (
-            "import json, os, time; "
-            f"out={str(output_dir)!r}; "
-            "os.makedirs(out, exist_ok=True); "
-            "print('[INFO] smoke training started', flush=True); "
-            "\nfor step in range(1, 7):\n"
-            "    loss=round(1.8/(step+1), 4)\n"
-            "    lr=round(0.0002*(1-step/10), 8)\n"
-            "    print({'loss': loss, 'grad_norm': round(0.4 + step*0.03, 4), 'learning_rate': lr, 'epoch': round(step/6, 4)}, flush=True)\n"
-            "    time.sleep(0.4)\n"
-            "\nopen(os.path.join(out, 'adapter_config.json'), 'w', encoding='utf-8').write(json.dumps({'engine':'smoke','status':'completed'})); "
-            "print('***** train metrics *****', flush=True); "
-            "print('train_loss = 0.12', flush=True); "
-            "print('***** train metrics end *****', flush=True)"
-        )
-        return LlamaFactoryCommand(command=["python", "-u", "-c", script], work_dir="/app", env={})
-
     model_path = config.get("base_model") or config.get("model_name_or_path")
-    dataset = config.get("dataset") or config.get("dataset_name")
-    dataset_dir = config.get("dataset_dir")
+    dataset = config.get("dataset") or config.get("dataset_dir")
     output_dir = config.get("output_dir") or f"/data/yg-ft/outputs/{config.get('name', 'training-job')}"
     command = [
         "llamafactory-cli",
         "train",
         "--stage",
-        _normalize_stage(config),
+        str(config.get("stage", "sft")).lower(),
         "--do_train",
         "true",
         "--model_name_or_path",
         str(model_path),
         "--dataset",
-        str(dataset or "default"),
+        str(dataset),
         "--template",
         str(config.get("template", "qwen")),
         "--finetuning_type",
@@ -256,28 +61,7 @@ def build_command(config: dict[str, Any], llama_factory_home: str = "/app/LLaMA-
         str(config.get("n_epochs", 3)),
         "--save_steps",
         str(config.get("save_steps", 50)),
-        "--logging_steps",
-        str(config.get("logging_steps", 10)),
-        "--overwrite_output_dir",
-        "true",
-        "--plot_loss",
-        "true",
     ]
-    if dataset_dir:
-        command.extend(["--dataset_dir", str(dataset_dir)])
-    _optional_arg(config, command, "--cutoff_len", "max_length", "cutoff_len")
-    _optional_arg(config, command, "--lr_scheduler_type", "lr_scheduler_type")
-    _optional_arg(config, command, "--warmup_ratio", "warmup_ratio")
-    _optional_arg(config, command, "--weight_decay", "weight_decay")
-    _optional_arg(config, command, "--lora_rank", "lora_rank", "rank")
-    _optional_arg(config, command, "--lora_alpha", "lora_alpha")
-    _optional_arg(config, command, "--lora_dropout", "lora_dropout")
-    _optional_arg(config, command, "--gradient_accumulation_steps", "gradient_accumulation_steps")
-    _optional_arg(config, command, "--val_size", "val_size")
-    _optional_arg(config, command, "--max_samples", "max_samples")
-    _optional_arg(config, command, "--preprocessing_num_workers", "preprocessing_num_workers")
-    _optional_bool_arg(config, command, "--fp16", "fp16")
-    _optional_bool_arg(config, command, "--bf16", "bf16")
     quantization_bit = int(config.get("quantization_bit", 0) or 0)
     if quantization_bit in {4, 8}:
         command.extend(["--quantization_bit", str(quantization_bit)])

compute/requirements.txt

@@ -1,6 +1,5 @@
 fastapi>=0.111.0
 uvicorn[standard]>=0.30.0
-python-multipart>=0.0.9
 pydantic>=2.7.0
 python-dotenv>=1.0.1
 httpx>=0.27.0

docker/README.md

@@ -56,7 +56,7 @@ $images | ForEach-Object { docker pull $_ }
 | PostgreSQL | `15432` | `5432` | 开发阶段内置数据库 |
 | Redis | `16379` | `6379` | 开发阶段内置缓存 |
 | Compute API | `19100` | `9100` | 算力服务器 API |
-| File Gateway | `19101` | `9100` | 当前由 Compute API 暴露文件网关契约，后续可拆为独立服务 |
+| File Gateway | `19101` | 后续服务端口 | 当前预留，后续拆出文件网关服务时使用 |
 
 注意：`8000` 是后端容器内部端口，不作为宿主机对外访问端口。宿主机或浏览器应访问 `http://<app-server-ip>:17861/modelTF/health`；前端 Nginx 容器在 Docker 网络内部访问 `http://backend-api:8000/modelTF/...`。
 
@@ -142,16 +142,6 @@ docker compose logs --tail=80 frontend
 
 如果使用企业统一 PostgreSQL/Redis，修改 `docker/app/.env`：
 
-如果前端 Nginx 日志出现 `open() "/usr/share/nginx/html/modelTF/login" failed` 或 `open() "/usr/share/nginx/html/login" failed`，说明当前容器没有加载项目的 Nginx 代理配置，`/modelTF/*` 被当成静态文件查找。处理方式：
-
-```bash
-cd <repo-root>/docker/app
-docker compose up -d --force-recreate frontend
-docker compose exec frontend nginx -T | grep -n "location.*modelTF" -A12
-```
-
-正常配置中应存在 `location ^~ /modelTF/`，并代理到 `BACKEND_PROXY_PASS`，默认是 `http://backend-api:8000`。
-
 ```env
 DATABASE_URL=postgresql+psycopg://<user>:<password>@<postgres-host>:15432/<db>
 REDIS_URL=redis://<redis-host>:16379/0
@@ -202,28 +192,10 @@ GET http://<compute-server-ip>:19100/modelTF/v1/compute/health
 ```text
 ../../compute -> /app/compute
 ${YG_FT_DATA_ROOT_HOST} -> /data/yg-ft
-${YG_FT_MODEL_ROOT_HOST} -> /data/yg-ft/models
-${YG_FT_DATASET_ROOT_HOST} -> /data/yg-ft/datasets
-${YG_FT_OUTPUT_ROOT_HOST} -> /data/yg-ft/outputs
-${COMPUTE_LOG_ROOT_HOST} -> /opt/yg-ft/logs/compute
-${TRAINING_LOG_ROOT_HOST} -> /opt/yg-ft/logs/training
+../../runtime/compute/logs -> /opt/yg-ft/logs/compute
+../../runtime/compute/training-logs -> /opt/yg-ft/logs/training
 ```
 
-算力服务器启动前必须先在宿主机创建持久化目录，基座模型、训练数据、训练产物和训练日志都应落在宿主机磁盘上，不能只写入容器层。推荐默认目录：
-
-```bash
-cd <repo-root>/docker/compute
-mkdir -p data/yg-ft/models \
-  data/yg-ft/datasets \
-  data/yg-ft/outputs \
-  data/yg-ft/logs/compute \
-  data/yg-ft/logs/training
-```
-
-默认 `docker/compute/.env.example` 使用 `./data/yg-ft`，该相对路径以 `docker/compute/docker-compose.yml` 所在目录为基准，因此实际宿主机目录是 `<repo-root>/docker/compute/data/yg-ft`。如企业环境模型盘、数据盘、产物盘分盘挂载，可在 `docker/compute/.env` 中分别调整 `YG_FT_MODEL_ROOT_HOST`、`YG_FT_DATASET_ROOT_HOST`、`YG_FT_OUTPUT_ROOT_HOST`、`COMPUTE_LOG_ROOT_HOST`、`TRAINING_LOG_ROOT_HOST`，容器内路径建议保持 `/data/yg-ft/models`、`/data/yg-ft/datasets`、`/data/yg-ft/outputs`，避免训练参数和节点配置复杂化。
-
-页面上传数据集时，文件先进入 Backend API，再由 Backend API 调用目标算力节点的 `POST /modelTF/compute/files/upload`，写入容器内 `/data/yg-ft/datasets/{dataset_id}/`。在默认开发配置下，宿主机可在 `<repo-root>/docker/compute/data/yg-ft/datasets/{dataset_id}/` 看到对应文件。仅创建 bind mount 不会自动让应用侧上传文件出现在算力目录，必须通过这条 File Gateway 链路同步。
-
 ## 应用与算力分离部署
 
 应用服务器只需要主动访问算力服务器，不要求算力服务器回调应用服务器。
@@ -235,7 +207,7 @@ COMPUTE_API_BASE_URL=http://<compute-server-ip>:19100
 FILE_GATEWAY_BASE_URL=http://<compute-server-ip>:19101
 COMPUTE_SERVICE_TOKEN=change_me
 COMPUTE_STATUS_SYNC_MODE=polling
-COMPUTE_POLL_INTERVAL_SECONDS=3
+COMPUTE_POLL_INTERVAL_SECONDS=10
 COMPUTE_POLL_BATCH_SIZE=100
 ```
 
@@ -250,8 +222,6 @@ Frontend
     <- Backend Worker 定时轮询 Compute API
 ```
 
-算力服务默认开启服务间鉴权。`docker/compute/.env` 中保持 `COMPUTE_AUTH_ENABLED=true`，并确保 `COMPUTE_SERVICE_TOKEN` 与 `docker/app/.env` 一致；健康检查路径仍可用于容器探活。
-
 ## 多算力节点部署
 
 多算力节点仍按“单机多 GPU 节点”部署。每台 GPU 服务器都独立部署一套 `docker/compute`：
@@ -264,8 +234,6 @@ gpu-node-03: docker/compute + /data/yg-ft + 19100/19101
 
 节点之间默认不互访。应用平台主动访问每个节点的 Compute API/File Gateway，并通过 `compute_nodes`、`resource_replicas`、`resource_sync_jobs` 统一调度和同步。
 
-节点地址、权重、标签、启用状态和本地路径在前端“算力节点”页面动态维护。新增或编辑节点后，点击“测试”会由 Backend API 主动访问该节点的 `GET /modelTF/v1/compute/health` 和 `GET /modelTF/compute/resources/gpus`，并把健康信息与 GPU 清单同步到 PostgreSQL。
-
 ## 常用命令
 
 重新构建应用镜像：

docker/app/.env.example

@@ -13,15 +13,15 @@ POSTGRES_PORT=15432
 REDIS_PORT=16379
 
 POSTGRES_DB=yg_ft
-POSTGRES_USER=root
-POSTGRES_PASSWORD=8811614287327Leo
-DATABASE_URL=postgresql+psycopg://root:8811614287327Leo@www.caoxiaozhu.com:5432/yg_ft
+POSTGRES_USER=yg_ft
+POSTGRES_PASSWORD=change_me
+DATABASE_URL=postgresql+psycopg://yg_ft:change_me@postgres:5432/yg_ft
 
 REDIS_URL=redis://redis:6379/0
 
-# PostgreSQL uses the shared external database. The local postgres service is disabled in docker-compose.yml.
-# Redis still uses the built-in service during current development.
-USE_BUILTIN_POSTGRES=false
+# Development uses the built-in PostgreSQL/Redis services in docker-compose.yml.
+# For enterprise infrastructure, replace DATABASE_URL/REDIS_URL and remove or disable those services.
+USE_BUILTIN_POSTGRES=true
 USE_BUILTIN_REDIS=true
 
 LOG_LEVEL=INFO
@@ -43,4 +43,3 @@ COMPUTE_MODE=real
 COMPUTE_STATUS_SYNC_MODE=polling
 COMPUTE_POLL_INTERVAL_SECONDS=10
 COMPUTE_POLL_BATCH_SIZE=100
-COMPUTE_REQUEST_TIMEOUT_SECONDS=5

docker/app/Dockerfile.backend

@@ -11,7 +11,7 @@ RUN pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple \
     && pip install -r /tmp/requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple \
     && rm -f /tmp/requirements.txt
 
-RUN python -c "import fastapi, uvicorn, psycopg, sqlalchemy, redis, jwt, passlib, httpx, alembic; print('backend dependency check ok')"
+RUN python -c "import fastapi, uvicorn, psycopg, sqlalchemy, redis, yaml, jwt, passlib, httpx, alembic; print('backend dependency check ok')"
 
 RUN mkdir -p /opt/yg-ft/logs/backend /data/yg-ft \
     && chmod -R 0775 /opt/yg-ft /data/yg-ft

docker/app/docker-compose.yml

@@ -21,8 +21,6 @@ services:
           ls -la /usr/share/nginx/html;
           exit 1;
         fi;
-        envsubst '$$BACKEND_PROXY_PASS' < /etc/nginx/templates/default.conf.template > /etc/nginx/conf.d/default.conf;
-        nginx -t;
         nginx -g 'daemon off;'
     networks:
       - yg-ft-app
@@ -32,6 +30,8 @@ services:
     image: ${BACKEND_API_IMAGE:-yg-ft-backend-api:latest}
     container_name: yg-ft-backend-api
     depends_on:
+      postgres:
+        condition: service_healthy
       redis:
         condition: service_healthy
     expose:
@@ -43,9 +43,9 @@ services:
       APP_NAME: ${APP_NAME:-YG Fine-Tune Platform API}
       MODELTF_ROUTE_PREFIX: ${MODELTF_ROUTE_PREFIX:-/modelTF}
       CORS_ALLOW_ORIGINS: ${CORS_ALLOW_ORIGINS:-http://localhost:16801,http://127.0.0.1:16801}
-      DATABASE_URL: ${DATABASE_URL:-postgresql+psycopg://root:8811614287327Leo@www.caoxiaozhu.com:5432/yg_ft}
+      DATABASE_URL: ${DATABASE_URL:-postgresql+psycopg://yg_ft:change_me@postgres:5432/yg_ft}
       REDIS_URL: ${REDIS_URL:-redis://redis:6379/0}
-      USE_BUILTIN_POSTGRES: ${USE_BUILTIN_POSTGRES:-false}
+      USE_BUILTIN_POSTGRES: ${USE_BUILTIN_POSTGRES:-true}
       USE_BUILTIN_REDIS: ${USE_BUILTIN_REDIS:-true}
       LOG_LEVEL: ${LOG_LEVEL:-INFO}
       LOG_DIR: ${LOG_DIR:-/opt/yg-ft/logs/backend}
@@ -58,9 +58,8 @@ services:
       FILE_GATEWAY_BASE_URL: ${FILE_GATEWAY_BASE_URL:-http://compute-api:9101}
       COMPUTE_MODE: ${COMPUTE_MODE:-real}
       COMPUTE_STATUS_SYNC_MODE: ${COMPUTE_STATUS_SYNC_MODE:-polling}
-      COMPUTE_POLL_INTERVAL_SECONDS: ${COMPUTE_POLL_INTERVAL_SECONDS:-3}
+      COMPUTE_POLL_INTERVAL_SECONDS: ${COMPUTE_POLL_INTERVAL_SECONDS:-10}
       COMPUTE_POLL_BATCH_SIZE: ${COMPUTE_POLL_BATCH_SIZE:-100}
-      COMPUTE_REQUEST_TIMEOUT_SECONDS: ${COMPUTE_REQUEST_TIMEOUT_SECONDS:-5}
       PYTHONPATH: /app
     volumes:
       - ../../backend:/app:ro
@@ -76,29 +75,27 @@ services:
       start_period: 20s
     restart: unless-stopped
 
-  # PostgreSQL uses the shared external database configured by DATABASE_URL in docker/app/.env.
-  # Keep this local service commented out unless development needs an isolated database again.
-  # postgres:
-  #   image: postgres:16-alpine
-  #   container_name: yg-ft-postgres
-  #   environment:
-  #     POSTGRES_DB: ${POSTGRES_DB:-yg_ft}
-  #     POSTGRES_USER: ${POSTGRES_USER:-yg_ft}
-  #     POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-change_me}
-  #     PGDATA: /var/lib/postgresql/data/pgdata
-  #   volumes:
-  #     - postgres_data:/var/lib/postgresql/data
-  #     - ../../backend/app/db/sql/001_platform_runtime.sql:/docker-entrypoint-initdb.d/001-platform-runtime.sql:ro
-  #   ports:
-  #     - "${POSTGRES_PORT:-15432}:5432"
-  #   networks:
-  #     - yg-ft-app
-  #   healthcheck:
-  #     test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
-  #     interval: 10s
-  #     timeout: 5s
-  #     retries: 5
-  #   restart: unless-stopped
+  postgres:
+    image: postgres:16-alpine
+    container_name: yg-ft-postgres
+    environment:
+      POSTGRES_DB: ${POSTGRES_DB:-yg_ft}
+      POSTGRES_USER: ${POSTGRES_USER:-yg_ft}
+      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-change_me}
+      PGDATA: /var/lib/postgresql/data/pgdata
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+      - ../../backend/app/db/sql/001_platform_runtime.sql:/docker-entrypoint-initdb.d/001-platform-runtime.sql:ro
+    ports:
+      - "${POSTGRES_PORT:-15432}:5432"
+    networks:
+      - yg-ft-app
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    restart: unless-stopped
 
   redis:
     image: redis:7-alpine
@@ -122,5 +119,5 @@ networks:
     name: yg-ft-app
 
 volumes:
-  # postgres_data:
+  postgres_data:
   redis_data:

docker/compute/.env

@@ -1,40 +0,0 @@
-COMPUTE_ENV=prod
-COMPUTE_HOST_ID=gpu-node-01
-COMPUTE_EXECUTION_MODE=real
-MODELTF_ROUTE_PREFIX=/modelTF
-# Five-digit host ports exposed outside the compute server.
-COMPUTE_API_PORT=19100
-FILE_GATEWAY_PORT=19101
-COMPUTE_API_IMAGE=yg-ft-compute-api:latest
-
-# The application server actively polls Compute API; compute server does not need reverse access.
-COMPUTE_AUTH_ENABLED=true
-COMPUTE_SERVICE_TOKEN=change_me
-ENABLE_APP_CALLBACK=false
-
-# LLaMA-Factory is provided by the official hiyouga/llamafactory base image.
-LLAMA_FACTORY_HOME=/app/LLaMA-Factory
-
-YG_FT_DATA_ROOT=/data/yg-ft
-YG_FT_DATA_ROOT_HOST=./data/yg-ft
-YG_FT_MODEL_ROOT=/data/yg-ft/models
-YG_FT_MODEL_ROOT_HOST=./data/yg-ft/models
-YG_FT_DATASET_ROOT=/data/yg-ft/datasets
-YG_FT_DATASET_ROOT_HOST=./data/yg-ft/datasets
-YG_FT_OUTPUT_ROOT=/data/yg-ft/outputs
-YG_FT_OUTPUT_ROOT_HOST=./data/yg-ft/outputs
-TRAINING_LOG_ROOT=/opt/yg-ft/logs/training
-TRAINING_LOG_ROOT_HOST=./data/yg-ft/logs/training
-COMPUTE_LOG_ROOT_HOST=./data/yg-ft/logs/compute
-
-# Optional fallback used when nvidia-smi is unavailable.
-# Leave COMPUTE_GPU_COUNT=0 on real GPU servers with working NVIDIA runtime.
-COMPUTE_GPU_COUNT=0
-COMPUTE_GPU_NAME=NVIDIA A800-SXM4-80GB
-COMPUTE_GPU_MEMORY_GB=80
-COMPUTE_GPU_POWER_LIMIT_W=300
-
-LOG_DIR=/opt/yg-ft/logs/compute
-CUDA_VISIBLE_DEVICES=all
-NVIDIA_VISIBLE_DEVICES=all
-NVIDIA_DRIVER_CAPABILITIES=compute,utility

docker/compute/.env.example

@@ -8,35 +8,14 @@ FILE_GATEWAY_PORT=19101
 COMPUTE_API_IMAGE=yg-ft-compute-api:latest
 
 # The application server actively polls Compute API; compute server does not need reverse access.
-COMPUTE_AUTH_ENABLED=true
 COMPUTE_SERVICE_TOKEN=change_me
 ENABLE_APP_CALLBACK=false
 
 # LLaMA-Factory is provided by the official hiyouga/llamafactory base image.
 LLAMA_FACTORY_HOME=/app/LLaMA-Factory
 
-# Persistent host directories on the compute server.
-# Create these directories before starting docker compose. They are mounted into
-# the container so base models, datasets, training outputs and logs survive
-# container recreation or image upgrades.
 YG_FT_DATA_ROOT=/data/yg-ft
-YG_FT_DATA_ROOT_HOST=./data/yg-ft
-YG_FT_MODEL_ROOT=/data/yg-ft/models
-YG_FT_MODEL_ROOT_HOST=./data/yg-ft/models
-YG_FT_DATASET_ROOT=/data/yg-ft/datasets
-YG_FT_DATASET_ROOT_HOST=./data/yg-ft/datasets
-YG_FT_OUTPUT_ROOT=/data/yg-ft/outputs
-YG_FT_OUTPUT_ROOT_HOST=./data/yg-ft/outputs
-TRAINING_LOG_ROOT=/opt/yg-ft/logs/training
-TRAINING_LOG_ROOT_HOST=./data/yg-ft/logs/training
-COMPUTE_LOG_ROOT_HOST=./data/yg-ft/logs/compute
-
-# Optional fallback used when nvidia-smi is unavailable.
-# Leave COMPUTE_GPU_COUNT=0 on real GPU servers with working NVIDIA runtime.
-COMPUTE_GPU_COUNT=0
-COMPUTE_GPU_NAME=NVIDIA A800-SXM4-80GB
-COMPUTE_GPU_MEMORY_GB=80
-COMPUTE_GPU_POWER_LIMIT_W=300
+YG_FT_DATA_ROOT_HOST=/data/yg-ft
 
 LOG_DIR=/opt/yg-ft/logs/compute
 CUDA_VISIBLE_DEVICES=all

docker/compute/data/yg-ft/README.md

@@ -1,38 +0,0 @@
-# YG-FT Compute 数据目录说明
-
-本目录挂载到 `yg-ft-compute-api` 容器的 `/data/yg-ft`，用于持久化存储训练相关的数据。
-
-## 目录结构
-
-```
-data/yg-ft/
-├── datasets/     # 数据集存储目录
-├── models/       # 模型文件存储目录
-├── outputs/      # 训练/推理输出结果目录
-└── logs/
-    ├── compute/  # 计算服务运行日志
-    └── training/ # 训练任务执行日志
-```
-
-## 各目录说明
-
-### datasets/
-训练和评估所使用的数据集文件，包括 JSON、JSONL、CSV 等格式。数据集由用户上传或通过平台创建，供 LLaMA-Factory 等训练引擎读取。
-
-### models/
-存放模型文件，包括：
-- 预训练基座模型（如 LLaMA、Qwen 等）
-- 微调后的自定义模型权重
-- 合并后的部署模型
-
-### outputs/
-训练任务和推理任务的输出结果，包括：
-- 训练过程中的 checkpoint 文件
-- 评估结果和指标报告
-- 推理生成的结果文本
-
-### logs/compute/
-计算服务（compute-api）的运行时日志，用于排查服务启动、GPU 调度、健康检查等问题。
-
-### logs/training/
-各训练任务的执行日志，记录训练过程状态、报错信息等，便于追踪单个任务的运行情况。

docker/compute/data/yg-ft/datasets/.gitignore

@@ -1,2 +0,0 @@
-*
-!.gitignore

docker/compute/data/yg-ft/logs/compute/.gitignore

@@ -1,2 +0,0 @@
-*
-!.gitignore

docker/compute/data/yg-ft/logs/training/.gitignore

@@ -1,2 +0,0 @@
-*
-!.gitignore

docker/compute/data/yg-ft/models/.gitignore

@@ -1,2 +0,0 @@
-*
-!.gitignore

docker/compute/data/yg-ft/outputs/.gitignore

@@ -1,2 +0,0 @@
-*
-!.gitignore

docker/compute/docker-compose.yml

@@ -5,26 +5,15 @@ services:
     gpus: all
     ports:
       - "${COMPUTE_API_PORT:-19100}:9100"
-      - "${FILE_GATEWAY_PORT:-19101}:9100"
     environment:
       COMPUTE_ENV: ${COMPUTE_ENV:-prod}
       COMPUTE_HOST_ID: ${COMPUTE_HOST_ID:-gpu-node-01}
       COMPUTE_EXECUTION_MODE: ${COMPUTE_EXECUTION_MODE:-real}
       MODELTF_ROUTE_PREFIX: ${MODELTF_ROUTE_PREFIX:-/modelTF}
-      COMPUTE_AUTH_ENABLED: ${COMPUTE_AUTH_ENABLED:-true}
       COMPUTE_SERVICE_TOKEN: ${COMPUTE_SERVICE_TOKEN:-change_me}
       ENABLE_APP_CALLBACK: ${ENABLE_APP_CALLBACK:-false}
       LLAMA_FACTORY_HOME: ${LLAMA_FACTORY_HOME:-/app/LLaMA-Factory}
       YG_FT_DATA_ROOT: ${YG_FT_DATA_ROOT:-/data/yg-ft}
-      YG_FT_MODEL_ROOT: ${YG_FT_MODEL_ROOT:-/data/yg-ft/models}
-      YG_FT_DATASET_ROOT: ${YG_FT_DATASET_ROOT:-/data/yg-ft/datasets}
-      YG_FT_OUTPUT_ROOT: ${YG_FT_OUTPUT_ROOT:-/data/yg-ft/outputs}
-      TRAINING_LOG_ROOT: ${TRAINING_LOG_ROOT:-/opt/yg-ft/logs/training}
-      COMPUTE_GPU_COUNT: ${COMPUTE_GPU_COUNT:-0}
-      COMPUTE_GPU_NAME: ${COMPUTE_GPU_NAME:-NVIDIA A800-SXM4-80GB}
-      COMPUTE_GPU_MEMORY_GB: ${COMPUTE_GPU_MEMORY_GB:-80}
-      COMPUTE_GPU_POWER_LIMIT_W: ${COMPUTE_GPU_POWER_LIMIT_W:-300}
-      MIN_TRAINING_GPU_MEMORY_GB: ${MIN_TRAINING_GPU_MEMORY_GB:-4}
       LOG_DIR: ${LOG_DIR:-/opt/yg-ft/logs/compute}
       CUDA_VISIBLE_DEVICES: ${CUDA_VISIBLE_DEVICES:-all}
       NVIDIA_VISIBLE_DEVICES: ${NVIDIA_VISIBLE_DEVICES:-all}
@@ -32,12 +21,9 @@ services:
       PYTHONPATH: /app
     volumes:
       - ../../compute:/app/compute:ro
-      - ${YG_FT_DATA_ROOT_HOST:-./data/yg-ft}:${YG_FT_DATA_ROOT:-/data/yg-ft}
-      - ${YG_FT_MODEL_ROOT_HOST:-./data/yg-ft/models}:${YG_FT_MODEL_ROOT:-/data/yg-ft/models}
-      - ${YG_FT_DATASET_ROOT_HOST:-./data/yg-ft/datasets}:${YG_FT_DATASET_ROOT:-/data/yg-ft/datasets}
-      - ${YG_FT_OUTPUT_ROOT_HOST:-./data/yg-ft/outputs}:${YG_FT_OUTPUT_ROOT:-/data/yg-ft/outputs}
-      - ${COMPUTE_LOG_ROOT_HOST:-./data/yg-ft/logs/compute}:${LOG_DIR:-/opt/yg-ft/logs/compute}
-      - ${TRAINING_LOG_ROOT_HOST:-./data/yg-ft/logs/training}:${TRAINING_LOG_ROOT:-/opt/yg-ft/logs/training}
+      - ${YG_FT_DATA_ROOT_HOST:-/data/yg-ft}:${YG_FT_DATA_ROOT:-/data/yg-ft}
+      - ../../runtime/compute/logs:/opt/yg-ft/logs/compute
+      - ../../runtime/compute/training-logs:/opt/yg-ft/logs/training
     networks:
       - yg-ft-compute
     healthcheck:

docker/nginx.conf.template

@@ -7,18 +7,11 @@ server {
 
   client_max_body_size 200m;
 
-  location ^~ /modelTF/ {
-    proxy_pass ${BACKEND_PROXY_PASS};
-    proxy_http_version 1.1;
-    proxy_set_header Host $host;
-    proxy_set_header X-Real-IP $remote_addr;
-    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-    proxy_set_header X-Forwarded-Proto $scheme;
-    proxy_read_timeout 300s;
-    proxy_send_timeout 300s;
+  location / {
+    try_files $uri $uri/ /index.html;
   }
 
-  location = /modelTF {
+  location /modelTF {
     proxy_pass ${BACKEND_PROXY_PASS};
     proxy_http_version 1.1;
     proxy_set_header Host $host;
@@ -34,8 +27,4 @@ server {
     expires 30d;
     add_header Cache-Control "public, immutable";
   }
-
-  location / {
-    try_files $uri $uri/ /index.html;
-  }
 }

docs/2026-07-23-development-summary.md

@@ -1,153 +0,0 @@
-# 2026-07-23 开发总结
-
-本文档汇总 2026-07-23 当天围绕模型微调平台完成的系统开发内容。当前开发不再按临时 Demo 口径处理，以下能力均按后续可继续演进到生产系统的代码和数据结构推进。
-
-## 1. 今日完成内容
-
-### 1.1 模型训练最小闭环增强
-
-| 功能点 | 作用 | 对应页面/模块 | 用户如何操作 |
-| --- | --- | --- | --- |
-| 训练任务创建前预检 | 在任务进入运行前校验模型、数据集、算力节点、路径和 LLaMA-Factory 参数，减少任务启动后才失败的情况 | 模型微调列表、创建模型微调、训练日志 | 在“模型微调”页面创建任务并提交，系统自动执行预检；失败时页面提示具体原因 |
-| 训练数据同步到算力节点 | 将应用侧选择的数据集文件同步到目标 Compute 节点本地磁盘，保证 LLaMA-Factory 能读取到真实训练文件 | 创建模型微调、训练日志、算力节点/资源副本 | 用户选择数据集后提交训练任务，系统自动同步数据集到算力节点，无需手工进入容器复制 |
-| 动态生成训练参数 | 根据用户在页面选择的训练数据、基座模型、训练方法、输出目录等动态生成 LLaMA-Factory 训练参数 | 创建模型微调、训练日志 | 用户在创建任务表单中选择模型和数据集，启动训练后可在训练日志中查看实际执行命令 |
-| 训练日志和状态轮询 | 应用侧主动轮询 Compute API 获取训练状态、进度、日志、指标和 checkpoint | 训练日志详情页 | 用户进入训练日志页，可查看任务状态、日志输出、指标和 checkpoint 信息 |
-| 训练失败信息回传 | 训练命令失败后将失败状态和日志内容回传应用侧，便于定位数据格式、模型路径或参数问题 | 模型微调列表、训练日志详情页 | 任务失败后，用户进入训练日志页查看失败日志和错误堆栈 |
-
-### 1.2 B. 模型资产、训练与 LLaMA-Factory 任务能力
-
-| 功能点 | 作用 | 对应页面/模块 | 用户如何操作 |
-| --- | --- | --- | --- |
-| 训练模型产物 artifact 展示 | 展示训练/合并产生的 Adapter、合并模型、量化模型等产物路径、大小、checksum | 模型管理 -> 训练模型列表 | 用户进入“模型管理”，切换到“训练模型”，展开某个模型行查看“模型产物” |
-| 模型血缘 lineage 展示 | 记录和展示基座模型、训练模型、合并产物之间的来源关系 | 模型管理 -> 训练模型列表 | 用户展开训练模型行，在“模型血缘”区域查看父资源、子资源和对应 Compute Job |
-| 模型导出任务 export job 状态展示 | 展示模型合并/导出的任务状态、输出目录和创建时间 | 模型管理 -> 训练模型列表 | 用户展开训练模型行，在“导出任务”区域查看导出任务状态 |
-| export 完成后回填 checksum | Compute export 完成后扫描产物文件，回填大小和 `checksum_sha256`，为后续一致性校验和制品治理做准备 | 后端模型资产模块、模型管理页面 | 用户无需手工操作，任务完成后页面自动展示产物大小和 checksum |
-| 合并模型任务接入 Compute | 模型合并不再只是生成模拟任务，而是通过 Compute 节点调用模型合并任务并记录导出任务 | 模型管理 -> 合并权重 | 用户在训练模型列表点击“合并权重”，提交后系统创建 Compute 合并任务 |
-
-### 1.3 D. 算力平台、部署与运维能力
-
-| 功能点 | 作用 | 对应页面/模块 | 用户如何操作 |
-| --- | --- | --- | --- |
-| 算力节点动态配置 | 支持维护 Compute API、File Gateway、权重、标签、启用状态、调度状态等信息 | 算力节点 -> 节点 | 用户进入“算力节点”，点击“新增节点”或“编辑”，填写节点地址和调度参数 |
-| 节点连通性测试 | 验证应用服务器是否能访问 Compute API，并返回 GPU 发现结果和延迟 | 算力节点 -> 节点 | 用户在节点列表点击“测试”，查看连接成功或失败提示 |
-| GPU 状态展示 | 展示每个算力节点的 GPU 利用率、显存、温度、功耗、进程信息 | 算力节点 -> GPU | 用户进入“算力节点”，切换到“GPU”页签查看 |
-| 训练队列展示 | 展示 Compute 侧任务队列、状态、进度、节点和 GPU 分配 | 算力节点 -> 队列 | 用户切换到“队列”页签查看当前运行或等待任务 |
-| 资源副本列表 | 展示数据集、模型、训练产物在算力节点本地磁盘上的副本路径和同步状态 | 算力节点 -> 资源副本 | 用户切换到“资源副本”页签，选择目标节点查看本地副本 |
-| 资源副本漂移检测 | 检查副本路径是否仍存在、大小是否可读取，并将异常副本标记为漂移 | 算力节点 -> 资源副本 | 用户点击“漂移检测”，系统调用 Compute API 检查节点本地路径 |
-| 资源副本 repair 重同步 | 对漂移、失败、待修复副本执行真实重同步，而不只是标记状态 | 算力节点 -> 资源副本 | 用户点击“修复副本”，系统从权威源重新上传或导入到目标算力节点 |
-| 应用侧轮询模式 | 应用服务主动轮询 Compute API，避免需要 Compute 侧反向访问应用侧 | 后端 Compute Gateway、部署架构 | 用户无感知，部署时只需保证应用侧能访问各 Compute 节点 |
-| 本地磁盘挂载约定 | 基座模型、训练数据、训练产物、训练日志均通过宿主机目录挂载到 Compute 容器 | docker/compute 部署 | 运维人员在启动 Compute 服务前准备并挂载 `docker/compute/data/yg-ft/*` 目录 |
-
-### 1.4 前端状态中文化
-
-| 功能点 | 作用 | 对应页面/模块 | 用户如何操作 |
-| --- | --- | --- | --- |
-| 统一状态字典 | 将 `pending`、`running`、`completed`、`failed`、`online`、`synced`、`drifted` 等状态统一展示为中文 | 通用前端组件 `ModelStatusTag`、状态工具 `status.ts` | 用户在各列表页看到中文状态，不再直接看到英文状态值 |
-| 模型管理状态中文化 | 合并状态、导出任务状态、产物状态使用中文展示 | 模型管理 | 用户查看训练模型列表和展开详情时看到“未合并/合并中/已合并/已完成/失败”等中文 |
-| 算力节点状态中文化 | 节点状态、GPU 状态、队列状态、副本同步状态使用中文展示 | 算力节点 | 用户查看“节点/GPU/队列/资源副本”时看到中文状态标签 |
-| 推理/对比/评测/用户状态中文化 | 将推理任务、对比模型加载、评测任务、用户状态统一为中文标签 | 模型推理、模型对比、模型评测、用户设置 | 用户查看相关页面时看到“启动中/已就绪/已完成/启用”等中文状态 |
-
-### 1.5 前端构建产物更新
-
-| 功能点 | 作用 | 对应页面/模块 | 用户如何操作 |
-| --- | --- | --- | --- |
-| 刷新 `frontend/dist` | 保证 Docker 前端 nginx 容器可以直接加载最新页面代码 | 前端部署 | 用户重新访问前端服务即可看到最新页面 |
-| 重启前后端服务验证 | 验证源码挂载和 dist 挂载方式下服务可正常加载最新代码 | docker/app、docker/compute | 开发人员重启 `backend-api`、`frontend`、`compute-api` 后验证页面和接口 |
-
-## 2. 今日涉及的主要代码位置
-
-| 文件 | 说明 |
-| --- | --- |
-| `backend/app/api/v1/endpoints/platform.py` | 新增/完善训练预检、资源同步、模型 artifact/lineage/export job 接口、资源副本漂移检测和 repair |
-| `backend/app/db/platform_store.py` | 新增模型产物、血缘、导出任务、副本同步结果、artifact 查询等数据访问能力 |
-| `compute/agent/process_manager.py` | export/artifact 扫描时补充文件大小和 checksum |
-| `frontend/src/api/modules/model.ts` | 增加训练模型 artifact、lineage、export job API |
-| `frontend/src/api/modules/compute.ts` | 增加副本漂移检测、repair API 和副本字段 |
-| `frontend/src/views/model/ModelManageView.vue` | 增加训练模型展开详情：产物、血缘、导出任务 |
-| `frontend/src/views/compute/ComputeNodesView.vue` | 增加资源副本页签、漂移检测、repair 操作和状态中文化 |
-| `frontend/src/utils/status.ts` | 新增统一状态中文映射和标签类型映射 |
-| `frontend/src/components/ModelStatusTag.vue` | 改为复用统一状态字典 |
-| `frontend/src/components/DataTablePage.vue` | 增加表格展开事件支持 |
-| `frontend/dist/*` | 前端生产构建产物已更新 |
-
-## 3. 今日验证结果
-
-| 验证项 | 结果 |
-| --- | --- |
-| Python 编译检查 | 通过：`python -m compileall backend compute scripts` |
-| 前端生产构建 | 通过：`npm run build` |
-| 前端入口 | 通过：`http://127.0.0.1:16801/` 返回 200 |
-| 后端健康检查 | 通过：`/modelTF/health` 返回正常 |
-| Compute 健康检查 | 通过：`/modelTF/health` 返回正常 |
-| 模型导出任务接口 | 通过：`/modelTF/model-manage/export-jobs` 返回正常 |
-| 算力节点列表接口 | 通过：`/modelTF/compute/nodes` 返回正常 |
-| 资源副本漂移检测 | 通过：当前节点副本检测返回 `drifted: 0` |
-
-## 4. 当前仍需注意的问题
-
-| 问题 | 影响 | 建议 |
-| --- | --- | --- |
-| 训练数据格式仍依赖 LLaMA-Factory 约定 | 如果用户上传的数据字段不符合模板要求，训练仍会失败 | 下一步增加数据集格式校验和模板转换预检 |
-| 单机 MX350 显存较小 | 真实训练大模型时容易因显存不足失败 | 当前环境用于链路验证；真实训练应使用高显存 GPU 节点 |
-| resource replica repair 对大型模型仍是同步调用 | 大模型重同步可能耗时较长 | 下一步将 repair 完整异步化，并展示 sync job 进度 |
-| artifact checksum 目前在 Compute 扫描阶段计算 | 对超大目录递归扫描可能较慢 | 下一步支持分文件 checksum、manifest 文件和后台扫描 |
-| 模型评测、模型推理仍未形成完整生产闭环 | 目前页面已有基础能力，但后端表结构、任务运行、日志、治理能力还需补齐 | 下一步将评测和推理纳入正式开发计划 |
-
-## 5. 下一步开发计划
-
-### 5.1 B. 模型资产、训练与 LLaMA-Factory 任务
-
-| 优先级 | 开发任务 | 目标页面/模块 | 交付结果 |
-| --- | --- | --- | --- |
-| P0 | 数据集格式预检 | 创建模型微调、数据集管理 | 提交训练前校验 Alpaca/ShareGPT/OpenAI Messages 等格式，提示缺失字段 |
-| P0 | 训练参数可视化确认 | 创建模型微调 | 提交前展示最终 LLaMA-Factory 参数预览，减少参数不一致问题 |
-| P0 | 训练任务失败诊断 | 训练日志详情 | 识别常见错误：数据字段缺失、模型路径不存在、显存不足、依赖缺失，并生成中文诊断 |
-| P1 | checkpoint 管理 | 训练日志详情、模型管理 | 展示 checkpoint 列表、大小、路径、保留策略，支持标记最佳 checkpoint |
-| P1 | 模型产物 manifest | 模型管理 | 为每个训练/导出产物生成 manifest，记录文件清单、大小、checksum、来源任务 |
-| P1 | 合并/导出任务详情页 | 模型管理、训练日志 | 展示合并任务日志、状态、产物、失败原因 |
-| P2 | 模型版本治理 | 模型管理 | 支持版本号、标签、发布状态、归档状态、审批状态 |
-| P2 | 训练模板管理 | 系统设置或训练配置 | 将 LLaMA-Factory 模板、数据格式、默认超参做成可维护配置 |
-
-### 5.2 D. 算力平台、部署与运维
-
-| 优先级 | 开发任务 | 目标页面/模块 | 交付结果 |
-| --- | --- | --- | --- |
-| P0 | repair 异步化 | 算力节点 -> 资源副本 | repair 创建 sync job 后后台执行，页面展示进度和失败原因 |
-| P0 | 多算力节点调度策略 | 算力节点、创建模型微调 | 支持按标签、权重、空闲 GPU、显存要求选择节点 |
-| P1 | 节点资源水位告警 | 算力节点、硬件监控 | 展示磁盘、GPU、显存、训练日志目录水位和告警状态 |
-| P1 | 节点维护窗口 | 算力节点 | 支持维护中节点不再调度新任务，已有任务可继续或迁移 |
-| P1 | 文件副本治理 | 算力节点 -> 资源副本 | 支持副本清理、重建、过期策略和跨节点一致性检查 |
-| P2 | 部署健康巡检脚本 | 部署运维 | 一键检查 app、backend、redis、pg、compute、GPU、挂载目录、端口连通 |
-| P2 | Compute Agent 插件标准 | 算力平台 | 抽象 LLaMA-Factory 接入规范，预留其他训练框架 |
-
-### 5.3 模型评测页面开发计划
-
-| 优先级 | 开发任务 | 目标页面/模块 | 交付结果 |
-| --- | --- | --- | --- |
-| P0 | 评测任务表结构和接口补齐 | 模型评测列表、创建评测、评测详情 | 建立 `eval_tasks`、`eval_dimensions`、`eval_sample_results` 等运行表和接口 |
-| P0 | 创建评测任务真实提交 | 创建评测 | 支持选择模型、数据集、评测维度、GPU/节点，提交后生成评测任务 |
-| P0 | 评测任务运行闭环 | 评测详情 | 支持状态、进度、日志、样本级结果回传 |
-| P1 | 评测维度管理 | 评测维度创建/编辑 | 支持规则、Prompt、评分器、权重、适用数据集配置 |
-| P1 | 样本级评分展示 | 评测详情 | 展示每条样本的输入、模型输出、评分、原因、人工复核状态 |
-| P1 | 综合报告生成 | 评测详情、排行榜 | 生成维度汇总、综合分、问题样本、改进建议 |
-| P2 | 评测审批和审计 | 审批流、审计日志 | 评测任务创建、发布报告、删除报告纳入治理 |
-
-### 5.4 模型推理页面开发计划
-
-| 优先级 | 开发任务 | 目标页面/模块 | 交付结果 |
-| --- | --- | --- | --- |
-| P0 | 推理任务表结构和接口补齐 | 模型推理列表、新建推理 | 建立 `inference_tasks`、`inference_task_models`、`chat_sessions`、`chat_messages` |
-| P0 | 模型加载/卸载真实闭环 | 模型推理列表 | 支持选择训练产物加载推理服务，展示加载状态、端口、进程和错误 |
-| P0 | 单模型对话持久化 | 模型对话 | 保存会话、消息、参数、响应耗时、token 统计 |
-| P1 | 多模型对比任务 | 模型对比 | 支持多模型同时加载、同一问题并发请求、结果对比展示 |
-| P1 | 推理资源管控 | 模型推理、算力节点 | 支持 GPU 选择、并发限制、空闲自动卸载、异常进程清理 |
-| P1 | 推理日志和调用审计 | 日志、审计中心 | 记录加载、卸载、对话请求、失败原因、用户和租户信息 |
-| P2 | 推理服务发布 | 模型管理、模型推理 | 支持将某个训练模型发布为内部推理服务，并配置访问权限 |
-
-## 6. 建议的下一阶段顺序
-
-1. 先完成 B+D 的训练稳定性增强：数据格式预检、训练参数预览、失败诊断、repair 异步化。
-2. 再补齐模型评测的真实任务闭环：任务表、创建任务、运行状态、样本结果。
-3. 然后补齐模型推理闭环：加载/卸载、对话持久化、多模型对比。
-4. 最后统一治理能力：审批、审计、租户隔离、资源配额、保留策略和运维巡检。
-

docs/2026-07-24-work-plan.md

@@ -1,281 +0,0 @@
-# 2026-07-24 工作计划
-
-本文基于 `docs/2026-07-23-development-summary.md`、当前 B+D 开发进度，以及 2026-07-24 已完成的训练预检、失败诊断、资源修复异步化能力整理。后续开发仍按正式系统演进推进，不以临时演示能力作为交付标准。
-
-## 1. 当前完成基线
-
-### 1.1 训练创建与预检
-
-- 对应页面：`模型微调 / 创建训练任务`
-- 已完成能力：
-  - 创建训练前调用训练预检接口。
-  - 展示真实 LLaMA-Factory 命令预览。
-  - 展示预检节点、错误、警告和中文诊断建议。
-  - 后端预检不创建任务、不落库、不占用调度锁。
-  - 后端兼容 `base_model/train_dataset_id` 和 `model_id/dataset_id` 两套字段。
-- 对应接口：
-  - `POST /modelTF/fine-tune/preflight`
-  - `POST /modelTF/fine-tune/command-preview`
-
-### 1.2 训练日志与失败诊断
-
-- 对应页面：`系统日志 / 训练日志`
-- 已完成能力：
-  - 训练失败或停止后可查询诊断建议。
-  - 根据训练日志和失败原因识别模型路径、数据集字段、CUDA/GPU、LLaMA-Factory 命令等常见问题。
-  - 页面以中文展示失败诊断。
-- 对应接口：
-  - `GET /modelTF/fine-tune/{task_id}/diagnostics`
-
-### 1.3 算力节点与资源修复
-
-- 对应页面：`算力平台 / 算力节点`
-- 已完成能力：
-  - 算力节点可配置、可测试连接、可查看 GPU 与健康状态。
-  - 资源副本 repair 改为异步提交。
-  - 页面展示资源同步任务进度，避免长请求阻塞页面。
-- 对应接口：
-  - `POST /modelTF/compute/nodes/{node_id}/replicas/repair`
-  - `GET /modelTF/compute/sync-jobs/{sync_id}`
-
-## 2. 未完成任务清单
-
-### P0：真实训练成功闭环
-
-- 对应页面：
-  - `模型管理 / 新增模型`
-  - `数据集管理 / 上传数据集`
-  - `模型微调 / 创建训练任务`
-  - `系统日志 / 训练日志`
-- 未完成内容：
-  - 基座模型必须支持算力服务器本地路径校验，避免选择 API 模型或应用侧路径后进入训练。
-  - 数据集必须支持上传后格式校验，提前发现 Alpaca、ShareGPT、DPO、CPT 字段不匹配问题。
-  - 训练成功后需要完成模型产物扫描、产物入库、训练任务状态回填。
-  - 训练失败时需要强制拉取最后日志片段，保证失败原因可见。
-- 后端开发：
-  - 增强模型路径校验，明确区分 `本地训练模型`、`API 模型`、`已训练模型`。
-  - 增加数据集格式校验服务，支持字段级错误返回。
-  - 完善训练任务完成后的 artifact 回填逻辑。
-- Compute 开发：
-  - 训练结束后扫描输出目录。
-  - 返回产物列表、文件大小、目录结构、训练日志路径。
-- 验收标准：
-  - 使用算力节点可访问的本地模型路径和合法数据集，可以完成一次真实 LLaMA-Factory 训练。
-  - 训练成功后页面能看到完成状态、输出目录、模型产物。
-  - 训练失败时页面能看到中文诊断和最后错误日志。
-
-### P0：LLaMA-Factory 参数映射完善
-
-- 对应页面：`模型微调 / 创建训练任务`
-- 未完成内容：
-  - SFT、DPO、CPT 参数映射仍需细化。
-  - LoRA、Full、QLoRA、量化导出参数需要按训练方式校验。
-  - 不同模板与数据集格式之间的兼容关系需要预检。
-- 后端开发：
-  - 建立训练参数标准化层。
-  - 建立训练方式到 LLaMA-Factory 参数的映射表。
-  - 对无效组合返回中文错误，例如 DPO 缺少 rejected 字段、CPT 不应使用 instruction/output 格式等。
-- 验收标准：
-  - 页面选择不同训练方式时，预检能返回准确命令。
-  - 无效参数组合不能启动训练。
-
-### P1：资源副本 repair 自动重同步
-
-- 对应页面：`算力平台 / 算力节点`
-- 未完成内容：
-  - repair 当前已异步化，但还需要基于权威源路径自动重同步。
-  - 修复完成后需要重新校验副本状态。
-- 后端开发：
-  - 为模型、数据集、训练产物定义权威源路径。
-  - repair job 根据权威源自动发起重传或重新扫描。
-  - 修复完成后更新 replica 状态、checksum、错误原因。
-- Compute 开发：
-  - 支持接收重同步请求。
-  - 支持按资源类型写入目标路径并返回校验信息。
-- 验收标准：
-  - 将副本标记为异常后，点击修复可自动完成重同步并恢复为正常。
-
-### P1：artifact checksum 与 manifest
-
-- 对应页面：
-  - `模型管理 / 已训练模型`
-  - `模型管理 / 模型详情`
-  - `算力平台 / 资源副本`
-- 未完成内容：
-  - artifact checksum 目前仍是预留字段。
-  - 大目录需要 manifest 文件，避免每次递归扫描成本过高。
-- 后端开发：
-  - 增加 artifact checksum 回填逻辑。
-  - 增加 manifest 解析和存储字段。
-- Compute 开发：
-  - export 或训练完成后扫描文件并生成 checksum。
-  - 对大模型目录生成 manifest。
-- 验收标准：
-  - 模型产物列表能展示 checksum、大小、文件数、生成时间。
-  - 副本校验可以基于 checksum 判断一致性。
-
-### P1：模型导出闭环
-
-- 对应页面：
-  - `模型管理 / 已训练模型`
-  - `模型管理 / 导出任务`
-  - `模型管理 / 合并权重`
-- 未完成内容：
-  - 导出任务、量化导出、导出日志、失败重试、产物下载仍需完善。
-- 后端开发：
-  - 完善 export job 创建、查询、取消、重试接口。
-  - 导出完成后登记 artifact 和 lineage。
-- Compute 开发：
-  - 支持 LoRA 合并、GGUF/量化导出、导出日志回传。
-- 验收标准：
-  - 已训练模型可发起导出。
-  - 导出状态、日志、产物可在页面查看。
-
-### P1：多算力节点调度增强
-
-- 对应页面：`算力平台 / 算力节点`
-- 未完成内容：
-  - 当前已有节点配置和基础调度，但生产级调度策略仍需增强。
-  - 需要支持节点标签、权重、启用状态、容量、手动指定节点。
-- 后端开发：
-  - 增强调度策略：标签匹配、权重、当前任务数、GPU 占用、显存约束。
-  - 增加任务排队和等待原因。
-  - 增加 GPU 分配释放的异常恢复。
-- 前端开发：
-  - 创建训练时支持可选手动指定节点。
-  - 算力节点页展示容量、排队数、当前任务。
-- 验收标准：
-  - 多节点时可以自动选择合适节点。
-  - 节点不可用时页面能明确展示不可调度原因。
-
-### P2：训练日志实时性优化
-
-- 对应页面：`系统日志 / 训练日志`
-- 未完成内容：
-  - 当前依赖应用侧轮询，日志实时性和失败最后日志仍需增强。
-- 后端开发：
-  - 支持日志 offset/tail 增量读取。
-  - 任务失败时强制同步最后日志片段。
-  - 日志接口返回来源、偏移量、是否截断。
-- 前端开发：
-  - 日志页按 offset 增量刷新。
-  - 失败时自动跳到底部并展示最后错误。
-- 验收标准：
-  - 训练过程中日志持续刷新。
-  - 失败后无需手动刷新即可看到最后错误。
-
-### P2：权限、审计和治理落点补齐
-
-- 对应页面：
-  - `用户中心`
-  - `项目管理`
-  - `模型管理`
-  - `数据集管理`
-  - `模型微调`
-  - `算力平台`
-- 未完成内容：
-  - 训练链路中的租户、项目、用户权限校验还需要细粒度补齐。
-  - 审计事件需要覆盖训练创建、启动、停止、删除、导出、资源修复。
-- 后端开发：
-  - 接口增加项目/租户上下文校验。
-  - 增加审计事件写入。
-  - 删除和高风险操作进入审批流。
-- 验收标准：
-  - 用户只能访问授权项目内的模型、数据集和训练任务。
-  - 关键操作可以在审计日志中查询。
-
-### P2：模型评测页面真实闭环
-
-- 对应页面：`模型评测`
-- 未完成内容：
-  - 评测任务创建、运行、日志、指标、结果对比仍需接入真实后端。
-- 后端开发：
-  - 评测任务表、评测指标表、评测日志接口。
-  - 支持指定模型、数据集、评测模板和指标。
-- Compute 开发：
-  - 支持评测任务执行器。
-  - 返回指标结果和日志。
-- 验收标准：
-  - 可创建评测任务并看到运行状态、指标结果和失败原因。
-
-### P2：模型推理页面真实闭环
-
-- 对应页面：
-  - `模型推理 / 推理服务`
-  - `模型推理 / 对话测试`
-  - `模型对比`
-- 未完成内容：
-  - 推理服务启动、停止、健康检查、会话请求、资源释放仍需完善。
-- 后端开发：
-  - 推理服务实例管理接口。
-  - 对话请求代理接口。
-  - 推理日志和资源占用查询。
-- Compute 开发：
-  - 支持启动本地模型推理服务。
-  - 支持停止服务和释放 GPU。
-- 验收标准：
-  - 可从页面启动一个已训练模型的推理服务。
-  - 可进行对话测试并查看服务状态。
-
-## 3. 推荐开发顺序
-
-1. 完成真实训练成功闭环。
-2. 完成数据集格式校验和 LLaMA-Factory 参数映射。
-3. 完成 artifact、checksum、manifest 和模型导出闭环。
-4. 完成资源副本 repair 自动重同步。
-5. 完成多算力节点调度增强。
-6. 完成训练日志实时性优化。
-7. 补齐权限、审批、审计治理落点。
-8. 启动模型评测真实闭环开发。
-9. 启动模型推理真实闭环开发。
-
-## 4. 下一轮优先执行任务
-
-### 任务 1：训练模型路径治理
-
-- 页面：`模型管理 / 新增模型`、`模型微调 / 创建训练任务`
-- 内容：
-  - 新增模型时区分是否可用于训练。
-  - API 模型不能作为 LLaMA-Factory 本地训练基座。
-  - 本地模型路径必须是算力节点可访问路径。
-- 验收：
-  - 选择不可训练模型时，训练创建页预检直接给出中文错误。
-
-### 任务 2：数据集格式校验
-
-- 页面：`数据集管理 / 上传数据集`、`模型微调 / 创建训练任务`
-- 内容：
-  - 上传后扫描样本字段。
-  - 支持 Alpaca、ShareGPT、DPO、CPT 校验。
-  - 返回字段缺失、类型错误、空样本等问题。
-- 验收：
-  - `111.json` 这类数据可以明确判断是否满足当前训练模板。
-
-### 任务 3：训练完成产物入库
-
-- 页面：`模型微调 / 任务列表`、`模型管理 / 已训练模型`
-- 内容：
-  - Compute 训练成功后返回输出目录。
-  - 应用侧轮询后创建 trained model 记录。
-  - 写入 artifact、lineage、export job 初始状态。
-- 验收：
-  - 训练完成后无需手动登记，模型管理中自动出现新模型。
-
-### 任务 4：失败日志最后片段拉取
-
-- 页面：`系统日志 / 训练日志`
-- 内容：
-  - 任务失败时立即拉取最后 N 行日志。
-  - 页面展示最后错误、诊断建议和原始日志。
-- 验收：
-  - 训练失败后页面不再只看到“失败”，可以直接看到失败原因。
-
-## 5. 当前测试注意事项
-
-- 当前环境中已有算力节点可访问，但 GPU 为 2GB 显存，预检会提示显存不足，这是符合预期的生产校验结果。
-- 若要验证真实训练成功，需要提前准备：
-  - 算力节点可访问的本地基座模型目录。
-  - 合法训练数据集文件。
-  - 足够显存的 GPU。
-  - Compute 容器内可用的 LLaMA-Factory 和 `llamafactory-cli`。
-- 当前前端 `dist` 已按要求参与构建更新，后续修改前端页面后需要重新执行 `npm run build`。

docs/backend-api-design.md

@@ -271,7 +271,7 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
 | POST | `/modelTF/dataset-manage` | 创建数据集 |
 | PUT | `/modelTF/dataset-manage/{id}` | 更新数据集 |
 | DELETE | `/modelTF/dataset-manage/{id}` | 删除数据集 |
-| POST | `/modelTF/dataset-manage/upload/{dataset_id}` | 上传文件，字段名 `files`；默认同步到启用的算力节点 `/data/yg-ft/datasets/{dataset_id}/` |
+| POST | `/modelTF/dataset-manage/upload/{dataset_id}` | 上传文件，字段名 `files` |
 | GET | `/modelTF/dataset-manage/download/{dataset_id}` | 打包下载数据集 |
 | GET | `/modelTF/dataset-manage/download/{dataset_id}/{file_id}` | 下载单文件 |
 
@@ -451,9 +451,7 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
 | GET | `/modelTF/fine-tune/{id}` | 训练任务详情 |
 | GET | `/modelTF/fine-tune/check-name?name=xxx` | 任务名查重 |
 | POST | `/modelTF/fine-tune` | 创建训练任务记录 |
-| POST | `/modelTF/fine-tune/{id}/command-preview` | 训练创建页/详情页命令预览，返回目标节点、Compute Job payload 和 LLaMA-Factory 命令 |
-| POST | `/modelTF/fine-tune/{id}/preflight` | 训练创建页启动前预检，校验节点、模型路径、数据集路径、引擎命令和训练参数 |
-| POST | `/modelTF/fine-tune/start` | 启动训练，应用侧选择算力节点并提交 Compute Job |
+| POST | `/modelTF/fine-tune/start` | 启动训练 |
 | PUT | `/modelTF/fine-tune/{id}` | 更新任务 |
 | POST | `/modelTF/fine-tune/stop/{id}` | 停止任务 |
 | DELETE | `/modelTF/fine-tune/{id}` | 删除任务 |
@@ -496,49 +494,6 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
 }
 ```
 
-训练启动前检查和命令预览：
-
-- 页面模块：`/fine-tune/create` 创建训练任务的“参数确认/启动训练”区域；`/training-log/:id` 训练详情页的“任务配置/命令查看”区域。
-- `POST /modelTF/fine-tune/{id}/command-preview`：不做远端路径强校验，只返回应用侧调度出的算力节点、标准 Compute Job payload、训练引擎命令和工作目录，供前端展示最终 LLaMA-Factory 启动命令。
-- `POST /modelTF/fine-tune/{id}/preflight`：启动前强校验，真实 `llama_factory` 会检查目标节点连通性、模型路径、数据集目录、LLaMA-Factory HOME、训练命令是否可用；`smoke` 引擎用于自动化闭环验收，会跳过模型/数据集路径检查。
-- `POST /modelTF/fine-tune/start`：内部先执行 preflight，预检失败返回 `409` 且任务保持 `pending`，预检通过后再写入 `syncing/queued/running` 运行态并提交 Compute Job。
-
-请求体可传启动覆盖参数：
-
-```json
-{
-  "requested_node_id": "node_xxx",
-  "gpus": [0],
-  "batch_size": 1,
-  "learning_rate": 0.0002,
-  "n_epochs": 1
-}
-```
-
-响应结构：
-
-```json
-{
-  "valid": true,
-  "errors": [],
-  "warnings": [],
-  "node": {
-    "id": "node_xxx",
-    "code": "gpu-node-01",
-    "scheduler_status": "online",
-    "gpu_count": 1
-  },
-  "job_payload": {},
-  "preview": {
-    "engine": "llama_factory",
-    "command": ["llamafactory-cli", "train", "..."],
-    "command_text": "llamafactory-cli train ...",
-    "work_dir": "/app/LLaMA-Factory",
-    "path_checks": []
-  }
-}
-```
-
 ### 7.2 训练日志详情页
 
 训练日志页还会联合调用：
@@ -886,7 +841,7 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
 | POST | `/modelTF/compute/nodes` | 新增算力节点 |
 | GET | `/modelTF/compute/nodes/{id}` | 算力节点详情 |
 | PUT | `/modelTF/compute/nodes/{id}` | 编辑节点地址、权重、标签、路径和启用状态 |
-| POST | `/modelTF/compute/nodes/{id}/test-connection` | 测试 Compute API/File Gateway 连通性，并同步节点健康信息和 GPU 清单 |
+| POST | `/modelTF/compute/nodes/{id}/test-connection` | 测试 Compute API/File Gateway 连通性 |
 | POST | `/modelTF/compute/nodes/{id}/enable` | 启用节点 |
 | POST | `/modelTF/compute/nodes/{id}/disable` | 禁用节点，不接收新任务 |
 | POST | `/modelTF/compute/nodes/{id}/drain` | 进入维护模式，已有任务跑完后下线 |
@@ -898,7 +853,6 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
 | GET | `/modelTF/compute/jobs/{id}` | 算力任务详情 |
 | POST | `/modelTF/compute/jobs/{id}/retry` | 重试任务 |
 | POST | `/modelTF/compute/jobs/{id}/priority` | 调整优先级 |
-| GET | `/modelTF/compute/jobs/{id}/logs` | 拉取算力任务训练日志，支持 tail/分页 |
 | POST | `/modelTF/internal/compute-sync/jobs/poll` | 应用平台主动轮询并同步算力任务状态 |
 | POST | `/modelTF/internal/compute-sync/resources` | 调度前同步数据集/模型到目标节点 |
 
@@ -908,82 +862,6 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
 - 每个可执行训练的节点都需要部署 `Compute API`、`Compute Agent`、`File Gateway` 和宿主机挂载的 LLaMA-Factory。
 - 节点之间默认不互相访问，应用平台主动访问所有节点的 Compute API/File Gateway。
 - 调度支持 `auto` 和 `manual`：普通用户默认自动调度，管理员或高级用户可手动指定节点。
-- 节点地址、权重、标签、启用状态、最大并发和本地路径都由 `/compute` 算力节点页面维护。
-- 连接测试由应用后端发起，依次探测算力侧 `GET /modelTF/v1/compute/health` 和 `GET /modelTF/compute/resources/gpus`；返回包可为裸 JSON，也可为 `{code,message,data}` 包装结构。
-
-新增/编辑节点请求：
-
-```json
-{
-  "code": "gpu-node-01",
-  "name": "A800 Node 01",
-  "api_base_url": "http://10.10.20.31:19100",
-  "file_gateway_url": "http://10.10.20.31:19101",
-  "enabled": true,
-  "scheduler_status": "offline",
-  "scheduler_weight": 100,
-  "tags": ["A800", "80GB", "llama_factory"],
-  "max_parallel_jobs": 4,
-  "data_root": "/data/yg-ft",
-  "model_root": "/data/yg-ft/models",
-  "log_root": "/opt/yg-ft/logs/training",
-  "description": "北京机房训练节点"
-}
-```
-
-启动成功后，响应中的训练任务会包含 `compute_node_id`、`compute_job_id`、`process_id`、`status`、`progress`、`output_dir`、`log_file` 等字段。应用侧后台 worker 会按 `COMPUTE_POLL_INTERVAL_SECONDS` 定时调用目标算力节点查询 Compute Job，并回写训练任务状态。
-
-算力任务日志查询参数：
-
-| 参数 | 类型 | 必填 | 说明 |
-| --- | --- | --- | --- |
-| `tail_lines` | int | 否 | 默认 `200`，返回最后 N 行，范围 `1-5000` |
-| `offset` | int | 否 | 从第 N 行开始读取；当传入 `offset` 或 `limit` 时分页优先，忽略默认 tail 行数 |
-| `limit` | int | 否 | 分页读取行数，范围 `1-5000` |
-
-响应字段包括 `content`、`metrics`、`total_lines`、`offset`、`limit`、`has_more`、`next_offset`。前端训练详情页、训练日志页和算力队列页可以用该接口增量读取日志，避免一次性拉取大文件。
-
-任务维度实时日志接口：`GET /modelTF/fine-tune/{task_id}/logs?tail_lines=500`。该接口由应用后端按任务绑定的 `compute_node_id` 和 `compute_job_id` 转发到目标算力节点日志接口；如果训练尚未创建 Compute Job 或远端日志暂时不可达，则返回任务 `failure_reason`，用于页面展示启动失败、预检失败和远端训练失败原因。
-
-算力任务重试：
-
-```json
-{
-  "force": false,
-  "priority": "high",
-  "requested_node_id": "node_xxx",
-  "gpus": [0]
-}
-```
-
-默认只允许 `failed`、`stopped` 任务重试；如确需重新执行已完成任务，需要显式传 `force=true`。重试会清空旧的运行时字段，重新调度节点并创建新的 Compute Job。
-
-算力任务优先级：
-
-```json
-{
-  "priority": "low|normal|high|urgent"
-}
-```
-
-第一版优先级写入任务 payload，并影响 `/modelTF/compute/queue` 的展示排序；后续如接入独立队列调度器，可保持接口不变，将该字段映射到调度器优先级。
-
-连接测试响应：
-
-```json
-{
-  "node_id": "node_xxx",
-  "success": true,
-  "latency_ms": 35,
-  "gpu_count": 8,
-  "health": {
-    "status": "ok",
-    "api_version": "v1",
-    "execution_mode": "real",
-    "capabilities": ["gpu_discovery", "llama_factory", "file_gateway", "job_polling"]
-  }
-}
-```
 
 算力节点响应字段：
 
@@ -1004,9 +882,6 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
   "data_root": "/data/yg-ft",
   "model_root": "/data/yg-ft/models",
   "log_root": "/opt/yg-ft/logs/compute",
-  "api_version": "v1",
-  "capabilities": ["gpu_discovery", "llama_factory"],
-  "description": "北京机房训练节点",
   "last_health_check_at": "2026-07-20T12:00:00+08:00",
   "health_detail": {
     "compute_api": "ok",
@@ -1041,15 +916,11 @@ GPU 响应字段：
 | 方法 | 路径 | 说明 |
 | --- | --- | --- |
 | POST | `/modelTF/compute/jobs` | 创建训练/评测/数据处理/推理任务 |
-| POST | `/modelTF/compute/jobs/preview` | 算力节点训练命令预览，不启动进程 |
-| POST | `/modelTF/compute/jobs/validate` | 算力节点训练启动前预检，校验参数、路径和引擎命令 |
 | GET | `/modelTF/compute/jobs/{id}` | 查询任务 |
 | POST | `/modelTF/compute/jobs/{id}/stop` | 停止任务 |
 | GET | `/modelTF/compute/jobs/{id}/logs` | 拉取日志 |
-| POST | `/modelTF/compute/files/check-paths` | 算力节点本地路径可用性检查 |
 | GET | `/modelTF/compute/resources/gpus` | 查询 GPU |
 | POST | `/modelTF/compute/files/upload` | 上传到算力本地磁盘 |
-| POST | `/modelTF/compute/files/import-local` | 从算力服务器本地路径导入到 `YG_FT_DATA_ROOT` |
 | GET | `/modelTF/compute/files/{id}/download` | 下载文件 |
 
 创建算力任务：
@@ -1086,71 +957,6 @@ GPU 响应字段：
 }
 ```
 
-当前 LLaMA-Factory 训练作业最小 payload：
-
-```json
-{
-  "id": "ft_xxx",
-  "name": "finance-sft-001",
-  "engine": "llama_factory",
-  "base_model": "/data/yg-ft/models/Qwen2.5-7B",
-  "model_name_or_path": "/data/yg-ft/models/Qwen2.5-7B",
-  "train_dataset_id": "ds_finance_train",
-  "dataset": "ygft_ds_finance_train",
-  "dataset_key": "ygft_ds_finance_train",
-  "dataset_dir": "/data/yg-ft/datasets/ds_finance_train",
-  "dataset_info": {
-    "ygft_ds_finance_train": {
-      "file_name": "train.jsonl",
-      "formatting": "alpaca",
-      "columns": {
-        "prompt": "instruction",
-        "query": "input",
-        "response": "output"
-      }
-    }
-  },
-  "output_dir": "/data/yg-ft/outputs/finance-sft-001",
-  "template": "qwen",
-  "train_method": "lora",
-  "gpus": [0],
-  "batch_size": 2,
-  "learning_rate": 0.0002,
-  "n_epochs": 3,
-  "save_steps": 50
-}
-```
-
-数据集启动规则：
-- 页面选择的是平台数据集 ID，后端提交 Compute Job 时会将其转换为 LLaMA-Factory 数据集 key。
-- 单文件数据集使用 `--dataset ygft_{dataset_id}`；多文件数据集使用 `--dataset ygft_{dataset_id}_1,ygft_{dataset_id}_2`。
-- `dataset_dir` 指向目标算力节点上的独立数据集目录 `/data/yg-ft/datasets/{dataset_id}`。
-- Compute API 在 preflight 和启动训练前根据 `dataset_info` 生成 `{dataset_dir}/dataset_info.json`，避免 LLaMA-Factory 读取全局 `/data/yg-ft/datasets/dataset_info.json` 失败。
-- `columns` 只声明训练文件实际存在的字段；`system`、`history` 等可选字段不能默认写入，否则样本缺少字段时 LLaMA-Factory 会在格式转换阶段报 `KeyError`。
-- 正式启动前，应用侧会把当前数据集文件内容同步到被调度的算力节点，确保在线编辑/版本切换后的训练文件被使用。
-- Preflight 会校验 `dataset_info.columns` 对应字段是否能在样本文件中找到，并校验 PyTorch CUDA 可用性、所选 GPU 是否存在、显存是否满足 `MIN_TRAINING_GPU_MEMORY_GB`。
-- Compute 健康检查返回 `torch_cuda`，用于区分 `nvidia-smi` 可见但 PyTorch CUDA 初始化失败的环境问题。
-
-应用侧轮询同步响应：
-
-```json
-{
-  "synced": 1,
-  "failed": [],
-  "items": [
-    {
-      "id": "ft_xxx",
-      "status": "running",
-      "progress": 35,
-      "compute_job_id": "ft_xxx",
-      "process_id": 52341,
-      "output_dir": "/data/yg-ft/outputs/finance-sft-001",
-      "log_file": "/opt/yg-ft/logs/training/ft_xxx.log"
-    }
-  ]
-}
-```
-
 手动指定节点时：
 
 ```json
@@ -1353,38 +1159,3 @@ LLaMA-Factory 引擎声明：
 | 存储管理 | `/storage` | `GET /modelTF/quotas/usage`、`GET /modelTF/files/{id}/download-url`、`GET /modelTF/retention-policies`、`PUT /modelTF/retention-policies/{id}` | 磁盘占用、下载、留存 |
 | 审计中心 | `/audit-logs`、`/login-logs`、`/download-logs` | `GET /modelTF/audit-logs`、`GET /modelTF/login-logs`、`GET /modelTF/download-logs` | 操作、登录、下载审计 |
 | 训练引擎管理 | `/training-engines` | `GET /modelTF/training-engines`、`GET /modelTF/training-engines/{id}`、`GET /modelTF/training-engines/{id}/schema`、`POST /modelTF/training-engines/{id}/health-check` | 引擎能力和健康 |
-## P1/P2/P3 runtime implementation note
-
-The current backend/compute implementation has connected the B+D training runtime features below:
-
-| Page module | API | Runtime behavior |
-| --- | --- | --- |
-| Training detail / logs `/training-log/:id` | `GET /modelTF/fine-tune/{id}/overview` | Returns task progress, parsed training metrics and real checkpoint records. |
-| Training detail / loss chart `/training-log/:id` | `GET /modelTF/fine-tune/{id}/metrics` | Reads `fine_tune_metrics`, populated from Compute log polling and log proxy access. |
-| Training detail / checkpoint list `/training-log/:id` | `GET /modelTF/fine-tune/{id}/checkpoints` | Reads `fine_tune_checkpoints`, populated from Compute scanning `output_dir/checkpoint-*`. |
-| Merge weights `/model-manage/merge` | `POST /modelTF/model-manage/merge` | Submits a real Compute job using `llamafactory-cli export`; records the job in `compute_jobs`; updates `trained_models.merging/merged/merged_path` when queried after completion. |
-| Compute ops / job detail | `GET /modelTF/compute/jobs/{job_id}` | Supports both fine-tune jobs and model merge/export jobs recorded in `compute_jobs`. |
-| Compute ops / job logs | `GET /modelTF/compute/jobs/{job_id}/logs` | Proxies logs from the assigned Compute node for training and merge/export jobs. |
-
-Operational diagnostic script:
-
-```bash
-APP_BASE_URL=http://localhost:17861 \
-COMPUTE_BASE_URL=http://localhost:19100 \
-COMPUTE_SERVICE_TOKEN=change_me \
-DATABASE_URL=postgresql+psycopg://user:password@host:5432/yg_ft \
-python scripts/ops_diagnostics.py
-```
-## P2/P3 runtime extension note
-
-This iteration extends the B+D runtime implementation with production-facing model asset governance and compute operations:
-
-| Page module | API | Description |
-| --- | --- | --- |
-| Trained model detail / artifacts | `GET /modelTF/model-manage/trained-models/{id}/artifacts` | Returns registered adapter, merged model and quantized/export artifacts from `model_artifacts`. |
-| Trained model detail / lineage | `GET /modelTF/model-manage/trained-models/{id}/lineage` | Returns upstream/downstream relations from `model_lineage`, including base model to fine-tuned model and merge/export relations. |
-| Merge/export task list | `GET /modelTF/model-manage/export-jobs?trained_model_id=xxx` | Returns model export and merge jobs from `model_export_jobs`. |
-| Compute node replicas | `GET /modelTF/compute/nodes/{id}/replicas/drift` | Checks whether model/dataset/output replicas still exist on the compute node local disk and updates replica status. |
-| Compute node replicas | `POST /modelTF/compute/nodes/{id}/replicas/repair` | Marks drifted replicas as `repair_pending` and creates a resource sync job for the operator/scheduler to process. |
-
-The scheduler now uses the `scheduler_locks` table while starting training tasks. Node selection, task state update, resource sync job creation and GPU pre-allocation are written in one database transaction to reduce multi-worker GPU contention.

docs/data-process-design.md

@@ -1,279 +0,0 @@
-# 数据处理接口与算法设计
-
-本文是 `team-development-plan.md` 板块 C 的落地契约，约束
-`/modelTF/data-process/*`、前端数据处理向导以及 PostgreSQL 数据模型。
-
-## 1. 处理闭环
-
-```text
-创建草稿任务
-  → 上传并登记源文件（格式、SHA-256、版本）
-  → 预处理（标准化、无效过滤、去重、可选脱敏）
-  → 构建可编辑预览（来源偏移与行号）
-  → 生成标准训练记录
-  → 质量评分与稳定数据集划分
-  → 人工编辑/恢复
-  → 幂等发布为数据集（保留完整来源链路）
-```
-
-任务只使用以下五种状态：
-
-```text
-pending ──start/generate──> running ──success──> completed
-   ▲                         │  ├──error───────> failed
-   │                         │  └──stop────────> stopped
-   └────────retry────────────┴────────retry─────┘
-```
-
-- `pending` 允许修改配置、增删源文件和重建预览。
-- `running` 拒绝重复启动、修改配置和删除任务。
-- `failed`、`stopped` 可重试；重试前清理上一次未完成结果。
-- `completed` 可编辑结果和发布；重复发布返回同一个数据集。
-- 非法状态转换返回 HTTP 409。
-- 每次生成分配独立 `generation_run_id`；停止或重试会使旧代次立即失效，
-  旧后台任务不能覆盖新代次的结果或状态。
-
-## 2. 接口契约
-
-所有路径由请求层统一添加 `/modelTF`，响应统一为
-`{ "code": 0, "message": "ok", "data": ... }`。
-
-### 任务与进度
-
-| 方法 | 路径 | 说明 |
-| --- | --- | --- |
-| GET | `/data-process` | 分页查询任务，支持 keyword/status/process_type |
-| POST | `/data-process` | 创建 `pending` 草稿 |
-| GET | `/data-process/{id}` | 查询任务详情，不内嵌全部结果 |
-| PUT | `/data-process/{id}` | 更新草稿配置 |
-| DELETE | `/data-process/{id}` | 软删除非运行任务 |
-| POST | `/data-process/{id}/start` | 重建预览并生成的一键编排入口 |
-| POST | `/data-process/{id}/generate` | 使用已确认预览生成结果 |
-| POST | `/data-process/{id}/stop` | 请求停止运行任务 |
-| GET | `/data-process/{id}/progress` | 查询阶段、进度与计数 |
-
-### 源文件与预览
-
-| 方法 | 路径 | 说明 |
-| --- | --- | --- |
-| POST | `/data-process/{id}/source-files` | multipart 上传，字段名 `files` |
-| DELETE | `/data-process/{id}/source-files/{file_id}` | 删除源文件及其预览 |
-| GET | `/data-process/{id}/source-files/{file_id}/content` | 按行窗口读取源文 |
-| POST | `/data-process/{id}/preview/build` | 后端预处理并重建预览 |
-| GET | `/data-process/{id}/preview` | 分页查询预览 |
-| POST | `/data-process/{id}/preview` | 手工增加预览条目 |
-| PUT | `/data-process/{id}/preview/{preview_id}` | 保存人工编辑 |
-| DELETE | `/data-process/{id}/preview/{preview_id}` | 删除预览条目 |
-
-上传批次先全部完成有界读取和解析，再在单个事务中登记；任一文件为空、超限、
-重复或格式非法时整批不落库，暂存原件也会一并清理。响应不回传整个文件，只返回
-逻辑对象引用、文件 ID、格式、原始字节数、记录数和原始 SHA-256。二进制文档必须
-由对应解析器显式处理；不支持的格式返回 415，绝不能静默替换成示例正文。
-
-原始上传字节与解析正文采用双层存储：原件默认保存在
-`backend/storage/data-process/<task_id>/<file_id>/v<version>/<安全文件名>`，数据库的
-`storage_object_id` 只保存 `local://data-process/...` 逻辑引用，不保存或返回宿主机
-绝对路径；完整解析正文继续保存在 `data_process_source_files.content`，列表摘要使用
-`content_preview`，因此 PDF、Office 等文件的预览无需反复解析原始二进制。可通过
-`DATA_PROCESS_STORAGE_DIR` 指定其他本地根目录；从 `start.sh` 启动时，该变量应在
-当前终端导出。历史 `db://data-process/...` 记录继续从数据库正文预览。
-
-单独删除源文件时先提交数据库软删除，再立即删除受控目录中的原件；若物理删除
-失败，接口仍按数据库结果返回成功并标记 `storage_cleanup_pending=true`，软删除记录
-中的逻辑引用可供运维补偿清理。任务软删除以及修改 `process_type` 导致的源文件
-软删除按留存数据处理，当前版本不自动物理清除。
-
-### 结果与发布
-
-| 方法 | 路径 | 说明 |
-| --- | --- | --- |
-| GET | `/data-process/{id}/results` | 分页查询，支持 keyword/status/split |
-| PUT | `/data-process/{id}/results/{result_id}` | 保存人工编辑并重评分 |
-| POST | `/data-process/{id}/results/{result_id}/restore` | 恢复生成时的原值 |
-| POST | `/data-process/{id}/publish` | 幂等发布为数据集 |
-
-## 3. 配置校验
-
-- `process_type`：`structured | unstructured | external`。
-- 数据集划分的 `train + validation + test` 必须等于 100，各项为 0～100。
-- `chunk_size` 为 16～32768 token；`chunk_overlap` 必须小于
-  `chunk_size`；`min_chunk_size` 不得大于 `chunk_size`。
-- `temperature` 为 0～2，`max_tokens` 为 1～32768。
-- 任务名称在未删除任务中唯一。
-- 选择 `generation_model_id` 后，启动生成时校验模型是否存在，并保存不含密钥的
-  模型版本快照。
-- 当前运行库沿用平台现有的单租户模式，不接受客户端提交 tenant/owner/operator
-  字段，避免伪造隔离上下文；接入平台可信认证上下文后再启用数据库中预留的
-  tenant/project 字段。
-
-## 4. 格式解析与标准化
-
-上传格式按处理类型约束：
-
-- 结构化数据支持 JSON、JSONL/NDJSON、CSV/TSV 和 XLSX。XLSX 能识别纵向、
-  横向合并单元格组成的多级表头，并稳定展平为 `销售.Q1` 一类字段；公式只读取
-  文件中已缓存的计算结果，不在服务端执行。
-- 非结构化数据支持 UTF-8/UTF-8 BOM 的 TXT、Markdown、JSON/JSONL，以及
-  文本型 PDF、DOCX 和 PPTX。PDF 按页抽取文本，DOCX 抽取段落与表格，PPTX
-  抽取幻灯片文本与表格，随后统一进入切片算法。
-- 旧版二进制 DOC、XLS、PPT 不直接解析，返回 415 并提示分别转换为
-  DOCX、XLSX、PPTX。
-- 扫描 PDF 没有文本层时明确提示需要 OCR；当前流程不执行 OCR。加密、损坏或
-  超出页数/工作表/行列/解压规模限制的文件整批拒绝。
-
-现代 Office 文件在交给解析库前检查 ZIP 成员路径、重复成员、加密标记、活动
-XML、单成员大小、总解压大小和压缩比，避免路径穿越、实体扩展与 ZIP bomb。
-
-结构化选项按固定顺序执行，关闭某项时不会隐式执行对应业务变换：
-
-1. `detect_structure`：展平嵌套对象；XLSX 上传解析阶段识别合并单元格和多级表头。
-2. `normalize_format`：字段名转 snake_case，执行 Unicode NFKC、换行和容器值规范化，
-   输出键顺序稳定的 canonical JSON；账号、邮编等字符串不会转成数值。
-3. `clean_invalid`：删除全空列和全空记录；存在 `id/uuid/key/code/*_id` 身份字段时，
-   删除身份字段残缺的行，但不会因备注等可选字段为空误删有效记录。
-4. `filter_anomaly`：仅对不少于 8 个样本的非身份数值字段使用 Tukey IQR 过滤离群行，
-   同时过滤明确乱码、不可打印或极端超长文本；小样本和 ID 字段不参与统计过滤。
-5. `deduplicate`：先按整行 canonical JSON 精确去重，再按非空
-   `id/uuid/key/code/*_id` 字段稳定保留首条；空关键值互不视为重复。
-6. `desensitize`：对结构化姓名字段和正文中的高置信上下文姓名、邮箱、手机号、
-   身份证号进行不可逆掩码，并分别记录命中数。
-
-非结构化“智能预处理”由六个可独立执行的底层选项组成：
-
-- `clean_invalid_content` 删除确定为空、不可读或纯重复符号的无效块。
-- `detect_document_structure` 识别 Markdown、中文章节和数字标题，切片不跨章节，
-  并在预览质量详情中保存 `heading_path`。
-- `merge_short_content` 在同一章节中合并短块，合并后不突破 `chunk_size`。
-- `filter_low_quality` 在生成前过滤乱码、不可打印、重复或极端超长内容。
-- `deduplicate_content` 先精确去重，再对足够长的内容进行保守近重复判断；数字或
-  否定含义变化时始终保留。
-- `preserve_context` 才启用相邻切片 overlap；关闭时切片不共享正文上下文，且上下文
-  永不跨文件或章节。
-
-表格、围栏代码块和连续列表保护是三个独立参数。启用时切点避开相应 Markdown
-块，关闭时允许按正常长度切分。
-
-脱敏是不可逆掩码：
-
-- 邮箱：`[EMAIL]`
-- 中国大陆手机号：`[PHONE]`
-- 18 位身份证号：`[ID_CARD]`
-- 高置信姓名：`[NAME]`
-
-源文件原文与脱敏后的预览分开保存，结果不得反向覆盖源文件。
-
-## 5. 切片算法
-
-首阶段只提供三种切片策略：
-
-- `structure` 先识别 Markdown、中文章节及编号标题，再由 LlamaIndex
-  `SentenceSplitter` 在章节内按段落和中英文句界限长；章节之间不共享 overlap。
-- `fixed` 使用 LlamaIndex `TokenTextSplitter` 按目标 token 窗口切分。
-- `custom` 使用用户给定分隔符，在找不到合适分隔点时回退到固定窗口。
-
-不提供 `semantic` 和旧 `heading` 配置；创建或更新任务时传入这些值会直接拒绝。
-LlamaIndex 只负责通用切分，原文 offset、行号、标题路径和 Markdown 保护块仍由
-项目适配层统一维护。
-
-首版使用可替换的确定性 token 估算器，中文字符、标点和英文词分别计数；
-所有偏移以 Python/JavaScript 都能稳定表达的 Unicode 文本偏移为准。
-
-算法必须满足：
-
-- 每轮游标严格前进，异常分隔符不能产生死循环。
-- overlap 是最大重叠量，尾部过短切片合并到上一片。
-- 代码块、Markdown 表格和连续列表在启用保护时不从中间切开。
-- 每个预览条目记录 `source_file_id`、字符偏移、起止行、token 数和算法版本。
-
-## 6. 生成与质量评分
-
-结构化记录优先识别以下字段：
-
-1. `instruction/input/output`
-2. `question/context/answer`
-3. `prompt/input/response`
-
-已有标准字段时只做标准化；需要语义生成时调用所选模型的 OpenAI 兼容接口，
-并固化模型 ID、模型版本、prompt、temperature、max_tokens 和 JSON mode 快照。
-模型地址可输入域名、`/v1` 基础地址或完整地址：例如输入
-`www.caoxiaozhu.com` 会规范为
-`https://www.caoxiaozhu.com/v1/chat/completions`，无需用户手工拼接路径。
-单条失败记录为 `invalid`，有限重试耗尽后继续处理下一条，避免整批丢失。
-
-每条结果总分为 0～100：
-
-```text
-总分 = 完整性 35% + 长度合理性 20% + 可读性 20%
-     + 来源相关性 15% + 非重复性 10%
-```
-
-- instruction 或 output 为空时格式硬失败并标记 `invalid`。
-- 开启短文本过滤且 output 低于 `min_output_length` 时标记过滤原因。
-- 评分详情、命中规则与过滤原因必须落库并返回前端，不只返回一个总分。
-
-## 7. 稳定划分
-
-划分不能依赖结果插入顺序。对每条记录计算：
-
-```text
-bucket = SHA256(task_id + ":" + result_id) mod 10000
-```
-
-按万分位阈值映射为 `train/validation/test`。同一任务重试、分页或进程重启后，
-同一结果仍落入相同 split。
-
-## 8. 发布与来源链路
-
-发布在一个数据库事务中完成：
-
-```text
-source_file
-  → data_process_task
-  → data_process_result
-  → dataset
-  → dataset_file + dataset_file_version
-  → dataset_record
-```
-
-只发布 `valid/modified` 且满足质量门槛的结果。输出 JSONL 先计算 checksum，
-再登记文件版本和记录。发布请求中的 split 会重新进行稳定划分。任务的
-`output_dataset_id` 是幂等键；重复调用返回已有数据集，目标数据集若已被外部
-删除则解除断链并重新发布。当前运行库只开放 `local` 存储类型，正文保存在
-当前平台的 `dataset_files.content`，不虚假宣称已上传 MinIO 或云存储。
-
-## 9. 安全边界
-
-- 文件名只保留 basename，响应不返回宿主机绝对路径。
-- 上传限制单文件、批次文件数与批次总大小，解析采用有界读取。
-- 外部数据源凭据不写日志、不进入 localStorage、不在详情接口回显。
-- 外部 PostgreSQL 只允许单条 `SELECT/WITH`、只读事务、5 秒连接超时、
-  30 秒语句超时和 50 MiB 响应上限；默认阻止回环、链路本地及私网地址。
-  可信内网部署必须显式设置 `DATA_PROCESS_ALLOW_PRIVATE_EXTERNAL_DB=true`。
-- SQL 迁移独立存放，应用启动不会隐式修改当前远程数据库。
-
-## 10. 迁移边界
-
-`backend/app/db/sql/002_data_process.sql` 只面向当前运行脚本
-`001_platform_runtime.sql` 的 TEXT/最小表模型。它会在执行前检查
-`datasets.id` 类型；若检测到 `docs/postgres-schema.sql` 的 UUID/JSONB 目标模型，
-会直接失败而不是进行一半成功、一半失败的危险迁移。目标模型后续应由独立
-Alembic 迁移和对应存储实现承接。
-
-`DataProcessStore.ensure_schema()` 仅供受控管理命令显式调用，API 路由和应用启动
-均不会自动执行该迁移。本次开发和测试没有修改任何远程数据库。
-
-在已加载 `DATABASE_URL` 的终端中可先只读检查：
-
-```bash
-cd backend
-.venv/bin/python -m app.modules.data_process.schema_cli --check
-```
-
-确认目标主机和数据库名称无误后，才显式执行：
-
-```bash
-cd backend
-.venv/bin/python -m app.modules.data_process.schema_cli --apply --yes
-```
-
-命令输出只显示主机、端口和数据库名，不显示用户名或密码。

docs/deployment-plan.md

@@ -228,20 +228,6 @@ GPU 算力服务器部署：
 
 多节点任务调度由应用平台统一完成。应用平台从 `compute_nodes` 读取节点地址、权重、标签、启用状态、维护状态和健康检查结果；从 `resource_replicas` 判断目标节点是否已有所需数据集/模型副本；缺失时创建 `resource_sync_jobs`，通过目标节点 File Gateway 同步资源。
 
-当前实现已支持在 `/compute` 算力节点页面新增和编辑节点。运维人员维护 `Compute API` 地址、`File Gateway` 地址、权重、标签、启用状态、最大并发和本地路径后，点击连接测试会由应用后端主动访问目标节点健康检查和 GPU 清单接口，并将 `health_detail`、`gpu_count`、`gpu_devices/gpus` 同步到 PostgreSQL。真实 GPU 服务器优先通过 `nvidia-smi` 发现 GPU；特殊环境可用 `COMPUTE_GPU_COUNT` 等环境变量声明兼容清单。
-
-训练运行闭环：
-
-- 前端启动训练后，Backend API 按 `compute_nodes` 的启用状态、调度状态、权重和并行任务数选择节点。
-- Backend API 向目标节点 `POST /modelTF/compute/jobs` 提交 LLaMA-Factory 训练作业，并在 `fine_tune_tasks.compute_job_id` 记录算力任务 ID。
-- Compute API 在真实模式下启动 `llamafactory-cli train` 子进程，训练日志写入 `TRAINING_LOG_ROOT/{job_id}.log`。
-- Backend API 启动后会运行应用侧轮询 worker，按 `COMPUTE_POLL_INTERVAL_SECONDS` 主动查询目标节点 `GET /modelTF/compute/jobs/{id}`，同步任务状态、进度、PID、输出目录、日志路径和产物索引。
-- 停止训练时，Backend API 优先调用目标节点 `POST /modelTF/compute/jobs/{id}/stop`，再回写应用任务状态。
-- 失败或停止任务可以通过 `POST /modelTF/compute/jobs/{id}/retry` 重试；重试会清空旧运行态，重新调度节点并创建 Compute Job。
-- 训练日志通过 `GET /modelTF/compute/jobs/{id}/logs` 读取，支持 `tail_lines`、`offset`、`limit`，用于训练详情页、训练日志页和日志平台采集。
-- Compute API 使用 `COMPUTE_SERVICE_TOKEN` 做服务间鉴权，应用侧请求携带 `X-Compute-Token`；健康检查接口保持可公开探活。
-- Compute API 会把本机训练作业登记到 `TRAINING_LOG_ROOT/compute-jobs.json`，服务重启后可恢复任务索引并继续暴露状态和日志。
-
 调度策略：
 
 - 默认自动调度，按节点健康、标签、GPU 空闲、队列长度、节点权重和资源副本命中率排序。
@@ -293,7 +279,7 @@ COMPUTE_API_BASE_URL=https://compute.internal:19100
 COMPUTE_SERVICE_TOKEN=***
 FILE_GATEWAY_BASE_URL=https://compute.internal:19101
 COMPUTE_STATUS_SYNC_MODE=polling
-COMPUTE_POLL_INTERVAL_SECONDS=3
+COMPUTE_POLL_INTERVAL_SECONDS=10
 COMPUTE_POLL_BATCH_SIZE=100
 ```
 
@@ -304,21 +290,10 @@ COMPUTE_ENV=prod
 COMPUTE_HOST_ID=gpu-node-01
 COMPUTE_API_PORT=19100
 FILE_GATEWAY_PORT=19101
-COMPUTE_AUTH_ENABLED=true
 COMPUTE_SERVICE_TOKEN=***
 ENABLE_APP_CALLBACK=false
 LLAMA_FACTORY_HOME=/app/LLaMA-Factory
 YG_FT_DATA_ROOT=/data/yg-ft
-YG_FT_DATA_ROOT_HOST=./data/yg-ft
-YG_FT_MODEL_ROOT=/data/yg-ft/models
-YG_FT_MODEL_ROOT_HOST=./data/yg-ft/models
-YG_FT_DATASET_ROOT=/data/yg-ft/datasets
-YG_FT_DATASET_ROOT_HOST=./data/yg-ft/datasets
-YG_FT_OUTPUT_ROOT=/data/yg-ft/outputs
-YG_FT_OUTPUT_ROOT_HOST=./data/yg-ft/outputs
-TRAINING_LOG_ROOT=/opt/yg-ft/logs/training
-TRAINING_LOG_ROOT_HOST=./data/yg-ft/logs/training
-COMPUTE_LOG_ROOT_HOST=./data/yg-ft/logs/compute
 LOG_DIR=/opt/yg-ft/logs/compute
 CUDA_VISIBLE_DEVICES=0,1,2,3
 ```
@@ -418,7 +393,7 @@ gpu-node-03 -> http://10.10.20.33:19100 / http://10.10.20.33:19101
 ```env
 ENABLE_APP_CALLBACK=false
 COMPUTE_SERVICE_TOKEN=change_me
-YG_FT_DATA_ROOT_HOST=./data/yg-ft
+YG_FT_DATA_ROOT_HOST=/data/yg-ft
 ```
 
 ## 12. 仍需确认的问题

docs/postgres-schema.sql

@@ -1061,20 +1061,8 @@ CREATE TABLE IF NOT EXISTS compute_nodes (
   name varchar(150) NOT NULL,
   host varchar(200) NOT NULL,
   api_base_url text NOT NULL,
-  file_gateway_url text NOT NULL DEFAULT '',
   storage_node_id uuid REFERENCES storage_nodes(id) ON DELETE SET NULL,
   status varchar(40) NOT NULL DEFAULT 'online',
-  scheduler_status varchar(40) NOT NULL DEFAULT 'online',
-  scheduler_weight integer NOT NULL DEFAULT 100,
-  enabled boolean NOT NULL DEFAULT true,
-  max_parallel_jobs integer NOT NULL DEFAULT 1,
-  data_root text NOT NULL DEFAULT '/data/yg-ft',
-  model_root text NOT NULL DEFAULT '/data/yg-ft/models',
-  log_root text NOT NULL DEFAULT '/opt/yg-ft/logs/training',
-  api_version varchar(40) NOT NULL DEFAULT 'v1',
-  capabilities jsonb NOT NULL DEFAULT '[]'::jsonb,
-  description text,
-  health_detail jsonb NOT NULL DEFAULT '{}'::jsonb,
   agent_version varchar(80),
   gpu_count integer NOT NULL DEFAULT 0,
   last_heartbeat_at timestamptz,
@@ -1446,15 +1434,10 @@ ALTER TABLE fine_tune_tasks ADD COLUMN IF NOT EXISTS tenant_id uuid REFERENCES t
 ALTER TABLE fine_tune_tasks ADD COLUMN IF NOT EXISTS project_id uuid REFERENCES projects(id) ON DELETE SET NULL;
 ALTER TABLE fine_tune_tasks ADD COLUMN IF NOT EXISTS owner_id uuid REFERENCES users(id) ON DELETE SET NULL;
 ALTER TABLE fine_tune_tasks ADD COLUMN IF NOT EXISTS approval_status approval_status NOT NULL DEFAULT 'not_required';
-ALTER TABLE fine_tune_tasks ADD COLUMN IF NOT EXISTS compute_node_id uuid REFERENCES compute_nodes(id) ON DELETE SET NULL;
 ALTER TABLE fine_tune_tasks ADD COLUMN IF NOT EXISTS compute_job_id uuid REFERENCES compute_jobs(id) ON DELETE SET NULL;
 ALTER TABLE fine_tune_tasks ADD COLUMN IF NOT EXISTS resume_checkpoint_id uuid REFERENCES fine_tune_checkpoints(id) ON DELETE SET NULL;
 CREATE INDEX IF NOT EXISTS idx_fine_tune_tasks_scope_status
   ON fine_tune_tasks(tenant_id, project_id, status, created_at DESC) WHERE deleted_at IS NULL;
-CREATE INDEX IF NOT EXISTS idx_fine_tune_tasks_compute_job
-  ON fine_tune_tasks(compute_job_id) WHERE deleted_at IS NULL;
-CREATE INDEX IF NOT EXISTS idx_fine_tune_tasks_node_status
-  ON fine_tune_tasks(compute_node_id, status, created_at DESC) WHERE deleted_at IS NULL;
 
 ALTER TABLE inference_tasks ADD COLUMN IF NOT EXISTS tenant_id uuid REFERENCES tenants(id) ON DELETE SET NULL;
 ALTER TABLE inference_tasks ADD COLUMN IF NOT EXISTS project_id uuid REFERENCES projects(id) ON DELETE SET NULL;

docs/system-development-plan.md

@@ -701,9 +701,7 @@ YG_FT/
 职责：
 
 - 生成训练 YAML 或 CLI。
-- 根据平台数据集 ID 生成 LLaMA-Factory dataset key 和 `{dataset_dir}/dataset_info.json`。
-- 单文件数据集使用一个 dataset key，多文件数据集拆分为多个 key 并通过逗号组合传入 `--dataset`。
-- 正式启动前由应用侧同步当前数据集文件到目标算力节点，确保训练使用最新激活版本。
+- 生成 dataset_info。
 - 设置 `CUDA_VISIBLE_DEVICES`。
 - 启动训练。
 - 解析日志。
@@ -1118,10 +1116,3 @@ YG_FT/
 | P1 | 数据处理、评测、推理、模型服务治理 | 形成训练前后闭环 |
 | P1 | 存储管理、checkpoint、审计中心 | 企业治理和运维收口 |
 | P2 | 训练引擎管理、外部 API、OIDC/LDAP、成本核算、人工评测 | 扩展能力，第一版可预留 |
-## P1/P2/P3 Development Status
-
-| Priority | Scope | Implemented content | Remaining production hardening |
-| --- | --- | --- | --- |
-| P1 | Training runtime assets | Compute scans real `checkpoint-*` directories; backend persists checkpoints to `fine_tune_checkpoints`; backend parses log metrics into `fine_tune_metrics`; completed training auto-registers a trained model with its real output directory. | Add full TensorBoard event parsing and retention/cleanup execution policies. |
-| P2 | Model merge/export | `/modelTF/model-manage/merge` submits `llamafactory-cli export` through Compute; merge/export jobs are recorded in `compute_jobs` and `model_export_jobs`; trained model artifacts are recorded in `model_artifacts`; base->trained and merge/export relations are recorded in `model_lineage`; APIs expose artifacts, lineage and export jobs. | Add frontend polling/status panels for export jobs and richer artifact checksum generation during long-running export. |
-| P3 | Compute ops | `compute_jobs` and `gpu_allocations` are updated from runtime polling; training start uses `scheduler_locks` to serialize node selection and GPU pre-allocation; terminal task states release GPU allocations; resource replica drift check and repair-pending APIs are available; `scripts/ops_diagnostics.py` checks backend, PostgreSQL, Compute health, GPU discovery and Compute jobs. | Add dedicated scheduler worker, automatic replica repair execution from authoritative source metadata, and lock timeout alarms. |

docs/评估工作台设计.md

@@ -1,22 +0,0 @@
-4.2.3 评估工作台
-负责训练后模型的质量评测和问题诊断。这是平台闭环的核心环节。
-
-
-评估流程图
-1.评估方式
-评估默认使用本地部署的 LLM 作为评审模型，不依赖外部 API。评审模型对每条测试数据从四个子维度打分：核心事实正确性、信息完整性、无幻觉、格式合规性（可选）。汇总为三档判定：正确、部分正确、错误。
-用户可以选择额外启动人工复核——平台按错误类型分层抽样建议五十到两百条题目，用户在界面上逐条确认或修改 LLM 评审的判定。人工复核的结果用于校准 LLM 评审——一致率超过百分之八十时 LLM 评审结果标记为"可信"，低于百分之六十时标记为"以人工为准"。
-平台同时提供自动指标作为参考——如关键字段匹配率。这些指标不单独作为判定依据，仅作为快速参考。
-
-2.错误诊断
-评审模型在完成评分后，额外输出一个错误分类标签。标签从五种固定类型中选择：混淆（模型回答的值像是另一个实体的属性值）、不完整（事实正确但缺少部分信息）、格式偏差（语义正确但措辞与预期不符）、幻觉（回答中存在标准答案没有的内容）、其他（不属于以上任何类型）。
-这种分类方式简单可落地——它是一个固定枚举的分类任务，评审模型的prompt 中已包含每种类型的定义和判别示例，不需要额外的自然语言聚类或机器学习算法。
-
-3.评估报告
-评估完成后自动生成报告，分为四个部分：总览面板：整体得分和各维度通过率。如果做了人工复核，展示 LLM 评审和人工判定的一致率及可信度标记。
-错误分类面板：按五种分类标签分组的错误列表，每组展示数量和占比。点击展开可查看具体错误样例（问题、标准答案、模型回答、评审模型的原因描述）。
-修复建议面板：根据错误分类的统计分布，自动生成方向性建议。如"混淆"类错误占比最高时，建议检查训练数据中指令相似但答案不同的样本对；"不完整"类占比最高时，建议统一同类问题的答案详略标准。某类错误的绝对数量不足百分之三时不单独给建议，"其他"类占比最高时提示用户人工分析错误样例。
-迭代对比面板：如果存在上轮评估记录，展示两轮各分类标签的数量变化。标注"混淆类错误从35条降到14条，下降60%，修复可能生效"。
-
-4.建议有效性的验证
-平台在每次评估完成后将分类标签的统计数据（每种标签的数量和占比）存入评估记录。下一轮评估的对应数据与之对比，计算差值和变化百分比。某类错误数量下降超过百分之二十，标注"调整可能生效"；变化不足百分之十，标注"调整可能未生效或生效不显著"；反向上升则标注"建议检查本次调整方向是否正确"。

frontend/README.md

@@ -22,9 +22,9 @@ npm install
 npm run dev
 ```
 
-开发服务器默认运行在 `http://localhost:16801`。
-
-后端 API 默认通过 Vite 代理转发到 `http://localhost:17861`（见 `vite.config.ts`）。
+通过根目录 `scripts/start-dev.sh` 启动时，开发服务器端口和后端代理端口来自
+`backend/config.yaml` 的 `server` 配置。单独运行 `npm run dev` 时默认使用前端
+`16801` 和后端 `17861`；可用 `FRONTEND_PORT`、`BACKEND_PORT` 环境变量覆盖。
 
 开发环境默认联调真实后端接口。如需进行隔离前端开发，可显式启用 Mock：
 

frontend/dist/assets/AppConfirmDialog-Bk_vjo5o.css

@@ -1 +0,0 @@
-.app-confirm-overlay[data-v-8d3bf6ad]{position:fixed;z-index:2000;top:0;right:0;bottom:0;left:0;display:grid;place-items:center;padding:20px;box-sizing:border-box;background:#0f172a70}.app-confirm-dialog[data-v-8d3bf6ad]{position:relative;width:min(480px,100%);overflow:hidden;background:#fff;border:1px solid #dfe3ea;border-radius:8px;box-shadow:0 12px 28px #0f172a29}.app-confirm-header[data-v-8d3bf6ad]{display:flex;min-height:52px;align-items:center;justify-content:space-between;gap:16px;padding:0 10px 0 20px;border-bottom:1px solid #e7eaf0}.app-confirm-heading[data-v-8d3bf6ad]{display:flex;min-width:0;align-items:center;gap:10px}.app-confirm-heading h2[data-v-8d3bf6ad]{margin:0;overflow:hidden;color:#273142;font-size:15px;font-weight:650;line-height:1.4;text-overflow:ellipsis;white-space:nowrap}.app-confirm-close[data-v-8d3bf6ad]{display:inline-grid;width:32px;height:32px;flex:0 0 32px;place-items:center;padding:0;color:#7b8495;background:transparent;border:0;border-radius:4px;cursor:pointer;transition:color .18s ease,background-color .18s ease}.app-confirm-close[data-v-8d3bf6ad]:hover{color:#273142;background:#f2f4f7}.app-confirm-close[data-v-8d3bf6ad]:focus-visible{outline:2px solid rgba(91,80,242,.45);outline-offset:1px}.app-confirm-icon[data-v-8d3bf6ad]{display:inline-grid;width:28px;height:28px;flex:0 0 28px;place-items:center;color:#a15c07;background:#fff8e6;border:1px solid #f3dfad;border-radius:6px;font-size:13px}.app-confirm-dialog.is-danger .app-confirm-icon[data-v-8d3bf6ad]{color:#c43232;background:#fff1f1;border-color:#f2c7c7}.app-confirm-dialog.is-primary .app-confirm-icon[data-v-8d3bf6ad]{color:#4f46e5;background:#f3f2ff;border-color:#d9d6ff}.app-confirm-body[data-v-8d3bf6ad]{padding:16px 20px 18px}.app-confirm-body p[data-v-8d3bf6ad]{margin:0;color:#5f6878;font-size:13px;line-height:1.7}.app-confirm-actions[data-v-8d3bf6ad]{display:flex;justify-content:flex-end;gap:8px;padding:10px 14px;background:#f8f9fb;border-top:1px solid #e7eaf0}.app-confirm-button[data-v-8d3bf6ad]{height:34px;min-width:72px;padding:0 13px;color:#344054;font-size:13px;font-weight:500;background:#fff;border:1px solid #cfd5df;border-radius:4px;cursor:pointer;transition:border-color .18s ease,background-color .18s ease,color .18s ease}.app-confirm-button[data-v-8d3bf6ad]:hover{color:#273142;background:#f2f4f7;border-color:#b9c1cd}.app-confirm-button[data-v-8d3bf6ad]:focus-visible{outline:2px solid rgba(91,80,242,.45);outline-offset:1px}.app-confirm-button.is-confirm[data-v-8d3bf6ad]{color:#fff;background:#a15c07;border-color:#a15c07}.app-confirm-button.is-confirm[data-v-8d3bf6ad]:hover{background:#844b06;border-color:#844b06}.app-confirm-dialog.is-danger .app-confirm-button.is-confirm[data-v-8d3bf6ad]{background:#c43232;border-color:#c43232}.app-confirm-dialog.is-danger .app-confirm-button.is-confirm[data-v-8d3bf6ad]:hover{background:#a92828;border-color:#a92828}.app-confirm-dialog.is-primary .app-confirm-button.is-confirm[data-v-8d3bf6ad]{background:#4f46e5;border-color:#4f46e5}.app-confirm-dialog.is-primary .app-confirm-button.is-confirm[data-v-8d3bf6ad]:hover{background:#4338ca;border-color:#4338ca}.app-confirm-enter-active[data-v-8d3bf6ad],.app-confirm-leave-active[data-v-8d3bf6ad]{transition:opacity .18s ease}.app-confirm-enter-active .app-confirm-dialog[data-v-8d3bf6ad],.app-confirm-leave-active .app-confirm-dialog[data-v-8d3bf6ad]{transition:opacity .18s ease,transform .18s ease}.app-confirm-enter-from[data-v-8d3bf6ad],.app-confirm-leave-to[data-v-8d3bf6ad]{opacity:0}.app-confirm-enter-from .app-confirm-dialog[data-v-8d3bf6ad],.app-confirm-leave-to .app-confirm-dialog[data-v-8d3bf6ad]{opacity:0;transform:translateY(4px)}@media(max-width:520px){.app-confirm-overlay[data-v-8d3bf6ad]{padding:12px}.app-confirm-actions[data-v-8d3bf6ad]{display:grid;grid-template-columns:repeat(2,minmax(0,1fr))}.app-confirm-button[data-v-8d3bf6ad]{height:auto;min-width:0;min-height:44px}}@media(prefers-reduced-motion:reduce){.app-confirm-enter-active[data-v-8d3bf6ad],.app-confirm-leave-active[data-v-8d3bf6ad],.app-confirm-enter-active .app-confirm-dialog[data-v-8d3bf6ad],.app-confirm-leave-active .app-confirm-dialog[data-v-8d3bf6ad]{transition:none}}

frontend/dist/assets/AppConfirmDialog-g7rxg-F7.js

@@ -1 +1 @@
-import{d as C,bf as g,D as E,H as O,o as _,e as B,s as D,Z as $,w as A,c as I,aL as K,q as t,n as v,aa as r,g as q,bg as M,y as u,z as N,P as k}from"./index-z4H3KYDz.js";import{_ as R}from"./_plugin-vue_export-helper-DlAUqK2U.js";const V={class:"app-confirm-header"},z={class:"app-confirm-heading"},H={class:"app-confirm-icon","aria-hidden":"true"},L={class:"app-confirm-body"},P={class:"app-confirm-actions"},S=C({__name:"AppConfirmDialog",setup(j,{expose:x}){const c=u(!1),m=u(),p=u(),d=`app-confirm-title-${g()}`,y=`app-confirm-message-${g()}`,n=N({title:"请确认操作",message:"",confirmText:"确定",cancelText:"取消",tone:"warning",closeOnOverlay:!1});let o=null,i=null;function s(a){c.value=!1;const e=o;o=null,e==null||e(a)}function h(a){return o&&s(!1),Object.assign(n,{confirmText:"确定",cancelText:"取消",tone:"warning",closeOnOverlay:!1,...a}),c.value=!0,new Promise(e=>{o=e})}function w(){n.closeOnOverlay&&s(!1)}function T(a){var b;if(a.key==="Escape"){a.preventDefault(),s(!1);return}if(a.key!=="Tab")return;const e=Array.from(((b=m.value)==null?void 0:b.querySelectorAll("button:not([disabled])"))??[]),l=e[0],f=e[e.length-1];!l||!f||(a.shiftKey&&document.activeElement===l?(a.preventDefault(),f.focus()):!a.shiftKey&&document.activeElement===f&&(a.preventDefault(),l.focus()))}return E(c,async a=>{var e;if(a){i=document.activeElement instanceof HTMLElement?document.activeElement:null,await k(),(e=p.value)==null||e.focus();return}await k(),i==null||i.focus(),i=null}),O(()=>{o==null||o(!1),o=null}),x({open:h}),(a,e)=>(_(),B(M,{to:"body"},[D($,{name:"app-confirm"},{default:A(()=>[c.value?(_(),I("div",{key:0,class:"app-confirm-overlay",onMousedown:K(w,["self"])},[t("section",{ref_key:"dialogRef",ref:m,class:v(["app-confirm-dialog",`is-${n.tone}`]),role:"alertdialog","aria-modal":!0,"aria-labelledby":d,"aria-describedby":y,onKeydown:T},[t("header",V,[t("div",z,[t("span",H,[t("i",{class:v(n.tone==="primary"?"fa fa-question-circle":"fa fa-exclamation-triangle")},null,2)]),t("h2",{id:d},r(n.title),1)]),t("button",{class:"app-confirm-close",type:"button","aria-label":"关闭确认弹窗",onClick:e[0]||(e[0]=l=>s(!1))},[...e[3]||(e[3]=[t("i",{class:"fa fa-times","aria-hidden":"true"},null,-1)])])]),t("div",L,[t("p",{id:y},r(n.message),1)]),t("footer",P,[t("button",{ref_key:"cancelButtonRef",ref:p,class:"app-confirm-button is-cancel",type:"button",onClick:e[1]||(e[1]=l=>s(!1))},r(n.cancelText),513),t("button",{class:"app-confirm-button is-confirm",type:"button",onClick:e[2]||(e[2]=l=>s(!0))},r(n.confirmText),1)])],34)],32)):q("",!0)]),_:1})]))}}),G=R(S,[["__scopeId","data-v-8d3bf6ad"]]);export{G as A};
+import{d as C,bn as g,D as E,H as O,o as _,e as B,s as D,Z as $,w as A,c as I,aL as K,q as a,n as v,aa as r,g as q,bm as M,y as u,z as N,P as k}from"./index-BKKvzUDD.js";import{_ as R}from"./_plugin-vue_export-helper-DlAUqK2U.js";const V={class:"app-confirm-header"},z={class:"app-confirm-heading"},H={class:"app-confirm-icon","aria-hidden":"true"},L={class:"app-confirm-body"},P={class:"app-confirm-actions"},S=C({__name:"AppConfirmDialog",setup(j,{expose:x}){const c=u(!1),m=u(),p=u(),d=`app-confirm-title-${g()}`,y=`app-confirm-message-${g()}`,n=N({title:"请确认操作",message:"",confirmText:"确定",cancelText:"取消",tone:"warning",closeOnOverlay:!1});let o=null,i=null;function s(t){c.value=!1;const e=o;o=null,e==null||e(t)}function h(t){return o&&s(!1),Object.assign(n,{confirmText:"确定",cancelText:"取消",tone:"warning",closeOnOverlay:!1,...t}),c.value=!0,new Promise(e=>{o=e})}function w(){n.closeOnOverlay&&s(!1)}function T(t){var b;if(t.key==="Escape"){t.preventDefault(),s(!1);return}if(t.key!=="Tab")return;const e=Array.from(((b=m.value)==null?void 0:b.querySelectorAll("button:not([disabled])"))??[]),l=e[0],f=e[e.length-1];!l||!f||(t.shiftKey&&document.activeElement===l?(t.preventDefault(),f.focus()):!t.shiftKey&&document.activeElement===f&&(t.preventDefault(),l.focus()))}return E(c,async t=>{var e;if(t){i=document.activeElement instanceof HTMLElement?document.activeElement:null,await k(),(e=p.value)==null||e.focus();return}await k(),i==null||i.focus(),i=null}),O(()=>{o==null||o(!1),o=null}),x({open:h}),(t,e)=>(_(),B(M,{to:"body"},[D($,{name:"app-confirm"},{default:A(()=>[c.value?(_(),I("div",{key:0,class:"app-confirm-overlay",onMousedown:K(w,["self"])},[a("section",{ref_key:"dialogRef",ref:m,class:v(["app-confirm-dialog",`is-${n.tone}`]),role:"alertdialog","aria-modal":!0,"aria-labelledby":d,"aria-describedby":y,onKeydown:T},[a("header",V,[a("div",z,[a("span",H,[a("i",{class:v(n.tone==="primary"?"fa fa-question-circle":"fa fa-exclamation-triangle")},null,2)]),a("h2",{id:d},r(n.title),1)]),a("button",{class:"app-confirm-close",type:"button","aria-label":"关闭确认弹窗",onClick:e[0]||(e[0]=l=>s(!1))},[...e[3]||(e[3]=[a("i",{class:"fa fa-times","aria-hidden":"true"},null,-1)])])]),a("div",L,[a("p",{id:y},r(n.message),1)]),a("footer",P,[a("button",{ref_key:"cancelButtonRef",ref:p,class:"app-confirm-button is-cancel",type:"button",onClick:e[1]||(e[1]=l=>s(!1))},r(n.cancelText),513),a("button",{class:"app-confirm-button is-confirm",type:"button",onClick:e[2]||(e[2]=l=>s(!0))},r(n.confirmText),1)])],34)],32)):q("",!0)]),_:1})]))}}),G=R(S,[["__scopeId","data-v-398df98e"]]);export{G as A};

frontend/dist/assets/AppConfirmDialog-zFD3Iwu_.css

@@ -0,0 +1 @@
+.app-confirm-overlay[data-v-398df98e]{position:fixed;z-index:2000;top:0;right:0;bottom:0;left:0;display:grid;place-items:center;padding:20px;box-sizing:border-box;background:#0f172a70}.app-confirm-dialog[data-v-398df98e]{position:relative;width:min(480px,100%);overflow:hidden;background:#fff;border:1px solid #dfe3ea;border-radius:8px;box-shadow:0 12px 28px #0f172a29}.app-confirm-header[data-v-398df98e]{display:flex;min-height:52px;align-items:center;justify-content:space-between;gap:16px;padding:0 10px 0 20px;border-bottom:1px solid #e7eaf0}.app-confirm-heading[data-v-398df98e]{display:flex;min-width:0;align-items:center;gap:10px}.app-confirm-heading h2[data-v-398df98e]{margin:0;overflow:hidden;color:#273142;font-size:15px;font-weight:650;line-height:1.4;text-overflow:ellipsis;white-space:nowrap}.app-confirm-close[data-v-398df98e]{display:inline-grid;width:32px;height:32px;flex:0 0 32px;place-items:center;padding:0;color:#7b8495;background:transparent;border:0;border-radius:4px;cursor:pointer;transition:color .18s ease,background-color .18s ease}.app-confirm-close[data-v-398df98e]:hover{color:#273142;background:#f2f4f7}.app-confirm-close[data-v-398df98e]:focus-visible{outline:2px solid rgba(91,80,242,.45);outline-offset:1px}.app-confirm-icon[data-v-398df98e]{display:inline-grid;width:28px;height:28px;flex:0 0 28px;place-items:center;color:#a15c07;background:#fff8e6;border:1px solid #f3dfad;border-radius:6px;font-size:13px}.app-confirm-dialog.is-danger .app-confirm-icon[data-v-398df98e]{color:#c43232;background:#fff1f1;border-color:#f2c7c7}.app-confirm-dialog.is-primary .app-confirm-icon[data-v-398df98e]{color:#4f46e5;background:#f3f2ff;border-color:#d9d6ff}.app-confirm-body[data-v-398df98e]{padding:16px 20px 18px}.app-confirm-body p[data-v-398df98e]{margin:0;color:#5f6878;font-size:13px;line-height:1.7}.app-confirm-actions[data-v-398df98e]{display:flex;justify-content:flex-end;gap:8px;padding:10px 14px;background:#f8f9fb;border-top:1px solid #e7eaf0}.app-confirm-button[data-v-398df98e]{height:34px;min-width:72px;padding:0 13px;color:#344054;font-size:13px;font-weight:500;background:#fff;border:1px solid #cfd5df;border-radius:4px;cursor:pointer;transition:border-color .18s ease,background-color .18s ease,color .18s ease}.app-confirm-button[data-v-398df98e]:hover{color:#273142;background:#f2f4f7;border-color:#b9c1cd}.app-confirm-button[data-v-398df98e]:focus-visible{outline:2px solid rgba(91,80,242,.45);outline-offset:1px}.app-confirm-button.is-confirm[data-v-398df98e]{color:#fff;background:#a15c07;border-color:#a15c07}.app-confirm-button.is-confirm[data-v-398df98e]:hover{background:#844b06;border-color:#844b06}.app-confirm-dialog.is-danger .app-confirm-button.is-confirm[data-v-398df98e]{background:#c43232;border-color:#c43232}.app-confirm-dialog.is-danger .app-confirm-button.is-confirm[data-v-398df98e]:hover{background:#a92828;border-color:#a92828}.app-confirm-dialog.is-primary .app-confirm-button.is-confirm[data-v-398df98e]{background:#4f46e5;border-color:#4f46e5}.app-confirm-dialog.is-primary .app-confirm-button.is-confirm[data-v-398df98e]:hover{background:#4338ca;border-color:#4338ca}.app-confirm-enter-active[data-v-398df98e],.app-confirm-leave-active[data-v-398df98e]{transition:opacity .18s ease}.app-confirm-enter-active .app-confirm-dialog[data-v-398df98e],.app-confirm-leave-active .app-confirm-dialog[data-v-398df98e]{transition:opacity .18s ease,transform .18s ease}.app-confirm-enter-from[data-v-398df98e],.app-confirm-leave-to[data-v-398df98e]{opacity:0}.app-confirm-enter-from .app-confirm-dialog[data-v-398df98e],.app-confirm-leave-to .app-confirm-dialog[data-v-398df98e]{opacity:0;transform:translateY(4px)}@media(max-width:520px){.app-confirm-overlay[data-v-398df98e]{padding:12px}.app-confirm-actions[data-v-398df98e]{display:grid;grid-template-columns:repeat(2,minmax(0,1fr))}.app-confirm-button[data-v-398df98e]{height:auto;min-width:0;min-height:44px}}@media(prefers-reduced-motion:reduce){.app-confirm-enter-active[data-v-398df98e],.app-confirm-leave-active[data-v-398df98e],.app-confirm-enter-active .app-confirm-dialog[data-v-398df98e],.app-confirm-leave-active .app-confirm-dialog[data-v-398df98e]{transition:none}}

frontend/dist/assets/CompareChatView-BZJACmmb.js

@@ -0,0 +1 @@
+import{a as N,E as B}from"./el-form-item-D5kF3B90.js";import{E as K}from"./index-TFUf94PZ.js";import{E as M}from"./index-BjEW7-SA.js";import{E as h}from"./index-BDEF353-.js";import{E as I}from"./el-divider-DjByQoml.js";import{E as R}from"./el-slider-DYENF1-i.js";import{d as F,G as z,e as V,w as l,ac as D,y as L,o as f,s as a,x as p,q as $,c as k,ad as j,aa as E,M as A,g as G,f as J,v as O,z as H,j as _,A as P}from"./index-BKKvzUDD.js";import"./el-popper-D6_hxRbQ.js";import"./el-tooltip-l0sNRNKZ.js";import"./el-input-number-BFR4pu1i.js";/* empty css               */import{P as Q}from"./PageCard-BoKXOzst.js";import{u as W}from"./usePolling-C6448AR2.js";import{a as X}from"./compare-CZ4TIoIW.js";import{_ as Y}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./index-GAnQrJsQ.js";import"./debounce-ByXPNh5F.js";import"./toNumber-Dkj3QRv9.js";import"./clamp-CbbY8h6F.js";import"./index-CWUnzf90.js";import"./index-BENk7lZo.js";import"./el-card-CApHJ1Gj.js";const Z={class:"model-list"},tt={key:0,class:"empty-hint"},et=F({__name:"CompareChatView",setup(ot){const b=D(),y=O(),x=b.params.id,n=L(null),e=H({systemPrompt:"",question:"",temperature:.7,topP:.9,topK:40,maxTokens:2048}),m=_(()=>{var s;if(!((s=n.value)!=null&&s.load_status))return[];try{return(typeof n.value.load_status=="string"?JSON.parse(n.value.load_status):n.value.load_status).loaded_models||[]}catch{return[]}}),T=_(()=>m.value.length>0&&m.value.every(s=>s.status==="ready"||s.status==="running")),g=_(()=>m.value.some(s=>s.status==="starting"));async function c(){try{n.value=await X(x)}catch{}}function C(){var u,i;if(!e.question.trim()){P.warning("请输入问题");return}if(g.value){P.warning("模型仍在启动中，请稍候");return}const s=new URLSearchParams({taskId:x,taskName:((u=n.value)==null?void 0:u.model_name)||((i=n.value)==null?void 0:i.name)||"",question:e.question,systemPrompt:e.systemPrompt,temperature:String(e.temperature),topP:String(e.topP),topK:String(e.topK),maxTokens:String(e.maxTokens)}),t=y.resolve(`/model-compare/result?${s.toString()}`).href;window.open(t,"_blank")}const{start:S}=W(c,5e3,{immediate:!1});return z(async()=>{await c(),S()}),(s,t)=>{const u=I,i=h,v=K,r=N,d=R,w=M,q=B;return f(),V(Q,{title:"模型对比配置"},{default:l(()=>[a(u,{"content-position":"left"},{default:l(()=>[...t[7]||(t[7]=[p("已启动模型",-1)])]),_:1}),$("div",Z,[(f(!0),k(A,null,j(m.value,(o,U)=>(f(),V(i,{key:U,type:o.status==="ready"||o.status==="running"?"success":o.status==="starting"?"warning":"danger",size:"large"},{default:l(()=>[p(E(o.model_name)+" ("+E(o.status)+") ",1)]),_:2},1032,["type"]))),128)),m.value.length?G("",!0):(f(),k("span",tt,"暂无已启动模型"))]),a(u,{"content-position":"left"},{default:l(()=>[...t[8]||(t[8]=[p("对话配置",-1)])]),_:1}),a(q,{"label-width":"120px",style:{"max-width":"700px"}},{default:l(()=>[a(r,{label:"系统提示词"},{default:l(()=>[a(v,{modelValue:e.systemPrompt,"onUpdate:modelValue":t[0]||(t[0]=o=>e.systemPrompt=o),type:"textarea",rows:3,placeholder:"可选"},null,8,["modelValue"])]),_:1}),a(r,{label:"问题"},{default:l(()=>[a(v,{modelValue:e.question,"onUpdate:modelValue":t[1]||(t[1]=o=>e.question=o),type:"textarea",rows:4,placeholder:"请输入要对比的问题"},null,8,["modelValue"])]),_:1}),a(r,{label:"Temperature"},{default:l(()=>[a(d,{modelValue:e.temperature,"onUpdate:modelValue":t[2]||(t[2]=o=>e.temperature=o),min:0,max:2,step:.1,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Top-p"},{default:l(()=>[a(d,{modelValue:e.topP,"onUpdate:modelValue":t[3]||(t[3]=o=>e.topP=o),min:0,max:1,step:.05,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Top-k"},{default:l(()=>[a(d,{modelValue:e.topK,"onUpdate:modelValue":t[4]||(t[4]=o=>e.topK=o),min:1,max:100,step:1,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Max Tokens"},{default:l(()=>[a(d,{modelValue:e.maxTokens,"onUpdate:modelValue":t[5]||(t[5]=o=>e.maxTokens=o),min:256,max:4096,step:128,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,null,{default:l(()=>[a(w,{type:"primary",disabled:!T.value||g.value,onClick:C},{default:l(()=>[...t[9]||(t[9]=[p(" 开始对比 ",-1)])]),_:1},8,["disabled"]),a(w,{onClick:t[6]||(t[6]=o=>J(y).back())},{default:l(()=>[...t[10]||(t[10]=[p("返回",-1)])]),_:1})]),_:1})]),_:1})]),_:1})}}}),qt=Y(et,[["__scopeId","data-v-5da55d96"]]);export{qt as default};

frontend/dist/assets/CompareChatView-DTOxFKd3.css

@@ -1 +0,0 @@
-.model-list[data-v-fbe74cb0]{display:flex;flex-wrap:wrap;gap:12px;margin-bottom:12px}.empty-hint[data-v-fbe74cb0]{color:#909399}

frontend/dist/assets/CompareChatView-DviGpt3Z.js

@@ -1 +0,0 @@
-import{a as B,E as K}from"./el-form-item-WQOcwC6B.js";import{E as M}from"./index-OPiDM056.js";import{E as h}from"./index-BHESvfAo.js";import{E as I}from"./el-tag-ulIZLhPi.js";import{E as R}from"./el-divider-C8c-JInP.js";import{E as F}from"./el-slider-OsPL6Uf1.js";import{d as L,G as z,e as V,w as s,ac as D,y as $,o as f,s as a,x as p,q as j,c as k,ad as A,aa as b,f as E,M as G,g as J,v as O,z as H,j as _,A as P}from"./index-z4H3KYDz.js";import"./el-popper-BJIpIz7j.js";import"./el-tooltip-l0sNRNKZ.js";import"./el-input-number-bsSWsFMW.js";import{P as Q}from"./PageCard-BWIJ3AEp.js";import{u as W}from"./usePolling-eiZ9wScN.js";import{a as X}from"./compare-Dtaakk2Y.js";import{a as Y}from"./status-Dl1fykxa.js";import{_ as Z}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./castArray-DuG_gufG.js";import"./_baseClone-DT8DugHo.js";import"./raf-CGFVbd0C.js";import"./index-C6iZvNqO.js";import"./index-CeeJbXQu.js";import"./debounce-DozRow7U.js";import"./toNumber-DmGFONFK.js";import"./clamp-Cqzl_HF6.js";import"./index-DJMIMot_.js";import"./index-DIW0xkWn.js";import"./el-card-DLSyQgTR.js";const tt={class:"model-list"},et={key:0,class:"empty-hint"},ot=L({__name:"CompareChatView",setup(at){const T=D(),y=O(),x=T.params.id,n=$(null),e=H({systemPrompt:"",question:"",temperature:.7,topP:.9,topK:40,maxTokens:2048}),m=_(()=>{var l;if(!((l=n.value)!=null&&l.load_status))return[];try{return(typeof n.value.load_status=="string"?JSON.parse(n.value.load_status):n.value.load_status).loaded_models||[]}catch{return[]}}),C=_(()=>m.value.length>0&&m.value.every(l=>l.status==="ready"||l.status==="running")),g=_(()=>m.value.some(l=>l.status==="starting"));async function c(){try{n.value=await X(x)}catch{}}function S(){var u,i;if(!e.question.trim()){P.warning("请输入问题");return}if(g.value){P.warning("模型仍在启动中，请稍候");return}const l=new URLSearchParams({taskId:x,taskName:((u=n.value)==null?void 0:u.model_name)||((i=n.value)==null?void 0:i.name)||"",question:e.question,systemPrompt:e.systemPrompt,temperature:String(e.temperature),topP:String(e.topP),topK:String(e.topK),maxTokens:String(e.maxTokens)}),t=y.resolve(`/model-compare/result?${l.toString()}`).href;window.open(t,"_blank")}const{start:q}=W(c,5e3,{immediate:!1});return z(async()=>{await c(),q()}),(l,t)=>{const u=R,i=I,v=M,r=B,d=F,w=h,U=K;return f(),V(Q,{title:"模型对比配置"},{default:s(()=>[a(u,{"content-position":"left"},{default:s(()=>[...t[7]||(t[7]=[p("已启动模型",-1)])]),_:1}),j("div",tt,[(f(!0),k(G,null,A(m.value,(o,N)=>(f(),V(i,{key:N,type:o.status==="ready"||o.status==="running"?"success":o.status==="starting"?"warning":"danger",size:"large"},{default:s(()=>[p(b(o.model_name)+" ("+b(E(Y)(o.status))+") ",1)]),_:2},1032,["type"]))),128)),m.value.length?J("",!0):(f(),k("span",et,"暂无已启动模型"))]),a(u,{"content-position":"left"},{default:s(()=>[...t[8]||(t[8]=[p("对话配置",-1)])]),_:1}),a(U,{"label-width":"120px",style:{"max-width":"700px"}},{default:s(()=>[a(r,{label:"系统提示词"},{default:s(()=>[a(v,{modelValue:e.systemPrompt,"onUpdate:modelValue":t[0]||(t[0]=o=>e.systemPrompt=o),type:"textarea",rows:3,placeholder:"可选"},null,8,["modelValue"])]),_:1}),a(r,{label:"问题"},{default:s(()=>[a(v,{modelValue:e.question,"onUpdate:modelValue":t[1]||(t[1]=o=>e.question=o),type:"textarea",rows:4,placeholder:"请输入要对比的问题"},null,8,["modelValue"])]),_:1}),a(r,{label:"Temperature"},{default:s(()=>[a(d,{modelValue:e.temperature,"onUpdate:modelValue":t[2]||(t[2]=o=>e.temperature=o),min:0,max:2,step:.1,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Top-p"},{default:s(()=>[a(d,{modelValue:e.topP,"onUpdate:modelValue":t[3]||(t[3]=o=>e.topP=o),min:0,max:1,step:.05,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Top-k"},{default:s(()=>[a(d,{modelValue:e.topK,"onUpdate:modelValue":t[4]||(t[4]=o=>e.topK=o),min:1,max:100,step:1,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Max Tokens"},{default:s(()=>[a(d,{modelValue:e.maxTokens,"onUpdate:modelValue":t[5]||(t[5]=o=>e.maxTokens=o),min:256,max:4096,step:128,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,null,{default:s(()=>[a(w,{type:"primary",disabled:!C.value||g.value,onClick:S},{default:s(()=>[...t[9]||(t[9]=[p(" 开始对比 ",-1)])]),_:1},8,["disabled"]),a(w,{onClick:t[6]||(t[6]=o=>E(y).back())},{default:s(()=>[...t[10]||(t[10]=[p("返回",-1)])]),_:1})]),_:1})]),_:1})]),_:1})}}}),Ut=Z(ot,[["__scopeId","data-v-fbe74cb0"]]);export{Ut as default};

frontend/dist/assets/CompareChatView-wm3b2ZD9.css

@@ -0,0 +1 @@
+.model-list[data-v-5da55d96]{display:flex;flex-wrap:wrap;gap:12px;margin-bottom:12px}.empty-hint[data-v-5da55d96]{color:#909399}

frontend/dist/assets/CompareResultView-CEEVA92i.js

@@ -1 +0,0 @@
-import{E as D}from"./el-alert-uXFexXb6.js";import{E as $}from"./index-BHESvfAo.js";import{E as z}from"./el-tag-ulIZLhPi.js";import{E as F}from"./el-card-DLSyQgTR.js";import{d as O,G as U,H as J,c as d,q as i,aa as c,f as v,s as N,w as m,M as K,ad as A,ac as G,y as g,o as r,x as p,e as y,g as H}from"./index-z4H3KYDz.js";import{_ as L}from"./MarkdownView.vue_vue_type_style_index_0_lang-DicHzhRR.js";import{a as W,c as j}from"./compare-Dtaakk2Y.js";import{_ as Q}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./vnode-Cz-7S2If.js";const X={class:"compare-result"},Y={class:"result-header"},Z={class:"header-actions"},tt={class:"result-grid"},et={class:"card-header"},st={class:"model-name"},ot={key:0,class:"error-text"},at={key:1,class:"streaming-text"},nt={key:3,class:"loading-text"},lt={key:4,class:"result-stats"},rt=O({__name:"CompareResultView",setup(it){const u=G(),q=u.query.taskId,k=decodeURIComponent(u.query.question||""),h=decodeURIComponent(u.query.systemPrompt||""),E=Number(u.query.temperature||.7),P=Number(u.query.topP||.9),I=Number(u.query.topK||40),b=Number(u.query.maxTokens||2048),w=u.query.taskName,l=g([]),C=g(!1),x=g([]),f=new Set;async function B(){if(!C.value){C.value=!0;try{const s=await W(q);let t=[];s.load_status&&(t=(typeof s.load_status=="string"?JSON.parse(s.load_status):s.load_status).loaded_models||[]),x.value=t,l.value=t.map(o=>({name:o.model_name||"模型",content:"",displayContent:"",isTyping:!1,status:"loading"})),await Promise.all(t.map((o,e)=>S(o,e)))}catch{}}}async function S(s,t){const o=Date.now();try{const e=await V(j({port:s.port,model_name:s.model_name,messages:[...h?[{role:"system",content:h}]:[],{role:"user",content:k}],temperature:E,top_p:P,top_k:I,max_tokens:b}),3e5),a=(e==null?void 0:e.response)||(e==null?void 0:e.content)||(e==null?void 0:e.data)||JSON.stringify(e),_=(Date.now()-o)/1e3;l.value[t].content=a,l.value[t].status="done",l.value[t].stats={totalTime:_,charsPerSec:_>0?Number((a.length/_).toFixed(1)):0},M(t,a)}catch(e){l.value[t].content="推理失败: "+(e.message||""),l.value[t].status="error"}}async function V(s,t){let o=null;try{return await Promise.race([s,new Promise((e,a)=>{o=setTimeout(()=>a(new Error("推理超时")),t)})])}finally{o&&clearTimeout(o)}}function M(s,t){let o=0;l.value[s].isTyping=!0;const e=Math.max(2,Math.ceil(t.length/30)),a=setInterval(()=>{o+=e,l.value[s].displayContent=t.slice(0,o),o>=t.length&&(clearInterval(a),f.delete(a),l.value[s].displayContent=t,l.value[s].isTyping=!1)},50);f.add(a)}return U(B),J(()=>{f.forEach(clearInterval),f.clear()}),(s,t)=>{const o=$,e=D,a=z,_=F;return r(),d("div",X,[i("div",Y,[i("h2",null,"对比结果"+c(v(w)?` - ${v(w)}`:""),1),i("div",Z,[N(o,{onClick:t[0]||(t[0]=n=>s.$router.push("/model-inference"))},{default:m(()=>[...t[1]||(t[1]=[p("返回列表",-1)])]),_:1})])]),N(e,{type:"info",closable:!1,"show-icon":"",class:"question-box"},{title:m(()=>[t[2]||(t[2]=i("strong",null,"问题：",-1)),p(c(v(k)),1)]),_:1}),i("div",tt,[(r(!0),d(K,null,A(l.value,(n,R)=>(r(),y(_,{key:R,shadow:"hover",class:"result-card"},{header:m(()=>[i("div",et,[i("span",st,c(n.name),1),n.status==="loading"?(r(),y(a,{key:0,type:"warning",size:"small"},{default:m(()=>[...t[3]||(t[3]=[p("生成中...",-1)])]),_:1})):n.status==="done"?(r(),y(a,{key:1,type:"success",size:"small"},{default:m(()=>[...t[4]||(t[4]=[p("完成",-1)])]),_:1})):(r(),y(a,{key:2,type:"danger",size:"small"},{default:m(()=>[...t[5]||(t[5]=[p("失败",-1)])]),_:1}))])]),default:m(()=>{var T;return[n.status==="error"?(r(),d("div",ot,c(n.content),1)):n.isTyping?(r(),d("div",at,c(n.displayContent),1)):n.displayContent?(r(),y(L,{key:2,content:n.displayContent},null,8,["content"])):(r(),d("div",nt,[...t[6]||(t[6]=[i("i",{class:"fa fa-spinner fa-spin"},null,-1),p(" 正在生成回答... ",-1)])])),n.stats?(r(),d("div",lt,[i("span",null,"耗时 "+c((T=n.stats.totalTime)==null?void 0:T.toFixed(1))+"s",1),i("span",null,"速度 "+c(n.stats.charsPerSec)+" 字/秒",1)])):H("",!0)]}),_:2},1024))),128))])])}}}),gt=Q(rt,[["__scopeId","data-v-8391f389"]]);export{gt as default};

frontend/dist/assets/CompareResultView-DcZvYm8N.css

@@ -1 +0,0 @@
-.compare-result[data-v-8391f389]{max-width:1200px;margin:0 auto}.result-header[data-v-8391f389]{display:flex;align-items:center;justify-content:space-between;margin-bottom:16px}.result-header h2[data-v-8391f389]{font-size:18px;font-weight:500;margin:0}.question-box[data-v-8391f389]{margin-bottom:20px}.result-grid[data-v-8391f389]{display:grid;grid-template-columns:repeat(auto-fit,minmax(420px,1fr));gap:16px}.result-card .card-header[data-v-8391f389]{display:flex;align-items:center;justify-content:space-between}.result-card .card-header .model-name[data-v-8391f389]{font-weight:500;color:#303133}.result-card .loading-text[data-v-8391f389],.result-card .error-text[data-v-8391f389]{color:#909399;min-height:80px;display:flex;align-items:center;justify-content:center}.result-card .error-text[data-v-8391f389]{color:#f56c6c}.result-card .streaming-text[data-v-8391f389]{min-height:80px;line-height:1.7;white-space:pre-wrap;word-break:break-word}.result-card .result-stats[data-v-8391f389]{display:flex;gap:16px;margin-top:12px;padding-top:12px;border-top:1px solid #ebeef5;font-size:12px;color:#909399}

frontend/dist/assets/CompareResultView-DlLosvn8.css

@@ -0,0 +1 @@
+.compare-result[data-v-4c4e03d7]{max-width:1200px;margin:0 auto}.result-header[data-v-4c4e03d7]{display:flex;align-items:center;justify-content:space-between;margin-bottom:16px}.result-header h2[data-v-4c4e03d7]{font-size:18px;font-weight:500;margin:0}.question-box[data-v-4c4e03d7]{margin-bottom:20px}.result-grid[data-v-4c4e03d7]{display:grid;grid-template-columns:repeat(auto-fit,minmax(420px,1fr));gap:16px}.result-card .card-header[data-v-4c4e03d7]{display:flex;align-items:center;justify-content:space-between}.result-card .card-header .model-name[data-v-4c4e03d7]{font-weight:500;color:#303133}.result-card .loading-text[data-v-4c4e03d7],.result-card .error-text[data-v-4c4e03d7]{color:#909399;min-height:80px;display:flex;align-items:center;justify-content:center}.result-card .error-text[data-v-4c4e03d7]{color:#f56c6c}.result-card .streaming-text[data-v-4c4e03d7]{min-height:80px;line-height:1.7;white-space:pre-wrap;word-break:break-word}.result-card .result-stats[data-v-4c4e03d7]{display:flex;gap:16px;margin-top:12px;padding-top:12px;border-top:1px solid #ebeef5;font-size:12px;color:#909399}

frontend/dist/assets/CompareResultView-yTJ0EdKT.js

@@ -0,0 +1 @@
+import{E as D}from"./el-alert-xVLeNUgJ.js";import{E as $}from"./index-BjEW7-SA.js";import{E as z}from"./index-BDEF353-.js";import{E as F}from"./el-card-CApHJ1Gj.js";import{d as O,G as U,H as J,c as d,q as i,aa as c,f as v,s as N,w as m,M as K,ad as A,ac as G,y as g,o as r,x as p,e as y,g as H}from"./index-BKKvzUDD.js";/* empty css               */import{_ as L}from"./MarkdownView.vue_vue_type_style_index_0_lang-BPqmX54I.js";import{a as W,c as j}from"./compare-CZ4TIoIW.js";import{_ as Q}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./vnode-78qeDweP.js";const X={class:"compare-result"},Y={class:"result-header"},Z={class:"header-actions"},tt={class:"result-grid"},et={class:"card-header"},st={class:"model-name"},ot={key:0,class:"error-text"},at={key:1,class:"streaming-text"},nt={key:3,class:"loading-text"},lt={key:4,class:"result-stats"},rt=O({__name:"CompareResultView",setup(it){const u=G(),q=u.query.taskId,k=decodeURIComponent(u.query.question||""),h=decodeURIComponent(u.query.systemPrompt||""),E=Number(u.query.temperature||.7),P=Number(u.query.topP||.9),I=Number(u.query.topK||40),b=Number(u.query.maxTokens||2048),w=u.query.taskName,l=g([]),C=g(!1),x=g([]),f=new Set;async function B(){if(!C.value){C.value=!0;try{const s=await W(q);let t=[];s.load_status&&(t=(typeof s.load_status=="string"?JSON.parse(s.load_status):s.load_status).loaded_models||[]),x.value=t,l.value=t.map(o=>({name:o.model_name||"模型",content:"",displayContent:"",isTyping:!1,status:"loading"})),await Promise.all(t.map((o,e)=>S(o,e)))}catch{}}}async function S(s,t){const o=Date.now();try{const e=await V(j({port:s.port,model_name:s.model_name,messages:[...h?[{role:"system",content:h}]:[],{role:"user",content:k}],temperature:E,top_p:P,top_k:I,max_tokens:b}),3e5),a=(e==null?void 0:e.response)||(e==null?void 0:e.content)||(e==null?void 0:e.data)||JSON.stringify(e),_=(Date.now()-o)/1e3;l.value[t].content=a,l.value[t].status="done",l.value[t].stats={totalTime:_,charsPerSec:_>0?Number((a.length/_).toFixed(1)):0},M(t,a)}catch(e){l.value[t].content="推理失败: "+(e.message||""),l.value[t].status="error"}}async function V(s,t){let o=null;try{return await Promise.race([s,new Promise((e,a)=>{o=setTimeout(()=>a(new Error("推理超时")),t)})])}finally{o&&clearTimeout(o)}}function M(s,t){let o=0;l.value[s].isTyping=!0;const e=Math.max(2,Math.ceil(t.length/30)),a=setInterval(()=>{o+=e,l.value[s].displayContent=t.slice(0,o),o>=t.length&&(clearInterval(a),f.delete(a),l.value[s].displayContent=t,l.value[s].isTyping=!1)},50);f.add(a)}return U(B),J(()=>{f.forEach(clearInterval),f.clear()}),(s,t)=>{const o=$,e=D,a=z,_=F;return r(),d("div",X,[i("div",Y,[i("h2",null,"对比结果"+c(v(w)?` - ${v(w)}`:""),1),i("div",Z,[N(o,{onClick:t[0]||(t[0]=n=>s.$router.push("/model-inference"))},{default:m(()=>[...t[1]||(t[1]=[p("返回列表",-1)])]),_:1})])]),N(e,{type:"info",closable:!1,"show-icon":"",class:"question-box"},{title:m(()=>[t[2]||(t[2]=i("strong",null,"问题：",-1)),p(c(v(k)),1)]),_:1}),i("div",tt,[(r(!0),d(K,null,A(l.value,(n,R)=>(r(),y(_,{key:R,shadow:"hover",class:"result-card"},{header:m(()=>[i("div",et,[i("span",st,c(n.name),1),n.status==="loading"?(r(),y(a,{key:0,type:"warning",size:"small"},{default:m(()=>[...t[3]||(t[3]=[p("生成中...",-1)])]),_:1})):n.status==="done"?(r(),y(a,{key:1,type:"success",size:"small"},{default:m(()=>[...t[4]||(t[4]=[p("完成",-1)])]),_:1})):(r(),y(a,{key:2,type:"danger",size:"small"},{default:m(()=>[...t[5]||(t[5]=[p("失败",-1)])]),_:1}))])]),default:m(()=>{var T;return[n.status==="error"?(r(),d("div",ot,c(n.content),1)):n.isTyping?(r(),d("div",at,c(n.displayContent),1)):n.displayContent?(r(),y(L,{key:2,content:n.displayContent},null,8,["content"])):(r(),d("div",nt,[...t[6]||(t[6]=[i("i",{class:"fa fa-spinner fa-spin"},null,-1),p(" 正在生成回答... ",-1)])])),n.stats?(r(),d("div",lt,[i("span",null,"耗时 "+c((T=n.stats.totalTime)==null?void 0:T.toFixed(1))+"s",1),i("span",null,"速度 "+c(n.stats.charsPerSec)+" 字/秒",1)])):H("",!0)]}),_:2},1024))),128))])])}}}),kt=Q(rt,[["__scopeId","data-v-4c4e03d7"]]);export{kt as default};

frontend/dist/assets/ComputeNodesView-BKuDW-um.css

@@ -0,0 +1 @@
+.compute-page[data-v-daf09639]{display:flex;flex-direction:column;gap:16px;min-height:0;height:100%;padding:24px;background:#fff}.compute-header[data-v-daf09639]{display:flex;justify-content:space-between;gap:16px;align-items:flex-start}.compute-header h1[data-v-daf09639]{margin:0;font-size:24px;font-weight:650;color:#111827}.compute-header p[data-v-daf09639]{margin:8px 0 0;color:#64748b}.header-actions[data-v-daf09639]{display:flex;align-items:center;gap:12px}.last-updated[data-v-daf09639],.muted[data-v-daf09639]{color:#64748b;font-size:12px}.summary-grid[data-v-daf09639]{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:12px}.summary-tile[data-v-daf09639]{border:1px solid #e5e7eb;border-radius:8px;padding:14px 16px;background:#f8fafc}.summary-tile span[data-v-daf09639]{display:block;color:#64748b;font-size:12px}.summary-tile strong[data-v-daf09639]{display:block;margin-top:8px;color:#111827;font-size:24px}.compute-tabs[data-v-daf09639]{flex:1;min-height:0}.compute-tabs[data-v-daf09639] .el-tabs__content{height:calc(100% - 56px)}.compute-tabs[data-v-daf09639] .el-tab-pane{height:100%}.mono[data-v-daf09639]{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px}.replica-toolbar[data-v-daf09639]{display:flex;gap:12px;align-items:center;margin-bottom:12px}.replica-toolbar .el-select[data-v-daf09639]{width:280px}@media(max-width:960px){.compute-header[data-v-daf09639],.header-actions[data-v-daf09639],.replica-toolbar[data-v-daf09639]{flex-direction:column;align-items:stretch}.summary-grid[data-v-daf09639]{grid-template-columns:repeat(2,minmax(0,1fr))}}

frontend/dist/assets/ComputeNodesView-BZOegsoz.css

@@ -1 +0,0 @@
-.compute-page[data-v-569994b3]{display:flex;flex-direction:column;gap:16px;min-height:0;height:100%;padding:24px;background:#fff}.compute-header[data-v-569994b3]{display:flex;justify-content:space-between;gap:16px;align-items:flex-start}.compute-header h1[data-v-569994b3]{margin:0;font-size:24px;font-weight:650;color:#111827}.compute-header p[data-v-569994b3]{margin:8px 0 0;color:#64748b}.header-actions[data-v-569994b3],.replica-toolbar[data-v-569994b3]{display:flex;align-items:center;gap:12px}.last-updated[data-v-569994b3],.muted[data-v-569994b3]{color:#64748b;font-size:12px}.last-updated[data-v-569994b3]{min-width:92px;text-align:right}.summary-grid[data-v-569994b3]{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:12px}.summary-tile[data-v-569994b3]{border:1px solid #e5e7eb;border-radius:8px;padding:14px 16px;background:#f8fafc}.summary-tile span[data-v-569994b3]{display:block;color:#64748b;font-size:12px}.summary-tile strong[data-v-569994b3]{display:block;margin-top:8px;color:#111827;font-size:24px}.compute-tabs[data-v-569994b3]{flex:1;min-height:0}.compute-tabs[data-v-569994b3] .el-tabs__content{height:calc(100% - 56px)}.compute-tabs[data-v-569994b3] .el-tab-pane{height:100%}.mono[data-v-569994b3]{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px}.replica-toolbar[data-v-569994b3]{margin-bottom:12px}.replica-toolbar .el-select[data-v-569994b3]{width:280px}.sync-progress[data-v-569994b3]{display:grid;gap:8px;margin-bottom:12px;border:1px solid #e5e7eb;border-radius:8px;padding:12px 14px;background:#f8fafc}.sync-progress>div[data-v-569994b3]{display:flex;align-items:center;gap:10px}.node-form-grid[data-v-569994b3]{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));column-gap:12px}.node-form-grid[data-v-569994b3] .el-input-number,.node-form-grid[data-v-569994b3] .el-select{width:100%}@media(max-width:960px){.compute-header[data-v-569994b3],.header-actions[data-v-569994b3],.replica-toolbar[data-v-569994b3]{flex-direction:column;align-items:stretch}.summary-grid[data-v-569994b3]{grid-template-columns:repeat(2,minmax(0,1fr))}.node-form-grid[data-v-569994b3]{grid-template-columns:1fr}}

frontend/dist/assets/DataConvertView-CaZ-PRiW.js

@@ -1 +0,0 @@
-import{a as N,E as v}from"./el-form-item-WQOcwC6B.js";import{E as b}from"./el-popper-BJIpIz7j.js";import{E as O}from"./index-OPiDM056.js";import{E as S}from"./index-BHESvfAo.js";import{E as g,a as E}from"./el-select-OcCw1rK5.js";import{d as J,e as V,w as e,o as w,q as t,s as l,x as i,z as x,A as C}from"./index-z4H3KYDz.js";import"./el-tooltip-l0sNRNKZ.js";import"./el-scrollbar-Cs4RtPFn.js";import"./el-tag-ulIZLhPi.js";import{P as U}from"./PageCard-BWIJ3AEp.js";import{_ as y}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./castArray-DuG_gufG.js";import"./_baseClone-DT8DugHo.js";import"./index-CeeJbXQu.js";import"./index-DJMIMot_.js";import"./raf-CGFVbd0C.js";import"./index-C6iZvNqO.js";import"./vnode-Cz-7S2If.js";import"./index-BANvEWbb.js";import"./scroll-CJzUJVH-.js";import"./clamp-Cqzl_HF6.js";import"./toNumber-DmGFONFK.js";import"./_baseIteratee-BSbGVKbQ.js";import"./el-card-DLSyQgTR.js";const F={class:"converter-panel"},L={class:"form-row"},T={class:"panel-footer"},B={class:"actions"},k=J({__name:"DataConvertView",setup(I){const a=x({outputName:"converted-data",encoding:"UTF-8"});function p(){C.info("当前仅完成界面设计，转换功能将在后续接入")}function d(){Object.assign(a,{outputName:"converted-data",encoding:"UTF-8"})}return(j,o)=>{const n=N,m=O,u=E,f=g,c=v,r=S,_=b;return w(),V(U,{class:"data-convert-page",title:"数据类型转换",subtitle:"将 JSON 文件转换为便于训练和评测使用的 JSONL 格式"},{default:e(()=>[t("div",F,[o[9]||(o[9]=t("div",{class:"panel-header"},[t("div",{class:"tool-icon","aria-hidden":"true"},[t("i",{class:"fa fa-exchange"})]),t("div",null,[t("h3",null,"JSON 转 JSONL"),t("p",null,"每条 JSON 数据将输出为 JSONL 文件中的一行记录")])],-1)),l(c,{class:"converter-form","label-position":"top"},{default:e(()=>[l(n,{label:"转换类型"},{default:e(()=>[...o[2]||(o[2]=[t("div",{class:"format-field","aria-label":"JSON 转 JSONL"},[t("span",null,"JSON"),t("i",{class:"fa fa-long-arrow-right","aria-hidden":"true"}),t("span",null,"JSONL")],-1)])]),_:1}),l(n,{label:"源文件",required:""},{default:e(()=>[t("button",{class:"upload-zone",type:"button",onClick:p},[...o[3]||(o[3]=[t("i",{class:"fa fa-cloud-upload","aria-hidden":"true"},null,-1),t("span",{class:"upload-content"},[t("strong",null,"点击选择或拖拽 JSON 文件到此处"),t("small",null,"仅支持 .json 格式，单文件不超过 200 MB")],-1),t("span",{class:"select-button"},"选择文件",-1)])])]),_:1}),t("div",L,[l(n,{label:"输出文件名"},{default:e(()=>[l(m,{modelValue:a.outputName,"onUpdate:modelValue":o[0]||(o[0]=s=>a.outputName=s)},{append:e(()=>[...o[4]||(o[4]=[i(".jsonl",-1)])]),_:1},8,["modelValue"])]),_:1}),l(n,{label:"字符编码"},{default:e(()=>[l(f,{modelValue:a.encoding,"onUpdate:modelValue":o[1]||(o[1]=s=>a.encoding=s),style:{width:"100%"}},{default:e(()=>[l(u,{label:"UTF-8",value:"UTF-8"})]),_:1},8,["modelValue"])]),_:1})]),o[5]||(o[5]=t("div",{class:"format-tip"},[t("i",{class:"fa fa-info-circle","aria-hidden":"true"}),t("span",null,"支持由 JSON 数组转换为 JSONL，每个数组元素输出为一行。")],-1))]),_:1}),t("div",T,[o[8]||(o[8]=t("span",{class:"prototype-label"},"当前为 UI 原型，暂不执行实际转换",-1)),t("div",B,[l(r,{onClick:d},{default:e(()=>[...o[6]||(o[6]=[i("重置",-1)])]),_:1}),l(_,{content:"转换功能将在后续开发中接入",placement:"top"},{default:e(()=>[t("span",null,[l(r,{type:"primary",disabled:""},{default:e(()=>[...o[7]||(o[7]=[i("开始转换",-1)])]),_:1})])]),_:1})])])])]),_:1})}}}),it=y(k,[["__scopeId","data-v-89a8493f"]]);export{it as default};

frontend/dist/assets/DataConvertView-DJuX-JIl.css

@@ -0,0 +1 @@
+.converter-panel[data-v-fcb69543]{width:100%;min-height:calc(100vh - 220px);border:1px solid #e4e7ed;border-radius:8px;background:#fff;display:flex;flex-direction:column}.panel-header[data-v-fcb69543]{min-height:72px;padding:16px 20px;border-bottom:1px solid #ebeef5;background:#fafafa;display:flex;align-items:center;gap:12px;box-sizing:border-box}.panel-header .tool-icon[data-v-fcb69543]{width:38px;height:38px;flex:0 0 auto;border-radius:6px;background:var(--el-color-primary-light-9);color:var(--primary-color);display:flex;align-items:center;justify-content:center}.panel-header h3[data-v-fcb69543]{margin:0;color:#303133;font-size:15px;font-weight:600}.panel-header p[data-v-fcb69543]{margin:4px 0 0;color:#909399;font-size:12px}.converter-form[data-v-fcb69543]{flex:1;padding:22px 24px 6px}.converter-form[data-v-fcb69543] .el-form-item{margin-bottom:20px}.converter-form[data-v-fcb69543] .el-form-item__label{padding-bottom:8px;color:#606266;font-size:13px}.format-field[data-v-fcb69543]{width:100%;min-height:40px;padding:0 14px;border:1px solid #dcdfe6;border-radius:4px;background:#f5f7fa;color:#303133;display:flex;align-items:center;gap:14px;box-sizing:border-box;font-size:13px;font-weight:500}.format-field i[data-v-fcb69543]{color:#909399}.upload-zone[data-v-fcb69543]{width:100%;min-height:112px;padding:20px;border:1px dashed #b8c4d1;border-radius:6px;background:#fafcff;color:#606266;cursor:pointer;font:inherit;display:flex;align-items:center;gap:14px;text-align:left;transition:border-color .2s ease,background .2s ease}.upload-zone[data-v-fcb69543]:hover,.upload-zone[data-v-fcb69543]:focus-visible{border-color:var(--primary-color);background:var(--el-color-primary-light-9);outline:none}.upload-zone>i[data-v-fcb69543]{color:var(--primary-color);font-size:24px}.upload-zone .upload-content[data-v-fcb69543]{min-width:0;display:flex;flex:1;flex-direction:column;gap:5px}.upload-zone strong[data-v-fcb69543]{color:#303133;font-size:13px;font-weight:500}.upload-zone small[data-v-fcb69543]{color:#909399;font-size:12px}.upload-zone .select-button[data-v-fcb69543]{min-height:32px;padding:0 14px;border:1px solid #dcdfe6;border-radius:4px;background:#fff;color:#606266;display:inline-flex;align-items:center;white-space:nowrap}.form-row[data-v-fcb69543]{display:grid;grid-template-columns:minmax(0,2fr) minmax(180px,1fr);gap:16px}.format-tip[data-v-fcb69543]{min-height:38px;padding:9px 12px;border-radius:4px;background:var(--el-color-primary-light-9);color:#606266;display:flex;align-items:center;gap:8px;box-sizing:border-box;font-size:12px}.format-tip i[data-v-fcb69543]{color:var(--primary-color)}.panel-footer[data-v-fcb69543]{min-height:64px;padding:12px 24px;border-top:1px solid #ebeef5;background:#fafafa;display:flex;align-items:center;justify-content:space-between;gap:20px;box-sizing:border-box}.panel-footer .prototype-label[data-v-fcb69543]{color:#909399;font-size:12px}.panel-footer .actions[data-v-fcb69543]{display:flex;gap:10px}@media(max-width:640px){.converter-form[data-v-fcb69543]{padding:18px 16px 4px}.form-row[data-v-fcb69543]{grid-template-columns:1fr;gap:0}.upload-zone[data-v-fcb69543]{align-items:flex-start;flex-wrap:wrap}.upload-zone .select-button[data-v-fcb69543]{margin-left:38px}.panel-footer[data-v-fcb69543]{padding:12px 16px;align-items:flex-end;flex-direction:column}}

frontend/dist/assets/DataConvertView-DYHO4l1y.js

@@ -0,0 +1 @@
+import{a as N,E as v}from"./el-form-item-D5kF3B90.js";import{E as b}from"./el-popper-D6_hxRbQ.js";import{E as O}from"./index-TFUf94PZ.js";import{E as S}from"./index-BjEW7-SA.js";import{E as g,a as E}from"./el-select-Dl-FRZQd.js";import{d as J,e as V,w as e,o as w,q as t,s as l,x as i,z as x,A as C}from"./index-BKKvzUDD.js";import"./el-tooltip-l0sNRNKZ.js";import"./el-scrollbar-ClJnvz-9.js";/* empty css               */import{P as U}from"./PageCard-BoKXOzst.js";import{_ as y}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./index-GAnQrJsQ.js";import"./index-CWUnzf90.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./vnode-78qeDweP.js";import"./index-D5ryD3I5.js";import"./index-BDEF353-.js";import"./scroll-BkzZKETR.js";import"./clamp-CbbY8h6F.js";import"./toNumber-Dkj3QRv9.js";import"./_baseIteratee-BAilevJ_.js";import"./el-card-CApHJ1Gj.js";const F={class:"converter-panel"},L={class:"form-row"},T={class:"panel-footer"},B={class:"actions"},k=J({__name:"DataConvertView",setup(I){const a=x({outputName:"converted-data",encoding:"UTF-8"});function p(){C.info("当前仅完成界面设计，转换功能将在后续接入")}function d(){Object.assign(a,{outputName:"converted-data",encoding:"UTF-8"})}return(j,o)=>{const n=N,m=O,u=E,f=g,c=v,r=S,_=b;return w(),V(U,{class:"data-convert-page",title:"数据类型转换",subtitle:"将 JSON 文件转换为便于训练和评测使用的 JSONL 格式"},{default:e(()=>[t("div",F,[o[9]||(o[9]=t("div",{class:"panel-header"},[t("div",{class:"tool-icon","aria-hidden":"true"},[t("i",{class:"fa fa-exchange"})]),t("div",null,[t("h3",null,"JSON 转 JSONL"),t("p",null,"每条 JSON 数据将输出为 JSONL 文件中的一行记录")])],-1)),l(c,{class:"converter-form","label-position":"top"},{default:e(()=>[l(n,{label:"转换类型"},{default:e(()=>[...o[2]||(o[2]=[t("div",{class:"format-field","aria-label":"JSON 转 JSONL"},[t("span",null,"JSON"),t("i",{class:"fa fa-long-arrow-right","aria-hidden":"true"}),t("span",null,"JSONL")],-1)])]),_:1}),l(n,{label:"源文件",required:""},{default:e(()=>[t("button",{class:"upload-zone",type:"button",onClick:p},[...o[3]||(o[3]=[t("i",{class:"fa fa-cloud-upload","aria-hidden":"true"},null,-1),t("span",{class:"upload-content"},[t("strong",null,"点击选择或拖拽 JSON 文件到此处"),t("small",null,"仅支持 .json 格式，单文件不超过 200 MB")],-1),t("span",{class:"select-button"},"选择文件",-1)])])]),_:1}),t("div",L,[l(n,{label:"输出文件名"},{default:e(()=>[l(m,{modelValue:a.outputName,"onUpdate:modelValue":o[0]||(o[0]=s=>a.outputName=s)},{append:e(()=>[...o[4]||(o[4]=[i(".jsonl",-1)])]),_:1},8,["modelValue"])]),_:1}),l(n,{label:"字符编码"},{default:e(()=>[l(f,{modelValue:a.encoding,"onUpdate:modelValue":o[1]||(o[1]=s=>a.encoding=s),style:{width:"100%"}},{default:e(()=>[l(u,{label:"UTF-8",value:"UTF-8"})]),_:1},8,["modelValue"])]),_:1})]),o[5]||(o[5]=t("div",{class:"format-tip"},[t("i",{class:"fa fa-info-circle","aria-hidden":"true"}),t("span",null,"支持由 JSON 数组转换为 JSONL，每个数组元素输出为一行。")],-1))]),_:1}),t("div",T,[o[8]||(o[8]=t("span",{class:"prototype-label"},"当前为 UI 原型，暂不执行实际转换",-1)),t("div",B,[l(r,{onClick:d},{default:e(()=>[...o[6]||(o[6]=[i("重置",-1)])]),_:1}),l(_,{content:"转换功能将在后续开发中接入",placement:"top"},{default:e(()=>[t("span",null,[l(r,{type:"primary",disabled:""},{default:e(()=>[...o[7]||(o[7]=[i("开始转换",-1)])]),_:1})])]),_:1})])])])]),_:1})}}}),rt=y(k,[["__scopeId","data-v-fcb69543"]]);export{rt as default};

frontend/dist/assets/DataConvertView-rvGLlrQa.css

@@ -1 +0,0 @@
-.converter-panel[data-v-89a8493f]{width:100%;min-height:calc(100vh - 220px);border:1px solid #e4e7ed;border-radius:8px;background:#fff;display:flex;flex-direction:column}.panel-header[data-v-89a8493f]{min-height:72px;padding:16px 20px;border-bottom:1px solid #ebeef5;background:#fafafa;display:flex;align-items:center;gap:12px;box-sizing:border-box}.panel-header .tool-icon[data-v-89a8493f]{width:38px;height:38px;flex:0 0 auto;border-radius:6px;background:var(--el-color-primary-light-9);color:var(--primary-color);display:flex;align-items:center;justify-content:center}.panel-header h3[data-v-89a8493f]{margin:0;color:#303133;font-size:15px;font-weight:600}.panel-header p[data-v-89a8493f]{margin:4px 0 0;color:#909399;font-size:12px}.converter-form[data-v-89a8493f]{flex:1;padding:22px 24px 6px}.converter-form[data-v-89a8493f] .el-form-item{margin-bottom:20px}.converter-form[data-v-89a8493f] .el-form-item__label{padding-bottom:8px;color:#606266;font-size:13px}.format-field[data-v-89a8493f]{width:100%;min-height:40px;padding:0 14px;border:1px solid #dcdfe6;border-radius:4px;background:#f5f7fa;color:#303133;display:flex;align-items:center;gap:14px;box-sizing:border-box;font-size:13px;font-weight:500}.format-field i[data-v-89a8493f]{color:#909399}.upload-zone[data-v-89a8493f]{width:100%;min-height:112px;padding:20px;border:1px dashed #b8c4d1;border-radius:6px;background:#fafcff;color:#606266;cursor:pointer;font:inherit;display:flex;align-items:center;gap:14px;text-align:left;transition:border-color .2s ease,background .2s ease}.upload-zone[data-v-89a8493f]:hover,.upload-zone[data-v-89a8493f]:focus-visible{border-color:var(--primary-color);background:var(--el-color-primary-light-9);outline:none}.upload-zone>i[data-v-89a8493f]{color:var(--primary-color);font-size:24px}.upload-zone .upload-content[data-v-89a8493f]{min-width:0;display:flex;flex:1;flex-direction:column;gap:5px}.upload-zone strong[data-v-89a8493f]{color:#303133;font-size:13px;font-weight:500}.upload-zone small[data-v-89a8493f]{color:#909399;font-size:12px}.upload-zone .select-button[data-v-89a8493f]{min-height:32px;padding:0 14px;border:1px solid #dcdfe6;border-radius:4px;background:#fff;color:#606266;display:inline-flex;align-items:center;white-space:nowrap}.form-row[data-v-89a8493f]{display:grid;grid-template-columns:minmax(0,2fr) minmax(180px,1fr);gap:16px}.format-tip[data-v-89a8493f]{min-height:38px;padding:9px 12px;border-radius:4px;background:var(--el-color-primary-light-9);color:#606266;display:flex;align-items:center;gap:8px;box-sizing:border-box;font-size:12px}.format-tip i[data-v-89a8493f]{color:var(--primary-color)}.panel-footer[data-v-89a8493f]{min-height:64px;padding:12px 24px;border-top:1px solid #ebeef5;background:#fafafa;display:flex;align-items:center;justify-content:space-between;gap:20px;box-sizing:border-box}.panel-footer .prototype-label[data-v-89a8493f]{color:#909399;font-size:12px}.panel-footer .actions[data-v-89a8493f]{display:flex;gap:10px}@media(max-width:640px){.converter-form[data-v-89a8493f]{padding:18px 16px 4px}.form-row[data-v-89a8493f]{grid-template-columns:1fr;gap:0}.upload-zone[data-v-89a8493f]{align-items:flex-start;flex-wrap:wrap}.upload-zone .select-button[data-v-89a8493f]{margin-left:38px}.panel-footer[data-v-89a8493f]{padding:12px 16px;align-items:flex-end;flex-direction:column}}

frontend/dist/assets/DataProcessListView-D--2V8HO.css

@@ -1 +0,0 @@
-.action-buttons[data-v-71f2711e]{display:flex;justify-content:center;gap:8px}

frontend/dist/assets/DataProcessListView-D6X_r5Q_.css

@@ -0,0 +1 @@
+.action-buttons[data-v-c0da9336]{display:flex;justify-content:center;gap:8px}

frontend/dist/assets/DataProcessListView-DUOxJsmF.js

@@ -1 +0,0 @@
-import{E as B}from"./index-BHESvfAo.js";import{E as C}from"./el-tag-ulIZLhPi.js";import{E as N}from"./el-table-BpaXJdL-.js";import{d as z,G as P,c as g,s as o,w as s,y as n,o as d,q as f,x as r,e as V,aa as c,A as I,v as L}from"./index-z4H3KYDz.js";import{E as M}from"./index-C_KL97zv.js";import"./el-checkbox-CODdaR7b.js";import{D as $}from"./DataTablePage-DLeZhncr.js";import{_ as S}from"./ModelStatusTag.vue_vue_type_script_setup_true_lang-CRiYR4QO.js";import{g as q,d as A}from"./dataProcess-CWOEiOM9.js";import{_ as G}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./el-scrollbar-Cs4RtPFn.js";import"./index-CeeJbXQu.js";import"./el-popper-BJIpIz7j.js";import"./index-DJMIMot_.js";import"./_baseClone-DT8DugHo.js";import"./_baseIteratee-BSbGVKbQ.js";import"./castArray-DuG_gufG.js";import"./debounce-DozRow7U.js";import"./toNumber-DmGFONFK.js";import"./raf-CGFVbd0C.js";import"./validator-Cv5bUbtV.js";import"./index-3Nih6or2.js";import"./clamp-Cqzl_HF6.js";import"./index-5agRb3jZ.js";import"./scroll-CJzUJVH-.js";import"./index-OPiDM056.js";import"./index-C6iZvNqO.js";import"./omit-8mGWGArl.js";import"./el-card-DLSyQgTR.js";import"./el-pagination-P9pb1Tpb.js";import"./el-select-OcCw1rK5.js";import"./vnode-Cz-7S2If.js";import"./index-BANvEWbb.js";import"./directive-BjB4tpmk.js";import"./el-tooltip-l0sNRNKZ.js";import"./status-Dl1fykxa.js";const R={class:"data-process-page",style:{height:"100%"}},j={key:1},F={class:"action-buttons"},H=z({__name:"DataProcessListView",setup(J){const y={structured:"结构化数据",unstructured:"非结构化数据",external:"外来数据源拉取"},h=L(),l=n([]),p=n(!1),m=n(null),u=n("");async function v(t=!1){t||(p.value=!0),u.value="";try{const a=await q({page:1,page_size:200});l.value=a.items}catch{u.value="数据处理任务加载失败，请稍后重试。"}finally{t||(p.value=!1)}}function x(t){const a=t.id;h.push({name:"data-process-detail",params:{id:a}})}async function b(t){try{await M.confirm(`确定删除数据处理任务“${t.name}”吗？删除后无法恢复。`,"确认删除",{type:"warning",confirmButtonText:"删除",cancelButtonText:"取消",confirmButtonClass:"el-button--danger"})}catch{return}m.value=t.id;try{await A(t.id),l.value=l.value.filter(a=>a.id!==t.id),I.success("数据处理任务已删除")}catch{}finally{m.value=null}}function D(t){if(!t)return"-";const a=new Date(t);return Number.isNaN(a.getTime())?t:a.toLocaleString("zh-CN",{hour12:!1})}function k(t){return t.source_dataset_name||t.source_dataset||"-"}function w(t){return t.output_dataset_name||t.output_dataset||"-"}return P(v),(t,a)=>{const i=N,E=C,_=B;return d(),g("div",R,[o($,{title:"",data:l.value,loading:p.value,searchable:"","search-fields":["name"],"create-text":"新建数据处理","create-to":"/data-process/create","row-key":"id","page-size":10,"empty-text":u.value||"暂无数据处理任务"},{columns:s(()=>[o(i,{label:"任务ID",prop:"id",align:"center",width:"100"}),o(i,{label:"任务名称",prop:"name",align:"center","show-overflow-tooltip":""}),o(i,{label:"任务状态",align:"center",width:"110"},{default:s(({row:e})=>[o(S,{status:e.status},null,8,["status"])]),_:1}),o(i,{label:"处理类型",align:"center",width:"140"},{default:s(({row:e})=>[e.process_type?(d(),V(E,{key:0,size:"small",type:"info",effect:"plain"},{default:s(()=>[r(c(y[e.process_type]||e.process_type),1)]),_:2},1024)):(d(),g("span",j,"-"))]),_:1}),o(i,{label:"源数据集",align:"center","show-overflow-tooltip":""},{default:s(({row:e})=>[r(c(k(e)),1)]),_:1}),o(i,{label:"输出数据集",align:"center","show-overflow-tooltip":""},{default:s(({row:e})=>[r(c(w(e)),1)]),_:1}),o(i,{label:"创建时间",align:"center",width:"190"},{default:s(({row:e})=>[r(c(D(e.create_time||e.created_at)),1)]),_:1})]),actions:s(({row:e})=>[f("div",F,[o(_,{type:"primary",link:"",size:"small",onClick:T=>x(e)},{default:s(()=>[...a[0]||(a[0]=[f("i",{class:"fa fa-file-text-o",style:{"margin-right":"4px"}},null,-1),r("详情 ",-1)])]),_:1},8,["onClick"]),o(_,{type:"danger",link:"",size:"small",loading:m.value===e.id,onClick:T=>b(e)},{default:s(()=>[...a[1]||(a[1]=[f("i",{class:"fa fa-trash-o",style:{"margin-right":"4px"}},null,-1),r("删除 ",-1)])]),_:1},8,["loading","onClick"])])]),_:1},8,["data","loading","empty-text"])])}}}),Ct=G(H,[["__scopeId","data-v-71f2711e"]]);export{Ct as default};

frontend/dist/assets/DataProcessListView-Ds7sKAJx.js

@@ -0,0 +1 @@
+import{E as b}from"./index-BjEW7-SA.js";import{E as k}from"./index-BDEF353-.js";import{E as x}from"./el-table-D8K5zLyz.js";import{d as v,c as m,s as e,w as a,y as D,o as l,q as p,x as i,e as C,aa as n,A as E,v as T}from"./index-BKKvzUDD.js";/* empty css               */import"./el-checkbox-qGZ2-RM2.js";import{D as z}from"./DataTablePage-C-EM-7-s.js";import{_ as B}from"./ModelStatusTag.vue_vue_type_script_setup_true_lang-VFIQNBU2.js";import{_ as V}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./el-scrollbar-ClJnvz-9.js";import"./index-GAnQrJsQ.js";import"./el-popper-D6_hxRbQ.js";import"./index-CWUnzf90.js";import"./_baseClone-B5RBzbh5.js";import"./_baseIteratee-BAilevJ_.js";import"./castArray-5uErZEc3.js";import"./debounce-ByXPNh5F.js";import"./toNumber-Dkj3QRv9.js";import"./raf-C-x62Pcl.js";import"./omit-BTSq4AYh.js";import"./index-TFUf94PZ.js";import"./index-DLzof2Fz.js";import"./el-card-CApHJ1Gj.js";import"./el-pagination-DOPYW-qb.js";import"./el-select-Dl-FRZQd.js";import"./vnode-78qeDweP.js";import"./index-D5ryD3I5.js";import"./scroll-BkzZKETR.js";import"./clamp-CbbY8h6F.js";import"./directive-DsZckUTJ.js";import"./index-dNo5H1ie.js";import"./validator-Fn0bm4xU.js";import"./index-CQXVR1Ud.js";import"./index-DfyOU94W.js";import"./el-tooltip-l0sNRNKZ.js";const L={class:"data-process-page",style:{height:"100%"}},N={key:1},I={class:"action-buttons"},P=v({__name:"DataProcessListView",setup(S){const u={structured:"结构化数据",unstructured:"非结构化数据",external:"外来数据源拉取"},d=T(),_=D([{id:183921,name:"客服问答数据清洗",status:"completed",process_type:"structured",source_dataset:"客服对话原始集",output_dataset:"客服对话清洗集",create_time:"2026-07-08 14:23:00"},{id:492015,name:"指令微调数据构造",status:"running",process_type:"unstructured",source_dataset:"通用语料库",output_dataset:"SFT 指令集",create_time:"2026-07-09 09:10:00"},{id:731948,name:"敏感信息脱敏处理",status:"pending",process_type:"structured",source_dataset:"用户反馈数据",create_time:"2026-07-09 16:45:00"},{id:582012,name:"多轮对话拼接",status:"failed",process_type:"structured",source_dataset:"单轮问答集",create_time:"2026-07-10 08:30:00"}]);function f(s){const r=s.id;d.push({name:"data-process-detail",params:{id:r}})}function g(s){E.info("删除功能开发中...")}function y(s){return s?new Date(s).toLocaleString("zh-CN",{hour12:!1}):"-"}return(s,r)=>{const o=x,h=k,c=b;return l(),m("div",L,[e(z,{title:"",data:_.value,searchable:"","search-fields":["name"],"create-text":"新建数据处理","create-to":"/data-process/create","row-key":"id","page-size":10},{columns:a(()=>[e(o,{label:"任务ID",prop:"id",align:"center",width:"100"}),e(o,{label:"任务名称",prop:"name",align:"center","show-overflow-tooltip":""}),e(o,{label:"任务状态",align:"center",width:"110"},{default:a(({row:t})=>[e(B,{status:t.status},null,8,["status"])]),_:1}),e(o,{label:"处理类型",align:"center",width:"140"},{default:a(({row:t})=>[t.process_type?(l(),C(h,{key:0,size:"small",type:"info",effect:"plain"},{default:a(()=>[i(n(u[t.process_type]||t.process_type),1)]),_:2},1024)):(l(),m("span",N,"-"))]),_:1}),e(o,{label:"源数据集",align:"center","show-overflow-tooltip":""},{default:a(({row:t})=>[i(n(t.source_dataset||"-"),1)]),_:1}),e(o,{label:"输出数据集",align:"center","show-overflow-tooltip":""},{default:a(({row:t})=>[i(n(t.output_dataset||"-"),1)]),_:1}),e(o,{label:"创建时间",align:"center",width:"190"},{default:a(({row:t})=>[i(n(y(t.create_time)),1)]),_:1})]),actions:a(({row:t})=>[p("div",I,[e(c,{type:"primary",link:"",size:"small",onClick:w=>f(t)},{default:a(()=>[...r[0]||(r[0]=[p("i",{class:"fa fa-file-text-o",style:{"margin-right":"4px"}},null,-1),i("详情 ",-1)])]),_:1},8,["onClick"]),e(c,{type:"danger",link:"",size:"small",onClick:w=>g(t)},{default:a(()=>[...r[1]||(r[1]=[p("i",{class:"fa fa-trash-o",style:{"margin-right":"4px"}},null,-1),i("删除 ",-1)])]),_:1},8,["onClick"])])]),_:1},8,["data"])])}}}),yt=V(P,[["__scopeId","data-v-c0da9336"]]);export{yt as default};