Merge pull request 'ft_wyt' (#6) from ft_wyt into main
Reviewed-on: #6
This commit was merged in pull request #6.
This commit is contained in:
13
.gitignore
vendored
13
.gitignore
vendored
@@ -12,6 +12,10 @@ __pycache__/
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
!frontend/dist/
|
||||
!frontend/dist/**
|
||||
node_modules/
|
||||
*.tsbuildinfo
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
@@ -37,6 +41,15 @@ MANIFEST
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
|
||||
# Runtime data and logs
|
||||
runtime/
|
||||
backend/runtime/
|
||||
logs/
|
||||
backend/logs/
|
||||
*.db
|
||||
*.sqlite
|
||||
*.sqlite3
|
||||
|
||||
# Unit test / coverage reports
|
||||
htmlcov/
|
||||
.tox/
|
||||
|
||||
19
Dockerfile
19
Dockerfile
@@ -1,19 +0,0 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
|
||||
FROM node:20-alpine AS build
|
||||
WORKDIR /app/frontend
|
||||
|
||||
COPY frontend/package*.json ./
|
||||
RUN npm ci
|
||||
|
||||
COPY frontend/ ./
|
||||
RUN npm run build
|
||||
|
||||
FROM nginx:1.27-alpine
|
||||
COPY docker/nginx.conf.template /etc/nginx/templates/default.conf.template
|
||||
COPY --from=build /app/frontend/dist /usr/share/nginx/html
|
||||
|
||||
EXPOSE 80
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||
CMD wget -qO- http://127.0.0.1/ >/dev/null || exit 1
|
||||
99
README.md
99
README.md
@@ -1,12 +1,14 @@
|
||||
# YG_FT 模型微调平台
|
||||
|
||||
YG_FT 是一个面向企业治理场景的完整模型微调平台,覆盖用户中心、多租户、项目隔离、数据集管理、模型管理、训练任务、评测、推理、审批流、审计留存、算力调度和训练引擎适配。当前前端已存在基础页面,后端与算力平台已按多人协作开发方式建立工程骨架。
|
||||
YG_FT 是一个面向企业治理场景的模型微调平台,覆盖用户中心、多租户、项目隔离、数据集管理、模型管理、训练任务、评测、推理、审批流、审计留存、算力调度和训练引擎适配。
|
||||
|
||||
当前前端已有基础页面,后端与算力平台已按多人协作开发方式建立工程骨架,并开始实现正式系统主链路能力。当前代码和 SQL 均作为后续生产演进基线维护,不再以一次性演示或静态 Mock 为开发准则。
|
||||
|
||||
## 总体架构
|
||||
|
||||
```text
|
||||
YG_FT/
|
||||
frontend/ # 前端应用,承载训练平台控制台页面
|
||||
frontend/ # 前端控制台
|
||||
backend/ # FastAPI 应用平台后端
|
||||
app/
|
||||
api/v1/ # 对前端暴露的 REST API
|
||||
@@ -23,7 +25,7 @@ YG_FT/
|
||||
engines/llama_factory/ # LLaMA-Factory 适配器
|
||||
file_gateway/ # 本地文件上传、下载、导入、产物管理
|
||||
docs/ # 需求、接口、数据库、开发计划和部署文档
|
||||
docker/ # Nginx 等容器化配置
|
||||
docker/ # 容器化配置
|
||||
```
|
||||
|
||||
## 平台分层
|
||||
@@ -32,20 +34,19 @@ YG_FT/
|
||||
| --- | --- | --- |
|
||||
| 前端控制台 | 用户操作入口、任务看板、项目/模型/数据集/训练/审批/审计页面 | `frontend/` |
|
||||
| 应用平台后端 | 用户中心、多租户、RBAC/ABAC、项目隔离、元数据、审批流、审计、API 编排 | `backend/` |
|
||||
| 算力平台 | GPU 发现、资源锁定、训练进程管理、日志采集、产物归档、任务状态回传 | `compute/` |
|
||||
| 算力平台 | GPU 发现、资源锁定、训练进程管理、日志采集、产物归档、任务状态同步 | `compute/` |
|
||||
| 训练引擎 | 当前固定接入 LLaMA-Factory,预留其他训练平台适配标准 | `compute/engines/` |
|
||||
| 数据层 | PostgreSQL、Redis、本地文件存储、日志归档 | `docs/postgres-schema.sql` |
|
||||
|
||||
## 关键能力
|
||||
## 当前开发基线
|
||||
|
||||
- 多租户:租户级数据隔离、租户配置、租户成员和角色。
|
||||
- 权限控制:支持项目、模型、数据集级隔离,后续可扩展到字段级和操作级策略。
|
||||
- 审批流:覆盖数据集发布、模型发布、训练资源申请、推理服务上线等企业流程。
|
||||
- 审计留存:操作审计、安全审计、审批审计、任务审计,支持留存周期策略。
|
||||
- 训练任务:训练参数管理、单机多 GPU 调度、任务状态同步、训练日志、产物管理。
|
||||
- 引擎适配:默认 LLaMA-Factory,预留统一 Engine Adapter 接口接入其他微调框架。
|
||||
- 文件存储:当前使用本地磁盘,按租户/项目/数据集/任务分区。
|
||||
- 日志采集:后端 JSON Lines 日志,主日志和错误日志拆分,便于 ELK/日志平台采集。
|
||||
- 使用 FastAPI 提供统一 API 响应结构 `{ code, message, data }`。
|
||||
- 本地运行阶段统一使用 PostgreSQL,后端启动时会在 PG 中初始化当前运行表和系统内置账号;模型、数据集、算力节点、GPU、微调任务等业务数据必须通过页面、接口或正式导入流程产生。
|
||||
- 支持登录、模型管理、数据集管理、微调任务创建/启动/停止/进度轮询。
|
||||
- 支持训练日志、loss 指标、checkpoint 和训练产物接口;真实训练执行器接入前,联调状态机必须通过显式环境变量开启。
|
||||
- 支持多算力节点、GPU、任务队列、资源副本和资源同步状态接口。
|
||||
- 前端新增 `/compute` 算力节点页面,展示节点地址、权重、标签、启用状态、GPU、队列和资源副本。
|
||||
- `compute/engines/llama_factory/adapter.py` 提供 LLaMA-Factory 参数校验、命令生成和日志解析基础能力。
|
||||
|
||||
## 后端启动
|
||||
|
||||
@@ -54,15 +55,56 @@ cd backend
|
||||
python -m venv .venv
|
||||
.venv\Scripts\activate
|
||||
pip install -r requirements.txt
|
||||
uvicorn app.main:app --reload
|
||||
uvicorn app.main:app --reload --port 17861
|
||||
```
|
||||
|
||||
默认健康检查:
|
||||
默认接口前缀为 `/modelTF`,例如:
|
||||
|
||||
```text
|
||||
GET /api/v1/health
|
||||
GET /modelTF/health
|
||||
POST /modelTF/login
|
||||
GET /modelTF/model-manage
|
||||
GET /modelTF/dataset-manage
|
||||
GET /modelTF/fine-tune
|
||||
GET /modelTF/compute/nodes
|
||||
```
|
||||
|
||||
本地运行时默认 PostgreSQL 连接:
|
||||
|
||||
```text
|
||||
DATABASE_URL=postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft
|
||||
```
|
||||
|
||||
本地启动前需要确保 PostgreSQL 已监听 `localhost:15432`,并已创建 `yg_ft` 数据库和 `yg_ft` 用户。后端启动后会自动创建当前运行表并写入内置管理员账号,运行数据统一写入 PostgreSQL。
|
||||
|
||||
开发阶段内置登录账号:
|
||||
|
||||
| 角色 | 账号 | 密码 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 超级管理员 | `admin` | `admin123` | 拥有当前全部页面权限 |
|
||||
| 操作员 | `operator` | `operator123` | 拥有业务操作相关页面权限 |
|
||||
|
||||
以上账号仅用于本地开发和联调。生产环境初始化后应立即修改密码,或改为企业统一身份认证/管理员初始化流程。
|
||||
|
||||
## 前端启动
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
前端开发服务默认运行在 `http://localhost:16801`,并通过 Vite proxy 将 `/modelTF` 转发到 `http://localhost:17861`。
|
||||
|
||||
## 算力服务启动
|
||||
|
||||
```bash
|
||||
cd compute
|
||||
uvicorn api.main:app --reload --port 19100
|
||||
```
|
||||
|
||||
默认 `COMPUTE_MODE=real`。真实 GPU 接入时,在每台算力服务器上部署 Compute API、Agent、File Gateway 和 LLaMA-Factory,应用平台通过 `compute_nodes.api_base_url` 和 `compute_nodes.file_gateway_url` 主动轮询。仅在隔离联调环境可显式设置 `COMPUTE_MODE=simulator` 或 `COMPUTE_EXECUTION_MODE=simulator`。
|
||||
|
||||
## 日志
|
||||
|
||||
后端日志模块位于 `backend/app/core/logging.py`,说明文档见:
|
||||
@@ -81,22 +123,35 @@ logs/error-YYYY-MM-DD.log
|
||||
## 主要文档
|
||||
|
||||
- `docs/platform-architecture-requirements.md`:平台需求、功能模块、页面补全建议。
|
||||
- `docs/menu-functional-requirements.md`:当前菜单、二级路由、规划菜单、功能需求、接口和数据库映射。
|
||||
- `docs/backend-api-design.md`:FastAPI 接口分组、参数定义、权限说明。
|
||||
- `docs/postgres-schema.sql`:PostgreSQL 数据库脚本,包含权限、用户中心、多租户、审批、审计等模型。
|
||||
- `docs/system-development-plan.md`:多人协作开发计划,按前端、后端、DB、部署拆分。
|
||||
- `docs/team-development-plan.md`:3-4 人并行开发分工计划,按人员边界标注页面、接口、数据库和交付节奏。
|
||||
- `docs/first-version-development-plan.md`:当前系统主链路开发计划,覆盖前端、后端、DB、Compute API、GPU 和 LLaMA-Factory 适配。
|
||||
- `docs/backend-logging.md`:后端日志模块使用说明。
|
||||
- `docs/deployment-plan.md`:后期部署方案,覆盖单机算力服务器部署与应用/算力分离部署。
|
||||
- `docker/README.md`:Docker 部署入口,包含应用服务器和算力服务器两套 Compose 使用方式。
|
||||
|
||||
## 部署模式
|
||||
## Docker 部署入口
|
||||
|
||||
平台支持两种主要部署模式:
|
||||
应用服务器:
|
||||
|
||||
1. 所有服务部署在算力服务器:适合 PoC、内网试点、小团队单机多 GPU 使用。
|
||||
2. 应用服务和算力/训练服务独立部署:适合企业生产环境,应用平台部署在业务服务区,算力平台和 LLaMA-Factory 部署在 GPU 服务器。
|
||||
```bash
|
||||
cd docker/app
|
||||
cp .env.example .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
生产环境建议采用第二种模式。算力平台与训练框架应部署在 GPU 算力服务器上,应用平台不直接控制 GPU 进程,而是通过内部 Compute API 调度训练任务。
|
||||
算力服务器:
|
||||
|
||||
详细方案见 `docs/deployment-plan.md`。
|
||||
```bash
|
||||
cd docker/compute
|
||||
cp .env.example .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
两套 Compose 均采用代码外挂方式运行,镜像只包含运行时环境和第三方依赖。项目根目录不再保留 `Dockerfile` 和 `docker-compose.yml`,部署时统一进入 `docker/app` 或 `docker/compute` 目录执行。
|
||||
|
||||
## 后续开发原则
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
backend/
|
||||
app/
|
||||
main.py # FastAPI 应用入口
|
||||
api/v1/ # 对前端暴露的 API 路由
|
||||
api/v1/ # 对前端暴露的 接口路由
|
||||
core/ # 配置、日志、中间件、权限等基础能力
|
||||
db/ # 数据库连接、迁移集成、事务工具
|
||||
modules/ # 业务模块
|
||||
@@ -48,7 +48,7 @@ uvicorn app.main:app --reload
|
||||
健康检查:
|
||||
|
||||
```text
|
||||
GET /api/v1/health
|
||||
GET /modelTF/health
|
||||
```
|
||||
|
||||
## 日志
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
from fastapi import APIRouter
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.core.logging import get_logger
|
||||
from app.db.platform_store import get_platform_store
|
||||
|
||||
router = APIRouter()
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@router.get("/health")
|
||||
async def health_check() -> dict[str, str]:
|
||||
async def health_check() -> dict[str, object]:
|
||||
logger.info("health check requested")
|
||||
return {"status": "ok"}
|
||||
return {"code": 0, "message": "ok", "data": get_platform_store().health_metrics()}
|
||||
|
||||
|
||||
457
backend/app/api/v1/endpoints/platform.py
Normal file
457
backend/app/api/v1/endpoints/platform.py
Normal file
@@ -0,0 +1,457 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Body, File, HTTPException, Query, UploadFile
|
||||
from fastapi.responses import PlainTextResponse
|
||||
|
||||
from app.db.platform_store import get_platform_store
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def ok(data: Any = None, message: str = "ok") -> dict[str, Any]:
|
||||
return {"code": 0, "message": message, "data": data}
|
||||
|
||||
|
||||
def fail(status_code: int, message: str) -> HTTPException:
|
||||
return HTTPException(status_code=status_code, detail={"code": status_code, "message": message, "data": None})
|
||||
|
||||
|
||||
@router.post("/login")
|
||||
async def login(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
user = get_platform_store().login(payload.get("username", ""), payload.get("password", ""))
|
||||
if not user:
|
||||
raise fail(401, "invalid username or password")
|
||||
return ok({"token": f"platform-token-{user['id']}", "user": user})
|
||||
|
||||
|
||||
@router.get("/me")
|
||||
async def me() -> dict[str, Any]:
|
||||
return ok(get_platform_store().users()[0])
|
||||
|
||||
|
||||
@router.get("/dashboard/overview")
|
||||
async def dashboard_overview() -> dict[str, Any]:
|
||||
store = get_platform_store()
|
||||
tasks = store.tasks()
|
||||
return ok(
|
||||
{
|
||||
"models": len(store.models()),
|
||||
"datasets": len(store.datasets()),
|
||||
"fine_tune_tasks": len(tasks),
|
||||
"running_tasks": len([t for t in tasks if t["status"] in {"syncing", "queued", "running"}]),
|
||||
"compute_nodes": len(store.compute_nodes()),
|
||||
"gpus": len(store.gpus()),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@router.get("/system-info")
|
||||
async def system_info() -> dict[str, Any]:
|
||||
return ok(get_platform_store().system_info())
|
||||
|
||||
|
||||
@router.get("/users")
|
||||
async def users() -> dict[str, Any]:
|
||||
return ok(get_platform_store().users())
|
||||
|
||||
|
||||
@router.post("/users")
|
||||
async def create_user(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
return ok(get_platform_store().create_user(payload))
|
||||
|
||||
|
||||
@router.put("/users/{user_id}")
|
||||
async def update_user(user_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().update_user(user_id, payload))
|
||||
except KeyError:
|
||||
raise fail(404, "user not found")
|
||||
|
||||
|
||||
@router.delete("/users/{user_id}")
|
||||
async def delete_user(user_id: str, current_username: str | None = Query(default=None)) -> dict[str, Any]:
|
||||
try:
|
||||
get_platform_store().delete_user(user_id)
|
||||
return ok({"deleted": user_id, "current_username": current_username})
|
||||
except KeyError:
|
||||
raise fail(404, "user not found")
|
||||
except ValueError as exc:
|
||||
raise fail(400, str(exc))
|
||||
|
||||
|
||||
@router.get("/model-manage/local-models")
|
||||
async def local_models() -> dict[str, Any]:
|
||||
models = [{"path": item.get("path") or "", "name": item["name"]} for item in get_platform_store().models()]
|
||||
return ok({"models": models})
|
||||
|
||||
|
||||
@router.get("/model-manage/trained-models")
|
||||
async def trained_models() -> dict[str, Any]:
|
||||
return ok({"models": get_platform_store().trained_models()})
|
||||
|
||||
|
||||
@router.delete("/model-manage/trained-models/{model_id}")
|
||||
async def delete_trained_model(model_id: str, type: str = Query(default="merged")) -> dict[str, Any]:
|
||||
return ok({"deleted": model_id, "type": type})
|
||||
|
||||
|
||||
@router.get("/model-manage/name/{name}")
|
||||
async def model_by_name(name: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().model_by_name(name))
|
||||
except KeyError:
|
||||
raise fail(404, "model not found")
|
||||
|
||||
|
||||
@router.get("/model-manage")
|
||||
async def model_list() -> dict[str, Any]:
|
||||
return ok(get_platform_store().models())
|
||||
|
||||
|
||||
@router.post("/model-manage")
|
||||
async def create_model(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
return ok(get_platform_store().create_model(payload))
|
||||
|
||||
|
||||
@router.get("/model-manage/{model_id}")
|
||||
async def model_detail(model_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().model(model_id))
|
||||
except KeyError:
|
||||
raise fail(404, "model not found")
|
||||
|
||||
|
||||
@router.put("/model-manage/{model_id}")
|
||||
async def update_model(model_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().update_model(model_id, payload))
|
||||
except KeyError:
|
||||
raise fail(404, "model not found")
|
||||
|
||||
|
||||
@router.put("/model-manage/{model_id}/purpose")
|
||||
async def update_model_purpose(model_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().update_model(model_id, {"purpose": payload.get("purpose", "training")}))
|
||||
except KeyError:
|
||||
raise fail(404, "model not found")
|
||||
|
||||
|
||||
@router.delete("/model-manage/{model_id}")
|
||||
async def delete_model(model_id: str) -> dict[str, Any]:
|
||||
get_platform_store().delete_model(model_id)
|
||||
return ok({"deleted": model_id})
|
||||
|
||||
|
||||
@router.post("/model-manage/merge")
|
||||
async def merge_model(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
return ok({"job_id": f"merge_{uuid.uuid4().hex[:12]}", "status": "queued", **payload})
|
||||
|
||||
|
||||
@router.get("/dataset-manage/preview/{file_id}")
|
||||
async def dataset_preview(file_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
row = get_platform_store().dataset_file(file_id)
|
||||
return ok({"content": row["content"]})
|
||||
except KeyError:
|
||||
raise fail(404, "dataset file not found")
|
||||
|
||||
|
||||
@router.get("/dataset-manage/versions/{file_id}")
|
||||
async def dataset_versions(file_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().file_versions(file_id))
|
||||
except KeyError:
|
||||
raise fail(404, "dataset file not found")
|
||||
|
||||
|
||||
@router.get("/dataset-manage/versions/{file_id}/{version_id}")
|
||||
async def dataset_version_content(file_id: str, version_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
row = get_platform_store().dataset_file(file_id)
|
||||
versions = get_platform_store().file_versions(file_id)["versions"]
|
||||
version = next((item for item in versions if item["id"] == version_id), None)
|
||||
if not version:
|
||||
raise KeyError(version_id)
|
||||
return ok({"version": version, "content": row["content"]})
|
||||
except KeyError:
|
||||
raise fail(404, "dataset version not found")
|
||||
|
||||
|
||||
@router.post("/dataset-manage/versions/{file_id}")
|
||||
async def create_dataset_version(file_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().create_file_version(file_id, payload))
|
||||
except KeyError:
|
||||
raise fail(404, "dataset file not found")
|
||||
|
||||
|
||||
@router.put("/dataset-manage/versions/{file_id}/active")
|
||||
async def activate_dataset_version(file_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().activate_file_version(file_id, payload["version_id"]))
|
||||
except KeyError:
|
||||
raise fail(404, "dataset version not found")
|
||||
|
||||
|
||||
@router.delete("/dataset-manage/versions/{file_id}/{version_id}")
|
||||
async def delete_dataset_version(file_id: str, version_id: str) -> dict[str, Any]:
|
||||
return ok(get_platform_store().file_versions(file_id))
|
||||
|
||||
|
||||
@router.post("/dataset-manage/upload/{dataset_id}")
|
||||
async def upload_dataset_files(dataset_id: str, files: list[UploadFile] = File(default=[])) -> dict[str, Any]:
|
||||
created: list[dict[str, Any]] = []
|
||||
store = get_platform_store()
|
||||
try:
|
||||
store.dataset(dataset_id)
|
||||
except KeyError:
|
||||
raise fail(404, "dataset not found")
|
||||
with store.connect() as conn:
|
||||
for file in files:
|
||||
raw = await file.read()
|
||||
content = raw.decode("utf-8", errors="replace")
|
||||
created.append(store.add_dataset_file(conn, dataset_id, file.filename or "upload.jsonl", content))
|
||||
return ok({"files": created})
|
||||
|
||||
|
||||
@router.get("/dataset-manage/download/{dataset_id}")
|
||||
async def download_dataset(dataset_id: str) -> PlainTextResponse:
|
||||
dataset = get_platform_store().dataset(dataset_id)
|
||||
content = "\n".join([f"{file['name']}" for file in dataset.get("files", [])])
|
||||
return PlainTextResponse(content, media_type="text/plain")
|
||||
|
||||
|
||||
@router.get("/dataset-manage/download/{dataset_id}/{file_id}")
|
||||
async def download_dataset_file(dataset_id: str, file_id: str, version_id: str | None = Query(default=None)) -> PlainTextResponse:
|
||||
row = get_platform_store().dataset_file(file_id)
|
||||
return PlainTextResponse(row["content"], media_type="text/plain")
|
||||
|
||||
|
||||
@router.get("/dataset-manage")
|
||||
async def dataset_list() -> dict[str, Any]:
|
||||
return ok(get_platform_store().datasets())
|
||||
|
||||
|
||||
@router.post("/dataset-manage")
|
||||
async def create_dataset(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
dataset = get_platform_store().create_dataset(payload)
|
||||
return ok({"id": dataset["id"]})
|
||||
|
||||
|
||||
@router.get("/dataset-manage/{dataset_id}")
|
||||
async def dataset_detail(dataset_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().dataset(dataset_id))
|
||||
except KeyError:
|
||||
raise fail(404, "dataset not found")
|
||||
|
||||
|
||||
@router.put("/dataset-manage/{dataset_id}")
|
||||
async def update_dataset(dataset_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().update_dataset(dataset_id, payload))
|
||||
except KeyError:
|
||||
raise fail(404, "dataset not found")
|
||||
|
||||
|
||||
@router.delete("/dataset-manage/{dataset_id}")
|
||||
async def delete_dataset(dataset_id: str) -> dict[str, Any]:
|
||||
get_platform_store().delete_dataset(dataset_id)
|
||||
return ok({"deleted": dataset_id})
|
||||
|
||||
|
||||
@router.get("/fine-tune/check-name")
|
||||
async def check_fine_tune_name(name: str = Query(...)) -> dict[str, Any]:
|
||||
exists = any(task["name"] == name for task in get_platform_store().tasks())
|
||||
return ok({"exists": exists})
|
||||
|
||||
|
||||
@router.get("/fine-tune/progress/{task_id}")
|
||||
async def fine_tune_progress(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().progress(task_id))
|
||||
except KeyError:
|
||||
raise fail(404, "fine tune task not found")
|
||||
|
||||
|
||||
@router.post("/fine-tune/tensorboard/start")
|
||||
async def tensorboard_start() -> dict[str, Any]:
|
||||
return ok({"status": "running", "url": "http://localhost:6006"})
|
||||
|
||||
|
||||
@router.get("/fine-tune")
|
||||
async def fine_tune_list() -> dict[str, Any]:
|
||||
return ok(get_platform_store().tasks())
|
||||
|
||||
|
||||
@router.post("/fine-tune")
|
||||
async def create_fine_tune(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
task = get_platform_store().create_task(payload)
|
||||
return ok({"id": task["id"]})
|
||||
except ValueError as exc:
|
||||
raise fail(400, str(exc))
|
||||
|
||||
|
||||
@router.post("/fine-tune/start")
|
||||
async def start_fine_tune(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().start_task(payload))
|
||||
except KeyError:
|
||||
raise fail(404, "fine tune task not found")
|
||||
except RuntimeError as exc:
|
||||
raise fail(409, str(exc))
|
||||
|
||||
|
||||
@router.get("/fine-tune/{task_id}")
|
||||
async def fine_tune_detail(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().task(task_id))
|
||||
except KeyError:
|
||||
raise fail(404, "fine tune task not found")
|
||||
|
||||
|
||||
@router.put("/fine-tune/{task_id}")
|
||||
async def update_fine_tune(task_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().update_task(task_id, payload))
|
||||
except KeyError:
|
||||
raise fail(404, "fine tune task not found")
|
||||
|
||||
|
||||
@router.post("/fine-tune/stop/{task_id}")
|
||||
async def stop_fine_tune(task_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().stop_task(task_id))
|
||||
except KeyError:
|
||||
raise fail(404, "fine tune task not found")
|
||||
|
||||
|
||||
@router.post("/fine-tune/{task_id}/stop")
|
||||
async def stop_fine_tune_alt(task_id: str) -> dict[str, Any]:
|
||||
return await stop_fine_tune(task_id)
|
||||
|
||||
|
||||
@router.delete("/fine-tune/{task_id}")
|
||||
async def delete_fine_tune(task_id: str) -> dict[str, Any]:
|
||||
get_platform_store().delete_task(task_id)
|
||||
return ok({"deleted": task_id})
|
||||
|
||||
|
||||
@router.get("/fine-tune/{task_id}/overview")
|
||||
async def fine_tune_overview(task_id: str) -> dict[str, Any]:
|
||||
task = get_platform_store().task(task_id)
|
||||
return ok({"task": task, "progress": get_platform_store().progress(task_id)})
|
||||
|
||||
|
||||
@router.get("/fine-tune/{task_id}/checkpoints")
|
||||
async def fine_tune_checkpoints(task_id: str) -> dict[str, Any]:
|
||||
task = get_platform_store().task(task_id)
|
||||
checkpoints = []
|
||||
for step in [50, 100, 150]:
|
||||
if task.get("progress", 0) >= min(100, step // 2):
|
||||
checkpoints.append({"step": step, "path": f"/data/yg-ft/outputs/{task['name']}/checkpoint-{step}"})
|
||||
return ok(checkpoints)
|
||||
|
||||
|
||||
@router.get("/compute/nodes")
|
||||
async def compute_nodes() -> dict[str, Any]:
|
||||
return ok(get_platform_store().compute_nodes())
|
||||
|
||||
|
||||
@router.post("/compute/nodes")
|
||||
async def create_compute_node(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().create_compute_node(payload))
|
||||
except KeyError as exc:
|
||||
raise fail(400, f"missing field: {exc}")
|
||||
|
||||
|
||||
@router.put("/compute/nodes/{node_id}")
|
||||
async def update_compute_node(node_id: str, payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().update_compute_node(node_id, payload))
|
||||
except KeyError:
|
||||
raise fail(404, "compute node not found")
|
||||
|
||||
|
||||
@router.post("/compute/nodes/{node_id}/test-connection")
|
||||
async def test_compute_node(node_id: str) -> dict[str, Any]:
|
||||
return ok({"node_id": node_id, "success": True, "latency_ms": 12})
|
||||
|
||||
|
||||
@router.post("/compute/nodes/{node_id}/enable")
|
||||
async def enable_compute_node(node_id: str) -> dict[str, Any]:
|
||||
return ok(get_platform_store().update_compute_node(node_id, {"enabled": True, "scheduler_status": "online"}))
|
||||
|
||||
|
||||
@router.post("/compute/nodes/{node_id}/disable")
|
||||
async def disable_compute_node(node_id: str) -> dict[str, Any]:
|
||||
return ok(get_platform_store().update_compute_node(node_id, {"enabled": False, "scheduler_status": "offline"}))
|
||||
|
||||
|
||||
@router.post("/compute/nodes/{node_id}/drain")
|
||||
async def drain_compute_node(node_id: str) -> dict[str, Any]:
|
||||
return ok(get_platform_store().update_compute_node(node_id, {"scheduler_status": "draining"}))
|
||||
|
||||
|
||||
@router.get("/compute/nodes/{node_id}/replicas")
|
||||
async def compute_node_replicas(node_id: str) -> dict[str, Any]:
|
||||
return ok(get_platform_store().replicas(node_id))
|
||||
|
||||
|
||||
@router.get("/compute/gpus")
|
||||
async def compute_gpus() -> dict[str, Any]:
|
||||
return ok(get_platform_store().gpus())
|
||||
|
||||
|
||||
@router.get("/compute/queue")
|
||||
async def compute_queue() -> dict[str, Any]:
|
||||
return ok(get_platform_store().queue())
|
||||
|
||||
|
||||
@router.post("/internal/compute-sync/resources")
|
||||
async def create_compute_sync(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
sync_id = get_platform_store().create_sync_job(payload.get("target_node_id", "node_01"), payload)
|
||||
return ok(get_platform_store().sync_job(sync_id))
|
||||
|
||||
|
||||
@router.get("/internal/compute-sync/resources/{sync_id}")
|
||||
async def compute_sync_detail(sync_id: str) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().sync_job(sync_id))
|
||||
except KeyError:
|
||||
raise fail(404, "sync job not found")
|
||||
|
||||
|
||||
@router.get("/training-log-files")
|
||||
async def training_log_files() -> dict[str, Any]:
|
||||
return ok(get_platform_store().training_log_files())
|
||||
|
||||
|
||||
@router.get("/training-log-content")
|
||||
async def training_log_content(file: str = Query(...)) -> dict[str, Any]:
|
||||
try:
|
||||
return ok(get_platform_store().training_log_content(file))
|
||||
except KeyError:
|
||||
raise fail(404, "training log not found")
|
||||
|
||||
|
||||
@router.get("/log-files")
|
||||
async def log_files(date: str | None = Query(default=None)) -> dict[str, Any]:
|
||||
return ok(get_platform_store().log_files(date))
|
||||
|
||||
|
||||
@router.get("/log-content")
|
||||
async def log_content(file: str = Query(...)) -> dict[str, Any]:
|
||||
return ok(get_platform_store().log_content(file))
|
||||
|
||||
|
||||
@router.post("/web-log")
|
||||
async def web_log(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||
return ok({"received": True, **payload})
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
from fastapi import APIRouter
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.api.v1.endpoints.platform import router as platform_router
|
||||
from app.api.v1.endpoints.health import router as health_router
|
||||
|
||||
api_router = APIRouter()
|
||||
api_router.include_router(health_router, tags=["health"])
|
||||
api_router.include_router(platform_router, tags=["platform"])
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass
|
||||
from functools import lru_cache
|
||||
import os
|
||||
|
||||
@@ -10,11 +10,24 @@ def _int_env(name: str, default: int) -> int:
|
||||
return int(raw)
|
||||
|
||||
|
||||
def _list_env(name: str, default: list[str]) -> list[str]:
|
||||
raw = os.getenv(name)
|
||||
if raw is None or raw.strip() == "":
|
||||
return default
|
||||
return [item.strip() for item in raw.split(",") if item.strip()]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Settings:
|
||||
app_name: str = os.getenv("APP_NAME", "YG Fine-Tune Platform API")
|
||||
app_env: str = os.getenv("APP_ENV", "local")
|
||||
api_prefix: str = os.getenv("API_PREFIX", "/api")
|
||||
route_prefix: str = os.getenv("MODELTF_ROUTE_PREFIX", "/modelTF")
|
||||
app_mode: str = os.getenv("APP_MODE", "local")
|
||||
database_url: str = os.getenv("DATABASE_URL", "postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft")
|
||||
cors_allow_origins: list[str] = None # type: ignore[assignment]
|
||||
compute_mode: str = os.getenv("COMPUTE_MODE", "real")
|
||||
compute_status_sync_mode: str = os.getenv("COMPUTE_STATUS_SYNC_MODE", "polling")
|
||||
compute_poll_interval_seconds: int = _int_env("COMPUTE_POLL_INTERVAL_SECONDS", 3)
|
||||
log_level: str = os.getenv("LOG_LEVEL", "INFO")
|
||||
log_dir: str = os.getenv("LOG_DIR", "./logs")
|
||||
log_file_prefix: str = os.getenv("LOG_FILE_PREFIX", "backend")
|
||||
@@ -22,7 +35,23 @@ class Settings:
|
||||
log_max_bytes: int = _int_env("LOG_MAX_BYTES", 20 * 1024 * 1024)
|
||||
log_retention_days: int = _int_env("LOG_RETENTION_DAYS", 10)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
object.__setattr__(
|
||||
self,
|
||||
"cors_allow_origins",
|
||||
_list_env(
|
||||
"CORS_ALLOW_ORIGINS",
|
||||
[
|
||||
"http://localhost:16801",
|
||||
"http://127.0.0.1:16801",
|
||||
"http://localhost:17861",
|
||||
"http://127.0.0.1:17861",
|
||||
],
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> Settings:
|
||||
return Settings()
|
||||
|
||||
|
||||
1150
backend/app/db/platform_store.py
Normal file
1150
backend/app/db/platform_store.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -1,4 +1,40 @@
|
||||
"""Database session factory placeholder.
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from collections.abc import Generator
|
||||
from contextlib import contextmanager
|
||||
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session, sessionmaker
|
||||
|
||||
|
||||
DATABASE_URL = os.getenv("DATABASE_URL", "postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft")
|
||||
|
||||
engine = create_engine(
|
||||
DATABASE_URL,
|
||||
pool_pre_ping=True,
|
||||
future=True,
|
||||
)
|
||||
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, expire_on_commit=False, future=True)
|
||||
|
||||
|
||||
def get_db() -> Generator[Session, None, None]:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def session_scope() -> Generator[Session, None, None]:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
Implement SQLAlchemy/SQLModel session management here when database development starts.
|
||||
"""
|
||||
|
||||
133
backend/app/db/sql/001_platform_runtime.sql
Normal file
133
backend/app/db/sql/001_platform_runtime.sql
Normal file
@@ -0,0 +1,133 @@
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id TEXT PRIMARY KEY,
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
password_hash TEXT NOT NULL,
|
||||
display_name TEXT NOT NULL,
|
||||
role TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
permissions TEXT NOT NULL,
|
||||
create_time TEXT NOT NULL,
|
||||
last_login TEXT,
|
||||
protected INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS models (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
type TEXT NOT NULL,
|
||||
purpose TEXT NOT NULL,
|
||||
model_source TEXT NOT NULL,
|
||||
description TEXT,
|
||||
path TEXT,
|
||||
api_url TEXT,
|
||||
api_key TEXT,
|
||||
online_model_name TEXT,
|
||||
create_time TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS trained_models (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
train_methods TEXT NOT NULL,
|
||||
base_model_path TEXT,
|
||||
create_time TEXT NOT NULL,
|
||||
merged INTEGER NOT NULL DEFAULT 0,
|
||||
merging INTEGER NOT NULL DEFAULT 0,
|
||||
merged_path TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS datasets (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
type TEXT NOT NULL,
|
||||
storage_type TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
task_id TEXT,
|
||||
size TEXT,
|
||||
count INTEGER NOT NULL DEFAULT 0,
|
||||
description TEXT,
|
||||
create_time TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS dataset_files (
|
||||
id TEXT PRIMARY KEY,
|
||||
dataset_id TEXT NOT NULL REFERENCES datasets(id) ON DELETE CASCADE,
|
||||
name TEXT NOT NULL,
|
||||
size TEXT,
|
||||
content TEXT NOT NULL,
|
||||
active_version_id TEXT NOT NULL,
|
||||
versions TEXT NOT NULL,
|
||||
create_time TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS compute_nodes (
|
||||
id TEXT PRIMARY KEY,
|
||||
code TEXT NOT NULL UNIQUE,
|
||||
name TEXT NOT NULL,
|
||||
api_base_url TEXT NOT NULL,
|
||||
file_gateway_url TEXT NOT NULL,
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
scheduler_status TEXT NOT NULL,
|
||||
scheduler_weight INTEGER NOT NULL DEFAULT 100,
|
||||
tags TEXT NOT NULL,
|
||||
gpu_count INTEGER NOT NULL DEFAULT 0,
|
||||
current_running_jobs INTEGER NOT NULL DEFAULT 0,
|
||||
max_parallel_jobs INTEGER NOT NULL DEFAULT 2,
|
||||
data_root TEXT NOT NULL,
|
||||
model_root TEXT NOT NULL,
|
||||
log_root TEXT NOT NULL,
|
||||
last_health_check_at TEXT,
|
||||
health_detail TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS gpus (
|
||||
id TEXT PRIMARY KEY,
|
||||
node_id TEXT NOT NULL REFERENCES compute_nodes(id) ON DELETE CASCADE,
|
||||
gpu_index INTEGER NOT NULL,
|
||||
uuid TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
memory_total_gb DOUBLE PRECISION NOT NULL,
|
||||
power_limit_w DOUBLE PRECISION NOT NULL,
|
||||
base_temperature INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS fine_tune_tasks (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
payload TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
progress INTEGER NOT NULL DEFAULT 0,
|
||||
process_id INTEGER,
|
||||
create_time TEXT NOT NULL,
|
||||
start_time TEXT,
|
||||
completed_at TEXT,
|
||||
compute_node_id TEXT REFERENCES compute_nodes(id) ON DELETE SET NULL,
|
||||
gpus TEXT NOT NULL,
|
||||
sync_job_id TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS resource_replicas (
|
||||
id TEXT PRIMARY KEY,
|
||||
node_id TEXT NOT NULL REFERENCES compute_nodes(id) ON DELETE CASCADE,
|
||||
resource_type TEXT NOT NULL,
|
||||
resource_id TEXT NOT NULL,
|
||||
local_path TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
sync_status TEXT NOT NULL,
|
||||
create_time TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS resource_sync_jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
target_node_id TEXT NOT NULL,
|
||||
resources TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
progress INTEGER NOT NULL DEFAULT 0,
|
||||
create_time TEXT NOT NULL,
|
||||
completed_at TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_fine_tune_status ON fine_tune_tasks(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_dataset_files_dataset ON dataset_files(dataset_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_gpus_node ON gpus(node_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_replicas_resource ON resource_replicas(resource_type, resource_id);
|
||||
@@ -1,4 +1,5 @@
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from app.api.v1.router import api_router
|
||||
from app.core.config import get_settings
|
||||
@@ -10,8 +11,15 @@ def create_app() -> FastAPI:
|
||||
configure_logging(settings)
|
||||
|
||||
app = FastAPI(title=settings.app_name)
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=settings.cors_allow_origins,
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
setup_request_logging(app)
|
||||
app.include_router(api_router, prefix=settings.api_prefix)
|
||||
app.include_router(api_router, prefix=settings.route_prefix)
|
||||
return app
|
||||
|
||||
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
name = "yg-ft-backend"
|
||||
version = "0.1.0"
|
||||
description = "Backend service for the model fine-tuning platform"
|
||||
requires-python = ">=3.11"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"fastapi>=0.111.0",
|
||||
"uvicorn[standard]>=0.30.0",
|
||||
"python-multipart>=0.0.9",
|
||||
"pydantic>=2.7.0",
|
||||
"sqlalchemy>=2.0.30",
|
||||
"asyncpg>=0.29.0",
|
||||
"psycopg[binary]>=3.2.1",
|
||||
"alembic>=1.13.1",
|
||||
"redis>=5.0.4",
|
||||
"httpx>=0.27.0",
|
||||
@@ -26,7 +26,7 @@ dev = [
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py311"
|
||||
target-version = "py312"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
|
||||
@@ -3,7 +3,7 @@ uvicorn[standard]>=0.30.0
|
||||
python-multipart>=0.0.9
|
||||
pydantic>=2.7.0
|
||||
sqlalchemy>=2.0.30
|
||||
asyncpg>=0.29.0
|
||||
psycopg[binary]>=3.2.1
|
||||
alembic>=1.13.1
|
||||
redis>=5.0.4
|
||||
httpx>=0.27.0
|
||||
|
||||
@@ -14,7 +14,7 @@ compute/
|
||||
tests/
|
||||
```
|
||||
|
||||
## 第一版职责
|
||||
## 开发职责
|
||||
|
||||
- GPU 发现、状态上报、锁定和释放。
|
||||
- 本地磁盘工作区管理。
|
||||
@@ -22,3 +22,8 @@ compute/
|
||||
- LLaMA-Factory 命令生成、日志解析、产物收集。
|
||||
- 分片上传、短时下载、离线导入。
|
||||
- 通过服务间 token 接受应用平台调用。
|
||||
|
||||
## 运行模式
|
||||
|
||||
- 默认 `COMPUTE_EXECUTION_MODE=real`,Compute API 只暴露健康检查和接口契约;真实训练执行器完成前,创建作业会返回未实现错误。
|
||||
- 仅隔离联调时可设置 `COMPUTE_EXECUTION_MODE=simulator`,启用内存状态机和合成 GPU/日志数据。该模式不得作为生产运行路径。
|
||||
|
||||
1
compute/__init__.py
Normal file
1
compute/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Compute platform package."""
|
||||
211
compute/api/main.py
Normal file
211
compute/api/main.py
Normal file
@@ -0,0 +1,211 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import math
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastapi import FastAPI, HTTPException
|
||||
|
||||
from compute.engines.llama_factory.adapter import build_command, parse_log_line
|
||||
|
||||
|
||||
def create_app() -> FastAPI:
|
||||
app = FastAPI(title="YG Fine-Tune Compute API")
|
||||
jobs: dict[str, dict[str, Any]] = {}
|
||||
route_prefix = os.getenv("MODELTF_ROUTE_PREFIX", "/modelTF").rstrip("/") or "/modelTF"
|
||||
|
||||
def now() -> float:
|
||||
return time.time()
|
||||
|
||||
def host_id() -> str:
|
||||
return os.getenv("COMPUTE_HOST_ID", "gpu-node-01")
|
||||
|
||||
def execution_mode() -> str:
|
||||
return os.getenv("COMPUTE_EXECUTION_MODE", os.getenv("COMPUTE_MODE", "real")).lower()
|
||||
|
||||
def job_status(job: dict[str, Any]) -> dict[str, Any]:
|
||||
if execution_mode() != "simulator":
|
||||
return job
|
||||
elapsed = max(0, int(now() - job["created_at"]))
|
||||
if job["status"] not in {"stopped", "failed", "completed"}:
|
||||
if elapsed < 5:
|
||||
job["status"] = "queued"
|
||||
job["progress"] = 12 + elapsed * 3
|
||||
elif elapsed < 60:
|
||||
job["status"] = "running"
|
||||
job["progress"] = min(96, 25 + int((elapsed - 5) / 55 * 70))
|
||||
else:
|
||||
job["status"] = "completed"
|
||||
job["progress"] = 100
|
||||
job["logs"] = generate_logs(job)
|
||||
return job
|
||||
|
||||
def generate_logs(job: dict[str, Any]) -> str:
|
||||
progress = int(job.get("progress", 0) or 0)
|
||||
points = max(1, min(80, progress))
|
||||
lines = [
|
||||
f"[INFO] compute_host_id={host_id()} job_id={job['id']} engine=llama_factory",
|
||||
f"[INFO] command={' '.join(job['command'])}",
|
||||
]
|
||||
for step in range(1, points + 1):
|
||||
if step % 4 != 0 and step != points:
|
||||
continue
|
||||
loss = max(0.11, 2.5 * math.exp(-step / 40))
|
||||
grad_norm = 0.4 + (step % 5) * 0.04
|
||||
lr = 0.0002 * max(0.05, 1 - step / 100)
|
||||
epoch = round(step / points * 3, 4)
|
||||
lines.append(
|
||||
"{"
|
||||
f"'loss': {loss:.4f}, 'grad_norm': {grad_norm:.4f}, "
|
||||
f"'learning_rate': {lr:.8f}, 'epoch': {epoch:.4f}"
|
||||
"}"
|
||||
)
|
||||
if job.get("status") == "completed":
|
||||
lines.extend(
|
||||
[
|
||||
"***** train metrics *****",
|
||||
"epoch = 3",
|
||||
"train_loss = 0.1181",
|
||||
"train_runtime = 1m 0s",
|
||||
"***** train metrics end *****",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
def gpu_resources() -> list[dict[str, Any]]:
|
||||
if execution_mode() != "simulator":
|
||||
return []
|
||||
active_jobs = [job_status(job) for job in jobs.values() if job["status"] in {"queued", "running"}]
|
||||
gpus: list[dict[str, Any]] = []
|
||||
for idx in range(4):
|
||||
task = next((job for job in active_jobs if idx in job.get("gpus", [])), None)
|
||||
busy = task is not None and task["status"] == "running"
|
||||
reserved = task is not None and task["status"] == "queued"
|
||||
gpus.append(
|
||||
{
|
||||
"id": idx,
|
||||
"uuid": f"GPU-{host_id().upper()}-{idx}",
|
||||
"name": os.getenv("COMPUTE_GPU_NAME", "NVIDIA A800-SXM4-80GB"),
|
||||
"status": "busy" if busy else "reserved" if reserved else "idle",
|
||||
"gpu_percent": 88 if busy else 25 if reserved else 4,
|
||||
"memory_used_gb": 58 if busy else 12 if reserved else 2,
|
||||
"memory_total_gb": 80,
|
||||
"temperature": 61 if busy else 45 if reserved else 36,
|
||||
"power_w": 215 if busy else 80 if reserved else 25,
|
||||
"power_limit_w": 300,
|
||||
"processes": [
|
||||
{
|
||||
"pid": task["pid"],
|
||||
"name": "llamafactory-cli",
|
||||
"task_name": task["name"],
|
||||
"memory_used_gb": 58 if busy else 12,
|
||||
}
|
||||
]
|
||||
if task
|
||||
else [],
|
||||
}
|
||||
)
|
||||
return gpus
|
||||
|
||||
@app.get(f"{route_prefix}/health")
|
||||
async def health_check() -> dict[str, str]:
|
||||
return {
|
||||
"status": "ok",
|
||||
"compute_host_id": os.getenv("COMPUTE_HOST_ID", "unknown"),
|
||||
}
|
||||
|
||||
@app.get(f"{route_prefix}/v1/compute/health")
|
||||
async def compute_health_check() -> dict[str, str | bool]:
|
||||
data_root = Path(os.getenv("YG_FT_DATA_ROOT", "/data/yg-ft"))
|
||||
llama_factory_home = Path(os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
|
||||
return {
|
||||
"status": "ok",
|
||||
"compute_host_id": os.getenv("COMPUTE_HOST_ID", "unknown"),
|
||||
"app_callback_enabled": os.getenv("ENABLE_APP_CALLBACK", "false").lower() == "true",
|
||||
"data_root": str(data_root),
|
||||
"data_root_exists": data_root.exists(),
|
||||
"llama_factory_home": str(llama_factory_home),
|
||||
"llama_factory_home_exists": llama_factory_home.exists(),
|
||||
"execution_mode": execution_mode(),
|
||||
}
|
||||
|
||||
@app.get(f"{route_prefix}/v1/compute/jobs")
|
||||
async def list_jobs_alias() -> dict[str, list[dict[str, Any]]]:
|
||||
return {"items": [job_status(job) for job in jobs.values()]}
|
||||
|
||||
@app.get(f"{route_prefix}/compute/resources/gpus")
|
||||
async def list_gpus() -> dict[str, Any]:
|
||||
return {"items": gpu_resources(), "compute_host_id": host_id()}
|
||||
|
||||
@app.post(f"{route_prefix}/compute/jobs")
|
||||
async def create_job(payload: dict[str, Any]) -> dict[str, Any]:
|
||||
try:
|
||||
command = build_command(payload, os.getenv("LLAMA_FACTORY_HOME", "/app/LLaMA-Factory"))
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
if execution_mode() != "simulator":
|
||||
raise HTTPException(
|
||||
status_code=501,
|
||||
detail="real compute executor is not implemented yet; set COMPUTE_EXECUTION_MODE=simulator only for isolated development",
|
||||
)
|
||||
job_id = str(payload.get("id") or f"job_{int(now() * 1000)}")
|
||||
job = {
|
||||
"id": job_id,
|
||||
"name": payload.get("name", job_id),
|
||||
"status": "queued",
|
||||
"progress": 10,
|
||||
"pid": int(52000 + now() % 10000),
|
||||
"gpus": payload.get("gpus") or [0],
|
||||
"created_at": now(),
|
||||
"command": command.command,
|
||||
"work_dir": command.work_dir,
|
||||
"artifacts": [],
|
||||
"logs": "",
|
||||
}
|
||||
jobs[job_id] = job
|
||||
return job_status(job)
|
||||
|
||||
@app.get(f"{route_prefix}/compute/jobs")
|
||||
async def list_jobs() -> dict[str, Any]:
|
||||
return {"items": [job_status(job) for job in jobs.values()]}
|
||||
|
||||
@app.get(f"{route_prefix}/compute/jobs/{{job_id}}")
|
||||
async def get_job(job_id: str) -> dict[str, Any]:
|
||||
job = jobs.get(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="job not found")
|
||||
return job_status(job)
|
||||
|
||||
@app.post(f"{route_prefix}/compute/jobs/{{job_id}}/stop")
|
||||
async def stop_job(job_id: str) -> dict[str, Any]:
|
||||
job = jobs.get(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="job not found")
|
||||
job["status"] = "stopped"
|
||||
job["progress"] = min(job.get("progress", 0), 99)
|
||||
return job
|
||||
|
||||
@app.get(f"{route_prefix}/compute/jobs/{{job_id}}/logs")
|
||||
async def job_logs(job_id: str) -> dict[str, Any]:
|
||||
job = jobs.get(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="job not found")
|
||||
job = job_status(job)
|
||||
metrics = [parse_log_line(line) for line in job["logs"].splitlines()]
|
||||
return {"job_id": job_id, "content": job["logs"], "metrics": [m for m in metrics if m]}
|
||||
|
||||
@app.post(f"{route_prefix}/compute/files/upload")
|
||||
async def upload_file(payload: dict[str, Any]) -> dict[str, Any]:
|
||||
file_id = str(payload.get("id") or f"file_{int(now() * 1000)}")
|
||||
return {"id": file_id, "status": "available", "local_path": f"/data/yg-ft/uploads/{file_id}"}
|
||||
|
||||
@app.get(f"{route_prefix}/compute/files/{{file_id}}/download")
|
||||
async def download_file(file_id: str) -> dict[str, Any]:
|
||||
return {"id": file_id, "status": "ready", "download_url": f"{route_prefix}/compute/files/{file_id}/download"}
|
||||
|
||||
return app
|
||||
|
||||
|
||||
app = create_app()
|
||||
80
compute/engines/llama_factory/adapter.py
Normal file
80
compute/engines/llama_factory/adapter.py
Normal file
@@ -0,0 +1,80 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LlamaFactoryCommand:
|
||||
command: list[str]
|
||||
work_dir: str
|
||||
env: dict[str, str]
|
||||
|
||||
|
||||
def validate_config(config: dict[str, Any]) -> list[str]:
|
||||
errors: list[str] = []
|
||||
if not config.get("base_model") and not config.get("model_name_or_path"):
|
||||
errors.append("base_model or model_name_or_path is required")
|
||||
if not config.get("dataset") and not config.get("dataset_dir"):
|
||||
errors.append("dataset or dataset_dir is required")
|
||||
learning_rate = float(config.get("learning_rate", 0.0002))
|
||||
if learning_rate <= 0:
|
||||
errors.append("learning_rate must be greater than zero")
|
||||
epochs = int(config.get("n_epochs", config.get("num_train_epochs", 1)))
|
||||
if epochs <= 0:
|
||||
errors.append("n_epochs must be greater than zero")
|
||||
return errors
|
||||
|
||||
|
||||
def build_command(config: dict[str, Any], llama_factory_home: str = "/app/LLaMA-Factory") -> LlamaFactoryCommand:
|
||||
errors = validate_config(config)
|
||||
if errors:
|
||||
raise ValueError("; ".join(errors))
|
||||
|
||||
model_path = config.get("base_model") or config.get("model_name_or_path")
|
||||
dataset = config.get("dataset") or config.get("dataset_dir")
|
||||
output_dir = config.get("output_dir") or f"/data/yg-ft/outputs/{config.get('name', 'training-job')}"
|
||||
command = [
|
||||
"llamafactory-cli",
|
||||
"train",
|
||||
"--stage",
|
||||
str(config.get("stage", "sft")).lower(),
|
||||
"--do_train",
|
||||
"true",
|
||||
"--model_name_or_path",
|
||||
str(model_path),
|
||||
"--dataset",
|
||||
str(dataset),
|
||||
"--template",
|
||||
str(config.get("template", "qwen")),
|
||||
"--finetuning_type",
|
||||
str(config.get("train_method", config.get("finetuning_type", "lora"))),
|
||||
"--output_dir",
|
||||
str(output_dir),
|
||||
"--per_device_train_batch_size",
|
||||
str(config.get("batch_size", 2)),
|
||||
"--learning_rate",
|
||||
str(config.get("learning_rate", 0.0002)),
|
||||
"--num_train_epochs",
|
||||
str(config.get("n_epochs", 3)),
|
||||
"--save_steps",
|
||||
str(config.get("save_steps", 50)),
|
||||
]
|
||||
quantization_bit = int(config.get("quantization_bit", 0) or 0)
|
||||
if quantization_bit in {4, 8}:
|
||||
command.extend(["--quantization_bit", str(quantization_bit)])
|
||||
return LlamaFactoryCommand(command=command, work_dir=str(Path(llama_factory_home)), env={})
|
||||
|
||||
|
||||
def parse_log_line(line: str) -> dict[str, float] | None:
|
||||
if "loss" not in line or "learning_rate" not in line:
|
||||
return None
|
||||
result: dict[str, float] = {}
|
||||
for key in ["loss", "grad_norm", "learning_rate", "epoch"]:
|
||||
match = re.search(rf"['\"]?{key}['\"]?\s*:\s*([-+]?\d+(?:\.\d+)?(?:[eE][-+]?\d+)?)", line)
|
||||
if match:
|
||||
result[key] = float(match.group(1))
|
||||
return result or None
|
||||
|
||||
5
compute/requirements.txt
Normal file
5
compute/requirements.txt
Normal file
@@ -0,0 +1,5 @@
|
||||
fastapi>=0.111.0
|
||||
uvicorn[standard]>=0.30.0
|
||||
pydantic>=2.7.0
|
||||
python-dotenv>=1.0.1
|
||||
httpx>=0.27.0
|
||||
@@ -1,15 +0,0 @@
|
||||
services:
|
||||
yg-ft-frontend:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
image: yg-ft-frontend:latest
|
||||
container_name: yg-ft-frontend
|
||||
ports:
|
||||
- "6801:80"
|
||||
environment:
|
||||
# Change this if the backend is deployed somewhere else.
|
||||
API_PROXY_PASS: "http://host.docker.internal:7861"
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
restart: unless-stopped
|
||||
267
docker/README.md
Normal file
267
docker/README.md
Normal file
@@ -0,0 +1,267 @@
|
||||
# Docker 部署说明
|
||||
|
||||
本目录按应用服务器和算力服务器拆分 Dockerfile 与 Docker Compose 文件。Compose 文件不包含 `build:`,不会在 `docker compose up` 时自动构建业务镜像。所有业务镜像需要先通过手动 `docker build` 构建,再由 Compose 启动。
|
||||
|
||||
## 基础镜像
|
||||
|
||||
| 镜像 | 用途 |
|
||||
| --- | --- |
|
||||
| `python:3.12-slim` | 应用后端基础镜像,后端运行环境要求 Python 3.12 及以上 |
|
||||
| `nginx:1.27-alpine` | 前端静态资源与 `/modelTF` 反向代理运行镜像 |
|
||||
| `hiyouga/llamafactory:latest` | 算力服务基础镜像,基于 LLaMA-Factory 官方镜像扩展 Compute API |
|
||||
| `postgres:16-alpine` | 开发阶段内置 PostgreSQL |
|
||||
| `redis:7-alpine` | 开发阶段内置 Redis |
|
||||
|
||||
一键拉取基础镜像:
|
||||
|
||||
```bash
|
||||
docker pull python:3.12-slim && \
|
||||
docker pull nginx:1.27-alpine && \
|
||||
docker pull hiyouga/llamafactory:latest && \
|
||||
docker pull postgres:16-alpine && \
|
||||
docker pull redis:7-alpine
|
||||
```
|
||||
|
||||
Windows PowerShell:
|
||||
|
||||
```powershell
|
||||
$images = @(
|
||||
"python:3.12-slim",
|
||||
"nginx:1.27-alpine",
|
||||
"hiyouga/llamafactory:latest",
|
||||
"postgres:16-alpine",
|
||||
"redis:7-alpine"
|
||||
)
|
||||
$images | ForEach-Object { docker pull $_ }
|
||||
```
|
||||
|
||||
如果部署环境不能访问外网,需要提前在可联网环境执行上述拉取命令,再用 `docker save` / `docker load` 导出导入。
|
||||
|
||||
## 业务镜像
|
||||
|
||||
| 镜像 | Dockerfile | 构建命令 |
|
||||
| --- | --- | --- |
|
||||
| `yg-ft-backend-api:latest` | `docker/app/Dockerfile.backend` | `docker build -f docker/app/Dockerfile.backend -t yg-ft-backend-api:latest .` |
|
||||
| `yg-ft-frontend-runtime:latest` | `docker/app/Dockerfile.frontend` | `docker build -f docker/app/Dockerfile.frontend -t yg-ft-frontend-runtime:latest .` |
|
||||
| `yg-ft-compute-api:latest` | `docker/compute/Dockerfile.compute` | `docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .` |
|
||||
|
||||
## 对外端口
|
||||
|
||||
所有宿主机对外端口统一使用 5 位端口。容器内部端口保持镜像默认端口,便于容器内服务和健康检查稳定。
|
||||
|
||||
| 服务 | 宿主机对外端口 | 容器内部端口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 前端 Nginx | `16801` | `80` | 前端页面入口 |
|
||||
| 后端 API | `17861` | `8000` | FastAPI 服务 |
|
||||
| PostgreSQL | `15432` | `5432` | 开发阶段内置数据库 |
|
||||
| Redis | `16379` | `6379` | 开发阶段内置缓存 |
|
||||
| Compute API | `19100` | `9100` | 算力服务器 API |
|
||||
| File Gateway | `19101` | 后续服务端口 | 当前预留,后续拆出文件网关服务时使用 |
|
||||
|
||||
注意:`8000` 是后端容器内部端口,不作为宿主机对外访问端口。宿主机或浏览器应访问 `http://<app-server-ip>:17861/modelTF/health`;前端 Nginx 容器在 Docker 网络内部访问 `http://backend-api:8000/modelTF/...`。
|
||||
|
||||
对应配置文件:
|
||||
|
||||
- `docker/app/.env.example`
|
||||
- `FRONTEND_PORT=16801`
|
||||
- `BACKEND_API_PORT=17861`
|
||||
- `POSTGRES_PORT=15432`
|
||||
- `REDIS_PORT=16379`
|
||||
- `docker/compute/.env.example`
|
||||
- `COMPUTE_API_PORT=19100`
|
||||
- `FILE_GATEWAY_PORT=19101`
|
||||
|
||||
## 运行模式
|
||||
|
||||
- 应用侧默认 `COMPUTE_MODE=real`,任务状态必须由真实算力同步逻辑更新。
|
||||
- 算力侧默认 `COMPUTE_EXECUTION_MODE=real`,真实执行器未完成前不会伪造训练作业。
|
||||
- 仅隔离联调时可显式设置 `COMPUTE_MODE=simulator` 或 `COMPUTE_EXECUTION_MODE=simulator`,该模式不得用于测试环境、生产环境或生产升级基线。
|
||||
|
||||
## 应用服务器部署
|
||||
|
||||
应用服务器包含前端 Nginx、Backend API、PostgreSQL、Redis。
|
||||
|
||||
当前 Compose 内置 PostgreSQL 使用 `backend/app/db/sql/001_platform_runtime.sql` 初始化运行库。`docs/postgres-schema.sql` 是完整目标架构设计,不应直接挂载为当前运行库初始化脚本,否则会与当前后端代码的运行表结构不兼容。
|
||||
|
||||
首次部署:
|
||||
|
||||
```bash
|
||||
cd <repo-root>
|
||||
|
||||
# 1. 使用当前 Windows/宿主机 npm 构建前端静态产物
|
||||
cd frontend
|
||||
npm ci
|
||||
npm run build
|
||||
cd ..
|
||||
|
||||
# 2. 手动构建业务镜像
|
||||
docker build -f docker/app/Dockerfile.backend -t yg-ft-backend-api:latest .
|
||||
docker build -f docker/app/Dockerfile.frontend -t yg-ft-frontend-runtime:latest .
|
||||
|
||||
# 3. 启动应用服务
|
||||
cd docker/app
|
||||
cp .env.example .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
后端镜像构建过程中会执行依赖导入自检,确认 `fastapi`、`uvicorn`、`psycopg`、`sqlalchemy`、`redis` 等运行依赖已安装。构建后也可以手动检查:
|
||||
|
||||
```bash
|
||||
docker run --rm yg-ft-backend-api:latest python -c "import psycopg; print(psycopg.__version__)"
|
||||
```
|
||||
|
||||
默认访问地址:
|
||||
|
||||
```text
|
||||
http://<app-server-ip>:16801
|
||||
```
|
||||
|
||||
应用侧代码和数据外挂:
|
||||
|
||||
```text
|
||||
../../backend -> /app
|
||||
../../frontend/dist -> /usr/share/nginx/html
|
||||
../../runtime/app/logs/backend -> /opt/yg-ft/logs/backend
|
||||
../../runtime/app/data -> /data/yg-ft
|
||||
```
|
||||
|
||||
前端容器启动前必须确保 `../../frontend/dist/index.html` 已存在。若前端 Nginx 日志出现 `directory index of "/usr/share/nginx/html/" is forbidden` 或 `rewrite or internal redirection cycle while internally redirecting to "/index.html"`,通常表示当前执行 `docker compose` 的项目目录下没有构建好的 `frontend/dist`,或挂载路径不是同一份代码目录。
|
||||
|
||||
```bash
|
||||
# 在执行 docker compose 的同一份代码目录中检查
|
||||
cd <repo-root>/frontend
|
||||
npm run build
|
||||
test -f dist/index.html && ls -lh dist/index.html
|
||||
|
||||
cd ../docker/app
|
||||
docker compose up -d --force-recreate frontend
|
||||
docker compose logs --tail=80 frontend
|
||||
```
|
||||
|
||||
如果使用 Windows npm 构建、WSL 中运行 Docker Compose,需要确认 Windows 路径和 WSL 路径指向同一份仓库。例如在 `D:\...\YG_FT\frontend` 构建不会自动生成 `/mnt/d/wuyongtao/Code/YG_FT/frontend/dist` 下的产物,除非二者本就是同一个目录。
|
||||
|
||||
如果使用企业统一 PostgreSQL/Redis,修改 `docker/app/.env`:
|
||||
|
||||
```env
|
||||
DATABASE_URL=postgresql+psycopg://<user>:<password>@<postgres-host>:15432/<db>
|
||||
REDIS_URL=redis://<redis-host>:16379/0
|
||||
USE_BUILTIN_POSTGRES=false
|
||||
USE_BUILTIN_REDIS=false
|
||||
```
|
||||
|
||||
生产环境如完全使用外部基础设施,可以删除或注释 Compose 中的 `postgres`、`redis` 服务及 `backend-api.depends_on` 中对应依赖。
|
||||
|
||||
## 算力服务器部署
|
||||
|
||||
算力服务器包含 Compute API、后续 Compute Agent、File Gateway、GPU runtime、本地训练数据目录和 LLaMA-Factory。`Dockerfile.compute` 基于 LLaMA-Factory 官方镜像:
|
||||
|
||||
```dockerfile
|
||||
FROM hiyouga/llamafactory:latest
|
||||
```
|
||||
|
||||
部署前需要安装:
|
||||
|
||||
- NVIDIA Driver
|
||||
- NVIDIA Container Toolkit
|
||||
- Docker Engine 和 Docker Compose Plugin
|
||||
- 本地训练数据目录,默认 `/data/yg-ft`
|
||||
|
||||
首次部署:
|
||||
|
||||
```bash
|
||||
cd <repo-root>
|
||||
|
||||
# 手动构建算力业务镜像
|
||||
docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .
|
||||
|
||||
# 启动算力服务
|
||||
cd docker/compute
|
||||
cp .env.example .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
健康检查:
|
||||
|
||||
```text
|
||||
GET http://<compute-server-ip>:19100/modelTF/health
|
||||
GET http://<compute-server-ip>:19100/modelTF/v1/compute/health
|
||||
```
|
||||
|
||||
算力侧代码和数据外挂:
|
||||
|
||||
```text
|
||||
../../compute -> /app/compute
|
||||
${YG_FT_DATA_ROOT_HOST} -> /data/yg-ft
|
||||
../../runtime/compute/logs -> /opt/yg-ft/logs/compute
|
||||
../../runtime/compute/training-logs -> /opt/yg-ft/logs/training
|
||||
```
|
||||
|
||||
## 应用与算力分离部署
|
||||
|
||||
应用服务器只需要主动访问算力服务器,不要求算力服务器回调应用服务器。
|
||||
|
||||
在 `docker/app/.env` 中配置:
|
||||
|
||||
```env
|
||||
COMPUTE_API_BASE_URL=http://<compute-server-ip>:19100
|
||||
FILE_GATEWAY_BASE_URL=http://<compute-server-ip>:19101
|
||||
COMPUTE_SERVICE_TOKEN=change_me
|
||||
COMPUTE_STATUS_SYNC_MODE=polling
|
||||
COMPUTE_POLL_INTERVAL_SECONDS=10
|
||||
COMPUTE_POLL_BATCH_SIZE=100
|
||||
```
|
||||
|
||||
交互链路:
|
||||
|
||||
```text
|
||||
Frontend
|
||||
-> Backend API
|
||||
-> Compute API
|
||||
-> Compute Agent / LLaMA-Factory
|
||||
-> 本地数据目录 / 模型目录 / 训练产物
|
||||
<- Backend Worker 定时轮询 Compute API
|
||||
```
|
||||
|
||||
## 多算力节点部署
|
||||
|
||||
多算力节点仍按“单机多 GPU 节点”部署。每台 GPU 服务器都独立部署一套 `docker/compute`:
|
||||
|
||||
```text
|
||||
gpu-node-01: docker/compute + /data/yg-ft + 19100/19101
|
||||
gpu-node-02: docker/compute + /data/yg-ft + 19100/19101
|
||||
gpu-node-03: docker/compute + /data/yg-ft + 19100/19101
|
||||
```
|
||||
|
||||
节点之间默认不互访。应用平台主动访问每个节点的 Compute API/File Gateway,并通过 `compute_nodes`、`resource_replicas`、`resource_sync_jobs` 统一调度和同步。
|
||||
|
||||
## 常用命令
|
||||
|
||||
重新构建应用镜像:
|
||||
|
||||
```bash
|
||||
docker build -f docker/app/Dockerfile.backend -t yg-ft-backend-api:latest .
|
||||
docker build -f docker/app/Dockerfile.frontend -t yg-ft-frontend-runtime:latest .
|
||||
```
|
||||
|
||||
重新构建算力镜像:
|
||||
|
||||
```bash
|
||||
docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .
|
||||
```
|
||||
|
||||
启动服务:
|
||||
|
||||
```bash
|
||||
cd docker/app
|
||||
docker compose up -d
|
||||
|
||||
cd ../compute
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
查看服务:
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
docker compose logs -f
|
||||
```
|
||||
45
docker/app/.env.example
Normal file
45
docker/app/.env.example
Normal file
@@ -0,0 +1,45 @@
|
||||
APP_ENV=prod
|
||||
APP_NAME=YG Fine-Tune Platform API
|
||||
MODELTF_ROUTE_PREFIX=/modelTF
|
||||
CORS_ALLOW_ORIGINS=http://localhost:16801,http://127.0.0.1:16801
|
||||
|
||||
FRONTEND_IMAGE=yg-ft-frontend-runtime:latest
|
||||
BACKEND_API_IMAGE=yg-ft-backend-api:latest
|
||||
|
||||
# Five-digit host ports exposed outside the application server.
|
||||
FRONTEND_PORT=16801
|
||||
BACKEND_API_PORT=17861
|
||||
POSTGRES_PORT=15432
|
||||
REDIS_PORT=16379
|
||||
|
||||
POSTGRES_DB=yg_ft
|
||||
POSTGRES_USER=yg_ft
|
||||
POSTGRES_PASSWORD=change_me
|
||||
DATABASE_URL=postgresql+psycopg://yg_ft:change_me@postgres:5432/yg_ft
|
||||
|
||||
REDIS_URL=redis://redis:6379/0
|
||||
|
||||
# Development uses the built-in PostgreSQL/Redis services in docker-compose.yml.
|
||||
# For enterprise infrastructure, replace DATABASE_URL/REDIS_URL and remove or disable those services.
|
||||
USE_BUILTIN_POSTGRES=true
|
||||
USE_BUILTIN_REDIS=true
|
||||
|
||||
LOG_LEVEL=INFO
|
||||
LOG_DIR=/opt/yg-ft/logs/backend
|
||||
LOG_FILE_PREFIX=backend
|
||||
LOG_ERROR_FILE_PREFIX=error
|
||||
LOG_MAX_BYTES=20971520
|
||||
LOG_RETENTION_DAYS=10
|
||||
|
||||
BACKEND_PROXY_PASS=http://backend-api:8000
|
||||
|
||||
# Split deployment: set these to the compute server address, for example http://10.10.20.31:19100.
|
||||
COMPUTE_API_BASE_URL=http://compute-api:9100
|
||||
COMPUTE_SERVICE_TOKEN=change_me
|
||||
FILE_GATEWAY_BASE_URL=http://compute-api:9101
|
||||
|
||||
# The application side polls Compute API for job state to avoid opening reverse network access.
|
||||
COMPUTE_MODE=real
|
||||
COMPUTE_STATUS_SYNC_MODE=polling
|
||||
COMPUTE_POLL_INTERVAL_SECONDS=10
|
||||
COMPUTE_POLL_BATCH_SIZE=100
|
||||
21
docker/app/Dockerfile.backend
Normal file
21
docker/app/Dockerfile.backend
Normal file
@@ -0,0 +1,21 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PIP_NO_CACHE_DIR=1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY backend/requirements.txt /tmp/requirements.txt
|
||||
RUN pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple \
|
||||
&& pip install -r /tmp/requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple \
|
||||
&& rm -f /tmp/requirements.txt
|
||||
|
||||
RUN python -c "import fastapi, uvicorn, psycopg, sqlalchemy, redis, jwt, passlib, httpx, alembic; print('backend dependency check ok')"
|
||||
|
||||
RUN mkdir -p /opt/yg-ft/logs/backend /data/yg-ft \
|
||||
&& chmod -R 0775 /opt/yg-ft /data/yg-ft
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
9
docker/app/Dockerfile.frontend
Normal file
9
docker/app/Dockerfile.frontend
Normal file
@@ -0,0 +1,9 @@
|
||||
|
||||
FROM nginx:1.27-alpine
|
||||
|
||||
RUN mkdir -p /usr/share/nginx/html
|
||||
|
||||
EXPOSE 80
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||
CMD test -f /usr/share/nginx/html/index.html && wget -qO- http://127.0.0.1/index.html >/dev/null || exit 1
|
||||
123
docker/app/docker-compose.yml
Normal file
123
docker/app/docker-compose.yml
Normal file
@@ -0,0 +1,123 @@
|
||||
services:
|
||||
frontend:
|
||||
image: ${FRONTEND_IMAGE:-yg-ft-frontend-runtime:latest}
|
||||
container_name: yg-ft-frontend
|
||||
depends_on:
|
||||
backend-api:
|
||||
condition: service_started
|
||||
ports:
|
||||
- "${FRONTEND_PORT:-16801}:80"
|
||||
environment:
|
||||
BACKEND_PROXY_PASS: ${BACKEND_PROXY_PASS:-http://backend-api:8000}
|
||||
volumes:
|
||||
- ../../frontend/dist:/usr/share/nginx/html:ro
|
||||
- ../nginx.conf.template:/etc/nginx/templates/default.conf.template:ro
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
if [ ! -f /usr/share/nginx/html/index.html ]; then
|
||||
echo "frontend dist is missing: build frontend first and ensure ../../frontend/dist is mounted";
|
||||
ls -la /usr/share/nginx/html;
|
||||
exit 1;
|
||||
fi;
|
||||
nginx -g 'daemon off;'
|
||||
networks:
|
||||
- yg-ft-app
|
||||
restart: unless-stopped
|
||||
|
||||
backend-api:
|
||||
image: ${BACKEND_API_IMAGE:-yg-ft-backend-api:latest}
|
||||
container_name: yg-ft-backend-api
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
expose:
|
||||
- "8000"
|
||||
ports:
|
||||
- "${BACKEND_API_PORT:-17861}:8000"
|
||||
environment:
|
||||
APP_ENV: ${APP_ENV:-prod}
|
||||
APP_NAME: ${APP_NAME:-YG Fine-Tune Platform API}
|
||||
MODELTF_ROUTE_PREFIX: ${MODELTF_ROUTE_PREFIX:-/modelTF}
|
||||
CORS_ALLOW_ORIGINS: ${CORS_ALLOW_ORIGINS:-http://localhost:16801,http://127.0.0.1:16801}
|
||||
DATABASE_URL: ${DATABASE_URL:-postgresql+psycopg://yg_ft:change_me@postgres:5432/yg_ft}
|
||||
REDIS_URL: ${REDIS_URL:-redis://redis:6379/0}
|
||||
USE_BUILTIN_POSTGRES: ${USE_BUILTIN_POSTGRES:-true}
|
||||
USE_BUILTIN_REDIS: ${USE_BUILTIN_REDIS:-true}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||
LOG_DIR: ${LOG_DIR:-/opt/yg-ft/logs/backend}
|
||||
LOG_FILE_PREFIX: ${LOG_FILE_PREFIX:-backend}
|
||||
LOG_ERROR_FILE_PREFIX: ${LOG_ERROR_FILE_PREFIX:-error}
|
||||
LOG_MAX_BYTES: ${LOG_MAX_BYTES:-20971520}
|
||||
LOG_RETENTION_DAYS: ${LOG_RETENTION_DAYS:-10}
|
||||
COMPUTE_API_BASE_URL: ${COMPUTE_API_BASE_URL:-http://compute-api:9100}
|
||||
COMPUTE_SERVICE_TOKEN: ${COMPUTE_SERVICE_TOKEN:-change_me}
|
||||
FILE_GATEWAY_BASE_URL: ${FILE_GATEWAY_BASE_URL:-http://compute-api:9101}
|
||||
COMPUTE_MODE: ${COMPUTE_MODE:-real}
|
||||
COMPUTE_STATUS_SYNC_MODE: ${COMPUTE_STATUS_SYNC_MODE:-polling}
|
||||
COMPUTE_POLL_INTERVAL_SECONDS: ${COMPUTE_POLL_INTERVAL_SECONDS:-10}
|
||||
COMPUTE_POLL_BATCH_SIZE: ${COMPUTE_POLL_BATCH_SIZE:-100}
|
||||
PYTHONPATH: /app
|
||||
volumes:
|
||||
- ../../backend:/app:ro
|
||||
- ../../runtime/app/logs/backend:/opt/yg-ft/logs/backend
|
||||
- ../../runtime/app/data:/data/yg-ft
|
||||
networks:
|
||||
- yg-ft-app
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/modelTF/health', timeout=3).read()\""]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
restart: unless-stopped
|
||||
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
container_name: yg-ft-postgres
|
||||
environment:
|
||||
POSTGRES_DB: ${POSTGRES_DB:-yg_ft}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-yg_ft}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-change_me}
|
||||
PGDATA: /var/lib/postgresql/data/pgdata
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
- ../../backend/app/db/sql/001_platform_runtime.sql:/docker-entrypoint-initdb.d/001-platform-runtime.sql:ro
|
||||
ports:
|
||||
- "${POSTGRES_PORT:-15432}:5432"
|
||||
networks:
|
||||
- yg-ft-app
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
restart: unless-stopped
|
||||
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
container_name: yg-ft-redis
|
||||
command: ["redis-server", "--appendonly", "yes"]
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
ports:
|
||||
- "${REDIS_PORT:-16379}:6379"
|
||||
networks:
|
||||
- yg-ft-app
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
yg-ft-app:
|
||||
name: yg-ft-app
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
redis_data:
|
||||
23
docker/compute/.env.example
Normal file
23
docker/compute/.env.example
Normal file
@@ -0,0 +1,23 @@
|
||||
COMPUTE_ENV=prod
|
||||
COMPUTE_HOST_ID=gpu-node-01
|
||||
COMPUTE_EXECUTION_MODE=real
|
||||
MODELTF_ROUTE_PREFIX=/modelTF
|
||||
# Five-digit host ports exposed outside the compute server.
|
||||
COMPUTE_API_PORT=19100
|
||||
FILE_GATEWAY_PORT=19101
|
||||
COMPUTE_API_IMAGE=yg-ft-compute-api:latest
|
||||
|
||||
# The application server actively polls Compute API; compute server does not need reverse access.
|
||||
COMPUTE_SERVICE_TOKEN=change_me
|
||||
ENABLE_APP_CALLBACK=false
|
||||
|
||||
# LLaMA-Factory is provided by the official hiyouga/llamafactory base image.
|
||||
LLAMA_FACTORY_HOME=/app/LLaMA-Factory
|
||||
|
||||
YG_FT_DATA_ROOT=/data/yg-ft
|
||||
YG_FT_DATA_ROOT_HOST=/data/yg-ft
|
||||
|
||||
LOG_DIR=/opt/yg-ft/logs/compute
|
||||
CUDA_VISIBLE_DEVICES=all
|
||||
NVIDIA_VISIBLE_DEVICES=all
|
||||
NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
||||
27
docker/compute/Dockerfile.compute
Normal file
27
docker/compute/Dockerfile.compute
Normal file
@@ -0,0 +1,27 @@
|
||||
|
||||
FROM hiyouga/llamafactory:latest
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PIP_NO_CACHE_DIR=1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends tini \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY compute/requirements.txt /tmp/requirements.txt
|
||||
RUN pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple \
|
||||
&& pip install -r /tmp/requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple \
|
||||
&& rm -f /tmp/requirements.txt
|
||||
|
||||
RUN mkdir -p /opt/yg-ft/logs/compute /opt/yg-ft/logs/training /data/yg-ft /app/LLaMA-Factory \
|
||||
&& chmod -R 0775 /opt/yg-ft /data/yg-ft /app/LLaMA-Factory
|
||||
|
||||
ENTRYPOINT ["/usr/bin/tini", "--"]
|
||||
|
||||
EXPOSE 9100
|
||||
|
||||
CMD ["uvicorn", "compute.api.main:app", "--host", "0.0.0.0", "--port", "9100"]
|
||||
39
docker/compute/docker-compose.yml
Normal file
39
docker/compute/docker-compose.yml
Normal file
@@ -0,0 +1,39 @@
|
||||
services:
|
||||
compute-api:
|
||||
image: ${COMPUTE_API_IMAGE:-yg-ft-compute-api:latest}
|
||||
container_name: yg-ft-compute-api
|
||||
gpus: all
|
||||
ports:
|
||||
- "${COMPUTE_API_PORT:-19100}:9100"
|
||||
environment:
|
||||
COMPUTE_ENV: ${COMPUTE_ENV:-prod}
|
||||
COMPUTE_HOST_ID: ${COMPUTE_HOST_ID:-gpu-node-01}
|
||||
COMPUTE_EXECUTION_MODE: ${COMPUTE_EXECUTION_MODE:-real}
|
||||
MODELTF_ROUTE_PREFIX: ${MODELTF_ROUTE_PREFIX:-/modelTF}
|
||||
COMPUTE_SERVICE_TOKEN: ${COMPUTE_SERVICE_TOKEN:-change_me}
|
||||
ENABLE_APP_CALLBACK: ${ENABLE_APP_CALLBACK:-false}
|
||||
LLAMA_FACTORY_HOME: ${LLAMA_FACTORY_HOME:-/app/LLaMA-Factory}
|
||||
YG_FT_DATA_ROOT: ${YG_FT_DATA_ROOT:-/data/yg-ft}
|
||||
LOG_DIR: ${LOG_DIR:-/opt/yg-ft/logs/compute}
|
||||
CUDA_VISIBLE_DEVICES: ${CUDA_VISIBLE_DEVICES:-all}
|
||||
NVIDIA_VISIBLE_DEVICES: ${NVIDIA_VISIBLE_DEVICES:-all}
|
||||
NVIDIA_DRIVER_CAPABILITIES: ${NVIDIA_DRIVER_CAPABILITIES:-compute,utility}
|
||||
PYTHONPATH: /app
|
||||
volumes:
|
||||
- ../../compute:/app/compute:ro
|
||||
- ${YG_FT_DATA_ROOT_HOST:-/data/yg-ft}:${YG_FT_DATA_ROOT:-/data/yg-ft}
|
||||
- ../../runtime/compute/logs:/opt/yg-ft/logs/compute
|
||||
- ../../runtime/compute/training-logs:/opt/yg-ft/logs/training
|
||||
networks:
|
||||
- yg-ft-compute
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:9100/modelTF/health', timeout=3).read()\""]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
yg-ft-compute:
|
||||
name: yg-ft-compute
|
||||
@@ -11,8 +11,8 @@ server {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
||||
location /api {
|
||||
proxy_pass ${API_PROXY_PASS};
|
||||
location /modelTF {
|
||||
proxy_pass ${BACKEND_PROXY_PASS};
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# 模型微调平台后端接口设计
|
||||
|
||||
> 后端建议使用 FastAPI,统一挂载 `/api` 前缀。本文根据当前 Vue 前端路由、API 模块、mock 数据和页面交互反推接口,并补充完整微调平台必须具备的用户中心、权限控制、审计、异步任务、文件版本与监控能力。
|
||||
> 后端建议使用 FastAPI,统一挂载 `/modelTF` 前缀。本文根据当前 Vue 前端路由、API 模块和页面交互契约梳理接口,并补充完整微调平台必须具备的用户中心、权限控制、审计、异步任务、文件版本与监控能力。前端 Mock 仅作为隔离开发辅助,不作为接口设计准则。
|
||||
|
||||
菜单、二级路由、规划菜单、接口和数据库的总览映射见 `docs/menu-functional-requirements.md`。后续新增接口时,必须同步标注对应页面/功能模块。
|
||||
|
||||
## 1. 通用约定
|
||||
|
||||
@@ -58,7 +60,7 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
### 2.1 登录
|
||||
|
||||
`POST /api/login`
|
||||
`POST /modelTF/login`
|
||||
|
||||
请求:
|
||||
|
||||
@@ -88,11 +90,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
}
|
||||
```
|
||||
|
||||
说明:前端当前登录接口已经要求返回 `user`,mock 里只返回 token,正式后端必须返回完整用户信息。
|
||||
说明:前端当前登录接口已经要求返回 `user`,正式后端必须返回完整用户信息。
|
||||
|
||||
### 2.2 当前用户
|
||||
|
||||
`GET /api/me`
|
||||
`GET /modelTF/me`
|
||||
|
||||
用于刷新页面后恢复用户信息和权限,避免完全依赖 localStorage。
|
||||
|
||||
@@ -100,11 +102,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 | 权限 |
|
||||
| --- | --- | --- | --- |
|
||||
| GET | `/api/users` | 用户列表 | `user-settings` |
|
||||
| POST | `/api/users` | 创建用户 | `user-settings` |
|
||||
| PUT | `/api/users/{id}` | 更新角色、状态、页面权限 | `user-settings` |
|
||||
| DELETE | `/api/users/{id}` | 删除用户 | `user-settings` |
|
||||
| PUT | `/api/users/{id}/password` | 重置密码 | `user-settings` |
|
||||
| GET | `/modelTF/users` | 用户列表 | `user-settings` |
|
||||
| POST | `/modelTF/users` | 创建用户 | `user-settings` |
|
||||
| PUT | `/modelTF/users/{id}` | 更新角色、状态、页面权限 | `user-settings` |
|
||||
| DELETE | `/modelTF/users/{id}` | 删除用户 | `user-settings` |
|
||||
| PUT | `/modelTF/users/{id}/password` | 重置密码 | `user-settings` |
|
||||
|
||||
创建用户请求:
|
||||
|
||||
@@ -133,7 +135,7 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
### 3.1 首页看板
|
||||
|
||||
`GET /api/dashboard/overview?period=7d`
|
||||
`GET /modelTF/dashboard/overview?period=7d`
|
||||
|
||||
返回:
|
||||
|
||||
@@ -158,11 +160,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
}
|
||||
```
|
||||
|
||||
说明:首页当前完全使用前端 mock,后端应提供聚合接口,避免前端拼多接口导致加载慢。
|
||||
说明:首页需要由后端提供正式聚合接口,避免前端拼多接口导致加载慢。
|
||||
|
||||
### 3.2 健康指标
|
||||
|
||||
`GET /api/health`
|
||||
`GET /modelTF/health`
|
||||
|
||||
响应字段兼容前端 `HealthMetrics`:
|
||||
|
||||
@@ -176,7 +178,7 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
### 3.3 平台性能
|
||||
|
||||
`GET /api/system-info`
|
||||
`GET /modelTF/system-info`
|
||||
|
||||
返回 CPU、内存、磁盘、GPU、网络、系统运行时间。GPU 进程字段建议包含 `pid`、`name`、`memory_used_gb`、`task_name`、`user`。
|
||||
|
||||
@@ -184,11 +186,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/log-files?date=2026-07-16` | 系统日志文件列表 |
|
||||
| GET | `/api/log-content?file=system.log` | 系统日志内容 |
|
||||
| GET | `/api/training-log-files` | 训练日志文件列表 |
|
||||
| GET | `/api/training-log-content?file=xxx.log` | 训练日志内容 |
|
||||
| POST | `/api/web-log` | 前端错误/行为日志 |
|
||||
| GET | `/modelTF/log-files?date=2026-07-16` | 系统日志文件列表 |
|
||||
| GET | `/modelTF/log-content?file=system.log` | 系统日志内容 |
|
||||
| GET | `/modelTF/training-log-files` | 训练日志文件列表 |
|
||||
| GET | `/modelTF/training-log-content?file=xxx.log` | 训练日志内容 |
|
||||
| POST | `/modelTF/web-log` | 前端错误/行为日志 |
|
||||
|
||||
日志内容应支持 `tail`、`offset`、`limit` 参数,避免一次返回超大文件。
|
||||
|
||||
@@ -198,14 +200,14 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/model-manage` | 模型列表 |
|
||||
| GET | `/api/model-manage/{id}` | 模型详情 |
|
||||
| GET | `/api/model-manage/name/{name}` | 按名称查询 |
|
||||
| POST | `/api/model-manage` | 创建模型 |
|
||||
| PUT | `/api/model-manage/{id}` | 编辑模型 |
|
||||
| DELETE | `/api/model-manage/{id}` | 删除模型 |
|
||||
| PUT | `/api/model-manage/{id}/purpose` | 修改用途 |
|
||||
| GET | `/api/model-manage/local-models` | 扫描本地模型目录 |
|
||||
| GET | `/modelTF/model-manage` | 模型列表 |
|
||||
| GET | `/modelTF/model-manage/{id}` | 模型详情 |
|
||||
| GET | `/modelTF/model-manage/name/{name}` | 按名称查询 |
|
||||
| POST | `/modelTF/model-manage` | 创建模型 |
|
||||
| PUT | `/modelTF/model-manage/{id}` | 编辑模型 |
|
||||
| DELETE | `/modelTF/model-manage/{id}` | 删除模型 |
|
||||
| PUT | `/modelTF/model-manage/{id}/purpose` | 修改用途 |
|
||||
| GET | `/modelTF/model-manage/local-models` | 扫描本地模型目录 |
|
||||
|
||||
创建/编辑请求:
|
||||
|
||||
@@ -234,10 +236,10 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/model-manage/trained-models` | 已训练模型列表 |
|
||||
| DELETE | `/api/model-manage/trained-models/{id}?type=merged\|lora` | 删除训练产物 |
|
||||
| POST | `/api/model-manage/merge` | 合并 LoRA 权重 |
|
||||
| GET | `/api/model-manage/trained-models/{model_name}/export` | 导出模型文件 |
|
||||
| GET | `/modelTF/model-manage/trained-models` | 已训练模型列表 |
|
||||
| DELETE | `/modelTF/model-manage/trained-models/{id}?type=merged\|lora` | 删除训练产物 |
|
||||
| POST | `/modelTF/model-manage/merge` | 合并 LoRA 权重 |
|
||||
| GET | `/modelTF/model-manage/trained-models/{model_name}/export` | 导出模型文件 |
|
||||
|
||||
合并请求:
|
||||
|
||||
@@ -264,14 +266,14 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/dataset-manage` | 数据集列表 |
|
||||
| GET | `/api/dataset-manage/{id}` | 数据集详情 |
|
||||
| POST | `/api/dataset-manage` | 创建数据集 |
|
||||
| PUT | `/api/dataset-manage/{id}` | 更新数据集 |
|
||||
| DELETE | `/api/dataset-manage/{id}` | 删除数据集 |
|
||||
| POST | `/api/dataset-manage/upload/{dataset_id}` | 上传文件,字段名 `files` |
|
||||
| GET | `/api/dataset-manage/download/{dataset_id}` | 打包下载数据集 |
|
||||
| GET | `/api/dataset-manage/download/{dataset_id}/{file_id}` | 下载单文件 |
|
||||
| GET | `/modelTF/dataset-manage` | 数据集列表 |
|
||||
| GET | `/modelTF/dataset-manage/{id}` | 数据集详情 |
|
||||
| POST | `/modelTF/dataset-manage` | 创建数据集 |
|
||||
| PUT | `/modelTF/dataset-manage/{id}` | 更新数据集 |
|
||||
| DELETE | `/modelTF/dataset-manage/{id}` | 删除数据集 |
|
||||
| POST | `/modelTF/dataset-manage/upload/{dataset_id}` | 上传文件,字段名 `files` |
|
||||
| GET | `/modelTF/dataset-manage/download/{dataset_id}` | 打包下载数据集 |
|
||||
| GET | `/modelTF/dataset-manage/download/{dataset_id}/{file_id}` | 下载单文件 |
|
||||
|
||||
创建数据集:
|
||||
|
||||
@@ -296,12 +298,12 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/dataset-manage/preview/{file_id}` | 当前版本内容预览 |
|
||||
| GET | `/api/dataset-manage/versions/{file_id}` | 文件版本列表 |
|
||||
| GET | `/api/dataset-manage/versions/{file_id}/{version_id}` | 读取历史版本 |
|
||||
| POST | `/api/dataset-manage/versions/{file_id}` | 保存为新版本 |
|
||||
| PUT | `/api/dataset-manage/versions/{file_id}/active` | 切换当前版本 |
|
||||
| DELETE | `/api/dataset-manage/versions/{file_id}/{version_id}` | 删除非当前、非初始版本 |
|
||||
| GET | `/modelTF/dataset-manage/preview/{file_id}` | 当前版本内容预览 |
|
||||
| GET | `/modelTF/dataset-manage/versions/{file_id}` | 文件版本列表 |
|
||||
| GET | `/modelTF/dataset-manage/versions/{file_id}/{version_id}` | 读取历史版本 |
|
||||
| POST | `/modelTF/dataset-manage/versions/{file_id}` | 保存为新版本 |
|
||||
| PUT | `/modelTF/dataset-manage/versions/{file_id}/active` | 切换当前版本 |
|
||||
| DELETE | `/modelTF/dataset-manage/versions/{file_id}/{version_id}` | 删除非当前、非初始版本 |
|
||||
|
||||
创建新版本:
|
||||
|
||||
@@ -318,21 +320,21 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
## 6. 数据处理
|
||||
|
||||
当前数据处理页面为本地模拟,正式后端建议实现以下接口。
|
||||
当前数据处理页面需要后端正式承接上传、切片、生成、编辑和发布流程,建议实现以下接口。
|
||||
|
||||
### 6.1 任务列表与详情
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/data-process` | 数据处理任务列表 |
|
||||
| POST | `/api/data-process` | 创建草稿任务 |
|
||||
| GET | `/api/data-process/{id}` | 任务详情 |
|
||||
| PUT | `/api/data-process/{id}` | 更新任务配置 |
|
||||
| DELETE | `/api/data-process/{id}` | 删除任务 |
|
||||
| POST | `/api/data-process/{id}/start` | 启动处理 |
|
||||
| POST | `/api/data-process/{id}/stop` | 停止处理 |
|
||||
| GET | `/api/data-process/{id}/progress` | 查询进度 |
|
||||
| GET | `/api/data-process/{id}/events` | SSE 实时进度 |
|
||||
| GET | `/modelTF/data-process` | 数据处理任务列表 |
|
||||
| POST | `/modelTF/data-process` | 创建草稿任务 |
|
||||
| GET | `/modelTF/data-process/{id}` | 任务详情 |
|
||||
| PUT | `/modelTF/data-process/{id}` | 更新任务配置 |
|
||||
| DELETE | `/modelTF/data-process/{id}` | 删除任务 |
|
||||
| POST | `/modelTF/data-process/{id}/start` | 启动处理 |
|
||||
| POST | `/modelTF/data-process/{id}/stop` | 停止处理 |
|
||||
| GET | `/modelTF/data-process/{id}/progress` | 查询进度 |
|
||||
| GET | `/modelTF/data-process/{id}/events` | SSE 实时进度 |
|
||||
|
||||
创建任务:
|
||||
|
||||
@@ -358,10 +360,10 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/api/data-process/{id}/source-files` | 上传源文件,字段名 `files` |
|
||||
| DELETE | `/api/data-process/{id}/source-files/{file_id}` | 移除源文件 |
|
||||
| POST | `/api/data-process/{id}/external/test` | 测试外部数据源连接 |
|
||||
| POST | `/api/data-process/{id}/external/pull` | 拉取外部数据并生成源文件 |
|
||||
| POST | `/modelTF/data-process/{id}/source-files` | 上传源文件,字段名 `files` |
|
||||
| DELETE | `/modelTF/data-process/{id}/source-files/{file_id}` | 移除源文件 |
|
||||
| POST | `/modelTF/data-process/{id}/external/test` | 测试外部数据源连接 |
|
||||
| POST | `/modelTF/data-process/{id}/external/pull` | 拉取外部数据并生成源文件 |
|
||||
|
||||
外部数据源请求:
|
||||
|
||||
@@ -383,11 +385,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/api/data-process/{id}/preview/build` | 根据源文件和配置生成预览切片 |
|
||||
| GET | `/api/data-process/{id}/preview` | 查询预览切片 |
|
||||
| PUT | `/api/data-process/{id}/preview/{preview_id}` | 编辑切片内容 |
|
||||
| POST | `/api/data-process/{id}/preview` | 手动新增切片 |
|
||||
| DELETE | `/api/data-process/{id}/preview/{preview_id}` | 删除切片 |
|
||||
| POST | `/modelTF/data-process/{id}/preview/build` | 根据源文件和配置生成预览切片 |
|
||||
| GET | `/modelTF/data-process/{id}/preview` | 查询预览切片 |
|
||||
| PUT | `/modelTF/data-process/{id}/preview/{preview_id}` | 编辑切片内容 |
|
||||
| POST | `/modelTF/data-process/{id}/preview` | 手动新增切片 |
|
||||
| DELETE | `/modelTF/data-process/{id}/preview/{preview_id}` | 删除切片 |
|
||||
|
||||
预览切片字段:
|
||||
|
||||
@@ -410,11 +412,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/api/data-process/{id}/generate` | 启动 LLM 生成 |
|
||||
| GET | `/api/data-process/{id}/results` | 查询结果明细 |
|
||||
| PUT | `/api/data-process/{id}/results/{result_id}` | 编辑结果 |
|
||||
| POST | `/api/data-process/{id}/results/{result_id}/restore` | 恢复原始结果 |
|
||||
| POST | `/api/data-process/{id}/publish` | 发布为数据集 |
|
||||
| POST | `/modelTF/data-process/{id}/generate` | 启动 LLM 生成 |
|
||||
| GET | `/modelTF/data-process/{id}/results` | 查询结果明细 |
|
||||
| PUT | `/modelTF/data-process/{id}/results/{result_id}` | 编辑结果 |
|
||||
| POST | `/modelTF/data-process/{id}/results/{result_id}/restore` | 恢复原始结果 |
|
||||
| POST | `/modelTF/data-process/{id}/publish` | 发布为数据集 |
|
||||
|
||||
结果字段:
|
||||
|
||||
@@ -445,17 +447,17 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/fine-tune` | 训练任务列表 |
|
||||
| GET | `/api/fine-tune/{id}` | 训练任务详情 |
|
||||
| GET | `/api/fine-tune/check-name?name=xxx` | 任务名查重 |
|
||||
| POST | `/api/fine-tune` | 创建训练任务记录 |
|
||||
| POST | `/api/fine-tune/start` | 启动训练 |
|
||||
| PUT | `/api/fine-tune/{id}` | 更新任务 |
|
||||
| POST | `/api/fine-tune/stop/{id}` | 停止任务 |
|
||||
| DELETE | `/api/fine-tune/{id}` | 删除任务 |
|
||||
| GET | `/api/fine-tune/progress/{id}` | 获取训练进度 |
|
||||
| GET | `/api/fine-tune/{id}/events` | SSE 训练日志/进度 |
|
||||
| POST | `/api/fine-tune/tensorboard/start` | 启动 TensorBoard |
|
||||
| GET | `/modelTF/fine-tune` | 训练任务列表 |
|
||||
| GET | `/modelTF/fine-tune/{id}` | 训练任务详情 |
|
||||
| GET | `/modelTF/fine-tune/check-name?name=xxx` | 任务名查重 |
|
||||
| POST | `/modelTF/fine-tune` | 创建训练任务记录 |
|
||||
| POST | `/modelTF/fine-tune/start` | 启动训练 |
|
||||
| PUT | `/modelTF/fine-tune/{id}` | 更新任务 |
|
||||
| POST | `/modelTF/fine-tune/stop/{id}` | 停止任务 |
|
||||
| DELETE | `/modelTF/fine-tune/{id}` | 删除任务 |
|
||||
| GET | `/modelTF/fine-tune/progress/{id}` | 获取训练进度 |
|
||||
| GET | `/modelTF/fine-tune/{id}/events` | SSE 训练日志/进度 |
|
||||
| POST | `/modelTF/fine-tune/tensorboard/start` | 启动 TensorBoard |
|
||||
|
||||
启动训练请求兼容前端 `FineTuneStartPayload`:
|
||||
|
||||
@@ -496,14 +498,14 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
训练日志页还会联合调用:
|
||||
|
||||
- `GET /api/fine-tune/{id}` 获取任务参数。
|
||||
- `GET /api/dataset-manage/{id}` 获取训练集信息。
|
||||
- `GET /api/system-info` 获取 GPU 状态。
|
||||
- `GET /api/training-log-files`、`GET /api/training-log-content` 获取日志。
|
||||
- `GET /modelTF/fine-tune/{id}` 获取任务参数。
|
||||
- `GET /modelTF/dataset-manage/{id}` 获取训练集信息。
|
||||
- `GET /modelTF/system-info` 获取 GPU 状态。
|
||||
- `GET /modelTF/training-log-files`、`GET /modelTF/training-log-content` 获取日志。
|
||||
|
||||
建议新增:
|
||||
|
||||
`GET /api/fine-tune/{id}/overview`
|
||||
`GET /modelTF/fine-tune/{id}/overview`
|
||||
|
||||
一次返回任务、数据集、GPU、日志摘要、指标曲线,减少页面聚合复杂度。
|
||||
|
||||
@@ -513,11 +515,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/model-eval` | 评测任务列表 |
|
||||
| GET | `/api/model-eval/{id}` | 评测详情 |
|
||||
| POST | `/api/model-eval/start` | 启动评测 |
|
||||
| DELETE | `/api/model-eval/{id}` | 删除评测 |
|
||||
| GET | `/api/model-eval/{id}/events` | SSE 评测进度 |
|
||||
| GET | `/modelTF/model-eval` | 评测任务列表 |
|
||||
| GET | `/modelTF/model-eval/{id}` | 评测详情 |
|
||||
| POST | `/modelTF/model-eval/start` | 启动评测 |
|
||||
| DELETE | `/modelTF/model-eval/{id}` | 删除评测 |
|
||||
| GET | `/modelTF/model-eval/{id}/events` | SSE 评测进度 |
|
||||
|
||||
启动评测:
|
||||
|
||||
@@ -550,11 +552,11 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/dimension` | 维度列表 |
|
||||
| GET | `/api/dimension/{id}` | 维度详情 |
|
||||
| POST | `/api/dimension` | 创建维度 |
|
||||
| PUT | `/api/dimension/{id}` | 编辑维度 |
|
||||
| DELETE | `/api/dimension/{id}` | 删除维度 |
|
||||
| GET | `/modelTF/dimension` | 维度列表 |
|
||||
| GET | `/modelTF/dimension/{id}` | 维度详情 |
|
||||
| POST | `/modelTF/dimension` | 创建维度 |
|
||||
| PUT | `/modelTF/dimension/{id}` | 编辑维度 |
|
||||
| DELETE | `/modelTF/dimension/{id}` | 删除维度 |
|
||||
|
||||
维度请求:
|
||||
|
||||
@@ -583,17 +585,17 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/model-compare` | 推理/对比任务列表 |
|
||||
| GET | `/api/model-compare/{id}` | 任务详情 |
|
||||
| POST | `/api/model-compare` | 创建任务 |
|
||||
| DELETE | `/api/model-compare/{id}` | 删除任务 |
|
||||
| POST | `/api/model-compare/{id}/load` | 加载任务内模型 |
|
||||
| POST | `/api/model-compare/{id}/unload` | 卸载任务内模型 |
|
||||
| GET | `/api/model-compare/{id}/load-status` | 查询加载状态 |
|
||||
| POST | `/api/model-compare/{id}/load-status` | 更新加载状态 |
|
||||
| POST | `/api/model-compare/{id}/start-model` | 启动单个模型服务 |
|
||||
| POST | `/api/model-compare/stop-by-pid` | 按 PID 停止模型 |
|
||||
| POST | `/api/model-compare/all/stop-all` | 停止全部旧模型服务 |
|
||||
| GET | `/modelTF/model-compare` | 推理/对比任务列表 |
|
||||
| GET | `/modelTF/model-compare/{id}` | 任务详情 |
|
||||
| POST | `/modelTF/model-compare` | 创建任务 |
|
||||
| DELETE | `/modelTF/model-compare/{id}` | 删除任务 |
|
||||
| POST | `/modelTF/model-compare/{id}/load` | 加载任务内模型 |
|
||||
| POST | `/modelTF/model-compare/{id}/unload` | 卸载任务内模型 |
|
||||
| GET | `/modelTF/model-compare/{id}/load-status` | 查询加载状态 |
|
||||
| POST | `/modelTF/model-compare/{id}/load-status` | 更新加载状态 |
|
||||
| POST | `/modelTF/model-compare/{id}/start-model` | 启动单个模型服务 |
|
||||
| POST | `/modelTF/model-compare/stop-by-pid` | 按 PID 停止模型 |
|
||||
| POST | `/modelTF/model-compare/all/stop-all` | 停止全部旧模型服务 |
|
||||
|
||||
创建任务:
|
||||
|
||||
@@ -617,12 +619,12 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/api/model-compare/stream-chat` | 流式对话,建议 SSE/chunked |
|
||||
| POST | `/api/model-compare/chat-with-port` | 指定端口非流式对话 |
|
||||
| POST | `/api/model-chat/batch` | API 模型批量对话 |
|
||||
| POST | `/api/model-chat/local/chat` | 本地模型对话 |
|
||||
| POST | `/api/model-chat/local/preload` | 预加载本地模型 |
|
||||
| POST | `/api/model-chat/trained/preload` | 预加载已训练模型 |
|
||||
| POST | `/modelTF/model-compare/stream-chat` | 流式对话,建议 SSE/chunked |
|
||||
| POST | `/modelTF/model-compare/chat-with-port` | 指定端口非流式对话 |
|
||||
| POST | `/modelTF/model-chat/batch` | API 模型批量对话 |
|
||||
| POST | `/modelTF/model-chat/local/chat` | 本地模型对话 |
|
||||
| POST | `/modelTF/model-chat/local/preload` | 预加载本地模型 |
|
||||
| POST | `/modelTF/model-chat/trained/preload` | 预加载已训练模型 |
|
||||
|
||||
流式请求:
|
||||
|
||||
@@ -649,10 +651,10 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/api/data-convert/jobs` | 创建转换任务,multipart 上传源文件 |
|
||||
| GET | `/api/data-convert/jobs/{id}` | 转换任务详情 |
|
||||
| GET | `/api/data-convert/jobs/{id}/download` | 下载转换结果 |
|
||||
| DELETE | `/api/data-convert/jobs/{id}` | 删除转换任务 |
|
||||
| POST | `/modelTF/data-convert/jobs` | 创建转换任务,multipart 上传源文件 |
|
||||
| GET | `/modelTF/data-convert/jobs/{id}` | 转换任务详情 |
|
||||
| GET | `/modelTF/data-convert/jobs/{id}/download` | 下载转换结果 |
|
||||
| DELETE | `/modelTF/data-convert/jobs/{id}` | 删除转换任务 |
|
||||
|
||||
请求字段:
|
||||
|
||||
@@ -667,18 +669,18 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/tools` | 工具列表 |
|
||||
| POST | `/api/tools` | 创建工具 |
|
||||
| GET | `/api/tools/{id}` | 工具详情 |
|
||||
| PUT | `/api/tools/{id}` | 编辑工具 |
|
||||
| DELETE | `/api/tools/{id}` | 删除工具 |
|
||||
| GET | `/modelTF/tools` | 工具列表 |
|
||||
| POST | `/modelTF/tools` | 创建工具 |
|
||||
| GET | `/modelTF/tools/{id}` | 工具详情 |
|
||||
| PUT | `/modelTF/tools/{id}` | 编辑工具 |
|
||||
| DELETE | `/modelTF/tools/{id}` | 删除工具 |
|
||||
|
||||
字段:`name`、`description`、`url`、`icon`、`visibility`、`owner_id`。
|
||||
|
||||
## 11. 后端开发需要补齐的关键点
|
||||
|
||||
1. 前端路由已有 `user-settings`、`user-create`、`user-permission`、`permission-denied`,但当前仓库缺少对应 Vue 文件;后端仍应先实现用户中心和权限接口。
|
||||
2. 数据处理主流程目前全在浏览器本地模拟,后端需要正式实现上传、切片、LLM 生成、结果编辑、发布数据集。
|
||||
2. 数据处理主流程需要后端正式实现上传、切片、LLM 生成、结果编辑、发布数据集。
|
||||
3. 训练、评测、数据处理、模型加载都不应同步阻塞 HTTP;建议接 Celery/RQ/Arq 或 FastAPI BackgroundTasks + 独立 worker。
|
||||
4. 文件内容不要全部入库;数据库保存元数据、版本、校验和、对象存储路径,内容放本地 NAS/MinIO。
|
||||
5. API Key、外部数据源密码必须加密存储,接口只回显脱敏。
|
||||
@@ -687,7 +689,7 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
## 12. 仍需确认的问题
|
||||
|
||||
1. 部署形态:单机多 GPU、K8s、多训练节点,还是只在一台服务器上调度?
|
||||
1. 部署形态已确认:多算力节点仍按“单机多 GPU 节点”管理,不引入 K8s;每台 GPU 服务器独立部署 Compute API/Agent/File Gateway/LLaMA-Factory。
|
||||
2. 文件存储:使用本地磁盘、NAS、MinIO,还是对象存储?是否需要断点续传?
|
||||
3. 训练框架:是否固定使用 LLaMA-Factory?是否还要支持 Transformers 原生、DeepSpeed、Accelerate?
|
||||
4. 权限粒度:页面级权限是否足够,还是需要到数据集/模型/任务的所有者与项目空间级权限?
|
||||
@@ -706,12 +708,12 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 | 权限 |
|
||||
| --- | --- | --- | --- |
|
||||
| GET | `/api/tenants` | 租户列表 | 平台管理员 |
|
||||
| POST | `/api/tenants` | 创建租户 | 平台管理员 |
|
||||
| GET | `/api/tenants/{id}` | 租户详情 | 租户管理员 |
|
||||
| PUT | `/api/tenants/{id}` | 更新租户 | 平台管理员 |
|
||||
| PUT | `/api/tenants/{id}/quota` | 设置租户配额 | 平台管理员 |
|
||||
| PUT | `/api/tenants/{id}/retention-policy` | 设置租户留存策略 | 平台管理员 |
|
||||
| GET | `/modelTF/tenants` | 租户列表 | 平台管理员 |
|
||||
| POST | `/modelTF/tenants` | 创建租户 | 平台管理员 |
|
||||
| GET | `/modelTF/tenants/{id}` | 租户详情 | 租户管理员 |
|
||||
| PUT | `/modelTF/tenants/{id}` | 更新租户 | 平台管理员 |
|
||||
| PUT | `/modelTF/tenants/{id}/quota` | 设置租户配额 | 平台管理员 |
|
||||
| PUT | `/modelTF/tenants/{id}/retention-policy` | 设置租户留存策略 | 平台管理员 |
|
||||
|
||||
创建租户:
|
||||
|
||||
@@ -738,15 +740,15 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/projects` | 当前用户可访问项目列表 |
|
||||
| POST | `/api/projects` | 创建项目 |
|
||||
| GET | `/api/projects/{id}` | 项目详情 |
|
||||
| PUT | `/api/projects/{id}` | 更新项目 |
|
||||
| POST | `/api/projects/{id}/archive` | 归档项目 |
|
||||
| GET | `/api/projects/{id}/members` | 项目成员 |
|
||||
| POST | `/api/projects/{id}/members` | 添加成员 |
|
||||
| PUT | `/api/projects/{id}/members/{user_id}` | 修改项目角色 |
|
||||
| DELETE | `/api/projects/{id}/members/{user_id}` | 移除成员 |
|
||||
| GET | `/modelTF/projects` | 当前用户可访问项目列表 |
|
||||
| POST | `/modelTF/projects` | 创建项目 |
|
||||
| GET | `/modelTF/projects/{id}` | 项目详情 |
|
||||
| PUT | `/modelTF/projects/{id}` | 更新项目 |
|
||||
| POST | `/modelTF/projects/{id}/archive` | 归档项目 |
|
||||
| GET | `/modelTF/projects/{id}/members` | 项目成员 |
|
||||
| POST | `/modelTF/projects/{id}/members` | 添加成员 |
|
||||
| PUT | `/modelTF/projects/{id}/members/{user_id}` | 修改项目角色 |
|
||||
| DELETE | `/modelTF/projects/{id}/members/{user_id}` | 移除成员 |
|
||||
|
||||
创建项目:
|
||||
|
||||
@@ -772,9 +774,9 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/resources/{resource_type}/{resource_id}/acl` | 查询资源授权 |
|
||||
| PUT | `/api/resources/{resource_type}/{resource_id}/acl` | 覆盖资源授权 |
|
||||
| POST | `/api/resources/{resource_type}/{resource_id}/share` | 快速分享给用户/项目角色 |
|
||||
| GET | `/modelTF/resources/{resource_type}/{resource_id}/acl` | 查询资源授权 |
|
||||
| PUT | `/modelTF/resources/{resource_type}/{resource_id}/acl` | 覆盖资源授权 |
|
||||
| POST | `/modelTF/resources/{resource_type}/{resource_id}/share` | 快速分享给用户/项目角色 |
|
||||
|
||||
授权请求:
|
||||
|
||||
@@ -801,14 +803,14 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/approvals` | 审批列表,支持 `type=pending/mine/done` |
|
||||
| POST | `/api/approvals` | 发起审批 |
|
||||
| GET | `/api/approvals/{id}` | 审批详情 |
|
||||
| POST | `/api/approvals/{id}/approve` | 通过 |
|
||||
| POST | `/api/approvals/{id}/reject` | 驳回 |
|
||||
| POST | `/api/approvals/{id}/cancel` | 撤回 |
|
||||
| GET | `/api/approval-templates` | 审批模板列表 |
|
||||
| PUT | `/api/approval-templates/{id}` | 更新审批模板 |
|
||||
| GET | `/modelTF/approvals` | 审批列表,支持 `type=pending/mine/done` |
|
||||
| POST | `/modelTF/approvals` | 发起审批 |
|
||||
| GET | `/modelTF/approvals/{id}` | 审批详情 |
|
||||
| POST | `/modelTF/approvals/{id}/approve` | 通过 |
|
||||
| POST | `/modelTF/approvals/{id}/reject` | 驳回 |
|
||||
| POST | `/modelTF/approvals/{id}/cancel` | 撤回 |
|
||||
| GET | `/modelTF/approval-templates` | 审批模板列表 |
|
||||
| PUT | `/modelTF/approval-templates/{id}` | 更新审批模板 |
|
||||
|
||||
发起审批:
|
||||
|
||||
@@ -831,17 +833,63 @@ page=1&page_size=20&keyword=xxx&sort=-created_at
|
||||
|
||||
### 13.5 算力资源与队列
|
||||
|
||||
应用平台对前端暴露 `/api/compute/*`,实际由 Compute Gateway 调用算力平台内部接口。
|
||||
应用平台对前端暴露 `/modelTF/compute/*`,实际由 Compute Gateway 调用算力平台内部接口。
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/compute/nodes` | 算力节点列表 |
|
||||
| GET | `/api/compute/gpus` | GPU 状态 |
|
||||
| GET | `/api/compute/queue` | 任务队列 |
|
||||
| GET | `/api/compute/jobs/{id}` | 算力任务详情 |
|
||||
| POST | `/api/compute/jobs/{id}/retry` | 重试任务 |
|
||||
| POST | `/api/compute/jobs/{id}/priority` | 调整优先级 |
|
||||
| POST | `/api/internal/compute-callbacks/jobs` | 算力平台任务回调 |
|
||||
| GET | `/modelTF/compute/nodes` | 算力节点列表 |
|
||||
| POST | `/modelTF/compute/nodes` | 新增算力节点 |
|
||||
| GET | `/modelTF/compute/nodes/{id}` | 算力节点详情 |
|
||||
| PUT | `/modelTF/compute/nodes/{id}` | 编辑节点地址、权重、标签、路径和启用状态 |
|
||||
| POST | `/modelTF/compute/nodes/{id}/test-connection` | 测试 Compute API/File Gateway 连通性 |
|
||||
| POST | `/modelTF/compute/nodes/{id}/enable` | 启用节点 |
|
||||
| POST | `/modelTF/compute/nodes/{id}/disable` | 禁用节点,不接收新任务 |
|
||||
| POST | `/modelTF/compute/nodes/{id}/drain` | 进入维护模式,已有任务跑完后下线 |
|
||||
| POST | `/modelTF/compute/nodes/{id}/health-check` | 主动触发节点健康检查 |
|
||||
| GET | `/modelTF/compute/nodes/{id}/engines` | 节点训练引擎和版本 |
|
||||
| GET | `/modelTF/compute/nodes/{id}/replicas` | 节点本地资源副本 |
|
||||
| GET | `/modelTF/compute/gpus` | GPU 状态 |
|
||||
| GET | `/modelTF/compute/queue` | 任务队列 |
|
||||
| GET | `/modelTF/compute/jobs/{id}` | 算力任务详情 |
|
||||
| POST | `/modelTF/compute/jobs/{id}/retry` | 重试任务 |
|
||||
| POST | `/modelTF/compute/jobs/{id}/priority` | 调整优先级 |
|
||||
| POST | `/modelTF/internal/compute-sync/jobs/poll` | 应用平台主动轮询并同步算力任务状态 |
|
||||
| POST | `/modelTF/internal/compute-sync/resources` | 调度前同步数据集/模型到目标节点 |
|
||||
|
||||
算力节点设计说明:
|
||||
|
||||
- 多算力节点仍按“单机多 GPU 节点”管理,每台 GPU 服务器是一条 `compute_nodes` 记录。
|
||||
- 每个可执行训练的节点都需要部署 `Compute API`、`Compute Agent`、`File Gateway` 和宿主机挂载的 LLaMA-Factory。
|
||||
- 节点之间默认不互相访问,应用平台主动访问所有节点的 Compute API/File Gateway。
|
||||
- 调度支持 `auto` 和 `manual`:普通用户默认自动调度,管理员或高级用户可手动指定节点。
|
||||
|
||||
算力节点响应字段:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "uuid",
|
||||
"code": "gpu-node-01",
|
||||
"name": "A800 Node 01",
|
||||
"api_base_url": "http://10.10.20.31:19100",
|
||||
"file_gateway_url": "http://10.10.20.31:19101",
|
||||
"enabled": true,
|
||||
"scheduler_status": "online",
|
||||
"scheduler_weight": 100,
|
||||
"tags": ["A800", "80GB", "llama_factory"],
|
||||
"gpu_count": 8,
|
||||
"current_running_jobs": 2,
|
||||
"max_parallel_jobs": 8,
|
||||
"data_root": "/data/yg-ft",
|
||||
"model_root": "/data/yg-ft/models",
|
||||
"log_root": "/opt/yg-ft/logs/compute",
|
||||
"last_health_check_at": "2026-07-20T12:00:00+08:00",
|
||||
"health_detail": {
|
||||
"compute_api": "ok",
|
||||
"file_gateway": "ok",
|
||||
"llama_factory": "ok"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
GPU 响应字段:
|
||||
|
||||
@@ -867,13 +915,13 @@ GPU 响应字段:
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/compute/jobs` | 创建训练/评测/数据处理/推理任务 |
|
||||
| GET | `/compute/jobs/{id}` | 查询任务 |
|
||||
| POST | `/compute/jobs/{id}/stop` | 停止任务 |
|
||||
| GET | `/compute/jobs/{id}/logs` | 拉取日志 |
|
||||
| GET | `/compute/resources/gpus` | 查询 GPU |
|
||||
| POST | `/compute/files/upload` | 上传到算力本地磁盘 |
|
||||
| GET | `/compute/files/{id}/download` | 下载文件 |
|
||||
| POST | `/modelTF/compute/jobs` | 创建训练/评测/数据处理/推理任务 |
|
||||
| GET | `/modelTF/compute/jobs/{id}` | 查询任务 |
|
||||
| POST | `/modelTF/compute/jobs/{id}/stop` | 停止任务 |
|
||||
| GET | `/modelTF/compute/jobs/{id}/logs` | 拉取日志 |
|
||||
| GET | `/modelTF/compute/resources/gpus` | 查询 GPU |
|
||||
| POST | `/modelTF/compute/files/upload` | 上传到算力本地磁盘 |
|
||||
| GET | `/modelTF/compute/files/{id}/download` | 下载文件 |
|
||||
|
||||
创建算力任务:
|
||||
|
||||
@@ -884,6 +932,13 @@ GPU 响应字段:
|
||||
"job_type": "fine_tune",
|
||||
"engine": "llama_factory",
|
||||
"priority": "normal",
|
||||
"scheduler": {
|
||||
"mode": "auto",
|
||||
"requested_node_id": null,
|
||||
"required_tags": ["A800"],
|
||||
"preferred_tags": ["llama_factory"],
|
||||
"min_gpu_memory_mb": 40960
|
||||
},
|
||||
"resource_request": {
|
||||
"gpu_count": 1,
|
||||
"gpu_ids": [0],
|
||||
@@ -897,21 +952,57 @@ GPU 响应字段:
|
||||
"dataset_path": "/data/ft-platform/.../datasets/train.jsonl",
|
||||
"training_args": {}
|
||||
},
|
||||
"callback_url": "http://app/api/internal/compute-callbacks/jobs"
|
||||
"status_sync_mode": "polling",
|
||||
"poll_interval_seconds": 10
|
||||
}
|
||||
```
|
||||
|
||||
手动指定节点时:
|
||||
|
||||
```json
|
||||
{
|
||||
"scheduler": {
|
||||
"mode": "manual",
|
||||
"requested_node_id": "uuid",
|
||||
"gpu_ids": [0, 1]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
调度前资源副本检查:
|
||||
|
||||
```json
|
||||
{
|
||||
"target_compute_node_id": "uuid",
|
||||
"resources": [
|
||||
{
|
||||
"resource_type": "model",
|
||||
"resource_id": "uuid",
|
||||
"required": true
|
||||
},
|
||||
{
|
||||
"resource_type": "dataset",
|
||||
"resource_id": "uuid",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"sync_if_missing": true
|
||||
}
|
||||
```
|
||||
|
||||
如果目标节点缺少数据集或模型副本,应用平台通过 File Gateway 创建 `resource_sync_jobs`,同步完成后再提交训练任务。
|
||||
|
||||
### 13.7 文件网关与离线导入
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/api/files/upload-session` | 创建分片上传会话 |
|
||||
| PUT | `/api/files/upload-session/{id}/parts/{part_no}` | 上传分片 |
|
||||
| POST | `/api/files/upload-session/{id}/complete` | 完成上传 |
|
||||
| GET | `/api/files/{id}/preview` | 文件预览 |
|
||||
| GET | `/api/files/{id}/download-url` | 获取短时下载链接 |
|
||||
| POST | `/api/import/local-model` | 从算力节点本地路径导入模型 |
|
||||
| POST | `/api/import/local-dataset` | 从算力节点本地路径导入数据集 |
|
||||
| POST | `/modelTF/files/upload-session` | 创建分片上传会话 |
|
||||
| PUT | `/modelTF/files/upload-session/{id}/parts/{part_no}` | 上传分片 |
|
||||
| POST | `/modelTF/files/upload-session/{id}/complete` | 完成上传 |
|
||||
| GET | `/modelTF/files/{id}/preview` | 文件预览 |
|
||||
| GET | `/modelTF/files/{id}/download-url` | 获取短时下载链接 |
|
||||
| POST | `/modelTF/import/local-model` | 从算力节点本地路径导入模型 |
|
||||
| POST | `/modelTF/import/local-dataset` | 从算力节点本地路径导入数据集 |
|
||||
|
||||
离线导入模型:
|
||||
|
||||
@@ -930,10 +1021,10 @@ GPU 响应字段:
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/training-engines` | 训练引擎列表 |
|
||||
| GET | `/api/training-engines/{id}` | 引擎详情 |
|
||||
| GET | `/api/training-engines/{id}/schema` | 参数 schema |
|
||||
| POST | `/api/training-engines/{id}/health-check` | 健康检查 |
|
||||
| GET | `/modelTF/training-engines` | 训练引擎列表 |
|
||||
| GET | `/modelTF/training-engines/{id}` | 引擎详情 |
|
||||
| GET | `/modelTF/training-engines/{id}/schema` | 参数 schema |
|
||||
| POST | `/modelTF/training-engines/{id}/health-check` | 健康检查 |
|
||||
|
||||
LLaMA-Factory 引擎声明:
|
||||
|
||||
@@ -953,11 +1044,11 @@ LLaMA-Factory 引擎声明:
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/fine-tune/{id}/checkpoints` | checkpoint 列表 |
|
||||
| POST | `/api/fine-tune/{id}/retry` | 失败任务重试 |
|
||||
| POST | `/api/fine-tune/{id}/resume` | 从 checkpoint 恢复训练 |
|
||||
| DELETE | `/api/fine-tune/{id}/checkpoints/{checkpoint_id}` | 删除 checkpoint,可能触发审批 |
|
||||
| PUT | `/api/fine-tune/{id}/checkpoint-retention` | 设置 checkpoint 保留策略 |
|
||||
| GET | `/modelTF/fine-tune/{id}/checkpoints` | checkpoint 列表 |
|
||||
| POST | `/modelTF/fine-tune/{id}/retry` | 失败任务重试 |
|
||||
| POST | `/modelTF/fine-tune/{id}/resume` | 从 checkpoint 恢复训练 |
|
||||
| DELETE | `/modelTF/fine-tune/{id}/checkpoints/{checkpoint_id}` | 删除 checkpoint,可能触发审批 |
|
||||
| PUT | `/modelTF/fine-tune/{id}/checkpoint-retention` | 设置 checkpoint 保留策略 |
|
||||
|
||||
默认保留策略:最近 3 个、最优 2 个、已发布模型关联 checkpoint 不自动删除、失败任务保留 14 天。
|
||||
|
||||
@@ -965,13 +1056,13 @@ LLaMA-Factory 引擎声明:
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| GET | `/api/audit-logs` | 操作审计 |
|
||||
| GET | `/api/login-logs` | 登录审计 |
|
||||
| GET | `/api/download-logs` | 下载审计 |
|
||||
| GET | `/api/retention-policies` | 留存策略 |
|
||||
| PUT | `/api/retention-policies/{id}` | 更新留存策略 |
|
||||
| GET | `/api/quotas/usage` | 配额使用 |
|
||||
| GET | `/api/usage/summary` | GPU 小时、磁盘、推理调用统计 |
|
||||
| GET | `/modelTF/audit-logs` | 操作审计 |
|
||||
| GET | `/modelTF/login-logs` | 登录审计 |
|
||||
| GET | `/modelTF/download-logs` | 下载审计 |
|
||||
| GET | `/modelTF/retention-policies` | 留存策略 |
|
||||
| PUT | `/modelTF/retention-policies/{id}` | 更新留存策略 |
|
||||
| GET | `/modelTF/quotas/usage` | 配额使用 |
|
||||
| GET | `/modelTF/usage/summary` | GPU 小时、磁盘、推理调用统计 |
|
||||
|
||||
第一版只做用量统计,不做账单计费;第二期可扩展成本核算。
|
||||
|
||||
@@ -983,88 +1074,88 @@ LLaMA-Factory 引擎声明:
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 登录页 | `/login` | `POST /api/login`、`GET /api/me`、`POST /api/logout` | 登录、恢复用户、退出 |
|
||||
| 用户中心 | `/user-settings` | `GET /api/users` | 用户列表、搜索、状态筛选 |
|
||||
| 创建用户 | `/user-settings/create` | `POST /api/users` | 创建本地用户 |
|
||||
| 用户权限 | `/user-settings/:id/permission` | `PUT /api/users/{id}`、`PUT /api/users/{id}/password` | 用户角色、状态、页面权限、重置密码 |
|
||||
| 无权限页 | `/permission-denied` | 无专属接口,可调用 `GET /api/me` | 展示当前用户权限和返回入口 |
|
||||
| 登录页 | `/login` | `POST /modelTF/login`、`GET /modelTF/me`、`POST /modelTF/logout` | 登录、恢复用户、退出 |
|
||||
| 用户中心 | `/user-settings` | `GET /modelTF/users` | 用户列表、搜索、状态筛选 |
|
||||
| 创建用户 | `/user-settings/create` | `POST /modelTF/users` | 创建本地用户 |
|
||||
| 用户权限 | `/user-settings/:id/permission` | `PUT /modelTF/users/{id}`、`PUT /modelTF/users/{id}/password` | 用户角色、状态、页面权限、重置密码 |
|
||||
| 无权限页 | `/permission-denied` | 无专属接口,可调用 `GET /modelTF/me` | 展示当前用户权限和返回入口 |
|
||||
|
||||
### 14.2 租户、项目和资源授权
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 租户管理 | `/tenants` | `GET /api/tenants`、`POST /api/tenants` | 租户列表、创建租户 |
|
||||
| 租户详情 | `/tenants/:id` | `GET /api/tenants/{id}`、`PUT /api/tenants/{id}`、`PUT /api/tenants/{id}/quota`、`PUT /api/tenants/{id}/retention-policy` | 租户配置、配额、留存 |
|
||||
| 项目列表 | `/projects` | `GET /api/projects`、`POST /api/projects` | 项目列表、创建项目 |
|
||||
| 项目详情 | `/projects/:id` | `GET /api/projects/{id}`、`PUT /api/projects/{id}`、`POST /api/projects/{id}/archive` | 项目概览、归档 |
|
||||
| 项目成员 | `/projects/:id/members` | `GET /api/projects/{id}/members`、`POST /api/projects/{id}/members`、`PUT /api/projects/{id}/members/{user_id}`、`DELETE /api/projects/{id}/members/{user_id}` | 成员和项目角色 |
|
||||
| 资源授权 | 资源详情弹窗或 `/projects/:id/permissions` | `GET /api/resources/{resource_type}/{resource_id}/acl`、`PUT /api/resources/{resource_type}/{resource_id}/acl`、`POST /api/resources/{resource_type}/{resource_id}/share` | 模型/数据集/任务级 ACL |
|
||||
| 租户管理 | `/tenants` | `GET /modelTF/tenants`、`POST /modelTF/tenants` | 租户列表、创建租户 |
|
||||
| 租户详情 | `/tenants/:id` | `GET /modelTF/tenants/{id}`、`PUT /modelTF/tenants/{id}`、`PUT /modelTF/tenants/{id}/quota`、`PUT /modelTF/tenants/{id}/retention-policy` | 租户配置、配额、留存 |
|
||||
| 项目列表 | `/projects` | `GET /modelTF/projects`、`POST /modelTF/projects` | 项目列表、创建项目 |
|
||||
| 项目详情 | `/projects/:id` | `GET /modelTF/projects/{id}`、`PUT /modelTF/projects/{id}`、`POST /modelTF/projects/{id}/archive` | 项目概览、归档 |
|
||||
| 项目成员 | `/projects/:id/members` | `GET /modelTF/projects/{id}/members`、`POST /modelTF/projects/{id}/members`、`PUT /modelTF/projects/{id}/members/{user_id}`、`DELETE /modelTF/projects/{id}/members/{user_id}` | 成员和项目角色 |
|
||||
| 资源授权 | 资源详情弹窗或 `/projects/:id/permissions` | `GET /modelTF/resources/{resource_type}/{resource_id}/acl`、`PUT /modelTF/resources/{resource_type}/{resource_id}/acl`、`POST /modelTF/resources/{resource_type}/{resource_id}/share` | 模型/数据集/任务级 ACL |
|
||||
|
||||
### 14.3 看板、监控和日志
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 服务看板 | `/dashboard` | `GET /api/dashboard/overview`、`GET /api/health` | 首页聚合、轻量健康指标 |
|
||||
| 平台性能 | `/hardware` | `GET /api/system-info`、`GET /api/compute/gpus` | CPU、内存、磁盘、GPU、任务占用 |
|
||||
| 系统日志 | `/logs` | `GET /api/log-files`、`GET /api/log-content` | 系统日志列表和内容 |
|
||||
| 训练日志页 | `/training-log/:id` | `GET /api/fine-tune/{id}/overview`、`GET /api/training-log-files`、`GET /api/training-log-content` | 训练日志、指标、GPU 状态 |
|
||||
| 服务看板 | `/dashboard` | `GET /modelTF/dashboard/overview`、`GET /modelTF/health` | 首页聚合、轻量健康指标 |
|
||||
| 平台性能 | `/hardware` | `GET /modelTF/system-info`、`GET /modelTF/compute/gpus` | CPU、内存、磁盘、GPU、任务占用 |
|
||||
| 系统日志 | `/logs` | `GET /modelTF/log-files`、`GET /modelTF/log-content` | 系统日志列表和内容 |
|
||||
| 训练日志页 | `/training-log/:id` | `GET /modelTF/fine-tune/{id}/overview`、`GET /modelTF/training-log-files`、`GET /modelTF/training-log-content` | 训练日志、指标、GPU 状态 |
|
||||
|
||||
### 14.4 模型管理
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 模型列表 | `/model-manage` | `GET /api/model-manage`、`DELETE /api/model-manage/{id}`、`PUT /api/model-manage/{id}/purpose` | 模型列表、删除审批入口、用途变更 |
|
||||
| 模型创建/编辑 | `/model-manage/create`、`/model-manage/:id/edit` | `GET /api/model-manage/{id}`、`POST /api/model-manage`、`PUT /api/model-manage/{id}`、`GET /api/model-manage/local-models` | 本地/API 模型登记 |
|
||||
| 离线导入模型 | 模型创建页或导入弹窗 | `POST /api/import/local-model` | 从算力节点本地路径导入 |
|
||||
| 已训练模型 | 模型列表/选择弹窗 | `GET /api/model-manage/trained-models`、`DELETE /api/model-manage/trained-models/{id}` | 训练产物列表和删除 |
|
||||
| 权重合并 | `/model-manage/merge` | `POST /api/model-manage/merge` | LoRA 合并任务 |
|
||||
| 模型导出 | 模型列表/详情 | `GET /api/model-manage/trained-models/{model_name}/export` | 导出下载,必要时触发审批 |
|
||||
| 模型列表 | `/model-manage` | `GET /modelTF/model-manage`、`DELETE /modelTF/model-manage/{id}`、`PUT /modelTF/model-manage/{id}/purpose` | 模型列表、删除审批入口、用途变更 |
|
||||
| 模型创建/编辑 | `/model-manage/create`、`/model-manage/:id/edit` | `GET /modelTF/model-manage/{id}`、`POST /modelTF/model-manage`、`PUT /modelTF/model-manage/{id}`、`GET /modelTF/model-manage/local-models` | 本地/API 模型登记 |
|
||||
| 离线导入模型 | 模型创建页或导入弹窗 | `POST /modelTF/import/local-model` | 从算力节点本地路径导入 |
|
||||
| 已训练模型 | 模型列表/选择弹窗 | `GET /modelTF/model-manage/trained-models`、`DELETE /modelTF/model-manage/trained-models/{id}` | 训练产物列表和删除 |
|
||||
| 权重合并 | `/model-manage/merge` | `POST /modelTF/model-manage/merge` | LoRA 合并任务 |
|
||||
| 模型导出 | 模型列表/详情 | `GET /modelTF/model-manage/trained-models/{model_name}/export` | 导出下载,必要时触发审批 |
|
||||
|
||||
### 14.5 数据集与数据处理
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 数据集列表 | `/dataset` | `GET /api/dataset-manage`、`DELETE /api/dataset-manage/{id}`、`GET /api/dataset-manage/download/{id}` | 数据集列表、删除、打包下载 |
|
||||
| 数据集创建/编辑 | `/dataset/create`、`/dataset/:id/edit` | `POST /api/dataset-manage`、`PUT /api/dataset-manage/{id}`、`POST /api/dataset-manage/upload/{dataset_id}` | 数据集元数据和文件上传 |
|
||||
| 离线导入数据集 | 数据集创建页或导入弹窗 | `POST /api/import/local-dataset` | 从算力节点目录导入 |
|
||||
| 数据集预览 | `/dataset/:id/preview` | `GET /api/dataset-manage/preview/{file_id}`、`GET /api/dataset-manage/versions/{file_id}`、`POST /api/dataset-manage/versions/{file_id}`、`PUT /api/dataset-manage/versions/{file_id}/active`、`DELETE /api/dataset-manage/versions/{file_id}/{version_id}` | 文件预览、版本、在线编辑 |
|
||||
| 数据处理列表 | `/data-process` | `GET /api/data-process`、`DELETE /api/data-process/{id}` | 处理任务列表 |
|
||||
| 数据处理创建向导 | `/data-process/create` | `POST /api/data-process`、`POST /api/data-process/{id}/source-files`、`POST /api/data-process/{id}/preview/build`、`POST /api/data-process/{id}/generate`、`POST /api/data-process/{id}/publish` | 创建、上传、预览、生成、发布 |
|
||||
| 数据处理详情 | `/data-process/:id` | `GET /api/data-process/{id}`、`GET /api/data-process/{id}/results`、`GET /api/data-process/{id}/progress`、`GET /api/data-process/{id}/events` | 详情、结果、进度 |
|
||||
| 数据转换 | `/data-convert` | `POST /api/data-convert/jobs`、`GET /api/data-convert/jobs/{id}`、`GET /api/data-convert/jobs/{id}/download` | JSON/JSONL 转换 |
|
||||
| 数据集列表 | `/dataset` | `GET /modelTF/dataset-manage`、`DELETE /modelTF/dataset-manage/{id}`、`GET /modelTF/dataset-manage/download/{id}` | 数据集列表、删除、打包下载 |
|
||||
| 数据集创建/编辑 | `/dataset/create`、`/dataset/:id/edit` | `POST /modelTF/dataset-manage`、`PUT /modelTF/dataset-manage/{id}`、`POST /modelTF/dataset-manage/upload/{dataset_id}` | 数据集元数据和文件上传 |
|
||||
| 离线导入数据集 | 数据集创建页或导入弹窗 | `POST /modelTF/import/local-dataset` | 从算力节点目录导入 |
|
||||
| 数据集预览 | `/dataset/:id/preview` | `GET /modelTF/dataset-manage/preview/{file_id}`、`GET /modelTF/dataset-manage/versions/{file_id}`、`POST /modelTF/dataset-manage/versions/{file_id}`、`PUT /modelTF/dataset-manage/versions/{file_id}/active`、`DELETE /modelTF/dataset-manage/versions/{file_id}/{version_id}` | 文件预览、版本、在线编辑 |
|
||||
| 数据处理列表 | `/data-process` | `GET /modelTF/data-process`、`DELETE /modelTF/data-process/{id}` | 处理任务列表 |
|
||||
| 数据处理创建向导 | `/data-process/create` | `POST /modelTF/data-process`、`POST /modelTF/data-process/{id}/source-files`、`POST /modelTF/data-process/{id}/preview/build`、`POST /modelTF/data-process/{id}/generate`、`POST /modelTF/data-process/{id}/publish` | 创建、上传、预览、生成、发布 |
|
||||
| 数据处理详情 | `/data-process/:id` | `GET /modelTF/data-process/{id}`、`GET /modelTF/data-process/{id}/results`、`GET /modelTF/data-process/{id}/progress`、`GET /modelTF/data-process/{id}/events` | 详情、结果、进度 |
|
||||
| 数据转换 | `/data-convert` | `POST /modelTF/data-convert/jobs`、`GET /modelTF/data-convert/jobs/{id}`、`GET /modelTF/data-convert/jobs/{id}/download` | JSON/JSONL 转换 |
|
||||
|
||||
### 14.6 微调训练
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 微调列表 | `/fine-tune` | `GET /api/fine-tune`、`POST /api/fine-tune/stop/{id}`、`DELETE /api/fine-tune/{id}` | 训练任务列表、停止、删除 |
|
||||
| 微调创建 | `/fine-tune/create` | `GET /api/fine-tune/check-name`、`POST /api/fine-tune`、`POST /api/fine-tune/start`、`GET /api/model-manage`、`GET /api/dataset-manage`、`GET /api/compute/gpus` | 参数配置、GPU 选择、启动训练 |
|
||||
| 训练详情/日志 | `/training-log/:id` | `GET /api/fine-tune/{id}`、`GET /api/fine-tune/progress/{id}`、`GET /api/fine-tune/{id}/events`、`GET /api/fine-tune/{id}/checkpoints` | 日志、进度、checkpoint |
|
||||
| 恢复/重试训练 | 训练详情页 | `POST /api/fine-tune/{id}/retry`、`POST /api/fine-tune/{id}/resume` | 从 checkpoint 重试或恢复 |
|
||||
| Checkpoint 管理 | 训练详情页、存储管理页 | `DELETE /api/fine-tune/{id}/checkpoints/{checkpoint_id}`、`PUT /api/fine-tune/{id}/checkpoint-retention` | 清理策略和删除 |
|
||||
| TensorBoard | 训练详情页 | `POST /api/fine-tune/tensorboard/start` | 启动 TensorBoard |
|
||||
| 微调列表 | `/fine-tune` | `GET /modelTF/fine-tune`、`POST /modelTF/fine-tune/stop/{id}`、`DELETE /modelTF/fine-tune/{id}` | 训练任务列表、停止、删除 |
|
||||
| 微调创建 | `/fine-tune/create` | `GET /modelTF/fine-tune/check-name`、`POST /modelTF/fine-tune`、`POST /modelTF/fine-tune/start`、`GET /modelTF/model-manage`、`GET /modelTF/dataset-manage`、`GET /modelTF/compute/gpus` | 参数配置、GPU 选择、启动训练 |
|
||||
| 训练详情/日志 | `/training-log/:id` | `GET /modelTF/fine-tune/{id}`、`GET /modelTF/fine-tune/progress/{id}`、`GET /modelTF/fine-tune/{id}/events`、`GET /modelTF/fine-tune/{id}/checkpoints` | 日志、进度、checkpoint |
|
||||
| 恢复/重试训练 | 训练详情页 | `POST /modelTF/fine-tune/{id}/retry`、`POST /modelTF/fine-tune/{id}/resume` | 从 checkpoint 重试或恢复 |
|
||||
| Checkpoint 管理 | 训练详情页、存储管理页 | `DELETE /modelTF/fine-tune/{id}/checkpoints/{checkpoint_id}`、`PUT /modelTF/fine-tune/{id}/checkpoint-retention` | 清理策略和删除 |
|
||||
| TensorBoard | 训练详情页 | `POST /modelTF/fine-tune/tensorboard/start` | 启动 TensorBoard |
|
||||
|
||||
### 14.7 评测、推理、对比和服务发布
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 评测列表 | `/model-eval` | `GET /api/model-eval`、`DELETE /api/model-eval/{id}` | 评测任务列表 |
|
||||
| 评测创建 | `/model-eval/create` | `POST /api/model-eval/start`、`GET /api/dimension`、`GET /api/model-manage/trained-models`、`GET /api/dataset-manage`、`GET /api/compute/gpus` | 选择模型、数据集、维度、GPU |
|
||||
| 评测详情 | `/model-eval/:id` | `GET /api/model-eval/{id}`、`GET /api/model-eval/{id}/events` | 综合结果、样本评分 |
|
||||
| 评测维度 | `/model-eval/dimension/:id/edit` | `GET /api/dimension/{id}`、`POST /api/dimension`、`PUT /api/dimension/{id}`、`DELETE /api/dimension/{id}` | 维度和 Prompt 管理 |
|
||||
| 推理列表 | `/model-inference` | `GET /api/model-compare`、`POST /api/model-compare/{id}/load`、`POST /api/model-compare/{id}/unload` | 推理任务和加载状态 |
|
||||
| 推理创建 | `/model-inference/create` | `POST /api/model-compare`、`GET /api/model-manage`、`GET /api/model-manage/trained-models`、`GET /api/compute/gpus` | 选择模型和 GPU |
|
||||
| 推理对话 | `/model-inference/chat/:id` | `GET /api/model-compare/{id}`、`POST /api/model-compare/stream-chat`、`POST /api/model-compare/chat-with-port` | 单模型对话 |
|
||||
| 模型对比 | `/model-compare/chat/:id`、`/model-compare/result` | `POST /api/model-chat/batch`、`POST /api/model-chat/local/chat`、`POST /api/model-chat/local/preload`、`POST /api/model-chat/trained/preload` | 多模型对比和预加载 |
|
||||
| 模型服务治理 | `/model-services`、`/model-services/:id` | `POST /api/approvals`、`GET /api/compute/jobs/{id}`、`GET /api/usage/summary` | 测试/生产服务发布、调用统计、下线审批 |
|
||||
| 评测列表 | `/model-eval` | `GET /modelTF/model-eval`、`DELETE /modelTF/model-eval/{id}` | 评测任务列表 |
|
||||
| 评测创建 | `/model-eval/create` | `POST /modelTF/model-eval/start`、`GET /modelTF/dimension`、`GET /modelTF/model-manage/trained-models`、`GET /modelTF/dataset-manage`、`GET /modelTF/compute/gpus` | 选择模型、数据集、维度、GPU |
|
||||
| 评测详情 | `/model-eval/:id` | `GET /modelTF/model-eval/{id}`、`GET /modelTF/model-eval/{id}/events` | 综合结果、样本评分 |
|
||||
| 评测维度 | `/model-eval/dimension/:id/edit` | `GET /modelTF/dimension/{id}`、`POST /modelTF/dimension`、`PUT /modelTF/dimension/{id}`、`DELETE /modelTF/dimension/{id}` | 维度和 Prompt 管理 |
|
||||
| 推理列表 | `/model-inference` | `GET /modelTF/model-compare`、`POST /modelTF/model-compare/{id}/load`、`POST /modelTF/model-compare/{id}/unload` | 推理任务和加载状态 |
|
||||
| 推理创建 | `/model-inference/create` | `POST /modelTF/model-compare`、`GET /modelTF/model-manage`、`GET /modelTF/model-manage/trained-models`、`GET /modelTF/compute/gpus` | 选择模型和 GPU |
|
||||
| 推理对话 | `/model-inference/chat/:id` | `GET /modelTF/model-compare/{id}`、`POST /modelTF/model-compare/stream-chat`、`POST /modelTF/model-compare/chat-with-port` | 单模型对话 |
|
||||
| 模型对比 | `/model-compare/chat/:id`、`/model-compare/result` | `POST /modelTF/model-chat/batch`、`POST /modelTF/model-chat/local/chat`、`POST /modelTF/model-chat/local/preload`、`POST /modelTF/model-chat/trained/preload` | 多模型对比和预加载 |
|
||||
| 模型服务治理 | `/model-services`、`/model-services/:id` | `POST /modelTF/approvals`、`GET /modelTF/compute/jobs/{id}`、`GET /modelTF/usage/summary` | 测试/生产服务发布、调用统计、下线审批 |
|
||||
|
||||
### 14.8 审批、审计、算力和存储运维
|
||||
|
||||
| 页面模块 | 路由/入口 | 接口 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 审批中心 | `/approvals`、`/approvals/pending`、`/approvals/mine`、`/approvals/:id` | `GET /api/approvals`、`POST /api/approvals`、`GET /api/approvals/{id}`、`POST /api/approvals/{id}/approve`、`POST /api/approvals/{id}/reject`、`POST /api/approvals/{id}/cancel` | 审批列表和审批动作 |
|
||||
| 审批设置 | `/approval-settings` | `GET /api/approval-templates`、`PUT /api/approval-templates/{id}` | 审批模板 |
|
||||
| 算力资源 | `/compute`、`/compute/gpus`、`/compute/queue`、`/compute/nodes` | `GET /api/compute/nodes`、`GET /api/compute/gpus`、`GET /api/compute/queue`、`POST /api/compute/jobs/{id}/retry`、`POST /api/compute/jobs/{id}/priority` | GPU、节点、队列 |
|
||||
| 存储管理 | `/storage` | `GET /api/quotas/usage`、`GET /api/files/{id}/download-url`、`GET /api/retention-policies`、`PUT /api/retention-policies/{id}` | 磁盘占用、下载、留存 |
|
||||
| 审计中心 | `/audit-logs`、`/login-logs`、`/download-logs` | `GET /api/audit-logs`、`GET /api/login-logs`、`GET /api/download-logs` | 操作、登录、下载审计 |
|
||||
| 训练引擎管理 | `/training-engines` | `GET /api/training-engines`、`GET /api/training-engines/{id}`、`GET /api/training-engines/{id}/schema`、`POST /api/training-engines/{id}/health-check` | 引擎能力和健康 |
|
||||
| 审批中心 | `/approvals`、`/approvals/pending`、`/approvals/mine`、`/approvals/:id` | `GET /modelTF/approvals`、`POST /modelTF/approvals`、`GET /modelTF/approvals/{id}`、`POST /modelTF/approvals/{id}/approve`、`POST /modelTF/approvals/{id}/reject`、`POST /modelTF/approvals/{id}/cancel` | 审批列表和审批动作 |
|
||||
| 审批设置 | `/approval-settings` | `GET /modelTF/approval-templates`、`PUT /modelTF/approval-templates/{id}` | 审批模板 |
|
||||
| 算力资源 | `/compute`、`/compute/gpus`、`/compute/queue`、`/compute/nodes` | `GET/POST/PUT /modelTF/compute/nodes`、`POST /modelTF/compute/nodes/{id}/test-connection`、`POST /modelTF/compute/nodes/{id}/enable`、`POST /modelTF/compute/nodes/{id}/disable`、`POST /modelTF/compute/nodes/{id}/drain`、`GET /modelTF/compute/gpus`、`GET /modelTF/compute/queue`、`POST /modelTF/compute/jobs/{id}/retry`、`POST /modelTF/compute/jobs/{id}/priority` | GPU、节点、队列、节点权重、标签、维护状态、资源副本 |
|
||||
| 存储管理 | `/storage` | `GET /modelTF/quotas/usage`、`GET /modelTF/files/{id}/download-url`、`GET /modelTF/retention-policies`、`PUT /modelTF/retention-policies/{id}` | 磁盘占用、下载、留存 |
|
||||
| 审计中心 | `/audit-logs`、`/login-logs`、`/download-logs` | `GET /modelTF/audit-logs`、`GET /modelTF/login-logs`、`GET /modelTF/download-logs` | 操作、登录、下载审计 |
|
||||
| 训练引擎管理 | `/training-engines` | `GET /modelTF/training-engines`、`GET /modelTF/training-engines/{id}`、`GET /modelTF/training-engines/{id}/schema`、`POST /modelTF/training-engines/{id}/health-check` | 引擎能力和健康 |
|
||||
|
||||
@@ -43,7 +43,7 @@ LOG_RETENTION_DAYS=10
|
||||
"@timestamp": "2026-07-16T13:20:10.123",
|
||||
"level": "INFO",
|
||||
"logger": "app.access",
|
||||
"message": "request completed method=GET path=/api/v1/health status_code=200 duration_ms=3.12 client=127.0.0.1",
|
||||
"message": "request completed method=GET path=/modelTF/health status_code=200 duration_ms=3.12 client=127.0.0.1",
|
||||
"module": "logging",
|
||||
"function": "request_logging_middleware",
|
||||
"file": "D:\\AI\\codex-code\\YG_FT\\backend\\app\\core\\logging.py",
|
||||
|
||||
@@ -63,7 +63,7 @@
|
||||
|
||||
### 4.1 适用场景
|
||||
|
||||
- PoC、试点环境、演示环境。
|
||||
- 开发联调、单机试运行、资源受限的早期上线环境。
|
||||
- 小团队共用一台单机多 GPU 服务器。
|
||||
- 网络隔离要求不高,部署资源有限。
|
||||
|
||||
@@ -111,11 +111,11 @@ flowchart LR
|
||||
| 服务 | 端口 | 暴露范围 |
|
||||
| --- | --- | --- |
|
||||
| Nginx | 80/443 | 用户网段 |
|
||||
| Backend API | 8000 | 仅 Nginx、本机 |
|
||||
| Compute API | 9100 | 仅 Backend API、本机 |
|
||||
| File Gateway | 9101 | 仅 Backend API、本机 |
|
||||
| PostgreSQL | 5432 | 本机或内网 |
|
||||
| Redis | 6379 | 本机或内网 |
|
||||
| Backend API | 17861 | 仅 Nginx、本机 |
|
||||
| Compute API | 19100 | 仅 Backend API、本机 |
|
||||
| File Gateway | 19101 | 仅 Backend API、本机 |
|
||||
| PostgreSQL | 15432 | 本机或内网 |
|
||||
| Redis | 16379 | 本机或内网 |
|
||||
|
||||
## 5. 方案二:应用服务与算力/训练服务独立部署
|
||||
|
||||
@@ -139,7 +139,7 @@ flowchart LR
|
||||
A --> L["LLaMA-Factory"]
|
||||
A --> G["GPU/CUDA"]
|
||||
A --> FS["算力服务器本地磁盘"]
|
||||
A -- "状态回调/日志摘要" --> B
|
||||
B -- "定时轮询任务状态/指标/产物索引" --> C
|
||||
```
|
||||
|
||||
### 5.3 部署边界
|
||||
@@ -170,8 +170,8 @@ GPU 算力服务器部署:
|
||||
- 协议:内部 HTTPS REST,后续可扩展 gRPC。
|
||||
- 鉴权:服务间 Token,生产建议 mTLS + IP 白名单。
|
||||
- 幂等:训练任务提交使用 `Idempotency-Key` 或 `job_id`。
|
||||
- 回调:算力平台向应用平台回调任务状态、指标摘要、产物索引。
|
||||
- 拉取:应用平台也可以定时轮询 Compute API,避免回调失败导致状态丢失。
|
||||
- 状态同步:默认由应用平台定时轮询 Compute API,拉取任务状态、指标摘要和产物索引。
|
||||
- 回调策略:第一阶段关闭算力侧回调,避免算力服务器访问应用服务器,减少双向网络策略开通。
|
||||
|
||||
文件互通:
|
||||
|
||||
@@ -196,9 +196,46 @@ GPU 算力服务器部署:
|
||||
### 5.6 风险
|
||||
|
||||
- 文件传输链路比单机部署复杂。
|
||||
- 需要处理跨服务器网络失败、回调失败、任务状态对账。
|
||||
- 需要处理跨服务器网络失败、轮询延迟、任务状态对账。
|
||||
- 需要明确模型、数据集、产物在应用侧和算力侧的索引关系。
|
||||
|
||||
### 5.7 多算力节点部署约定
|
||||
|
||||
多算力节点阶段仍然按“单机多 GPU 节点”部署,每台 GPU 服务器都是一个独立算力节点。每个参与调度的节点都必须部署:
|
||||
|
||||
- Compute API。
|
||||
- Compute Agent。
|
||||
- File Gateway。
|
||||
- LLaMA-Factory 宿主机目录和训练依赖。
|
||||
- CUDA、NVIDIA Driver、NCCL、PyTorch。
|
||||
- 本地数据盘 `/data/yg-ft`。
|
||||
- 本地日志和训练产物目录。
|
||||
|
||||
网络策略保持单向:
|
||||
|
||||
```text
|
||||
应用服务器 -> 算力节点 A Compute API/File Gateway
|
||||
应用服务器 -> 算力节点 B Compute API/File Gateway
|
||||
应用服务器 -> 算力节点 C Compute API/File Gateway
|
||||
```
|
||||
|
||||
默认不要求:
|
||||
|
||||
```text
|
||||
算力节点 -> 应用服务器
|
||||
算力节点 A -> 算力节点 B
|
||||
```
|
||||
|
||||
多节点任务调度由应用平台统一完成。应用平台从 `compute_nodes` 读取节点地址、权重、标签、启用状态、维护状态和健康检查结果;从 `resource_replicas` 判断目标节点是否已有所需数据集/模型副本;缺失时创建 `resource_sync_jobs`,通过目标节点 File Gateway 同步资源。
|
||||
|
||||
调度策略:
|
||||
|
||||
- 默认自动调度,按节点健康、标签、GPU 空闲、队列长度、节点权重和资源副本命中率排序。
|
||||
- 支持管理员/高级用户手动指定节点或 GPU。
|
||||
- `disabled` 节点不参与调度。
|
||||
- `draining` 节点不接收新任务,但允许已有任务跑完。
|
||||
- `maintenance/offline` 节点只允许查看和清理,不允许提交训练任务。
|
||||
|
||||
## 6. Compute API 接入标准
|
||||
|
||||
为预留其他训练平台,应用平台只依赖统一算力接口,不直接依赖 LLaMA-Factory 命令。
|
||||
@@ -234,13 +271,16 @@ compute/engines/openrlhf/
|
||||
|
||||
```env
|
||||
APP_ENV=prod
|
||||
API_PREFIX=/api
|
||||
DATABASE_URL=postgresql+asyncpg://yg_ft:***@postgres:5432/yg_ft
|
||||
MODELTF_ROUTE_PREFIX=/modelTF
|
||||
DATABASE_URL=postgresql+psycopg://yg_ft:***@postgres:5432/yg_ft
|
||||
REDIS_URL=redis://redis:6379/0
|
||||
LOG_DIR=/opt/yg-ft/logs/backend
|
||||
COMPUTE_API_BASE_URL=https://compute.internal:9100
|
||||
COMPUTE_API_BASE_URL=https://compute.internal:19100
|
||||
COMPUTE_SERVICE_TOKEN=***
|
||||
FILE_GATEWAY_BASE_URL=https://compute.internal:9101
|
||||
FILE_GATEWAY_BASE_URL=https://compute.internal:19101
|
||||
COMPUTE_STATUS_SYNC_MODE=polling
|
||||
COMPUTE_POLL_INTERVAL_SECONDS=10
|
||||
COMPUTE_POLL_BATCH_SIZE=100
|
||||
```
|
||||
|
||||
算力平台:
|
||||
@@ -248,11 +288,11 @@ FILE_GATEWAY_BASE_URL=https://compute.internal:9101
|
||||
```env
|
||||
COMPUTE_ENV=prod
|
||||
COMPUTE_HOST_ID=gpu-node-01
|
||||
COMPUTE_API_PORT=9100
|
||||
FILE_GATEWAY_PORT=9101
|
||||
APP_CALLBACK_BASE_URL=https://app.internal/api/v1/compute/callbacks
|
||||
APP_SERVICE_TOKEN=***
|
||||
LLAMA_FACTORY_HOME=/opt/LLaMA-Factory
|
||||
COMPUTE_API_PORT=19100
|
||||
FILE_GATEWAY_PORT=19101
|
||||
COMPUTE_SERVICE_TOKEN=***
|
||||
ENABLE_APP_CALLBACK=false
|
||||
LLAMA_FACTORY_HOME=/app/LLaMA-Factory
|
||||
YG_FT_DATA_ROOT=/data/yg-ft
|
||||
LOG_DIR=/opt/yg-ft/logs/compute
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
@@ -282,12 +322,12 @@ CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
|
||||
## 10. 部署检查清单
|
||||
|
||||
- PostgreSQL 已初始化 `docs/postgres-schema.sql`。
|
||||
- PostgreSQL 已初始化当前运行脚本 `backend/app/db/sql/001_platform_runtime.sql`;`docs/postgres-schema.sql` 作为目标架构设计,后续通过迁移体系逐步收敛。
|
||||
- Redis 可连通。
|
||||
- 后端 `GET /api/v1/health` 正常。
|
||||
- 后端 `GET /modelTF/health` 正常。
|
||||
- Compute API 健康检查正常。
|
||||
- Compute Agent 能识别 GPU、显存、CUDA 版本。
|
||||
- LLaMA-Factory 能在命令行完成最小训练样例。
|
||||
- LLaMA-Factory 能在命令行完成最小训练作业。
|
||||
- 应用平台能提交训练任务到 Compute API。
|
||||
- 任务状态能从算力平台同步回应用平台。
|
||||
- 数据集上传、离线导入、产物下载路径权限正确。
|
||||
@@ -295,11 +335,72 @@ CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
- ERROR 日志能触发告警。
|
||||
- 日志、数据集、模型、产物所在磁盘容量有监控和告警。
|
||||
|
||||
## 11. 仍需确认的问题
|
||||
## 11. Docker Compose 文件规划
|
||||
|
||||
当前项目按应用服务器和算力服务器拆分了两套 Docker 部署文件,均采用代码外挂方式运行:
|
||||
|
||||
```text
|
||||
docker/
|
||||
app/
|
||||
Dockerfile.backend # Backend API 运行时镜像,代码通过 volume 挂载到 /app
|
||||
Dockerfile.frontend # Nginx 前端运行时镜像,frontend/dist 通过 volume 挂载
|
||||
docker-compose.yml # 应用服务器:frontend、backend-api、postgres、redis
|
||||
.env.example
|
||||
compute/
|
||||
Dockerfile.compute # CUDA + Python + Compute API 运行时镜像
|
||||
docker-compose.yml # 算力服务器:compute-api,预留 agent/file gateway 拆分
|
||||
.env.example
|
||||
```
|
||||
|
||||
项目根目录不再保留 `Dockerfile` 和 `docker-compose.yml`,避免与拆分部署入口混淆。
|
||||
|
||||
应用服务器启动:
|
||||
|
||||
```bash
|
||||
cd docker/app
|
||||
cp .env.example .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
算力服务器启动:
|
||||
|
||||
```bash
|
||||
cd docker/compute
|
||||
cp .env.example .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
应用服务器与算力服务器独立部署时,需要在 `docker/app/.env` 中配置:
|
||||
|
||||
```env
|
||||
COMPUTE_API_BASE_URL=http://<compute-server-ip>:19100
|
||||
FILE_GATEWAY_BASE_URL=http://<compute-server-ip>:19101
|
||||
COMPUTE_SERVICE_TOKEN=change_me
|
||||
```
|
||||
|
||||
这些地址在当前 Docker 阶段通过环境变量动态配置。后续多算力节点阶段建议升级为数据库配置,由应用平台从 `compute_nodes` 表读取节点地址、权重、标签、健康状态和启用状态,并在“算力节点管理”页面维护。
|
||||
|
||||
多节点后,每台算力服务器各自进入 `docker/compute` 启动一套算力服务,并在应用平台中登记为一条 `compute_nodes` 记录:
|
||||
|
||||
```text
|
||||
gpu-node-01 -> http://10.10.20.31:19100 / http://10.10.20.31:19101
|
||||
gpu-node-02 -> http://10.10.20.32:19100 / http://10.10.20.32:19101
|
||||
gpu-node-03 -> http://10.10.20.33:19100 / http://10.10.20.33:19101
|
||||
```
|
||||
|
||||
算力服务器需要在 `docker/compute/.env` 中配置:
|
||||
|
||||
```env
|
||||
ENABLE_APP_CALLBACK=false
|
||||
COMPUTE_SERVICE_TOKEN=change_me
|
||||
YG_FT_DATA_ROOT_HOST=/data/yg-ft
|
||||
```
|
||||
|
||||
## 12. 仍需确认的问题
|
||||
|
||||
- 生产环境是否已有统一 ELK/OpenSearch、Filebeat/Vector 标准配置。
|
||||
- 数据库和 Redis 是否由企业基础设施统一提供,还是由项目自行部署。
|
||||
- PostgreSQL/Redis 开发阶段采用项目自带部署;生产阶段是否切换企业统一基础设施,以及对应 SLA 仍需确认。
|
||||
- 是否需要 PostgreSQL 主备、备份恢复、审计日志长期归档的明确 SLA。
|
||||
- 大文件上传是否需要断点续传、限速、病毒扫描或 DLP 检测。
|
||||
- 应用服务器与算力服务器之间是否允许双向访问,还是只能应用侧主动访问算力侧。
|
||||
- 是否需要未来支持多台 GPU 节点调度;如果需要,Compute API 需要提前设计节点注册和调度策略。
|
||||
- 应用服务器与算力服务器默认只开通应用侧主动访问算力侧;如后续需要实时回调,再单独评估双向网络策略。
|
||||
- 多算力节点已按单机多 GPU 节点扩展设计;仍需确认是否需要节点组、租户绑定节点、同步限速和资源副本清理审批。
|
||||
|
||||
145
docs/first-version-development-plan.md
Normal file
145
docs/first-version-development-plan.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# 当前系统主链路开发计划
|
||||
|
||||
> 说明:本计划描述当前正在开发的系统能力。代码、接口和 SQL 均按后续生产演进基线维护,不以一次性演示、静态 Mock 或样例数据作为开发准则。联调辅助能力必须显式配置启用,并不得成为默认运行路径。
|
||||
|
||||
## 1. 阶段目标
|
||||
|
||||
当前阶段需要完成模型微调平台的主链路工程基础:
|
||||
|
||||
```text
|
||||
登录
|
||||
-> 模型管理
|
||||
-> 数据集管理
|
||||
-> 创建微调任务
|
||||
-> 调度算力节点与 GPU
|
||||
-> 检查并同步模型/数据集资源
|
||||
-> 启动训练任务
|
||||
-> 轮询任务状态、GPU 占用、训练日志、loss 曲线
|
||||
-> 训练完成后登记训练产物
|
||||
```
|
||||
|
||||
该阶段是正式系统的第一批可运行能力,不再初始化业务样例数据。系统只允许初始化内置管理员/运维账号,模型、数据集、算力节点、GPU、训练任务和资源副本必须通过页面、接口、算力 Agent 扫描或正式导入流程产生。
|
||||
|
||||
## 2. 运行模式
|
||||
|
||||
| 模式 | 说明 | 当前要求 |
|
||||
| --- | --- | --- |
|
||||
| `real` | 面向真实部署,等待 Compute API、Agent、File Gateway 和 LLaMA-Factory 执行器回写状态 | 默认模式 |
|
||||
| `simulator` | 仅用于隔离联调,无真实 GPU 时临时推进任务状态、GPU 状态和训练日志 | 必须显式开启,不得用于生产基线 |
|
||||
|
||||
后端默认 `COMPUTE_MODE=real`。在该模式下,任务状态不再按时间自动推进,必须由后续真实算力同步逻辑更新。算力服务默认 `COMPUTE_EXECUTION_MODE=real`,真实训练执行器未完成前,创建训练作业会返回明确的未实现错误,避免误认为已经完成生产训练能力。
|
||||
|
||||
训练相关能力必须沉淀在 `compute/engines/` 适配层,不允许在应用平台后端直接拼接或执行训练命令。
|
||||
|
||||
## 3. 当前开发范围
|
||||
|
||||
### 3.1 应用平台后端
|
||||
|
||||
对应目录:
|
||||
|
||||
```text
|
||||
backend/app/
|
||||
api/v1/endpoints/platform.py
|
||||
core/
|
||||
db/
|
||||
```
|
||||
|
||||
已建立能力:
|
||||
- 统一 API 响应结构 `{ code, message, data }`。
|
||||
- PostgreSQL 运行表初始化,当前执行脚本位于 `backend/app/db/sql/001_platform_runtime.sql`。
|
||||
- 内置管理员账号初始化,业务数据不再自动写入样例记录。
|
||||
- 登录、当前用户、用户列表与权限页面接口。
|
||||
- 模型管理、训练产物列表、权重合并任务入口。
|
||||
- 数据集管理、文件上传、预览、版本管理和下载。
|
||||
- 微调任务创建、启动、停止、删除、进度查询、checkpoint 查询。
|
||||
- 系统健康指标、系统信息、训练日志、系统日志接口。
|
||||
- 算力节点、GPU、队列、资源副本、资源同步任务接口。
|
||||
|
||||
待继续开发:
|
||||
- 接入正式 ORM/Repository/Service 分层和 Alembic 迁移。
|
||||
- 将任务状态更新改为应用侧定时轮询 Compute API/File Gateway 后落库。
|
||||
- 完成项目/模型/数据集级权限隔离校验。
|
||||
- 完成审批流、审计留存、配额、资源申请和多租户上下文。
|
||||
- 增加正式异常码、接口鉴权、中间件、幂等控制和分页规范。
|
||||
|
||||
### 3.2 算力平台服务
|
||||
|
||||
对应目录:
|
||||
|
||||
```text
|
||||
compute/
|
||||
api/main.py
|
||||
agent/
|
||||
engines/llama_factory/
|
||||
file_gateway/
|
||||
```
|
||||
|
||||
已建立能力:
|
||||
- `/modelTF/health` 与 `/modelTF/v1/compute/health` 节点健康检查。
|
||||
- LLaMA-Factory 参数校验、命令生成和训练日志指标解析。
|
||||
- Compute API 作业、GPU、文件网关接口壳。
|
||||
- 显式 `simulator` 模式下的内存状态机,用于隔离联调。
|
||||
|
||||
待继续开发:
|
||||
- 真实 GPU 发现:接入 `nvidia-smi` 或 NVML。
|
||||
- GPU 锁定与释放:一张 GPU 同一时间只分配给一个训练或推理任务。
|
||||
- LLaMA-Factory 真实执行器:生成 YAML/命令、启动进程、停止进程、采集 PID。
|
||||
- 训练日志采集:读取宿主机挂载日志文件,解析 loss、learning rate、epoch 等指标。
|
||||
- Checkpoint/adapter/merged model 扫描与产物登记。
|
||||
- File Gateway:本地磁盘文件上传、下载、校验、导入和跨节点资源同步。
|
||||
|
||||
### 3.3 前端页面
|
||||
|
||||
已接入页面:
|
||||
- `/login`:登录接口。
|
||||
- `/model-manage`:模型列表、模型来源、训练产物。
|
||||
- `/dataset`、`/dataset/:id/preview`:数据集列表、预览、版本。
|
||||
- `/fine-tune`、`/fine-tune/create`:微调任务创建、启动、状态轮询。
|
||||
- `/training-log/:id`:训练日志和 loss 曲线。
|
||||
- `/hardware`:平台 GPU 与系统性能。
|
||||
- `/compute`:算力节点、GPU、队列、资源副本。
|
||||
|
||||
前端 Mock 默认关闭。仅在隔离前端开发时可设置 `VITE_ENABLE_MOCK=true`,真实联调和后续生产演进均以 `/modelTF` 后端接口为准。
|
||||
|
||||
待继续开发:
|
||||
- 补齐多租户、项目管理、审批中心、审计中心、配额管理页面。
|
||||
- 完成算力节点管理表单,包括节点地址、权重、标签、启用状态和健康检查结果。
|
||||
- 完成模型/数据集导入页面,支持本地路径扫描和归属项目选择。
|
||||
- 推理服务页面需接入真实后端任务接口,移除页面内本地假对话路径。
|
||||
|
||||
### 3.4 数据库
|
||||
|
||||
当前运行 SQL:
|
||||
|
||||
```text
|
||||
backend/app/db/sql/001_platform_runtime.sql
|
||||
```
|
||||
|
||||
架构目标 SQL:
|
||||
|
||||
```text
|
||||
docs/postgres-schema.sql
|
||||
```
|
||||
|
||||
当前运行 SQL 用于支持已开发接口落库;架构目标 SQL 包含用户中心、多租户、项目隔离、审批、审计、配额、评测等完整模型。后续需要通过 Alembic 将二者收敛为统一迁移体系,生产升级只走迁移脚本,不依赖手工改表。
|
||||
|
||||
## 4. 验收标准
|
||||
|
||||
- 启动后端必须连接 PostgreSQL,不允许回退到 SQLite。
|
||||
- 后端启动只初始化系统内置账号,不初始化模型、数据集、算力节点、GPU、训练任务等业务样例数据。
|
||||
- 前端默认请求真实 `/modelTF` 接口,除非显式设置 `VITE_ENABLE_MOCK=true`。
|
||||
- 默认 `real` 模式下任务状态不自动伪造完成,必须等待真实算力同步。
|
||||
- 显式 `simulator` 模式只能用于隔离联调,部署文档必须标注不得用于生产。
|
||||
- 登录后可以进入主界面,并可通过页面/API 创建真实业务记录。
|
||||
- 代码、接口路由、配置项、数据库表名不得使用 `demo` 命名。
|
||||
|
||||
## 5. 后续开发计划
|
||||
|
||||
| 阶段 | 重点 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| 阶段 1 | 数据库迁移体系 | 将当前运行 SQL 与架构 SQL 收敛到 Alembic 迁移 |
|
||||
| 阶段 2 | 后端领域分层 | 拆分用户、模型、数据集、训练、算力、审计等模块 |
|
||||
| 阶段 3 | 真实 Compute Agent | GPU 发现、资源锁定、进程管理、日志采集 |
|
||||
| 阶段 4 | LLaMA-Factory 训练执行 | YAML/命令生成、进程启动/停止、checkpoint 和 adapter 扫描 |
|
||||
| 阶段 5 | 企业治理 | 多租户、项目隔离、审批流、审计留存、配额和资源申请 |
|
||||
| 阶段 6 | 多算力节点调度 | 基于 `compute_nodes`、标签、权重、资源副本和节点健康实现调度策略 |
|
||||
98
docs/menu-functional-requirements.md
Normal file
98
docs/menu-functional-requirements.md
Normal file
@@ -0,0 +1,98 @@
|
||||
# 菜单与功能需求总览
|
||||
|
||||
> 本文根据当前前端侧边栏、路由、需求文档、接口文档、部署文档和 SQL 脚本整理。当前代码和 SQL 均按正式系统开发基线维护;Mock、Simulator 只能作为显式联调能力,不作为默认开发准则。
|
||||
|
||||
## 1. 菜单分层
|
||||
|
||||
### 1.1 当前侧边栏菜单
|
||||
|
||||
| 一级分组 | 菜单 | 路由 | 权限码 | 当前状态 | 主要功能 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 服务看板 | 服务看板 | `/dashboard` | `dashboard` | 已有页面,接口需继续完善 | 总览指标、服务状态、训练统计、最近任务、健康入口 |
|
||||
| 模型服务 | 模型训练 | `/fine-tune` | `fine-tune` | 已接入主链路 | 训练任务列表、创建训练、启动/停止、进度、训练日志、checkpoint |
|
||||
| 模型服务 | 模型评测 | `/model-eval` | `model-eval` | 前端页面已有,后端待完整实现 | 评测任务、评测维度、样本评分、综合结果 |
|
||||
| 模型服务 | 模型推理 | `/model-inference` | `model-inference` | 前端页面已有,后端待完整实现 | 推理任务、模型加载、单模型对话、模型对比入口 |
|
||||
| 模型服务 | 模型管理 | `/model-manage` | `model-manage` | 已接入主链路 | 基座模型登记、本地/API 模型、训练产物、权重合并、模型导出 |
|
||||
| 数据治理 | 数据集管理 | `/dataset` | `dataset` | 已接入主链路 | 数据集列表、上传、预览、在线编辑、版本、下载、删除审批入口 |
|
||||
| 数据治理 | 数据处理 | `/data-process` | `data-process` | 前端页面已有,后端待完整实现 | 文档上传、切片预览、LLM 生成、结果编辑、发布数据集 |
|
||||
| 其他工具 | 数据类型转换 | `/data-convert` | `data-convert` | 前端页面已有,后端待实现 | JSON/JSONL/Markdown 等格式转换任务 |
|
||||
| 算力资源 | 算力节点 | `/compute` | `compute` | 已接入节点管理接口 | 节点地址、权重、标签、启用状态、GPU、队列、资源副本 |
|
||||
| 系统设置 | 用户设置 | `/user-settings` | `user-settings` | 已接入基础用户接口 | 用户列表、创建用户、启停、页面权限 |
|
||||
| 系统设置 | 平台性能 | `/hardware` | `hardware` | 已有接口,需接真实采集 | CPU、内存、磁盘、GPU、进程、网络监控 |
|
||||
| 系统设置 | 查看日志 | `/logs` | `logs` | 已有接口,需接真实日志文件 | 后端日志、error 日志、训练日志索引、日志内容查看 |
|
||||
|
||||
### 1.2 当前二级和隐藏路由
|
||||
|
||||
| 页面 | 路由 | 归属菜单 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 登录 | `/login` | 独立入口 | 登录后进入主界面 |
|
||||
| 使用文档 | `/guide` | 独立入口 | 当前系统使用说明 |
|
||||
| 创建训练任务 | `/fine-tune/create` | 模型训练 | 训练参数、模型/数据集/GPU 选择 |
|
||||
| 训练日志 | `/training-log/:id` | 模型训练 | 日志、指标、checkpoint、任务概览 |
|
||||
| 新建评测 | `/model-eval/create` | 模型评测 | 模型、数据集、维度、GPU 选择 |
|
||||
| 评测详情 | `/model-eval/:id` | 模型评测 | 维度汇总、样本结果、人工复核预留 |
|
||||
| 评测维度创建/编辑 | `/model-eval/dimension/create`、`/model-eval/dimension/:id/edit` | 模型评测 | 评测规则、Prompt、评分器配置 |
|
||||
| 新建推理 | `/model-inference/create` | 模型推理 | 推理任务和模型加载配置 |
|
||||
| 模型对话 | `/model-inference/chat/:id` | 模型推理 | 单模型对话 |
|
||||
| 模型对比 | `/model-compare/chat/:id`、`/model-compare/result` | 模型推理 | 多模型对比和结果页 |
|
||||
| 添加/编辑模型 | `/model-manage/create`、`/model-manage/:id/edit` | 模型管理 | 模型登记、用途、来源、路径/API 配置 |
|
||||
| 合并权重 | `/model-manage/merge` | 模型管理 | LoRA/Adapter 合并任务 |
|
||||
| 数据处理创建/详情 | `/data-process/create`、`/data-process/:id` | 数据处理 | 数据处理向导和任务详情 |
|
||||
| 数据集创建/编辑/预览 | `/dataset/create`、`/dataset/:id/edit`、`/dataset/:id/preview` | 数据集管理 | 数据集元数据、文件、版本与内容 |
|
||||
| 自定义工具 | `/tools`、`/tools/create`、`/tools/:id/edit` | 规划入口 | 路由存在,当前侧边栏未展示,后续可归入“其他工具” |
|
||||
| 算力子页 | `/compute/gpus`、`/compute/queue`、`/compute/nodes` | 算力节点 | 当前可作为页签或深链 |
|
||||
| 创建用户/权限设置 | `/user-settings/create`、`/user-settings/:id/permission` | 用户设置 | 用户创建和页面权限 |
|
||||
| 无权限页 | `/permission-denied` | 系统页 | 路由守卫无权限跳转 |
|
||||
|
||||
### 1.3 企业治理待补菜单
|
||||
|
||||
| 建议菜单分组 | 菜单 | 建议路由 | 优先级 | 必要性 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 组织与项目 | 租户管理 | `/tenants`、`/tenants/:id` | P0 | 多租户隔离、配额、留存策略入口 |
|
||||
| 组织与项目 | 项目空间 | `/projects`、`/projects/:id`、`/projects/:id/members` | P0 | 项目级模型/数据集/任务隔离 |
|
||||
| 组织与项目 | 资源授权 | `/projects/:id/permissions` 或资源详情弹窗 | P0 | 模型/数据集/任务级 ACL |
|
||||
| 治理中心 | 审批中心 | `/approvals`、`/approvals/:id` | P0 | 删除、发布、导出、停止他人任务等高风险动作 |
|
||||
| 治理中心 | 审批设置 | `/approval-settings` | P1 | 审批模板、审批人规则、超时策略 |
|
||||
| 治理中心 | 审计中心 | `/audit-logs`、`/login-logs`、`/download-logs` | P1 | 操作审计、登录审计、下载审计、导出 |
|
||||
| 运维中心 | 存储管理 | `/storage` | P1 | 本地磁盘占用、临时文件、checkpoint 清理、留存 |
|
||||
| 运维中心 | 训练引擎管理 | `/training-engines` | P2 | LLaMA-Factory 和后续引擎能力 schema、健康检查 |
|
||||
| 模型服务 | 模型服务治理 | `/model-services`、`/model-services/:id` | P1 | 测试/生产服务发布、调用统计、下线审批 |
|
||||
|
||||
## 2. 菜单对应接口和数据库
|
||||
|
||||
| 菜单/模块 | 主要接口 | 当前运行 SQL | 目标 SQL |
|
||||
| --- | --- | --- | --- |
|
||||
| 登录、用户设置 | `/modelTF/login`、`/modelTF/me`、`/modelTF/users` | `users` | `users`、`login_sessions`、`permissions`、`role_permissions`、`user_permission_overrides` |
|
||||
| 服务看板 | `/modelTF/dashboard/overview`、`/modelTF/health` | 复用模型/数据集/任务/算力表 | `system_metric_snapshots`、`web_logs`、各业务表聚合 |
|
||||
| 模型管理 | `/modelTF/model-manage`、`/modelTF/model-manage/trained-models`、`/modelTF/model-manage/merge` | `models`、`trained_models` | `models`、`trained_models`、`storage_objects`、`local_import_jobs`、`resource_acl` |
|
||||
| 数据集管理 | `/modelTF/dataset-manage`、`/modelTF/dataset-manage/upload/{id}`、`/preview`、`/versions` | `datasets`、`dataset_files` | `datasets`、`dataset_files`、`dataset_file_versions`、`dataset_records`、`storage_objects` |
|
||||
| 模型训练 | `/modelTF/fine-tune`、`/start`、`/progress`、`/checkpoints` | `fine_tune_tasks`、`trained_models` | `fine_tune_tasks`、`fine_tune_metrics`、`fine_tune_checkpoints`、`compute_jobs`、`gpu_allocations` |
|
||||
| 训练日志 | `/modelTF/training-log-files`、`/modelTF/training-log-content` | 由任务表生成索引 | 日志文件元数据、`fine_tune_metrics`、`audit_logs` |
|
||||
| 算力节点 | `/modelTF/compute/nodes`、`/compute/gpus`、`/compute/queue`、`/compute/nodes/{id}/replicas` | `compute_nodes`、`gpus`、`resource_replicas`、`resource_sync_jobs` | `compute_nodes`、`gpu_devices`、`compute_node_engines`、`compute_jobs`、`resource_replicas`、`resource_sync_jobs` |
|
||||
| 平台性能 | `/modelTF/system-info`、`/modelTF/compute/gpus` | `gpus`、任务表 | `system_metric_snapshots`、`gpu_devices`、`compute_jobs` |
|
||||
| 查看日志 | `/modelTF/log-files`、`/modelTF/log-content`、`/modelTF/web-log` | 文件日志 | `web_logs`、`audit_logs`,大日志进入日志平台 |
|
||||
| 模型评测 | `/modelTF/model-eval`、`/modelTF/dimension` | 当前运行 SQL 未覆盖 | `eval_tasks`、`eval_dimensions`、`eval_sample_results`、`eval_dimension_summaries` |
|
||||
| 模型推理/对比 | `/modelTF/model-compare`、`/modelTF/model-chat/*` | 当前运行 SQL 未覆盖 | `inference_tasks`、`inference_task_models`、`chat_sessions`、`chat_messages` |
|
||||
| 数据处理 | `/modelTF/data-process/*` | 当前运行 SQL 未覆盖 | `data_process_tasks`、`data_process_source_files`、`data_process_preview_items`、`data_process_results` |
|
||||
| 数据转换/自定义工具 | `/modelTF/data-convert/jobs`、`/modelTF/tools` | 当前运行 SQL 未覆盖 | `data_convert_jobs`、`custom_tools` |
|
||||
| 租户/项目/资源授权 | `/modelTF/tenants`、`/modelTF/projects`、`/modelTF/resources/{type}/{id}/acl` | 当前运行 SQL 未覆盖 | `tenants`、`tenant_users`、`projects`、`project_members`、`resource_acl` |
|
||||
| 审批/审计/留存/配额 | `/modelTF/approvals`、`/modelTF/audit-logs`、`/modelTF/retention-policies`、`/modelTF/quotas/usage` | 当前运行 SQL 未覆盖 | `approval_templates`、`approval_instances`、`approval_steps`、`audit_logs`、`retention_policies`、`quotas`、`quota_usage` |
|
||||
|
||||
## 3. 文档和脚本检查结论
|
||||
|
||||
| 对象 | 当前结论 | 本次补充 |
|
||||
| --- | --- | --- |
|
||||
| 需求文档 | `docs/platform-architecture-requirements.md` 和 `docs/system-development-plan.md` 已覆盖多租户、项目隔离、审批、审计、多算力节点、应用/算力分离部署;缺少一份按当前菜单组织的总览 | 新增本文作为菜单和功能需求总览 |
|
||||
| 接口文档 | `docs/backend-api-design.md` 已统一 `/modelTF`,并已有页面/接口映射;需要明确引用菜单总览,避免开发只看接口不看页面入口 | 在接口文档增加菜单总览引用 |
|
||||
| 开发计划 | `docs/system-development-plan.md` 已按工作包列出页面、接口和 DB;需要把本文作为任务认领入口 | 在开发计划增加菜单总览引用 |
|
||||
| 部署文档 | `docs/deployment-plan.md`、`docker/README.md` 已覆盖应用/算力分离、单机多 GPU、本地磁盘、真实模式默认、Docker 拆分 | 暂无新增部署配置要求 |
|
||||
| 目标 SQL | `docs/postgres-schema.sql` 覆盖完整目标模型,包含用户、权限、多租户、项目、审批、审计、模型、数据集、训练、评测、推理、算力、存储、导入、服务治理 | 暂不需要新增目标表 |
|
||||
| 当前运行 SQL | `backend/app/db/sql/001_platform_runtime.sql` 只覆盖已接入运行接口的最小表集 | 后续每实现一个 P0/P1 菜单模块,应同步补运行 SQL 或迁移脚本;不能再以样例数据补功能 |
|
||||
|
||||
## 4. 后续补充原则
|
||||
|
||||
- 新增侧边栏菜单时,必须同步补齐:路由、权限码、接口文档、DB 表/迁移、审计动作、部署依赖。
|
||||
- 新增后端接口时,必须在 `docs/backend-api-design.md` 标注对应页面/功能模块。
|
||||
- 新增表结构时,目标模型写入 `docs/postgres-schema.sql`,当前可执行落库写入 `backend/app/db/sql/` 或 Alembic 迁移。
|
||||
- 与训练、评测、推理、数据处理相关的异步任务必须落库,不能依赖前端本地状态。
|
||||
- 与算力相关的功能默认走真实模式;Simulator 只能显式开启用于隔离联调。
|
||||
@@ -80,20 +80,20 @@ flowchart LR
|
||||
|
||||
- 应用平台调用算力平台必须携带 `X-Service-Token` 或 mTLS 证书。
|
||||
- 算力平台不信任前端用户身份,只信任应用平台下发的租户、项目、任务和资源上下文。
|
||||
- 所有任务回调必须带签名,避免伪造状态。
|
||||
- 第一阶段不默认启用算力侧回调,应用平台通过定时轮询 Compute API 同步任务状态;如未来启用回调,必须带签名,避免伪造状态。
|
||||
|
||||
核心通信接口:
|
||||
|
||||
| 方向 | 接口 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| 应用 -> 算力 | `POST /compute/jobs` | 创建训练/评测/数据处理/推理任务 |
|
||||
| 应用 -> 算力 | `POST /compute/jobs/{id}/stop` | 停止任务 |
|
||||
| 应用 -> 算力 | `GET /compute/jobs/{id}` | 查询任务状态 |
|
||||
| 应用 -> 算力 | `GET /compute/jobs/{id}/logs` | 拉取日志 |
|
||||
| 应用 -> 算力 | `GET /compute/resources/gpus` | 查询 GPU 状态 |
|
||||
| 应用 -> 算力 | `POST /compute/files/upload` | 上传文件到算力本地磁盘 |
|
||||
| 应用 -> 算力 | `GET /compute/files/{object_id}/download` | 下载文件 |
|
||||
| 算力 -> 应用 | `POST /api/internal/compute-callbacks/jobs` | 回调任务状态、指标、产物 |
|
||||
| 应用 -> 算力 | `POST /modelTF/compute/jobs` | 创建训练/评测/数据处理/推理任务 |
|
||||
| 应用 -> 算力 | `POST /modelTF/compute/jobs/{id}/stop` | 停止任务 |
|
||||
| 应用 -> 算力 | `GET /modelTF/compute/jobs/{id}` | 查询任务状态 |
|
||||
| 应用 -> 算力 | `GET /modelTF/compute/jobs/{id}/logs` | 拉取日志 |
|
||||
| 应用 -> 算力 | `GET /modelTF/compute/resources/gpus` | 查询 GPU 状态 |
|
||||
| 应用 -> 算力 | `POST /modelTF/compute/files/upload` | 上传文件到算力本地磁盘 |
|
||||
| 应用 -> 算力 | `GET /modelTF/compute/files/{object_id}/download` | 下载文件 |
|
||||
| 应用 -> 算力 | `GET /modelTF/compute/jobs?status=running` | 定时轮询任务状态、指标、产物索引 |
|
||||
|
||||
## 3. 本地磁盘存储设计
|
||||
|
||||
@@ -149,6 +149,8 @@ flowchart LR
|
||||
|
||||
## 4. 单机多 GPU 资源调度
|
||||
|
||||
第一版可以先以单个算力节点跑通主链路,但数据模型、接口和页面需要按多算力节点预留。多节点阶段仍然采用“每台 GPU 服务器 = 一个单机多 GPU 节点”的模式,不引入 Kubernetes。
|
||||
|
||||
### 4.1 GPU 资源模型
|
||||
|
||||
每张 GPU 需要记录:
|
||||
@@ -183,6 +185,25 @@ flowchart LR
|
||||
- 支持任务队列优先级:`low`、`normal`、`high`、`urgent`。
|
||||
- 高优任务是否可抢占低优任务,需要审批或管理员权限。
|
||||
|
||||
### 4.4 多算力节点升级策略
|
||||
|
||||
多算力节点阶段的推荐决策:
|
||||
|
||||
- 每个可执行训练任务的 GPU 节点都部署 `Compute API`、`Compute Agent`、`File Gateway`、LLaMA-Factory、CUDA/PyTorch 训练环境和本地数据盘。
|
||||
- 应用服务器可以主动访问所有算力节点的 `Compute API/File Gateway`。
|
||||
- 算力节点之间默认不互相访问,不做节点间点对点同步;所有调度、状态同步和资源分发由应用平台统一编排。
|
||||
- 长期坚持每台算力服务器本地磁盘,因此需要 `resource_replicas` 记录数据集、基座模型、checkpoint、adapter、导出模型在哪些节点已有本地副本。
|
||||
- 调度前必须检查目标节点是否已有模型和数据集副本;缺失时由应用平台通过目标节点 File Gateway 创建同步任务,完成后再启动训练。
|
||||
- 调度支持自动和手动两种模式:普通用户默认自动调度,管理员/高级用户可手动指定节点、GPU、标签或节点组。
|
||||
|
||||
多节点自动调度建议:
|
||||
|
||||
1. 过滤 `enabled = true` 且 `scheduler_status = online` 的节点。
|
||||
2. 按训练引擎、GPU 型号、显存、节点标签、租户/项目配额过滤。
|
||||
3. 优先选择已存在所需模型/数据集副本的节点,减少跨节点复制。
|
||||
4. 同等条件下按空闲 GPU、队列长度、节点权重和最近健康检查排序。
|
||||
5. `draining` 节点不接收新任务,但允许已有任务完成。
|
||||
|
||||
## 5. 训练引擎接入标准
|
||||
|
||||
### 5.1 引擎抽象
|
||||
@@ -421,6 +442,9 @@ LLaMA-Factory 适配器负责:
|
||||
- 队列中的任务。
|
||||
- 资源配额:租户/项目/用户维度。
|
||||
- 算力节点 Agent 状态。
|
||||
- 算力节点新增/编辑、连接测试、启用/禁用、维护模式。
|
||||
- 节点权重、标签、训练引擎版本、LLaMA-Factory 健康状态。
|
||||
- 节点本地资源副本:数据集、模型、checkpoint、adapter 和导出模型缓存。
|
||||
- 训练引擎健康状态。
|
||||
|
||||
### 8.5 文件与存储管理
|
||||
@@ -532,54 +556,54 @@ LLaMA-Factory 适配器负责:
|
||||
|
||||
租户:
|
||||
|
||||
- `GET /api/tenants`
|
||||
- `POST /api/tenants`
|
||||
- `GET /api/tenants/{id}`
|
||||
- `PUT /api/tenants/{id}`
|
||||
- `PUT /api/tenants/{id}/quota`
|
||||
- `PUT /api/tenants/{id}/retention-policy`
|
||||
- `GET /modelTF/tenants`
|
||||
- `POST /modelTF/tenants`
|
||||
- `GET /modelTF/tenants/{id}`
|
||||
- `PUT /modelTF/tenants/{id}`
|
||||
- `PUT /modelTF/tenants/{id}/quota`
|
||||
- `PUT /modelTF/tenants/{id}/retention-policy`
|
||||
|
||||
项目:
|
||||
|
||||
- `GET /api/projects`
|
||||
- `POST /api/projects`
|
||||
- `GET /api/projects/{id}`
|
||||
- `PUT /api/projects/{id}`
|
||||
- `POST /api/projects/{id}/archive`
|
||||
- `GET /api/projects/{id}/members`
|
||||
- `POST /api/projects/{id}/members`
|
||||
- `PUT /api/projects/{id}/members/{user_id}`
|
||||
- `DELETE /api/projects/{id}/members/{user_id}`
|
||||
- `GET /modelTF/projects`
|
||||
- `POST /modelTF/projects`
|
||||
- `GET /modelTF/projects/{id}`
|
||||
- `PUT /modelTF/projects/{id}`
|
||||
- `POST /modelTF/projects/{id}/archive`
|
||||
- `GET /modelTF/projects/{id}/members`
|
||||
- `POST /modelTF/projects/{id}/members`
|
||||
- `PUT /modelTF/projects/{id}/members/{user_id}`
|
||||
- `DELETE /modelTF/projects/{id}/members/{user_id}`
|
||||
|
||||
资源授权:
|
||||
|
||||
- `GET /api/resources/{resource_type}/{resource_id}/acl`
|
||||
- `PUT /api/resources/{resource_type}/{resource_id}/acl`
|
||||
- `POST /api/resources/{resource_type}/{resource_id}/share`
|
||||
- `GET /modelTF/resources/{resource_type}/{resource_id}/acl`
|
||||
- `PUT /modelTF/resources/{resource_type}/{resource_id}/acl`
|
||||
- `POST /modelTF/resources/{resource_type}/{resource_id}/share`
|
||||
|
||||
审批:
|
||||
|
||||
- `GET /api/approvals`
|
||||
- `POST /api/approvals`
|
||||
- `GET /api/approvals/{id}`
|
||||
- `POST /api/approvals/{id}/approve`
|
||||
- `POST /api/approvals/{id}/reject`
|
||||
- `POST /api/approvals/{id}/cancel`
|
||||
- `GET /modelTF/approvals`
|
||||
- `POST /modelTF/approvals`
|
||||
- `GET /modelTF/approvals/{id}`
|
||||
- `POST /modelTF/approvals/{id}/approve`
|
||||
- `POST /modelTF/approvals/{id}/reject`
|
||||
- `POST /modelTF/approvals/{id}/cancel`
|
||||
|
||||
算力:
|
||||
|
||||
- `GET /api/compute/nodes`
|
||||
- `GET /api/compute/gpus`
|
||||
- `GET /api/compute/queue`
|
||||
- `POST /api/compute/jobs/{id}/retry`
|
||||
- `POST /api/compute/jobs/{id}/priority`
|
||||
- `GET /modelTF/compute/nodes`
|
||||
- `GET /modelTF/compute/gpus`
|
||||
- `GET /modelTF/compute/queue`
|
||||
- `POST /modelTF/compute/jobs/{id}/retry`
|
||||
- `POST /modelTF/compute/jobs/{id}/priority`
|
||||
|
||||
训练引擎:
|
||||
|
||||
- `GET /api/training-engines`
|
||||
- `GET /api/training-engines/{id}`
|
||||
- `POST /api/training-engines/{id}/health-check`
|
||||
- `GET /api/training-engines/{id}/schema`
|
||||
- `GET /modelTF/training-engines`
|
||||
- `GET /modelTF/training-engines/{id}`
|
||||
- `POST /modelTF/training-engines/{id}/health-check`
|
||||
- `GET /modelTF/training-engines/{id}/schema`
|
||||
|
||||
## 10. 端到端业务流程
|
||||
|
||||
@@ -650,6 +674,8 @@ LLaMA-Factory 适配器负责:
|
||||
10. 是否需要对外提供标准 API 给其他系统调用训练、评测、推理能力?
|
||||
11. 是否需要接入企业统一身份认证,例如 LDAP、OIDC、企业微信、钉钉?
|
||||
12. 是否需要成本核算:按租户/项目统计 GPU 小时、磁盘占用、模型调用量?
|
||||
13. 多算力节点是否需要节点组、租户绑定节点或项目绑定节点策略?
|
||||
14. 跨节点资源同步是否需要限速、同步窗口和管理员审批?
|
||||
|
||||
## 13. 推荐决策补充
|
||||
|
||||
@@ -658,15 +684,17 @@ LLaMA-Factory 适配器负责:
|
||||
1. 本地磁盘主存储放在算力服务器,应用服务器只保留上传临时文件。临时文件默认保留 24 小时,成功转发到算力文件网关后可立即进入清理队列。
|
||||
2. 第一版不支持 MIG、GPU 分片和多任务共享同一张 GPU。一张 GPU 同一时间只分配给一个训练任务或一个推理服务。GPU 数据模型预留 `partition_type`、`parent_gpu_uuid`、`memory_total_mb`,便于后续扩展 MIG。
|
||||
3. 第一版不做自动抢占。支持任务优先级和排队;停止他人任务需要审批或平台管理员权限。
|
||||
4. 第一版必须支持离线导入已有模型和数据集目录。导入由算力 Agent 扫描、校验、登记,并归属指定租户和项目。
|
||||
5. 模型发布区分测试服务和生产服务。测试服务项目内可启动并默认限流;生产服务必须审批。
|
||||
6. 数据脱敏和数据质量评分作为数据处理模块的一等能力进入第一期,先实现规则版脱敏、格式校验、重复率、完整性、长度分布等指标。
|
||||
7. 人工评测/复核作为第二期功能,但第一期需在数据库和页面入口预留人工复核状态与修订字段。
|
||||
8. 第一版支持从 checkpoint 手动恢复训练,不做自动失败续训。失败任务可选择 checkpoint 重试。
|
||||
9. 第一版必须支持 checkpoint 自动清理策略:默认保留最近 3 个、最优 2 个;已发布模型关联 checkpoint 不自动删除;失败任务 checkpoint 默认保留 14 天。
|
||||
10. 第一版提供内部 API,第二期再开放面向其他系统的标准 API、API Key、限流和 Webhook。
|
||||
11. 第一版使用本地账号,预留 OIDC/LDAP 字段和认证 provider 抽象;第二期接入企业统一身份认证。
|
||||
12. 第一版做 GPU 小时、磁盘占用、任务时长、推理调用量等用量统计;第二期再做成本单价和账单核算。
|
||||
4. 多算力节点仍按“单机多 GPU 节点”管理,每个节点独立部署算力服务和 LLaMA-Factory;节点之间不互相访问,由应用平台统一调度和资源同步。
|
||||
5. 多节点调度默认自动选择节点,同时支持管理员/高级用户手动指定节点;调度优先考虑节点健康、标签、权重、空闲 GPU、队列长度和资源副本是否已存在。
|
||||
6. 第一版必须支持离线导入已有模型和数据集目录。导入由算力 Agent 扫描、校验、登记,并归属指定租户和项目。
|
||||
7. 模型发布区分测试服务和生产服务。测试服务项目内可启动并默认限流;生产服务必须审批。
|
||||
8. 数据脱敏和数据质量评分作为数据处理模块的一等能力进入第一期,先实现规则版脱敏、格式校验、重复率、完整性、长度分布等指标。
|
||||
9. 人工评测/复核作为第二期功能,但第一期需在数据库和页面入口预留人工复核状态与修订字段。
|
||||
10. 第一版支持从 checkpoint 手动恢复训练,不做自动失败续训。失败任务可选择 checkpoint 重试。
|
||||
11. 第一版必须支持 checkpoint 自动清理策略:默认保留最近 3 个、最优 2 个;已发布模型关联 checkpoint 不自动删除;失败任务 checkpoint 默认保留 14 天。
|
||||
12. 第一版提供内部 API,第二期再开放面向其他系统的标准 API、API Key、限流和 Webhook。
|
||||
13. 第一版使用本地账号,预留 OIDC/LDAP 字段和认证 provider 抽象;第二期接入企业统一身份认证。
|
||||
14. 第一版做 GPU 小时、磁盘占用、任务时长、推理调用量等用量统计;第二期再做成本单价和账单核算。
|
||||
|
||||
以上决策需要同步反映在接口文档、数据库 SQL、前端页面和部署方案中。第一版实现不再阻塞于这些问题的反复确认,除非实际部署环境与假设明显冲突。
|
||||
|
||||
|
||||
@@ -1147,6 +1147,116 @@ CREATE INDEX IF NOT EXISTS idx_gpu_allocations_scope ON gpu_allocations(tenant_i
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uq_gpu_allocations_active_gpu
|
||||
ON gpu_allocations(gpu_device_id) WHERE released_at IS NULL;
|
||||
|
||||
-- Multi compute-node scheduling and local-cache metadata.
|
||||
-- Each GPU server is modeled as one compute node. Nodes do not call each other;
|
||||
-- the application platform schedules jobs and syncs resources through each node's File Gateway.
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS file_gateway_url text;
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS enabled boolean NOT NULL DEFAULT true;
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS scheduler_status varchar(40) NOT NULL DEFAULT 'online';
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS scheduler_weight integer NOT NULL DEFAULT 100 CHECK (scheduler_weight >= 0);
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS tags text[] NOT NULL DEFAULT '{}';
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS service_token_encrypted text;
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS data_root text NOT NULL DEFAULT '/data/yg-ft';
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS model_root text NOT NULL DEFAULT '/data/yg-ft/models';
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS log_root text NOT NULL DEFAULT '/opt/yg-ft/logs/compute';
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS max_parallel_jobs integer NOT NULL DEFAULT 1 CHECK (max_parallel_jobs >= 0);
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS current_running_jobs integer NOT NULL DEFAULT 0 CHECK (current_running_jobs >= 0);
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS last_health_check_at timestamptz;
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS health_detail jsonb NOT NULL DEFAULT '{}'::jsonb;
|
||||
ALTER TABLE compute_nodes ADD COLUMN IF NOT EXISTS drain_reason text;
|
||||
CREATE INDEX IF NOT EXISTS idx_compute_nodes_scheduler
|
||||
ON compute_nodes(enabled, scheduler_status, scheduler_weight DESC, last_health_check_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_compute_nodes_tags_gin
|
||||
ON compute_nodes USING gin(tags);
|
||||
|
||||
ALTER TABLE compute_jobs ADD COLUMN IF NOT EXISTS scheduler_mode varchar(40) NOT NULL DEFAULT 'auto';
|
||||
ALTER TABLE compute_jobs ADD COLUMN IF NOT EXISTS requested_node_id uuid REFERENCES compute_nodes(id) ON DELETE SET NULL;
|
||||
ALTER TABLE compute_jobs ADD COLUMN IF NOT EXISTS assigned_at timestamptz;
|
||||
ALTER TABLE compute_jobs ADD COLUMN IF NOT EXISTS scheduler_reason text;
|
||||
CREATE INDEX IF NOT EXISTS idx_compute_jobs_requested_node
|
||||
ON compute_jobs(requested_node_id, status, created_at DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS compute_node_engines (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
compute_node_id uuid NOT NULL REFERENCES compute_nodes(id) ON DELETE CASCADE,
|
||||
engine_id uuid REFERENCES training_engines(id) ON DELETE SET NULL,
|
||||
engine_code varchar(80) NOT NULL,
|
||||
engine_version varchar(80),
|
||||
home_path text,
|
||||
status varchar(40) NOT NULL DEFAULT 'available',
|
||||
capability jsonb NOT NULL DEFAULT '{}'::jsonb,
|
||||
last_health_check_at timestamptz,
|
||||
health_detail jsonb NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
updated_at timestamptz NOT NULL DEFAULT now(),
|
||||
UNIQUE (compute_node_id, engine_code)
|
||||
);
|
||||
SELECT touch_updated_at('compute_node_engines');
|
||||
CREATE INDEX IF NOT EXISTS idx_compute_node_engines_node_status
|
||||
ON compute_node_engines(compute_node_id, status, engine_code);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS resource_replicas (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
tenant_id uuid REFERENCES tenants(id) ON DELETE SET NULL,
|
||||
project_id uuid REFERENCES projects(id) ON DELETE SET NULL,
|
||||
resource_type varchar(80) NOT NULL,
|
||||
resource_id uuid NOT NULL,
|
||||
storage_object_id uuid REFERENCES storage_objects(id) ON DELETE SET NULL,
|
||||
compute_node_id uuid NOT NULL REFERENCES compute_nodes(id) ON DELETE CASCADE,
|
||||
local_path text NOT NULL,
|
||||
status varchar(40) NOT NULL DEFAULT 'available',
|
||||
sync_status varchar(40) NOT NULL DEFAULT 'synced',
|
||||
checksum_sha256 char(64),
|
||||
byte_size bigint NOT NULL DEFAULT 0 CHECK (byte_size >= 0),
|
||||
version varchar(120),
|
||||
pinned boolean NOT NULL DEFAULT false,
|
||||
last_verified_at timestamptz,
|
||||
expires_at timestamptz,
|
||||
failure_reason text,
|
||||
metadata jsonb NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
updated_at timestamptz NOT NULL DEFAULT now(),
|
||||
UNIQUE (resource_type, resource_id, compute_node_id)
|
||||
);
|
||||
SELECT touch_updated_at('resource_replicas');
|
||||
CREATE INDEX IF NOT EXISTS idx_resource_replicas_resource
|
||||
ON resource_replicas(resource_type, resource_id, status);
|
||||
CREATE INDEX IF NOT EXISTS idx_resource_replicas_node_status
|
||||
ON resource_replicas(compute_node_id, status, sync_status, updated_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_resource_replicas_scope
|
||||
ON resource_replicas(tenant_id, project_id, resource_type, updated_at DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS resource_sync_jobs (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
tenant_id uuid REFERENCES tenants(id) ON DELETE SET NULL,
|
||||
project_id uuid REFERENCES projects(id) ON DELETE SET NULL,
|
||||
target_compute_node_id uuid NOT NULL REFERENCES compute_nodes(id) ON DELETE CASCADE,
|
||||
source_compute_node_id uuid REFERENCES compute_nodes(id) ON DELETE SET NULL,
|
||||
resource_type varchar(80) NOT NULL,
|
||||
resource_id uuid NOT NULL,
|
||||
storage_object_id uuid REFERENCES storage_objects(id) ON DELETE SET NULL,
|
||||
status task_status NOT NULL DEFAULT 'pending',
|
||||
transfer_mode varchar(40) NOT NULL DEFAULT 'app_proxy',
|
||||
source_uri text,
|
||||
target_path text NOT NULL,
|
||||
byte_size bigint NOT NULL DEFAULT 0 CHECK (byte_size >= 0),
|
||||
checksum_sha256 char(64),
|
||||
progress numeric(5,2) NOT NULL DEFAULT 0 CHECK (progress >= 0 AND progress <= 100),
|
||||
failure_reason text,
|
||||
requested_by uuid REFERENCES users(id) ON DELETE SET NULL,
|
||||
started_at timestamptz,
|
||||
completed_at timestamptz,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
SELECT touch_updated_at('resource_sync_jobs');
|
||||
CREATE INDEX IF NOT EXISTS idx_resource_sync_jobs_status
|
||||
ON resource_sync_jobs(status, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_resource_sync_jobs_target
|
||||
ON resource_sync_jobs(target_compute_node_id, status, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_resource_sync_jobs_resource
|
||||
ON resource_sync_jobs(resource_type, resource_id, status);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS resource_acl (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
tenant_id uuid REFERENCES tenants(id) ON DELETE CASCADE,
|
||||
|
||||
@@ -431,9 +431,9 @@ Expected: 三项检查全部 PASS。
|
||||
|
||||
- [ ] **Step 3: 启动页面并逐步验证四步交互**
|
||||
|
||||
Run: `cd frontend && npm run dev -- --host 0.0.0.0 --port 6801`
|
||||
Run: `cd frontend && npm run dev -- --host 0.0.0.0 --port 16801`
|
||||
|
||||
Browser checks at `http://localhost:6801/data-process/create`:
|
||||
Browser checks at `http://localhost:16801/data-process/create`:
|
||||
|
||||
1. 第一步上传文本并选择非结构化数据。
|
||||
2. 第二步点击至少三个右侧切片,确认左侧滚动目标和高亮范围变化。
|
||||
|
||||
@@ -156,7 +156,7 @@
|
||||
- API Key 管理。
|
||||
- OpenAPI 文档。
|
||||
- 请求限流。
|
||||
- Webhook 回调。
|
||||
- Webhook 回调作为外部系统集成的可选能力,不作为算力状态同步默认方案。
|
||||
- 外部系统发起训练、评测、推理。
|
||||
|
||||
### 1.11 企业统一身份认证
|
||||
@@ -412,11 +412,11 @@ YG_FT/
|
||||
|
||||
联调接口:
|
||||
|
||||
- `GET /api/users`
|
||||
- `POST /api/users`
|
||||
- `PUT /api/users/{id}`
|
||||
- `DELETE /api/users/{id}`
|
||||
- `PUT /api/users/{id}/password`
|
||||
- `GET /modelTF/users`
|
||||
- `POST /modelTF/users`
|
||||
- `PUT /modelTF/users/{id}`
|
||||
- `DELETE /modelTF/users/{id}`
|
||||
- `PUT /modelTF/users/{id}/password`
|
||||
|
||||
#### 项目空间
|
||||
|
||||
@@ -439,12 +439,12 @@ YG_FT/
|
||||
|
||||
联调接口:
|
||||
|
||||
- `GET /api/projects`
|
||||
- `POST /api/projects`
|
||||
- `GET /api/projects/{id}`
|
||||
- `PUT /api/projects/{id}`
|
||||
- `GET /api/projects/{id}/members`
|
||||
- `POST /api/projects/{id}/members`
|
||||
- `GET /modelTF/projects`
|
||||
- `POST /modelTF/projects`
|
||||
- `GET /modelTF/projects/{id}`
|
||||
- `PUT /modelTF/projects/{id}`
|
||||
- `GET /modelTF/projects/{id}/members`
|
||||
- `POST /modelTF/projects/{id}/members`
|
||||
|
||||
#### 审批中心
|
||||
|
||||
@@ -466,10 +466,10 @@ YG_FT/
|
||||
|
||||
联调接口:
|
||||
|
||||
- `GET /api/approvals`
|
||||
- `POST /api/approvals/{id}/approve`
|
||||
- `POST /api/approvals/{id}/reject`
|
||||
- `POST /api/approvals/{id}/cancel`
|
||||
- `GET /modelTF/approvals`
|
||||
- `POST /modelTF/approvals/{id}/approve`
|
||||
- `POST /modelTF/approvals/{id}/reject`
|
||||
- `POST /modelTF/approvals/{id}/cancel`
|
||||
|
||||
#### 算力资源中心
|
||||
|
||||
@@ -487,13 +487,16 @@ YG_FT/
|
||||
- 任务队列。
|
||||
- Agent 健康状态。
|
||||
- 租户/项目配额。
|
||||
- 算力节点新增、编辑、连接测试、启用、禁用、维护模式。
|
||||
- 节点权重、标签、训练引擎版本和本地资源副本。
|
||||
- 自动调度和手动指定节点入口。
|
||||
|
||||
联调接口:
|
||||
|
||||
- `GET /api/compute/gpus`
|
||||
- `GET /api/compute/queue`
|
||||
- `GET /api/compute/nodes`
|
||||
- `GET /api/quotas/usage`
|
||||
- `GET /modelTF/compute/gpus`
|
||||
- `GET /modelTF/compute/queue`
|
||||
- `GET /modelTF/compute/nodes`
|
||||
- `GET /modelTF/quotas/usage`
|
||||
|
||||
### 5.2 现有页面改造
|
||||
|
||||
@@ -529,9 +532,9 @@ YG_FT/
|
||||
|
||||
交付接口:
|
||||
|
||||
- `POST /api/login`
|
||||
- `POST /api/logout`
|
||||
- `GET /api/me`
|
||||
- `POST /modelTF/login`
|
||||
- `POST /modelTF/logout`
|
||||
- `GET /modelTF/me`
|
||||
|
||||
#### Tenant / Project 模块
|
||||
|
||||
@@ -545,9 +548,9 @@ YG_FT/
|
||||
|
||||
交付接口:
|
||||
|
||||
- `/api/tenants`
|
||||
- `/api/projects`
|
||||
- `/api/projects/{id}/members`
|
||||
- `/modelTF/tenants`
|
||||
- `/modelTF/projects`
|
||||
- `/modelTF/projects/{id}/members`
|
||||
|
||||
#### Resource ACL 模块
|
||||
|
||||
@@ -559,24 +562,35 @@ YG_FT/
|
||||
|
||||
交付接口:
|
||||
|
||||
- `GET /api/resources/{resource_type}/{resource_id}/acl`
|
||||
- `PUT /api/resources/{resource_type}/{resource_id}/acl`
|
||||
- `GET /modelTF/resources/{resource_type}/{resource_id}/acl`
|
||||
- `PUT /modelTF/resources/{resource_type}/{resource_id}/acl`
|
||||
|
||||
#### Compute Gateway 模块
|
||||
|
||||
开发内容:
|
||||
|
||||
- 算力平台客户端。
|
||||
- 算力节点管理:地址、File Gateway、权重、标签、启用状态、维护状态。
|
||||
- 自动/手动调度策略。
|
||||
- 调度前检查模型/数据集在目标节点的资源副本。
|
||||
- 缺失资源时创建同步任务,通过目标节点 File Gateway 写入本地磁盘。
|
||||
- 创建 compute job。
|
||||
- 查询状态和日志。
|
||||
- 状态回调验签。
|
||||
- 应用侧定时轮询 Compute API,同步任务状态、日志摘要和产物索引。
|
||||
- 任务状态映射。
|
||||
|
||||
交付接口:
|
||||
|
||||
- `GET /api/compute/gpus`
|
||||
- `GET /api/compute/queue`
|
||||
- `POST /api/internal/compute-callbacks/jobs`
|
||||
- `GET /modelTF/compute/gpus`
|
||||
- `GET /modelTF/compute/queue`
|
||||
- `GET/POST/PUT /modelTF/compute/nodes`
|
||||
- `POST /modelTF/compute/nodes/{id}/test-connection`
|
||||
- `POST /modelTF/compute/nodes/{id}/enable`
|
||||
- `POST /modelTF/compute/nodes/{id}/disable`
|
||||
- `POST /modelTF/compute/nodes/{id}/drain`
|
||||
- `GET /modelTF/compute/nodes/{id}/replicas`
|
||||
- `POST /modelTF/internal/compute-sync/jobs/poll`
|
||||
- `POST /modelTF/internal/compute-sync/resources`
|
||||
|
||||
#### Fine Tune 模块
|
||||
|
||||
@@ -592,7 +606,7 @@ YG_FT/
|
||||
|
||||
交付接口:
|
||||
|
||||
- 沿用 `docs/backend-api-design.md` 的 `/api/fine-tune` 系列。
|
||||
- 沿用 `docs/backend-api-design.md` 的 `/modelTF/fine-tune` 系列。
|
||||
|
||||
#### Approval 模块
|
||||
|
||||
@@ -605,8 +619,8 @@ YG_FT/
|
||||
|
||||
交付接口:
|
||||
|
||||
- `/api/approvals`
|
||||
- `/api/approval-templates`
|
||||
- `/modelTF/approvals`
|
||||
- `/modelTF/approval-templates`
|
||||
|
||||
#### Audit 模块
|
||||
|
||||
@@ -620,8 +634,8 @@ YG_FT/
|
||||
|
||||
交付接口:
|
||||
|
||||
- `GET /api/audit-logs`
|
||||
- `GET /api/login-logs`
|
||||
- `GET /modelTF/audit-logs`
|
||||
- `GET /modelTF/login-logs`
|
||||
|
||||
### 6.2 P1 模块
|
||||
|
||||
@@ -646,13 +660,13 @@ YG_FT/
|
||||
|
||||
接口:
|
||||
|
||||
- `POST /compute/jobs`
|
||||
- `GET /compute/jobs/{id}`
|
||||
- `POST /compute/jobs/{id}/stop`
|
||||
- `GET /compute/jobs/{id}/logs`
|
||||
- `GET /compute/resources/gpus`
|
||||
- `POST /compute/files/upload`
|
||||
- `GET /compute/files/{id}/download`
|
||||
- `POST /modelTF/compute/jobs`
|
||||
- `GET /modelTF/compute/jobs/{id}`
|
||||
- `POST /modelTF/compute/jobs/{id}/stop`
|
||||
- `GET /modelTF/compute/jobs/{id}/logs`
|
||||
- `GET /modelTF/compute/resources/gpus`
|
||||
- `POST /modelTF/compute/files/upload`
|
||||
- `GET /modelTF/compute/files/{id}/download`
|
||||
|
||||
职责:
|
||||
|
||||
@@ -660,7 +674,9 @@ YG_FT/
|
||||
- 校验服务 token。
|
||||
- 调用 Agent。
|
||||
- 聚合状态。
|
||||
- 回调应用平台。
|
||||
- 提供任务状态查询接口,供应用平台定时轮询。
|
||||
- 每个单机多 GPU 算力节点都部署一套 Compute API,不依赖其他算力节点。
|
||||
- 暴露节点健康、GPU、训练引擎、资源副本和文件网关状态。
|
||||
|
||||
### 7.2 Compute Agent
|
||||
|
||||
@@ -735,9 +751,12 @@ YG_FT/
|
||||
| `quota_usage` | 配额使用 |
|
||||
| `storage_nodes` | 存储节点 |
|
||||
| `compute_nodes` | 算力节点 |
|
||||
| `compute_node_engines` | 算力节点训练引擎能力 |
|
||||
| `gpu_devices` | GPU 设备 |
|
||||
| `gpu_allocations` | GPU 分配记录 |
|
||||
| `compute_jobs` | 算力任务 |
|
||||
| `resource_replicas` | 数据集/模型/产物在算力节点的本地副本 |
|
||||
| `resource_sync_jobs` | 应用平台编排的资源同步任务 |
|
||||
| `training_engines` | 训练引擎 |
|
||||
| `retention_policies` | 保留策略 |
|
||||
| `cleanup_jobs` | 清理任务 |
|
||||
@@ -782,12 +801,14 @@ YG_FT/
|
||||
- `file-gateway`
|
||||
- `llama-factory-env`
|
||||
|
||||
多算力节点阶段,每台单机多 GPU 服务器都部署以上组件和宿主机挂载的 LLaMA-Factory。算力节点之间默认不互相访问,由应用平台统一调度和同步资源。
|
||||
|
||||
配置:
|
||||
|
||||
- `COMPUTE_NODE_ID`
|
||||
- `SERVICE_TOKEN`
|
||||
- `APP_CALLBACK_URL`
|
||||
- `DATA_ROOT=/data/ft-platform`
|
||||
- `ENABLE_APP_CALLBACK=false`
|
||||
- `DATA_ROOT=/data/yg-ft`
|
||||
- `LLAMA_FACTORY_PATH`
|
||||
- `PYTHON_ENV_PATH`
|
||||
- `GPU_VISIBLE_DEVICES`
|
||||
@@ -798,7 +819,7 @@ YG_FT/
|
||||
|
||||
- 前端只访问应用平台。
|
||||
- 应用平台可访问算力平台内部 API。
|
||||
- 算力平台可回调应用平台 internal callback。
|
||||
- 第一阶段只开通应用平台主动访问算力平台内部 API,状态同步采用应用侧轮询。
|
||||
- 算力平台不直接暴露给公网。
|
||||
- 文件下载通过应用平台签发令牌。
|
||||
|
||||
@@ -939,9 +960,9 @@ YG_FT/
|
||||
|
||||
### 算力组
|
||||
|
||||
- CE-A:Compute API、Agent、GPU 调度。
|
||||
- CE-A:Compute API、Agent、GPU 调度、多节点健康检查。
|
||||
- CE-B:LLaMA-Factory Adapter、日志解析、产物管理。
|
||||
- CE-C:File Gateway、本地磁盘、离线导入。
|
||||
- CE-C:File Gateway、本地磁盘、离线导入、资源副本同步。
|
||||
|
||||
### 数据库/部署组
|
||||
|
||||
@@ -954,7 +975,8 @@ YG_FT/
|
||||
| --- | --- | --- |
|
||||
| M1 基础治理 | 第 1-2 周 | 登录、用户、租户、项目、权限 |
|
||||
| M2 资源管理 | 第 3-4 周 | 模型、数据集、文件网关、离线导入 |
|
||||
| M3 训练主链路 | 第 5-7 周 | GPU 调度、LLaMA-Factory 训练、日志指标、产物 |
|
||||
| M3 训练主链路 | 第 5-7 周 | GPU 调度、LLaMA-Factory 训练、日志指标、产物、资源副本检查 |
|
||||
| M3.5 多节点预留 | 第 7-8 周 | compute_nodes 管理、节点权重/标签、自动/手动调度、资源同步任务 |
|
||||
| M4 数据处理和评测 | 第 8-9 周 | 数据处理、质量评分、自动评测 |
|
||||
| M5 推理发布 | 第 10-11 周 | 推理服务、模型对比、生产发布审批 |
|
||||
| M6 企业治理收口 | 第 12 周 | 审批、审计、配额、清理、部署文档 |
|
||||
@@ -967,7 +989,8 @@ YG_FT/
|
||||
- 项目成员只能访问授权项目资源。
|
||||
- 模型和数据集支持资源级授权。
|
||||
- 上传数据集后可预览、编辑版本、用于训练。
|
||||
- 训练任务可指定 GPU 并成功运行。
|
||||
- 训练任务可自动调度节点/GPU,也可由管理员手动指定节点/GPU 并成功运行。
|
||||
- 调度前可识别目标节点是否已有数据集和模型副本,缺失时能创建同步任务。
|
||||
- 训练日志和指标实时可见。
|
||||
- 训练产物可登记、合并、发布测试服务。
|
||||
- 评测任务可生成样本级结果。
|
||||
@@ -1007,6 +1030,7 @@ YG_FT/
|
||||
- `docs/postgres-schema.sql`:初始数据库模型。
|
||||
- `docs/platform-architecture-requirements.md`:架构与功能需求补充。
|
||||
- `docs/system-development-plan.md`:多人开发拆分和实施计划。
|
||||
- `docs/team-development-plan.md`:3-4 人并行开发人员分工、模块边界、接口范围和里程碑。
|
||||
|
||||
后续建议再补两份执行文档:
|
||||
|
||||
@@ -1015,72 +1039,72 @@ YG_FT/
|
||||
|
||||
## 16. 页面模块开发工作包
|
||||
|
||||
本节用于多人并行开发时认领任务。每个工作包都标明对应页面、前端内容、后端接口、DB 表和部署/算力依赖。
|
||||
本节用于多人并行开发时认领任务。每个工作包都标明对应页面、前端内容、后端接口、DB 表和部署/算力依赖。当前菜单、二级路由、规划菜单和接口/数据库映射总览见 `docs/menu-functional-requirements.md`;按 3-4 人落地的具体人员边界和开发节奏见 `docs/team-development-plan.md`。
|
||||
|
||||
### 16.1 基础入口与用户权限
|
||||
|
||||
| 工作包 | 页面/路由 | 前端开发 | 后端开发 | DB 关联 | 依赖 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 登录和会话 | `/login` | 登录表单、错误提示、登录后跳转、会话过期处理 | `POST /api/login`、`GET /api/me`、JWT 中间件 | `users`、`login_sessions`、`v_user_effective_permissions` | 无 |
|
||||
| 登录和会话 | `/login` | 登录表单、错误提示、登录后跳转、会话过期处理 | `POST /modelTF/login`、`GET /modelTF/me`、JWT 中间件 | `users`、`login_sessions`、`v_user_effective_permissions` | 无 |
|
||||
| 主布局和权限菜单 | `/` | 菜单按权限过滤、用户信息、项目切换器 | 当前用户权限、当前项目上下文 | `permissions`、`role_permissions`、`project_members` | 登录 |
|
||||
| 用户中心 | `/user-settings`、`/user-settings/create`、`/user-settings/:id/permission` | 用户列表、创建、禁用、重置密码、页面权限 | `/api/users` 系列 | `users`、`user_permissions`、`tenant_users` | 租户/项目 |
|
||||
| 用户中心 | `/user-settings`、`/user-settings/create`、`/user-settings/:id/permission` | 用户列表、创建、禁用、重置密码、页面权限 | `/modelTF/users` 系列 | `users`、`user_permissions`、`tenant_users` | 租户/项目 |
|
||||
| 无权限页 | `/permission-denied` | 无权限说明、返回入口 | 权限异常返回统一错误码 | 无新增 | 主布局 |
|
||||
|
||||
### 16.2 租户和项目空间
|
||||
|
||||
| 工作包 | 页面/路由 | 前端开发 | 后端开发 | DB 关联 | 依赖 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 租户管理 | `/tenants`、`/tenants/:id` | 租户列表、创建/禁用、配额、留存策略 | `/api/tenants` 系列 | `tenants`、`tenant_users`、`quotas`、`retention_policies` | 用户中心 |
|
||||
| 项目列表和详情 | `/projects`、`/projects/:id` | 项目列表、创建、归档、资源概览 | `/api/projects` 系列 | `projects`、`quota_usage` | 租户 |
|
||||
| 项目成员 | `/projects/:id/members` | 成员列表、添加成员、角色选择 | `/api/projects/{id}/members` 系列 | `project_members` | 项目 |
|
||||
| 资源授权 | `/projects/:id/permissions`、资源详情弹窗 | ACL 表格、用户/角色授权 | `/api/resources/{type}/{id}/acl` | `resource_acl` | 项目、资源表 |
|
||||
| 租户管理 | `/tenants`、`/tenants/:id` | 租户列表、创建/禁用、配额、留存策略 | `/modelTF/tenants` 系列 | `tenants`、`tenant_users`、`quotas`、`retention_policies` | 用户中心 |
|
||||
| 项目列表和详情 | `/projects`、`/projects/:id` | 项目列表、创建、归档、资源概览 | `/modelTF/projects` 系列 | `projects`、`quota_usage` | 租户 |
|
||||
| 项目成员 | `/projects/:id/members` | 成员列表、添加成员、角色选择 | `/modelTF/projects/{id}/members` 系列 | `project_members` | 项目 |
|
||||
| 资源授权 | `/projects/:id/permissions`、资源详情弹窗 | ACL 表格、用户/角色授权 | `/modelTF/resources/{type}/{id}/acl` | `resource_acl` | 项目、资源表 |
|
||||
|
||||
### 16.3 数据集和数据处理
|
||||
|
||||
| 工作包 | 页面/路由 | 前端开发 | 后端开发 | DB 关联 | 依赖 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 数据集列表 | `/dataset` | 列表、搜索、项目过滤、下载、删除审批入口 | `GET/DELETE /api/dataset-manage` | `datasets`、`resource_acl`、`approval_instances` | 项目上下文 |
|
||||
| 数据集创建/上传 | `/dataset/create`、`/dataset/:id/edit` | 表单、分片上传、离线导入弹窗 | `POST /api/dataset-manage`、`POST /api/files/upload-session`、`POST /api/import/local-dataset` | `datasets`、`dataset_files`、`storage_objects`、`file_upload_sessions`、`local_import_jobs` | 文件网关 |
|
||||
| 数据集预览/版本 | `/dataset/:id/preview` | 文件列表、内容预览、版本栏、在线编辑 | `/api/dataset-manage/preview`、`/versions` 系列 | `dataset_file_versions`、`dataset_records` | 数据集上传 |
|
||||
| 数据处理列表 | `/data-process` | 任务列表、状态、输出数据集跳转 | `/api/data-process` 系列 | `data_process_tasks` | 数据集 |
|
||||
| 数据处理创建向导 | `/data-process/create` | 任务配置、模型选择、源文件、预览切片、生成、结果编辑、发布 | `/api/data-process/{id}/source-files`、`preview/build`、`generate`、`publish` | `data_process_source_files`、`data_process_preview_items`、`data_process_results` | 数据集、模型、文件网关 |
|
||||
| 数据处理详情 | `/data-process/:id` | 运行统计、失败原因、结果表格 | `GET /api/data-process/{id}`、`results`、`events` | `data_process_tasks`、`data_process_results` | 数据处理任务 |
|
||||
| 数据转换 | `/data-convert` | 文件上传、转换状态、下载 | `/api/data-convert/jobs` 系列 | `data_convert_jobs`、`storage_objects` | 文件网关 |
|
||||
| 数据集列表 | `/dataset` | 列表、搜索、项目过滤、下载、删除审批入口 | `GET/DELETE /modelTF/dataset-manage` | `datasets`、`resource_acl`、`approval_instances` | 项目上下文 |
|
||||
| 数据集创建/上传 | `/dataset/create`、`/dataset/:id/edit` | 表单、分片上传、离线导入弹窗 | `POST /modelTF/dataset-manage`、`POST /modelTF/files/upload-session`、`POST /modelTF/import/local-dataset` | `datasets`、`dataset_files`、`storage_objects`、`file_upload_sessions`、`local_import_jobs` | 文件网关 |
|
||||
| 数据集预览/版本 | `/dataset/:id/preview` | 文件列表、内容预览、版本栏、在线编辑 | `/modelTF/dataset-manage/preview`、`/versions` 系列 | `dataset_file_versions`、`dataset_records` | 数据集上传 |
|
||||
| 数据处理列表 | `/data-process` | 任务列表、状态、输出数据集跳转 | `/modelTF/data-process` 系列 | `data_process_tasks` | 数据集 |
|
||||
| 数据处理创建向导 | `/data-process/create` | 任务配置、模型选择、源文件、预览切片、生成、结果编辑、发布 | `/modelTF/data-process/{id}/source-files`、`preview/build`、`generate`、`publish` | `data_process_source_files`、`data_process_preview_items`、`data_process_results` | 数据集、模型、文件网关 |
|
||||
| 数据处理详情 | `/data-process/:id` | 运行统计、失败原因、结果表格 | `GET /modelTF/data-process/{id}`、`results`、`events` | `data_process_tasks`、`data_process_results` | 数据处理任务 |
|
||||
| 数据转换 | `/data-convert` | 文件上传、转换状态、下载 | `/modelTF/data-convert/jobs` 系列 | `data_convert_jobs`、`storage_objects` | 文件网关 |
|
||||
|
||||
### 16.4 模型管理和训练
|
||||
|
||||
| 工作包 | 页面/路由 | 前端开发 | 后端开发 | DB 关联 | 依赖 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 模型列表 | `/model-manage` | 列表、用途标签、授权、导出、删除审批 | `/api/model-manage` 系列 | `models`、`trained_models`、`resource_acl` | 项目上下文 |
|
||||
| 模型创建/离线导入 | `/model-manage/create`、`/model-manage/:id/edit` | 本地/API 模型表单、离线导入 | `POST/PUT /api/model-manage`、`POST /api/import/local-model` | `models`、`storage_objects`、`local_import_jobs` | 文件网关 |
|
||||
| 权重合并 | `/model-manage/merge` | 选择训练产物、合并状态 | `POST /api/model-manage/merge`、`GET /api/compute/jobs/{id}` | `trained_models`、`compute_jobs` | 算力平台 |
|
||||
| 微调列表 | `/fine-tune` | 任务列表、停止、删除、日志入口 | `/api/fine-tune` 系列 | `fine_tune_tasks` | 模型、数据集 |
|
||||
| 微调创建 | `/fine-tune/create` | 训练参数、GPU 选择、命令预览、审批提示 | `POST /api/fine-tune`、`POST /api/fine-tune/start`、`GET /api/compute/gpus` | `fine_tune_tasks`、`compute_jobs`、`gpu_allocations` | 算力平台、审批 |
|
||||
| 训练日志和 checkpoint | `/training-log/:id` | 日志 tail、指标曲线、checkpoint、恢复/重试 | `GET /api/fine-tune/{id}/overview`、`checkpoints`、`resume`、`retry` | `fine_tune_metrics`、`fine_tune_checkpoints` | 训练任务 |
|
||||
| 训练引擎管理 | `/training-engines` | 引擎列表、schema、健康检查 | `/api/training-engines` 系列 | `training_engines` | 算力 Agent |
|
||||
| 模型列表 | `/model-manage` | 列表、用途标签、授权、导出、删除审批 | `/modelTF/model-manage` 系列 | `models`、`trained_models`、`resource_acl` | 项目上下文 |
|
||||
| 模型创建/离线导入 | `/model-manage/create`、`/model-manage/:id/edit` | 本地/API 模型表单、离线导入 | `POST/PUT /modelTF/model-manage`、`POST /modelTF/import/local-model` | `models`、`storage_objects`、`local_import_jobs` | 文件网关 |
|
||||
| 权重合并 | `/model-manage/merge` | 选择训练产物、合并状态 | `POST /modelTF/model-manage/merge`、`GET /modelTF/compute/jobs/{id}` | `trained_models`、`compute_jobs` | 算力平台 |
|
||||
| 微调列表 | `/fine-tune` | 任务列表、停止、删除、日志入口 | `/modelTF/fine-tune` 系列 | `fine_tune_tasks` | 模型、数据集 |
|
||||
| 微调创建 | `/fine-tune/create` | 训练参数、GPU 选择、命令预览、审批提示 | `POST /modelTF/fine-tune`、`POST /modelTF/fine-tune/start`、`GET /modelTF/compute/gpus` | `fine_tune_tasks`、`compute_jobs`、`gpu_allocations` | 算力平台、审批 |
|
||||
| 训练日志和 checkpoint | `/training-log/:id` | 日志 tail、指标曲线、checkpoint、恢复/重试 | `GET /modelTF/fine-tune/{id}/overview`、`checkpoints`、`resume`、`retry` | `fine_tune_metrics`、`fine_tune_checkpoints` | 训练任务 |
|
||||
| 训练引擎管理 | `/training-engines` | 引擎列表、schema、健康检查 | `/modelTF/training-engines` 系列 | `training_engines` | 算力 Agent |
|
||||
|
||||
### 16.5 评测、推理和发布
|
||||
|
||||
| 工作包 | 页面/路由 | 前端开发 | 后端开发 | DB 关联 | 依赖 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 评测列表 | `/model-eval` | 列表、分数、状态、删除 | `/api/model-eval` 系列 | `eval_tasks` | 模型、数据集 |
|
||||
| 评测创建 | `/model-eval/create` | 模型/数据集/维度/GPU 选择、指标配置 | `POST /api/model-eval/start`、`GET /api/dimension` | `eval_tasks`、`eval_dimensions`、`compute_jobs` | 算力平台 |
|
||||
| 评测详情 | `/model-eval/:id` | 综合评价、维度汇总、样本结果、人工复核预留 | `GET /api/model-eval/{id}`、`events` | `eval_sample_results`、`eval_dimension_summaries` | 评测任务 |
|
||||
| 评测维度 | `/model-eval/dimension/:id/edit` | 维度表单、Prompt 编辑 | `/api/dimension` 系列 | `eval_dimensions` | 模型管理 |
|
||||
| 推理列表/创建 | `/model-inference`、`/model-inference/create` | 任务列表、模型选择、GPU 选择、加载状态 | `/api/model-compare`、`load/unload` | `inference_tasks`、`inference_task_models` | 算力平台 |
|
||||
| 评测列表 | `/model-eval` | 列表、分数、状态、删除 | `/modelTF/model-eval` 系列 | `eval_tasks` | 模型、数据集 |
|
||||
| 评测创建 | `/model-eval/create` | 模型/数据集/维度/GPU 选择、指标配置 | `POST /modelTF/model-eval/start`、`GET /modelTF/dimension` | `eval_tasks`、`eval_dimensions`、`compute_jobs` | 算力平台 |
|
||||
| 评测详情 | `/model-eval/:id` | 综合评价、维度汇总、样本结果、人工复核预留 | `GET /modelTF/model-eval/{id}`、`events` | `eval_sample_results`、`eval_dimension_summaries` | 评测任务 |
|
||||
| 评测维度 | `/model-eval/dimension/:id/edit` | 维度表单、Prompt 编辑 | `/modelTF/dimension` 系列 | `eval_dimensions` | 模型管理 |
|
||||
| 推理列表/创建 | `/model-inference`、`/model-inference/create` | 任务列表、模型选择、GPU 选择、加载状态 | `/modelTF/model-compare`、`load/unload` | `inference_tasks`、`inference_task_models` | 算力平台 |
|
||||
| 推理对话 | `/model-inference/chat/:id` | 单模型流式对话、历史消息 | `stream-chat`、`chat-with-port` | `chat_sessions`、`chat_messages` | 推理服务 |
|
||||
| 模型对比 | `/model-compare/chat/:id`、`/model-compare/result` | 多模型对话、结果对比 | `/api/model-chat/*` | `chat_sessions`、`chat_messages` | 推理任务 |
|
||||
| 模型服务治理 | `/model-services`、`/model-services/:id` | 测试/生产服务、发布申请、下线、调用统计 | `POST /api/approvals`、`GET /api/usage/summary` | `model_services`、`approval_instances` | 审批、算力平台 |
|
||||
| 模型对比 | `/model-compare/chat/:id`、`/model-compare/result` | 多模型对话、结果对比 | `/modelTF/model-chat/*` | `chat_sessions`、`chat_messages` | 推理任务 |
|
||||
| 模型服务治理 | `/model-services`、`/model-services/:id` | 测试/生产服务、发布申请、下线、调用统计 | `POST /modelTF/approvals`、`GET /modelTF/usage/summary` | `model_services`、`approval_instances` | 审批、算力平台 |
|
||||
|
||||
### 16.6 审批、审计、算力和存储运维
|
||||
|
||||
| 工作包 | 页面/路由 | 前端开发 | 后端开发 | DB 关联 | 依赖 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 审批中心 | `/approvals`、`/approvals/pending`、`/approvals/mine`、`/approvals/:id` | 审批列表、详情、通过/驳回/撤回 | `/api/approvals` 系列 | `approval_instances`、`approval_steps` | 用户、项目 |
|
||||
| 审批模板 | `/approval-settings` | 动作策略、审批人规则、超时配置 | `/api/approval-templates` 系列 | `approval_templates` | 租户/项目 |
|
||||
| 算力资源中心 | `/compute`、`/compute/gpus`、`/compute/queue`、`/compute/nodes` | GPU 卡片、节点状态、队列、优先级 | `/api/compute/*`、`/compute/*` 内部接口 | `compute_nodes`、`gpu_devices`、`compute_jobs`、`gpu_allocations` | Compute API/Agent |
|
||||
| 存储管理 | `/storage` | 磁盘占用、大文件、临时文件、checkpoint 清理 | `GET /api/quotas/usage`、`GET/PUT /api/retention-policies` | `storage_nodes`、`storage_objects`、`cleanup_jobs`、`retention_policies` | 文件网关 |
|
||||
| 审计中心 | `/audit-logs`、`/login-logs`、`/download-logs` | 筛选、详情、导出 | `GET /api/audit-logs`、`GET /api/login-logs`、`GET /api/download-logs` | `audit_logs`、`login_sessions` | 审计中间件 |
|
||||
| 审批中心 | `/approvals`、`/approvals/pending`、`/approvals/mine`、`/approvals/:id` | 审批列表、详情、通过/驳回/撤回 | `/modelTF/approvals` 系列 | `approval_instances`、`approval_steps` | 用户、项目 |
|
||||
| 审批模板 | `/approval-settings` | 动作策略、审批人规则、超时配置 | `/modelTF/approval-templates` 系列 | `approval_templates` | 租户/项目 |
|
||||
| 算力资源中心 | `/compute`、`/compute/gpus`、`/compute/queue`、`/compute/nodes` | GPU 卡片、节点状态、队列、优先级、节点新增/编辑、连接测试、启用/禁用、维护模式、节点权重/标签、本地资源副本 | `/modelTF/compute/*`、`/modelTF/compute/*` 内部接口 | `compute_nodes`、`compute_node_engines`、`gpu_devices`、`compute_jobs`、`gpu_allocations`、`resource_replicas`、`resource_sync_jobs` | Compute API/Agent/File Gateway |
|
||||
| 存储管理 | `/storage` | 磁盘占用、大文件、临时文件、checkpoint 清理 | `GET /modelTF/quotas/usage`、`GET/PUT /modelTF/retention-policies` | `storage_nodes`、`storage_objects`、`cleanup_jobs`、`retention_policies` | 文件网关 |
|
||||
| 审计中心 | `/audit-logs`、`/login-logs`、`/download-logs` | 筛选、详情、导出 | `GET /modelTF/audit-logs`、`GET /modelTF/login-logs`、`GET /modelTF/download-logs` | `audit_logs`、`login_sessions` | 审计中间件 |
|
||||
|
||||
### 16.7 页面开发优先级建议
|
||||
|
||||
|
||||
208
docs/team-development-plan.md
Normal file
208
docs/team-development-plan.md
Normal file
@@ -0,0 +1,208 @@
|
||||
# 多人并行开发分工计划
|
||||
|
||||
> 本文基于 `docs/menu-functional-requirements.md`、`docs/backend-api-design.md`、`docs/postgres-schema.sql`、`docs/system-development-plan.md` 和当前前端路由整理,用于 3-4 人并行开发。开发口径以正式系统演进为准,不以临时演示或静态 Mock 作为交付标准。
|
||||
|
||||
## 1. 分工原则
|
||||
|
||||
- 每位开发尽量独立负责一组页面、后端模块、数据库表和联调脚本,避免多人同时改同一个业务文件。
|
||||
- 公共接口契约先在 `docs/backend-api-design.md` 更新,再进入代码实现。
|
||||
- 前端 API 模块按业务域维护:`system.ts`、`model.ts`、`dataset.ts`、`fineTune.ts`、`compute.ts`、`eval.ts`、`log.ts`。
|
||||
- 后端业务代码按 `backend/app/modules/<domain>/` 拆分,路由统一挂载 `/modelTF`。
|
||||
- 数据库按迁移脚本推进,目标模型以 `docs/postgres-schema.sql` 为准,当前运行库脚本以 `backend/app/db/sql/` 为准。
|
||||
- 所有写操作必须预留审计;涉及删除、导出、发布、停止他人任务等高风险动作必须预留审批入口。
|
||||
|
||||
## 2. 4 人开发拆分
|
||||
|
||||
### A. 平台基础与企业治理
|
||||
|
||||
负责人边界:
|
||||
|
||||
- 前端目录:`frontend/src/views/login/`、`frontend/src/views/system/`,后续新增 `tenants`、`projects`、`approvals`、`audit` 页面目录。
|
||||
- 前端 API:`frontend/src/api/modules/system.ts`、`frontend/src/api/modules/log.ts`。
|
||||
- 后端模块:`auth`、`tenant`、`project`、`approval`、`audit`、`retention`、`system`。
|
||||
- 数据库表:`users`、`permissions`、`roles`、`role_permissions`、`user_permission_overrides`、`tenants`、`tenant_users`、`projects`、`project_members`、`resource_acl`、`approval_templates`、`approval_instances`、`approval_steps`、`audit_logs`、`retention_policies`。
|
||||
|
||||
对应页面和功能:
|
||||
|
||||
| 页面/模块 | 路由 | 功能点 | 接口 |
|
||||
| --- | --- | --- | --- |
|
||||
| 登录 | `/login` | 登录、Token 写入、登录失败提示、会话恢复 | `POST /modelTF/login`、`GET /modelTF/me` |
|
||||
| 用户设置 | `/user-settings`、`/user-settings/create`、`/user-settings/:id/permission` | 用户列表、创建、启停、重置密码、页面权限 | `/modelTF/users/*`、`/modelTF/permissions` |
|
||||
| 平台性能 | `/hardware` | 系统资源、进程、GPU 摘要 | `/modelTF/system-info`、`/modelTF/compute/gpus` |
|
||||
| 查看日志 | `/logs`、`/training-log/:id` | 后端日志、error 日志、训练日志索引和内容 | `/modelTF/log-files`、`/modelTF/log-content`、`/modelTF/training-log-*` |
|
||||
| 租户管理 | `/tenants`、`/tenants/:id` | 租户、配额、留存策略 | `/modelTF/tenants/*`、`/modelTF/retention-policies/*` |
|
||||
| 项目空间 | `/projects`、`/projects/:id`、`/projects/:id/members` | 项目、成员、项目角色 | `/modelTF/projects/*` |
|
||||
| 资源授权 | `/resources/:type/:id/acl` 或弹窗 | 模型/数据集/任务 ACL | `/modelTF/resources/{type}/{id}/acl` |
|
||||
| 审批中心 | `/approvals`、`/approval-settings` | 审批待办、审批历史、审批模板 | `/modelTF/approvals/*`、`/modelTF/approval-templates/*` |
|
||||
| 审计中心 | `/audit-logs` | 操作审计、登录审计、导出 | `/modelTF/audit-logs` |
|
||||
|
||||
开发计划:
|
||||
|
||||
| 阶段 | 交付内容 |
|
||||
| --- | --- |
|
||||
| 第 1 周 | 完成登录、当前用户、用户列表、权限码、日志查询接口;完善当前运行 SQL。 |
|
||||
| 第 2 周 | 完成租户、项目、项目成员、资源 ACL 后端和基础页面。 |
|
||||
| 第 3 周 | 完成审批实例、审批模板、审计日志查询和导出。 |
|
||||
| 第 4 周 | 接入其他模块写操作审计和审批拦截,补充权限测试。 |
|
||||
|
||||
验收标准:
|
||||
|
||||
- 所有业务列表按租户、项目、资源 ACL 过滤。
|
||||
- 普通用户无法访问未授权项目、模型和数据集。
|
||||
- 高风险动作有审批或管理员旁路规则。
|
||||
- 日志文件和审计日志可按时间、用户、动作、资源筛选。
|
||||
|
||||
### B. 模型资产、训练与 LLaMA-Factory 任务
|
||||
|
||||
负责人边界:
|
||||
|
||||
- 前端目录:`frontend/src/views/model/`、`frontend/src/views/fine-tune/`。
|
||||
- 前端 API:`frontend/src/api/modules/model.ts`、`frontend/src/api/modules/fineTune.ts`。
|
||||
- 后端模块:`model`、`fine_tune`、`engine_registry`、`compute_gateway` 中的训练编排部分。
|
||||
- 数据库表:`models`、`trained_models`、`fine_tune_tasks`、`fine_tune_metrics`、`fine_tune_checkpoints`、`training_engines`、`training_engine_capabilities`、`compute_jobs`、`gpu_allocations`。
|
||||
|
||||
对应页面和功能:
|
||||
|
||||
| 页面/模块 | 路由 | 功能点 | 接口 |
|
||||
| --- | --- | --- | --- |
|
||||
| 模型管理 | `/model-manage` | 基座模型、API 模型、训练产物列表、筛选、删除审批入口 | `/modelTF/model-manage`、`/modelTF/model-manage/trained-models` |
|
||||
| 添加/编辑模型 | `/model-manage/create`、`/model-manage/:id/edit` | 模型登记、本地路径/API 配置、能力标签 | `/modelTF/model-manage`、`/modelTF/model-manage/{id}` |
|
||||
| 合并权重 | `/model-manage/merge` | LoRA/Adapter 合并、产物登记 | `/modelTF/model-manage/merge` |
|
||||
| 模型训练 | `/fine-tune` | 训练任务列表、状态、启动、停止、删除审批入口 | `/modelTF/fine-tune`、`/modelTF/fine-tune/{id}/start`、`/stop` |
|
||||
| 创建训练任务 | `/fine-tune/create` | 选择模型、数据集、超参、GPU、节点策略 | `/modelTF/fine-tune`、`/modelTF/model-manage`、`/modelTF/dataset-manage`、`/modelTF/compute/*` |
|
||||
| 训练日志 | `/training-log/:id` | 实时日志、loss 曲线、checkpoint、产物 | `/modelTF/fine-tune/{id}/progress`、`/metrics`、`/checkpoints`、`/modelTF/training-log-*` |
|
||||
|
||||
开发计划:
|
||||
|
||||
| 阶段 | 交付内容 |
|
||||
| --- | --- |
|
||||
| 第 1 周 | 完成模型 CRUD、训练任务 CRUD、训练参数校验和接口联调。 |
|
||||
| 第 2 周 | 完成训练启动、停止、状态轮询、日志和指标落库。 |
|
||||
| 第 3 周 | 完成 LLaMA-Factory 参数映射、checkpoint 列表、训练产物登记。 |
|
||||
| 第 4 周 | 完成权重合并、失败恢复、权限隔离和审计接入。 |
|
||||
|
||||
验收标准:
|
||||
|
||||
- 训练任务不能绕过项目、模型、数据集权限。
|
||||
- 训练任务状态以应用侧轮询 Compute API 为主。
|
||||
- 训练命令只能由训练引擎适配层生成,不在页面或应用 API 中拼命令。
|
||||
- checkpoint、日志、产物均可追溯到任务、节点、GPU 和项目。
|
||||
|
||||
### C. 数据集、数据处理、评测与推理
|
||||
|
||||
负责人边界:
|
||||
|
||||
- 前端目录:`frontend/src/views/dataset/`、`frontend/src/views/data-process/`、`frontend/src/views/data-convert/`、`frontend/src/views/eval/`、`frontend/src/views/inference/`、`frontend/src/views/compare/`、`frontend/src/views/tools/`。
|
||||
- 前端 API:`dataset.ts`、`eval.ts`、`compare.ts`,必要时新增 `dataProcess.ts`、`dataConvert.ts`、`inference.ts`。
|
||||
- 后端模块:`dataset`、`data_process`、`eval`、`inference`、`file_gateway` 中的数据资产登记部分。
|
||||
- 数据库表:`datasets`、`dataset_files`、`dataset_file_versions`、`dataset_records`、`data_process_tasks`、`data_process_source_files`、`data_process_preview_items`、`data_process_results`、`data_convert_jobs`、`eval_tasks`、`eval_dimensions`、`eval_sample_results`、`inference_tasks`、`chat_sessions`、`chat_messages`、`custom_tools`。
|
||||
|
||||
对应页面和功能:
|
||||
|
||||
| 页面/模块 | 路由 | 功能点 | 接口 |
|
||||
| --- | --- | --- | --- |
|
||||
| 数据集管理 | `/dataset` | 数据集列表、搜索、版本、下载、删除审批入口 | `/modelTF/dataset-manage` |
|
||||
| 数据集创建/编辑 | `/dataset/create`、`/dataset/:id/edit` | 元数据、文件上传、格式识别、项目归属 | `/modelTF/dataset-manage`、`/upload/{id}` |
|
||||
| 数据集预览 | `/dataset/:id/preview` | 分页预览、在线编辑、版本对比 | `/modelTF/dataset-manage/{id}/preview`、`/versions` |
|
||||
| 数据处理 | `/data-process`、`/data-process/create`、`/data-process/:id` | 文档上传、切片、脱敏、质量评分、发布数据集 | `/modelTF/data-process/*` |
|
||||
| 数据类型转换 | `/data-convert` | JSON/JSONL/Markdown 转换任务 | `/modelTF/data-convert/jobs/*` |
|
||||
| 模型评测 | `/model-eval`、`/model-eval/create`、`/model-eval/:id` | 评测任务、维度、样本级结果、人工复核预留 | `/modelTF/model-eval/*`、`/modelTF/dimension/*` |
|
||||
| 模型推理/对比 | `/model-inference/*`、`/model-compare/*` | 模型加载、对话、对比、结果沉淀 | `/modelTF/model-chat/*`、`/modelTF/model-compare/*` |
|
||||
| 自定义工具 | `/tools`、`/tools/create`、`/tools/:id/edit` | 工具登记、参数 schema、启停 | `/modelTF/tools/*` |
|
||||
|
||||
开发计划:
|
||||
|
||||
| 阶段 | 交付内容 |
|
||||
| --- | --- |
|
||||
| 第 1 周 | 完成数据集 CRUD、上传、预览、版本接口和页面联调。 |
|
||||
| 第 2 周 | 完成数据处理任务、切片预览、质量评分和发布数据集。 |
|
||||
| 第 3 周 | 完成评测任务、评测维度、样本结果查询。 |
|
||||
| 第 4 周 | 完成推理会话、模型对比、数据转换、自定义工具基础能力。 |
|
||||
|
||||
验收标准:
|
||||
|
||||
- 数据文件必须登记存储对象、checksum、版本和项目归属。
|
||||
- 数据处理产物发布为数据集时保留来源链路。
|
||||
- 评测和推理必须记录使用的模型版本、数据集版本和参数快照。
|
||||
- 下载、删除、导出等动作必须接入审计和审批策略。
|
||||
|
||||
### D. 算力平台、部署与运维
|
||||
|
||||
负责人边界:
|
||||
|
||||
- 前端目录:`frontend/src/views/compute/`,协助 `system/HardwareView.vue`。
|
||||
- 前端 API:`frontend/src/api/modules/compute.ts`。
|
||||
- 后端模块:`compute_gateway`、`engine_registry`、`file_gateway`、`system` 中的资源采集部分。
|
||||
- 算力目录:`compute/api/`、`compute/agent/`、`compute/engines/llama_factory/`、`compute/file_gateway/`。
|
||||
- 部署目录:`docker/app/`、`docker/compute/`、`docker/README.md`、`docs/deployment-plan.md`。
|
||||
- 数据库表:`compute_nodes`、`gpu_devices`、`compute_node_engines`、`compute_jobs`、`gpu_allocations`、`resource_replicas`、`resource_sync_jobs`、`system_metric_snapshots`、`storage_objects`。
|
||||
|
||||
对应页面和功能:
|
||||
|
||||
| 页面/模块 | 路由 | 功能点 | 接口 |
|
||||
| --- | --- | --- | --- |
|
||||
| 算力节点 | `/compute`、`/compute?tab=nodes` | 节点地址、File Gateway 地址、权重、标签、启用状态、连接测试 | `/modelTF/compute/nodes/*` |
|
||||
| GPU 资源 | `/compute?tab=gpus`、`/hardware` | GPU 显存、利用率、温度、分配状态、节点归属 | `/modelTF/compute/gpus`、`/modelTF/system-info` |
|
||||
| 任务队列 | `/compute?tab=queue` | 队列、优先级、占用 GPU、任务状态 | `/modelTF/compute/queue` |
|
||||
| 资源副本 | `/compute` 节点详情 | 模型/数据集在算力节点上的同步状态 | `/modelTF/compute/nodes/{id}/replicas` |
|
||||
| 文件网关 | 无独立页面,供模型/数据/训练调用 | 上传、下载、离线导入、产物归档 | 应用侧 `/modelTF/*` 编排,算力侧内部 File Gateway API |
|
||||
| 部署运维 | 文档和 Compose | 应用/算力分离部署、端口、镜像、日志、健康检查 | Docker Compose、健康检查接口 |
|
||||
|
||||
开发计划:
|
||||
|
||||
| 阶段 | 交付内容 |
|
||||
| --- | --- |
|
||||
| 第 1 周 | 完成 Compute API 健康检查、GPU 发现、节点登记和连接测试。 |
|
||||
| 第 2 周 | 完成任务状态查询、应用侧轮询、资源副本状态同步。 |
|
||||
| 第 3 周 | 完成 LLaMA-Factory 容器/宿主机路径适配、日志采集、训练进程管理。 |
|
||||
| 第 4 周 | 完成应用/算力两套 Docker Compose、部署文档、故障排查脚本。 |
|
||||
|
||||
验收标准:
|
||||
|
||||
- 多算力节点阶段仍按“每台算力服务器 = 单机多 GPU 节点”设计。
|
||||
- 每台算力服务器都部署 Compute API、Agent、File Gateway 和 LLaMA-Factory。
|
||||
- 应用服务器只需主动访问所有算力节点,不要求算力节点反向访问应用服务器。
|
||||
- 节点地址、权重、标签、启用状态必须可动态维护。
|
||||
|
||||
## 3. 3 人开发合并方案
|
||||
|
||||
如果团队只有 3 人,建议合并为:
|
||||
|
||||
| 开发人员 | 合并内容 | 不建议合并的原因 |
|
||||
| --- | --- | --- |
|
||||
| A | 平台基础与企业治理 | 该部分是所有模块的权限和隔离底座,不宜再叠加训练或数据主链路。 |
|
||||
| B | 模型资产、训练与 LLaMA-Factory 任务 | 模型和训练强耦合,适合一人端到端打通。 |
|
||||
| C | 数据集、数据处理、评测、推理、算力部署协同 | 数据链路和评测推理使用相同数据/模型资产;算力底层可先由 C 搭骨架,后续扩人拆出 D。 |
|
||||
|
||||
若进入真实 GPU 联调阶段,必须优先把 D 独立出来,否则训练问题、部署问题和业务问题会混在一起,排障效率会明显下降。
|
||||
|
||||
## 4. 公共契约和协作节奏
|
||||
|
||||
公共契约负责人建议由 A 兼任,所有人遵守:
|
||||
|
||||
| 契约 | 文件 | 变更规则 |
|
||||
| --- | --- | --- |
|
||||
| 路由前缀 | `backend/app/core/config.py`、`backend/app/api/v1/router.py`、接口文档 | 统一 `/modelTF`,不得新增 `/api` 前缀 |
|
||||
| 响应结构 | `frontend/src/api/request.ts`、后端 schema | 统一 `{ code, message, data }` |
|
||||
| 权限码 | `frontend/src/types/index.ts`、`permissions` 表、接口文档 | 新菜单先登记权限码再开发 |
|
||||
| 项目隔离 | `project_id`、`tenant_id`、`resource_acl` | 所有模型、数据集、任务必须带项目归属 |
|
||||
| 审计动作 | `audit_logs`、后端审计中间件/服务 | 写操作默认审计 |
|
||||
| 异步状态 | 任务表、`compute_jobs` | 统一 `pending/running/completed/failed/stopped` |
|
||||
| 文件存储 | `storage_objects`、File Gateway | 不暴露宿主机绝对路径给前端 |
|
||||
|
||||
建议节奏:
|
||||
|
||||
- 每周一上午同步接口契约和数据库迁移计划。
|
||||
- 每天下午固定一次跨模块联调窗口,优先处理阻塞其他人的接口。
|
||||
- 每个模块 PR 必须包含页面入口、接口说明、SQL/迁移、最小验证步骤。
|
||||
- 公共文件如 `frontend/src/types/index.ts`、`backend/app/core/*`、`docs/backend-api-design.md` 由对应 owner 统一合并,其他人通过小 PR 提交变更。
|
||||
|
||||
## 5. 里程碑
|
||||
|
||||
| 里程碑 | 目标 | 必须完成 |
|
||||
| --- | --- | --- |
|
||||
| M1 基础可用 | 用户登录、模型/数据集/训练主链路可运行 | A 登录权限,B 模型训练,C 数据集,D 单节点 GPU 状态 |
|
||||
| M2 企业隔离 | 多租户、项目、资源 ACL 接入主链路 | 所有资源按租户/项目过滤,审计落库 |
|
||||
| M3 训练闭环 | LLaMA-Factory 真实训练、日志、checkpoint、产物登记 | 应用侧轮询 Compute API,训练产物可在模型管理查看 |
|
||||
| M4 治理闭环 | 审批、审计、留存、导出、部署文档完善 | 高风险动作审批,审计可检索,应用/算力分离部署可复现 |
|
||||
|
||||
2
frontend/.gitignore
vendored
2
frontend/.gitignore
vendored
@@ -1,5 +1,7 @@
|
||||
node_modules
|
||||
dist
|
||||
!dist/
|
||||
!dist/**
|
||||
dist-ssr
|
||||
*.local
|
||||
|
||||
|
||||
@@ -22,18 +22,17 @@ npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
开发服务器默认运行在 `http://localhost:6801`。
|
||||
开发服务器默认运行在 `http://localhost:16801`。
|
||||
|
||||
后端 API 默认通过 Vite 代理转发到 `http://localhost:7861`(见 `vite.config.ts`)。
|
||||
后端 API 默认通过 Vite 代理转发到 `http://localhost:17861`(见 `vite.config.ts`)。
|
||||
|
||||
开发环境默认启用前端 Mock。如需联调真实后端,使用:
|
||||
开发环境默认联调真实后端接口。如需进行隔离前端开发,可显式启用 Mock:
|
||||
|
||||
```bash
|
||||
VITE_ENABLE_MOCK=false npm run dev
|
||||
VITE_ENABLE_MOCK=true npm run dev
|
||||
```
|
||||
|
||||
生产构建默认不包含 Mock;仅在演示构建中可显式设置
|
||||
`VITE_ENABLE_MOCK=true`。
|
||||
真实联调、测试环境和生产环境不应启用 Mock。
|
||||
|
||||
## 构建
|
||||
|
||||
|
||||
1
frontend/dist/assets/AppConfirmDialog-g7rxg-F7.js
vendored
Normal file
1
frontend/dist/assets/AppConfirmDialog-g7rxg-F7.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{d as C,bn as g,D as E,H as O,o as _,e as B,s as D,Z as $,w as A,c as I,aL as K,q as a,n as v,aa as r,g as q,bm as M,y as u,z as N,P as k}from"./index-BKKvzUDD.js";import{_ as R}from"./_plugin-vue_export-helper-DlAUqK2U.js";const V={class:"app-confirm-header"},z={class:"app-confirm-heading"},H={class:"app-confirm-icon","aria-hidden":"true"},L={class:"app-confirm-body"},P={class:"app-confirm-actions"},S=C({__name:"AppConfirmDialog",setup(j,{expose:x}){const c=u(!1),m=u(),p=u(),d=`app-confirm-title-${g()}`,y=`app-confirm-message-${g()}`,n=N({title:"请确认操作",message:"",confirmText:"确定",cancelText:"取消",tone:"warning",closeOnOverlay:!1});let o=null,i=null;function s(t){c.value=!1;const e=o;o=null,e==null||e(t)}function h(t){return o&&s(!1),Object.assign(n,{confirmText:"确定",cancelText:"取消",tone:"warning",closeOnOverlay:!1,...t}),c.value=!0,new Promise(e=>{o=e})}function w(){n.closeOnOverlay&&s(!1)}function T(t){var b;if(t.key==="Escape"){t.preventDefault(),s(!1);return}if(t.key!=="Tab")return;const e=Array.from(((b=m.value)==null?void 0:b.querySelectorAll("button:not([disabled])"))??[]),l=e[0],f=e[e.length-1];!l||!f||(t.shiftKey&&document.activeElement===l?(t.preventDefault(),f.focus()):!t.shiftKey&&document.activeElement===f&&(t.preventDefault(),l.focus()))}return E(c,async t=>{var e;if(t){i=document.activeElement instanceof HTMLElement?document.activeElement:null,await k(),(e=p.value)==null||e.focus();return}await k(),i==null||i.focus(),i=null}),O(()=>{o==null||o(!1),o=null}),x({open:h}),(t,e)=>(_(),B(M,{to:"body"},[D($,{name:"app-confirm"},{default:A(()=>[c.value?(_(),I("div",{key:0,class:"app-confirm-overlay",onMousedown:K(w,["self"])},[a("section",{ref_key:"dialogRef",ref:m,class:v(["app-confirm-dialog",`is-${n.tone}`]),role:"alertdialog","aria-modal":!0,"aria-labelledby":d,"aria-describedby":y,onKeydown:T},[a("header",V,[a("div",z,[a("span",H,[a("i",{class:v(n.tone==="primary"?"fa fa-question-circle":"fa fa-exclamation-triangle")},null,2)]),a("h2",{id:d},r(n.title),1)]),a("button",{class:"app-confirm-close",type:"button","aria-label":"关闭确认弹窗",onClick:e[0]||(e[0]=l=>s(!1))},[...e[3]||(e[3]=[a("i",{class:"fa fa-times","aria-hidden":"true"},null,-1)])])]),a("div",L,[a("p",{id:y},r(n.message),1)]),a("footer",P,[a("button",{ref_key:"cancelButtonRef",ref:p,class:"app-confirm-button is-cancel",type:"button",onClick:e[1]||(e[1]=l=>s(!1))},r(n.cancelText),513),a("button",{class:"app-confirm-button is-confirm",type:"button",onClick:e[2]||(e[2]=l=>s(!0))},r(n.confirmText),1)])],34)],32)):q("",!0)]),_:1})]))}}),G=R(S,[["__scopeId","data-v-398df98e"]]);export{G as A};
|
||||
1
frontend/dist/assets/AppConfirmDialog-zFD3Iwu_.css
vendored
Normal file
1
frontend/dist/assets/AppConfirmDialog-zFD3Iwu_.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.app-confirm-overlay[data-v-398df98e]{position:fixed;z-index:2000;top:0;right:0;bottom:0;left:0;display:grid;place-items:center;padding:20px;box-sizing:border-box;background:#0f172a70}.app-confirm-dialog[data-v-398df98e]{position:relative;width:min(480px,100%);overflow:hidden;background:#fff;border:1px solid #dfe3ea;border-radius:8px;box-shadow:0 12px 28px #0f172a29}.app-confirm-header[data-v-398df98e]{display:flex;min-height:52px;align-items:center;justify-content:space-between;gap:16px;padding:0 10px 0 20px;border-bottom:1px solid #e7eaf0}.app-confirm-heading[data-v-398df98e]{display:flex;min-width:0;align-items:center;gap:10px}.app-confirm-heading h2[data-v-398df98e]{margin:0;overflow:hidden;color:#273142;font-size:15px;font-weight:650;line-height:1.4;text-overflow:ellipsis;white-space:nowrap}.app-confirm-close[data-v-398df98e]{display:inline-grid;width:32px;height:32px;flex:0 0 32px;place-items:center;padding:0;color:#7b8495;background:transparent;border:0;border-radius:4px;cursor:pointer;transition:color .18s ease,background-color .18s ease}.app-confirm-close[data-v-398df98e]:hover{color:#273142;background:#f2f4f7}.app-confirm-close[data-v-398df98e]:focus-visible{outline:2px solid rgba(91,80,242,.45);outline-offset:1px}.app-confirm-icon[data-v-398df98e]{display:inline-grid;width:28px;height:28px;flex:0 0 28px;place-items:center;color:#a15c07;background:#fff8e6;border:1px solid #f3dfad;border-radius:6px;font-size:13px}.app-confirm-dialog.is-danger .app-confirm-icon[data-v-398df98e]{color:#c43232;background:#fff1f1;border-color:#f2c7c7}.app-confirm-dialog.is-primary .app-confirm-icon[data-v-398df98e]{color:#4f46e5;background:#f3f2ff;border-color:#d9d6ff}.app-confirm-body[data-v-398df98e]{padding:16px 20px 18px}.app-confirm-body p[data-v-398df98e]{margin:0;color:#5f6878;font-size:13px;line-height:1.7}.app-confirm-actions[data-v-398df98e]{display:flex;justify-content:flex-end;gap:8px;padding:10px 14px;background:#f8f9fb;border-top:1px solid #e7eaf0}.app-confirm-button[data-v-398df98e]{height:34px;min-width:72px;padding:0 13px;color:#344054;font-size:13px;font-weight:500;background:#fff;border:1px solid #cfd5df;border-radius:4px;cursor:pointer;transition:border-color .18s ease,background-color .18s ease,color .18s ease}.app-confirm-button[data-v-398df98e]:hover{color:#273142;background:#f2f4f7;border-color:#b9c1cd}.app-confirm-button[data-v-398df98e]:focus-visible{outline:2px solid rgba(91,80,242,.45);outline-offset:1px}.app-confirm-button.is-confirm[data-v-398df98e]{color:#fff;background:#a15c07;border-color:#a15c07}.app-confirm-button.is-confirm[data-v-398df98e]:hover{background:#844b06;border-color:#844b06}.app-confirm-dialog.is-danger .app-confirm-button.is-confirm[data-v-398df98e]{background:#c43232;border-color:#c43232}.app-confirm-dialog.is-danger .app-confirm-button.is-confirm[data-v-398df98e]:hover{background:#a92828;border-color:#a92828}.app-confirm-dialog.is-primary .app-confirm-button.is-confirm[data-v-398df98e]{background:#4f46e5;border-color:#4f46e5}.app-confirm-dialog.is-primary .app-confirm-button.is-confirm[data-v-398df98e]:hover{background:#4338ca;border-color:#4338ca}.app-confirm-enter-active[data-v-398df98e],.app-confirm-leave-active[data-v-398df98e]{transition:opacity .18s ease}.app-confirm-enter-active .app-confirm-dialog[data-v-398df98e],.app-confirm-leave-active .app-confirm-dialog[data-v-398df98e]{transition:opacity .18s ease,transform .18s ease}.app-confirm-enter-from[data-v-398df98e],.app-confirm-leave-to[data-v-398df98e]{opacity:0}.app-confirm-enter-from .app-confirm-dialog[data-v-398df98e],.app-confirm-leave-to .app-confirm-dialog[data-v-398df98e]{opacity:0;transform:translateY(4px)}@media(max-width:520px){.app-confirm-overlay[data-v-398df98e]{padding:12px}.app-confirm-actions[data-v-398df98e]{display:grid;grid-template-columns:repeat(2,minmax(0,1fr))}.app-confirm-button[data-v-398df98e]{height:auto;min-width:0;min-height:44px}}@media(prefers-reduced-motion:reduce){.app-confirm-enter-active[data-v-398df98e],.app-confirm-leave-active[data-v-398df98e],.app-confirm-enter-active .app-confirm-dialog[data-v-398df98e],.app-confirm-leave-active .app-confirm-dialog[data-v-398df98e]{transition:none}}
|
||||
1
frontend/dist/assets/CompareChatView-BZJACmmb.js
vendored
Normal file
1
frontend/dist/assets/CompareChatView-BZJACmmb.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{a as N,E as B}from"./el-form-item-D5kF3B90.js";import{E as K}from"./index-TFUf94PZ.js";import{E as M}from"./index-BjEW7-SA.js";import{E as h}from"./index-BDEF353-.js";import{E as I}from"./el-divider-DjByQoml.js";import{E as R}from"./el-slider-DYENF1-i.js";import{d as F,G as z,e as V,w as l,ac as D,y as L,o as f,s as a,x as p,q as $,c as k,ad as j,aa as E,M as A,g as G,f as J,v as O,z as H,j as _,A as P}from"./index-BKKvzUDD.js";import"./el-popper-D6_hxRbQ.js";import"./el-tooltip-l0sNRNKZ.js";import"./el-input-number-BFR4pu1i.js";/* empty css */import{P as Q}from"./PageCard-BoKXOzst.js";import{u as W}from"./usePolling-C6448AR2.js";import{a as X}from"./compare-CZ4TIoIW.js";import{_ as Y}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./index-GAnQrJsQ.js";import"./debounce-ByXPNh5F.js";import"./toNumber-Dkj3QRv9.js";import"./clamp-CbbY8h6F.js";import"./index-CWUnzf90.js";import"./index-BENk7lZo.js";import"./el-card-CApHJ1Gj.js";const Z={class:"model-list"},tt={key:0,class:"empty-hint"},et=F({__name:"CompareChatView",setup(ot){const b=D(),y=O(),x=b.params.id,n=L(null),e=H({systemPrompt:"",question:"",temperature:.7,topP:.9,topK:40,maxTokens:2048}),m=_(()=>{var s;if(!((s=n.value)!=null&&s.load_status))return[];try{return(typeof n.value.load_status=="string"?JSON.parse(n.value.load_status):n.value.load_status).loaded_models||[]}catch{return[]}}),T=_(()=>m.value.length>0&&m.value.every(s=>s.status==="ready"||s.status==="running")),g=_(()=>m.value.some(s=>s.status==="starting"));async function c(){try{n.value=await X(x)}catch{}}function C(){var u,i;if(!e.question.trim()){P.warning("请输入问题");return}if(g.value){P.warning("模型仍在启动中,请稍候");return}const s=new URLSearchParams({taskId:x,taskName:((u=n.value)==null?void 0:u.model_name)||((i=n.value)==null?void 0:i.name)||"",question:e.question,systemPrompt:e.systemPrompt,temperature:String(e.temperature),topP:String(e.topP),topK:String(e.topK),maxTokens:String(e.maxTokens)}),t=y.resolve(`/model-compare/result?${s.toString()}`).href;window.open(t,"_blank")}const{start:S}=W(c,5e3,{immediate:!1});return z(async()=>{await c(),S()}),(s,t)=>{const u=I,i=h,v=K,r=N,d=R,w=M,q=B;return f(),V(Q,{title:"模型对比配置"},{default:l(()=>[a(u,{"content-position":"left"},{default:l(()=>[...t[7]||(t[7]=[p("已启动模型",-1)])]),_:1}),$("div",Z,[(f(!0),k(A,null,j(m.value,(o,U)=>(f(),V(i,{key:U,type:o.status==="ready"||o.status==="running"?"success":o.status==="starting"?"warning":"danger",size:"large"},{default:l(()=>[p(E(o.model_name)+" ("+E(o.status)+") ",1)]),_:2},1032,["type"]))),128)),m.value.length?G("",!0):(f(),k("span",tt,"暂无已启动模型"))]),a(u,{"content-position":"left"},{default:l(()=>[...t[8]||(t[8]=[p("对话配置",-1)])]),_:1}),a(q,{"label-width":"120px",style:{"max-width":"700px"}},{default:l(()=>[a(r,{label:"系统提示词"},{default:l(()=>[a(v,{modelValue:e.systemPrompt,"onUpdate:modelValue":t[0]||(t[0]=o=>e.systemPrompt=o),type:"textarea",rows:3,placeholder:"可选"},null,8,["modelValue"])]),_:1}),a(r,{label:"问题"},{default:l(()=>[a(v,{modelValue:e.question,"onUpdate:modelValue":t[1]||(t[1]=o=>e.question=o),type:"textarea",rows:4,placeholder:"请输入要对比的问题"},null,8,["modelValue"])]),_:1}),a(r,{label:"Temperature"},{default:l(()=>[a(d,{modelValue:e.temperature,"onUpdate:modelValue":t[2]||(t[2]=o=>e.temperature=o),min:0,max:2,step:.1,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Top-p"},{default:l(()=>[a(d,{modelValue:e.topP,"onUpdate:modelValue":t[3]||(t[3]=o=>e.topP=o),min:0,max:1,step:.05,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Top-k"},{default:l(()=>[a(d,{modelValue:e.topK,"onUpdate:modelValue":t[4]||(t[4]=o=>e.topK=o),min:1,max:100,step:1,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,{label:"Max Tokens"},{default:l(()=>[a(d,{modelValue:e.maxTokens,"onUpdate:modelValue":t[5]||(t[5]=o=>e.maxTokens=o),min:256,max:4096,step:128,"show-input":"",style:{"max-width":"500px"}},null,8,["modelValue"])]),_:1}),a(r,null,{default:l(()=>[a(w,{type:"primary",disabled:!T.value||g.value,onClick:C},{default:l(()=>[...t[9]||(t[9]=[p(" 开始对比 ",-1)])]),_:1},8,["disabled"]),a(w,{onClick:t[6]||(t[6]=o=>J(y).back())},{default:l(()=>[...t[10]||(t[10]=[p("返回",-1)])]),_:1})]),_:1})]),_:1})]),_:1})}}}),qt=Y(et,[["__scopeId","data-v-5da55d96"]]);export{qt as default};
|
||||
1
frontend/dist/assets/CompareChatView-wm3b2ZD9.css
vendored
Normal file
1
frontend/dist/assets/CompareChatView-wm3b2ZD9.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.model-list[data-v-5da55d96]{display:flex;flex-wrap:wrap;gap:12px;margin-bottom:12px}.empty-hint[data-v-5da55d96]{color:#909399}
|
||||
1
frontend/dist/assets/CompareResultView-DlLosvn8.css
vendored
Normal file
1
frontend/dist/assets/CompareResultView-DlLosvn8.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.compare-result[data-v-4c4e03d7]{max-width:1200px;margin:0 auto}.result-header[data-v-4c4e03d7]{display:flex;align-items:center;justify-content:space-between;margin-bottom:16px}.result-header h2[data-v-4c4e03d7]{font-size:18px;font-weight:500;margin:0}.question-box[data-v-4c4e03d7]{margin-bottom:20px}.result-grid[data-v-4c4e03d7]{display:grid;grid-template-columns:repeat(auto-fit,minmax(420px,1fr));gap:16px}.result-card .card-header[data-v-4c4e03d7]{display:flex;align-items:center;justify-content:space-between}.result-card .card-header .model-name[data-v-4c4e03d7]{font-weight:500;color:#303133}.result-card .loading-text[data-v-4c4e03d7],.result-card .error-text[data-v-4c4e03d7]{color:#909399;min-height:80px;display:flex;align-items:center;justify-content:center}.result-card .error-text[data-v-4c4e03d7]{color:#f56c6c}.result-card .streaming-text[data-v-4c4e03d7]{min-height:80px;line-height:1.7;white-space:pre-wrap;word-break:break-word}.result-card .result-stats[data-v-4c4e03d7]{display:flex;gap:16px;margin-top:12px;padding-top:12px;border-top:1px solid #ebeef5;font-size:12px;color:#909399}
|
||||
1
frontend/dist/assets/CompareResultView-yTJ0EdKT.js
vendored
Normal file
1
frontend/dist/assets/CompareResultView-yTJ0EdKT.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as D}from"./el-alert-xVLeNUgJ.js";import{E as $}from"./index-BjEW7-SA.js";import{E as z}from"./index-BDEF353-.js";import{E as F}from"./el-card-CApHJ1Gj.js";import{d as O,G as U,H as J,c as d,q as i,aa as c,f as v,s as N,w as m,M as K,ad as A,ac as G,y as g,o as r,x as p,e as y,g as H}from"./index-BKKvzUDD.js";/* empty css */import{_ as L}from"./MarkdownView.vue_vue_type_style_index_0_lang-BPqmX54I.js";import{a as W,c as j}from"./compare-CZ4TIoIW.js";import{_ as Q}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./vnode-78qeDweP.js";const X={class:"compare-result"},Y={class:"result-header"},Z={class:"header-actions"},tt={class:"result-grid"},et={class:"card-header"},st={class:"model-name"},ot={key:0,class:"error-text"},at={key:1,class:"streaming-text"},nt={key:3,class:"loading-text"},lt={key:4,class:"result-stats"},rt=O({__name:"CompareResultView",setup(it){const u=G(),q=u.query.taskId,k=decodeURIComponent(u.query.question||""),h=decodeURIComponent(u.query.systemPrompt||""),E=Number(u.query.temperature||.7),P=Number(u.query.topP||.9),I=Number(u.query.topK||40),b=Number(u.query.maxTokens||2048),w=u.query.taskName,l=g([]),C=g(!1),x=g([]),f=new Set;async function B(){if(!C.value){C.value=!0;try{const s=await W(q);let t=[];s.load_status&&(t=(typeof s.load_status=="string"?JSON.parse(s.load_status):s.load_status).loaded_models||[]),x.value=t,l.value=t.map(o=>({name:o.model_name||"模型",content:"",displayContent:"",isTyping:!1,status:"loading"})),await Promise.all(t.map((o,e)=>S(o,e)))}catch{}}}async function S(s,t){const o=Date.now();try{const e=await V(j({port:s.port,model_name:s.model_name,messages:[...h?[{role:"system",content:h}]:[],{role:"user",content:k}],temperature:E,top_p:P,top_k:I,max_tokens:b}),3e5),a=(e==null?void 0:e.response)||(e==null?void 0:e.content)||(e==null?void 0:e.data)||JSON.stringify(e),_=(Date.now()-o)/1e3;l.value[t].content=a,l.value[t].status="done",l.value[t].stats={totalTime:_,charsPerSec:_>0?Number((a.length/_).toFixed(1)):0},M(t,a)}catch(e){l.value[t].content="推理失败: "+(e.message||""),l.value[t].status="error"}}async function V(s,t){let o=null;try{return await Promise.race([s,new Promise((e,a)=>{o=setTimeout(()=>a(new Error("推理超时")),t)})])}finally{o&&clearTimeout(o)}}function M(s,t){let o=0;l.value[s].isTyping=!0;const e=Math.max(2,Math.ceil(t.length/30)),a=setInterval(()=>{o+=e,l.value[s].displayContent=t.slice(0,o),o>=t.length&&(clearInterval(a),f.delete(a),l.value[s].displayContent=t,l.value[s].isTyping=!1)},50);f.add(a)}return U(B),J(()=>{f.forEach(clearInterval),f.clear()}),(s,t)=>{const o=$,e=D,a=z,_=F;return r(),d("div",X,[i("div",Y,[i("h2",null,"对比结果"+c(v(w)?` - ${v(w)}`:""),1),i("div",Z,[N(o,{onClick:t[0]||(t[0]=n=>s.$router.push("/model-inference"))},{default:m(()=>[...t[1]||(t[1]=[p("返回列表",-1)])]),_:1})])]),N(e,{type:"info",closable:!1,"show-icon":"",class:"question-box"},{title:m(()=>[t[2]||(t[2]=i("strong",null,"问题:",-1)),p(c(v(k)),1)]),_:1}),i("div",tt,[(r(!0),d(K,null,A(l.value,(n,R)=>(r(),y(_,{key:R,shadow:"hover",class:"result-card"},{header:m(()=>[i("div",et,[i("span",st,c(n.name),1),n.status==="loading"?(r(),y(a,{key:0,type:"warning",size:"small"},{default:m(()=>[...t[3]||(t[3]=[p("生成中...",-1)])]),_:1})):n.status==="done"?(r(),y(a,{key:1,type:"success",size:"small"},{default:m(()=>[...t[4]||(t[4]=[p("完成",-1)])]),_:1})):(r(),y(a,{key:2,type:"danger",size:"small"},{default:m(()=>[...t[5]||(t[5]=[p("失败",-1)])]),_:1}))])]),default:m(()=>{var T;return[n.status==="error"?(r(),d("div",ot,c(n.content),1)):n.isTyping?(r(),d("div",at,c(n.displayContent),1)):n.displayContent?(r(),y(L,{key:2,content:n.displayContent},null,8,["content"])):(r(),d("div",nt,[...t[6]||(t[6]=[i("i",{class:"fa fa-spinner fa-spin"},null,-1),p(" 正在生成回答... ",-1)])])),n.stats?(r(),d("div",lt,[i("span",null,"耗时 "+c((T=n.stats.totalTime)==null?void 0:T.toFixed(1))+"s",1),i("span",null,"速度 "+c(n.stats.charsPerSec)+" 字/秒",1)])):H("",!0)]}),_:2},1024))),128))])])}}}),kt=Q(rt,[["__scopeId","data-v-4c4e03d7"]]);export{kt as default};
|
||||
1
frontend/dist/assets/ComputeNodesView-BKuDW-um.css
vendored
Normal file
1
frontend/dist/assets/ComputeNodesView-BKuDW-um.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.compute-page[data-v-daf09639]{display:flex;flex-direction:column;gap:16px;min-height:0;height:100%;padding:24px;background:#fff}.compute-header[data-v-daf09639]{display:flex;justify-content:space-between;gap:16px;align-items:flex-start}.compute-header h1[data-v-daf09639]{margin:0;font-size:24px;font-weight:650;color:#111827}.compute-header p[data-v-daf09639]{margin:8px 0 0;color:#64748b}.header-actions[data-v-daf09639]{display:flex;align-items:center;gap:12px}.last-updated[data-v-daf09639],.muted[data-v-daf09639]{color:#64748b;font-size:12px}.summary-grid[data-v-daf09639]{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:12px}.summary-tile[data-v-daf09639]{border:1px solid #e5e7eb;border-radius:8px;padding:14px 16px;background:#f8fafc}.summary-tile span[data-v-daf09639]{display:block;color:#64748b;font-size:12px}.summary-tile strong[data-v-daf09639]{display:block;margin-top:8px;color:#111827;font-size:24px}.compute-tabs[data-v-daf09639]{flex:1;min-height:0}.compute-tabs[data-v-daf09639] .el-tabs__content{height:calc(100% - 56px)}.compute-tabs[data-v-daf09639] .el-tab-pane{height:100%}.mono[data-v-daf09639]{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:12px}.replica-toolbar[data-v-daf09639]{display:flex;gap:12px;align-items:center;margin-bottom:12px}.replica-toolbar .el-select[data-v-daf09639]{width:280px}@media(max-width:960px){.compute-header[data-v-daf09639],.header-actions[data-v-daf09639],.replica-toolbar[data-v-daf09639]{flex-direction:column;align-items:stretch}.summary-grid[data-v-daf09639]{grid-template-columns:repeat(2,minmax(0,1fr))}}
|
||||
1
frontend/dist/assets/ComputeNodesView-Cy5hxd1h.js
vendored
Normal file
1
frontend/dist/assets/ComputeNodesView-Cy5hxd1h.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DashboardView-BMPZwafh.css
vendored
Normal file
1
frontend/dist/assets/DashboardView-BMPZwafh.css
vendored
Normal file
File diff suppressed because one or more lines are too long
2
frontend/dist/assets/DashboardView-JKcOpMUU.js
vendored
Normal file
2
frontend/dist/assets/DashboardView-JKcOpMUU.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DataConvertView-DJuX-JIl.css
vendored
Normal file
1
frontend/dist/assets/DataConvertView-DJuX-JIl.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.converter-panel[data-v-fcb69543]{width:100%;min-height:calc(100vh - 220px);border:1px solid #e4e7ed;border-radius:8px;background:#fff;display:flex;flex-direction:column}.panel-header[data-v-fcb69543]{min-height:72px;padding:16px 20px;border-bottom:1px solid #ebeef5;background:#fafafa;display:flex;align-items:center;gap:12px;box-sizing:border-box}.panel-header .tool-icon[data-v-fcb69543]{width:38px;height:38px;flex:0 0 auto;border-radius:6px;background:var(--el-color-primary-light-9);color:var(--primary-color);display:flex;align-items:center;justify-content:center}.panel-header h3[data-v-fcb69543]{margin:0;color:#303133;font-size:15px;font-weight:600}.panel-header p[data-v-fcb69543]{margin:4px 0 0;color:#909399;font-size:12px}.converter-form[data-v-fcb69543]{flex:1;padding:22px 24px 6px}.converter-form[data-v-fcb69543] .el-form-item{margin-bottom:20px}.converter-form[data-v-fcb69543] .el-form-item__label{padding-bottom:8px;color:#606266;font-size:13px}.format-field[data-v-fcb69543]{width:100%;min-height:40px;padding:0 14px;border:1px solid #dcdfe6;border-radius:4px;background:#f5f7fa;color:#303133;display:flex;align-items:center;gap:14px;box-sizing:border-box;font-size:13px;font-weight:500}.format-field i[data-v-fcb69543]{color:#909399}.upload-zone[data-v-fcb69543]{width:100%;min-height:112px;padding:20px;border:1px dashed #b8c4d1;border-radius:6px;background:#fafcff;color:#606266;cursor:pointer;font:inherit;display:flex;align-items:center;gap:14px;text-align:left;transition:border-color .2s ease,background .2s ease}.upload-zone[data-v-fcb69543]:hover,.upload-zone[data-v-fcb69543]:focus-visible{border-color:var(--primary-color);background:var(--el-color-primary-light-9);outline:none}.upload-zone>i[data-v-fcb69543]{color:var(--primary-color);font-size:24px}.upload-zone .upload-content[data-v-fcb69543]{min-width:0;display:flex;flex:1;flex-direction:column;gap:5px}.upload-zone strong[data-v-fcb69543]{color:#303133;font-size:13px;font-weight:500}.upload-zone small[data-v-fcb69543]{color:#909399;font-size:12px}.upload-zone .select-button[data-v-fcb69543]{min-height:32px;padding:0 14px;border:1px solid #dcdfe6;border-radius:4px;background:#fff;color:#606266;display:inline-flex;align-items:center;white-space:nowrap}.form-row[data-v-fcb69543]{display:grid;grid-template-columns:minmax(0,2fr) minmax(180px,1fr);gap:16px}.format-tip[data-v-fcb69543]{min-height:38px;padding:9px 12px;border-radius:4px;background:var(--el-color-primary-light-9);color:#606266;display:flex;align-items:center;gap:8px;box-sizing:border-box;font-size:12px}.format-tip i[data-v-fcb69543]{color:var(--primary-color)}.panel-footer[data-v-fcb69543]{min-height:64px;padding:12px 24px;border-top:1px solid #ebeef5;background:#fafafa;display:flex;align-items:center;justify-content:space-between;gap:20px;box-sizing:border-box}.panel-footer .prototype-label[data-v-fcb69543]{color:#909399;font-size:12px}.panel-footer .actions[data-v-fcb69543]{display:flex;gap:10px}@media(max-width:640px){.converter-form[data-v-fcb69543]{padding:18px 16px 4px}.form-row[data-v-fcb69543]{grid-template-columns:1fr;gap:0}.upload-zone[data-v-fcb69543]{align-items:flex-start;flex-wrap:wrap}.upload-zone .select-button[data-v-fcb69543]{margin-left:38px}.panel-footer[data-v-fcb69543]{padding:12px 16px;align-items:flex-end;flex-direction:column}}
|
||||
1
frontend/dist/assets/DataConvertView-DYHO4l1y.js
vendored
Normal file
1
frontend/dist/assets/DataConvertView-DYHO4l1y.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{a as N,E as v}from"./el-form-item-D5kF3B90.js";import{E as b}from"./el-popper-D6_hxRbQ.js";import{E as O}from"./index-TFUf94PZ.js";import{E as S}from"./index-BjEW7-SA.js";import{E as g,a as E}from"./el-select-Dl-FRZQd.js";import{d as J,e as V,w as e,o as w,q as t,s as l,x as i,z as x,A as C}from"./index-BKKvzUDD.js";import"./el-tooltip-l0sNRNKZ.js";import"./el-scrollbar-ClJnvz-9.js";/* empty css */import{P as U}from"./PageCard-BoKXOzst.js";import{_ as y}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./index-GAnQrJsQ.js";import"./index-CWUnzf90.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./vnode-78qeDweP.js";import"./index-D5ryD3I5.js";import"./index-BDEF353-.js";import"./scroll-BkzZKETR.js";import"./clamp-CbbY8h6F.js";import"./toNumber-Dkj3QRv9.js";import"./_baseIteratee-BAilevJ_.js";import"./el-card-CApHJ1Gj.js";const F={class:"converter-panel"},L={class:"form-row"},T={class:"panel-footer"},B={class:"actions"},k=J({__name:"DataConvertView",setup(I){const a=x({outputName:"converted-data",encoding:"UTF-8"});function p(){C.info("当前仅完成界面设计,转换功能将在后续接入")}function d(){Object.assign(a,{outputName:"converted-data",encoding:"UTF-8"})}return(j,o)=>{const n=N,m=O,u=E,f=g,c=v,r=S,_=b;return w(),V(U,{class:"data-convert-page",title:"数据类型转换",subtitle:"将 JSON 文件转换为便于训练和评测使用的 JSONL 格式"},{default:e(()=>[t("div",F,[o[9]||(o[9]=t("div",{class:"panel-header"},[t("div",{class:"tool-icon","aria-hidden":"true"},[t("i",{class:"fa fa-exchange"})]),t("div",null,[t("h3",null,"JSON 转 JSONL"),t("p",null,"每条 JSON 数据将输出为 JSONL 文件中的一行记录")])],-1)),l(c,{class:"converter-form","label-position":"top"},{default:e(()=>[l(n,{label:"转换类型"},{default:e(()=>[...o[2]||(o[2]=[t("div",{class:"format-field","aria-label":"JSON 转 JSONL"},[t("span",null,"JSON"),t("i",{class:"fa fa-long-arrow-right","aria-hidden":"true"}),t("span",null,"JSONL")],-1)])]),_:1}),l(n,{label:"源文件",required:""},{default:e(()=>[t("button",{class:"upload-zone",type:"button",onClick:p},[...o[3]||(o[3]=[t("i",{class:"fa fa-cloud-upload","aria-hidden":"true"},null,-1),t("span",{class:"upload-content"},[t("strong",null,"点击选择或拖拽 JSON 文件到此处"),t("small",null,"仅支持 .json 格式,单文件不超过 200 MB")],-1),t("span",{class:"select-button"},"选择文件",-1)])])]),_:1}),t("div",L,[l(n,{label:"输出文件名"},{default:e(()=>[l(m,{modelValue:a.outputName,"onUpdate:modelValue":o[0]||(o[0]=s=>a.outputName=s)},{append:e(()=>[...o[4]||(o[4]=[i(".jsonl",-1)])]),_:1},8,["modelValue"])]),_:1}),l(n,{label:"字符编码"},{default:e(()=>[l(f,{modelValue:a.encoding,"onUpdate:modelValue":o[1]||(o[1]=s=>a.encoding=s),style:{width:"100%"}},{default:e(()=>[l(u,{label:"UTF-8",value:"UTF-8"})]),_:1},8,["modelValue"])]),_:1})]),o[5]||(o[5]=t("div",{class:"format-tip"},[t("i",{class:"fa fa-info-circle","aria-hidden":"true"}),t("span",null,"支持由 JSON 数组转换为 JSONL,每个数组元素输出为一行。")],-1))]),_:1}),t("div",T,[o[8]||(o[8]=t("span",{class:"prototype-label"},"当前为 UI 原型,暂不执行实际转换",-1)),t("div",B,[l(r,{onClick:d},{default:e(()=>[...o[6]||(o[6]=[i("重置",-1)])]),_:1}),l(_,{content:"转换功能将在后续开发中接入",placement:"top"},{default:e(()=>[t("span",null,[l(r,{type:"primary",disabled:""},{default:e(()=>[...o[7]||(o[7]=[i("开始转换",-1)])]),_:1})])]),_:1})])])])]),_:1})}}}),rt=y(k,[["__scopeId","data-v-fcb69543"]]);export{rt as default};
|
||||
12
frontend/dist/assets/DataProcessCreateView-BEQjbuB9.js
vendored
Normal file
12
frontend/dist/assets/DataProcessCreateView-BEQjbuB9.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DataProcessCreateView-DyU55R2b.css
vendored
Normal file
1
frontend/dist/assets/DataProcessCreateView-DyU55R2b.css
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DataProcessDetailView-B1q0P6u8.css
vendored
Normal file
1
frontend/dist/assets/DataProcessDetailView-B1q0P6u8.css
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DataProcessDetailView-DrQeE7-I.js
vendored
Normal file
1
frontend/dist/assets/DataProcessDetailView-DrQeE7-I.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DataProcessListView-D6X_r5Q_.css
vendored
Normal file
1
frontend/dist/assets/DataProcessListView-D6X_r5Q_.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.action-buttons[data-v-c0da9336]{display:flex;justify-content:center;gap:8px}
|
||||
1
frontend/dist/assets/DataProcessListView-Ds7sKAJx.js
vendored
Normal file
1
frontend/dist/assets/DataProcessListView-Ds7sKAJx.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as b}from"./index-BjEW7-SA.js";import{E as k}from"./index-BDEF353-.js";import{E as x}from"./el-table-D8K5zLyz.js";import{d as v,c as m,s as e,w as a,y as D,o as l,q as p,x as i,e as C,aa as n,A as E,v as T}from"./index-BKKvzUDD.js";/* empty css */import"./el-checkbox-qGZ2-RM2.js";import{D as z}from"./DataTablePage-C-EM-7-s.js";import{_ as B}from"./ModelStatusTag.vue_vue_type_script_setup_true_lang-VFIQNBU2.js";import{_ as V}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./el-scrollbar-ClJnvz-9.js";import"./index-GAnQrJsQ.js";import"./el-popper-D6_hxRbQ.js";import"./index-CWUnzf90.js";import"./_baseClone-B5RBzbh5.js";import"./_baseIteratee-BAilevJ_.js";import"./castArray-5uErZEc3.js";import"./debounce-ByXPNh5F.js";import"./toNumber-Dkj3QRv9.js";import"./raf-C-x62Pcl.js";import"./omit-BTSq4AYh.js";import"./index-TFUf94PZ.js";import"./index-DLzof2Fz.js";import"./el-card-CApHJ1Gj.js";import"./el-pagination-DOPYW-qb.js";import"./el-select-Dl-FRZQd.js";import"./vnode-78qeDweP.js";import"./index-D5ryD3I5.js";import"./scroll-BkzZKETR.js";import"./clamp-CbbY8h6F.js";import"./directive-DsZckUTJ.js";import"./index-dNo5H1ie.js";import"./validator-Fn0bm4xU.js";import"./index-CQXVR1Ud.js";import"./index-DfyOU94W.js";import"./el-tooltip-l0sNRNKZ.js";const L={class:"data-process-page",style:{height:"100%"}},N={key:1},I={class:"action-buttons"},P=v({__name:"DataProcessListView",setup(S){const u={structured:"结构化数据",unstructured:"非结构化数据",external:"外来数据源拉取"},d=T(),_=D([{id:183921,name:"客服问答数据清洗",status:"completed",process_type:"structured",source_dataset:"客服对话原始集",output_dataset:"客服对话清洗集",create_time:"2026-07-08 14:23:00"},{id:492015,name:"指令微调数据构造",status:"running",process_type:"unstructured",source_dataset:"通用语料库",output_dataset:"SFT 指令集",create_time:"2026-07-09 09:10:00"},{id:731948,name:"敏感信息脱敏处理",status:"pending",process_type:"structured",source_dataset:"用户反馈数据",create_time:"2026-07-09 16:45:00"},{id:582012,name:"多轮对话拼接",status:"failed",process_type:"structured",source_dataset:"单轮问答集",create_time:"2026-07-10 08:30:00"}]);function f(s){const r=s.id;d.push({name:"data-process-detail",params:{id:r}})}function g(s){E.info("删除功能开发中...")}function y(s){return s?new Date(s).toLocaleString("zh-CN",{hour12:!1}):"-"}return(s,r)=>{const o=x,h=k,c=b;return l(),m("div",L,[e(z,{title:"",data:_.value,searchable:"","search-fields":["name"],"create-text":"新建数据处理","create-to":"/data-process/create","row-key":"id","page-size":10},{columns:a(()=>[e(o,{label:"任务ID",prop:"id",align:"center",width:"100"}),e(o,{label:"任务名称",prop:"name",align:"center","show-overflow-tooltip":""}),e(o,{label:"任务状态",align:"center",width:"110"},{default:a(({row:t})=>[e(B,{status:t.status},null,8,["status"])]),_:1}),e(o,{label:"处理类型",align:"center",width:"140"},{default:a(({row:t})=>[t.process_type?(l(),C(h,{key:0,size:"small",type:"info",effect:"plain"},{default:a(()=>[i(n(u[t.process_type]||t.process_type),1)]),_:2},1024)):(l(),m("span",N,"-"))]),_:1}),e(o,{label:"源数据集",align:"center","show-overflow-tooltip":""},{default:a(({row:t})=>[i(n(t.source_dataset||"-"),1)]),_:1}),e(o,{label:"输出数据集",align:"center","show-overflow-tooltip":""},{default:a(({row:t})=>[i(n(t.output_dataset||"-"),1)]),_:1}),e(o,{label:"创建时间",align:"center",width:"190"},{default:a(({row:t})=>[i(n(y(t.create_time)),1)]),_:1})]),actions:a(({row:t})=>[p("div",I,[e(c,{type:"primary",link:"",size:"small",onClick:w=>f(t)},{default:a(()=>[...r[0]||(r[0]=[p("i",{class:"fa fa-file-text-o",style:{"margin-right":"4px"}},null,-1),i("详情 ",-1)])]),_:1},8,["onClick"]),e(c,{type:"danger",link:"",size:"small",onClick:w=>g(t)},{default:a(()=>[...r[1]||(r[1]=[p("i",{class:"fa fa-trash-o",style:{"margin-right":"4px"}},null,-1),i("删除 ",-1)])]),_:1},8,["onClick"])])]),_:1},8,["data"])])}}}),yt=V(P,[["__scopeId","data-v-c0da9336"]]);export{yt as default};
|
||||
1
frontend/dist/assets/DataTablePage-C-EM-7-s.js
vendored
Normal file
1
frontend/dist/assets/DataTablePage-C-EM-7-s.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as M}from"./index-TFUf94PZ.js";import{E as Z}from"./index-BjEW7-SA.js";import{E as A}from"./el-card-CApHJ1Gj.js";import{E as G}from"./el-pagination-DOPYW-qb.js";import{E as H,a as J}from"./el-table-D8K5zLyz.js";import{v as O}from"./directive-DsZckUTJ.js";import{E as b}from"./index-dNo5H1ie.js";import{d as Q,y as f,D as _,c as D,s as p,w as r,o as u,q as s,h as w,e as g,g as v,x as y,aa as k,Z as W,K as X,j as $,v as Y}from"./index-BKKvzUDD.js";import"./el-scrollbar-ClJnvz-9.js";import"./el-popper-D6_hxRbQ.js";/* empty css */import"./el-select-Dl-FRZQd.js";import"./el-tooltip-l0sNRNKZ.js";import"./el-checkbox-qGZ2-RM2.js";import{_ as ee}from"./_plugin-vue_export-helper-DlAUqK2U.js";const te={class:"data-table-page"},ae={class:"table-toolbar"},le={class:"table-title"},oe={class:"toolbar-actions"},se={key:0,class:"batch-bar"},ne={class:"batch-info"},re={class:"table-body"},ie={class:"table-pagination"},ce=Q({__name:"DataTablePage",props:{title:{},data:{},loading:{type:Boolean,default:!1},searchable:{type:Boolean,default:!1},searchFields:{},multiSelect:{type:Boolean,default:!1},createText:{},createTo:{},deleteFn:{},rowKey:{default:"id"},emptyText:{default:"暂无数据"},pageSize:{default:10},pageSizes:{default:()=>[10,20,50,100]}},emits:["create","refresh"],setup(l,{expose:F,emit:V}){const a=l,x=V,P=Y(),S=f(),o=f(new Set),d=f(""),m=f(1),h=f(a.pageSize);function z(t){return t[a.rowKey]??t.name??t.id}const T=$(()=>{var e;if(!d.value.trim()||!((e=a.searchFields)!=null&&e.length))return a.data;const t=d.value.toLowerCase().trim();return a.data.filter(i=>a.searchFields.some(n=>String(i[n]??"").toLowerCase().includes(t)))}),I=$(()=>{const t=(m.value-1)*h.value;return T.value.slice(t,t+h.value)});function K(t){o.value=new Set(t.map(z))}async function E(t,e="确定要删除这条记录吗?"){if(a.deleteFn)try{await b.confirm(e,"确认删除",{type:"warning",confirmButtonText:"确定",cancelButtonText:"取消"}),await a.deleteFn(t),x("refresh")}catch{}}async function N(){if(!(o.value.size===0||!a.deleteFn))try{await b.confirm(`确定要删除选中的 ${o.value.size} 条记录吗?`,"批量删除",{type:"warning"});const t=a.data.filter(n=>o.value.has(z(n)));let e=0,i=0;for(const n of t)try{await a.deleteFn(n),e++}catch{i++}o.value.clear(),x("refresh"),i===0?b.alert(`成功删除 ${e} 条记录`,"成功",{type:"success"}):b.alert(`成功 ${e} 条,失败 ${i} 条`,"部分失败",{type:"warning"})}catch{}}function C(){var t;o.value.clear(),(t=S.value)==null||t.clearSelection()}function L(){a.createTo?P.push(a.createTo):x("create")}return _(()=>a.data,()=>{var t;o.value.clear(),m.value=1,(t=S.value)==null||t.clearSelection()}),_(()=>d.value,()=>{m.value=1}),F({handleDelete:E,clearSelection:C}),(t,e)=>{const i=M,n=Z,B=H,R=J,U=G,j=A,q=O;return u(),D("div",te,[p(j,{shadow:"never",class:"table-card"},{default:r(()=>[s("div",ae,[w(t.$slots,"title",{},()=>[s("h2",le,k(l.title),1)],!0),s("div",oe,[l.searchable?(u(),g(i,{key:0,modelValue:d.value,"onUpdate:modelValue":e[0]||(e[0]=c=>d.value=c),placeholder:`搜索${l.title}...`,clearable:"",class:"search-input"},{prefix:r(()=>[...e[3]||(e[3]=[s("i",{class:"fa fa-search"},null,-1)])]),_:1},8,["modelValue","placeholder"])):v("",!0),l.createText?(u(),g(n,{key:1,type:"primary",onClick:L},{default:r(()=>[e[4]||(e[4]=s("i",{class:"fa fa-plus",style:{"margin-right":"4px"}},null,-1)),y(" "+k(l.createText),1)]),_:1})):v("",!0),w(t.$slots,"toolbar-extra",{},void 0,!0)])]),p(W,{name:"el-zoom-in-top"},{default:r(()=>[l.multiSelect&&o.value.size>0?(u(),D("div",se,[s("span",ne,[e[6]||(e[6]=y(" 已选择 ",-1)),s("strong",null,k(o.value.size),1),e[7]||(e[7]=y(" 项 ",-1)),p(n,{link:"",type:"primary",onClick:C},{default:r(()=>[...e[5]||(e[5]=[y("取消选择",-1)])]),_:1})]),p(n,{type:"danger",size:"small",onClick:N},{default:r(()=>[e[8]||(e[8]=s("i",{class:"fa fa-trash",style:{"margin-right":"4px"}},null,-1)),y(" 批量删除 ("+k(o.value.size)+") ",1)]),_:1})])):v("",!0)]),_:1}),s("div",re,[X((u(),g(R,{ref_key:"tableRef",ref:S,data:I.value,"row-key":l.rowKey,height:"100%",style:{width:"100%"},"empty-text":l.emptyText,onSelectionChange:K},{default:r(()=>[l.multiSelect?(u(),g(B,{key:0,type:"selection",width:"50",align:"center","reserve-selection":"",fixed:"left"})):v("",!0),w(t.$slots,"columns",{},void 0,!0),t.$slots.actions?(u(),g(B,{key:1,label:"操作",align:"center",width:"220",fixed:"right"},{default:r(({row:c})=>[w(t.$slots,"actions",{row:c,handleDelete:E},void 0,!0)]),_:3})):v("",!0)]),_:3},8,["data","row-key","empty-text"])),[[q,l.loading]])]),s("div",ie,[p(U,{"current-page":m.value,"onUpdate:currentPage":e[1]||(e[1]=c=>m.value=c),"page-size":h.value,"onUpdate:pageSize":e[2]||(e[2]=c=>h.value=c),background:"",layout:"total, sizes, prev, pager, next, jumper","page-sizes":l.pageSizes,total:T.value.length},null,8,["current-page","page-size","page-sizes","total"])])]),_:3})])}}}),Te=ee(ce,[["__scopeId","data-v-e6e41965"]]);export{Te as D};
|
||||
1
frontend/dist/assets/DataTablePage-CHPswbwi.css
vendored
Normal file
1
frontend/dist/assets/DataTablePage-CHPswbwi.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
@charset "UTF-8";.data-table-page[data-v-e6e41965]{width:100%;height:100%;box-sizing:border-box;display:flex;flex-direction:column;min-height:0}.data-table-page .table-card[data-v-e6e41965]{margin-bottom:0;width:100%;flex:1;display:flex;flex-direction:column;min-height:0;overflow:hidden}.data-table-page .table-toolbar[data-v-e6e41965]{display:flex;align-items:center;justify-content:space-between;margin-bottom:8px;padding:0 12px;flex-wrap:wrap;gap:12px}.data-table-page .table-title[data-v-e6e41965]{font-size:18px;font-weight:600;color:#1e293b;margin:0;letter-spacing:-.3px}.data-table-page .toolbar-actions[data-v-e6e41965]{display:flex;align-items:center;gap:12px}.data-table-page .search-input[data-v-e6e41965]{width:280px}.data-table-page .search-input[data-v-e6e41965] .el-input__wrapper{border-radius:8px;box-shadow:0 0 0 1px #e2e8f0 inset;transition:all .2s}.data-table-page .search-input[data-v-e6e41965] .el-input__wrapper.is-focus{box-shadow:0 0 0 1px var(--primary-color) inset}.data-table-page .batch-bar[data-v-e6e41965]{display:flex;align-items:center;justify-content:space-between;background:#4f46e50d;border:1px solid rgba(79,70,229,.1);border-radius:8px;padding:10px 16px;margin:0 24px 16px}.data-table-page .batch-bar .batch-info[data-v-e6e41965]{font-size:13px;color:var(--primary-color)}.data-table-page .batch-bar .batch-info strong[data-v-e6e41965]{font-weight:600;margin:0 4px}.data-table-page .table-body[data-v-e6e41965]{flex:1;min-height:0}.data-table-page .table-pagination[data-v-e6e41965]{display:flex;justify-content:flex-end;align-items:center;padding:16px 24px;border-top:1px solid #f1f5f9}.data-table-page .table-pagination[data-v-e6e41965] .el-pagination{margin-top:0}.data-table-page[data-v-e6e41965] .el-card__body{display:flex;flex-direction:column;padding:12px 0 0;flex:1;min-height:0}.data-table-page[data-v-e6e41965] .el-table__inner-wrapper:before{display:none}.data-table-page[data-v-e6e41965] .el-table td.el-table__cell{padding:18px 0!important}.data-table-page[data-v-e6e41965] .el-table__cell .cell{line-height:22px;white-space:nowrap}.data-table-page[data-v-e6e41965] .el-table .action-buttons{display:flex;align-items:center;justify-content:center;flex-wrap:nowrap;gap:8px}
|
||||
2
frontend/dist/assets/DatasetCreateView-ChBCun2m.js
vendored
Normal file
2
frontend/dist/assets/DatasetCreateView-ChBCun2m.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DatasetCreateView-D2g2qvbH.css
vendored
Normal file
1
frontend/dist/assets/DatasetCreateView-D2g2qvbH.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.file-item[data-v-85ac7099]{display:flex;align-items:center;justify-content:space-between;padding:8px 12px;border:1px solid #ebeef5;border-radius:4px;margin-top:8px}.file-info[data-v-85ac7099]{display:flex;align-items:center;gap:8px}.file-name[data-v-85ac7099]{color:#303133}.file-size[data-v-85ac7099]{color:#909399;font-size:12px}.record-count[data-v-85ac7099]{margin-top:8px;font-size:13px;color:#909399}
|
||||
1
frontend/dist/assets/DatasetListView-BD0YX55E.js
vendored
Normal file
1
frontend/dist/assets/DatasetListView-BD0YX55E.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as z}from"./index-BjEW7-SA.js";import{E}from"./index-BDEF353-.js";import{E as T}from"./el-table-D8K5zLyz.js";import{d as L,G as A,e as b,w as e,y as u,j as V,o as k,q as r,s as a,x as s,g as $,aa as p,f as y,b1 as B,b2 as N,n as w,v as P,A as S}from"./index-BKKvzUDD.js";/* empty css */import"./el-checkbox-qGZ2-RM2.js";import{D as M}from"./DataTablePage-C-EM-7-s.js";import{g as R,d as G,a as I}from"./dataset-BJ2ikehQ.js";import{_ as j}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./el-scrollbar-ClJnvz-9.js";import"./index-GAnQrJsQ.js";import"./el-popper-D6_hxRbQ.js";import"./index-CWUnzf90.js";import"./_baseClone-B5RBzbh5.js";import"./_baseIteratee-BAilevJ_.js";import"./castArray-5uErZEc3.js";import"./debounce-ByXPNh5F.js";import"./toNumber-Dkj3QRv9.js";import"./raf-C-x62Pcl.js";import"./omit-BTSq4AYh.js";import"./index-TFUf94PZ.js";import"./index-DLzof2Fz.js";import"./el-card-CApHJ1Gj.js";import"./el-pagination-DOPYW-qb.js";import"./el-select-Dl-FRZQd.js";import"./vnode-78qeDweP.js";import"./index-D5ryD3I5.js";import"./scroll-BkzZKETR.js";import"./clamp-CbbY8h6F.js";import"./directive-DsZckUTJ.js";import"./index-dNo5H1ie.js";import"./validator-Fn0bm4xU.js";import"./index-CQXVR1Ud.js";import"./index-DfyOU94W.js";import"./el-tooltip-l0sNRNKZ.js";const q={class:"capsule-tabs"},O={class:"action-buttons"},U=L({__name:"DatasetListView",setup(Y){const C=P(),c=u(!1),m=u([]),n=u("upload"),D=V(()=>n.value==="task"?m.value.filter(l=>l.source==="task"):m.value.filter(l=>l.source!=="task"));async function f(){c.value=!0;try{m.value=await R()||[]}catch{}finally{c.value=!1}}async function _(l){await G(l.id),S.success("删除成功")}function h(l){C.push(`/dataset/${l.id}/preview`)}function x(l){window.open(I(l.id),"_blank")}return A(f),(l,i)=>{const o=T,g=E,d=z;return k(),b(M,{title:"数据集管理",data:D.value,loading:c.value,searchable:"","search-fields":["name","description"],"create-text":n.value==="upload"?"上传数据集":"","create-to":"/dataset/create","delete-fn":_,"row-key":"id",onRefresh:f},{title:e(()=>[r("div",q,[r("button",{class:w(["capsule-tab-item",{active:n.value==="upload"}]),onClick:i[0]||(i[0]=t=>n.value="upload")}," 上传任务 ",2),r("button",{class:w(["capsule-tab-item",{active:n.value==="task"}]),onClick:i[1]||(i[1]=t=>n.value="task")}," 数据任务 ",2)])]),columns:e(()=>[n.value==="task"?(k(),b(o,{key:0,label:"任务ID",prop:"task_id",align:"center",width:"100"})):$("",!0),a(o,{label:"数据集名称",prop:"name",align:"center"}),a(o,{label:"数据类型",align:"center",width:"110"},{default:e(({row:t})=>[a(g,{type:"primary",size:"small"},{default:e(()=>[s(p(y(B)[String(t.type).toLowerCase()]||t.type||"-"),1)]),_:2},1024)]),_:1}),a(o,{label:"存储位置",align:"center",width:"110"},{default:e(({row:t})=>[a(g,{type:"success",size:"small"},{default:e(()=>[s(p(y(N)[t.storage_type]||t.storage_type||"-"),1)]),_:2},1024)]),_:1}),a(o,{label:"大小",align:"center",width:"100"},{default:e(({row:t})=>[s(p(t.size&&t.size!=="0 B"&&t.size!=="0"?t.size:"-"),1)]),_:1}),a(o,{label:"数据条数",align:"center",width:"100"},{default:e(({row:t})=>[s(p(t.count||0),1)]),_:1}),a(o,{label:"描述",align:"center","show-overflow-tooltip":""},{default:e(({row:t})=>[s(p(t.description||"-"),1)]),_:1}),a(o,{label:"创建时间",align:"center",width:"180"},{default:e(({row:t})=>[s(p(t.create_time?new Date(t.create_time).toLocaleString("zh-CN"):"-"),1)]),_:1})]),actions:e(({row:t})=>[r("div",O,[a(d,{type:"primary",link:"",size:"small",onClick:v=>h(t)},{default:e(()=>[...i[2]||(i[2]=[r("i",{class:"fa fa-eye",style:{"margin-right":"4px"}},null,-1),s("详情 ",-1)])]),_:1},8,["onClick"]),a(d,{type:"success",link:"",size:"small",onClick:v=>x(t)},{default:e(()=>[...i[3]||(i[3]=[r("i",{class:"fa fa-download",style:{"margin-right":"4px"}},null,-1),s("下载 ",-1)])]),_:1},8,["onClick"]),a(d,{type:"danger",link:"",size:"small",onClick:v=>_(t)},{default:e(()=>[...i[4]||(i[4]=[r("i",{class:"fa fa-trash-o",style:{"margin-right":"4px"}},null,-1),s("删除 ",-1)])]),_:1},8,["onClick"])])]),_:1},8,["data","loading","create-text"])}}}),Et=j(U,[["__scopeId","data-v-a2aaedec"]]);export{Et as default};
|
||||
1
frontend/dist/assets/DatasetListView-BZ24MdYw.css
vendored
Normal file
1
frontend/dist/assets/DatasetListView-BZ24MdYw.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
@charset "UTF-8";.capsule-tabs[data-v-a2aaedec]{display:flex;background:#f1f5f9;padding:3px;border-radius:8px;gap:2px;border:1px solid #e2e8f0}.capsule-tab-item[data-v-a2aaedec]{border:0;background:transparent;padding:6px 20px;font-size:13px;font-weight:500;color:#64748b;cursor:pointer;border-radius:6px;transition:all .2s ease;outline:none}.capsule-tab-item[data-v-a2aaedec]:hover{color:#1e293b}.capsule-tab-item.active[data-v-a2aaedec]{background:#fff;color:#4f46e5;box-shadow:0 1px 3px #0000000f,0 1px 2px #0000000a;font-weight:600}
|
||||
6
frontend/dist/assets/DatasetPreviewView-CVHwvkKZ.js
vendored
Normal file
6
frontend/dist/assets/DatasetPreviewView-CVHwvkKZ.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DatasetPreviewView-DO7Vb8Tp.css
vendored
Normal file
1
frontend/dist/assets/DatasetPreviewView-DO7Vb8Tp.css
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DimensionCreateView-DvcDsfrQ.js
vendored
Normal file
1
frontend/dist/assets/DimensionCreateView-DvcDsfrQ.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{a as k,E as M}from"./el-form-item-D5kF3B90.js";import{E as q}from"./index-TFUf94PZ.js";import{E as R}from"./index-BjEW7-SA.js";import{d as S,G as B,e as F,w as i,j as y,y as _,z as I,o as O,s as r,f as m,aB as P,x as h,aa as j,A as x,v as A,ac as L}from"./index-BKKvzUDD.js";import{P as N}from"./PageCard-BoKXOzst.js";import{E as T,D as U}from"./DimensionFormFields-Bhy2Lp-a.js";import{b as z,u as G,c as H}from"./eval-D3O1S_j5.js";import{g as J}from"./model-gCK34aBo.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./index-GAnQrJsQ.js";import"./el-card-CApHJ1Gj.js";import"./_plugin-vue_export-helper-DlAUqK2U.js";import"./el-select-Dl-FRZQd.js";import"./index-CWUnzf90.js";import"./vnode-78qeDweP.js";import"./el-popper-D6_hxRbQ.js";import"./el-scrollbar-ClJnvz-9.js";import"./index-D5ryD3I5.js";import"./index-BDEF353-.js";import"./scroll-BkzZKETR.js";import"./clamp-CbbY8h6F.js";import"./toNumber-Dkj3QRv9.js";import"./_baseIteratee-BAilevJ_.js";import"./el-checkbox-qGZ2-RM2.js";import"./omit-BTSq4AYh.js";import"./el-radio-group-BkCoTeQL.js";import"./el-input-number-BFR4pu1i.js";import"./index-BENk7lZo.js";import"./el-slider-DYENF1-i.js";import"./debounce-ByXPNh5F.js";import"./el-switch-DbCDiaMc.js";import"./validator-Fn0bm4xU.js";import"./el-tooltip-l0sNRNKZ.js";/* empty css *//* empty css */const Be=S({__name:"DimensionCreateView",setup(K){const c=L(),f=A(),n=_(),p=_(!1),u=y(()=>!!c.params.id),s=y(()=>c.params.id),d=_([]);let e=I({name:"",type:"",description:"",eval_model:"",eval_method:"",eval_prompt:"",is_active:!0,is_default:!1,bleu_n:1,output_precision:3,score_min:0,score_max:5,pass_threshold:3});const b={name:[{required:!0,message:"请输入维度名称",trigger:"blur"},{max:50,message:"不超过 50 字符",trigger:"blur"}],type:[{required:!0,message:"请选择指标类型",trigger:"change"}],eval_model:[{required:!0,message:"请选择大模型",trigger:"change"}],eval_method:[{required:!0,message:"请选择评估方式",trigger:"change"}],eval_prompt:[{required:!0,message:"请填写评估 Prompt",trigger:"blur"}]};async function w(){var o,t;if(s.value)try{const a=await z(s.value);Object.assign(e,{name:a.name||"",type:a.type||"",description:a.description||"",eval_model:a.eval_model||"",eval_method:a.eval_method||(a.type?(t=(o=T[a.type])==null?void 0:o[0])==null?void 0:t.value:""),eval_prompt:a.eval_prompt||"",is_active:a.is_active!==!1,is_default:!!a.is_default,bleu_n:a.bleu_n??1,output_precision:a.output_precision??3,score_min:a.score_min??0,score_max:a.score_max??5,pass_threshold:a.pass_threshold??3})}catch{}}async function E(){try{const o=await J()||[];d.value=o.filter(t=>t.purpose==="evaluation")}catch{d.value=[]}}async function D(){n.value&&await n.value.validate(async o=>{if(o){p.value=!0;try{const t={name:e.name,type:e.type,description:e.description,eval_model:e.type==="text_similarity"?null:e.eval_model,eval_method:e.eval_method,eval_prompt:e.type==="text_similarity"?null:e.eval_prompt,is_active:e.is_active,is_default:e.is_default,create_time:new Date().toISOString()};e.type==="text_similarity"&&(t.bleu_n=e.bleu_n,t.output_precision=e.output_precision),e.type==="metric"&&(t.score_min=e.score_min,t.score_max=e.score_max,t.pass_threshold=e.pass_threshold),u.value&&s.value?(await G(s.value,t),x.success("更新成功")):(await H(t),x.success("创建成功")),f.push("/model-eval")}catch{}finally{p.value=!1}}})}function V(){f.back()}return B(()=>{E(),w()}),(o,t)=>{const a=q,v=k,g=R,C=M;return O(),F(N,{title:u.value?"编辑评测维度":"添加评测维度"},{default:i(()=>[r(C,{ref_key:"formRef",ref:n,model:m(e),rules:b,"label-width":"130px",style:{"max-width":"760px"}},{default:i(()=>[r(v,{label:"维度名称",prop:"name"},{default:i(()=>[r(a,{modelValue:m(e).name,"onUpdate:modelValue":t[0]||(t[0]=l=>m(e).name=l),placeholder:"请输入维度名称",maxlength:"50","show-word-limit":""},null,8,["modelValue"])]),_:1}),r(U,{modelValue:m(e),"onUpdate:modelValue":t[1]||(t[1]=l=>P(e)?e.value=l:e=l),"eval-models":d.value},null,8,["modelValue","eval-models"]),r(v,null,{default:i(()=>[r(g,{type:"primary",loading:p.value,onClick:D},{default:i(()=>[h(j(u.value?"保存":"创建"),1)]),_:1},8,["loading"]),r(g,{onClick:V},{default:i(()=>[...t[2]||(t[2]=[h("取消",-1)])]),_:1})]),_:1})]),_:1},8,["model"])]),_:1},8,["title"])}}});export{Be as default};
|
||||
70
frontend/dist/assets/DimensionFormFields-Bhy2Lp-a.js
vendored
Normal file
70
frontend/dist/assets/DimensionFormFields-Bhy2Lp-a.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/DimensionFormFields-jElmd60O.css
vendored
Normal file
1
frontend/dist/assets/DimensionFormFields-jElmd60O.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.method-desc[data-v-c192dba1]{color:#909399;font-size:12px;margin-left:6px}
|
||||
1
frontend/dist/assets/EvalCreateView-DT45yzX7.js
vendored
Normal file
1
frontend/dist/assets/EvalCreateView-DT45yzX7.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/EvalCreateView-DW15zgvp.css
vendored
Normal file
1
frontend/dist/assets/EvalCreateView-DW15zgvp.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.step-form[data-v-41b02b05]{max-width:760px}.field-tip[data-v-41b02b05]{color:#909399;font-size:12px;line-height:20px}.source-alert[data-v-41b02b05]{margin:-4px 0 20px 120px;max-width:640px}.step-form[data-v-4a03e7a1]{max-width:760px}.step-form-wide[data-v-4a03e7a1]{max-width:920px}.basic-metric-form[data-v-55c3e9f0]{max-width:760px}.start-eval-step[data-v-1470a4e6]{max-width:920px}.create-wizard-layout[data-v-63694ec5]{display:flex;flex-direction:column;min-height:620px;margin:-20px;background:#fff}.wizard-main[data-v-63694ec5]{flex:1;padding:32px}.wizard-steps-container[data-v-63694ec5]{margin-bottom:32px;padding-bottom:24px;border-bottom:1px dashed #e2e8f0}.custom-wizard-steps[data-v-63694ec5]{display:flex;align-items:center;justify-content:space-between;max-width:1120px;margin:0 auto;padding:0 20px}.step-item[data-v-63694ec5]{display:flex;flex:none;align-items:center}.step-connector[data-v-63694ec5]{flex:1;height:2px;margin:0 12px;background:#e2e8f0;transition:background-color .2s ease}.step-connector.is-active[data-v-63694ec5]{background:#5146e5}.step-node[data-v-63694ec5]{display:flex;align-items:center;gap:12px}.step-icon[data-v-63694ec5]{display:flex;width:30px;height:30px;flex:0 0 30px;align-items:center;justify-content:center;box-sizing:border-box;border:2px solid #cbd5e1;border-radius:50%;background:#fff;color:#64748b;font-size:13px;font-weight:650;transition:all .2s ease}.step-title[data-v-63694ec5]{color:#64748b;font-size:15px;font-weight:600;white-space:nowrap}.step-description[data-v-63694ec5]{margin-top:3px;color:#94a3b8;font-size:12px;white-space:nowrap}.step-item.is-active .step-icon[data-v-63694ec5]{border-color:#5146e5;background:#eef2ff;color:#5146e5}.step-item.is-active .step-title[data-v-63694ec5],.step-item.is-completed .step-title[data-v-63694ec5]{color:#1e293b}.step-item.is-completed .step-icon[data-v-63694ec5]{border-color:#5146e5;background:#5146e5;color:#fff}.wizard-content[data-v-63694ec5]{max-width:920px;min-height:400px;margin:0 auto}.wizard-footer[data-v-63694ec5]{display:flex;min-height:64px;flex-shrink:0;align-items:center;justify-content:flex-end;gap:12px;padding:0 32px;border-top:1px solid #e2e8f0;background:#fff;box-shadow:0 -4px 6px -1px #0f172a05}.footer-back[data-v-63694ec5]{margin-right:auto}.footer-button-icon[data-v-63694ec5]{margin-right:6px}.footer-button-icon.is-right[data-v-63694ec5]{margin-right:0;margin-left:6px}@media(max-width:1100px){.step-description[data-v-63694ec5]{display:none}}@media(max-width:760px){.wizard-main[data-v-63694ec5]{padding:24px 20px}.custom-wizard-steps[data-v-63694ec5]{padding:0}.step-connector[data-v-63694ec5]{margin:0 8px}.step-text[data-v-63694ec5]{display:none}.wizard-footer[data-v-63694ec5]{padding:0 20px}}
|
||||
1
frontend/dist/assets/EvalDetailView-Clfyt0He.css
vendored
Normal file
1
frontend/dist/assets/EvalDetailView-Clfyt0He.css
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/EvalDetailView-DJayGyvy.js
vendored
Normal file
1
frontend/dist/assets/EvalDetailView-DJayGyvy.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/EvalView-CfhhxItX.css
vendored
Normal file
1
frontend/dist/assets/EvalView-CfhhxItX.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
@charset "UTF-8";.eval-page[data-v-8012e1f7]{width:100%;height:100%;display:flex;flex-direction:column;min-height:0}.capsule-tabs[data-v-8012e1f7]{display:flex;background:#f1f5f9;padding:3px;border-radius:8px;gap:2px;border:1px solid #e2e8f0}.capsule-tab-item[data-v-8012e1f7]{border:0;background:transparent;padding:6px 20px;font-size:13px;font-weight:500;color:#64748b;cursor:pointer;border-radius:6px;transition:all .2s ease;outline:none}.capsule-tab-item[data-v-8012e1f7]:hover{color:#1e293b}.capsule-tab-item.active[data-v-8012e1f7]{background:#fff;color:#4f46e5;box-shadow:0 1px 3px #0000000f,0 1px 2px #0000000a;font-weight:600}
|
||||
1
frontend/dist/assets/EvalView-EyPQOCUX.js
vendored
Normal file
1
frontend/dist/assets/EvalView-EyPQOCUX.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as V}from"./index-BjEW7-SA.js";import{E as B}from"./el-table-D8K5zLyz.js";import{d as D,G as T,c as $,e as _,w as o,g as C,y as p,o as v,s as a,q as i,x as b,aa as L,n as m,v as N,A as z}from"./index-BKKvzUDD.js";import{E as M}from"./index-dNo5H1ie.js";/* empty css */import"./el-checkbox-qGZ2-RM2.js";import{D as y}from"./DataTablePage-C-EM-7-s.js";import{g as A,d as G}from"./eval-D3O1S_j5.js";import{_ as P}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./el-scrollbar-ClJnvz-9.js";import"./index-GAnQrJsQ.js";import"./el-popper-D6_hxRbQ.js";import"./index-CWUnzf90.js";import"./_baseClone-B5RBzbh5.js";import"./_baseIteratee-BAilevJ_.js";import"./castArray-5uErZEc3.js";import"./debounce-ByXPNh5F.js";import"./toNumber-Dkj3QRv9.js";import"./raf-C-x62Pcl.js";import"./validator-Fn0bm4xU.js";import"./index-CQXVR1Ud.js";import"./clamp-CbbY8h6F.js";import"./index-DfyOU94W.js";import"./scroll-BkzZKETR.js";import"./index-TFUf94PZ.js";import"./index-DLzof2Fz.js";import"./omit-BTSq4AYh.js";import"./el-card-CApHJ1Gj.js";import"./el-pagination-DOPYW-qb.js";import"./el-select-Dl-FRZQd.js";import"./vnode-78qeDweP.js";import"./index-D5ryD3I5.js";import"./index-BDEF353-.js";import"./directive-DsZckUTJ.js";import"./el-tooltip-l0sNRNKZ.js";const R={class:"eval-page"},S={class:"capsule-tabs"},q={class:"capsule-tabs"},I=D({__name:"EvalView",setup(Q){const f=N(),r=p("tasks"),c=p(!1),d=p([]),w=p([{rank:1,name:"GPT-4",score:92.5},{rank:2,name:"Claude-3",score:90.1},{rank:3,name:"Qwen-Max",score:85.3}]);async function u(){c.value=!0;try{d.value=await A()||[]}catch{d.value=[]}finally{c.value=!1}}async function k(s){await M.confirm("确定要删除这个评测任务吗?","确认删除",{type:"warning"}),await G(s.id),z.success("删除成功"),u()}function E(){f.push("/model-eval/create")}function n(s){return r.value===s}function x(s){f.push({name:"model-eval-detail",params:{id:s.id}})}return T(()=>{u()}),(s,e)=>{const t=B,g=V;return v(),$("div",R,[r.value==="tasks"?(v(),_(y,{key:0,title:"",data:d.value,loading:c.value,"delete-fn":k,"create-text":"创建评测任务",onCreate:E,"row-key":"id",onRefresh:u},{title:o(()=>[i("div",S,[i("button",{class:m(["capsule-tab-item",{active:n("tasks")}]),onClick:e[0]||(e[0]=l=>r.value="tasks")},"评测任务",2),i("button",{class:m(["capsule-tab-item",{active:n("leaderboard")}]),onClick:e[1]||(e[1]=l=>r.value="leaderboard")},"排行榜",2)])]),columns:o(()=>[a(t,{label:"任务名称",prop:"eval_task_name",align:"center"}),a(t,{label:"评测模型",prop:"model_name",align:"center"}),a(t,{label:"数据集",prop:"dataset",align:"center"}),a(t,{label:"指标",prop:"metric",align:"center"}),a(t,{label:"评分",prop:"score",width:"100",align:"center"}),a(t,{label:"状态",prop:"status",width:"100",align:"center"}),a(t,{label:"创建时间",align:"center",width:"180"},{default:o(({row:l})=>[b(L(l.create_time?new Date(l.create_time).toLocaleString("zh-CN"):"-"),1)]),_:1})]),actions:o(({row:l})=>[a(g,{link:"",type:"primary",size:"small",onClick:h=>x(l)},{default:o(()=>[...e[4]||(e[4]=[i("i",{class:"fa fa-file-text-o",style:{"margin-right":"4px"}},null,-1),b("详情 ",-1)])]),_:1},8,["onClick"]),a(g,{link:"",type:"danger",size:"small",onClick:h=>k(l)},{default:o(()=>[...e[5]||(e[5]=[i("i",{class:"fa fa-trash-o",style:{"margin-right":"4px"}},null,-1),b("删除 ",-1)])]),_:1},8,["onClick"])]),_:1},8,["data","loading"])):C("",!0),r.value==="leaderboard"?(v(),_(y,{key:1,title:"",data:w.value,"row-key":"rank"},{title:o(()=>[i("div",q,[i("button",{class:m(["capsule-tab-item",{active:n("tasks")}]),onClick:e[2]||(e[2]=l=>r.value="tasks")},"评测任务",2),i("button",{class:m(["capsule-tab-item",{active:n("leaderboard")}]),onClick:e[3]||(e[3]=l=>r.value="leaderboard")},"排行榜",2)])]),columns:o(()=>[a(t,{label:"排名",prop:"rank",width:"80",align:"center"}),a(t,{label:"模型名称",prop:"name",align:"center"}),a(t,{label:"综合评分",prop:"score",align:"center"})]),_:1},8,["data"])):C("",!0)])}}}),xe=P(I,[["__scopeId","data-v-8012e1f7"]]);export{xe as default};
|
||||
2
frontend/dist/assets/FineTuneCreateView-B4Xgm6l7.js
vendored
Normal file
2
frontend/dist/assets/FineTuneCreateView-B4Xgm6l7.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/FineTuneCreateView-Dbr8mf56.css
vendored
Normal file
1
frontend/dist/assets/FineTuneCreateView-Dbr8mf56.css
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/FineTuneListView-DB12Hfwf.css
vendored
Normal file
1
frontend/dist/assets/FineTuneListView-DB12Hfwf.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.el-popover{--el-popover-bg-color:var(--el-bg-color-overlay);--el-popover-font-size:var(--el-font-size-base);--el-popover-border-color:var(--el-border-color-lighter);--el-popover-padding:12px;--el-popover-padding-large:18px 20px;--el-popover-title-font-size:16px;--el-popover-title-text-color:var(--el-text-color-primary);--el-popover-border-radius:4px}.el-popover.el-popper{background:var(--el-popover-bg-color);border-radius:var(--el-popover-border-radius);border:1px solid var(--el-popover-border-color);min-width:150px;padding:var(--el-popover-padding);z-index:var(--el-index-popper);color:var(--el-text-color-regular);line-height:1.4;font-size:var(--el-popover-font-size);box-shadow:var(--el-box-shadow-light);overflow-wrap:break-word;box-sizing:border-box}.el-popover.el-popper--plain{padding:var(--el-popover-padding-large)}.el-popover__title{color:var(--el-popover-title-text-color);font-size:var(--el-popover-title-font-size);margin-bottom:12px;line-height:1}.el-popover__reference:focus:not(.focusing),.el-popover__reference:focus:hover{outline-width:0}.el-popover.el-popper.is-dark{--el-popover-bg-color:var(--el-text-color-primary);--el-popover-border-color:var(--el-text-color-primary);--el-popover-title-text-color:var(--el-bg-color);color:var(--el-bg-color)}.el-popover.el-popper:focus:active,.el-popover.el-popper:focus{outline-width:0}.progress-value[data-v-5c76183f]{color:var(--primary-color);font-weight:600}.filter-header[data-v-5c76183f]{display:inline-flex;align-items:center;gap:6px}.filter-badge[data-v-5c76183f]{line-height:1}.filter-icon[data-v-5c76183f]{cursor:pointer;font-size:12px;color:#c0c4cc;transition:color .2s}.filter-icon[data-v-5c76183f]:hover,.filter-icon.active[data-v-5c76183f]{color:#1890ff}.filter-options[data-v-5c76183f]{display:flex;flex-direction:column;gap:8px;max-height:240px;overflow-y:auto}.filter-actions[data-v-5c76183f]{text-align:right;margin-top:8px;border-top:1px solid #ebeef5;padding-top:8px}
|
||||
1
frontend/dist/assets/FineTuneListView-DJrDesBG.js
vendored
Normal file
1
frontend/dist/assets/FineTuneListView-DJrDesBG.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/GuideView-B0JJuoOs.js
vendored
Normal file
1
frontend/dist/assets/GuideView-B0JJuoOs.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{_ as s}from"./_plugin-vue_export-helper-DlAUqK2U.js";import{o as t,c,q as o}from"./index-BKKvzUDD.js";const n={},r={class:"simple-page"};function a(_,e){return t(),c("section",r,[...e[0]||(e[0]=[o("h1",null,"使用文档",-1),o("p",null,"第一版系统已接入后端、数据集、模型、微调任务、算力节点和训练日志主链路。",-1)])])}const p=s(n,[["render",a],["__scopeId","data-v-87551c13"]]);export{p as default};
|
||||
1
frontend/dist/assets/GuideView-CaCcm19L.css
vendored
Normal file
1
frontend/dist/assets/GuideView-CaCcm19L.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.simple-page[data-v-87551c13]{padding:24px}
|
||||
1
frontend/dist/assets/HardwareView-B7yq-27k.js
vendored
Normal file
1
frontend/dist/assets/HardwareView-B7yq-27k.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/HardwareView-DcounnQ-.css
vendored
Normal file
1
frontend/dist/assets/HardwareView-DcounnQ-.css
vendored
Normal file
File diff suppressed because one or more lines are too long
12
frontend/dist/assets/InferenceChatView-BP3Wudle.js
vendored
Normal file
12
frontend/dist/assets/InferenceChatView-BP3Wudle.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/InferenceChatView-dVyzm1FM.css
vendored
Normal file
1
frontend/dist/assets/InferenceChatView-dVyzm1FM.css
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/InferenceCreateView-DrIqOCaE.js
vendored
Normal file
1
frontend/dist/assets/InferenceCreateView-DrIqOCaE.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as T}from"./el-alert-xVLeNUgJ.js";import{a as A,E as D}from"./el-form-item-D5kF3B90.js";import{E as L}from"./index-TFUf94PZ.js";import{E as j}from"./index-BjEW7-SA.js";import{E as z,b as H,a as J}from"./el-select-Dl-FRZQd.js";import{E as K}from"./el-divider-DjByQoml.js";import{d as Q,G as W,e as u,w as o,at as X,y as i,z as Y,o as s,s as a,x as b,c as v,ad as k,M as w,g as Z,j as p,A as B,v as ee}from"./index-BKKvzUDD.js";import"./el-scrollbar-ClJnvz-9.js";import"./el-popper-D6_hxRbQ.js";/* empty css */import{P as te}from"./PageCard-BoKXOzst.js";import{g as le,a as ae}from"./model-gCK34aBo.js";import"./vnode-78qeDweP.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./index-GAnQrJsQ.js";import"./index-CWUnzf90.js";import"./index-D5ryD3I5.js";import"./index-BDEF353-.js";import"./scroll-BkzZKETR.js";import"./clamp-CbbY8h6F.js";import"./toNumber-Dkj3QRv9.js";import"./_baseIteratee-BAilevJ_.js";import"./el-card-CApHJ1Gj.js";import"./_plugin-vue_export-helper-DlAUqK2U.js";const Ie=Q({__name:"InferenceCreateView",setup(oe){const E=ee(),f=i(),c=i(!1),m=i(""),V=i([]),h=i([]),_=i([]),x=p(()=>V.value.map(t=>({key:`db-${t.id}`,id:t.id,name:t.name,source:"database",model_path:t.path||""}))),M=p(()=>h.value.map(t=>({key:`trained-${t.id}`,id:t.id,name:t.name,source:"trained",model_path:t.merged_path||t.base_model_path||"",merged:t.merged,merging:t.merging,disabled:t.merged===!1}))),G=p(()=>{const t={};for(const e of[...x.value,...M.value])t[e.key]=e;return t}),n=Y({name:"",description:"",model_key:"",gpu_id:0}),I={name:[{required:!0,message:"请输入推理名称",trigger:"blur"}],model_key:[{required:!0,message:"请选择模型",trigger:"change"}]},O=p(()=>G.value[n.model_key]);async function S(){f.value&&await f.value.validate(async t=>{if(!t)return;const e=O.value;if(!e){B.warning("请选择模型");return}c.value=!0,m.value="正在启动模型服务...";try{await new Promise(r=>setTimeout(r,1200)),B.success("模型已启动"),E.push({path:"/model-inference/chat/mock",query:{model:e.name}})}finally{c.value=!1,m.value=""}})}function $(){E.back()}async function q(){try{const[t,e,r]=await Promise.all([le(),ae(),X()]);V.value=t||[],h.value=(e==null?void 0:e.models)||[],_.value=(r==null?void 0:r.gpu)||[],_.value.length>0&&(n.gpu_id=0)}catch{}}return W(q),(t,e)=>{const r=L,d=A,F=K,g=J,C=H,P=z,N=T,U=j,R=D;return s(),u(te,{title:"新建推理"},{default:o(()=>[a(R,{ref_key:"formRef",ref:f,model:n,rules:I,"label-width":"100px"},{default:o(()=>[a(d,{label:"推理名称",prop:"name"},{default:o(()=>[a(r,{modelValue:n.name,"onUpdate:modelValue":e[0]||(e[0]=l=>n.name=l),placeholder:"请输入推理名称",maxlength:"50","show-word-limit":"",style:{"max-width":"400px"}},null,8,["modelValue"])]),_:1}),a(d,{label:"描述"},{default:o(()=>[a(r,{modelValue:n.description,"onUpdate:modelValue":e[1]||(e[1]=l=>n.description=l),type:"textarea",rows:2,maxlength:"200","show-word-limit":"",style:{"max-width":"400px"}},null,8,["modelValue"])]),_:1}),a(F,{"content-position":"left"},{default:o(()=>[...e[4]||(e[4]=[b("选择模型",-1)])]),_:1}),a(d,{label:"选择模型",prop:"model_key"},{default:o(()=>[a(P,{modelValue:n.model_key,"onUpdate:modelValue":e[2]||(e[2]=l=>n.model_key=l),placeholder:"请选择模型",filterable:"",style:{width:"400px"}},{default:o(()=>[a(C,{label:"本地模型"},{default:o(()=>[(s(!0),v(w,null,k(x.value,l=>(s(),u(g,{key:l.key,label:l.name,value:l.key},null,8,["label","value"]))),128))]),_:1}),a(C,{label:"已训练模型"},{default:o(()=>[(s(!0),v(w,null,k(M.value,l=>(s(),u(g,{key:l.key,label:l.name+(l.disabled?"(未合并)":""),value:l.key,disabled:l.disabled},null,8,["label","value","disabled"]))),128))]),_:1})]),_:1},8,["modelValue"])]),_:1}),a(d,{label:"GPU"},{default:o(()=>[a(P,{modelValue:n.gpu_id,"onUpdate:modelValue":e[3]||(e[3]=l=>n.gpu_id=l),style:{width:"400px"}},{default:o(()=>[(s(!0),v(w,null,k(_.value,(l,y)=>(s(),u(g,{key:y,label:`${l.name} (GPU${y})`,value:y},null,8,["label","value"]))),128))]),_:1},8,["modelValue"])]),_:1}),m.value?(s(),u(d,{key:0,label:"启动状态"},{default:o(()=>[a(N,{title:m.value,type:"info",closable:!1,"show-icon":""},null,8,["title"])]),_:1})):Z("",!0),a(d,null,{default:o(()=>[a(U,{type:"primary",loading:c.value,onClick:S},{default:o(()=>[...e[5]||(e[5]=[b("开始推理",-1)])]),_:1},8,["loading"]),a(U,{onClick:$},{default:o(()=>[...e[6]||(e[6]=[b("取消",-1)])]),_:1})]),_:1})]),_:1},8,["model"])]),_:1})}}});export{Ie as default};
|
||||
1
frontend/dist/assets/InferenceListView-mZ19jYab.js
vendored
Normal file
1
frontend/dist/assets/InferenceListView-mZ19jYab.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/LoginView-BkI86ygw.js
vendored
Normal file
1
frontend/dist/assets/LoginView-BkI86ygw.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as $,a as q}from"./el-form-item-D5kF3B90.js";import{E as z}from"./index-TFUf94PZ.js";import{u as K,E as P}from"./index-BjEW7-SA.js";import{E as R}from"./el-checkbox-qGZ2-RM2.js";import{b as T,i as U,d as N,u as A,a as D,o as u,c as k,e as h,w as t,r as M,f as w,E as j,g,n as E,h as V,j as b,k as C,l as G,m as H,p as J,q as n,s,t as O,v as Q,x as y,y as B,z as W,A as X}from"./index-BKKvzUDD.js";import{_ as S}from"./logo-CxXS7KxG.js";import{_ as Y}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./index-GAnQrJsQ.js";import"./omit-BTSq4AYh.js";const Z=T({type:{type:String,values:["primary","success","warning","info","danger","default"],default:void 0},underline:{type:[Boolean,String],values:[!0,!1,"always","never","hover"],default:void 0},disabled:Boolean,href:{type:String,default:""},target:{type:String,default:"_self"},icon:{type:U}}),ee={click:l=>l instanceof MouseEvent},ae=["href","target"];var ne=N({name:"ElLink",__name:"link",props:Z,emits:ee,setup(l,{emit:m}){const o=l,i=m,r=A("link");K({scope:"el-link",from:"The underline option (boolean)",replacement:"'always' | 'hover' | 'never'",version:"3.0.0",ref:"https://element-plus.org/en-US/component/link.html#underline"},b(()=>C(o.underline)));const a=D("link"),c=b(()=>{var e;return[a.b(),a.m(o.type??((e=r.value)==null?void 0:e.type)??"default"),a.is("disabled",o.disabled),a.is("underline",d.value==="always"),a.is("hover-underline",d.value==="hover"&&!o.disabled)]}),d=b(()=>{var e;return C(o.underline)?o.underline?"hover":"never":o.underline??((e=r.value)==null?void 0:e.underline)??"hover"});function p(e){o.disabled||i("click",e)}return(e,f)=>(u(),k("a",{class:E(c.value),href:l.disabled||!l.href?void 0:l.href,target:l.disabled||!l.href?void 0:l.target,onClick:p},[l.icon?(u(),h(w(j),{key:0},{default:t(()=>[(u(),h(M(l.icon)))]),_:1})):g("v-if",!0),e.$slots.default?(u(),k("span",{key:1,class:E(w(a).e("inner"))},[V(e.$slots,"default")],2)):g("v-if",!0),e.$slots.icon?V(e.$slots,"icon",{key:2}):g("v-if",!0)],10,ae))}}),le=ne;const oe=G(le),se={class:"login-page"},te={class:"login-panel"},re={class:"login-form-wrap"},ie={class:"login-options"},de=N({__name:"LoginView",setup(l){const m=Q(),o=H(),i=B(),r=B(!1),a=W({username:"",password:""}),c={username:[{required:!0,message:"请输入账号",trigger:"blur"}],password:[{required:!0,message:"请输入密码",trigger:"blur"}]};async function d(){i.value&&await i.value.validate(async p=>{if(p){r.value=!0;try{await o.login(a.username,a.password),X.success("登录成功"),m.push("/dashboard")}catch{}finally{r.value=!1}}})}return(p,e)=>{const f=z,_=q,x=R,F=oe,I=P,L=$;return u(),k("div",se,[e[8]||(e[8]=J('<section class="login-visual" aria-labelledby="platform-title" data-v-713185bd><div class="login-visual-content" data-v-713185bd><div class="login-visual-copy" data-v-713185bd><h1 id="platform-title" data-v-713185bd>远光软件微调平台</h1><p data-v-713185bd>大模型微调、评测与推理的一体化工作台</p></div><div class="login-visual-footer" aria-label="平台核心能力" data-v-713185bd><span data-v-713185bd>数据准备</span><span aria-hidden="true" data-v-713185bd>·</span><span data-v-713185bd>模型训练</span><span aria-hidden="true" data-v-713185bd>·</span><span data-v-713185bd>效果评测</span></div></div></section>',1)),n("main",te,[e[6]||(e[6]=n("div",{class:"brand-lockup",role:"img","aria-label":"远光软件"},[n("span",{class:"brand-logo-crop brand-logo-crop-mark","aria-hidden":"true"},[n("img",{src:S,alt:""})]),n("span",{class:"brand-logo-crop brand-logo-crop-wordmark","aria-hidden":"true"},[n("img",{src:S,alt:""})])],-1)),n("div",re,[e[5]||(e[5]=n("div",{class:"login-heading"},[n("span",null,"账号登录"),n("h2",null,"欢迎回来"),n("p",null,"登录后继续使用微调平台")],-1)),s(L,{ref_key:"loginFormRef",ref:i,model:a,rules:c,"label-position":"top",size:"large",onKeyup:O(d,["enter"])},{default:t(()=>[s(_,{label:"账号",prop:"username"},{default:t(()=>[s(f,{modelValue:a.username,"onUpdate:modelValue":e[0]||(e[0]=v=>a.username=v),placeholder:"请输入账号",clearable:""},null,8,["modelValue"])]),_:1}),s(_,{label:"密码",prop:"password"},{default:t(()=>[s(f,{modelValue:a.password,"onUpdate:modelValue":e[1]||(e[1]=v=>a.password=v),type:"password",placeholder:"请输入密码","show-password":""},null,8,["modelValue"])]),_:1}),n("div",ie,[s(x,null,{default:t(()=>[...e[2]||(e[2]=[y("记住密码",-1)])]),_:1}),s(F,{type:"primary",underline:!1},{default:t(()=>[...e[3]||(e[3]=[y("忘记密码?",-1)])]),_:1})]),s(I,{type:"primary",class:"login-btn",loading:r.value,onClick:d},{default:t(()=>[...e[4]||(e[4]=[y(" 登录 ",-1)])]),_:1},8,["loading"])]),_:1},8,["model"])]),e[7]||(e[7]=n("footer",null,"© 2026 远光软件",-1))])])}}}),Ee=Y(de,[["__scopeId","data-v-713185bd"]]);export{Ee as default};
|
||||
1
frontend/dist/assets/LoginView-BnSabjr1.css
vendored
Normal file
1
frontend/dist/assets/LoginView-BnSabjr1.css
vendored
Normal file
File diff suppressed because one or more lines are too long
3
frontend/dist/assets/LogsView-Bt-ZCtJy.js
vendored
Normal file
3
frontend/dist/assets/LogsView-Bt-ZCtJy.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/LogsView-Np1qgq3l.css
vendored
Normal file
1
frontend/dist/assets/LogsView-Np1qgq3l.css
vendored
Normal file
File diff suppressed because one or more lines are too long
4
frontend/dist/assets/MainLayout-C4Xu6RsD.css
vendored
Normal file
4
frontend/dist/assets/MainLayout-C4Xu6RsD.css
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/MainLayout-D_KBFIU-.js
vendored
Normal file
1
frontend/dist/assets/MainLayout-D_KBFIU-.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/MarkdownView-C0RlwvJo.css
vendored
Normal file
1
frontend/dist/assets/MarkdownView-C0RlwvJo.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.markdown-view{line-height:1.7;word-break:break-word}.markdown-view p{margin:0 0 8px}.markdown-view h1,.markdown-view h2,.markdown-view h3,.markdown-view h4{margin:16px 0 8px;font-weight:600}.markdown-view ul,.markdown-view ol{padding-left:24px;margin:0 0 8px}.markdown-view code{background:#0000000f;padding:2px 6px;border-radius:4px;font-family:SFMono-Regular,Consolas,monospace;font-size:.9em}.markdown-view pre{background:#1e1e1e;color:#d4d4d4;padding:12px 16px;border-radius:6px;overflow-x:auto;margin:0 0 8px}.markdown-view pre code{background:none;padding:0;color:inherit}.markdown-view table{border-collapse:collapse;margin:0 0 8px}.markdown-view table th,.markdown-view table td{border:1px solid #dcdfe6;padding:6px 12px}.markdown-view blockquote{border-left:4px solid #dcdfe6;padding-left:12px;color:#909399;margin:0 0 8px}.markdown-view a{color:#1890ff}
|
||||
58
frontend/dist/assets/MarkdownView.vue_vue_type_style_index_0_lang-BPqmX54I.js
vendored
Normal file
58
frontend/dist/assets/MarkdownView.vue_vue_type_style_index_0_lang-BPqmX54I.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/MergeWeightsView-B8jVfM-U.css
vendored
Normal file
1
frontend/dist/assets/MergeWeightsView-B8jVfM-U.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.model-status[data-v-2e30aa99]{margin-top:20px}
|
||||
1
frontend/dist/assets/MergeWeightsView-D_WkK77Q.js
vendored
Normal file
1
frontend/dist/assets/MergeWeightsView-D_WkK77Q.js
vendored
Normal file
@@ -0,0 +1 @@
|
||||
import{E as A}from"./el-alert-xVLeNUgJ.js";import{a as N,E as T}from"./el-form-item-D5kF3B90.js";import{E as B}from"./index-TFUf94PZ.js";import{E as I}from"./index-BjEW7-SA.js";import{E as R}from"./index-BDEF353-.js";import{d as P,z as q,G as D,e as d,w as o,j as y,y as E,o as n,s as t,f as F,ar as U,x as m,aa as W,c as j,g as z,A as M,v as G,ac as H}from"./index-BKKvzUDD.js";/* empty css */import{P as L}from"./PageCard-BoKXOzst.js";import{a as O,m as S}from"./model-gCK34aBo.js";import{_ as J}from"./_plugin-vue_export-helper-DlAUqK2U.js";import"./vnode-78qeDweP.js";import"./castArray-5uErZEc3.js";import"./_baseClone-B5RBzbh5.js";import"./raf-C-x62Pcl.js";import"./index-DLzof2Fz.js";import"./index-GAnQrJsQ.js";import"./el-card-CApHJ1Gj.js";const K={key:0,class:"model-status"},Q=P({__name:"MergeWeightsView",setup(X){const v=H(),b=G(),s=E(!1),u=y(()=>v.query.model||""),w=y(()=>v.query.method||"lora"),i=E([]),_=y(()=>i.value.find(l=>l.name===u.value)),a=q({model_name:u.value,train_method:w.value,base_model_path:""});async function V(){try{const l=await O();i.value=(l==null?void 0:l.models)||[];const e=i.value.find(p=>p.name===u.value);a.base_model_path=(e==null?void 0:e.base_model_path)||""}catch{}}async function k(){if(!a.model_name||!a.base_model_path){M.warning("缺少模型信息");return}s.value=!0;try{await S({model_name:a.model_name,train_method:a.train_method,base_model_path:a.base_model_path}),M.success("合并成功"),b.push("/model-manage")}catch{}finally{s.value=!1}}function x(){b.back()}return D(V),(l,e)=>{const p=A,f=B,r=N,h=I,C=T,c=R;return n(),d(L,{title:"合并权重"},{default:o(()=>[t(p,{type:"info",closable:!1,"show-icon":"",title:"将 LoRA 适配器权重合并到基座模型,合并后可直接用于推理部署。",style:{"margin-bottom":"20px"}}),t(C,{"label-width":"120px",style:{"max-width":"600px"}},{default:o(()=>[t(r,{label:"模型名称"},{default:o(()=>[t(f,{modelValue:a.model_name,"onUpdate:modelValue":e[0]||(e[0]=g=>a.model_name=g),disabled:""},null,8,["modelValue"])]),_:1}),t(r,{label:"训练方法"},{default:o(()=>[t(f,{"model-value":F(U)[a.train_method]||a.train_method,disabled:""},null,8,["model-value"])]),_:1}),t(r,{label:"基座模型路径"},{default:o(()=>[t(f,{modelValue:a.base_model_path,"onUpdate:modelValue":e[1]||(e[1]=g=>a.base_model_path=g),placeholder:"基座模型路径"},null,8,["modelValue"])]),_:1}),t(r,null,{default:o(()=>[t(h,{type:"primary",loading:s.value,onClick:k},{default:o(()=>[m(W(s.value?"合并中...":"开始合并"),1)]),_:1},8,["loading"]),t(h,{onClick:x},{default:o(()=>[...e[2]||(e[2]=[m("取消",-1)])]),_:1})]),_:1})]),_:1}),_.value?(n(),j("div",K,[_.value.merging?(n(),d(c,{key:0,type:"warning"},{default:o(()=>[...e[3]||(e[3]=[m("合并中",-1)])]),_:1})):_.value.merged?(n(),d(c,{key:1,type:"success"},{default:o(()=>[...e[4]||(e[4]=[m("已合并",-1)])]),_:1})):(n(),d(c,{key:2,type:"info"},{default:o(()=>[...e[5]||(e[5]=[m("未合并",-1)])]),_:1}))])):z("",!0)]),_:1})}}}),fe=J(Q,[["__scopeId","data-v-2e30aa99"]]);export{fe as default};
|
||||
1
frontend/dist/assets/ModelCreateView-Np2j_mdr.js
vendored
Normal file
1
frontend/dist/assets/ModelCreateView-Np2j_mdr.js
vendored
Normal file
File diff suppressed because one or more lines are too long
1
frontend/dist/assets/ModelManageView-B__x03pD.css
vendored
Normal file
1
frontend/dist/assets/ModelManageView-B__x03pD.css
vendored
Normal file
@@ -0,0 +1 @@
|
||||
@charset "UTF-8";.capsule-tabs[data-v-f06fa847]{display:flex;background:#f1f5f9;padding:3px;border-radius:8px;gap:2px;border:1px solid #e2e8f0}.capsule-tab-item[data-v-f06fa847]{border:0;background:transparent;padding:6px 20px;font-size:13px;font-weight:500;color:#64748b;cursor:pointer;border-radius:6px;transition:all .2s ease;outline:none}.capsule-tab-item[data-v-f06fa847]:hover{color:#1e293b}.capsule-tab-item.active[data-v-f06fa847]{background:#fff;color:#4f46e5;box-shadow:0 1px 3px #0000000f,0 1px 2px #0000000a;font-weight:600}.action-buttons[data-v-f06fa847]{display:flex;justify-content:center;gap:8px}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user