feat: 更新后端平台模块、Compute引擎、前端组件及构建产物

- 更新 backend 平台 API、platform_store、compute_gateway sync
- 更新 compute agent/engine/adapter 及 API
- 更新 Docker 部署配置(app/compute)
- 新增 frontend/src/utils/ 工具模块
- 新增 scripts/ops_diagnostics.py 运维诊断脚本
- 新增 docs/2026-07-23-development-summary.md 开发总结
- 重构 frontend/dist 构建产物(新 hash)
- 更新前端多个视图组件及 API 模块

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
wuyongtao
2026-07-23 19:32:42 +08:00
parent f04dc479bb
commit b28cfbc6fa
193 changed files with 2647 additions and 424 deletions

View File

@@ -701,7 +701,9 @@ YG_FT/
职责:
- 生成训练 YAML 或 CLI。
- 生成 dataset_info。
- 根据平台数据集 ID 生成 LLaMA-Factory dataset key 和 `{dataset_dir}/dataset_info.json`
- 单文件数据集使用一个 dataset key多文件数据集拆分为多个 key 并通过逗号组合传入 `--dataset`
- 正式启动前由应用侧同步当前数据集文件到目标算力节点,确保训练使用最新激活版本。
- 设置 `CUDA_VISIBLE_DEVICES`
- 启动训练。
- 解析日志。
@@ -1116,3 +1118,10 @@ YG_FT/
| P1 | 数据处理、评测、推理、模型服务治理 | 形成训练前后闭环 |
| P1 | 存储管理、checkpoint、审计中心 | 企业治理和运维收口 |
| P2 | 训练引擎管理、外部 API、OIDC/LDAP、成本核算、人工评测 | 扩展能力,第一版可预留 |
## P1/P2/P3 Development Status
| Priority | Scope | Implemented content | Remaining production hardening |
| --- | --- | --- | --- |
| P1 | Training runtime assets | Compute scans real `checkpoint-*` directories; backend persists checkpoints to `fine_tune_checkpoints`; backend parses log metrics into `fine_tune_metrics`; completed training auto-registers a trained model with its real output directory. | Add full TensorBoard event parsing and retention/cleanup execution policies. |
| P2 | Model merge/export | `/modelTF/model-manage/merge` submits `llamafactory-cli export` through Compute; merge/export jobs are recorded in `compute_jobs` and `model_export_jobs`; trained model artifacts are recorded in `model_artifacts`; base->trained and merge/export relations are recorded in `model_lineage`; APIs expose artifacts, lineage and export jobs. | Add frontend polling/status panels for export jobs and richer artifact checksum generation during long-running export. |
| P3 | Compute ops | `compute_jobs` and `gpu_allocations` are updated from runtime polling; training start uses `scheduler_locks` to serialize node selection and GPU pre-allocation; terminal task states release GPU allocations; resource replica drift check and repair-pending APIs are available; `scripts/ops_diagnostics.py` checks backend, PostgreSQL, Compute health, GPU discovery and Compute jobs. | Add dedicated scheduler worker, automatic replica repair execution from authoritative source metadata, and lock timeout alarms. |