Complete AI TrainingYourJobSkills for your job

Skills / ml-ops

remote-gpu-trainer

Deploy, monitor, and debug long GPU jobs on RENTED/remote instances (AutoDL, RunPod, vast.ai, Lambda, Slurm, K8s): teardown/billing safety, spot resilience, resumable checkpointing, OOM/NaN triage.

AI & Automation

remote-gpu-trainer — Remote GPU Job Orchestration

Overview

Deploy and babysit long-running GPU jobs on rented boxes you don't own, across any platform, and get the result off the box before the meter or a preemption kills it. The core insight: you are a short-term tenant on someone else's machine — so the job is to detach the work, make the result outlive the instance, and stop the meter safely, not to provision a cluster.

This skill is platform-agnostic at the core, platform-specific at the edges: a fixed set of operating principles + a 6-phase lifecycle that hold everywhere, plus one profile per platform (profiles/<platform>.md) that owns every concrete path, proxy, billing verb, and spot semantic. Its defensible value is the union the big orchestrators skip: Chinese cgroup-isolated rentals + bare-SSH cheap boxes + the disk-budget / monitoring / teardown reality that is the job on metered hardware.

When to Use This Skill

Use whenever the user deploys, trains, monitors, or troubleshoots a long-running GPU job on a RENTED or remote instance they do not own — training, eval, ablation sweeps, batch inference, or large data processing — on AutoDL, RunPod, vast.ai, Lambda, Paperspace, Chinese platforms (恒源云/矩池云/Featurize/ 揽睿星舟), a bare SSH box, Slurm, or Kubernetes; single OR multi-instance. Triggers (multilingual): "远程 GPU 训练", "GPU 租赁",

Subscribers only

The full skill, its 46 bundled files and every download is included with every paid Complete AI plan.

Details

SourceHanyuyuan6/remote-gpu-trainer
LicenseMIT
Risk labelsafe ("critical" means the skill may run commands or touch files — read before use)
FilesLICENSE, README.md, SKILL.md, evals/README.md, evals/RESULTS.md, evals/cases.jsonl, evals/run_evals.py, examples/autodl_sweep/README.md, examples/autodl_sweep/queue_1.txt, profiles/_schema.md, profiles/autodl.md, profiles/china.md, profiles/generic-ssh.md, profiles/lambda.md, profiles/paperspace.md, profiles/runpod.md, profiles/vastai.md, references/china-network.md, references/gotchas_universal.md, references/lifecycle_checklist.md, references/monitoring_patterns.md, references/multinode.md, references/parallel_ablation.md, references/principles.md, references/self-improvement.md, references/spot-resilience.md, references/ssh_transport.md, references/training/by-domain.md, references/training/checkpoint-resume.md, references/training/convergence-debugging.md, references/training/data-pipeline.md, references/training/distributed-launch.md, references/training/oom-memory.md, references/training/precision-stability.md, references/training/throughput-profiling.md, scripts/aggregate_to_fs.sh, scripts/check_staleness.py, scripts/download_loop.sh, scripts/gpu_health.sh, scripts/health_patrol.sh.template, scripts/mem_monitor.sh, scripts/reap_vram_zombies.sh, scripts/run_one.sh.template, scripts/run_queue.sh.template, scripts/setup-china-mirrors.sh, scripts/verify_local.py
Added2026-06-20