mirror of
https://github.com/dante-gpu/dantegpu-core.git
synced 2026-11-02 10:08:03 +09:00
0raclus
1672858674
Adds the "real model on a rented GPU" proof the rental-proof was building toward.
scripts/gpu-model-proof.sh — one command:
- ensures mlx-lm in a throwaway venv (Apple Silicon / Metal),
- starts a JetStream nats-server + the provider-daemon,
- dispatches an ai-inference task over NATS whose payload (scripts/gpu_infer.py)
loads a real 4-bit quantized LLM and generates text on the GPU,
- prints the daemon-captured model output and tears everything down.
Verified on an Apple M4 Max: the daemon receives the task over NATS, runs
mlx-community/Llama-3.2-1B-Instruct-4bit on the Metal GPU (device "Device(gpu, 0)",
~235 tokens/sec, 0.81 GB GPU peak), and returns a real coherent answer with
exitCode 0. So the full chain is real end to end: detect GPU -> rent -> dispatch
over NATS -> daemon executes -> real model runs on the GPU -> output returned.
scripts/gpu_infer.py is the MLX inference payload (swap --model for any
mlx-community repo; a PyTorch payload works for CUDA/NVIDIA). docs/PROOF.md and
the README document the run; the only thing still unproven is *distributed*
multi-node inference (one model sharded across many rented GPUs).