mirror of
https://github.com/dante-gpu/dantegpu-core.git
synced 2026-11-02 10:08:03 +09:00
0raclus
1672858674
Adds the "real model on a rented GPU" proof the rental-proof was building toward.
scripts/gpu-model-proof.sh — one command:
- ensures mlx-lm in a throwaway venv (Apple Silicon / Metal),
- starts a JetStream nats-server + the provider-daemon,
- dispatches an ai-inference task over NATS whose payload (scripts/gpu_infer.py)
loads a real 4-bit quantized LLM and generates text on the GPU,
- prints the daemon-captured model output and tears everything down.
Verified on an Apple M4 Max: the daemon receives the task over NATS, runs
mlx-community/Llama-3.2-1B-Instruct-4bit on the Metal GPU (device "Device(gpu, 0)",
~235 tokens/sec, 0.81 GB GPU peak), and returns a real coherent answer with
exitCode 0. So the full chain is real end to end: detect GPU -> rent -> dispatch
over NATS -> daemon executes -> real model runs on the GPU -> output returned.
scripts/gpu_infer.py is the MLX inference payload (swap --model for any
mlx-community repo; a PyTorch payload works for CUDA/NVIDIA). docs/PROOF.md and
the README document the run; the only thing still unproven is *distributed*
multi-node inference (one model sharded across many rented GPUs).
DanteGPU Core documentation
Start with the root README for what the system is and the honest status. This folder holds the supporting docs.
| Doc | What it covers |
|---|---|
| STATUS.md | Per-service reality matrix (working / partial / stub / removed) |
| ARCHITECTURE.md | System architecture, services, data flow |
| ARCHITECTURE-TR.md | Architecture, Turkish |
| API_DOCUMENTATION.md | HTTP API reference |
| RUNBOOK.md | Operations runbook |
Project-level docs live at the repository root: README, ROADMAP, CONTRIBUTING.
Service-specific notes live next to each service (for example
billing-payment-service/README.md).