GPU Tunnels

Run open-weight models on your own GPU and call them through the normal Tarqa API, at $0 Tarqa token cost. A small agent on your machine connects out to Tarqa and passes requests to a model server you already run locally (Ollama, LM Studio or vLLM).

Set it up

1. 1
   Register the machine

   In the dashboard, open Build → GPU Tunnel and click Register Agent. Give it a name, OS, GPU info and total memory. This registers the machine, not a model yet.
2. 2
   Download the config and start the agent

   The dashboard shows your Agent ID and a tarqa-tunnel.config.json filled in with your details. Save it, then run the agent on the GPU machine:

   cURL

   Copy

   cd packages/tunnel-agent
   npm install
   npm run build
   node dist/src/cli.js start --config ~/.tarqa/tarqa-tunnel.config.json

3. 3
   Register a model

   When the agent shows Online, click Register Model. Pick the agent, choose one of the local models it found (or type one in), and set the local endpoint, e.g. http://localhost:11434 for Ollama.
4. 4
   Use it

   The model appears in the chat model picker as hosted-<id> with a tunnel badge. Every request goes over the tunnel to your machine and back.

Call it from the API

Use the hosted-<id> shown in the dashboard (not the raw local model name) like any other model:

curl -X POST "https://api.tarqaai.com/api/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hosted-<your-model-id>",
    "messages": [{"role": "user", "content": "Hello from local GPU!"}]
  }'

Tool calling works too, with the same tools / tool_calls format, as long as the local server is OpenAI-compatible (LM Studio, vLLM). Ollama doesn't forward tool definitions yet.

Security

FAQ