Local inference and operations

Keep the app and model on your machine.

IdeaForge can use Ollama, LM Studio, or another OpenAI-compatible server on loopback. The simplest and most reliable arrangement serves the static app locally too, so the browser never has to cross from a public HTTPS page into a local HTTP service.

Choose a topology

Run from a clone without Docker

There are no dependencies and no build step. Node is used for the repository tools; the app itself only needs a static server over a real origin.

npm run serve
# open http://127.0.0.1:8765

Do not open index.html with file://. ES modules, IndexedDB, installability, and the service worker all depend on an origin.

1. Start Ollama, LM Studio, or another OpenAI-compatible server.
2. Open http://127.0.0.1:8765.
3. Select the matching local provider.
4. Prefer a server address such as http://127.0.0.1:11434/v1. Press Check the connection.
5. Choose an installed model from the suggestions, or type its exact identifier. Leave the wrap-up field blank and the same model runs that call too.

Run the source Compose stack

docker compose -f docker/compose.yml up -d
docker compose -f docker/compose.yml exec ollama ollama pull <model-name>
# open http://127.0.0.1:8765

The app is built from the current tree with docker/Dockerfile.app. The image copies only the publishable static tree into nginx. Ollama's model store is a named volume, so an ordinary docker compose down does not discard downloaded model data. Do not add -v unless removing that volume is intentional.

If port 11434 is already occupied, publish the container on a different host port and put that same address in IdeaForge:

IDEAFORGE_OLLAMA_PORT=11435 docker compose -f docker/compose.yml up -d
# Server address: http://127.0.0.1:11435/v1

In PowerShell, set the variable first:

$env:IDEAFORGE_OLLAMA_PORT = '11435'
docker compose -f docker/compose.yml up -d

Run published images

The standalone release Compose file needs no clone and no local app build:

curl -fsSLO https://raw.githubusercontent.com/jaypetez/ideaforge/main/docker/compose.release.yml
docker compose -f compose.release.yml up -d
docker compose -f compose.release.yml exec ollama ollama pull <model-name>
# open http://127.0.0.1:8765

To refresh an existing installation, pull before recreating the services:

docker compose -f compose.release.yml pull
docker compose -f compose.release.yml up -d

The release file shares the same Compose project and model-volume name as the source stack. This avoids downloading the model again when switching between a released app and a locally built one, but it also means the two stacks should not run at the same time.

Containerise only the static app

docker run -d --name ideaforge \
  -p 127.0.0.1:8765:80 \
  ghcr.io/jaypetez/ideaforge:latest

Run Ollama or LM Studio natively, then point IdeaForge at its loopback API. This is the practical route when the host's native model runtime has the right GPU support and a Linux container does not, including the normal Apple Silicon arrangement.

GPU and CPU behaviour

Both repository Compose definitions reserve NVIDIA GPU devices. Docker does not silently ignore a missing NVIDIA device driver: service creation fails with a clear device-driver error. That is intentional, because an unnoticed CPU fallback looks like an extremely slow application rather than a deployment fault.

• For the release file, follow the marked GPU BLOCK instructions in the file if you deliberately want CPU-only Ollama.
• For other GPU stacks, prefer native Ollama plus the app-only container, or maintain a local Compose override appropriate to that host. The repository file does not claim to configure every GPU runtime.
• A CPU model can still answer correctly, but cold loads and long prompts may take much longer. Use a model that fits available memory rather than treating repeated request deadlines as a browser problem.

See Ollama's Docker guidance and Docker's Compose GPU guidance for host prerequisites.

Use an address IdeaForge can safely reach

CORS, local-network permission, and Safari

When the app is local

A page at http://127.0.0.1:8765 calling a loopback model stays within the local address space. Ollama's normal localhost origins cover this route. LM Studio still needs CORS enabled for browser access.

When the app is hosted over HTTPS

Two independent gates must pass:

1. The model server must allow the page's origin. For Ollama, set OLLAMA_ORIGINS to the exact origin that hosts IdeaForge and restart Ollama. The origin has a scheme and host, but no path; for the public Pages deployment it is https://jaypetez.github.io. Ollama reads the environment when it starts, so changing another shell without restarting the running service has no effect.
2. Chrome must grant Local Network Access. IdeaForge marks the fetch with targetAddressSpace: "local". Chrome shows its permission prompt only after you explicitly press Check the connection; allow it for the site.

LM Studio exposes CORS in its server settings and through the lms server start options. Ollama documents allowed origins in its official FAQ. Chrome documents the current permission model in Local Network Access.

Validate a real model, not a graceful fallback

IdeaForge is designed to continue from its built-in question bank when inference fails. That resilience makes a superficial end-to-end check unsafe: an interview can finish and export even if no model request succeeded. The repository's validate:local harness therefore checks both application state and what actually left the browser.

Run against a native or already-running Ollama

In a POSIX shell:

OLLAMA_URL=http://127.0.0.1:11434 \
IDEAFORGE_MODEL=<model-name> \
npm run validate:local

In PowerShell:

$env:OLLAMA_URL = 'http://127.0.0.1:11434'
$env:IDEAFORGE_MODEL = '<model-name>'
npm run validate:local

Validate a running app container

Set IDEAFORGE_URL so the harness drives that origin instead of serving the working tree:

$env:IDEAFORGE_URL = 'http://127.0.0.1:8765'
$env:OLLAMA_URL = 'http://127.0.0.1:11434'
$env:IDEAFORGE_MODEL = '<model-name>'
npm run validate:local

Run the containerised validator

docker compose -f docker/compose.yml -f docker/compose.validate.yml run --rm validate

The validation container shares Ollama's network namespace. That makes 127.0.0.1:11434 genuinely point to Ollama inside the container while preserving IdeaForge's loopback-only policy. It does not create a special test-only remote-server exception.

What the validator proves

• Ollama answers, and the requested model is actually pulled.
• A host name does not resolve to two different Ollama servers with different model fingerprints.
• The model loads before the browser run, and its reported model size is resident in GPU memory when GPU validation is required.
• A warm decode sample is fast enough for the configured memory-bandwidth floor, without misclassifying a two-token first response as a speed test.
• The real app boots with no uncaught error or CSP violation.
• The browser reaches the same Ollama instance as the Node harness.
• The app reads the real model list and sees the selected model.
• Every model-sourced question and the final synthesis produce actual browser POST requests; saved questionSource values agree.
• No turn silently falls back to the checklist, the refined prompt is generated, and the model remains in VRAM through the wrap-up when still loaded.

Useful validation controls

For the containerised run, pass controls that are not declared in the Compose file with docker compose run -e, for example -e VALIDATE_MODE=handsfree or -e VALIDATE_ALLOW_CPU=1.

Implementation references

The development stack is docker/compose.yml, the standalone release stack is docker/compose.release.yml, and the validation overlay is docker/compose.validate.yml. The real-model checks themselves are in tools/validate-local.mjs.

Loopback parsing and Chrome's fetch option are in src/providers/http.js, while local presets, model discovery, CORS diagnostics, and Safari guidance are in src/providers/openaiCompat.js.