home-classifier

Self-hosted, CPU-only zero-shot text classification using GLiClass v3. The image runs GLiClass's built-in Ray Serve HTTP API; there is no custom API or inference wrapper. Supply candidate labels with each request. No training, external inference API, GPU, database, or PolyLoRA adapters are required.

Model provisioning

The image contains dependencies and configuration, not model weights. Neither the image build nor the runtime downloads models. Provision a complete Hugging Face model snapshot separately, then mount it read-only at /models/gliclass, readable by UID/GID 10001. HF_HUB_OFFLINE=1 is set in the image.

Start with knowledgator/gliclass-edge-v3.0 for the smallest v3 model (about 131 MB of weights). The same local-path configuration can select gliclass-base-v3.0, gliclass-large-v3.0, gliclass-modern-base-v3.0, or gliclass-modern-large-v3.0 from Knowledgator. Larger models need more memory and CPU time. Model size is not total runtime memory. Evaluate accuracy on your own texts and labels, especially for Portuguese.

For example, on a machine with internet access:

uv sync --frozen
uv run hf download knowledgator/gliclass-edge-v3.0 \
  --revision MODEL_COMMIT \
  --local-dir ./models/gliclass-edge-v3.0

Replace MODEL_COMMIT with a commit from the model's Hugging Face repository for reproducibility. Keep the model configuration, safetensors weights (including shards/index if applicable), and all tokenizer assets. For Kubernetes, provision the snapshot on a PVC with a separate job/init container. No Hugging Face token is needed for these public models.

Container

docker build -t home-classifier .
docker run --rm --init \
  -p 127.0.0.1:8000:8000 \
  --cpus=4 --memory=4g --shm-size=2g --stop-timeout=60 \
  --mount type=bind,source="$(pwd)/models/gliclass-edge-v3.0",target=/models/gliclass,readonly \
  home-classifier

The runtime is non-root and installs PyTorch from its CPU-only wheel index. Four CPUs and 4 GiB RAM are an initial allocation for the edge model, not a sizing guarantee for other models or workloads. Ray needs writable /tmp and shared memory. For a read-only root filesystem, additionally pass --read-only --tmpfs /tmp:rw,size=1g,mode=1777. Monitor temporary storage/log usage for long-running deployments.

The image entrypoint is python -m gliclass.serve --config /app/serve_config.yaml --host 0.0.0.0. Additional arguments override configuration, for example home-classifier --model /models/another-v3. To change multiple settings, copy and edit the complete shipped serve_config.yaml, then mount it at /app/serve_config.yaml read-only. YAML files replace the configuration; they do not merge with the shipped defaults. Preserve device: cpu, dtype: float32, and num_gpus_per_replica: 0 (upstream defaults require CUDA). Keep the internal port at 8000 to match the Docker health check; change the host port mapping instead of http_port or --port. There is no MODEL_PATH environment variable; use the mount location or --model.

Defaults

Setting	Value
Model path	/models/gliclass
Device / dtype	CPU / float32
Replicas / GPUs	1 / 0
Ray CPU reservation / PyTorch threads	2 / 2
Input sequence limit	512 tokens, including label/prompt tokens
Default multi-label threshold	0.5
Maximum inference batch	4 requests
Batch collection wait	20 ms
Ongoing requests / queued requests	8 / 16 (Ray Serve limits)
Compilation, GPU memory calibration, adapters	Disabled

The CPU reservation is a Ray scheduling resource, not a CPU quota. Use container CPU/memory limits. If tuning parallelism, adjust num_cpus_per_replica, tokenizer_threads (also controls PyTorch threads), and RAYON_NUM_THREADS together. Leave resources for Ray's control processes.

Keep the largest precompiled_batch_sizes entry equal to max_batch_size: GLiClass uses this list for batch sizing even with compilation disabled. Inputs beyond the sequence limit are truncated by the tokenizer, not rejected. Candidate label count is not capped by this service; label tokens still consume the input budget. Use small label sets and bounded text lengths.

Zero-shot HTTP API

curl --fail-with-body http://localhost:8000/gliclass \
  -H 'Content-Type: application/json' \
  -d '{
    "text": "This is a great product!",
    "labels": ["positive", "negative", "neutral"],
    "threshold": 0.5,
    "multi_label": true
  }'

The response is a JSON array of {"label": "...", "score": 0.0} objects. Actual scores depend on the model and input.

• text: one nonempty string; labels: a nonempty array of candidate strings.
• multi_label: true (default): independent sigmoid scores; return labels at or above threshold (default 0.5). An empty result is valid.
• multi_label: false: return one best label with a softmax score. Upstream ignores threshold in this mode; it does not provide an abstention cutoff.
• Send concurrent requests for dynamic batching. Do not send a texts array: upstream only processes its first item.

Only zero-shot usage is configured/documented here. The unmodified upstream API also accepts optional prompts/examples; no custom filtering layer is added. There are no custom /docs, /healthz, or /readyz routes.

Health and deployment

Probe	Endpoint	What it checks
Liveness / Docker health	GET /-/healthz	Ray HTTP proxy, not model inference
Startup / readiness	GET /gliclass/adapter-cache	Model replica responds; returns enabled: false
End-to-end smoke test	POST /gliclass	Real classification

The adapter-cache endpoint is used only as a lightweight upstream replica probe; adapters remain disabled. Give startup several minutes for larger models. Inference blocks the replica's event loop, so readiness can wait behind queued batches. Size readiness timeoutSeconds and failureThreshold to tolerate the measured worst-case batch and queue latency for your model, not just idle probe latency; otherwise a healthy busy pod can flap unready. Readiness proves model initialization, not inference quality. Do not use the client's prefixed /gliclass/-/healthz as a probe.

For Kubernetes: one container/replica per pod, a read-only model PVC, writable /tmp, memory-backed /dev/shm, CPU/memory requests and limits, and a termination grace period of at least 60 seconds. Expose only port 8000. Do not expose Ray's control-plane/dashboard ports or use host networking.

Trusted internal use only: upstream has no authentication and limited request validation. There is no application-level body-size limit. Restrict network access and put authentication, request-size limits, and timeouts at your ingress if needed. Queue limits do not replace ingress limits.

Forgejo publishing

.forgejo/workflows/docker-image.yml follows home-whisper: pushes to master and manual runs on master publish git.brennoflavio.com.br/brennoflavio/home-classifier:YYYYMMDDHHMMSS (UTC), using REGISTRY_USERNAME and REGISTRY_PASSWORD repository secrets. The workflow maintains a build-cache registry tag. Builds and publishing do not run tests or lint checks; these can be run locally. The workflow does not create releases, deploy the service, or provision model weights.

Development and validation

uv sync --frozen
uv run pytest
uv run ruff check .
uv run ruff format --check .

Default tests check CPU configuration, effective batch sizing, CLI overrides, and Ray API/protobuf compatibility without downloading models or starting Ray. Live HTTP tests are skipped unless CLASSIFIER_URL is set. Against a running container:

CLASSIFIER_URL=http://localhost:8000 uv run pytest tests/test_http.py -v

The HTTP tests also run without development dependencies:

CLASSIFIER_URL=http://localhost:8000 python3 tests/test_http.py -v

For a local server with an already provisioned model:

HF_HUB_OFFLINE=1 HF_HUB_DISABLE_TELEMETRY=1 RAY_USAGE_STATS_ENABLED=0 \
  RAYON_NUM_THREADS=2 OPENBLAS_NUM_THREADS=1 \
  uv run python -m gliclass.serve --config serve_config.yaml \
  --model /absolute/path/to/gliclass-v3 --host 127.0.0.1

Dependencies are locked in uv.lock. GLiClass 0.1.20 calls private Ray APIs: Ray is pinned to 2.55.1, and protobuf is constrained below 7 for that Ray version. Re-run container startup and live inference tests when upgrading dependencies; an import or --help check alone does not catch these incompatibilities.

References: GLiClass serving, edge v3 model.