Jump to content

Comfac GPU Scaling and AI Research Goals

From MediawikiCIT

Comfac GPU Scaling and AI Research Goals

Objective

To develop and scale a high-performance AMD-based AI compute cluster, capable of running large-scale models (e.g., Qwen 2.5 235B) and supporting educational and R&D initiatives through open collaboration with partner schools.

Goals and Steps

1. Platform and Motherboard Selection

  • Identify and procure a motherboard or server platform that supports extensive GPU scaling and PCIe bifurcation (similar to the setup demonstrated by PewDiePie).
  • Ensure compatibility with ROCm and vLLM for distributed inference and multi-GPU coordination.

2. Initial Scaling (Pilot Models)

  • Begin with well-known, stable models to validate infrastructure performance and reliability.
  • Pilot hardware: AMD Radeon PRO R9700 AI or equivalent AI-focused GPU.
  • Validate thermal performance, power delivery, and driver stability for continuous inference workloads.

3. Progressive Hardware Replication

  • Once stable results are achieved with R9700 PRO, replicate the same environment using RX 7900 XTX and other AMD GPUs to benchmark performance scaling.
  • Document compatibility issues, driver updates, and quantization performance metrics.

4. Cluster and Swarm Development

  • Establish a Cluster System for large-model distributed inference and training.
  • Build a Swarm System capable of parallelizing smaller AI instances (e.g., 7700 and lower-end GPU nodes) for local and academic deployment.
  • Optimize inter-node communication, synchronization, and monitoring tools for mixed hardware setups.

5. Funding and Laboratory Deployment

  • Fund the creation of a dedicated AI Lab focused on testing, documentation, and educational use.
  • Provide access to partner schools for research, benchmarking, and AI model fine-tuning.

6. Open Compute and Tokenization Participation

  • Study and participate in open-source projects that allow community-based compute contributions (similar to Folding@home).
  • Learn and experiment with decentralized compute-sharing models that enable contributors to sell tokens or compute time securely and transparently.

Applied Model Evaluation — Synopsis Mail-Agent Test (2026-07-31)

First concrete output under the AI research program: an empirical model-selection harness, built for the Synopsis "talk to your email" agent but reusable for CSAMA and 2B evaluation.

> Evaluator: Kimi Code CLI / k1.6 · Methodology: live infra checks + real API tool-call sweeps + mailbox-anchored question verification. Full detail: work/comfac-synopsys/docs/260731-eod-model-selection-session.md and plans/260731-115026-model-selection-test.md.

  • Method over benchmarks: candidates are gated on tool-calling fidelity, citation honesty, and admitting absence — not prose quality or public benchmark scores. A single fabricated citation fails a candidate outright; every failure carries a failure-mode tag (fabrication / no-search / false-absence / wrong-args / shallow-search / misread / multi-hop-fail) so "failed" distinguishes untrustworthy from merely fixable.
  • Slate (7 candidates, all tools-gated on real calls): qwen3-vl:8b-instruct-q8_0 (control), qwen3-vl:8b-instruct, qwen3.5:9b (thinking on/off arm), gemma4:12b, qwen3:14b, ibm/granite4.1:8b, fablevibes:14b-a3b (community MoE prune — provenance caveat).
  • Hardware finding: 15 GiB usable VRAM (RX 9060 XT class) excludes 27B-class models at Q4 (17 GB+) — concrete evidence for the 20 GB+ procurement path (RX 7900 XT / Radeon PRO R9700) in the scaling goals above.
  • Small-model finding: sub-4B models mostly cannot run a tool loop (granite3.3:2b, phi4-mini failed to emit tool calls at all; llama3.2:3b marginal). For CPU/micro-model (CSAMA) roles, plan non-tool architectures (extractive prefilter → small-model polish).
  • Infra prerequisites proven: Ollama as an enabled systemd service, VPN-reachable from the batch host, no auto-sleep, 100% GPU placement — the pattern any "chief model host" in the cluster must replicate.
  • Status: gated run blocked on one human step (Justin's 20 verified questions). Harness + scorer + golden-set format in work/comfac-synopsys/harness/.

Reference

End Goal

To make Comfac and its academic partners a recognized hub for open, scalable, and sustainable AI research using AMD technologies.

Retrieved from "https://mediawiki.comfac.net/index.php?title=Comfac_GPU_Scaling_and_AI_Research_Goals&oldid=265"