Skip to content

.environment.md

Latest commit

 

History

History
79 lines (64 loc) · 3.22 KB

.environment.md

File metadata and controls

79 lines (64 loc) · 3.22 KB
Raw

Local development environment

This is the current-machine supplement to the canonical environment record. It is intentionally a snapshot, not the old branch's attempt log.

Last checked: 2026-07-21.

Host and toolchain

  • NixOS x86_64 host (ledbx in the source branch record), AMD Ryzen 9 7900, approximately 30 GiB system RAM.
  • Local accelerator: NVIDIA GeForce RTX 5070 Ti / discrete Blackwell (sm_120), 16 GiB VRAM. The host driver is healthy (595.71.05, CUDA 13.2 compatibility, 166 MiB display use and 0% compute at the 2026-07-21 check); the restricted workspace sandbox hides the device, so runtime commands need device access.
  • CMake, Ninja and nvcc are deliberately supplied by the checked-in flake; they are not assumed to be globally installed.
  • The pinned CUDA shell exports the NixOS driver-library path and TRITON_LIBCUDA_PATH=/run/opengl-driver/lib, avoiding Triton's unavailable /sbin/ldconfig probe.

Reproducible shells

CPU configure/build/test:

nix develop .#default --command \
  cmake -S . -B build-nix-cpu -G Ninja -DVLLM_CPP_CUDA=OFF \
    -DCMAKE_BUILD_TYPE=RelWithDebInfo
nix develop .#default --command cmake --build build-nix-cpu -j4
nix develop .#default --command \
  ctest --test-dir build-nix-cpu --output-on-failure

CUDA configure/build/test (runtime commands require access outside the restricted workspace sandbox):

nix develop .#cuda --command bash -lc \
  'cmake -S . -B build-nix-cuda-gcc14 -G Ninja \
    -DVLLM_CPP_CUDA=ON \
    -DCMAKE_CUDA_COMPILER="$CMAKE_CUDA_COMPILER" \
    -DCMAKE_CUDA_HOST_COMPILER="$CMAKE_CUDA_HOST_COMPILER" \
    -DVLLM_CPP_CUDA_ARCHITECTURES=120a \
    -DCMAKE_BUILD_TYPE=RelWithDebInfo'
nix develop .#cuda --command cmake --build build-nix-cuda-gcc14 -j4
nix develop .#cuda --command \
  ctest --test-dir build-nix-cuda-gcc14 --output-on-failure

Local Python/JIT caches are repository-local and ignored: .venv-vllm/, .hf-cache/, .vllm-cache/, .torchinductor-cache/, .triton-cache/, and .flashinfer-cache/.

Local Qwen3.5 diagnostic

Qwen/Qwen3.5-4B is the local plain-BF16 model: it exercises the hybrid GDN / full-attention architecture within this GPU's memory budget. Its real cached weight-loader gate is:

HF_HOME=$PWD/.hf-cache nix develop .#default --command \
  build-nix-cpu/tests/test_qwen35_plain_weights --no-skip

The current transplant has a clean CPU build, focused loader/registry/forward tests, and the real-weight topology/load gate. CUDA 12.9/GCC 14 configures and the production library compiles for sm_120a; the real 4B CUDA test passes 1664/1664 with identical retained-host/direct-device prompt and output token IDs. Exact commands and exceptions are recorded in the active state entry. Token-for-token comparison against a current pinned vLLM oracle, execution traces, memory measurement and every-axis performance remain PENDING. Historical vLLM 0.24 local results from the retired branch are diagnostic/VOID for the present parity denominator and are not reproduced here.

The deterministic ShareGPT generator, vLLM token/logit dumper, and optional UVM sentinel live under tools/bench/. The bpftrace sentinel is observational and requires root plus matching NVIDIA open-kernel symbols.