API reference

Every Python module of the stokastisk package and its public names. Generated from the source by scripts/release/build_docs_site.py.

Names exported at the top level (import stokastisk) are in stokastisk.__all__; submodule names are imported as shown, e.g. from stokastisk.serving import ServerProcess.

stokastisk

Stokastisk: dual-Tesla-T4 (SM75, CUDA 13.0) SDK around the bundled vLLM 0.25.1 engine.

stokastisk.benchmarks

Reproducible dual-GPU benchmarks (import lazily: they need vLLM / CUDA only when run).

stokastisk.benchmarks.baseline

Shared engine benchmark: identical workload for every strategy, ALL runs recorded (median/p95/min/max, never only the best).

Name Summary
class Workload
  Workload.prompts(self)
def measure(llm, workload: Workload, sampler_interval: float=0.5) Time workload on an already-loaded engine. Warm-up runs are discarded; every measured repeat is kept.
def run_engine(config, model: str, workloads: 'Workload | list[Workload]', *, label: str | None=None, sampler_interval: float=0.5) Load ONE in-process stokastisk.LLM and measure every workload on it. Call from a __main__-guarded script (vLLM spawns).

stokastisk.benchmarks.communication

Transfer-size sweep: host-staged (torch / native C++) vs NCCL send/recv. NCCL runs inside a 2-rank torchrun job.

Name Summary
def bench_host_staged(src: int=0, dst: int=1, *, sizes=SIZES, iters: int=20, warmup: int=5, chunk_bytes: int=4 << 20, n_buffers: int=2, native=None) Verifies the payload (byte-exact) on every size, then times iters transfers. native = NativeHostStaged class to use C++ instead.
def bench_nccl(rank: int, *, sizes=SIZES, iters: int=20, warmup: int=5) Inside an initialised 2-rank NCCL group. Each rank returns its own view; rank 1 (receiver) completion time is the transfer latency.
def compare(host_staged: dict, nccl: dict) Per size: ratio of median latencies and which transport wins. Decision rule: host-staged is integrated nowhere unless it wins.

stokastisk.benchmarks.gpu_balance

Compute / communication / idle accounting and imbalance for replica runs.

Name Summary
def from_replica_record(rec: dict[str, Any], workload: str)

stokastisk.benchmarks.pipeline_parallel

PP=2 (experimental). Stage-level compute/communication times are NOT exposed by vLLM's public API: they are reported as NOT_MEASURED

Name Summary
def config_for(**kw)
def run(model: str, workloads, **kw)

stokastisk.benchmarks.replica_parallel

Independent TP=1 replicas vs one engine. Offline batch generation reports BATCH completion latency (individual request completion

Name Summary
def run(model: str, workloads, devices=(0, 1), **kw)

stokastisk.benchmarks.report

Combine benchmark records into one JSON + a markdown table. Missing strategies are listed as NOT_RUN, never omitted silently.

Name Summary
def comparison_table(records: dict[str, dict[str, Any]], workload: str)
def relative_to(records: dict[str, dict[str, Any]], workload: str, base: str='tp1')

stokastisk.benchmarks.tensor_parallel

TP=1 vs TP=2 on an identical workload.

Name Summary
def config_for(tp: int, **kw)
def run(model: str, workloads, tp: int, **kw)

stokastisk.communication

EXPERIMENTAL communication layer: host-staged transport, NCCL timing helpers, transport registry. Not used by the vLLM engine.

stokastisk.communication.cuda_transport

EXPERIMENTAL host-staged GPU->GPU transfer for platforms without CUDA peer access.

Name Summary
class HostStagedTransport
  HostStagedTransport.init(self, src_device: int, dst_device: int, *, chunk_bytes: int=4 << 20, n_buffers: int=2)
  HostStagedTransport.transfer(self, src_tensor, dst_tensor) Copy src_tensor (on src device) into dst_tensor (on dst device); returns timing. Blocks until complete.

stokastisk.communication.native

Optional ctypes loader for native/host_staged_xfer.cpp (C++ / CUDA Runtime). Built on demand with nvcc; absence is not an error.

Name Summary
def build_native(src: str | os.PathLike, out_dir: str | os.PathLike, arch: str='sm_75')
def load_native(path: str | os.PathLike | None=None)
class NativeHostStaged
  NativeHostStaged.init(self, src_device: int, dst_device: int, *, chunk_bytes: int=4 << 20, n_buffers: int=2, lib=None)
  NativeHostStaged.transfer(self, src_tensor, dst_tensor)
  NativeHostStaged.close(self)

stokastisk.communication.nccl

NCCL timing helpers for a torchrun/torch.distributed process group (2 ranks). NCCL is the reference transport; it is never replaced.

Name Summary
def time_p2p(nbytes: int, rank: int, *, iters: int=20, warmup: int=5) rank 0 sends nbytes to rank 1 (NCCL send/recv). Returns per-iteration seconds measured on this rank (barrier-aligned).
def time_allreduce(nbytes: int, *, iters: int=20, warmup: int=5)

stokastisk.communication.pinned_memory

A small ring of page-locked host buffers (cudaHostAlloc via PyTorch's pinned allocator).

Name Summary
class PinnedBufferRing
  PinnedBufferRing.init(self, n_buffers: int, nbytes: int)
  PinnedBufferRing.host_bytes(self)

stokastisk.communication.synchronization

Buffer ownership tracking for the host-staged pipeline.

Name Summary
class BufferTicket
  BufferTicket.init(self)

stokastisk.communication.transport_registry

Registry of known transports with an honest status. Nothing is selected automatically.

Name Summary
def list_transports()

stokastisk.compat

Static Tesla T4 / SM75 / TRITON_ATTN compatibility inspection of the bundled vLLM 0.25.1.

Name Summary
def feature_matrix() A copy of the feature table (safe to mutate).
def get_feature(name: str)
def check_feature(name: str) Return the feature record, or raise if the feature cannot work on this build. UNKNOWN_REQUIRES_EVIDENCE is returned, not raised: it is untested, not forbidden.
def check_dtype(dtype: str | None, capability: tuple[int, int]=SM75) Reject an explicit bfloat16 below SM80 with the same reasoning as vllm/platforms/cuda.py, but before an engine is started.
def check_quantization(method: str | None, capability: tuple[int, int]=SM75) Compare a vLLM quantization method with the minimum capability declared in the source. Unknown methods are reported UNKNOWN_REQUIRES_EVIDENCE, never assumed to work.
def read_registry(package_root: str | pathlib.Path | None=None) architecture name -> (module, class) from vllm/model_executor/models/registry.py, read with ast (no import).
def uses_removed_backend(module_file: pathlib.Path, root: pathlib.Path) True if the architecture's OWN module imports names from the Flash-family placeholder module at import time, i.e. the model is built on a removed backend.
def find_missing_imports(module_file: pathlib.Path, root: pathlib.Path, *, max_depth: int=6) Chains of unconditional vllm imports from module_file that end in a module absent from the shipped package.
def check_model(model: str | None=None, *, architectures: Iterable[str] | None=None, package_root: str | pathlib.Path | None=None, capability: tuple[int, int]=SM75, dtype: str | None='half', quantization: str | None=None, mode: str | None=None) Static, OFFLINE pre-flight for a model on the T4 build (nothing is downloaded and no GPU is touched).

stokastisk.config

Typed configuration with T4-safe defaults and environment overrides.

Name Summary
class StokastiskConfig
  StokastiskConfig.strategy(self) Effective strategy. None keeps the baseline meaning: TP=2 -> tensor_parallel, otherwise single_gpu (PP>1 -> pipeline_parallel).
  StokastiskConfig.world_size(self)
  StokastiskConfig.is_experimental(self)
  StokastiskConfig.validate(self)
  StokastiskConfig.from_env(cls, prefix: str='STOKASTISK_')
  StokastiskConfig.apply_environment(self) Export NCCL settings (never overriding values the user already set).
  StokastiskConfig.engine_kwargs(self)
  StokastiskConfig.as_dict(self)

stokastisk.cuda

Optional CUDA stream/event/memory helpers (PyTorch-backed). Not used by the vLLM engine path.

stokastisk.cuda.device

Thin, optional CUDA device helpers on top of PyTorch. They NEVER touch vLLM's internal streams or contexts.

Name Summary
def device_count()
def on_device(index: int)
def synchronize(index: int | None=None)

stokastisk.cuda.events

Event helpers: dependency edges between streams (also across devices) and GPU-side timing.

Name Summary
def new_event(*, timing: bool=False)
def record(event, stream=None)
def make_wait(stream, event) cudaStreamWaitEvent: stream will not run later work until event completes (event may belong to another device).
def elapsed_ms(start, end)

stokastisk.cuda.memory

Memory helpers: free/total per device, pinned allocation, timed async copies.

Name Summary
def mem_info(device: int)
def pinned_empty(nbytes: int)
def timed_copy(dst, src, stream) Async copy on stream timed with CUDA events on the stream's device.

stokastisk.cuda.streams

Stream helpers. Create your own streams; do not enqueue onto vLLM's streams from outside vLLM.

Name Summary
def create_stream(device: int, *, priority: int=0)
def stream_synchronize(stream)

stokastisk.dependencies

Feature -> third-party import names. The single registry that the SDK (serving, engine), the Kaggle installer and the unit tests share.

Name Summary
def missing(names: Iterable[str]) Import names that cannot be found (find_spec returning None, or raising for a half-installed namespace).
def required_imports(*profiles: str) Locked + baseline import names for the given profiles (order preserved, no duplicates).
def missing_for(*profiles: str)

stokastisk.diagnostics

Runtime diagnostics. torch is imported lazily so this module works on CPU-only hosts.

Name Summary
def discover_gpus()
def peer_access_matrix()
def check_native_extensions() Import every bundled extension; value is 'ok' or the error text.
def flash_attn_native_loaded() FlashAttention native modules present in this process (expected: none).
def prohibited_report() Detect FlashAttention / FlashInfer / FlashMLA distributions, importable modules and shipped files (expected: all empty).
def validate_environment(*, require_gpus: int=2, strict_torch: bool=True) Validate the runtime against the SDK build target (CUDA 13.0 / SM75 / torch 2.11 cu130).
def report() Non-raising environment report (safe on CPU hosts).

stokastisk.engine

LLM facade over the bundled vLLM engine (Tesla T4 / SM75 / TRITON_ATTN / fp16).

Name Summary
class LLM Thin wrapper: applies the T4 defaults (NCCL configured not to depend on peer access, TRITON_ATTN, pinned structured-output backend), then delegates to vllm.LLM.
  LLM.init(self, model: str, *, config: StokastiskConfig | None=None, **kwargs: Any)
  LLM.selected_attention_backend(self) Backend recorded in the live engine's VllmConfig (config-level evidence; the worker's own log line and the Triton cache are the kernel-level evidence).
  LLM.verify_attention_backend(self)
  LLM.structured_output_backend(self) The structured-output backend recorded in the live engine's VllmConfig (pinned to 'xgrammar'; never 'auto').
  LLM.verify_structured_output_backend(self)
  LLM.engine(self)
  LLM.closed(self) True once :meth:close has run, or when the engine failed to start.
  LLM.generate(self, prompts, sampling_params=None, *, timeout: float | None=None, **kwargs) Pass-through to vllm.LLM.generate. An empty prompt list returns [] without touching the engine; non-str or empty prompts raise TypeError /
  LLM.chat(self, messages, sampling_params=None, *, timeout: float | None=None, **kwargs) Pass-through to vllm.LLM.chat (the chat template is applied by vLLM). messages is a list of {"role", "content"} dicts or a list of such lists.
  LLM.complete(self, prompt: str, *, max_tokens: int=64, temperature: float=0.0, timeout: float | None=None) Generate one completion for prompt and return its text.
  LLM.generate_structured(self, prompts, schema, *, max_tokens: int=64, temperature: float=0.0, timeout: float | None=None) Generate JSON constrained to schema by the engine's pinned structured-output backend, then validate every output against the schema.
  LLM.close(self) Shut the engine (and, for TP>1, its worker processes) down explicitly, then release cached CUDA memory. Idempotent.

stokastisk.errors

Stokastisk exception hierarchy.

Name Summary
class StokastiskError Base class for all SDK errors.
class EnvironmentValidationError The runtime does not match the SDK build target.
class GPUDiscoveryError Required GPUs are missing or have the wrong architecture.
class EngineError vLLM engine failed to initialise or generate.
class ConfigurationError Invalid Stokastisk configuration (e.g. a FlashAttention backend was requested).
class AttentionBackendError The engine did not end up with the requested Triton attention backend.
class EngineClosedError The engine was closed (explicitly, or because it never started); create a new stokastisk.LLM.
class EngineTimeoutError A generation call did not finish within the caller's timeout. The request may still be running inside the engine.
class EngineCloseError The engine shut down with an error. The SDK still released everything it could; __cause__ carries the original failure.
class StructuredOutputError A structured-output request failed, or its output violated the schema that was requested.
class MissingDependencyError A third-party package needed by the requested feature is not importable. missing lists the import names; feature names the feature.
  MissingDependencyError.init(self, feature: str, missing, hint: str='')
class ServerError Base class for failures of the OpenAI-compatible server process.
class ServerStartupError The server process exited, or did not become ready, before wait_ready returned.
  ServerStartupError.init(self, reason: str, *, exit_code, command, log_path, log_excerpt: str, timeout=None, waited=None)
class ServerHTTPError The server answered an API request with an error status or an unparsable body. status is the HTTP status (None for a transport failure).
  ServerHTTPError.init(self, message: str, *, status=None, body: str='')
class ServerShutdownError The server process could not be stopped cleanly (it had to be killed, or children survived).
class UnsupportedParallelismError The requested TP/PP/replica combination is not valid for the two-T4 target.
class InsufficientMemoryError The model (weights + KV cache) does not fit the requested GPU layout.
class TopologyError GPU topology could not be determined or does not match the requested layout.
class ReplicaError A replica worker failed to start, died, or returned an error.
class StokastiskCompatibilityError A model, dtype, quantization method or vLLM feature cannot work on the Tesla T4 / SM75 / TRITON_ATTN build.

stokastisk.limits

Hard resource limits of the Kaggle target and helpers that enforce / report them.

Name Summary
class KaggleLimits
def available_cpus() CPUs this process may really use (affinity-aware; equals nproc), never more than the Kaggle budget of 4 for planning.
def cpu_budget()
def threads_per_worker(tensor_parallel_size: int) Split the CPU budget across the TP worker processes (4 cores, TP=2 -> 2 threads each) so CPU-side libraries never oversubscribe.
def cpu_thread_env(tensor_parallel_size: int)
def system_ram_gib()
def dir_size_bytes(path: str | os.PathLike)
def working_dir_status(path: str | None=None) Usage of the (quota-limited) output directory against the 20 GiB Kaggle limit.
def check_resources() Compare the host with the Kaggle limits. Never raises. Statuses: PASS (within the limit) / WARNING (host differs from the Kaggle design).

stokastisk.nccl

NCCL validation helpers (launch with python -m torch.distributed.run --standalone --nproc_per_node=2).

Name Summary
def run_collective_selftest(expected_world: int=2) Run all-reduce / all-gather / reduce-scatter / broadcast / barrier over the torchrun world. Never raises.

stokastisk.parallel

Application-level parallelism: independent GPU replicas, token-aware scheduling, rule-based recommendations.

stokastisk.parallel.memory

Weight / KV-cache memory estimation from a model's config.json (an ESTIMATE, labelled as such everywhere it is shown).

Name Summary
class ModelSpec
  ModelSpec.head_dim_(self)
  ModelSpec.params(self)
  ModelSpec.weight_mib(self)
  ModelSpec.kv_mib(self, tokens: int)
  ModelSpec.unsupported_reason(self)
  ModelSpec.to_dict(self)
  ModelSpec.from_hf_config(cls, name: str, cfg: dict[str, Any], **kw)
def load_model_spec(model: 'str | ModelSpec', *, local_files_only: bool=False) ModelSpec from a ModelSpec, a local model directory, or a Hub repo id (downloads ONLY config.json).

stokastisk.parallel.queue

Request records and a deterministic FIFO queue.

Name Summary
def estimate_tokens(text: str) Cheap, deterministic input-token estimate (~4 characters per token). Replace with a tokenizer when exactness matters.
class Request
  Request.cost(self) Token-aware work estimate: prompt tokens (prefill) + requested output tokens (decode).
class RequestQueue
  RequestQueue.init(self, maxsize: int | None=None)
  RequestQueue.put(self, r: Request, *, front: bool=False)
  RequestQueue.get(self)
  RequestQueue.pending_tokens(self)

stokastisk.parallel.recommender

Rule-based, deterministic parallelism recommendation. It only RECOMMENDS: the caller decides (Recommendation.to_config()).

Name Summary
class Recommendation
  Recommendation.to_dict(self)
  Recommendation.to_config(self, **kw) A validated StokastiskConfig for this recommendation. Raises for non-runnable outcomes.
def default_recommendation(topology) Model-independent summary used by Topology.recommended_parallelism.
def recommend_parallelism(model, topology=None, workload: str='latency', *, max_model_len: int=4096, gpu_memory_utilization: float=0.88, min_concurrent_seqs: int=1)

stokastisk.parallel.replica

ReplicaPool: independent TP=1 vLLM engines, one per GPU, behind a deterministic token-aware scheduler.

Name Summary
class ReplicaPool
  ReplicaPool.init(self, model: str, devices: list[int] | tuple[int, ...]=(0, 1), *, config: StokastiskConfig | None=None, ready_timeout: float=900.0, max_attempts: int=2, engine_kwargs: dict[str, Any] | None=None)
  ReplicaPool.restart_dead(self, timeout: float=900.0) Restart every worker that has exited. Never called implicitly.
  ReplicaPool.generate(self, prompts: list[str], *, max_tokens: int=64, temperature: float=0.0, timeout: float=1800.0)
  ReplicaPool.close(self, timeout: float=60.0) Stop every worker. Always completes the whole cleanup (shutdown message, join, terminate, kill, queue close); then, if any worker had to be terminated, exited

stokastisk.parallel.routing

Weighted least-loaded routing (deterministic; ties go to the lowest worker id).

Name Summary
class WorkerLoad
  WorkerLoad.available(self)
def effective_speed(w: WorkerLoad, reference_speed: float | None=None) Speed used for routing. An observed speed is tokens/s; the default 1.0 is unit-less, so an unobserved worker is
def projected_load(w: WorkerLoad, extra_tokens: int, reference_speed: float | None=None) Estimated time-to-drain (arbitrary units): queued tokens + the new work, divided by the worker's speed.
def choose_worker(workers: list[WorkerLoad], cost_tokens: int)

stokastisk.parallel.scheduler

Deterministic, token-aware, weighted least-loaded scheduler. Pure Python: no CUDA, no processes (the pool owns those).

Name Summary
class Decision
class Scheduler
  Scheduler.init(self, workers: list[WorkerLoad], *, max_attempts: int=2, speed_alpha: float=0.3)
  Scheduler.assign(self, req: Request)
  Scheduler.assign_batch(self, reqs: list[Request], *, longest_first: bool=True) Greedy LPT (longest-processing-time-first) placement; deterministic. Returns worker id -> requests; unplaceable ones under key -1.
  Scheduler.complete(self, req_id: int, *, seconds: float | None=None)
  Scheduler.mark_dead(self, worker_id: int) Mark a worker dead and return its in-flight requests that may still be retried elsewhere.
  Scheduler.revive(self, worker_id: int)

stokastisk.parallel.worker

One replica worker process: one physical GPU, one TP=1 vLLM engine, one full copy of the weights.

Name Summary
def gpu_identity(physical_index: int) uuid + memory.used of a PHYSICAL GPU via nvidia-smi (independent of CUDA_VISIBLE_DEVICES, no CUDA context).
def worker_main(worker_id: int, device: int, model: str, config_dict: dict, engine_kwargs: dict, in_q, out_q)

stokastisk.prohibited

Policy and scanners for the Stokastisk "no FlashAttention / FlashInfer / FlashMLA" rule.

Name Summary
def is_prohibited_path(path: str) True if any component of path (a /-separated path inside a wheel or tree) names a Flash-family module or package.
def is_binary(path: str)
def native_markers(data: bytes) Flash-family markers present in a native object's bytes.
def scan_members(names: Iterable[str], read) Scan a set of archive/tree members. read(name) -> bytes is only called for files whose content must be inspected.
def scan_wheel(path: str | pathlib.Path)
def scan_tree(root: str | pathlib.Path)
def flash_in_maps(lines: Iterable[str]) Flash-family shared objects in the lines of /proc/<pid>/maps (path column only).
def loaded_flash_libraries(pid: int | str='self') Flash-family shared objects mapped by process pid (empty list = none; unreadable maps raise OSError, never silently pass).

stokastisk.serving

OpenAI-compatible serving for the T4 build: launch the bundled vllm serve with the Stokastisk T4 defaults, wait for /health, read /metrics, stream tokens.

Name Summary
def check_server_requirements(names: Sequence[str] | None=None) Return the import names that cannot be found (empty list = nothing missing). Uses find_spec only (no import side effects).
def validate_model_id(model: Any) A Hugging Face id (org/name) or a path. Rejects empty strings, whitespace/control characters and anything that vLLM's argument parser would read as an option.
def validate_host(host: Any)
def validate_port(port: Any)
def port_is_free(host: str, port: int) True if nothing is listening on (host, port) and the address can be bound.
def find_free_port(host: str='127.0.0.1')
def build_serve_command(model: str, config: StokastiskConfig | None=None, *, host: str='127.0.0.1', port: int=8000, served_model_name: str | None=None, quantization: str | None=None, extra_args: Sequence[str]=(), python: str | None=None) The exact argv that ServerProcess runs. Pure function: no process, no import of vllm.
class ServerProcess Supervise one vllm serve child. Use as a context manager. Never started implicitly, never left running by with.
  ServerProcess.init(self, model: str, config: StokastiskConfig | None=None, *, host: str='127.0.0.1', port: int=8000, served_model_name: str | None=None, quantization: str | None=None, extra_args: Sequence[str]=(), cuda_visible_devices: str | None=None, log_path: str | None=None, python: str | None=None)
  ServerProcess.base_url(self)
  ServerProcess.pid(self)
  ServerProcess.running(self)
  ServerProcess.log_tail(self, max_lines: int=40)
  ServerProcess.start(self)
  ServerProcess.health(self, timeout: float=5.0) True if GET /health answers 200 right now.
  ServerProcess.wait_ready(self, timeout: float=600.0, interval: float=2.0) Poll GET /health until HTTP 200. Raises :class:ServerStartupError if the child exits first or the timeout passes (with the exit code, the command,
  ServerProcess.start_and_wait(self, timeout: float=600.0)
  ServerProcess.restart(self, timeout: float=600.0) Stop the server (graceful first) and start a fresh one on the same port; returns once it is healthy.
  ServerProcess.stop(self, grace: float=30.0) SIGINT the process group (vLLM's own graceful shutdown), then SIGTERM, then SIGKILL; then make sure no process of the group survives.
def chat(base_url: str, model: str, messages: list[dict[str, str]], *, max_tokens: int=64, temperature: float=0.0, timeout: float=120.0) Non-streaming /v1/chat/completions; returns the assistant text.
def stream_chat(base_url: str, model: str, messages: list[dict[str, str]], *, max_tokens: int=64, temperature: float=0.0, timeout: float=120.0) Yield content deltas from /v1/chat/completions with stream=true (server-sent events: data: {...} lines, terminated by data: [DONE]).
def list_models(base_url: str, timeout: float=10.0)
def fetch_metrics(base_url: str, timeout: float=10.0) Parse the Prometheus text at /metrics into {sample_name{labels}: value} (comment lines skipped; non-numeric samples ignored).

stokastisk.structured

Structured (JSON-schema) output helpers: pure validation that needs no GPU, plus the sampling-parameter builder.

Name Summary
def check_schema(schema: Any) Return schema (a dict, or a JSON string parsed to one) if it is a valid JSON Schema, else raise :class:StructuredOutputError.
def validate_json_output(text: str, schema: Any) Parse text as JSON and validate it against schema. Returns the parsed object; raises :class:StructuredOutputError (never returns invalid data).
def json_schema_params(schema: Any, *, max_tokens: int=64, temperature: float=0.0) vllm.SamplingParams that constrain the output to schema. The backend is the engine's (pinned) one; a request cannot choose another.

stokastisk.telemetry

Optional GPU / communication / worker telemetry. Every probe degrades to an explicit 'unavailable'.

stokastisk.telemetry.communication_profiler

Wall-clock + CUDA-event timing for communication/compute segments. Segments are recorded; nothing is inferred.

Name Summary
class SegmentTimer
  SegmentTimer.init(self)
  SegmentTimer.wall(self, name: str)
  SegmentTimer.cuda(self, name: str, stream=None) GPU-side duration via CUDA events (synchronises the stream when the block exits).
  SegmentTimer.totals(self)
  SegmentTimer.report(self)

stokastisk.telemetry.cuda_profiler

Nsight availability probe. Nsight is never required: an unusable profiler is an explicit SKIPPED.

Name Summary
def nsight_status()

stokastisk.telemetry.metrics

Pure statistics + latency helpers (no GPU needed).

Name Summary
def percentile(values: Sequence[float], q: float) Linear-interpolated percentile (q in [0,100]); raises on empty input.
def summarize(values: Sequence[float]) min / median / p95 / max / mean over ALL runs (never only the best one).
def imbalance_percent(t0: float, t1: float) abs(T0 - T1) / max(T0, T1) * 100. 0 = perfectly balanced; 0.0 when both are 0.
def throughput(tokens: int, seconds: float)
def token_latencies(first_token_s: float | None, total_s: float, n_tokens: int) TTFT and mean inter-token latency from a single request's timing.

stokastisk.telemetry.nvml

GPU sampling: NVML if present, else nvidia-smi, else an explicit 'unavailable'. Counters Kaggle does not expose stay None.

Name Summary
def sample_once()
def pcie_throughput_kib_s() PCIe RX/TX in KiB/s per GPU via NVML, or None when NVML / the counter is unavailable.
class GpuSampler Background sampler: with GpuSampler(0.5) as s: ...; s.summary(). Overhead = one nvidia-smi call per interval.
  GpuSampler.init(self, interval: float=0.5)
  GpuSampler.summary(self)

stokastisk.telemetry.report

Local JSON evidence + optional OpenTelemetry spans. All metrics are available as JSON without any network telemetry.

Name Summary
def write_json(path: str | pathlib.Path, obj: Any)
def span(name: str, **attrs: Any) OpenTelemetry span if opentelemetry is installed AND a provider is configured; otherwise a no-op. Never required.
def provenance() Versions + identity every benchmark records.

stokastisk.telemetry.worker_metrics

Per-worker accounting used by the balance benchmark: compute vs communication vs idle, and queue wait.

Name Summary
class WorkerMetrics
  WorkerMetrics.idle_s(self)
  WorkerMetrics.to_dict(self)
def balance_report(w0: WorkerMetrics, w1: WorkerMetrics) Imbalance on compute time and on total busy time, reported separately (utilisation alone does not prove balance).

stokastisk.topology

GPU topology discovery (torch / NVML / nvidia-smi / sysfs; all but torch optional).

stokastisk.topology.discovery

discover_topology(): torch -> NVML -> nvidia-smi -> sysfs, every layer optional. Nothing is hardcoded to Kaggle's PHB.

Name Summary
def discover_topology(*, with_recommendation: bool=True)

stokastisk.topology.gpu_capabilities

Basic GPU properties from PyTorch (the primary provider). cuda-python is an optional, never-required refinement.

Name Summary
def torch_devices()
def cuda_python_version()

stokastisk.topology.models

Plain, JSON-serialisable topology records. A value that could not be measured is None — never invented.

Name Summary
class DeviceInfo
class PeerAccess CUDA peer access per ordered pair. matrix[src][dst] is torch.cuda.can_device_access_peer(src, dst) (None = not measured).
  PeerAccess.can_access(self, src: int, dst: int)
  PeerAccess.all_pairs_available(self)
  PeerAccess.any_pair_available(self)
class Topology
  Topology.gpu_count(self)
  Topology.pair_link(self, a: int, b: int)
  Topology.to_dict(self)

stokastisk.topology.nvml

Optional NVML (pynvml / nvidia-ml-py) and nvidia-smi helpers. Neither is required: every function degrades to None.

Name Summary
def nvml_devices() Per-GPU dict from NVML, or None when NVML is unusable.
def smi_query(fields: str) Rows of nvidia-smi --query-gpu=<fields>, or None when nvidia-smi is missing/fails.
def smi_topo_matrix()

stokastisk.topology.pcie

PCIe / link classification from nvidia-smi topo -m text, and PCIe link speed from sysfs.

Name Summary
def classify_cell(cell: str)
def parse_topo_matrix(text: str) {"0-1": "PHB"} (lower index first). Unknown / unparsable cells become UNKNOWN; an unparsable matrix yields {}.
def pcie_sysfs(pci_bus_id: str | None) Current/max PCIe generation + width from /sys (no root needed). Missing files -> missing keys.

stokastisk.topology.peer_access

CUDA peer access, measured with torch.cuda.can_device_access_peer in BOTH directions. Never forced, never assumed.

Name Summary
def measure_peer_access(device_count: int | None=None)