1. Getting started
Notebook settings
| Setting | Value |
|---|---|
| Accelerator | GPU T4 × 2 |
| Internet | ON |
| Python | 3.13 image |
Install
One cell:
!python -m pip install --index-url https://pypi.org/simple "stokastisk @ https://huggingface.co/waqasm86/stokastisk/resolve/main/wheels/stokastisk-0.1.0-5-cp313-cp313-linux_x86_64.whl#sha256=fe80d7c36fa78cf8a26feb1e00bb5f1f006ca45a027450353603976320ae0b3e"
pip downloads the 38 MB wheel from the Hugging Face CDN, refuses it unless its SHA256 matches, and installs 19 pinned dependencies from PyPI. Kaggle's own PyTorch 2.11.0+cu130 and Triton 3.6.0 are never touched. The URL names the build‑4 wheel (project version 0.1.0); the current file and hash are in RELEASE.json.
Why this form and not pip install stokastisk --extra-index-url …? That also works, but --extra-index-url makes pip consider PyPI as well as the extra index, so a different project with the same name could be chosen. The pinned command above names the exact file and uses PyPI only for ordinary dependencies. To go back to build 3, see INSTALL.md.
Check the install
import stokastisk, json print(stokastisk.__version__) # 0.1.0 print(json.dumps(stokastisk.prohibited_report(), indent=1)) # "clean": true
clean: true means no FlashAttention, FlashInfer or FlashMLA code is present.
First generation
Engine code runs in an ordinary notebook cell:
from stokastisk import LLM, StokastiskConfig config = StokastiskConfig(max_model_len=2048) # one T4, FP16, TRITON_ATTN with LLM("Qwen/Qwen2.5-0.5B-Instruct", config=config) as llm: print(llm.verify_attention_backend()) # TRITON_ATTN print(llm.complete("The capital of France is", max_tokens=16)) print(llm.chat([{"role": "user", "content": "Say hello in five words."}])[0].outputs[0].text)
with releases the GPUs on exit. If you do not use with, call llm.close().
Running as a script
vLLM starts its engine in a separate (spawned) process. In a script (!python demo.py), keep engine code under a guard:
%%writefile demo.py import stokastisk if __name__ == "__main__": with stokastisk.LLM("Qwen/Qwen2.5-0.5B-Instruct", config=stokastisk.StokastiskConfig(max_model_len=2048)) as llm: print(llm.complete("Hello", max_tokens=8))
Both forms ran successfully on Kaggle (plain cells in notebooks 18 and 19, script in notebook 16).
Which model fits?
Each T4 has about 15 GiB. FP16 weights take about 2 bytes per parameter, plus KV cache.
| Model size | Use |
|---|---|
| up to about 3B | one GPU, tensor_parallel_size=1 (fastest) |
| about 7B (14 GiB of weights) | both GPUs: tensor_parallel_size=2 |
Ask the SDK: stokastisk.recommend_parallelism("Qwen/Qwen2.5-Coder-7B-Instruct") — see Using both GPUs — and stokastisk.check_model("Qwen/Qwen2.5-Coder-7B-Instruct", mode="TP=2") to see whether that exact model and mode were run on Kaggle (Checks and diagnostics).
Next: Configuration.