Environment, model choice and fallbacks

Choose a setup for your machine

For Mac walkthroughs, follow the macOS guide and 21-day plan. Begin Python and UI exercises before deciding whether local inference fits your machine.

Use gemma3:4b (~3.4 GB download) for the default chatbot, try qwen3:8b (~5.2 GB download) as a comparative second model, and fall back to gemma3:1b if troubleshooting startup or memory use. Published download sizes are not RAM footprints; active context, backend and other applications consume memory. On Apple Silicon, the CPU and GPU share system memory. See Ollama Gemma 3, Ollama Qwen 3.

Example setup commands on macOS, from your chosen workspace:

uname -m     # expect arm64 with native Apple Silicon Terminal
python3 --version
python3 -m venv .venv
source .venv/bin/activate
ollama pull gemma3:4b
ollama run gemma3:4b

Install Python first from the official macOS release or package manager and install Ollama from ollama.com/download before the Ollama commands. Do not modify macOS' system Python.

Before installing anything

Record operating system, RAM, GPU/VRAM (if present), available disk space, and comfortable weekly time. No GPU is required for Python, the interactive NN tools, or a rule-based chatbot. Local LLM inference speed is hardware-dependent.

Configuration	Starting point	Comment
Low-end CPU machine	`qwen3:0.6b`	Fastest test of wiring, lower response quality
~8 GB RAM laptop	`gemma3:1b`	Good small-model starting point
~16 GB RAM	`gemma3:4b`	Try if performance permits
Dedicated GPU / enough RAM	`qwen3:8b`	Larger and heavier; not a prerequisite

Model tags, availability and size change: check the official Ollama library before class. Download size is not peak runtime memory. Quantization affects memory use and quality. GPU inference is optional.

Cross-platform environment

Install Python using the OS-approved installer/package manager.
Make a virtual environment: python3 -m venv .venv (Windows: py -m venv .venv).
Activate it: POSIX . .venv/bin/activate, Windows PowerShell .venv\Scripts\Activate.ps1.
From the extracted course root, use python -m pip install -r examples/requirements.txt inside the venv.
Use Git for source. Ignore .venv, secrets and generated files.
Install Ollama, then ollama pull gemma3:4b if resources permit; use gemma3:1b as a smaller fallback.

Fallback routes

No model fits: use an external hosted inference endpoint if permitted and affordable, behind a small client adapter; otherwise use the deterministic bot for UI work.
Offline: use pre-downloaded videos/resources and an already-pulled model; the slides' CDN runtime and YouTube embeds do require internet, whereas the Markdown does not.
Docker unavailable: skip Session 08 until later. Package with requirements.txt and a short README.
Small screen: don't force the video+code slide together; use the original source in a separate browser tab.

Basic safety

Don't paste tokens or private files into an LLM. Keep Ollama on loopback; avoid exposing port 11434 or an unauthenticated Streamlit UI to untrusted networks. Treat model text as untrusted and always validate structured output.