┌─────────────────────────────────────────────────────────────────┐
│                    Local vs Cloud Models                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  LOCAL MODELS                    CLOUD MODELS                   │
│  ────────────                    ────────────                   │
│  + Complete privacy              + No hardware needed           │
│  + No per-token costs            + Always latest models        │
│  + Works offline                 + Highest capability          │
│  + Data never leaves device      + Managed infrastructure      │
│  - Requires hardware             - Per-token costs             │
│  - Limited model selection       - Requires internet           │
│  - Self-managed                  - Data sent to provider       │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

# Download and install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Or use Homebrew
brew install ollama

# Download and install
curl -fsSL https://ollama.com/install.sh | sh

# Download from ollama.com/download
# Run the installer
# Ollama will start automatically

# Start the Ollama service
ollama serve

# The service runs at http://localhost:11434

# Pull Llama 3.1 (recommended)
ollama pull llama3.1

# Pull specific size
ollama pull llama3.1:8b
ollama pull llama3.1:70b

# Pull other popular models
ollama pull mistral
ollama pull codellama
ollama pull qwen2.5

# Quick test
ollama run llama3.1 "Hello, how are you?"

# Interactive mode
ollama run llama3.1

# List installed models
ollama list

┌─────────────────────────────────────────────────────────────────┐
│  Settings > Local Models                                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Ollama Configuration                                            │
│                                                                  │
│  Endpoint: http://localhost:11434                                │
│                                                                  │
│  Status: [*] Connected                                          │
│                                                                  │
│  Available Models:                                               │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │ [x] llama3.1:8b          (4.7 GB)                       │    │
│  │ [x] mistral:7b           (4.1 GB)                       │    │
│  │ [x] codellama:13b        (7.4 GB)                       │    │
│  │ [ ] qwen2.5:72b          (41 GB) - Insufficient RAM     │    │
│  └─────────────────────────────────────────────────────────┘    │
│                                                                  │
│                                      [Test Connection] [Save]   │
└─────────────────────────────────────────────────────────────────┘

# On the Ollama server, allow external connections
OLLAMA_HOST=0.0.0.0:11434 ollama serve

# In HIVE, configure the remote endpoint
http://your-server-ip:11434

ollama pull llama3.1:8b

ollama pull mistral

ollama pull codellama:13b

ollama pull qwen2.5:14b

ollama pull phi3:mini

CPU-Only Operation:
  - Modern multi-core CPU (8+ cores recommended)
  - 16 GB RAM minimum
  - SSD storage
  - Expect slow responses (10-60 seconds)

For 7B Parameter Models:
  - 8 GB RAM
  - 5 GB disk space
  - CPU or entry-level GPU

GPU-Accelerated Operation:
  - NVIDIA GPU with 8+ GB VRAM
  - 32 GB system RAM
  - NVMe SSD
  - CUDA 11.8+ drivers

For 13B-34B Parameter Models:
  - 16-24 GB VRAM
  - 32 GB system RAM
  - 20-50 GB disk space

┌─────────────────────────────────────────────────────────────────┐
│  Model Size vs Quality/Speed Tradeoff                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Quality                                                         │
│    ^                                                             │
│    │                                          70B               │
│    │                                       * * *                │
│    │                                   34B                       │
│    │                              * * *                         │
│    │                         13B                                 │
│    │                     * * *                                  │
│    │                7B                                          │
│    │           * * *                                            │
│    │      3B                                                    │
│    │  * *                                                       │
│    └─────────────────────────────────────────────────────> Speed │
│                                                                  │
│  Smaller models = Faster but lower quality                      │
│  Larger models = Slower but higher quality                      │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

# Check GPU detection
ollama ps

# Force GPU usage
export OLLAMA_NUM_GPU=999

# Limit GPU memory (useful for sharing)
export OLLAMA_GPU_MEMORY=6g

# Reduce memory footprint
export OLLAMA_NUM_PARALLEL=1

# Use quantized models (smaller, faster)
ollama pull llama3.1:8b-q4_0

# Keep models loaded for faster response
export OLLAMA_KEEP_ALIVE=60m

# Pull quantized version
ollama pull llama3.1:8b-q4_K_M

Agent: Local Research Assistant
Framework: ollama
Model: llama3.1:8b
Settings:
  temperature: 0.7
  context_length: 4096
  num_predict: 2048

Swarm: Hybrid Pipeline
Agents:
  - name: Data Processor
    framework: ollama
    model: llama3.1:8b
    # Local for privacy and cost

  - name: Quality Reviewer
    framework: anthropic
    model: claude-sonnet-4-20250514
    # Cloud for highest quality

  - name: Code Assistant
    framework: ollama
    model: codellama:13b
    # Local for code tasks

1. Download from lmstudio.ai
2. Install and launch
3. Download models from built-in browser
4. Start local server
5. Configure HIVE with endpoint: http://localhost:1234/v1

# Clone repository
git clone https://github.com/oobabooga/text-generation-webui

# Run with API
python server.py --api

# Connect HIVE to: http://localhost:5000

# Build llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make

# Run server
./server -m model.gguf --port 8080

Symptom: "Error: could not connect to ollama app"

Solutions:
1. Check if Ollama is running:
   ps aux | grep ollama

2. Start Ollama service:
   ollama serve

3. Check logs:
   journalctl -u ollama (Linux)
   cat ~/.ollama/logs/server.log

Symptom: "Error pulling model"

Solutions:
1. Check disk space:
   df -h

2. Check network connection:
   curl -I https://ollama.com

3. Try alternative model source:
   ollama pull llama3.1:8b --insecure

Symptom: Very slow responses

Solutions:
1. Verify GPU is being used:
   ollama ps
   nvidia-smi

2. Use smaller or quantized model:
   ollama pull llama3.1:8b-q4_0

3. Reduce context length in agent settings

4. Check for other GPU processes:
   nvidia-smi

Symptom: "CUDA out of memory" or system freeze

Solutions:
1. Use smaller model variant
2. Use quantized version
3. Reduce context length
4. Close other applications
5. Enable GPU memory limit:
   export OLLAMA_GPU_MEMORY=6g

For GPT-4o Mini equivalent quality:

Cloud cost: $38/month
Local GPU: $1,200 upfront + $10/month electricity

Break-even: ~40 months

For higher volume (100M tokens/month):
Cloud cost: $380/month
Break-even: ~3 months

Begin with:
1. Small 7B parameter model
2. Test on your hardware
3. Gradually increase size
4. Find your sweet spot

Match model to task:
- Simple tasks: phi3:mini (fast, small)
- General tasks: llama3.1:8b (balanced)
- Code tasks: codellama:13b (specialized)
- Complex tasks: llama3.1:70b (quality)

# Update installed models
ollama pull llama3.1:8b

# Check for updates regularly
ollama list

# GPU monitoring
watch -n1 nvidia-smi

# System resources
htop