# Project Notebooks

This folder holds all runnable notebooks for the project:
- `zero123++.ipynb`: run Zero123++ for single-image novel view synthesis.
- `render_&_depth_est.ipynb`: render meshes and estimate depth maps (Blender + Objaverse pipeline).
- `fastr.ipynb`: Fast3R mesh extraction and visualization utilities (Blender/Open3D).
- `openlrm.ipynb`: OpenLRM rendering experiments (Blender-based).
- `evaluate_&_visualization.ipynb`: compute PSNR/SSIM/LPIPS and build figure grids from generated views.

Most notebooks were authored for Google Colab (metadata shows GPU = T4). The steps below describe a local setup.

## Prerequisites
- OS: Linux or Windows; Linux is strongly recommended for Blender CLI and CUDA tooling.
- GPU: NVIDIA GPU with recent driver and CUDA 11.8+ (needed for PyTorch/Zero123++).
- Python 3.10+ (Conda recommended).
- Blender 3.6+ installed and invokable as `blender` from the shell (required for the `bpy` cells in `render_&_depth_est.ipynb`, `fastr.ipynb`, and `openlrm.ipynb`).
- (Optional) Jupyter/VS Code for running notebooks locally.

## Environment setup (PyTorch + Python deps)
```bash
# create env
conda create -n csc2529 python=3.10 -y
conda activate csc2529

# install PyTorch (CUDA 11.8 build shown; change if your CUDA version differs)
pip install --extra-index-url https://download.pytorch.org/whl/cu118 \
  torch torchvision torchaudio

# core Python packages used across notebooks
pip install numpy pillow matplotlib scikit-image lpips open3d objaverse \
  imageio tqdm pandas jupyter
```

## Blender Python deps
The Blender-bundled Python is separate from your Conda env. Install needed packages into Blender so `import bpy, mathutils, numpy, open3d, objaverse` succeed:
```bash
# replace /path/to/blender with your blender executable
/path/to/blender -b --python - <<'PY'
import sys, subprocess
pkgs = ["numpy", "open3d", "objaverse", "tqdm", "imageio", "requests"]
subprocess.check_call([sys.executable, "-m", "pip", "install", *pkgs])
PY
```

## Data
- `evaldata.zip` lives in the repo root. The evaluation notebook’s first cell does `!unzip evaldata.zip` to create folders `zero/`, `render/`, `fast3r/`, and `lrm/` with example images. If running locally, unzip from repo root: `unzip evaldata.zip`.
- Other notebooks expect you to point to local assets/checkpoints (e.g., Zero123++ weights, Objaverse assets). Edit the path cells accordingly.

## Running the notebooks
1. From repo root: `cd code`.
2. Launch Jupyter: `jupyter notebook` (or open via VS Code).
3. For each notebook:
   - `evaluate_&_visualization.ipynb`: Run cells top-to-bottom. Requires the unzipped `zero/`, `render/`, `fast3r/mesh*.png`, and `lrm/` images. LPIPS installs inline; GPU is preferred.
   - `zero123++.ipynb`: Set the checkpoint/model paths in the first config cell. Run on GPU for reasonable speed.
   - `render_&_depth_est.ipynb`: Uses Blender + Objaverse; ensure Blender Python has the deps above. Provide object IDs/paths as prompted in the config cells.
   - `fastr.ipynb`: Uses Blender/Open3D to load and visualize meshes; update mesh/input paths in the config cell.
   - `openlrm.ipynb`: Blender-based rendering; point to your OpenLRM assets/checkpoints before running.

## Tips
- If Blender cannot import a module, reinstall that module using Blender’s Python as shown above.
- Keep the working directory as the repository root or `code/` so relative paths (e.g., `../evaldata.zip`, `zero/view_00.png`) resolve correctly.
- If you prefer Colab, uploading `evaldata.zip` and the notebooks works out of the box; select a GPU runtime.
