Skip to content

CG Spectrum → NotebookLM | Full Course Scraper

Scrapes all enrolled courses from CG Spectrum and saves everything — text, PDFs, slides, zip files, video links — organized and ready for Google NotebookLM.

What It Does

Feature Details
All courses Discovers every course you're enrolled in automatically
Full curriculum Scrapes every module and lesson in order
File downloads PDFs, PowerPoints, zip packages, .unitypackage, etc.
Video links Captures Vimeo / YouTube / Wistia embed URLs
External resources Captures all linked references per lesson
HTML to Markdown Converts lesson content to clean, readable Markdown
Resume support Re-running skips already-scraped pages
Per-course output One folder + combined file per course

Prerequisites

Setup & Run

# 1. Restore NuGet packages
dotnet restore

# 2. Build and install Playwright browser (one-time)
dotnet build
pwsh bin/Debug/net8.0/playwright.ps1 install chromium

# Alternative if you don't have PowerShell:
dotnet tool install --global Microsoft.Playwright.CLI
playwright install chromium

# 3. Run
dotnet run

What Happens Step by Step

1. Chrome opens and navigates to CG Spectrum login
2. Auto-clicks "Sign in with Google"
3. YOU complete the Google login (password / 2FA) - 3 min window
4. Scraper scans your dashboard/enrollments page
5. Discovers all enrolled courses automatically
6. For each course:
   a. Reads the full curriculum / lesson list
   b. Visits every lesson page
   c. Extracts text and saves as .md
   d. Downloads PDFs, slides, zips to _assets/
   e. Captures video embed URLs
7. Saves per-lesson files + combined per-course file
8. Saves master index of all courses

Output Structure

CGSpectrum_NotebookLM/
|-- _MASTER_INDEX.md                         (overview of all courses)
|
|-- 01_Core_Game_Programming_Unreal/
|   |-- 00_course_overview.md
|   |-- 001_Introduction.md
|   |-- 002_Setting_Up_Unreal.md
|   |-- ...
|   |-- _COMBINED_Core_Game_Programming.md   (upload this to NotebookLM)
|   +-- _assets/
|       |-- lecture_slides.pdf
|       |-- starter_project.zip
|       +-- ...
|
|-- 02_Art_Fundamentals/
|   |-- ...
|   |-- _COMBINED_Art_Fundamentals.md
|   +-- _assets/
|
+-- .progress   (tracks scraped URLs for resume support)

Uploading to NotebookLM

One notebook per course (recommended): 1. Go to notebooklm.google.com 2. New notebook, name it after the course 3. Add sources -> Upload files 4. Upload _COMBINED_CourseName.md 5. Also upload PDFs from the _assets/ folder 6. Ask any question about your course material!

NotebookLM limits to know: - 50 sources per notebook max - 500,000 words per source (combined .md files fit within this) - PDFs up to 200MB each

Troubleshooting

Problem Fix
SSO button not found Manually click "Sign in with Google" in the browser
No courses discovered Navigate to your enrollments page, then press Enter
No lessons found Scraper falls back to sequential "Next" button crawling
File download 403 error File needs auth - scraper syncs cookies automatically
Playwright install fails Run: dotnet tool install --global Microsoft.Playwright.CLI
Slow scraping Normal - 900ms delay between pages to avoid rate limiting

Configuration (top of Program.cs)

const bool DOWNLOAD_FILES   = true;        // set false to skip file downloads
const long MAX_FILE_BYTES   = 50_000_000;  // skip files larger than 50 MB
const int  PAGE_DELAY_MS    = 900;         // delay between pages (ms)
const int  LOGIN_TIMEOUT_MS = 180_000;     // time to complete Google login (ms)

For personal study use only. Check CG Spectrum's Terms of Service.