Viking Content Crawler

When to Use

Use this skill when the user wants to crawl content from websites and import it into Viking AI Search to build a searchable knowledge base. This covers news sites, blogs, academic papers, GitHub repositories, product documentation, RSS feeds, and similar web content sources.

The agent writes crawler code tailored to the target sites, outputs data in a fixed JSONL schema, and then hands off to the vs-item-onboarding skill for dataset creation and import.

Do not use this skill when:

• The user already has a local file ready to import (use vs-item-onboarding directly).
• The user wants to import from a database (use vs-item-onboarding directly with MySQL).

Fixed Schema

All crawled records MUST conform to this schema. Every record is a flat JSON object written as one line in a JSONL file.