The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The skill documentation references the official SentencePiece repository on GitHub for C++ installation and documentation. This is a well-known repository maintained by Google.
[COMMAND_EXECUTION]: The instructions include standard shell commands for package management, build processes, and library usage, such as pip install, cmake, make, and spm_train.
[PRIVILEGE_ESCALATION]: Documentation for the C++ quick start includes a sudo make install command. This is a standard and expected procedure for installing software to system-wide paths on Unix-like operating systems.
[INDIRECT_PROMPT_INJECTION]: The skill documentation describes how to process external text data for training and tokenization, which is the primary function of the tokenizer.
Ingestion points: External text files (e.g., data.txt, corpus.txt) and Python iterators (e.g., wikitext dataset) are used as training inputs.
Boundary markers: None explicitly defined in the provided introductory examples.
Capability inventory: The library provides text encoding, decoding, and model training capabilities; usage examples demonstrate file system access for reading training corpora and writing model files.
Sanitization: Not addressed in the basic documentation examples, as is common for library usage guides.

sentencepiece