The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The skill provides instructions to download source code from Google's official GitHub repository at https://github.com/google/sentencepiece.git for C++ installation.
[PRIVILEGE_ESCALATION]: The installation documentation includes the use of sudo make install to install the compiled C++ library into system directories. This is a standard administrative procedure for system-wide software installation.
[COMMAND_EXECUTION]: The skill includes shell commands for building the project using cmake and make, as well as using the spm_train command-line utility to generate tokenization models from text files.
[INDIRECT_PROMPT_INJECTION]:
Ingestion points: The skill processes untrusted text data from local files (e.g., data.txt) or external datasets (e.g., HuggingFace's wikitext) during the model training phase in SKILL.md and references/training.md.
Boundary markers: No explicit boundary markers or instructions to ignore embedded commands are present in the training code examples.
Capability inventory: The skill uses the SentencePiece library to analyze text. It does not perform network requests or unauthorized file modifications based on the contents of the processed text.
Sanitization: The tokenizer treats input as raw Unicode text and does not perform filtering or sanitization, which is expected behavior for a subword tokenization tool.

sentencepiece