Bio Foundation Housekeeping

Add validated metadata models and a queryable catalog to an existing bioinformatics project. This is an independent entry point when the layout already exists; if it does not, complete bioinformatics-project as a separate setup task.

Instructions

1. Confirm bioinformatics-project has established input/output boundaries, project records, and a pinned environment. Do not create a competing project layout.
2. Adapt schemas/project-metadata.yaml for the sample, run, file, result, and provenance records the project needs. Keep identifiers stable and declare types, required fields, enumerations, and patterns in the schema.
3. Generate Pydantic models with scripts/generate_models.py. The command pins LinkML and Pydantic, rejects changed outputs, imports the generated module, and can assert that expected classes exist.
4. Validate the complete metadata bundle with the generated MetadataBundle model. Reject unexpected fields and malformed types before checking relationships.
5. Check unique identifiers and foreign keys across record collections before writing outputs. At minimum, verify run-to-sample, file-to-run, result-to-input/output-file, and provenance-to-result links.
6. Normalize validated records into one Parquet table per record class plus bridge tables for multivalued relationships. Register the tables and their relative paths, row counts, and SHA-256 values in DuckDB.
7. Exercise the full boundary with scripts/build_metadata_catalog.py and the bundled valid, model-invalid, and foreign-key-invalid fixtures. Use scripts/build_sample_catalog.py only for the smaller sample-only smoke path.
8. Before adopting a project-specific extension or migrating stored records, run scripts/check_schema_compatibility.py. Optional slots and new classes are compatible; required additions and constraint changes are reported as breaking. Keep a versioned input/expected migration fixture like fixtures/schema-migration-v1-to-v1.1.json for every supported transition.

Quick Reference