[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MBC5G00S3KXS9871M4XEYQ":3,"skill-same:01M6MBC5G00S3KXS9871M4XEYQ":110},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":41,"linkOnly":42,"markdownContent":43,"markdownBody":44,"frontMatter":45,"refName":29,"commitSha":60,"skillDirectory":61,"forksCount":62,"repositoryHtmlUrl":63,"repositoryHomepageUrl":64,"ownerHtmlUrl":65,"ownerBlogUrl":66,"ownerDescription":67,"updatedAt":68,"install":69,"resources":81},"01M6MBC5G00S3KXS9871M4XEYQ","waypoint-bio",null,"Use when working with Outpost Bio's open microbiome foundation models - the Waypoint checkpoints (Waypoint-6m, Waypoint-45m, Waypoint-170m), the Atlas pretraining corpus, the Compass eight-task benchmark, or the `waypoint` CLI from the `waypoint-bio` package. Covers embedding microbiome samples, fine-tuning on taxonomic abundance data, benchmarking a checkpoint on Compass, pretraining a GPT-2 model on taxonomic abundance profiles, and converting MetaPhlAn, Kraken2, QIIME 2, or MGnify abundance tables into waypoint format.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1575,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],"waypoint","waypoint bio","read","2026","when","embedding","microbiome","scikit-learn","token","life","main","cli","model","access","open","tuning","Transformers","network","benchmark","learn","benchmarking","repos","2026-11-10 10:12:30",false,"---\nname: waypoint-bio\ndescription: Use when working with Outpost Bio's open microbiome foundation models - the Waypoint checkpoints (Waypoint-6m, Waypoint-45m, Waypoint-170m), the Atlas pretraining corpus, the Compass eight-task benchmark, or the `waypoint` CLI from the `waypoint-bio` package. Covers embedding microbiome samples, fine-tuning on taxonomic abundance data, benchmarking a checkpoint on Compass, pretraining a GPT-2 model on taxonomic abundance profiles, and converting MetaPhlAn, Kraken2, QIIME 2, or MGnify abundance tables into waypoint format.\nlicense: MIT\ncompatibility: Requires Python 3.10+ with `waypoint-bio` (pulls torch, transformers, datasets, peft, scikit-learn). Needs network access and a Hugging Face token with access granted to the gated outpost-bio repos. A GPU is strongly recommended for pretraining and benchmarking.\nmetadata:\n  version: '1.1'\n  category: life-sciences\n  maintainer: Kalaris Labs\n  upstream-version: waypoint-bio 1.0.2 (PyPI); GitHub main 1.0.4\n  last-reviewed: '2026-08-17'\n  openclaw:\n    primaryEnv: HF_TOKEN\n    envVars:\n    - name: HF_TOKEN\n      required: true\n      description: Hugging Face read token with access to the gated outpost-bio\u002FWaypoint-*, outpost-bio\u002FAtlas, and outpost-bio\u002FCompass repos.\n  tags: microbiome, foundation models, metagenomics, benchmarks\n---\n\n# Waypoint: Outpost Bio's Open Microbiome Foundation Models\n\n## Overview\n\nOutpost Bio open-sourced three artefacts under Apache 2.0, described in\n[Treloar et al., bioRxiv 2026.05.02.722381](https:\u002F\u002Fwww.biorxiv.org\u002Fcontent\u002F10.64898\u002F2026.05.02.722381v2):\n\n| Artefact | What it is | Hugging Face |\n| --- | --- | --- |\n| **Waypoint** | GPT-2-style causal LMs over taxonomic tokens, 6M–170M params | `outpost-bio\u002FWaypoint-6m`, `-45m`, `-170m` |\n| **Atlas** | 539,308 microbiome samples scraped from MGnify (485,377 pretrain \u002F 53,931 benchmark) | `outpost-bio\u002FAtlas` |\n| **Compass** | Eight downstream tasks over four studies | `outpost-bio\u002FCompass` |\n\nThe unifying idea: a microbiome sample is a *sentence*. Each taxon is one token, tokens are ordered\nby descending abundance z-score, and the model is trained with next-token prediction. A pretrained\ncheckpoint then supplies sample-level embeddings or a fine-tuning backbone for prediction tasks.\n\nAll of it is driven by one CLI, `waypoint`, with five subcommands: `prepare-dataset`, `embed`,\n`finetune`, `benchmark`, `pretrain`.\n\n## When to use\n\n- Embedding 16S\u002Fshotgun taxonomic profiles into fixed-size vectors for clustering, visualisation, or\n  a downstream classifier.\n- Fine-tuning a Waypoint checkpoint to predict a phenotype, treatment, or continuous readout from\n  community composition.\n- Scoring your own microbiome model against Compass so the number is comparable to the paper.\n- Pretraining a taxonomic language model on Atlas or on your own corpus.\n- Converting profiler output (MetaPhlAn, Kraken2\u002FBracken, QIIME 2, MGnify TSVs) into the input format\n  these tools expect.\n\n**Do not reach for this** when you have fewer than ~1,000 labelled samples — see\n[Scientific caveats](#scientific-caveats). A random forest on relative abundances is the better tool\nthere, and the paper says so.\n\n## Setup\n\n```bash\npip install waypoint-bio       # installs the `waypoint` command\n```\n\nAtlas, Compass, and every Waypoint checkpoint are **gated**. Access is auto-approved, but you must\nclick through once per repo and then authenticate:\n\n1. Request access on each repo page you need: [Waypoint-6m](https:\u002F\u002Fhuggingface.co\u002Foutpost-bio\u002FWaypoint-6m),\n   [Waypoint-45m](https:\u002F\u002Fhuggingface.co\u002Foutpost-bio\u002FWaypoint-45m),\n   [Waypoint-170m](https:\u002F\u002Fhuggingface.co\u002Foutpost-bio\u002FWaypoint-170m),\n   [Atlas](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Foutpost-bio\u002FAtlas),\n   [Compass](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Foutpost-bio\u002FCompass).\n2. Authenticate locally:\n\n   ```bash\n   hf auth login          # or: export HF_TOKEN=hf_...\n   ```\n\nA 401\u002F403 from any subcommand almost always means access was never requested on that specific repo —\na token alone is not enough. Use a read-scoped token. The tokenizer loads via\n`trust_remote_code=True`, so pin a `revision` if you need the remote code fixed across runs.\n\n## The waypoint data format\n\nEverything except `prepare-dataset` consumes **waypoint format**: a `.parquet` \u002F `.csv` \u002F `.tsv`\nwhose rows are samples, with two aligned list-columns plus any label columns you need.\n\n| Column | Type | Notes |\n| --- | --- | --- |\n| `Taxa` | `list[str]` | Full lineage strings, `;`-separated: `k__Bacteria; p__Firmicutes; ...; g__Lactobacillus` |\n| `Relative Abundances` | `list[float]` | Same length as `Taxa`, same order |\n| *(any)* | scalar | Targets, covariates, or a `Split` column |\n\nPrefer parquet. CSV\u002FTSV stores the lists as `repr` strings and round-trips through `ast.literal_eval`.\n\n**Give full lineages, not bare names.** The tokenizer extracts the genus segment (`g__`) from each\nlineage and falls back to the most specific higher rank when genus is missing. Bare names disable\nthat fallback entirely.\n\n## Workflow\n\n### 1. Get your data into waypoint format\n\nIf you already have a sample × taxa (or taxa × sample) abundance matrix with lineage labels:\n\n```bash\nwaypoint prepare-dataset \\\n    --input abundance_matrix.tsv \\\n    --metadata sample_labels.csv \\\n    --output dataset.parquet\n```\n\nOrientation is auto-detected from the first column header (`taxonomy`, `lineage`, `taxon`, `otu`,\n`#otu id` ⇒ taxa-as-rows); override with `--orientation`. Rows are normalised to sum to 1 unless you\npass `--no_normalize`, and zeros are dropped unless you pass `--keep_zeros`.\n\n`prepare-dataset` cannot read profiler output directly — MetaPhlAn uses `|` separators, Kraken2\nreports encode the hierarchy as indentation, and QIIME 2\u002FSILVA prefixes the domain `d__` instead of\n`k__` (which the tokenizer silently ignores). Use the bundled converter for those:\n\n```bash\npython scripts\u002Fprofiler_to_waypoint.py \\\n    --input merged_metaphlan.tsv --format metaphlan \\\n    --output dataset.parquet\n\npython scripts\u002Fprofiler_to_waypoint.py \\\n    --input reports\u002F*.kreport --format kraken \\\n    --output dataset.parquet\n\npython scripts\u002Fprofiler_to_waypoint.py \\\n    --input feature-table.tsv --format qiime2 \\\n    --output dataset.parquet\n```\n\nSee `references\u002Fdata-preparation.md` for every input layout, rank handling, and the `d__`\u002F`|` gotchas.\n\n### 2. Check vocabulary coverage before anything else\n\nWaypoint's vocabulary is fixed at pretraining time from Atlas. Taxa absent from it become `\u003Cunk>` and\nare **silently dropped** by `waypoint embed`; the paper names this as the models' main limitation. A\nsample whose taxa are all out-of-vocabulary yields a degenerate `[BOS][EOS]` embedding.\n\n```bash\npython scripts\u002Fvocab_coverage.py --model outpost-bio\u002FWaypoint-6m --data dataset.parquet\n```\n\nIt reports per-sample and abundance-weighted coverage and flags samples below a threshold. Treat\nmedian abundance-weighted coverage under ~0.8 as a reason to re-examine your taxonomy labels before\ntrusting any downstream number.\n\n### 3. Embed samples\n\n```bash\nwaypoint embed \\\n    --model outpost-bio\u002FWaypoint-6m \\\n    --data dataset.parquet \\\n    --output embeddings.parquet\n```\n\nOutput is indexed by sample ID with columns `dim_0 … dim_{H-1}` (`H` = 256 for 6m, 512 for 45m,\n768 for 170m). Defaults: `--pooling last_token`, `--batch_size 32`, `--max_length 512`, device\nauto-detected (`cuda` → `mps` → `cpu`).\n\nKeep `--pooling last_token` unless you have a reason to change it: it matches how the checkpoints\nwere pretrained and how `benchmark` and `finetune` pool. `mean` is a reasonable alternative for\nunsupervised use; `first_token`\u002F`cls_token` return the BOS position and carry little signal in a\ncausal LM.\n\n### 4. Fine-tune on your labels\n\n```bash\n# classification\nwaypoint finetune \\\n    --model outpost-bio\u002FWaypoint-45m \\\n    --data dataset.parquet \\\n    --output_dir outputs\u002Fft_disease \\\n    --task_type classification \\\n    --target \"Disease Status\" \\\n    --config configs\u002Ffinetune_classification.yaml\n\n# regression, with a categorical covariate one-hot appended to the pooled embedding\nwaypoint finetune \\\n    --model outpost-bio\u002FWaypoint-45m \\\n    --data dataset.parquet \\\n    --output_dir outputs\u002Fft_degradation \\\n    --task_type regression \\\n    --target \"Degradation Rate\" \\\n    --covariate_column Drug \\\n    --config configs\u002Ffinetune_regression.yaml\n```\n\nConfig paths resolve against the bundled `waypoint_bio\u002Fconfigs\u002F` tree, so `configs\u002F...` works from\nany directory without cloning.\n\nDefaults worth overriding for small datasets: `warmup_steps: 1000` (drop to ~50 so warmup finishes\nbefore early stopping), `num_epochs: 1` in the shipped configs (raise it — early stopping on\nvalidation loss is what actually terminates training), and `use_lora: true` when VRAM is tight\n(~1% of parameters trained; adapters are merged back before saving, so the checkpoint stays a plain\n`AutoModel`).\n\nSplits default to a random 80\u002F10\u002F10. **Set `split_column` to a `Split` column whenever samples are\ncorrelated** — repeated measures, one donor sampled over time, technical replicates — or a random\nsplit leaks and the test score is meaningless.\n\nOutputs land in `--output_dir`: `best_model\u002F` (loadable by `embed`\u002F`benchmark`),\n`test_metrics.json`, `training_log.csv` + `.html`, and `finetune_results.json`.\n\n### 5. Benchmark on Compass\n\n```bash\nwaypoint benchmark --model outpost-bio\u002FWaypoint-6m --output_dir outputs\u002Fbenchmark\nwaypoint benchmark --model outputs\u002Fpretrain\u002Fbest_model --tasks 1 6 --output_dir outputs\u002Fsmoke\n```\n\nFine-tunes a fresh head per task and writes `benchmark_results.json`. Classification tasks score\nmacro-F1; the one regression task scores R² clamped to [0, 1]; `final_score` is the unweighted mean\nacross tasks. Full task table, metric keys, and result-file schema: `references\u002Fcompass-benchmark.md`.\n\n### 6. Pretrain\n\n```bash\nwaypoint pretrain \\\n    --model_config configs\u002Fmodels\u002Fgpt2-45m.yaml \\\n    --pretrain_config configs\u002Fpretraining.yaml \\\n    --output_dir outputs\u002Fpretrain_45m\n```\n\nDownloads Atlas, builds a taxonomic tokenizer from the corpus, computes per-token abundance\nmean\u002Fstd for z-score ordering, then trains with next-token prediction and early stopping. Add\n`--data my_corpus.parquet` to pretrain on your own waypoint-format corpus instead, and\n`--max_samples N` for a smoke test.\n\nNine architectures ship, from `gpt2-6m.yaml` (8 layers, 256 hidden) to `gpt2-170m.yaml` (24 layers,\n768 hidden); per-head dimension is fixed at 64 throughout. `references\u002Fcli-reference.md` has the\nfull table and every config key.\n\n## Scientific caveats\n\nThese are load-bearing. Ignoring them produces numbers that look fine and mean nothing.\n\n- **Below ~1,000 labelled examples, Waypoint underperforms a random forest on raw abundances.** The\n  paper's crossover against the RF baseline sits near **10,000** training examples. Fit the baseline\n  first; only adopt the transformer if it wins on your data.\n- **Out-of-vocabulary taxa are dropped, not flagged.** Every Compass dataset carries some. Run\n  `scripts\u002Fvocab_coverage.py` and report the coverage alongside your results.\n- **45M, not 170M, was the best benchmark model.** Pretraining loss keeps falling with scale, but\n  downstream Compass score does not — start at 6m or 45m and only scale up if it demonstrably helps.\n- **Genus-level tokenisation is the default**, so species-level distinctions are collapsed. Changing\n  `taxon_rank` requires re-pretraining, not just re-tokenising.\n- **Compositional data.** Relative abundances are constrained to sum to 1; differences in one taxon\n  induce apparent changes in others. This affects interpretation of any per-taxon attribution.\n- **Batch and study effects dominate microbiome data.** Atlas spans MGnify pipelines v1.0–v5.0 and\n  four sequencing modalities. Never let a study or run boundary coincide with your label boundary.\n- **Not a clinical or diagnostic tool.** The model cards state this explicitly.\n\n## References\n\n- `references\u002Fcli-reference.md` — every subcommand flag, every config key, the model-size table.\n- `references\u002Fcompass-benchmark.md` — the eight tasks, filters, metrics, `benchmark_results.json` schema.\n- `references\u002Fdata-preparation.md` — waypoint format, profiler conversions, taxonomy string rules.\n- `references\u002Fpython-api.md` — using the tokenizer, datasets, heads, and checkpoints from Python.\n\n## Scripts\n\n- `scripts\u002Fprofiler_to_waypoint.py` — MetaPhlAn \u002F Kraken2 \u002F QIIME 2 \u002F generic lineage tables → waypoint format.\n- `scripts\u002Fvocab_coverage.py` — tokenizer coverage report for a waypoint-format file.\n\n## Upstream\n\nCode [github.com\u002FOutpost-Bio\u002Fwaypoint](https:\u002F\u002Fgithub.com\u002FOutpost-Bio\u002Fwaypoint) ·\npackage `waypoint-bio` ·\npaper [bioRxiv 2026.05.02.722381](https:\u002F\u002Fwww.biorxiv.org\u002Fcontent\u002F10.64898\u002F2026.05.02.722381v2) ·\ncommunity [Waypoint Slack](https:\u002F\u002Fjoin.slack.com\u002Ft\u002Foutpostbio-waypoint\u002Fshared_invite\u002Fzt-3w6ivgtba-WJOCkdxiISxQpwVq9ZZxTA) ·\ncontact `waypoint@outpost.bio`.\n\nCite Treloar, N. J., Ur-Rehman, S., Yang, J., & Outpost Bio (2026). *Learning the Language of the\nMicrobiome with Transformers.* bioRxiv. Per-artefact DOIs are listed at\n[outpost.bio\u002Fcitations](https:\u002F\u002Fwww.outpost.bio\u002Fcitations).\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm tool versions, the reference genome\u002Fannotation build and the input formats (FASTQ, BAM, VCF, h5ad).\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run the pipeline on a small subset (one sample, one chromosome, a few thousand cells) first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Check QC metrics, sample identities, genome build consistency and batch effects before interpreting results.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Genome builds or identifiers do not match between inputs | Stop and harmonize (liftover, ID mapping) before continuing. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Do not interpret biological significance beyond what the statistics support; report multiple-testing correction.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `scikit-bio`: Python library scikit-bio for biological sequence and community-ecology analysis: DNA\u002FRNA\u002Fprotein sequences, pair_align alignment, phylogen…\n- `distributed-llm-pretraining-torchtitan`: Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP).\n","# Waypoint: Outpost Bio's Open Microbiome Foundation Models\n\n## Overview\n\nOutpost Bio open-sourced three artefacts under Apache 2.0, described in\n[Treloar et al., bioRxiv 2026.05.02.722381](https:\u002F\u002Fwww.biorxiv.org\u002Fcontent\u002F10.64898\u002F2026.05.02.722381v2):\n\n| Artefact | What it is | Hugging Face |\n| --- | --- | --- |\n| **Waypoint** | GPT-2-style causal LMs over taxonomic tokens, 6M–170M params | `outpost-bio\u002FWaypoint-6m`, `-45m`, `-170m` |\n| **Atlas** | 539,308 microbiome samples scraped from MGnify (485,377 pretrain \u002F 53,931 benchmark) | `outpost-bio\u002FAtlas` |\n| **Compass** | Eight downstream tasks over four studies | `outpost-bio\u002FCompass` |\n\nThe unifying idea: a microbiome sample is a *sentence*. Each taxon is one token, tokens are ordered\nby descending abundance z-score, and the model is trained with next-token prediction. A pretrained\ncheckpoint then supplies sample-level embeddings or a fine-tuning backbone for prediction tasks.\n\nAll of it is driven by one CLI, `waypoint`, with five subcommands: `prepare-dataset`, `embed`,\n`finetune`, `benchmark`, `pretrain`.\n\n## When to use\n\n- Embedding 16S\u002Fshotgun taxonomic profiles into fixed-size vectors for clustering, visualisation, or\n  a downstream classifier.\n- Fine-tuning a Waypoint checkpoint to predict a phenotype, treatment, or continuous readout from\n  community composition.\n- Scoring your own microbiome model against Compass so the number is comparable to the paper.\n- Pretraining a taxonomic language model on Atlas or on your own corpus.\n- Converting profiler output (MetaPhlAn, Kraken2\u002FBracken, QIIME 2, MGnify TSVs) into the input format\n  these tools expect.\n\n**Do not reach for this** when you have fewer than ~1,000 labelled samples — see\n[Scientific caveats](#scientific-caveats). A random forest on relative abundances is the better tool\nthere, and the paper says so.\n\n## Setup\n\n```bash\npip install waypoint-bio       # installs the `waypoint` command\n```\n\nAtlas, Compass, and every Waypoint checkpoint are **gated**. Access is auto-approved, but you must\nclick through once per repo and then authenticate:\n\n1. Request access on each repo page you need: [Waypoint-6m](https:\u002F\u002Fhuggingface.co\u002Foutpost-bio\u002FWaypoint-6m),\n   [Waypoint-45m](https:\u002F\u002Fhuggingface.co\u002Foutpost-bio\u002FWaypoint-45m),\n   [Waypoint-170m](https:\u002F\u002Fhuggingface.co\u002Foutpost-bio\u002FWaypoint-170m),\n   [Atlas](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Foutpost-bio\u002FAtlas),\n   [Compass](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Foutpost-bio\u002FCompass).\n2. Authenticate locally:\n\n   ```bash\n   hf auth login          # or: export HF_TOKEN=hf_...\n   ```\n\nA 401\u002F403 from any subcommand almost always means access was never requested on that specific repo —\na token alone is not enough. Use a read-scoped token. The tokenizer loads via\n`trust_remote_code=True`, so pin a `revision` if you need the remote code fixed across runs.\n\n## The waypoint data format\n\nEverything except `prepare-dataset` consumes **waypoint format**: a `.parquet` \u002F `.csv` \u002F `.tsv`\nwhose rows are samples, with two aligned list-columns plus any label columns you need.\n\n| Column | Type | Notes |\n| --- | --- | --- |\n| `Taxa` | `list[str]` | Full lineage strings, `;`-separated: `k__Bacteria; p__Firmicutes; ...; g__Lactobacillus` |\n| `Relative Abundances` | `list[float]` | Same length as `Taxa`, same order |\n| *(any)* | scalar | Targets, covariates, or a `Split` column |\n\nPrefer parquet. CSV\u002FTSV stores the lists as `repr` strings and round-trips through `ast.literal_eval`.\n\n**Give full lineages, not bare names.** The tokenizer extracts the genus segment (`g__`) from each\nlineage and falls back to the most specific higher rank when genus is missing. Bare names disable\nthat fallback entirely.\n\n## Workflow\n\n### 1. Get your data into waypoint format\n\nIf you already have a sample × taxa (or taxa × sample) abundance matrix with lineage labels:\n\n```bash\nwaypoint prepare-dataset \\\n    --input abundance_matrix.tsv \\\n    --metadata sample_labels.csv \\\n    --output dataset.parquet\n```\n\nOrientation is auto-detected from the first column header (`taxonomy`, `lineage`, `taxon`, `otu`,\n`#otu id` ⇒ taxa-as-rows); override with `--orientation`. Rows are normalised to sum to 1 unless you\npass `--no_normalize`, and zeros are dropped unless you pass `--keep_zeros`.\n\n`prepare-dataset` cannot read profiler output directly — MetaPhlAn uses `|` separators, Kraken2\nreports encode the hierarchy as indentation, and QIIME 2\u002FSILVA prefixes the domain `d__` instead of\n`k__` (which the tokenizer silently ignores). Use the bundled converter for those:\n\n```bash\npython scripts\u002Fprofiler_to_waypoint.py \\\n    --input merged_metaphlan.tsv --format metaphlan \\\n    --output dataset.parquet\n\npython scripts\u002Fprofiler_to_waypoint.py \\\n    --input reports\u002F*.kreport --format kraken \\\n    --output dataset.parquet\n\npython scripts\u002Fprofiler_to_waypoint.py \\\n    --input feature-table.tsv --format qiime2 \\\n    --output dataset.parquet\n```\n\nSee `references\u002Fdata-preparation.md` for every input layout, rank handling, and the `d__`\u002F`|` gotchas.\n\n### 2. Check vocabulary coverage before anything else\n\nWaypoint's vocabulary is fixed at pretraining time from Atlas. Taxa absent from it become `\u003Cunk>` and\nare **silently dropped** by `waypoint embed`; the paper names this as the models' main limitation. A\nsample whose taxa are all out-of-vocabulary yields a degenerate `[BOS][EOS]` embedding.\n\n```bash\npython scripts\u002Fvocab_coverage.py --model outpost-bio\u002FWaypoint-6m --data dataset.parquet\n```\n\nIt reports per-sample and abundance-weighted coverage and flags samples below a threshold. Treat\nmedian abundance-weighted coverage under ~0.8 as a reason to re-examine your taxonomy labels before\ntrusting any downstream number.\n\n### 3. Embed samples\n\n```bash\nwaypoint embed \\\n    --model outpost-bio\u002FWaypoint-6m \\\n    --data dataset.parquet \\\n    --output embeddings.parquet\n```\n\nOutput is indexed by sample ID with columns `dim_0 … dim_{H-1}` (`H` = 256 for 6m, 512 for 45m,\n768 for 170m). Defaults: `--pooling last_token`, `--batch_size 32`, `--max_length 512`, device\nauto-detected (`cuda` → `mps` → `cpu`).\n\nKeep `--pooling last_token` unless you have a reason to change it: it matches how the checkpoints\nwere pretrained and how `benchmark` and `finetune` pool. `mean` is a reasonable alternative for\nunsupervised use; `first_token`\u002F`cls_token` return the BOS position and carry little signal in a\ncausal LM.\n\n### 4. Fine-tune on your labels\n\n```bash\n# classification\nwaypoint finetune \\\n    --model outpost-bio\u002FWaypoint-45m \\\n    --data dataset.parquet \\\n    --output_dir outputs\u002Fft_disease \\\n    --task_type classification \\\n    --target \"Disease Status\" \\\n    --config configs\u002Ffinetune_classification.yaml\n\n# regression, with a categorical covariate one-hot appended to the pooled embedding\nwaypoint finetune \\\n    --model outpost-bio\u002FWaypoint-45m \\\n    --data dataset.parquet \\\n    --output_dir outputs\u002Fft_degradation \\\n    --task_type regression \\\n    --target \"Degradation Rate\" \\\n    --covariate_column Drug \\\n    --config configs\u002Ffinetune_regression.yaml\n```\n\nConfig paths resolve against the bundled `waypoint_bio\u002Fconfigs\u002F` tree, so `configs\u002F...` works from\nany directory without cloning.\n\nDefaults worth overriding for small datasets: `warmup_steps: 1000` (drop to ~50 so warmup finishes\nbefore early stopping), `num_epochs: 1` in the shipped configs (raise it — early stopping on\nvalidation loss is what actually terminates training), and `use_lora: true` when VRAM is tight\n(~1% of parameters trained; adapters are merged back before saving, so the checkpoint stays a plain\n`AutoModel`).\n\nSplits default to a random 80\u002F10\u002F10. **Set `split_column` to a `Split` column whenever samples are\ncorrelated** — repeated measures, one donor sampled over time, technical replicates — or a random\nsplit leaks and the test score is meaningless.\n\nOutputs land in `--output_dir`: `best_model\u002F` (loadable by `embed`\u002F`benchmark`),\n`test_metrics.json`, `training_log.csv` + `.html`, and `finetune_results.json`.\n\n### 5. Benchmark on Compass\n\n```bash\nwaypoint benchmark --model outpost-bio\u002FWaypoint-6m --output_dir outputs\u002Fbenchmark\nwaypoint benchmark --model outputs\u002Fpretrain\u002Fbest_model --tasks 1 6 --output_dir outputs\u002Fsmoke\n```\n\nFine-tunes a fresh head per task and writes `benchmark_results.json`. Classification tasks score\nmacro-F1; the one regression task scores R² clamped to [0, 1]; `final_score` is the unweighted mean\nacross tasks. Full task table, metric keys, and result-file schema: `references\u002Fcompass-benchmark.md`.\n\n### 6. Pretrain\n\n```bash\nwaypoint pretrain \\\n    --model_config configs\u002Fmodels\u002Fgpt2-45m.yaml \\\n    --pretrain_config configs\u002Fpretraining.yaml \\\n    --output_dir outputs\u002Fpretrain_45m\n```\n\nDownloads Atlas, builds a taxonomic tokenizer from the corpus, computes per-token abundance\nmean\u002Fstd for z-score ordering, then trains with next-token prediction and early stopping. Add\n`--data my_corpus.parquet` to pretrain on your own waypoint-format corpus instead, and\n`--max_samples N` for a smoke test.\n\nNine architectures ship, from `gpt2-6m.yaml` (8 layers, 256 hidden) to `gpt2-170m.yaml` (24 layers,\n768 hidden); per-head dimension is fixed at 64 throughout. `references\u002Fcli-reference.md` has the\nfull table and every config key.\n\n## Scientific caveats\n\nThese are load-bearing. Ignoring them produces numbers that look fine and mean nothing.\n\n- **Below ~1,000 labelled examples, Waypoint underperforms a random forest on raw abundances.** The\n  paper's crossover against the RF baseline sits near **10,000** training examples. Fit the baseline\n  first; only adopt the transformer if it wins on your data.\n- **Out-of-vocabulary taxa are dropped, not flagged.** Every Compass dataset carries some. Run\n  `scripts\u002Fvocab_coverage.py` and report the coverage alongside your results.\n- **45M, not 170M, was the best benchmark model.** Pretraining loss keeps falling with scale, but\n  downstream Compass score does not — start at 6m or 45m and only scale up if it demonstrably helps.\n- **Genus-level tokenisation is the default**, so species-level distinctions are collapsed. Changing\n  `taxon_rank` requires re-pretraining, not just re-tokenising.\n- **Compositional data.** Relative abundances are constrained to sum to 1; differences in one taxon\n  induce apparent changes in others. This affects interpretation of any per-taxon attribution.\n- **Batch and study effects dominate microbiome data.** Atlas spans MGnify pipelines v1.0–v5.0 and\n  four sequencing modalities. Never let a study or run boundary coincide with your label boundary.\n- **Not a clinical or diagnostic tool.** The model cards state this explicitly.\n\n## References\n\n- `references\u002Fcli-reference.md` — every subcommand flag, every config key, the model-size table.\n- `references\u002Fcompass-benchmark.md` — the eight tasks, filters, metrics, `benchmark_results.json` schema.\n- `references\u002Fdata-preparation.md` — waypoint format, profiler conversions, taxonomy string rules.\n- `references\u002Fpython-api.md` — using the tokenizer, datasets, heads, and checkpoints from Python.\n\n## Scripts\n\n- `scripts\u002Fprofiler_to_waypoint.py` — MetaPhlAn \u002F Kraken2 \u002F QIIME 2 \u002F generic lineage tables → waypoint format.\n- `scripts\u002Fvocab_coverage.py` — tokenizer coverage report for a waypoint-format file.\n\n## Upstream\n\nCode [github.com\u002FOutpost-Bio\u002Fwaypoint](https:\u002F\u002Fgithub.com\u002FOutpost-Bio\u002Fwaypoint) ·\npackage `waypoint-bio` ·\npaper [bioRxiv 2026.05.02.722381](https:\u002F\u002Fwww.biorxiv.org\u002Fcontent\u002F10.64898\u002F2026.05.02.722381v2) ·\ncommunity [Waypoint Slack](https:\u002F\u002Fjoin.slack.com\u002Ft\u002Foutpostbio-waypoint\u002Fshared_invite\u002Fzt-3w6ivgtba-WJOCkdxiISxQpwVq9ZZxTA) ·\ncontact `waypoint@outpost.bio`.\n\nCite Treloar, N. J., Ur-Rehman, S., Yang, J., & Outpost Bio (2026). *Learning the Language of the\nMicrobiome with Transformers.* bioRxiv. Per-artefact DOIs are listed at\n[outpost.bio\u002Fcitations](https:\u002F\u002Fwww.outpost.bio\u002Fcitations).\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm tool versions, the reference genome\u002Fannotation build and the input formats (FASTQ, BAM, VCF, h5ad).\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run the pipeline on a small subset (one sample, one chromosome, a few thousand cells) first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Check QC metrics, sample identities, genome build consistency and batch effects before interpreting results.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Genome builds or identifiers do not match between inputs | Stop and harmonize (liftover, ID mapping) before continuing. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Do not interpret biological significance beyond what the statistics support; report multiple-testing correction.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `scikit-bio`: Python library scikit-bio for biological sequence and community-ecology analysis: DNA\u002FRNA\u002Fprotein sequences, pair_align alignment, phylogen…\n- `distributed-llm-pretraining-torchtitan`: Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP).\n",{"name":5,"license":15,"metadata":46,"description":7,"compatibility":59},{"tags":47,"version":48,"category":49,"openclaw":50,"maintainer":56,"last-reviewed":57,"upstream-version":58},"microbiome, foundation models, metagenomics, benchmarks","1.1","life-sciences",{"envVars":51,"primaryEnv":53},[52],{"name":53,"required":54,"description":55},"HF_TOKEN",true,"Hugging Face read token with access to the gated outpost-bio\u002FWaypoint-*, outpost-bio\u002FAtlas, and outpost-bio\u002FCompass repos.","Kalaris Labs","2026-08-17","waypoint-bio 1.0.2 (PyPI); GitHub main 1.0.4","Requires Python 3.10+ with `waypoint-bio` (pulls torch, transformers, datasets, peft, scikit-learn). Needs network access and a Hugging Face token with access granted to the gated outpost-bio repos. A GPU is strongly recommended for pretraining and benchmarking.","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio",491,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-14 14:06:13",{"source":8,"skillName":5,"commands":70,"prompt":80,"zipAvailable":54},[71,74,77],{"tool":72,"command":73},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill waypoint-bio",{"tool":75,"command":76},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill waypoint-bio",{"tool":78,"command":79},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill waypoint-bio","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002FSKILL.md",[82,88,92,96,100,106],{"path":83,"resourceType":84,"resourceTypeDesc":85,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Fcli-reference.md","REFERENCE","参考文档",8988,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002Freferences\u002Fcli-reference.md",{"path":89,"resourceType":84,"resourceTypeDesc":85,"sizeBytes":90,"sourceHtmlUrl":91},"references\u002Fcompass-benchmark.md",6952,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002Freferences\u002Fcompass-benchmark.md",{"path":93,"resourceType":84,"resourceTypeDesc":85,"sizeBytes":94,"sourceHtmlUrl":95},"references\u002Fdata-preparation.md",8397,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002Freferences\u002Fdata-preparation.md",{"path":97,"resourceType":84,"resourceTypeDesc":85,"sizeBytes":98,"sourceHtmlUrl":99},"references\u002Fpython-api.md",8440,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002Freferences\u002Fpython-api.md",{"path":101,"resourceType":102,"resourceTypeDesc":103,"sizeBytes":104,"sourceHtmlUrl":105},"scripts\u002Fprofiler_to_waypoint.py","SCRIPT","脚本",16893,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002Fscripts\u002Fprofiler_to_waypoint.py",{"path":107,"resourceType":102,"resourceTypeDesc":103,"sizeBytes":108,"sourceHtmlUrl":109},"scripts\u002Fvocab_coverage.py",8318,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fwaypoint-bio\u002Fscripts\u002Fvocab_coverage.py",[111,120,128,136,144,152],{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":115,"sourceHtmlUrl":116,"categories":117,"keywords":118,"publishedAt":119},"01M7DV332YQMGD8WTRJ9DZQXTB","worktable-review-with-annotations","Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FSKILL.md",[],[],"2026-11-15 14:19:17",{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":124,"sourceHtmlUrl":125,"categories":126,"keywords":127,"publishedAt":119},"01M7DV2YGHCZVBEB1AHYB6CGHE","worktable-find-and-synthesize-context","Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FSKILL.md",[],[],{"skillNo":129,"name":130,"nameZh":6,"description":131,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":132,"sourceHtmlUrl":133,"categories":134,"keywords":135,"publishedAt":119},"01M7DV2TQN1H11ME2SAF9MXN7V","worktable-create-or-update-docs","Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FSKILL.md",[],[],{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":140,"sourceHtmlUrl":141,"categories":142,"keywords":143,"publishedAt":119},"01M7DV2NE452NEA8TXATKSNAJH","worktable-create-or-manage-records","Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FSKILL.md",[],[],{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":148,"sourceHtmlUrl":149,"categories":150,"keywords":151,"publishedAt":119},"01M7DV2HENRBR2KSCVM064WH0Y","worktable-create-or-manage-interactive-html","Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FSKILL.md",[],[],{"skillNo":153,"name":154,"nameZh":6,"description":155,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":156,"sourceHtmlUrl":157,"categories":158,"keywords":159,"publishedAt":119},"01M7DV2CADR5R7RQFFYPNAGR9Y","worktable-collaborate-threads","Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FSKILL.md",[],[]]