Pathology Foundation Models × Spatial Transcriptomics
HEST-1k NCBI10 — H-Optimus-0 vs UNI2-h geometry, alignment, and morphology–expression coupling
This dashboard presents a side-by-side geometric and biological audit of two pathology foundation models applied to the same H&E tiles aligned to Visium spots. Rather than treating embeddings as opaque black boxes, we quantify representation geometry (PCA spectrum, norms, pairwise distances), cross-model congruence (cosine, CKA, k-NN agreement, Mantel, cluster ARI/NMI), and morphology–transcriptome coupling (Mantel vs expression PCs, PC–PC correlation heatmaps).
Sample NCBI10: WSI 8073×8593 px at
0.8519 MPP · 138 spots · 15997 genes ·
mean inter-model cosine similarity 0.0295.
1. Methods & experimental design
Reproducible pipeline from HEST-1k sample through dual foundation-model embedding extraction.
Data
HEST-1k pairs H&E WSIs with Visium spatial transcriptomics. We use sample NCBI10:
- WSI pyramid: 7 levels, level-0 8073×8593
- Microns/pixel: 0.8519
- Spot diameter (level-0 px): 131
- Expression matrix: 138 × 15997 (sparsity 0.8102)
Tile extraction
For each Visium centroid, a square crop of side ≈ spot diameter is taken from the WSI and resized to 224×224 for both models (standard FM input). Coordinates are stored in level-0 pixel space for spatial plotting.
- H-Optimus-0: ImageNet-style mean/std specialized for H&E
- UNI2-h: model
pretrained_cfgtransform viatimm - Inference: CPU, batch-wise,
torch.inference_mode()
Analysis metrics
- Linear CKA — representation similarity invariant to isotropic scaling / orthogonal transform
- k-NN agreement — local neighborhood overlap in cosine space
- Mantel (Spearman) — rank correlation of pairwise distance matrices (models, space, expression)
- k-means + silhouette / ARI / NMI — discrete partition concordance
- Expression PCs — log1p(CP10k) on top-variance genes, then PCA
H-Optimus-0
| Organization | Bioptimus |
|---|---|
| Parameters | ~1.1B |
| Architecture | ViT (DINOv2-style) |
| Output dim | 1536 |
| HF repo | bioptimus/H-optimus-0 |
| Mean L2 norm | 30.995 |
| PC1 / cum. PC1–5 | 17.86% / 44.8% |
| Silhouette (k=4) | 0.229 |
UNI2-h
| Organization | Mahmood Lab / Harvard-BWH |
|---|---|
| Parameters | 681M |
| Architecture | ViT-H/14 DINOv2 + reg tokens |
| Output dim | 1536 |
| HF repo | MahmoodLab/UNI2-h |
| Mean L2 norm | 13.72 |
| PC1 / cum. PC1–5 | 18.84% / 42.48% |
| Silhouette (k=4) | 0.182 |
2. Dataset quality control
Spot-level library complexity and gene detection before interpreting morphology embeddings.
Top genes by total counts
Mitochondrial % is zero for all spots in this HEST export (gene annotation flag), so mito QC is non-informative here. Sparsity = fraction of zero entries in the count matrix.
3. Spatial maps
Visium spots overlaid on the H&E thumbnail. Change the color channel to audit different covariates.
How to read this panel
Each marker is one Visium spot. Spatial structure in UMI / gene detection often reflects tissue cellularity and technical capture efficiency. If inter-model cosine similarity is spatially structured, the two FMs disagree more in some microenvironments (e.g., stroma vs epithelium) than others.
Coloring by k-means clusters tests whether morphology partitions form contiguous tissue domains — a weak but useful sanity check that embeddings encode histology rather than pure batch noise.
Coloring by expression PC1 vs morphology PC1 is a first visual check for shared spatial programs.
4. Embedding geometry
PCA spectra, UMAP manifolds, and pairwise distance structure for each foundation model.
Geometric reading
A steep PCA spectrum (large PC1 fraction) implies a dominant global mode — often stain intensity, tissue vs background residual, or bulk cellularity. Flatter spectra suggest more multi-factor morphological diversity.
UMAP is for visualization only; cluster labels and Mantel tests are computed in the original 1536-d (or PCA) spaces. Pairwise distance histograms characterize manifold concentration: tighter distances can indicate representation collapse or highly homogeneous tissue.
L2-norm differences across models are expected — each model uses different pretraining and projection heads. Compare structure (CKA, Mantel, NN) rather than raw norms.
5. Cross-model alignment
Do H-Optimus-0 and UNI2-h organize the same spots similarly?
Interpretation guide
- Linear CKA ≈ 1: representations related by linear transform; ≈ 0: unrelated.
- Mean cosine on paired spots can be low even when geometry is similar after rotation — prefer CKA / Mantel for geometry.
- NN agreement: local retrieval consistency (important for spot-neighborhood methods).
- Mantel ρ (models): global ranking of pairwise dissimilarities agrees.
- ARI / NMI: discrete cluster partitions agree beyond chance.
Low paired cosine with moderate CKA is common when models live in different cones of Rd but share relative structure.
6. Morphology ↔ expression coupling
Link tile embeddings to transcriptomic state via expression PCs and Mantel tests.
Expression features: log1p(counts-per-10k) on the 2,000 highest-variance genes, then PCA. Heatmaps show Pearson r between morphology PC1–5 and expression PC1–5. Strong off-diagonal or PC1–PC1 coupling would indicate that the dominant morphological axis tracks the dominant transcriptional program (e.g., proliferation, immune infiltrate, stroma).
7. Unsupervised morphological domains
k-means (k=4) in each embedding space; compare composition by library size and inter-model agreement.
8. Discussion, limitations & next experiments
Findings framing
With n=138 spots on a single sample, this is a methodological quickstart, not a population-scale benchmark. The value is in the analysis recipe: dual-FM embedding, multi-metric alignment, and explicit morphology–expression tests that can scale to full HEST-1k cohorts.
Limitations
- Single sample (NCBI10); no cross-slide batch effects or tissue-type stratification.
- H-Optimus-1 was not used (gated access pending); H-Optimus-0 is the deployed Bioptimus model.
- Tile MPP is approximate from Visium spot diameter; not a full multi-resolution pyramid embedding.
- No supervised labels (pathologist annotations, mutations) — clusters are unsupervised only.
- CKA/Mantel assume the chosen kernels/distances; cosine vs Euclidean can shift ranks.
Natural extensions
- Multi-sample HEST cohort; linear probe for tissue type / cancer subtype.
- Gene-set scores (Hallmark, cell-type signatures) as continuous labels for embedding regression.
- Attention/rollout or tile retrieval to explain clusters morphologically.
- Compare against CONCH, Virchow, Prov-GigaPath under the same tile grid.
- Slide-level MIL aggregation for diagnosis / survival tasks.
References (selected)
- HEST-1k — spatial transcriptomics + H&E foundation-model benchmark (Mahmood Lab).
- UNI / UNI2 — Chen et al., nature-scale pathology foundation models (Mahmood Lab).
- H-Optimus — Bioptimus open pathology FMs.
- Kornblith et al., Similarity of Neural Network Representations Revisited (CKA).
- Mantel 1967; spatial statistics in ecology / spatial omics.
Generated by scripts/build_dashboard.py · analytics snapshot also written to
outputs/NCBI10_phd_analytics.json · research / non-clinical use only.