Pathology Foundation Models × Spatial Transcriptomics
HEST-1k NCBI10 — H-Optimus-0 vs UNI2-h geometry, alignment, and morphology–expression coupling
This dashboard presents a side-by-side geometric and biological audit of two pathology foundation models applied to the same H&E tiles aligned to Visium spots. Rather than treating embeddings as opaque black boxes, we quantify representation geometry (PCA spectrum, norms, pairwise distances), cross-model congruence (cosine, CKA, k-NN agreement, Mantel, cluster ARI/NMI), and morphology–transcriptome coupling (Mantel vs expression PCs, PC–PC correlation heatmaps).
Sample NCBI10: WSI 8073×8593 px at
0.8519 MPP · 138 spots · 15997 genes ·
mean inter-model cosine similarity 0.0295.
1. Methods & experimental design
Reproducible pipeline from HEST-1k sample through dual foundation-model embedding extraction.
Data
HEST-1k pairs H&E WSIs with Visium spatial transcriptomics. We use sample NCBI10:
- WSI pyramid: 7 levels, level-0 8073×8593
- Microns/pixel: 0.8519
- Spot diameter (level-0 px): 131
- Expression matrix: 138 × 15997 (sparsity 0.8102)
Tile extraction
For each Visium centroid, a square crop of side ≈ spot diameter is taken from the WSI and resized to 224×224 for both models (standard FM input). Coordinates are stored in level-0 pixel space for spatial plotting.
- H-Optimus-0: ImageNet-style mean/std specialized for H&E
- UNI2-h: model
pretrained_cfgtransform viatimm - Inference: CPU, batch-wise,
torch.inference_mode()
Analysis metrics
- Linear CKA — representation similarity invariant to isotropic scaling / orthogonal transform
- k-NN agreement — local neighborhood overlap in cosine space
- Mantel (Spearman) — rank correlation of pairwise distance matrices (models, space, expression)
- k-means + silhouette / ARI / NMI — discrete partition concordance
- Expression PCs — log1p(CP10k) on top-variance genes, then PCA
H-Optimus-0
| Organization | Bioptimus |
|---|---|
| Parameters | ~1.1B |
| Architecture | ViT (DINOv2-style) |
| Output dim | 1536 |
| HF repo | bioptimus/H-optimus-0 |
| Mean L2 norm | 30.995 |
| PC1 / cum. PC1–5 | 17.86% / 44.8% |
| Silhouette (k=4) | 0.229 |
UNI2-h
| Organization | Mahmood Lab / Harvard-BWH |
|---|---|
| Parameters | 681M |
| Architecture | ViT-H/14 DINOv2 + reg tokens |
| Output dim | 1536 |
| HF repo | MahmoodLab/UNI2-h |
| Mean L2 norm | 13.72 |
| PC1 / cum. PC1–5 | 18.84% / 42.48% |
| Silhouette (k=4) | 0.182 |
2. Dataset quality control
Spot-level library complexity and gene detection before interpreting morphology embeddings.
Top genes by total counts
Mitochondrial % is zero for all spots in this HEST export (gene annotation flag), so mito QC is non-informative here. Sparsity = fraction of zero entries in the count matrix.
3. Spatial maps
Visium spots overlaid on the H&E thumbnail. Change the color channel to audit different covariates.
How to read this panel
Each marker is one Visium spot. Spatial structure in UMI / gene detection often reflects tissue cellularity and technical capture efficiency. If inter-model cosine similarity is spatially structured, the two FMs disagree more in some microenvironments (e.g., stroma vs epithelium) than others.
Coloring by k-means clusters tests whether morphology partitions form contiguous tissue domains — a weak but useful sanity check that embeddings encode histology rather than pure batch noise.
Coloring by expression PC1 vs morphology PC1 is a first visual check for shared spatial programs.
4. Embedding geometry
PCA spectra, UMAP manifolds, and pairwise distance structure for each foundation model. Drag the 3D views below to rotate manifolds.
Geometric reading
A steep PCA spectrum (large PC1 fraction) implies a dominant global mode — often stain intensity, tissue vs background residual, or bulk cellularity. Flatter spectra suggest more multi-factor morphological diversity.
UMAP is for visualization only; cluster labels and Mantel tests are computed in the original 1536-d (or PCA) spaces. Pairwise distance histograms characterize manifold concentration: tighter distances can indicate representation collapse or highly homogeneous tissue.
L2-norm differences across models are expected — each model uses different pretraining and projection heads. Compare structure (CKA, Mantel, NN) rather than raw norms.
4b. Interactive 3D manifolds
Rotate / zoom with the mouse. Color encodes biology or model agreement — switch channels live.
Reading 3D embeddings
- UMAP 3D — local neighborhood layout in three axes; good for spotting discrete domains and bridges.
- PCA 3D — linear variance-maximizing view; axes are orthogonal and variance-ordered (PC1 ≥ PC2 ≥ PC3).
- Expression PC1–3 — transcriptomic state space; color by morphology cluster to test shared structure.
- Spatial 3D — tissue x/y with UMI (or selected metric) as height — a “relief map” of capture efficiency / cellularity.
- Joint model space — H-Optimus-0 PC1 × UNI2-h PC1 × cosine similarity: spots where models agree sit high on Z.
Drag to rotate · scroll to zoom · double-click to reset. Plotly WebGL when available.
5. Cross-model alignment
Do H-Optimus-0 and UNI2-h organize the same spots similarly?
Interpretation guide
- Linear CKA ≈ 1: representations related by linear transform; ≈ 0: unrelated.
- Mean cosine on paired spots can be low even when geometry is similar after rotation — prefer CKA / Mantel for geometry.
- NN agreement: local retrieval consistency (important for spot-neighborhood methods).
- Mantel ρ (models): global ranking of pairwise dissimilarities agrees.
- ARI / NMI: discrete cluster partitions agree beyond chance.
Low paired cosine with moderate CKA is common when models live in different cones of Rd but share relative structure.
6. Morphology ↔ expression coupling
Link tile embeddings to transcriptomic state via expression PCs and Mantel tests.
Expression features: log1p(counts-per-10k) on the 2,000 highest-variance genes, then PCA. Heatmaps show Pearson r between morphology PC1–5 and expression PC1–5. Strong off-diagonal or PC1–PC1 coupling would indicate that the dominant morphological axis tracks the dominant transcriptional program (e.g., proliferation, immune infiltrate, stroma).
7. Unsupervised morphological domains
k-means (k=4) in each embedding space; compare composition by library size and inter-model agreement.
8. Discussion, limitations & next experiments
Findings framing
With n=138 spots on a single sample, this is a methodological quickstart, not a population-scale benchmark. The value is in the analysis recipe: dual-FM embedding, multi-metric alignment, and explicit morphology–expression tests that can scale to full HEST-1k cohorts.
Limitations
- Single sample (NCBI10); no cross-slide batch effects or tissue-type stratification.
- H-Optimus-1 was not used (gated access pending); H-Optimus-0 is the deployed Bioptimus model.
- Tile MPP is approximate from Visium spot diameter; not a full multi-resolution pyramid embedding.
- No supervised labels (pathologist annotations, mutations) — clusters are unsupervised only.
- CKA/Mantel assume the chosen kernels/distances; cosine vs Euclidean can shift ranks.
Natural extensions
- Multi-sample HEST cohort; linear probe for tissue type / cancer subtype.
- Gene-set scores (Hallmark, cell-type signatures) as continuous labels for embedding regression.
- Attention/rollout or tile retrieval to explain clusters morphologically.
- Compare against CONCH, Virchow, Prov-GigaPath under the same tile grid.
- Slide-level MIL aggregation for diagnosis / survival tasks.
References (selected)
- HEST-1k — spatial transcriptomics + H&E foundation-model benchmark (Mahmood Lab).
- UNI / UNI2 — Chen et al., nature-scale pathology foundation models (Mahmood Lab).
- H-Optimus — Bioptimus open pathology FMs.
- Kornblith et al., Similarity of Neural Network Representations Revisited (CKA).
- Mantel 1967; spatial statistics in ecology / spatial omics.
Generated by scripts/build_dashboard.py · analytics snapshot also written to
outputs/NCBI10_phd_analytics.json · research / non-clinical use only.