Rubric validation
How robust SchemaScope's scores are: how much they move when the weights change, how well our own quality checks agree with the OBO Dashboard, and how well scores agree with expert ratings.
Rubric v4.1 · 2,688 resources · computed 27 Oct 2026 from the evaluation of 27 Oct 2026. Reproduce with npm run validate:rubric.
Sensitivity to the weights
Each criterion's weight drawn uniformly from 0.7–1.3 (1000 draws, fixed seed); scores recomputed from the stored points. Penalties and the inconsistency cap are unchanged.
One criterion at a time
| Criterion | Grade changes, weight −30% | +30% | Criterion dropped | Rank correlation if dropped |
|---|---|---|---|---|
| Findable | 42 | 41 | 430 | 0.982 |
| Accessible | 101 | 47 | 350 | 0.962 |
| Interoperable | 69 | 24 | 195 | 0.984 |
| Reusable | 58 | 20 | 149 | 0.971 |
| Ontology consistency & quality | 22 | 44 | 87 | 0.998 |
| Maintenance | 48 | 74 | 507 | 0.964 |
| Community adoption | 79 | 110 | 583 | 0.992 |
Most borderline resources
- Hypertension Ontology40 (D), grade changes in 50% of draws
- Space Life Sciences Ontology70 (B), grade changes in 50% of draws
- European Science Vocabulary39 (E), grade changes in 49% of draws
- Cardiovascular Disease Ontology55 (C), grade changes in 48% of draws
- Teleost taxonomy ontology40 (D), grade changes in 48% of draws
- CIDOC Conceptual Reference Model55 (C), grade changes in 47% of draws
- Discourse Elements Ontology55 (C), grade changes in 47% of draws
- Document Components Ontology55 (C), grade changes in 47% of draws
- Data Use Ontology55 (C), grade changes in 47% of draws
- Environmental conditions, treatments and exposures ontology85 (A), grade changes in 47% of draws
- PharmGKB Gene39 (E), grade changes in 47% of draws
- Pathogen Host Interaction Phenotype Ontology85 (A), grade changes in 47% of draws
- Publishing Workflow Ontology55 (C), grade changes in 47% of draws
- Scholarly Contributions and Roles Ontology55 (C), grade changes in 47% of draws
- Ascomycete phenotype ontology70 (B), grade changes in 46% of draws
Our quality checks vs the OBO Dashboard
Ontologies with both a SchemaScope ROBOT run and an OBO Dashboard result. The two may check different files (the Dashboard usually checks the -base release).
No ontology has both results yet. This fills in as our quality checks run (npm run quality:all) and the catalog is re-evaluated.
Agreement with expert ratings
No expert ratings yet. Ratings (1–5 per resource) go in data/validation/expert-ratings.csv; the comparison appears here once at least five resources are rated. Would you like to rate resources in your field? Get in touch.