AR6

Workshop / Syntype

Syntype

An open benchmark of tabular models on ecological data. Version 0.1 was corrected in public the day it came out.

State
On 16 Oct 2026 the repository holds three commits, all from 30 September: version 0.1 and its public correction, which marks the forest results invalid and the trait results preliminary; the fixed v0.1.1 announced that day has no commit yet. m
commits per week, 12 weeks · 0 in the last 7 days · as of 14 Oct
Runs on
Apple Silicon (MPS), CUDA or CPU
Built with
Python 3.11+, uv, CatBoost and scikit-learn, PyTorch and NVIDIA’s structured-data-models
Data
Eleven open datasets (four trait databases, seven forest inventories), each downloaded under its own licence; nothing redistributed
Code
Public, Apache-2.0: github.com/syntype/syntype

Known limits

  • Version 0.1 is not usable as it stands: forest results invalid, trait results preliminary. c
  • Its sixteen tasks are eight targets from four databases, one of them counted twice, and the tests were too optimistic. c
  • Only two foundation models were evaluated, Kumo Tabular and TabICLv2; TabPFN was not. m
  • The protocol favours in-context models: a cap of 8,000 rows, an untuned CatBoost, point metrics only. m

Sources and questions

Register · 5 entries

  1. pause

    Version 0.1.1 was announced on 30 September to fix the four blocking issues; sixteen days later the repository has no new commit. m

  2. milestone

    A research model reviews the field for Syntype: missing species traits are a real need that phylogenetic imputation already serves; the open question is transfer to poorly documented species. i

  3. correction

    A second model reviewed Syntype v0.1 and found four blocking issues. Forest results declared invalid, trait results preliminary, the same evening. m

    1. det. 30 Sept · correctedA second model’s review finds four blocking issues: forest results invalid, trait results preliminary.
    2. det. 30 Sept · measuredv0.1 published: the foundation model wins on species traits, not on forest inventories.
  4. measure

    Light fine-tuning of Kumo Tabular (small) on three trait databases, tested on the fourth, brought no measurable gain: better than the untuned model on 2 of 8 traits (preliminary). m

  5. decision

    Values that the source databases imputed themselves are removed from the targets, along with features derived from the target; the audit found three such leaks before any result was published. m