**Paper:** TabSTAR: A Foundation Tabular Model With Semantically Target-Aware Representations > While deep learning has achieved remarkable success across many domains, it > has historically underperformed on tabular learning tasks, which remain > dominated by gradient boosting decision trees (GBDTs). However, recent > advancements are paving the way for Tabular Foundation Models, which can > leverage real-world knowledge and generalize across diverse datasets, > particularly when the data contains free-text. Although incorporating language > model capabilities into tabular tasks has been explored, most existing methods > utilize static, target-agnostic textual representations, limiting their > effectiveness. We introduce TabSTAR: a Foundation Tabular Model with > Semantically Target-Aware Representations. TabSTAR is designed to enable > transfer learning on tabular data with textual features, with an architecture > free of dataset-specific parameters. It unfreezes a pretrained text encoder and > takes as input target tokens, which provide the model with the context needed > to learn task-specific embeddings. TabSTAR achieves state-of-the-art > performance for both medium- and large-sized datasets across known benchmarks > of classification tasks with text features, and its pretraining phase exhibits > scaling laws in the number of datasets, offering a pathway for further > performance improvements.
翻訳待ち・原文を表示
入力 未確認→ 出力 未確認
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
分類準備中
入力 未確認→ 出力 未確認
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
分類準備中
入力 未確認→ 出力 未確認
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
分類準備中
入力 未確認→ 出力 未確認
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
分類準備中
入力 未確認→ 出力 未確認
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
分類準備中
入力 未確認→ 出力 未確認
TabSTAR-v2 classification checkpoint, epoch 17 of `cls50k-active-261004_233909`. Pretrained checkpoint; this file records how it was trained and how it differs from the original TabSTAR (v1). - **Task type**: classification only (separate model per task type) - **Pool**: decontaminated T4 pool `t4_tabstarv1.1_runs_decontamination`, 200,142 classification tables; 50,000 sampled per epoch (`--num_datasets 50000`, `--pool_mode active`) - **Learning rate**: 1e-4, **weight decay**: 1e-3, linear warmup over the first 10% of steps - **Epochs**: 25 planned, early-stopping patience 5 - **Batch size**: 512 global (micro-batch 32, no gradient accumulation) - **Text encoder**: `intfloat/e5-small-v2`, top 6 layers unfrozen, text batch size 1024 - **Finetunability check**: every epoch, 30 minute timeout per check; epoch 17 is among the top finetunability Elo epochs Downstream finetuning (TabArena-Lite) uses the repo defaults, none tuned per checkpoint:
翻訳待ち・原文を表示
分類準備中
入力 未確認→ 出力 未確認
TabSTAR-v2 regression checkpoint, epoch 16 of `reg50k-active-261004_233909`. Pretrained checkpoint; this file records how it was trained and how it differs from the original TabSTAR (v1). - **Task type**: regression only (separate model per task type) - **Pool**: decontaminated T4 pool `t4_tabstarv1.1_runs_decontamination`, 200,013 regression tables; 50,000 sampled per epoch (`--num_datasets 50000`, `--pool_mode active`) - **Learning rate**: 1e-4, **weight decay**: 1e-3, linear warmup over the first 10% of steps - **Epochs**: 25 planned, early-stopping patience 5 - **Batch size**: 512 global (micro-batch 32, no gradient accumulation) - **Text encoder**: `intfloat/e5-small-v2`, top 6 layers unfrozen, text batch size 1024 - **Regression head**: dual head, a scalar sub-head plus a distributional sub-head over 200 bins - **Finetunability check**: every epoch, 30 minute timeout per check; epoch 16 had the top finetunability Elo Downstream finetuning (TabArena-Lite) uses the repo defaults, none tuned per checkpoint:
翻訳待ち・原文を表示
分類準備中
入力 未確認→ 出力 未確認
