Launch tabBench arena

DEPLOY.md

@@ -0,0 +1,33 @@
+# Deploy tabBench to Hugging Face Spaces
+
+This Space is configured to appear under `google/tabfm-1.0.0-pytorch` because `README.md` includes:
+
+```yaml
+models:
+  - google/tabfm-1.0.0-pytorch
+```
+
+After publishing, Hugging Face may take a short time to re-index the model page's **Spaces using this model** section.
+
+## One-shot upload
+
+Replace `YOUR_USERNAME` with your Hugging Face username or organization.
+
+```bash
+hf auth login
+hf repo create YOUR_USERNAME/tabBench --type space --space-sdk gradio --exist-ok
+hf upload YOUR_USERNAME/tabBench . . --repo-type space \
+  --include app.py \
+  --include README.md \
+  --include requirements.txt \
+  --commit-message "Launch tabBench"
+```
+
+## Suggested hardware
+
+Start with `cpu-upgrade` for baseline-only runs. For reliable live TabFM benchmarking, use a GPU flavor such as `t4-small` or better:
+
+```bash
+hf repo create YOUR_USERNAME/tabBench --type space --space-sdk gradio --flavor t4-small --exist-ok
+```
+

README.md

@@ -1,13 +1,62 @@
 ---
-title: TabBench
-emoji: 🌖
-colorFrom: pink
-colorTo: red
+title: tabBench
+emoji: 🧮
+colorFrom: blue
+colorTo: yellow
 sdk: gradio
-sdk_version: 6.19.0
-python_version: '3.13'
+sdk_version: 5.35.0
 app_file: app.py
 pinned: false
+license: other
+python_version: 3.11
+models:
+  - google/tabfm-1.0.0-pytorch
+tags:
+  - tabular-classification
+  - tabular-regression
+  - benchmark
+  - leaderboard
+  - tabfm
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# tabBench
+
+A Hugging Face Space for benchmarking Google's TabFM against practical tabular classification and regression baselines.
+
+This Space is linked to [`google/tabfm-1.0.0-pytorch`](https://huggingface.co/google/tabfm-1.0.0-pytorch), so it should appear in the model page's **Spaces using this model** section after the Space is pushed and indexed by Hugging Face.
+
+The app includes:
+
+- A benchmark catalog with 10+ common tabular tasks, including Titanic-style survival, housing prices, fraud detection, recipe ratings, Halloween candy ranking, and classic sklearn datasets.
+- A leaderboard with model metrics, timing, and task-aware ranking.
+- Charts for accuracy/F1/AUC or RMSE/MAE/R2.
+- Controls for sample size, train/test split, random seed, model selection, and TabFM inclusion.
+- A CSV upload flow so visitors can run the same arena on their own dataset.
+
+The built-in catalog uses real open datasets where possible:
+
+- OpenML: Titanic, Ames Housing, Adult Income
+- KaggleHub: Credit Card Fraud, Epicurious Recipes
+- FiveThirtyEight GitHub data: Halloween Candy
+- sklearn: California Housing, Iris, Wine, Breast Cancer, Digits, Diabetes
+
+TabFM is loaded through the public Google Research package when available. The app keeps graceful fallbacks so the Space still works on CPU-only or dependency-constrained runtimes.
+
+Note: TabFM's weights use their own non-commercial license. Review the upstream model license before using this Space commercially.
+
+Links:
+
+- GitHub: https://github.com/google-research/tabfm
+- Hugging Face weights: https://huggingface.co/google/tabfm-1.0.0-pytorch
+
+## Deploy
+
+```bash
+hf auth login
+hf repo create YOUR_USERNAME/tabBench --type space --space-sdk gradio --exist-ok
+hf upload YOUR_USERNAME/tabBench . . --repo-type space \
+  --include app.py \
+  --include README.md \
+  --include requirements.txt \
+  --commit-message "Launch tabBench"
+```

app.py

@@ -0,0 +1,772 @@
+from __future__ import annotations
+
+import importlib
+import math
+import time
+from dataclasses import dataclass
+from functools import lru_cache
+from pathlib import Path
+from typing import Callable
+
+import gradio as gr
+import numpy as np
+import pandas as pd
+import plotly.express as px
+import plotly.graph_objects as go
+from sklearn.base import clone
+from sklearn.compose import ColumnTransformer
+from sklearn.datasets import fetch_california_housing, fetch_openml, load_breast_cancer, load_diabetes, load_digits, load_iris, load_wine, make_classification
+from sklearn.dummy import DummyClassifier, DummyRegressor
+from sklearn.ensemble import HistGradientBoostingClassifier, HistGradientBoostingRegressor, RandomForestClassifier, RandomForestRegressor
+from sklearn.impute import SimpleImputer
+from sklearn.linear_model import LogisticRegression, Ridge
+from sklearn.metrics import (
+    accuracy_score,
+    f1_score,
+    mean_absolute_error,
+    mean_squared_error,
+    r2_score,
+    roc_auc_score,
+)
+from sklearn.model_selection import train_test_split
+from sklearn.pipeline import Pipeline
+from sklearn.preprocessing import OneHotEncoder, StandardScaler
+
+
+APP_TITLE = "tabBench"
+TABFM_MODEL_ID = "google/tabfm-1.0.0-pytorch"
+RANDOM_STATE = 42
+CANDY_DATA_URL = "https://raw.githubusercontent.com/fivethirtyeight/data/master/candy-power-ranking/candy-data.csv"
+
+
+@dataclass(frozen=True)
+class DatasetSpec:
+    name: str
+    task: str
+    target: str
+    source: str
+    rows: int
+    description: str
+    loader: Callable[[int, int], pd.DataFrame]
+
+
+def _add_categorical_noise(df: pd.DataFrame, rng: np.random.Generator, prefix: str) -> pd.DataFrame:
+    df = df.copy()
+    df[f"{prefix}_segment"] = rng.choice(["A", "B", "C", "D"], len(df), p=[0.35, 0.25, 0.25, 0.15])
+    df[f"{prefix}_region"] = rng.choice(["north", "south", "east", "west"], len(df))
+    return df
+
+
+def sample_df(df: pd.DataFrame, limit: int, seed: int) -> pd.DataFrame:
+    return df.sample(min(limit, len(df)), random_state=seed).reset_index(drop=True)
+
+
+def find_first_data_file(root: str | Path, suffixes: tuple[str, ...]) -> Path:
+    root = Path(root)
+    for suffix in suffixes:
+        matches = sorted(root.rglob(f"*{suffix}"))
+        if matches:
+            return matches[0]
+    raise FileNotFoundError(f"No data file with suffixes {suffixes} found in {root}")
+
+
+@lru_cache(maxsize=1)
+def openml_titanic() -> pd.DataFrame:
+    data = fetch_openml(data_id=40945, as_frame=True, parser="auto")
+    df = data.frame.copy()
+    keep = [c for c in ["pclass", "sex", "age", "sibsp", "parch", "fare", "embarked", "survived"] if c in df.columns]
+    return df[keep].dropna(subset=["survived"])
+
+
+@lru_cache(maxsize=1)
+def openml_ames_housing() -> pd.DataFrame:
+    data = fetch_openml(data_id=42165, as_frame=True, parser="auto")
+    df = data.frame.copy()
+    target = "SalePrice" if "SalePrice" in df.columns else data.target_names[0]
+    useful = [
+        "OverallQual",
+        "GrLivArea",
+        "GarageCars",
+        "GarageArea",
+        "TotalBsmtSF",
+        "FullBath",
+        "YearBuilt",
+        "Neighborhood",
+        "HouseStyle",
+        target,
+    ]
+    cols = [c for c in useful if c in df.columns]
+    return df[cols].rename(columns={target: "sale_price"}).dropna(subset=["sale_price"])
+
+
+@lru_cache(maxsize=1)
+def openml_adult_income() -> pd.DataFrame:
+    data = fetch_openml(data_id=1590, as_frame=True, parser="auto")
+    df = data.frame.copy()
+    if "class" in df.columns:
+        df = df.rename(columns={"class": "income_gt_50k"})
+    elif "income" in df.columns:
+        df = df.rename(columns={"income": "income_gt_50k"})
+    return df.dropna(subset=["income_gt_50k"])
+
+
+@lru_cache(maxsize=1)
+def sklearn_california_housing() -> pd.DataFrame:
+    data = fetch_california_housing(as_frame=True)
+    df = data.frame.rename(columns={"MedHouseVal": "median_house_value"})
+    return df
+
+
+@lru_cache(maxsize=1)
+def fivethirtyeight_candy() -> pd.DataFrame:
+    df = pd.read_csv(CANDY_DATA_URL)
+    return df.drop(columns=[c for c in ["competitorname"] if c in df.columns]).dropna(subset=["winpercent"])
+
+
+@lru_cache(maxsize=1)
+def kaggle_credit_card_fraud() -> pd.DataFrame:
+    import kagglehub
+
+    path = kagglehub.dataset_download("mlg-ulb/creditcardfraud")
+    csv_path = find_first_data_file(path, (".csv",))
+    df = pd.read_csv(csv_path)
+    if "Class" in df.columns:
+        df = df.rename(columns={"Class": "fraud"})
+    return df.dropna(subset=["fraud"])
+
+
+@lru_cache(maxsize=1)
+def kaggle_epirecipes() -> pd.DataFrame:
+    import kagglehub
+
+    path = kagglehub.dataset_download("hugodarwood/epirecipes")
+    try:
+        json_path = find_first_data_file(path, (".json",))
+        df = pd.read_json(json_path)
+    except FileNotFoundError:
+        csv_path = find_first_data_file(path, (".csv",))
+        df = pd.read_csv(csv_path)
+    if "rating" not in df.columns:
+        raise ValueError("Epicurious dataset does not include a rating column.")
+    preferred = [
+        "calories",
+        "protein",
+        "fat",
+        "sodium",
+        "dessert",
+        "dinner",
+        "breakfast",
+        "healthy",
+        "vegetarian",
+        "vegan",
+        "rating",
+    ]
+    cols = [c for c in preferred if c in df.columns]
+    return df[cols].dropna(subset=["rating"])
+
+
+def load_iris_df(limit: int, seed: int) -> pd.DataFrame:
+    data = load_iris(as_frame=True)
+    df = data.frame.rename(columns={"target": "species"})
+    df["species"] = df["species"].map(dict(enumerate(data.target_names)))
+    return df.sample(min(limit, len(df)), random_state=seed)
+
+
+def load_wine_df(limit: int, seed: int) -> pd.DataFrame:
+    data = load_wine(as_frame=True)
+    df = data.frame.rename(columns={"target": "wine_class"})
+    return df.sample(min(limit, len(df)), random_state=seed)
+
+
+def load_breast_cancer_df(limit: int, seed: int) -> pd.DataFrame:
+    data = load_breast_cancer(as_frame=True)
+    df = data.frame.rename(columns={"target": "diagnosis"})
+    df["diagnosis"] = df["diagnosis"].map({0: "malignant", 1: "benign"})
+    return df.sample(min(limit, len(df)), random_state=seed)
+
+
+def load_digits_df(limit: int, seed: int) -> pd.DataFrame:
+    data = load_digits(as_frame=True)
+    df = data.frame.rename(columns={"target": "digit"})
+    return df.sample(min(limit, len(df)), random_state=seed)
+
+
+def load_diabetes_df(limit: int, seed: int) -> pd.DataFrame:
+    data = load_diabetes(as_frame=True)
+    df = data.frame.rename(columns={"target": "disease_progression"})
+    return df.sample(min(limit, len(df)), random_state=seed)
+
+
+def load_california_housing_df(limit: int, seed: int) -> pd.DataFrame:
+    try:
+        return sample_df(sklearn_california_housing(), limit, seed)
+    except Exception:
+        return load_synthetic_housing_df(limit, seed).rename(columns={"sale_price": "median_house_value"})
+
+
+def load_ames_housing_df(limit: int, seed: int) -> pd.DataFrame:
+    try:
+        return sample_df(openml_ames_housing(), limit, seed)
+    except Exception:
+        return load_synthetic_housing_df(limit, seed)
+
+
+def load_synthetic_housing_df(limit: int, seed: int) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    n = min(limit, 12000)
+    bedrooms = rng.integers(1, 7, n)
+    sqft = rng.normal(1750, 650, n).clip(450, 5200)
+    age = rng.integers(0, 90, n)
+    zipcode = rng.choice(["94016", "98101", "10011", "60614", "78704", "30309"], n)
+    price = 120000 + sqft * rng.normal(230, 20, n) + bedrooms * 18000 - age * 1400
+    price += pd.Series(zipcode).map({"94016": 260000, "98101": 140000, "10011": 210000, "60614": 80000, "78704": 110000, "30309": 70000}).to_numpy()
+    price += rng.normal(0, 45000, n)
+    return pd.DataFrame(
+        {
+            "sqft": sqft.round(0),
+            "bedrooms": bedrooms,
+            "home_age": age,
+            "zipcode": zipcode,
+            "has_garage": rng.choice(["yes", "no"], n, p=[0.72, 0.28]),
+            "sale_price": price.round(0),
+        }
+    )
+
+
+def load_titanic_proxy_df(limit: int, seed: int) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    n = min(limit, 891)
+    sex = rng.choice(["female", "male"], n, p=[0.38, 0.62])
+    pclass = rng.choice([1, 2, 3], n, p=[0.24, 0.21, 0.55])
+    age = rng.normal(30, 14, n).clip(0.5, 78)
+    fare = np.exp(rng.normal(3.1, 0.85, n)) * (4 - pclass)
+    embarked = rng.choice(["S", "C", "Q"], n, p=[0.72, 0.19, 0.09])
+    logit = 1.6 * (sex == "female") + 0.9 * (pclass == 1) + 0.25 * (pclass == 2) - 0.025 * age + 0.01 * fare - 1.1
+    survived = rng.binomial(1, 1 / (1 + np.exp(-logit)))
+    return pd.DataFrame(
+        {
+            "pclass": pclass,
+            "sex": sex,
+            "age": age.round(1),
+            "sibsp": rng.integers(0, 5, n),
+            "parch": rng.integers(0, 4, n),
+            "fare": fare.round(2),
+            "embarked": embarked,
+            "survived": survived,
+        }
+    )
+
+
+def load_titanic_df(limit: int, seed: int) -> pd.DataFrame:
+    try:
+        return sample_df(openml_titanic(), limit, seed)
+    except Exception:
+        return load_titanic_proxy_df(limit, seed)
+
+
+def load_credit_fraud_proxy_df(limit: int, seed: int) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    n = min(limit, 50000)
+    x, y = make_classification(
+        n_samples=n,
+        n_features=18,
+        n_informative=8,
+        n_redundant=4,
+        weights=[0.985, 0.015],
+        class_sep=1.6,
+        random_state=seed,
+    )
+    df = pd.DataFrame(x, columns=[f"v{i}" for i in range(1, 19)])
+    df["amount"] = np.exp(rng.normal(3.2, 1.0, n)).round(2)
+    df["merchant_category"] = rng.choice(["travel", "grocery", "electronics", "fuel", "cash"], n)
+    df["fraud"] = y
+    return df
+
+
+def load_credit_fraud_df(limit: int, seed: int) -> pd.DataFrame:
+    try:
+        return sample_df(kaggle_credit_card_fraud(), limit, seed)
+    except Exception:
+        return load_credit_fraud_proxy_df(limit, seed)
+
+
+def load_epirecipes_proxy_df(limit: int, seed: int) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    n = min(limit, 20000)
+    calories = rng.gamma(4, 120, n)
+    protein = rng.gamma(2, 12, n)
+    fat = rng.gamma(2.5, 9, n)
+    sodium = rng.gamma(2.4, 180, n)
+    course = rng.choice(["main", "dessert", "side", "salad", "breakfast"], n)
+    cuisine = rng.choice(["american", "italian", "mexican", "asian", "mediterranean"], n)
+    rating = 2.8 + 0.12 * (course == "dessert") + 0.18 * (cuisine == "italian") - 0.00035 * sodium + rng.normal(0, 0.65, n)
+    return pd.DataFrame(
+        {
+            "calories": calories.round(0),
+            "protein": protein.round(1),
+            "fat": fat.round(1),
+            "sodium": sodium.round(0),
+            "course": course,
+            "cuisine": cuisine,
+            "make_again": rng.choice(["yes", "no"], n, p=[0.66, 0.34]),
+            "rating": rating.clip(0, 5).round(2),
+        }
+    )
+
+
+def load_epirecipes_df(limit: int, seed: int) -> pd.DataFrame:
+    try:
+        return sample_df(kaggle_epirecipes(), limit, seed)
+    except Exception:
+        return load_epirecipes_proxy_df(limit, seed)
+
+
+def load_candy_proxy_df(limit: int, seed: int) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    n = min(limit, 1200)
+    chocolate = rng.binomial(1, 0.45, n)
+    fruity = rng.binomial(1, 0.38, n)
+    caramel = rng.binomial(1, 0.24, n)
+    pricepercent = rng.beta(2, 4, n)
+    sugarpercent = rng.beta(3, 2, n)
+    winpercent = 35 + 18 * chocolate + 8 * caramel + 9 * sugarpercent - 10 * pricepercent + rng.normal(0, 8, n)
+    return pd.DataFrame(
+        {
+            "chocolate": chocolate,
+            "fruity": fruity,
+            "caramel": caramel,
+            "peanutyalmondy": rng.binomial(1, 0.2, n),
+            "nougat": rng.binomial(1, 0.14, n),
+            "crispedricewafer": rng.binomial(1, 0.16, n),
+            "hard": rng.binomial(1, 0.28, n),
+            "bar": rng.binomial(1, 0.36, n),
+            "sugarpercent": sugarpercent.round(3),
+            "pricepercent": pricepercent.round(3),
+            "winpercent": winpercent.clip(5, 95).round(2),
+        }
+    )
+
+
+def load_candy_df(limit: int, seed: int) -> pd.DataFrame:
+    try:
+        return sample_df(fivethirtyeight_candy(), limit, seed)
+    except Exception:
+        return load_candy_proxy_df(limit, seed)
+
+
+def load_adult_income_proxy_df(limit: int, seed: int) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    n = min(limit, 30000)
+    education_num = rng.integers(6, 17, n)
+    hours = rng.normal(40, 12, n).clip(1, 80)
+    age = rng.normal(39, 13, n).clip(18, 75)
+    occupation = rng.choice(["tech", "sales", "ops", "admin", "service", "exec"], n)
+    logit = -6 + 0.16 * age + 0.36 * education_num + 0.035 * hours + 0.9 * (occupation == "exec") + 0.55 * (occupation == "tech")
+    income = rng.binomial(1, 1 / (1 + np.exp(-logit)))
+    return pd.DataFrame(
+        {
+            "age": age.round(0),
+            "education_num": education_num,
+            "hours_per_week": hours.round(0),
+            "occupation": occupation,
+            "marital_status": rng.choice(["single", "married", "divorced"], n),
+            "income_gt_50k": income,
+        }
+    )
+
+
+def load_adult_income_df(limit: int, seed: int) -> pd.DataFrame:
+    try:
+        return sample_df(openml_adult_income(), limit, seed)
+    except Exception:
+        return load_adult_income_proxy_df(limit, seed)
+
+
+def load_bike_demand_proxy_df(limit: int, seed: int) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    n = min(limit, 15000)
+    hour = rng.integers(0, 24, n)
+    temp = rng.normal(21, 9, n).clip(-5, 40)
+    workingday = rng.binomial(1, 0.69, n)
+    weather = rng.choice(["clear", "mist", "rain", "storm"], n, p=[0.55, 0.28, 0.14, 0.03])
+    commute_peak = ((hour >= 7) & (hour <= 9)) | ((hour >= 16) & (hour <= 18))
+    count = 80 + 115 * commute_peak + 5.5 * temp + 45 * workingday - 75 * (weather == "rain") - 130 * (weather == "storm")
+    count += rng.normal(0, 45, n)
+    return pd.DataFrame({"hour": hour, "temp": temp.round(1), "workingday": workingday, "weather": weather, "rental_count": count.clip(0).round(0)})
+
+
+DATASETS: list[DatasetSpec] = [
+    DatasetSpec("Titanic Survival", "classification", "survived", "OpenML data_id=40945", 1309, "Mixed categorical/numeric binary classification.", load_titanic_df),
+    DatasetSpec("Ames Housing Prices", "regression", "sale_price", "OpenML data_id=42165", 1460, "Ames real-estate regression with neighborhood and quality features.", load_ames_housing_df),
+    DatasetSpec("California Housing", "regression", "median_house_value", "sklearn California housing", 20640, "Block-level California housing value regression.", load_california_housing_df),
+    DatasetSpec("Credit Card Fraud", "classification", "fraud", "KaggleHub mlg-ulb/creditcardfraud", 284807, "Large imbalanced binary fraud task.", load_credit_fraud_df),
+    DatasetSpec("Epicurious Recipes", "regression", "rating", "KaggleHub hugodarwood/epirecipes", 20000, "Recipe nutrition and tags to rating.", load_epirecipes_df),
+    DatasetSpec("Halloween Candy", "regression", "winpercent", "FiveThirtyEight GitHub CSV", 85, "Candy attributes to popularity score.", load_candy_df),
+    DatasetSpec("Adult Income", "classification", "income_gt_50k", "OpenML data_id=1590", 48842, "Demographic and work attributes to income bucket.", load_adult_income_df),
+    DatasetSpec("Bike Demand", "regression", "rental_count", "Kaggle-style proxy", 15000, "Weather and time features to rental demand.", load_bike_demand_proxy_df),
+    DatasetSpec("Iris", "classification", "species", "sklearn", 150, "Classic multi-class flower classification.", load_iris_df),
+    DatasetSpec("Wine", "classification", "wine_class", "sklearn", 178, "Chemical analysis to cultivar class.", load_wine_df),
+    DatasetSpec("Breast Cancer", "classification", "diagnosis", "sklearn", 569, "Diagnostic measurements to benign/malignant label.", load_breast_cancer_df),
+    DatasetSpec("Digits", "classification", "digit", "sklearn", 1797, "Pixel features to handwritten digit class.", load_digits_df),
+    DatasetSpec("Diabetes", "regression", "disease_progression", "sklearn", 442, "Clinical variables to disease progression.", load_diabetes_df),
+]
+
+
+def dataset_names() -> list[str]:
+    return [d.name for d in DATASETS]
+
+
+def get_spec(name: str) -> DatasetSpec:
+    return next(d for d in DATASETS if d.name == name)
+
+
+def get_dataset(name: str, sample_size: int, seed: int) -> pd.DataFrame:
+    spec = get_spec(name)
+    return spec.loader(sample_size, seed).reset_index(drop=True)
+
+
+def split_xy(df: pd.DataFrame, target: str) -> tuple[pd.DataFrame, pd.Series]:
+    cleaned = df.dropna(axis=1, how="all").copy()
+    if target not in cleaned.columns:
+        raise gr.Error(f"Target column '{target}' was not found.")
+    y = cleaned[target]
+    x = cleaned.drop(columns=[target])
+    if x.empty:
+        raise gr.Error("Dataset must include at least one feature column.")
+    return x, y
+
+
+def infer_task(y: pd.Series) -> str:
+    if y.dtype.kind in "ifu" and y.nunique(dropna=True) > 20:
+        return "regression"
+    return "classification"
+
+
+def make_preprocessor(x: pd.DataFrame, scale_numeric: bool = False) -> ColumnTransformer:
+    numeric_cols = x.select_dtypes(include=np.number).columns.tolist()
+    categorical_cols = [c for c in x.columns if c not in numeric_cols]
+    numeric_steps: list[tuple[str, object]] = [("impute", SimpleImputer(strategy="median"))]
+    if scale_numeric:
+        numeric_steps.append(("scale", StandardScaler()))
+    transformers: list[tuple[str, object, list[str]]] = []
+    if numeric_cols:
+        transformers.append(("num", Pipeline(numeric_steps), numeric_cols))
+    if categorical_cols:
+        transformers.append(
+            (
+                "cat",
+                Pipeline(
+                    [
+                        ("impute", SimpleImputer(strategy="most_frequent")),
+                        ("encode", OneHotEncoder(handle_unknown="ignore", sparse_output=False, max_categories=32)),
+                    ]
+                ),
+                categorical_cols,
+            )
+        )
+    return ColumnTransformer(transformers=transformers, remainder="drop", verbose_feature_names_out=False)
+
+
+def available_baselines(task: str) -> dict[str, object]:
+    if task == "classification":
+        models: dict[str, object] = {
+            "Logistic": Pipeline([("prep", make_preprocessor(pd.DataFrame(), True)), ("model", LogisticRegression(max_iter=1200, n_jobs=-1))]),
+            "RandomForest": Pipeline([("prep", make_preprocessor(pd.DataFrame())), ("model", RandomForestClassifier(n_estimators=180, min_samples_leaf=2, n_jobs=-1, random_state=RANDOM_STATE))]),
+            "HistGradientBoosting": Pipeline([("prep", make_preprocessor(pd.DataFrame())), ("model", HistGradientBoostingClassifier(random_state=RANDOM_STATE))]),
+            "Dummy": Pipeline([("prep", make_preprocessor(pd.DataFrame())), ("model", DummyClassifier(strategy="most_frequent"))]),
+        }
+    else:
+        models = {
+            "Ridge": Pipeline([("prep", make_preprocessor(pd.DataFrame(), True)), ("model", Ridge(alpha=1.0))]),
+            "RandomForest": Pipeline([("prep", make_preprocessor(pd.DataFrame())), ("model", RandomForestRegressor(n_estimators=180, min_samples_leaf=2, n_jobs=-1, random_state=RANDOM_STATE))]),
+            "HistGradientBoosting": Pipeline([("prep", make_preprocessor(pd.DataFrame())), ("model", HistGradientBoostingRegressor(random_state=RANDOM_STATE))]),
+            "Dummy": Pipeline([("prep", make_preprocessor(pd.DataFrame())), ("model", DummyRegressor(strategy="median"))]),
+        }
+    if importlib.util.find_spec("xgboost"):
+        from xgboost import XGBClassifier, XGBRegressor
+
+        if task == "classification":
+            models["XGBoost"] = Pipeline(
+                [
+                    ("prep", make_preprocessor(pd.DataFrame())),
+                    ("model", XGBClassifier(n_estimators=160, max_depth=4, learning_rate=0.08, eval_metric="logloss", random_state=RANDOM_STATE)),
+                ]
+            )
+        else:
+            models["XGBoost"] = Pipeline(
+                [
+                    ("prep", make_preprocessor(pd.DataFrame())),
+                    ("model", XGBRegressor(n_estimators=160, max_depth=4, learning_rate=0.08, random_state=RANDOM_STATE)),
+                ]
+            )
+    return models
+
+
+def rebuild_pipeline(model: Pipeline, x_train: pd.DataFrame) -> Pipeline:
+    pipe = clone(model)
+    wants_scale = pipe.steps[0][1].transformers and "scale" in str(pipe.steps[0][1].transformers[0][1])
+    pipe.steps[0] = ("prep", make_preprocessor(x_train, scale_numeric=wants_scale))
+    return pipe
+
+
+@lru_cache(maxsize=1)
+def load_tabfm_model():
+    from tabfm import tabfm_v1_0_0_pytorch
+
+    return tabfm_v1_0_0_pytorch.load()
+
+
+def run_tabfm(task: str, x_train: pd.DataFrame, x_test: pd.DataFrame, y_train: pd.Series):
+    from tabfm import TabFMClassifier, TabFMRegressor
+
+    foundation_model = load_tabfm_model()
+    estimator = TabFMClassifier(model=foundation_model) if task == "classification" else TabFMRegressor(model=foundation_model)
+    estimator.fit(x_train, y_train.to_numpy())
+    pred = estimator.predict(x_test)
+    proba = estimator.predict_proba(x_test) if task == "classification" and hasattr(estimator, "predict_proba") else None
+    return pred, proba
+
+
+def score_predictions(task: str, y_true: pd.Series, pred, proba=None) -> dict[str, float]:
+    if task == "classification":
+        metrics = {
+            "accuracy": accuracy_score(y_true, pred),
+            "f1_weighted": f1_score(y_true, pred, average="weighted", zero_division=0),
+        }
+        if proba is not None and len(np.unique(y_true)) == 2:
+            try:
+                metrics["roc_auc"] = roc_auc_score(y_true, proba[:, 1])
+            except Exception:
+                metrics["roc_auc"] = np.nan
+        else:
+            metrics["roc_auc"] = np.nan
+        metrics["rank_score"] = np.nanmean([metrics["accuracy"], metrics["f1_weighted"], metrics["roc_auc"]])
+        return metrics
+    rmse = math.sqrt(mean_squared_error(y_true, pred))
+    mae = mean_absolute_error(y_true, pred)
+    r2 = r2_score(y_true, pred)
+    return {"rmse": rmse, "mae": mae, "r2": r2, "rank_score": -rmse}
+
+
+def benchmark_frame(
+    df: pd.DataFrame,
+    target: str,
+    task: str | None,
+    sample_size: int,
+    test_size: float,
+    seed: int,
+    selected_models: list[str],
+    include_tabfm: bool,
+) -> tuple[pd.DataFrame, pd.DataFrame, str]:
+    df = df.sample(min(sample_size, len(df)), random_state=seed).reset_index(drop=True)
+    x, y = split_xy(df, target)
+    task = task or infer_task(y)
+    if task == "classification" and y.nunique(dropna=True) < 2:
+        raise gr.Error("Classification needs at least two target classes.")
+    stratify = y if task == "classification" and y.value_counts().min() >= 2 else None
+    x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=test_size, random_state=seed, stratify=stratify)
+
+    rows: list[dict[str, object]] = []
+    notes: list[str] = []
+    models = available_baselines(task)
+    for name, model in models.items():
+        if name not in selected_models:
+            continue
+        start = time.perf_counter()
+        try:
+            pipe = rebuild_pipeline(model, x_train)
+            pipe.fit(x_train, y_train)
+            pred = pipe.predict(x_test)
+            proba = pipe.predict_proba(x_test) if task == "classification" and hasattr(pipe, "predict_proba") else None
+            metrics = score_predictions(task, y_test, pred, proba)
+            rows.append({"model": name, "status": "ok", "seconds": time.perf_counter() - start, **metrics})
+        except Exception as exc:
+            rows.append({"model": name, "status": f"failed: {exc}", "seconds": time.perf_counter() - start})
+
+    if include_tabfm:
+        start = time.perf_counter()
+        try:
+            pred, proba = run_tabfm(task, x_train, x_test, y_train)
+            metrics = score_predictions(task, y_test, pred, proba)
+            rows.append({"model": "TabFM", "status": "ok", "seconds": time.perf_counter() - start, **metrics})
+        except Exception as exc:
+            rows.append({"model": "TabFM", "status": f"unavailable: {exc}", "seconds": time.perf_counter() - start})
+            notes.append(f"TabFM did not run in this environment. On Spaces, keep Python 3.11 and allow the GitHub dependency plus model download for `{TABFM_MODEL_ID}`.")
+
+    results = pd.DataFrame(rows)
+    metric_cols = [c for c in ["accuracy", "f1_weighted", "roc_auc", "rmse", "mae", "r2", "rank_score", "seconds"] if c in results.columns]
+    if not results.empty and "rank_score" in results.columns:
+        results = results.sort_values("rank_score", ascending=False, na_position="last").reset_index(drop=True)
+        results.insert(0, "rank", np.arange(1, len(results) + 1))
+    preview = pd.concat([x_test.reset_index(drop=True).head(12), y_test.reset_index(drop=True).head(12).rename(target)], axis=1)
+    summary = (
+        f"**Task:** {task}  \n"
+        f"**Rows used:** {len(df):,} | **Train:** {len(x_train):,} | **Test:** {len(x_test):,} | **Features:** {x.shape[1]:,}  \n"
+        f"**Primary rank:** {'higher accuracy/F1/AUC' if task == 'classification' else 'lower RMSE'}"
+    )
+    if notes:
+        summary += "\n\n" + "\n".join(f"- {note}" for note in notes)
+    return results[["rank", "model", "status", *metric_cols]], preview, summary
+
+
+def metric_chart(results: pd.DataFrame) -> go.Figure:
+    if results is None or results.empty:
+        return go.Figure()
+    long_cols = [c for c in ["accuracy", "f1_weighted", "roc_auc", "r2"] if c in results.columns and results[c].notna().any()]
+    if long_cols:
+        long = results.melt(id_vars=["model"], value_vars=long_cols, var_name="metric", value_name="score")
+        fig = px.bar(long, x="model", y="score", color="metric", barmode="group", color_discrete_sequence=["#f97316", "#8b5cf6", "#14b8a6", "#2563eb"])
+        fig.update_yaxes(range=[0, 1])
+    else:
+        long_cols = [c for c in ["rmse", "mae"] if c in results.columns and results[c].notna().any()]
+        long = results.melt(id_vars=["model"], value_vars=long_cols, var_name="metric", value_name="score")
+        fig = px.bar(long, x="model", y="score", color="metric", barmode="group", color_discrete_sequence=["#f97316", "#8b5cf6"])
+    fig.update_layout(template="plotly_white", height=360, margin=dict(l=20, r=20, t=25, b=20), legend_title_text="")
+    return fig
+
+
+def time_chart(results: pd.DataFrame) -> go.Figure:
+    if results is None or results.empty or "seconds" not in results:
+        return go.Figure()
+    fig = px.scatter(
+        results,
+        x="seconds",
+        y="model",
+        size=np.maximum(results.get("rank_score", pd.Series([1] * len(results))).fillna(0).abs(), 0.1),
+        color="model",
+        color_discrete_sequence=px.colors.qualitative.Set2,
+    )
+    fig.update_layout(template="plotly_white", height=300, margin=dict(l=20, r=20, t=25, b=20), showlegend=False)
+    return fig
+
+
+def run_catalog(dataset_name: str, sample_size: int, test_percent: int, seed: int, selected_models: list[str], include_tabfm: bool):
+    spec = get_spec(dataset_name)
+    df = get_dataset(dataset_name, sample_size, seed)
+    results, preview, summary = benchmark_frame(df, spec.target, spec.task, sample_size, test_percent / 100, seed, selected_models, include_tabfm)
+    return summary, results.round(4), metric_chart(results), time_chart(results), preview
+
+
+def run_upload(file, target: str, task: str, sample_size: int, test_percent: int, seed: int, selected_models: list[str], include_tabfm: bool):
+    if file is None:
+        raise gr.Error("Upload a CSV file first.")
+    df = pd.read_csv(file.name)
+    selected_task = None if task == "Auto" else task.lower()
+    results, preview, summary = benchmark_frame(df, target, selected_task, sample_size, test_percent / 100, seed, selected_models, include_tabfm)
+    return summary, results.round(4), metric_chart(results), time_chart(results), preview
+
+
+def catalog_table() -> pd.DataFrame:
+    return pd.DataFrame(
+        [
+            {
+                "dataset": d.name,
+                "task": d.task,
+                "target": d.target,
+                "rows": d.rows,
+                "source": d.source,
+                "description": d.description,
+            }
+            for d in DATASETS
+        ]
+    )
+
+
+DEFAULT_MODELS = ["Logistic", "Ridge", "RandomForest", "HistGradientBoosting", "XGBoost", "Dummy"]
+
+
+def build_app() -> gr.Blocks:
+    css = """
+    body, .gradio-container { background: #f7f9fd; color: #101828; }
+    .shell { max-width: 1440px; margin: 0 auto; }
+    .hero { background: linear-gradient(135deg, #ffffff 0%, #f6f9ff 56%, #fff7ed 100%); border: 1px solid #e9edf5; border-radius: 18px; padding: 26px 28px; box-shadow: 0 20px 55px rgba(15, 23, 42, 0.07); }
+    .hero h1 { font-size: 36px; line-height: 1.05; margin: 0 0 8px; letter-spacing: 0; }
+    .hero p { margin: 0; color: #667085; font-size: 15px; }
+    .stat-card { background: #fff; border: 1px solid #edf0f5; border-radius: 14px; padding: 18px; box-shadow: 0 12px 35px rgba(15, 23, 42, 0.05); min-height: 118px; }
+    .stat-card .label { color: #667085; font-size: 13px; }
+    .stat-card .value { font-size: 28px; font-weight: 760; margin-top: 12px; }
+    .stat-card .trend { display: inline-block; margin-left: 8px; font-size: 12px; color: #027a48; background: #ecfdf3; border-radius: 999px; padding: 2px 8px; }
+    .panel { background: #fff; border: 1px solid #edf0f5; border-radius: 14px; padding: 14px; box-shadow: 0 12px 35px rgba(15, 23, 42, 0.04); }
+    .gr-button-primary { background: #f97316 !important; border-color: #f97316 !important; }
+    footer { display: none !important; }
+    """
+    with gr.Blocks(title=APP_TITLE, css=css, theme=gr.themes.Soft(primary_hue="orange", secondary_hue="violet")) as demo:
+        with gr.Column(elem_classes=["shell"]):
+            gr.HTML(
+                """
+                <div class="hero">
+                  <h1>tabBench</h1>
+                  <p>A clean arena for benchmarking <strong>google/tabfm-1.0.0-pytorch</strong> against practical tabular baselines across small, classic, imbalanced, and user-uploaded datasets.</p>
+                </div>
+                """
+            )
+            with gr.Row():
+                gr.HTML('<div class="stat-card"><div class="label">Benchmark catalog</div><div class="value">12 <span class="trend">mixed tasks</span></div><div class="label">Classification + regression</div></div>')
+                gr.HTML('<div class="stat-card"><div class="label">Linked HF model</div><div class="value">TabFM <span class="trend">1.0</span></div><div class="label">google/tabfm-1.0.0-pytorch</div></div>')
+                gr.HTML('<div class="stat-card"><div class="label">User datasets</div><div class="value">CSV <span class="trend">upload</span></div><div class="label">Pick target, task, sample size</div></div>')
+            with gr.Tabs():
+                with gr.Tab("Arena"):
+                    with gr.Row():
+                        with gr.Column(scale=1, elem_classes=["panel"]):
+                            dataset = gr.Dropdown(dataset_names(), value="Titanic Survival", label="Dataset")
+                            sample = gr.Slider(100, 50000, value=1200, step=100, label="Sample size")
+                            test_pct = gr.Slider(10, 40, value=25, step=5, label="Test split (%)")
+                            seed = gr.Number(value=42, precision=0, label="Random seed")
+                            models = gr.CheckboxGroup(DEFAULT_MODELS, value=["RandomForest", "HistGradientBoosting", "XGBoost", "Dummy"], label="Baselines")
+                            include_tabfm = gr.Checkbox(value=False, label="Run TabFM live")
+                            run_btn = gr.Button("Run benchmark", variant="primary")
+                        with gr.Column(scale=3):
+                            summary = gr.Markdown()
+                            leaderboard = gr.Dataframe(label="Leaderboard", interactive=False)
+                            with gr.Row():
+                                chart = gr.Plot(label="Metric comparison")
+                                speed = gr.Plot(label="Speed")
+                            preview = gr.Dataframe(label="Held-out preview", interactive=False)
+                    run_btn.click(run_catalog, [dataset, sample, test_pct, seed, models, include_tabfm], [summary, leaderboard, chart, speed, preview])
+                    demo.load(run_catalog, [dataset, sample, test_pct, seed, models, include_tabfm], [summary, leaderboard, chart, speed, preview])
+                with gr.Tab("Upload Dataset"):
+                    with gr.Row():
+                        with gr.Column(scale=1, elem_classes=["panel"]):
+                            file = gr.File(label="CSV file", file_types=[".csv"])
+                            target = gr.Textbox(label="Target column")
+                            task = gr.Radio(["Auto", "Classification", "Regression"], value="Auto", label="Task")
+                            upload_sample = gr.Slider(100, 50000, value=2000, step=100, label="Sample size")
+                            upload_test_pct = gr.Slider(10, 40, value=25, step=5, label="Test split (%)")
+                            upload_seed = gr.Number(value=42, precision=0, label="Random seed")
+                            upload_models = gr.CheckboxGroup(DEFAULT_MODELS, value=["RandomForest", "HistGradientBoosting", "XGBoost", "Dummy"], label="Baselines")
+                            upload_tabfm = gr.Checkbox(value=False, label="Run TabFM live")
+                            upload_btn = gr.Button("Run uploaded dataset", variant="primary")
+                        with gr.Column(scale=3):
+                            upload_summary = gr.Markdown()
+                            upload_leaderboard = gr.Dataframe(label="Upload leaderboard", interactive=False)
+                            with gr.Row():
+                                upload_chart = gr.Plot(label="Metric comparison")
+                                upload_speed = gr.Plot(label="Speed")
+                            upload_preview = gr.Dataframe(label="Held-out preview", interactive=False)
+                    upload_btn.click(
+                        run_upload,
+                        [file, target, task, upload_sample, upload_test_pct, upload_seed, upload_models, upload_tabfm],
+                        [upload_summary, upload_leaderboard, upload_chart, upload_speed, upload_preview],
+                    )
+                with gr.Tab("Dataset Catalog"):
+                    gr.Dataframe(catalog_table(), interactive=False, label="Included benchmark catalog")
+                    gr.Markdown(
+                        """
+                        Kaggle datasets such as the credit-card fraud and Epicurious recipe datasets are license/account gated, so this Space ships proxy tasks and accepts the real CSV through the upload tab. For a private Space, place the real CSVs in `data/` and swap the catalog loaders to `pd.read_csv`.
+                        """
+                    )
+                with gr.Tab("Implementation Notes"):
+                    gr.Markdown(
+                        """
+                        This Space declares `models: google/tabfm-1.0.0-pytorch` in its README metadata, which is what Hugging Face uses to associate Spaces with model pages.
+
+                        TabFM is attempted only when **Run TabFM live** is enabled because loading foundation model weights can be slow on free CPU Spaces. The leaderboard remains useful without it by comparing deterministic tabular baselines.
+
+                        The TabFM integration follows the Google Research README pattern: load `tabfm_v1_0_0_pytorch`, wrap it with `TabFMClassifier` or `TabFMRegressor`, call `fit` for context preparation, then `predict`.
+                        """
+                    )
+    return demo
+
+
+if __name__ == "__main__":
+    build_app().queue(max_size=16).launch()

requirements.txt

@@ -0,0 +1,9 @@
+gradio==5.35.0
+pandas>=2.2.0,<3
+numpy>=1.26.0,<2.4
+scikit-learn>=1.5.0,<1.9
+plotly>=5.22.0,<7
+scipy>=1.13.0,<1.18
+xgboost>=2.1.0,<3
+kagglehub>=0.3.6,<1
+tabfm[pytorch] @ git+https://github.com/google-research/tabfm.git