Sgd / Regressor Layer

Stochastic Gradient Descent Regression: Linear model fitted by minimizing a regularized empirical loss with SGD.

Mathematical formulation: $w min \frac{1}{n} i = 1 \sum n L (y_{i}, X_{i}^{T} w) + α R (w)$ where:

L is the loss function
R is the regularization term
α is regularization strength
w are model parameters

Key characteristics:

Online learning capability
Memory efficiency
Various loss functions
Multiple regularization options
Adaptive learning rates

Advantages:

Handles large datasets
Out-of-core learning
Fast convergence options
Flexible optimization
Resource efficient

Common applications:

Large-scale learning
Streaming data
Online prediction
Real-time updates
Resource-constrained environments

Outputs:

Predicted Table: Results with predictions
Validation Results: Cross-validation metrics
Test Metric: Hold-out performance
Feature Importances: Model coefficients

Table

Predicted Table

Validation Results

Test Metric

Feature Importances

SelectFeatures

[column, ...]

Feature column selection for SGD Regression:

Data requirements:

Preprocessing needs:
- Standardization crucial
- Scale to [0,1] or [-1,1]
- Handle missing values
- Remove outliers
Feature properties:
- Numeric values only
- Dense preferred
- Sparse supported
- Finite numbers
Quality considerations:
- Feature correlation
- Signal strength
- Noise levels
- Data distribution
Online learning:
- Feature stability
- Streaming compatibility
- Consistent scaling
- Real-time features
Performance impact:
- Memory footprint
- Update speed
- Convergence rate
- Training stability

Note: If empty, uses all numeric columns except target

SelectTarget

column

Target column specification for SGD Regression:

Requirements:

Data characteristics:
- Numeric continuous
- Finite values
- No missing data
- Stationary preferred
Preprocessing:
- Standardization recommended
- Outlier treatment
- Distribution check
- Scale considerations
Online aspects:
- Target stability
- Value range
- Update frequency
- Drift monitoring
Quality checks:
- Value distribution
- Noise assessment
- Temporal patterns
- Error structure
Performance considerations:
- Update efficiency
- Memory usage
- Scaling impact
- Convergence behavior

Note: Must be a single numeric column

Params

oneof

DefaultParams

Default configuration for SGD Regression:

Core settings:

Optimization:
- Squared error loss
- L2 regularization (α=0.0001)
- Inverse scaling learning rate
Training control:
- 1000 max iterations
- 0.001 tolerance
- Shuffled samples
- No early stopping
Learning rate:
- Initial rate (η₀) = 0.01
- Power_t = 0.25
- Invscaling schedule

Best suited for:

Initial modeling
Standard regression
Online learning
Large datasets

CustomParams

Customizable parameters for SGD Regression:

Parameter categories:

Model architecture:
- Loss function
- Regularization
- Feature scaling
Optimization:
- Learning rates
- Iterations
- Convergence criteria
Training control:
- Early stopping
- Validation
- Warm start

Trade-offs:

Speed vs accuracy
Memory vs performance
Stability vs adaptivity

Loss

enum

SquaredError

Loss functions for SGD optimization:

Selection criteria:

Error distribution
Outlier sensitivity
Prediction requirements
Robustness needs

Impact:

Convergence behavior
Solution stability
Prediction characteristics
Error handling

SquaredError ~

Standard squared error loss:

Formula: $L (y, \overset{y}{^}) = \frac{1}{2} (y - \overset{y}{^})^{2}$

Properties:

Ordinary least squares
Sensitive to outliers
Faster convergence
Differentiable

Best for:

Gaussian noise
Clean data
Standard regression
Initial modeling

Huber ~

Huber robust loss function:

Formula: $L (y, \overset{y}{^}) = {\frac{1}{2} (y - \overset{y}{^})^{2} ϵ ∣ y - \overset{y}{^} ∣ - \frac{1}{2} ϵ^{2} ∣ y - \overset{y}{^} ∣ \leq ϵ otherwise$

Properties:

Combines squared and absolute loss
Robust to outliers
Smooth transition
Adaptive behavior

Best for:

Noisy data
Outlier presence
Robust modeling
General purpose

EpsilonInsensitive ~

Epsilon-insensitive loss:

Formula: $L (y, \overset{y}{^}) = max (0, ∣ y - \overset{y}{^} ∣ - ϵ)$

Properties:

Ignores small errors
SVR-like behavior
Linear penalization
Zone of insensitivity

Best for:

Support vector regression
Tolerance zones
Sparse solutions
Specific error bounds

SquaredEpsilonInsensitive ~

Squared epsilon-insensitive loss:

Formula: $L (y, \overset{y}{^}) = max (0, ∣ y - \overset{y}{^} ∣ - ϵ)^{2}$

Properties:

Quadratic beyond epsilon
Smooth transitions
Combined characteristics
Flexible penalization

Best for:

Smooth solutions
Controlled tolerance
Mixed behavior needs
Advanced regression

Penalty

enum

Regularization terms for model complexity control:

Purpose:

Prevent overfitting
Control complexity
Feature selection
Improve generalization

Selection impact:

Solution sparsity
Model complexity
Feature weights
Training stability

L2 ~

L2 (Ridge) regularization:

Formula: $R (w) = \frac{1}{2} ∣∣ w ∣ ∣_{2}^{2}$

Properties:

Squared magnitude penalty
Smooth solutions
All features contribute
Stable training

Best for:

Correlated features
Dense solutions
General purpose
Default choice

L1 ~

L1 (Lasso) regularization:

Formula: $R (w) = ∣∣ w ∣ ∣_{1}$

Properties:

Absolute magnitude penalty
Sparse solutions
Feature selection
Variable elimination

Best for:

Feature selection
Sparse models
High dimensionality
Important feature identification

ElasticNet ~

ElasticNet mixed regularization:

Formula: $R (w) = ρ ∣∣ w ∣ ∣_{1} + \frac{1 - ρ}{2} ∣∣ w ∣ ∣_{2}^{2}$

Properties:

Combines L1 and L2
Balanced sparsity
Group selection
Flexible control

Best for:

Correlated features
Group selection
Mixed behavior needs
Advanced modeling

None ~

No regularization:

Formula: $R (w) = 0$

Properties:

No complexity penalty
Maximum flexibility
Uncontrolled weights
Pure optimization

Best for:

Simple problems
Clean data
Feature studies
When overfitting not concern

Alpha

f64

0.0001

Regularization strength parameter:

Mathematical impact: $w min L (w) + α R (w)$

Typical ranges:

Weak: 1e-5 to 1e-4
Standard: 1e-4 to 1e-3
Strong: 1e-3 to 1e-1

Effects:

Controls overfitting
Affects model complexity
Influences learning rate
Impacts convergence

L1Ratio

f64

0.15

ElasticNet mixing parameter:

Formula: $R (w) = ρ ∣∣ w ∣ ∣_{1} + \frac{1 - ρ}{2} ∣∣ w ∣ ∣_{2}^{2}$ where ρ is l1_ratio

Range: [0, 1]

0: Pure L2 (Ridge)
1: Pure L1 (Lasso)
Between: Mixed penalty

Selection guide:

High correlation: Lower values
Feature selection: Higher values
Balanced: Around 0.5

FitIntercept

bool

true

Intercept calculation control:

Model forms: With intercept: $y = Xw + b + ϵ$ Without intercept: $y = Xw + ϵ$

Effects when True:

Fits bias term
Centers predictions
Accounts for mean shift
Better general fit

Effects when False:

Forces origin fitting
No bias term
Assumes centered data
Domain-specific needs

Selection guide:

True: Most cases (default)
False: Zero-intercept theory
False: Pre-centered data
False: Physical constraints

MaxIter

u64

1000

Maximum number of training epochs:

Considerations:

Dataset size:
- Larger data: More iterations
- Smaller data: Fewer iterations
Convergence needs:
- Simple problems: 100-500
- Complex problems: 1000+
- Online learning: Continuous
Resource constraints:
- Time limitations
- Compute availability
- Memory constraints

Tol

f64

0.001

Stopping criterion tolerance:

Usage: Stop when: $∣ L os s_{t} - L os s_{t - 1} ∣ < t o l$

Typical values:

Strict: 1e-4 or smaller
Standard: 1e-3
Relaxed: 1e-2

Impact:

Convergence speed
Solution precision
Training duration
Resource usage

Shuffle

bool

true

Training data shuffling control:

Effects when True:

Prevents cycling patterns
Improves convergence
Reduces bias
Better generalization

Effects when False:

Deterministic order
Reproducible results
Order-sensitive cases
Time series data

Epsilon

f64

0.1

Loss function parameter:

Usage in losses:

Huber:
- Outlier threshold
- Linear vs quadratic
Epsilon-Insensitive:
- Error tolerance zone
- Prediction margin

Selection guide:

Small: Precise fitting
Large: More tolerance
Based on noise level

LearningRate

enum

Invscaling

Learning rate schedules for SGD optimization:

Purpose:

Control step sizes
Ensure convergence
Balance exploration/exploitation
Optimize learning process

Impact:

Convergence speed
Solution stability
Training dynamics
Final performance

Constant ~

Constant learning rate:

Formula: $η_{t} = η_{0}$

Properties:

Fixed step size
Simple implementation
Requires careful tuning
No adaptation

Best for:

Well-conditioned problems
Short training runs
Known optimal rate
Simple datasets

Optimal ~

Optimal learning rate schedule:

Formula: $η_{t} = \frac{1}{α ( t + t _{0} )}$ where t₀ is auto-determined

Properties:

Theoretically motivated
Automatic scaling
Provable convergence
Robust behavior

Best for:

Convex problems
Long training runs
Theoretical guarantees
Standard optimization

Invscaling ~

Inverse scaling learning rate:

Formula: $η_{t} = η_{0} t^{- p o w er_t}$

Properties:

Power-law decay
Controllable decay
Smooth reduction
Flexible schedule

Best for:

General purpose
Default choice
Custom decay needs
Balanced convergence

Adaptive ~

Adaptive learning rate:

Algorithm:

Start with η = η₀
Divide by 5 if no improvement
Check every n_iter_no_change epochs
Stop when tolerance reached

Properties:

Dynamic adaptation
Performance based
Automatic tuning
Early stopping capable

Best for:

Unknown problems
Difficult convergence
Automatic tuning needs
Robust training

Eta0

f64

0.01

Initial learning rate:

Impact:

Controls initial step size
Affects convergence speed
Influences stability

Typical values:

Conservative: 0.001
Standard: 0.01
Aggressive: 0.1

Note: Crucial for constant and invscaling

PowerT

f64

0.25

Inverse scaling exponent:

Formula: $η_{t} = η_{0} / t^{p o w er_t}$

Common values:

0.25: Default choice
0.5: Faster decay
0.1: Slower decay

Effect:

Controls learning rate decay
Affects convergence speed
Balances exploration/exploitation

EarlyStopping

bool

false

Early stopping control:

Functionality:

Monitors validation score
Prevents overfitting
Saves computation
Optimizes training

Requirements:

Validation fraction > 0
Sufficient data
Patience setting
Score monitoring

WarmStart

bool

false

When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.

When True:

Reuses previous solution
Continues training
Incremental learning
Parameter evolution

Best for:

Online learning
Parameter studies
Incremental fitting
Transfer learning

ValidationFraction

f64

0.1

Early stopping validation size:

Range: (0, 1)

Guidelines:

Small data: 0.2-0.3
Medium data: 0.1-0.2
Large data: 0.05-0.1

Trade-offs:

Validation quality
Training data size
Computational cost

NIterNoChange

u32

Early stopping patience:

Function:

Consecutive iterations threshold
No improvement tolerance
Stopping decision
Convergence check

Typical values:

Aggressive: 3-5
Standard: 5-10
Patient: 10-20

RandomState

u32

Random number generator seed:

Controls:

Data shuffling
Initialization
Stochastic processes

Usage:

Fixed: Reproducibility
Different: Robustness checks
None: Random behavior
Documentation important

GridSearch

Hyperparameter optimization for SGD Regression:

Search space organization:

Model configuration:
- Loss functions
- Regularization types
- Learning schedules
Optimization parameters:
- Learning rates
- Convergence controls
- Training limits
Early stopping:
- Validation settings
- Stopping criteria
- Patience values

Computational impact:

Time: O(n_params * n_samples * max_iter)
Memory: O(n_features * n_params)
Storage: O(n_params)

Loss

[enum, ...]

SquaredError

Loss functions for SGD optimization:

Selection criteria:

Error distribution
Outlier sensitivity
Prediction requirements
Robustness needs

Impact:

Convergence behavior
Solution stability
Prediction characteristics
Error handling

SquaredError ~

Standard squared error loss:

Formula: $L (y, \overset{y}{^}) = \frac{1}{2} (y - \overset{y}{^})^{2}$

Properties:

Ordinary least squares
Sensitive to outliers
Faster convergence
Differentiable

Best for:

Gaussian noise
Clean data
Standard regression
Initial modeling

Huber ~

Huber robust loss function:

Formula: $L (y, \overset{y}{^}) = {\frac{1}{2} (y - \overset{y}{^})^{2} ϵ ∣ y - \overset{y}{^} ∣ - \frac{1}{2} ϵ^{2} ∣ y - \overset{y}{^} ∣ \leq ϵ otherwise$

Properties:

Combines squared and absolute loss
Robust to outliers
Smooth transition
Adaptive behavior

Best for:

Noisy data
Outlier presence
Robust modeling
General purpose

EpsilonInsensitive ~

Epsilon-insensitive loss:

Formula: $L (y, \overset{y}{^}) = max (0, ∣ y - \overset{y}{^} ∣ - ϵ)$

Properties:

Ignores small errors
SVR-like behavior
Linear penalization
Zone of insensitivity

Best for:

Support vector regression
Tolerance zones
Sparse solutions
Specific error bounds

SquaredEpsilonInsensitive ~

Squared epsilon-insensitive loss:

Formula: $L (y, \overset{y}{^}) = max (0, ∣ y - \overset{y}{^} ∣ - ϵ)^{2}$

Properties:

Quadratic beyond epsilon
Smooth transitions
Combined characteristics
Flexible penalization

Best for:

Smooth solutions
Controlled tolerance
Mixed behavior needs
Advanced regression

Penalty

[enum, ...]

Regularization terms for model complexity control:

Purpose:

Prevent overfitting
Control complexity
Feature selection
Improve generalization

Selection impact:

Solution sparsity
Model complexity
Feature weights
Training stability

L2 ~

L2 (Ridge) regularization:

Formula: $R (w) = \frac{1}{2} ∣∣ w ∣ ∣_{2}^{2}$

Properties:

Squared magnitude penalty
Smooth solutions
All features contribute
Stable training

Best for:

Correlated features
Dense solutions
General purpose
Default choice

L1 ~

L1 (Lasso) regularization:

Formula: $R (w) = ∣∣ w ∣ ∣_{1}$

Properties:

Absolute magnitude penalty
Sparse solutions
Feature selection
Variable elimination

Best for:

Feature selection
Sparse models
High dimensionality
Important feature identification

ElasticNet ~

ElasticNet mixed regularization:

Formula: $R (w) = ρ ∣∣ w ∣ ∣_{1} + \frac{1 - ρ}{2} ∣∣ w ∣ ∣_{2}^{2}$

Properties:

Combines L1 and L2
Balanced sparsity
Group selection
Flexible control

Best for:

Correlated features
Group selection
Mixed behavior needs
Advanced modeling

None ~

No regularization:

Formula: $R (w) = 0$

Properties:

No complexity penalty
Maximum flexibility
Uncontrolled weights
Pure optimization

Best for:

Simple problems
Clean data
Feature studies
When overfitting not concern

Alpha

[f64, ...]

0.0001

Regularization strength search space:

Search ranges:

Log scale (recommended):
- [1e-5, 1e-4, 1e-3, 1e-2, 1e-1]
- Wide coverage
Fine-tuning:
- [0.0001, 0.0003, 0.001, 0.003]
- Narrow, focused
Problem-specific:
- Strong: [0.1, 0.3, 1.0]
- Weak: [1e-5, 3e-5, 1e-4]

L1Ratio

[f64, ...]

0.15

ElasticNet mixing parameter search:

Search spaces:

Coarse grid:
- [0.1, 0.5, 0.9]
- Basic coverage
Fine grid:
- [0.1, 0.3, 0.5, 0.7, 0.9]
- Detailed search
Extreme values:
- [0.01, 0.1, 0.9, 0.99]
- Near-pure penalties

FitIntercept

[bool, ...]

true

Intercept inclusion search:

Options:

Single option:
- [true]: Standard modeling
Complete search:
- [true, false]: Compare both

Selection impact:

Model flexibility
Prediction bias
Theory alignment

MaxIter

[u64, ...]

1000

Maximum iterations search space:

Search patterns:

Basic range:
- [100, 500, 1000]
- Standard problems
Extended search:
- [500, 1000, 2000, 5000]
- Complex problems
Convergence study:
- [100, 300, 1000, 3000]
- Log-scale spacing

Tol

[f64, ...]

0.001

Convergence tolerance search:

Search ranges:

Standard scale:
- [1e-4, 1e-3, 1e-2]
- Common values
High precision:
- [1e-5, 1e-4, 1e-3]
- Strict convergence
Quick convergence:
- [1e-3, 1e-2, 1e-1]
- Faster training

Shuffle

[bool, ...]

true

Data shuffling evaluation:

Options:

Single mode:
- [true]: Standard training
Comparison:
- [true, false]: Full evaluation

Impact study:

Convergence behavior
Training stability
Order sensitivity

Epsilon

[f64, ...]

0.1

Loss parameter search space:

Search ranges:

Standard:
- [0.05, 0.1, 0.2]
- Common values
Fine-grained:
- [0.01, 0.05, 0.1, 0.15]
- Detailed search
Problem-specific:
- Based on error scale
- Data characteristics

LearningRate

[enum, ...]

Invscaling

Learning rate schedules for SGD optimization:

Purpose:

Control step sizes
Ensure convergence
Balance exploration/exploitation
Optimize learning process

Impact:

Convergence speed
Solution stability
Training dynamics
Final performance

Constant ~

Constant learning rate:

Formula: $η_{t} = η_{0}$

Properties:

Fixed step size
Simple implementation
Requires careful tuning
No adaptation

Best for:

Well-conditioned problems
Short training runs
Known optimal rate
Simple datasets

Optimal ~

Optimal learning rate schedule:

Formula: $η_{t} = \frac{1}{α ( t + t _{0} )}$ where t₀ is auto-determined

Properties:

Theoretically motivated
Automatic scaling
Provable convergence
Robust behavior

Best for:

Convex problems
Long training runs
Theoretical guarantees
Standard optimization

Invscaling ~

Inverse scaling learning rate:

Formula: $η_{t} = η_{0} t^{- p o w er_t}$

Properties:

Power-law decay
Controllable decay
Smooth reduction
Flexible schedule

Best for:

General purpose
Default choice
Custom decay needs
Balanced convergence

Adaptive ~

Adaptive learning rate:

Algorithm:

Start with η = η₀
Divide by 5 if no improvement
Check every n_iter_no_change epochs
Stop when tolerance reached

Properties:

Dynamic adaptation
Performance based
Automatic tuning
Early stopping capable

Best for:

Unknown problems
Difficult convergence
Automatic tuning needs
Robust training

Eta0

[f64, ...]

0.01

Initial learning rate search:

Search spaces:

Log scale:
- [0.001, 0.01, 0.1]
- Wide range
Fine-tuning:
- [0.005, 0.01, 0.02]
- Narrow range
Schedule-specific:
- Constant: Smaller range
- Adaptive: Larger range

PowerT

[f64, ...]

0.25

Learning rate decay search:

Search ranges:

Standard:
- [0.1, 0.25, 0.5]
- Common values
Extended:
- [0.1, 0.25, 0.5, 0.75]
- Wider range

Note: Only for invscaling

EarlyStopping

[bool, ...]

false

Early stopping evaluation:

Options:

Single mode:
- [false]: Full training
- [true]: With stopping
Comparison:
- [true, false]: Compare both

Impact analysis:

Training duration
Model performance
Resource usage

WarmStart

[bool, ...]

false

When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.

Options:

Single mode:
- [false]: Fresh starts
- [true]: Solution reuse
Comparison:
- [true, false]: Compare both

Study focus:

Convergence speed
Solution quality
Training efficiency

ValidationFraction

f64

0.1

Validation set size control:

Typical values:

Small data: 0.2
Medium data: 0.1
Large data: 0.05

Considerations:

Dataset size
Validation stability
Training needs

NIterNoChange

u32

Early stopping patience control:

Typical ranges:

Aggressive: 3-5
Standard: 5-10
Patient: 10-20

Impact:

Training duration
Convergence quality
Resource usage

RandomState

u32

Random seed configuration:

Usage:

Fixed: Reproducibility
Varied: Robustness checks
None: Random behavior

Best practices:

Document seed values
Test multiple seeds
Ensure replicability

RefitScore

enum

R2score

Regression model evaluation metrics:

Purpose:

Model performance evaluation
Error measurement
Quality assessment
Model comparison

Selection criteria:

Error distribution
Scale sensitivity
Domain requirements
Business objectives

Default ~

Model's native scoring method:

Typically R² score
Model-specific implementation
Standard evaluation
Quick assessment

R2score ~

Coefficient of determination (R²):

Formula: $R^{2} = 1 - \frac{\sum ( y _{i} - y ^ _{i} ) ^{2}}{\sum ( y _{i} - y ˉ ) ^{2}}$

Properties:

Range: (-∞, 1]
1: Perfect prediction
0: Constant model
Negative: Worse than mean

Best for:

General performance
Variance explanation
Model comparison
Standard reporting

ExplainedVariance ~

Explained variance score:

Formula: $1 - \frac{Va r ( y - y ^ )}{Va r ( y )}$

Properties:

Range: (-∞, 1]
Accounts for bias
Variance focus
Similar to R²

Best for:

Variance analysis
Bias assessment
Model stability

MaxError ~

Maximum absolute error:

Formula: $max (∣ y_{i} - \overset{y}{^}_{i} ∣)$

Properties:

Worst case error
Original scale
Sensitive to outliers
Upper error bound

Best for:

Critical applications
Error bounds
Safety margins
Risk assessment

NegMeanAbsoluteError ~

Negative mean absolute error:

Formula: $- \frac{1}{n} \sum ∣ y_{i} - \overset{y}{^}_{i} ∣$

Properties:

Linear error scale
Robust to outliers
Original units
Negated for optimization

Best for:

Robust evaluation
Interpretable errors
Outlier presence

NegMeanSquaredError ~

Negative mean squared error:

Formula: $- \frac{1}{n} \sum (y_{i} - \overset{y}{^}_{i})^{2}$

Properties:

Squared error scale
Outlier sensitive
Squared units
Negated for optimization

Best for:

Standard optimization
Large error penalty
Statistical analysis

NegRootMeanSquaredError ~

Negative root mean squared error:

Formula: $- \frac{1}{n} \sum (y_{i} - \overset{y}{^}_{i})^{2}$

Properties:

Original scale
Outlier sensitive
Interpretable units
Negated for optimization

Best for:

Standard reporting
Interpretable errors
Model comparison

NegMeanSquaredLogError ~

Negative mean squared logarithmic error:

Formula: $- \frac{1}{n} \sum (lo g (1 + y_{i}) - lo g (1 + \overset{y}{^}_{i}))^{2}$

Properties:

Relative error scale
For positive values
Sensitive to ratios
Negated for optimization

Best for:

Exponential growth
Relative differences
Positive predictions

NegMedianAbsoluteError ~

Negative median absolute error:

Formula: $- m e d ian (∣ y_{i} - \overset{y}{^}_{i} ∣)$

Properties:

Highly robust
Original scale
Outlier resistant
Negated for optimization

Best for:

Robust evaluation
Heavy-tailed errors
Outlier presence

NegMeanPoissonDeviance ~

Negative Poisson deviance:

Formula: $- 2 \sum (y_{i} lo g (\frac{y _{i}}{y ^ _{i}}) - (y_{i} - \overset{y}{^}_{i}))$

Properties:

For count data
Non-negative values
Poisson assumption
Negated for optimization

Best for:

Count prediction
Event frequency
Rate modeling

NegMeanGammaDeviance ~

Negative Gamma deviance:

Formula: $- 2 \sum (lo g (\frac{y _{i}}{y ^ _{i}}) + \frac{y _{i} - y ^ _{i}}{y ^ _{i}})$

Properties:

For positive continuous data
Constant CV assumption
Relative errors
Negated for optimization

Best for:

Positive continuous data
Multiplicative errors
Financial modeling

NegMeanAbsolutePercentageError ~

Negative mean absolute percentage error:

Formula: $- \frac{100}{n} \sum ∣ \frac{y _{i} - y ^ _{i}}{y _{i}} ∣$

Properties:

Percentage scale
Scale independent
For non-zero targets
Negated for optimization

Best for:

Relative performance
Scale-free comparison
Business metrics

D2AbsoluteErrorScore ~

D² score with absolute error:

Formula: $1 - \frac{\sum ∣ y _{i} - y ^ _{i} ∣}{\sum ∣ y _{i} - y ˉ ∣}$

Properties:

Range: (-∞, 1]
Robust version of R²
Linear error scale
Outlier resistant

Best for:

Robust evaluation
Non-normal errors
Alternative to R²

D2PinballScore ~

D² score with pinball loss:

Properties:

Quantile focus
Asymmetric errors
Risk assessment
Distribution modeling

Best for:

Quantile regression
Risk analysis
Asymmetric costs
Distribution tails

D2TweedieScore ~

D² score with Tweedie deviance:

Properties:

Compound Poisson-Gamma
Flexible dispersion
Mixed distributions
Insurance modeling

Best for:

Insurance claims
Mixed continuous-discrete
Compound distributions
Specialized modeling

Split

oneof

DefaultSplit

Standard train-test split configuration optimized for general classification tasks.

Configuration:

Test size: 20% (0.2)
Random seed: 98
Shuffling: Enabled
Stratification: Based on target distribution

Advantages:

Preserves class distribution
Provides reliable validation
Suitable for most datasets

Best for:

Medium to large datasets
Independent observations
Initial model evaluation

Splitting uses the ShuffleSplit strategy or StratifiedShuffleSplit strategy depending on the field stratified. Note: If shuffle is false then stratified must be false.

CustomSplit

Configurable train-test split parameters for specialized requirements. Allows fine-tuning of data division strategy for specific use cases or constraints.

Use cases:

Time series data
Grouped observations
Specific train/test ratios
Custom validation schemes

RandomState

u64

Random seed for reproducible splits. Ensures:

Consistent train/test sets
Reproducible experiments
Comparable model evaluations

Same seed guarantees identical splits across runs.

Shuffle

bool

true

Data shuffling before splitting. Effects:

true: Randomizes order, better for i.i.d. data
false: Maintains order, important for time series

When to disable:

Time dependent data
Sequential patterns
Grouped observations

TrainSize

f64

0.8

Proportion of data for training. Considerations:

Larger (e.g., 0.8-0.9): Better model learning
Smaller (e.g., 0.5-0.7): Better validation

Common splits:

0.8: Standard (80/20 split)
0.7: More validation emphasis
0.9: More training emphasis

Stratified

bool

false

Maintain class distribution in splits. Important when:

Classes are imbalanced
Small classes present
Representative splits needed

Requirements:

Classification tasks only
Cannot use with shuffle=false
Sufficient samples per class

Cv

oneof

DefaultCv

Standard cross-validation configuration using stratified 3-fold splitting.

Configuration:

Folds: 3
Method: StratifiedKFold
Stratification: Preserves class proportions

Advantages:

Balanced evaluation
Reasonable computation time
Good for medium-sized datasets

Limitations:

May be insufficient for small datasets
Higher variance than larger fold counts
May miss some data patterns

CustomCv

Configurable stratified k-fold cross-validation for specific validation requirements.

Features:

Adjustable fold count with NFolds determining the number of splits.
Stratified sampling
Preserved class distributions

Use cases:

Small datasets (more folds)
Large datasets (fewer folds)
Detailed model evaluation
Robust performance estimation

NFolds

u32

Number of cross-validation folds. Guidelines:

3-5: Large datasets, faster training
5-10: Standard choice, good balance
10+: Small datasets, thorough evaluation

Trade-offs:

More folds: Better evaluation, slower training
Fewer folds: Faster training, higher variance

Must be at least 2.

KfoldCv

K-fold cross-validation without stratification. Divides data into k consecutive folds for iterative validation.

Process:

Splits data into k equal parts
Each fold serves as validation once
Remaining k-1 folds form training set

Use cases:

Regression problems
Large, balanced datasets
When stratification unnecessary
Continuous target variables

Limitations:

May not preserve class distributions
Less suitable for imbalanced data
Can create biased splits with ordered data

NSplits

u32

Number of folds for cross-validation. Selection guide: Recommended values:

5: Standard choice (default)
3: Large datasets/quick evaluation
10: Thorough evaluation/smaller datasets

Trade-offs:

Higher values: More thorough, computationally expensive
Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

RandomState

u64

Random seed for fold generation when shuffling. Important for:

Reproducible results
Consistent fold assignments
Benchmark comparisons
Debugging and validation

Set specific value for reproducibility across runs.

Shuffle

bool

true

Whether to shuffle data before splitting into folds. Effects:

true: Randomized fold composition (recommended)
false: Sequential splitting

Enable when:

Data may have ordering
Better fold independence needed

Disable for:

Time series data
Ordered observations

StratifiedKfoldCv

Stratified K-fold cross-validation maintaining class proportions across folds.

Key features:

Preserves class distribution in each fold
Handles imbalanced datasets
Ensures representative splits

Best for:

Classification problems
Imbalanced class distributions
When class proportions matter

Requirements:

Classification tasks only
Sufficient samples per class
Categorical target variable

NSplits

u32

Number of stratified folds. Guidelines: Typical values:

5: Standard for most cases
3: Quick evaluation/large datasets
10: Detailed evaluation/smaller datasets

Considerations:

Must allow sufficient samples per class per fold
Balance between stability and computation time
Consider smallest class size when choosing

RandomState

u64

Seed for reproducible stratified splits. Ensures:

Consistent fold assignments
Reproducible results
Comparable experiments
Systematic validation

Fixed seed guarantees identical stratified splits.

Shuffle

bool

false

Data shuffling before stratified splitting. Impact:

true: Randomizes while maintaining stratification
false: Maintains data order within strata

Use cases:

true: Independent observations
false: Grouped or sequential data

Class proportions maintained regardless of setting.

ShuffleSplitCv

Random permutation cross-validator with independent sampling.

Characteristics:

Random sampling for each split
Independent train/test sets
More flexible than K-fold
Can have overlapping test sets

Advantages:

Control over test size
Fresh splits each iteration
Good for large datasets

Limitations:

Some samples might never be tested
Others might be tested multiple times
No guarantee of complete coverage

NSplits

u32

Number of random splits to perform. Consider: Common values:

5: Standard evaluation
10: More thorough assessment
3: Quick estimates

Trade-offs:

More splits: Better estimation, longer runtime
Fewer splits: Faster, less stable estimates

Balance between computation and stability.

RandomState

u64

Random seed for reproducible shuffling. Controls:

Split randomization
Sample selection
Result reproducibility

Important for:

Debugging
Comparative studies
Result verification

TestSize

f64

0.2

Proportion of samples for test set. Guidelines: Common ratios:

0.2: Standard (80/20 split)
0.25: More validation emphasis
0.1: More training data

Considerations:

Dataset size
Model complexity
Validation requirements

It must be between 0.0 and 1.0.

StratifiedShuffleSplitCv

Stratified random permutation cross-validator combining shuffle-split with stratification.

Features:

Maintains class proportions
Random sampling within strata
Independent splits
Flexible test size

Ideal for:

Imbalanced datasets
Large-scale problems
When class distributions matter
Flexible validation schemes

NSplits

u32

Number of stratified random splits. Guidelines: Recommended values:

5: Standard evaluation
10: Detailed analysis
3: Quick assessment

Consider:

Sample size per class
Computational resources
Stability requirements

RandomState

u64

Seed for reproducible stratified sampling. Ensures:

Consistent class proportions
Reproducible splits
Comparable experiments

Critical for:

Benchmarking
Research studies
Quality assurance

TestSize

f64

0.2

Fraction of samples for stratified test set. Best practices: Common splits:

0.2: Balanced evaluation
0.3: More thorough testing
0.15: Preserve training size

Consider:

Minority class size
Overall dataset size
Validation objectives

It must be between 0.0 and 1.0.

Time Series cross-validator. Provides train/test indices to split time series data samples that are observed at fixed time intervals, in train/test sets. It is a variation of k-fold which returns first k folds as train set and the k + 1th fold as test set. Note that unlike standard cross-validation methods, successive training sets are supersets of those that come before them. Also, it adds all surplus data to the first training partition, which is always used to train the model. Key features:

Maintains temporal dependence
Expanding window approach
Forward-chaining splits
No future data leakage

Use cases:

Sequential data
Financial forecasting
Temporal predictions
Time-dependent patterns

Note: Training sets are supersets of previous iterations.

NSplits

u32

Number of temporal splits. Considerations: Typical values:

5: Standard forward chaining
3: Limited historical data
10: Long time series

Impact:

Affects training window growth
Determines validation points
Influences computational load

MaxTrainSize

u64

Maximum size of training set. Should be strictly less than the number of samples. Applications:

0: Use all available past data
>0: Rolling window of fixed size

Use cases:

Limit historical relevance
Control computational cost
Handle concept drift
Memory constraints

TestSize

u64

Number of samples in each test set. When 0:

Auto-calculated as n_samples/(n_splits+1)
Ensures equal-sized test sets

Considerations:

Forecast horizon
Validation requirements
Available future data

Gap

u64

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

Avoid data leakage
Model forecast lag
Buffer periods

Common scenarios:

0: Continuous prediction
>0: Forward gap for realistic evaluation
Match business forecasting needs