LassoLarsIc / Regressor Layer

Model selection criterion:

AIC (Akaike Information Criterion):

• Penalty = 2k
• Efficient prediction
• Less conservative
• Asymptotically unbiased

BIC (Bayesian Information Criterion):

• Penalty = k·log(n)
• Consistent selection
• More conservative
• True model recovery

Selection guide:

• Prediction focus: AIC
• Model selection: BIC
• Small samples: AIC
• Large samples: BIC

Intercept calculation control:

Model forms: With intercept: $$y=X\beta +b+ϵ$$ Without intercept: $$y=X\beta +ϵ$$

Effects when True:

• Centers predictions
• Affects IC calculation
• Improves model fit
• Counts in complexity

Effects when False:

• Origin-constrained
• Simpler model
• Lower complexity
• Theory-driven cases

Maximum number of iterations to perform.

Impacts:

• Path length limit
• Model candidates
• IC computation
• Resource usage

Typical ranges:

• Small problems: 100-300
• Standard: 500
• Complex: 1000+

Note: Each step evaluates IC

Numerical precision parameter:

Usage: $$eps\times machine\_precision$$

Effects:

• Cholesky stability
• Path computation
• IC calculation
• Solution reliability

Typical values:

• Standard: 1.0
• Ill-conditioned: 10-100
• Extreme cases: 100+

Whether to restrict coefficients to be >= 0.

Mathematical impact: $$\underset{\beta \ge 0}{\min }RSS(\beta )+IC\_penalty(k)$$

Applications:

• Signal processing
• Physical quantities
• Concentration studies
• Non-negative data

Effects:

• Constrains path
• Affects IC values
• Modifies selection
• Domain meaning

The estimated noise variance of the data.

Usage:

• 0: Auto-estimated via OLS
• >0: User-specified value

Impact on IC:

• RSS scaling
• Model selection
• Complexity balance
• Criterion accuracy

Best practices:

• Use domain knowledge
• Validate estimates
• Consider uncertainty
• Monitor sensitivity

Model selection criterion:

AIC (Akaike Information Criterion):

• Penalty = 2k
• Efficient prediction
• Less conservative
• Asymptotically unbiased

BIC (Bayesian Information Criterion):

• Penalty = k·log(n)
• Consistent selection
• More conservative
• True model recovery

Selection guide:

• Prediction focus: AIC
• Model selection: BIC
• Small samples: AIC
• Large samples: BIC

Intercept inclusion search:

Options:

1. Standard: [true]

   • Normal modeling
   • Better fit

2. Zero-intercept: [false]

   • Theory-driven
   • Simpler models

3. Compare: [true, false]

   • Impact analysis
   • Complexity study

Maximum iterations search:

Search patterns:

1. Basic range:

   • [100, 300, 500]
   • Standard problems

2. Extended search:

   • [500, 1000, 2000]
   • Complex problems

3. Comprehensive:

   • Based on features
   • Path complexity
   • Resource limits

Numerical precision search:

Search ranges:

1. Standard stability:

   • [1.0, 10.0]
   • Normal cases

2. Ill-conditioned:

   • [10.0, 100.0, 1000.0]
   • Numerical issues

3. Extreme cases:

   • Higher values
   • Stability critical

Positive constraint search:

Options:

1. Unconstrained: [false]

   • Standard modeling
   • Faster computation

2. Constrained: [true]

   • Physical meaning
   • Domain requirements

3. Compare: [true, false]

   • Impact study
   • Selection effects

Noise variance search space:

Search options:

1. Auto-estimate: [0.0]

   • OLS-based
   • Data-driven

2. Fixed values:

   • Domain knowledge
   • Prior estimates

3. Range study:

   • Sensitivity analysis
   • Selection stability

Regression model evaluation metrics:

Purpose:

• Model performance evaluation
• Error measurement
• Quality assessment
• Model comparison

Selection criteria:

• Error distribution
• Scale sensitivity
• Domain requirements
• Business objectives

Random seed for reproducible splits. Ensures:

• Consistent train/test sets
• Reproducible experiments
• Comparable model evaluations

Same seed guarantees identical splits across runs.

Data shuffling before splitting. Effects:

• true: Randomizes order, better for i.i.d. data
• false: Maintains order, important for time series

When to disable:

• Time dependent data
• Sequential patterns
• Grouped observations

Proportion of data for training. Considerations:

• Larger (e.g., 0.8-0.9): Better model learning
• Smaller (e.g., 0.5-0.7): Better validation

Common splits:

• 0.8: Standard (80/20 split)
• 0.7: More validation emphasis
• 0.9: More training emphasis

Maintain class distribution in splits. Important when:

• Classes are imbalanced
• Small classes present
• Representative splits needed

Requirements:

• Classification tasks only
• Cannot use with shuffle=false
• Sufficient samples per class

Number of cross-validation folds. Guidelines:

• 3-5: Large datasets, faster training
• 5-10: Standard choice, good balance
• 10+: Small datasets, thorough evaluation

Trade-offs:

• More folds: Better evaluation, slower training
• Fewer folds: Faster training, higher variance

Must be at least 2.

Number of folds for cross-validation. Selection guide: Recommended values:

• 5: Standard choice (default)
• 3: Large datasets/quick evaluation
• 10: Thorough evaluation/smaller datasets

Trade-offs:

• Higher values: More thorough, computationally expensive
• Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

Random seed for fold generation when shuffling. Important for:

• Reproducible results
• Consistent fold assignments
• Benchmark comparisons
• Debugging and validation

Set specific value for reproducibility across runs.

Whether to shuffle data before splitting into folds. Effects:

• true: Randomized fold composition (recommended)
• false: Sequential splitting

Enable when:

• Data may have ordering
• Better fold independence needed

Disable for:

• Time series data
• Ordered observations

Number of stratified folds. Guidelines: Typical values:

• 5: Standard for most cases
• 3: Quick evaluation/large datasets
• 10: Detailed evaluation/smaller datasets

Considerations:

• Must allow sufficient samples per class per fold
• Balance between stability and computation time
• Consider smallest class size when choosing

Seed for reproducible stratified splits. Ensures:

• Consistent fold assignments
• Reproducible results
• Comparable experiments
• Systematic validation

Fixed seed guarantees identical stratified splits.

Data shuffling before stratified splitting. Impact:

• true: Randomizes while maintaining stratification
• false: Maintains data order within strata

Use cases:

• true: Independent observations
• false: Grouped or sequential data

Class proportions maintained regardless of setting.

Number of random splits to perform. Consider: Common values:

• 5: Standard evaluation
• 10: More thorough assessment
• 3: Quick estimates

Trade-offs:

• More splits: Better estimation, longer runtime
• Fewer splits: Faster, less stable estimates

Balance between computation and stability.

Random seed for reproducible shuffling. Controls:

• Split randomization
• Sample selection
• Result reproducibility

Important for:

• Debugging
• Comparative studies
• Result verification

Proportion of samples for test set. Guidelines: Common ratios:

• 0.2: Standard (80/20 split)
• 0.25: More validation emphasis
• 0.1: More training data

Considerations:

• Dataset size
• Model complexity
• Validation requirements

It must be between 0.0 and 1.0.

Number of stratified random splits. Guidelines: Recommended values:

• 5: Standard evaluation
• 10: Detailed analysis
• 3: Quick assessment

Consider:

• Sample size per class
• Computational resources
• Stability requirements

Seed for reproducible stratified sampling. Ensures:

• Consistent class proportions
• Reproducible splits
• Comparable experiments

Critical for:

• Benchmarking
• Research studies
• Quality assurance

Fraction of samples for stratified test set. Best practices: Common splits:

• 0.2: Balanced evaluation
• 0.3: More thorough testing
• 0.15: Preserve training size

Consider:

• Minority class size
• Overall dataset size
• Validation objectives

It must be between 0.0 and 1.0.

Number of temporal splits. Considerations: Typical values:

• 5: Standard forward chaining
• 3: Limited historical data
• 10: Long time series

Impact:

• Affects training window growth
• Determines validation points
• Influences computational load

Maximum size of training set. Should be strictly less than the number of samples. Applications:

• 0: Use all available past data
• >0: Rolling window of fixed size

Use cases:

• Limit historical relevance
• Control computational cost
• Handle concept drift
• Memory constraints

Number of samples in each test set. When 0:

• Auto-calculated as n_samples/(n_splits+1)
• Ensures equal-sized test sets

Considerations:

• Forecast horizon
• Validation requirements
• Available future data

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

• Avoid data leakage
• Model forecast lag
• Buffer periods

Common scenarios:

• 0: Continuous prediction
• >0: Forward gap for realistic evaluation
• Match business forecasting needs