Lasso / Regressor Layer

L1 regularization strength:

Mathematical impact: $$\underset{\beta }{\min }\frac{1}{2n}||y-X\beta |{|}_2^2+\alpha ||\beta |{|}_1$$

Effects:

• Controls sparsity
• Feature selection
• Coefficient shrinkage
• Solution stability

Typical values:

• Weak: 0.01-0.1
• Standard: 1.0
• Strong: 10.0-100.0

Selection guide:

• Higher: More sparsity
• Lower: More features
• Zero: Standard OLS

Intercept calculation control:

Model forms: With intercept: $$y=X\beta +b+ϵ$$ Without intercept: $$y=X\beta +ϵ$$

Effects when True:

• Centers predictions
• Accounts for bias
• Improves fit
• Standard modeling

Effects when False:

• Forces origin fitting
• No bias term
• Domain-specific needs
• Theoretical constraints

Maximum iteration limit:

Impact:

• Convergence control
• Solution quality
• Computation time
• Resource usage

Typical ranges:

• Simple problems: 100-500
• Standard: 1000
• Complex: 2000-5000

Guidelines:

• Increase for harder problems
• Monitor convergence
• Balance with tolerance

Optimization tolerance threshold:

Convergence criterion:

• Checks dual gap
• Controls precision
• Affects iterations

Typical values:

• Strict: 1e-6 to 1e-5
• Standard: 1e-4
• Relaxed: 1e-3

Trade-offs:

• Precision vs speed
• Solution quality
• Computation time

When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution.

When True:

• Reuses previous solution
• Continues optimization
• Preserves coefficients
• Speeds convergence

Applications:

• Parameter studies
• Path algorithms
• Transfer learning
• Incremental fitting

Best practices:

• Monitor convergence
• Check stability
• Validate results
• Document usage

When set to True, forces the coefficients to be positive.

Mathematical impact: $$\underset{\beta \ge 0}{\min }\frac{1}{2n}||y-X\beta |{|}_2^2+\alpha ||\beta |{|}_1$$

Applications:

• Signal processing
• Image reconstruction
• Chemical concentrations
• Physical quantities

Effects:

• Constrains solutions
• Ensures interpretability
• May slow convergence
• Domain-specific needs

The seed of the pseudo random number generator that selects a random feature to update. Used when selection == 'random'.

Controls:

• Random selection mode
• Coefficient initialization
• Stochastic processes

Usage patterns:

1. Development:

   • Fixed seed
   • Reproducible results
   • Debug capability

2. Production:

   • Multiple seeds
   • Robustness checks
   • Stability testing

If set to 'random', a random coefficient is updated every iteration rather than looping over features sequentially by default.

Algorithm variations:

1. Random selection:

   • Stochastic updates
   • Faster convergence
   • Better for high tolerance

2. Cyclic updates:

   • Sequential processing
   • Deterministic behavior
   • Traditional approach

Performance impact:

• Convergence speed
• Solution stability
• Memory patterns

Regularization strength search space:

Search strategies:

1. Log scale (recommended):

   • [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0]
   • Wide parameter coverage
   • Common choice

2. Fine-tuning:

   • [0.1, 0.3, 0.7, 1.0, 1.5]
   • Around promising values
   • Narrow search

3. Problem-specific:

   • Based on domain knowledge
   • Data characteristics
   • Target sparsity level

Intercept inclusion search space:

Options:

1. Single mode:

   • [true]: Standard modeling

2. Complete search:

   • [true, false]: Compare both

Selection impact:

• Model flexibility
• Prediction bias
• Solution sparsity
• Theoretical alignment

Maximum iterations search space:

Search patterns:

1. Basic range:

   • [100, 500, 1000]
   • Standard problems

2. Extended search:

   • [500, 1000, 2000, 5000]
   • Complex problems

3. Convergence study:

   • [100, 300, 1000, 3000]
   • Log-scale spacing

Convergence tolerance search space:

Search ranges:

1. Standard scale:

   • [1e-4, 1e-3, 1e-2]
   • Common values

2. High precision:

   • [1e-6, 1e-5, 1e-4]
   • Strict convergence

3. Quick convergence:

   • [1e-3, 1e-2, 1e-1]
   • Faster training

When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.

Options:

1. Single mode:

   • [false]: Fresh starts
   • [true]: Solution reuse

2. Comparison:

   • [true, false]: Compare both

Study focus:

• Convergence speed
• Solution quality
• Training efficiency

Positive constraint evaluation:

Search options:

1. Unconstrained: [false]

   • Standard modeling
   • Faster convergence

2. Constrained: [true]

   • Physical meanings
   • Domain requirements

3. Comparison: [true, false]

   • Impact analysis
   • Trade-off study

If set to 'random', a random coefficient is updated every iteration rather than looping over features sequentially by default.

Algorithm variations:

1. Random selection:

   • Stochastic updates
   • Faster convergence
   • Better for high tolerance

2. Cyclic updates:

   • Sequential processing
   • Deterministic behavior
   • Traditional approach

Performance impact:

• Convergence speed
• Solution stability
• Memory patterns

Random number generation seed:

Usage:

• Fixed: Reproducibility
• Varied: Robustness checks
• None: Random behavior

Best practices:

• Document seed values
• Test multiple seeds
• Ensure replicability

Regression model evaluation metrics:

Purpose:

• Model performance evaluation
• Error measurement
• Quality assessment
• Model comparison

Selection criteria:

• Error distribution
• Scale sensitivity
• Domain requirements
• Business objectives

Random seed for reproducible splits. Ensures:

• Consistent train/test sets
• Reproducible experiments
• Comparable model evaluations

Same seed guarantees identical splits across runs.

Data shuffling before splitting. Effects:

• true: Randomizes order, better for i.i.d. data
• false: Maintains order, important for time series

When to disable:

• Time dependent data
• Sequential patterns
• Grouped observations

Proportion of data for training. Considerations:

• Larger (e.g., 0.8-0.9): Better model learning
• Smaller (e.g., 0.5-0.7): Better validation

Common splits:

• 0.8: Standard (80/20 split)
• 0.7: More validation emphasis
• 0.9: More training emphasis

Maintain class distribution in splits. Important when:

• Classes are imbalanced
• Small classes present
• Representative splits needed

Requirements:

• Classification tasks only
• Cannot use with shuffle=false
• Sufficient samples per class

Number of cross-validation folds. Guidelines:

• 3-5: Large datasets, faster training
• 5-10: Standard choice, good balance
• 10+: Small datasets, thorough evaluation

Trade-offs:

• More folds: Better evaluation, slower training
• Fewer folds: Faster training, higher variance

Must be at least 2.

Number of folds for cross-validation. Selection guide: Recommended values:

• 5: Standard choice (default)
• 3: Large datasets/quick evaluation
• 10: Thorough evaluation/smaller datasets

Trade-offs:

• Higher values: More thorough, computationally expensive
• Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

Random seed for fold generation when shuffling. Important for:

• Reproducible results
• Consistent fold assignments
• Benchmark comparisons
• Debugging and validation

Set specific value for reproducibility across runs.

Whether to shuffle data before splitting into folds. Effects:

• true: Randomized fold composition (recommended)
• false: Sequential splitting

Enable when:

• Data may have ordering
• Better fold independence needed

Disable for:

• Time series data
• Ordered observations

Number of stratified folds. Guidelines: Typical values:

• 5: Standard for most cases
• 3: Quick evaluation/large datasets
• 10: Detailed evaluation/smaller datasets

Considerations:

• Must allow sufficient samples per class per fold
• Balance between stability and computation time
• Consider smallest class size when choosing

Seed for reproducible stratified splits. Ensures:

• Consistent fold assignments
• Reproducible results
• Comparable experiments
• Systematic validation

Fixed seed guarantees identical stratified splits.

Data shuffling before stratified splitting. Impact:

• true: Randomizes while maintaining stratification
• false: Maintains data order within strata

Use cases:

• true: Independent observations
• false: Grouped or sequential data

Class proportions maintained regardless of setting.

Number of random splits to perform. Consider: Common values:

• 5: Standard evaluation
• 10: More thorough assessment
• 3: Quick estimates

Trade-offs:

• More splits: Better estimation, longer runtime
• Fewer splits: Faster, less stable estimates

Balance between computation and stability.

Random seed for reproducible shuffling. Controls:

• Split randomization
• Sample selection
• Result reproducibility

Important for:

• Debugging
• Comparative studies
• Result verification

Proportion of samples for test set. Guidelines: Common ratios:

• 0.2: Standard (80/20 split)
• 0.25: More validation emphasis
• 0.1: More training data

Considerations:

• Dataset size
• Model complexity
• Validation requirements

It must be between 0.0 and 1.0.

Number of stratified random splits. Guidelines: Recommended values:

• 5: Standard evaluation
• 10: Detailed analysis
• 3: Quick assessment

Consider:

• Sample size per class
• Computational resources
• Stability requirements

Seed for reproducible stratified sampling. Ensures:

• Consistent class proportions
• Reproducible splits
• Comparable experiments

Critical for:

• Benchmarking
• Research studies
• Quality assurance

Fraction of samples for stratified test set. Best practices: Common splits:

• 0.2: Balanced evaluation
• 0.3: More thorough testing
• 0.15: Preserve training size

Consider:

• Minority class size
• Overall dataset size
• Validation objectives

It must be between 0.0 and 1.0.

Number of temporal splits. Considerations: Typical values:

• 5: Standard forward chaining
• 3: Limited historical data
• 10: Long time series

Impact:

• Affects training window growth
• Determines validation points
• Influences computational load

Maximum size of training set. Should be strictly less than the number of samples. Applications:

• 0: Use all available past data
• >0: Rolling window of fixed size

Use cases:

• Limit historical relevance
• Control computational cost
• Handle concept drift
• Memory constraints

Number of samples in each test set. When 0:

• Auto-calculated as n_samples/(n_splits+1)
• Ensures equal-sized test sets

Considerations:

• Forecast horizon
• Validation requirements
• Available future data

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

• Avoid data leakage
• Model forecast lag
• Buffer periods

Common scenarios:

• 0: Continuous prediction
• >0: Forward gap for realistic evaluation
• Match business forecasting needs