BayesianRidge / Regressor Layer

Maximum iterations for variational optimization:

Update sequence:

1. Update weights (w)
2. Update α (noise precision)
3. Update λ (weight precision)

Typical ranges:

• Simple problems: 100-200
• Standard: 300
• Complex: 500-1000

Convergence factors:

• Data complexity
• Prior specifications
• Desired precision
• Initial values

Convergence tolerance threshold:

Stopping criterion: $$||w_{t+1}-w_t||<tol$$

Typical values:

• Strict: 1e-4 to 1e-5
• Standard: 1e-3
• Relaxed: 1e-2

Impact:

• Solution precision
• Computation time
• Parameter stability

Noise precision shape parameter:

Prior specification: $$p(\alpha )=Gamma(\alpha |{\alpha }_1,{\alpha }_2)$$

Effects:

• Controls noise model
• Influences uncertainty
• Affects robustness

Selection guide:

• Uninformative: 1e-6
• Informative: >1
• Based on noise knowledge

Noise precision rate parameter:

Prior specification: $$E[\alpha ]=\frac{{\alpha }_1}{{\alpha }_2}$$

Impact:

• Prior variance
• Uncertainty scale
• Model flexibility

Selection:

• Match with alpha_1
• Data-driven choice
• Prior knowledge

Weight precision shape parameter:

Prior structure: $$p(\lambda )=Gamma(\lambda |{\lambda }_1,{\lambda }_2)$$

Effects:

• Regularization strength
• Weight shrinkage
• Model complexity

Usage:

• Weak prior: 1e-6
• Strong prior: >1
• Problem-dependent

Weight precision rate parameter:

Expected precision: $$E[\lambda ]=\frac{{\lambda }_1}{{\lambda }_2}$$

Controls:

• Prior variance
• Regularization scale
• Model flexibility

Selection:

• Match with lambda_1
• Based on scale
• Prior knowledge

Initial noise precision value:

Role in model:

• Starting point for α
• Initial uncertainty scale
• Convergence impact

Selection guide:

• Standard: 1.0
• High noise: <1.0
• Low noise: >1.0
• Data-driven choice

Initial weight precision value:

Impact on model:

• Starting regularization
• Initial weight scale
• Convergence behavior

Selection guide:

• Standard: 1.0
• Stronger reg: >1.0
• Weaker reg: <1.0
• Problem-specific

Objective function monitoring: whether to compute the objective function at each step of the model.

When True:

• Tracks convergence
• Monitors likelihood
• Enables diagnostics
• Validates updates

Trade-offs:

• Additional computation
• Better monitoring
• Memory usage
• Debugging capability

Intercept calculation control:

Model forms: With intercept: $$y=Xw+b+ϵ$$ Without intercept: $$y=Xw+ϵ$$

Effects when True:

• Centers predictions
• Improves fit
• Not regularized
• Standard modeling

Effects when False:

• Origin-constrained
• Full regularization
• Theory-driven
• Domain-specific cases

Iteration limit search space:

Search patterns:

1. Basic range:

   • [100, 300, 500]
   • Standard problems

2. Extended search:

   • [300, 500, 1000]
   • Complex problems

3. Convergence study:

   • [200, 400, 800]
   • Iteration impact

Tolerance threshold search:

Search ranges:

1. High precision:

   • [1e-5, 1e-4, 1e-3]
   • Accurate solutions

2. Standard range:

   • [1e-4, 1e-3, 1e-2]
   • Balanced approach

3. Quick convergence:

   • [1e-3, 1e-2]
   • Faster solutions

Noise precision shape parameter search:

Search spaces:

1. Uninformative priors:

   • [1e-6, 1e-4, 1e-2]
   • Weak assumptions

2. Informative priors:

   • [0.1, 1.0, 10.0]
   • Strong beliefs

3. Mixed approach:

   • Multiple scales
   • Prior sensitivity

Noise precision rate parameter search:

Search strategies:

1. Match alpha_1:

   • Similar values
   • Balanced priors

2. Scale exploration:

   • [1e-6, 1e-4, 1e-2]
   • Various scales

3. Prior impact:

   • Different combinations
   • Sensitivity analysis

Weight precision shape parameter search:

Search patterns:

1. Weak regularization:

   • [1e-6, 1e-4, 1e-2]
   • Flexible weights

2. Strong regularization:

   • [0.1, 1.0, 10.0]
   • Strict control

3. Comprehensive:

   • Mixed strengths
   • Effect study

Weight precision rate parameter search:

Search spaces:

1. Match lambda_1:

   • Balanced priors
   • Standard approach

2. Scale variation:

   • [1e-6, 1e-4, 1e-2]
   • Different scales

3. Regularization impact:

   • Various combinations
   • Control study

Initial noise precision search:

Search ranges:

1. Standard range:

   • [0.1, 1.0, 10.0]
   • Common values

2. Wide range:

   • [0.01, 1.0, 100.0]
   • Exploration

3. Problem-specific:

   • Based on noise
   • Data characteristics

Initial weight precision search:

Search patterns:

1. Conservative:

   • [0.1, 1.0, 10.0]
   • Standard range

2. Exploratory:

   • [0.01, 1.0, 100.0]
   • Wide coverage

3. Focused:

   • Around promising values
   • Based on results

Score computation evaluation:

Options:

1. No monitoring: [false]

   • Faster computation
   • Standard training

2. With monitoring: [true]

   • Convergence tracking
   • Diagnostic information

3. Compare: [true, false]

   • Performance impact
   • Resource trade-off

Intercept inclusion search:

Options:

1. Standard: [true]

   • Default modeling
   • Better fit

2. Zero-intercept: [false]

   • Theory-driven
   • Full regularization

3. Compare: [true, false]

   • Model evaluation
   • Impact analysis

Regression model evaluation metrics:

Purpose:

• Model performance evaluation
• Error measurement
• Quality assessment
• Model comparison

Selection criteria:

• Error distribution
• Scale sensitivity
• Domain requirements
• Business objectives

Random seed for reproducible splits. Ensures:

• Consistent train/test sets
• Reproducible experiments
• Comparable model evaluations

Same seed guarantees identical splits across runs.

Data shuffling before splitting. Effects:

• true: Randomizes order, better for i.i.d. data
• false: Maintains order, important for time series

When to disable:

• Time dependent data
• Sequential patterns
• Grouped observations

Proportion of data for training. Considerations:

• Larger (e.g., 0.8-0.9): Better model learning
• Smaller (e.g., 0.5-0.7): Better validation

Common splits:

• 0.8: Standard (80/20 split)
• 0.7: More validation emphasis
• 0.9: More training emphasis

Maintain class distribution in splits. Important when:

• Classes are imbalanced
• Small classes present
• Representative splits needed

Requirements:

• Classification tasks only
• Cannot use with shuffle=false
• Sufficient samples per class

Number of cross-validation folds. Guidelines:

• 3-5: Large datasets, faster training
• 5-10: Standard choice, good balance
• 10+: Small datasets, thorough evaluation

Trade-offs:

• More folds: Better evaluation, slower training
• Fewer folds: Faster training, higher variance

Must be at least 2.

Number of folds for cross-validation. Selection guide: Recommended values:

• 5: Standard choice (default)
• 3: Large datasets/quick evaluation
• 10: Thorough evaluation/smaller datasets

Trade-offs:

• Higher values: More thorough, computationally expensive
• Lower values: Faster, potentially higher variance

Must be at least 2 for valid cross-validation.

Random seed for fold generation when shuffling. Important for:

• Reproducible results
• Consistent fold assignments
• Benchmark comparisons
• Debugging and validation

Set specific value for reproducibility across runs.

Whether to shuffle data before splitting into folds. Effects:

• true: Randomized fold composition (recommended)
• false: Sequential splitting

Enable when:

• Data may have ordering
• Better fold independence needed

Disable for:

• Time series data
• Ordered observations

Number of stratified folds. Guidelines: Typical values:

• 5: Standard for most cases
• 3: Quick evaluation/large datasets
• 10: Detailed evaluation/smaller datasets

Considerations:

• Must allow sufficient samples per class per fold
• Balance between stability and computation time
• Consider smallest class size when choosing

Seed for reproducible stratified splits. Ensures:

• Consistent fold assignments
• Reproducible results
• Comparable experiments
• Systematic validation

Fixed seed guarantees identical stratified splits.

Data shuffling before stratified splitting. Impact:

• true: Randomizes while maintaining stratification
• false: Maintains data order within strata

Use cases:

• true: Independent observations
• false: Grouped or sequential data

Class proportions maintained regardless of setting.

Number of random splits to perform. Consider: Common values:

• 5: Standard evaluation
• 10: More thorough assessment
• 3: Quick estimates

Trade-offs:

• More splits: Better estimation, longer runtime
• Fewer splits: Faster, less stable estimates

Balance between computation and stability.

Random seed for reproducible shuffling. Controls:

• Split randomization
• Sample selection
• Result reproducibility

Important for:

• Debugging
• Comparative studies
• Result verification

Proportion of samples for test set. Guidelines: Common ratios:

• 0.2: Standard (80/20 split)
• 0.25: More validation emphasis
• 0.1: More training data

Considerations:

• Dataset size
• Model complexity
• Validation requirements

It must be between 0.0 and 1.0.

Number of stratified random splits. Guidelines: Recommended values:

• 5: Standard evaluation
• 10: Detailed analysis
• 3: Quick assessment

Consider:

• Sample size per class
• Computational resources
• Stability requirements

Seed for reproducible stratified sampling. Ensures:

• Consistent class proportions
• Reproducible splits
• Comparable experiments

Critical for:

• Benchmarking
• Research studies
• Quality assurance

Fraction of samples for stratified test set. Best practices: Common splits:

• 0.2: Balanced evaluation
• 0.3: More thorough testing
• 0.15: Preserve training size

Consider:

• Minority class size
• Overall dataset size
• Validation objectives

It must be between 0.0 and 1.0.

Number of temporal splits. Considerations: Typical values:

• 5: Standard forward chaining
• 3: Limited historical data
• 10: Long time series

Impact:

• Affects training window growth
• Determines validation points
• Influences computational load

Maximum size of training set. Should be strictly less than the number of samples. Applications:

• 0: Use all available past data
• >0: Rolling window of fixed size

Use cases:

• Limit historical relevance
• Control computational cost
• Handle concept drift
• Memory constraints

Number of samples in each test set. When 0:

• Auto-calculated as n_samples/(n_splits+1)
• Ensures equal-sized test sets

Considerations:

• Forecast horizon
• Validation requirements
• Available future data

Number of samples to exclude from the end of each train set before the test set.Gap between train and test sets. Uses:

• Avoid data leakage
• Model forecast lag
• Buffer periods

Common scenarios:

• 0: Continuous prediction
• >0: Forward gap for realistic evaluation
• Match business forecasting needs