module econometron.utils.estimation.optimization

• genetic_algorithm_kalman function from The module
• simulated_annealing_kalman function from The module

Overview

The econometron.utils.estimation.optimization module provides tools for maximum likelihood estimation (MLE) of parameters in state-space models using global optimization techniques such as the Genetic Algorithm (GA) and Simulated Annealing (SA). These methods are particularly effective for non-linear and non-convex optimization problems, where traditional gradient-based approaches might get trapped in local optima. The module integrates seamlessly with the Kalman filter (via kalman_objective from econometron.filters) to compute the likelihood of state-space models, which are widely used in econometrics, time-series analysis, signal processing, and beyond—including Dynamic Stochastic General Equilibrium (DSGE) models. The functions handle general linear state-space models with Gaussian noise, making them versatile for various applications. The function kalman_objective computes the negative log-likelihood, which GA and SA minimize to find the parameter estimates θ̂.

Maximum Likelihood Estimation (MLE)

MLE involves finding the parameters that make the observed data most probable. The Kalman filter, implemented in econometron.filters.Kalman, computes the log-likelihood p(y|θ) by filtering the state-space model. The following functions wrap this likelihood computation in global optimization routines:

• genetic_algorithm_kalman
• simulated_annealing_kalman

Formally, given observed data $y$ and parameters $\theta$, MLE solves:

where $p(y \mid \theta)$ is the likelihood of the observed data under parameters $\theta$. For numerical stability, the log-likelihood is maximized instead of the raw likelihood. In this module, the Kalman filter evaluates the log-likelihood of candidate parameters, while GA and SA minimize its negative form, ensuring robust solutions even when the likelihood surface has multiple local optima or irregularities. This approach makes the module highly versatile, suitable for DSGE models, financial time-series, stochastic volatility models, and signal processing applications.

Functions

1. genetic_algorithm_kalman(...)

Purpose:
Implements a Genetic Algorithm (GA) for MLE of state-space model parameters. GA is a population-based evolutionary method that mimics natural selection, making it well-suited for complex, non-convex likelihood surfaces with multiple local optima.

Parameters:

Name	Type	Description	Default
y	np.ndarray or pd.DataFrame	Observations (shape: m × T or T × m). Auto-transposes if needed.	None
x0	np.ndarray	Initial parameter vector (shape: n_params).	None
lb	np.ndarray	Lower bounds for parameters.	None
ub	np.ndarray	Upper bounds for parameters.	None
param_names	list[str]	Names of parameters to estimate.	None
fixed_params	dict	Fixed parameters not estimated.	None
update_state_space	Callable	Function mapping params → state-space matrices.	None
pop_size	int	Population size.	50
n_gen	int	Number of generations.	100
crossover_rate	float	Probability of crossover (0–1).	0.8
mutation_rate	float	Probability of mutation (0–1).	0.1
elite_frac	float	Fraction of elites preserved.	0.1
seed	int	Random seed.	24
verbose	bool	Print optimization progress.	True

Returns:

• result: dict with
  • x: optimized parameters
  • fun: negative log-likelihood at optimum
  • nfev: number of function evaluations
  • message: optimization status
• table: summary of parameter estimates, SEs, log-likelihood, method

Workflow:

1. Prepare data y.
2. Define objective via kalman_objective.
3. Run GA with crossover, mutation, elitism.
4. Return best parameter set and results table.

2. simulated_annealing_kalman(...)

Purpose:
Implements Simulated Annealing (SA) for MLE of state-space model parameters. SA is a probabilistic method inspired by metallurgy that allows escaping local minima by occasionally accepting worse solutions based on a temperature schedule.

Parameters:

Name	Type	Description	Default
y	np.ndarray or pd.DataFrame	Observations (m × T or T × m).	None
x0	np.ndarray	Initial parameter vector.	None
lb	np.ndarray	Lower bounds for parameters.	None
ub	np.ndarray	Upper bounds for parameters.	None
param_names	list[str]	Names of parameters to estimate.	None
fixed_params	dict	Fixed parameters.	None
update_state_space	Callable	Function mapping params → state-space matrices.	None
T0	float	Initial temperature.	5.0
rt	float	Temperature reduction factor.	0.9
nt	int	Iterations per temperature level.	5
ns	int	Trials per iteration.	10
seed	int	Random seed.	42
max_evals	int	Max evaluations.	1,000,000
eps	float	Convergence threshold.	0.001

Returns:

• Same format as GA (result, table).

Workflow:

1. Start from x0 with T0.
2. Perturb parameters, evaluate likelihood.
3. Accept better/worse solutions probabilistically.
4. Reduce temperature by factor rt.
5. Stop when eps or max_evals reached.

Usage Example

import numpy as np
from econometron.utils.estimation.optimization import genetic_algorithm_kalman, simulated_annealing_kalman

# Generate synthetic data
np.random.seed(42)
T = 100
y = np.random.randn(2, T)

# Define state-space update
def update_state_space(params):
    return {
        'A': np.array([[params['phi'], 0], [0, 0.8]]),
        'D': np.eye(2),
        'Q': np.eye(2) * params['sigma_w'],
        'R': np.eye(2) * 0.05,
        'x0': np.zeros((2, 1)),
        'P0': np.eye(2)
    }

param_names = ['phi', 'sigma_w']
fixed_params = {'sigma_v': 0.05}
x0 = np.array([0.5, 0.1])
lb = np.array([0.01, 0.001])
ub = np.array([0.99, 1.0])

# Run Genetic Algorithm
ga_result = genetic_algorithm_kalman(y, x0, lb, ub, param_names, fixed_params, update_state_space)
print("GA Results:", ga_result['table'])

# Run Simulated Annealing
sa_result = simulated_annealing_kalman(y, x0, lb, ub, param_names, fixed_params, update_state_space)
print("SA Results:", sa_result['table'])

Notes

• Integration with Kalman Filter: Both functions rely on kalman_objective from econometron.filters.

• Input Flexibility: Accepts numpy.ndarray and pandas.DataFrame.

• Optimization Algorithms:

  • GA explores via crossover, mutation, elitism.
  • SA balances exploration/exploitation via a temperature schedule.

• Results Table: Built with create_results_table.

• Applicability: Useful in DSGE, financial time-series, signal processing, and control systems.

• Error Handling: GA includes explicit handling; SA relies on optimizer.

Dependencies

• econometron.filters (Kalman filter, likelihood).
• econometron.utils.optimizers (GA & SA).
• numpy, pandas.

Example Use Case

These optimizers are especially useful when estimating state-space models with complex likelihoods, such as:

• DSGE models with multiple regimes
• Financial time-series with stochastic volatility
• Signal processing (e.g., noisy tracking systems)

GA is ideal for broad exploration when little is known. SA works well when a good initial guess exists, refining solutions while avoiding local minima.