fitproblem - Interface between models and fitters

Fitness	Manage parameters, data, and theory function evaluation.
FitProblem	models is a sequence of Fitness instances. Note that they
CovarianceMixin	Add methods for cov, show_cov and show_err to a bumps problem definition.
load_problem	Load a model file.

Interface between the models and the fitters.

Fitness defines the interface that model evaluators can follow. These models can be bundled together into a FitProblem() and sent to bumps.fitters.FitDriver for optimization and uncertainty analysis.

Summary of problem attributes:

# Used by fitters
nllf(p: Optional[Vector]) -> float  # main calculation
bounds() -> Tuple(Vector, Vector)    # or equivalent sequence
setp(p: Vector) -> None
getp() -> Vector
residuals() -> Vector  # for LM, MPFit
parameter_residuals() -> Vector  # for LM, MPFit
constraints_nllf() -> float # for LM, MPFit;  constraint cost is spread across the individual residuals
randomize() -> None # for multistart
resynth_data() -> None  # for Monte Carlo resampling of maximum likelihood
restore_data() -> None # for Monte Carlo resampling of maximum likelihood
name: str  # DREAM uses this
chisq() -> float
chisq_str() -> str
labels() -> List[str]
summarize() -> str
show() -> None
load(input_path: str) -> None
save(output_path: str) -> None
plot(figfile: str, view: str) -> None

# Set/used by bumps.cli
model_reset() -> None # called by load_model
path: str  # set by load_model
name: str # set by load_model
title: str = filename # set by load_moel
options: List[str]  # from sys.argv[1:]
undefined:List[int]  # when loading a save .par file, these parameters weren't defined
store: str # set by make_store
output_path: str # set by make_store
simulate_data(noise: float) -> None # for --simulate in opts
cov() -> Matrix   # for --cov in opts

class bumps.fitproblem.CovarianceMixin

Bases: object

Add methods for cov, show_cov and show_err to a bumps problem definition.

This is done as a mixin because not all problems are FitProblem. See for example bumps.pdfwrapper.PDF.

cov(x)

Return an estimate of the covariance of the fit.

Depending on the fitter and the problem, this may be computed from existing evaluations within the fitter, or from numerical differentiation around the minimum.

If the problem has residuals available, then the covariance is derived from the Jacobian:

x = fit.problem.getp()
J = bumps.lsqerror.jacobian(fit.problem, x)
cov = bumps.lsqerror.jacobian_cov(J)

Otherwise, the numerical differentiation will use the Hessian estimated from nllf:

x = fit.problem.getp()
H = bumps.lsqerror.hessian(fit.problem, x)
cov = bumps.lsqerror.hessian_cov(H)

show_cov(x, cov)

show_err(x, dx)

Display the error approximation from the covariance matrix.

err is the standard deviation computed from the covariance matrix. It is available as result.dx from the simple fitter, or using:

from bumps import lsqerror

dx = lsqerror.stderr(problem.cov(x))

Warning: cost to compute cov grows as the cube of the number of parameters.

class bumps.fitproblem.FitProblem(models: FitnessType | List[FitnessType], weights=None, name=None, constraints=None, penalty_nllf=None, freevars=None, auto_tag=False)

Bases: Generic[FitnessType], CovarianceMixin

models is a sequence of Fitness instances. Note that they do not need to all be of the same class.

weights is an optional scale factor for each model. A weighted fit returns nllf \(L = \sum w_k^2 L_k\). If an individual nllf is the sum squared residuals then this is equivalent to scaling the measurement uncertainty by \(1/w\). Unless the measurement uncertainty is unknown, weights should be in [0, 1], representing an unknown systematic uncertainty spread across the individual measurements.

freevars is parameter.FreeVariables instance defining the per-model parameter assignments. See Free Variables for details.

Additional parameters:

name name of the problem

constraints is a list of Constraint objects, which have a method to calculate the nllf for that constraint. Also supports an alternate form which cannot be serialized: A function which returns the negative log likelihood of seeing the parameters independent from the fitness function. Use this for example to check for feasible regions of the search space, or to add constraints that cannot be easily calculated per parameter. Ideally, the constraints nllf will increase as you go farther from the feasible region so that the fit will be directed toward feasible values.

penalty_nllf is the nllf to use for fitness when constraints or model parameter bounds are not satisfied. The total nllf is the squared distance from the boundary plus the penalty so that the derivative points the search back to the feasible region. The penalty should be larger than any nllf you might see near the boundary so that the fit doesn’t get stuck outside, but small enough that penalty plus distance is different from penalty. The default is 1e12.

Total nllf is the sum of the parameter nllf, the constraints nllf and the depending on whether constraints is greater than soft_limit, either the fitness nllf or the penalty nllf.

New in 0.9.0: weights are now squared when computing the sum rather than linear.

bounds()

Return the bounds for each parameter as a 2 x N array

chisq(nllf: float | ndarray[tuple[Any, ...], dtype[_ScalarT]] | None = None, norm: bool = True, compact: bool = True)

Returns chisq as a floating point value.

See documentation for chisq_str().

chisq_str(nllf: float | None = None, norm: bool = True, compact: bool = True)

Return a string representing the chisq equivalent of the nllf.

If nllf is provided then use that instead of calling the model evaluator. Fail if compact is False.

If the model has strictly gaussian independent uncertainties then the negative log likelihood function will return 0.5*sum(residuals**2), which is 1/2*chisq. Since we are printing normalized chisq, we multiply the model nllf by 2/DOF before displaying the value. This is different from the problem nllf function, which includes the cost of the cost of the penalty constraints in the total nllf.

Parameter priors, if any, are treated as independent models in the total nllf. The constraint value is displayed separately.

Deprecated: norm:bool and compact:bool are ignored.

constraints: Sequence[Constraint] | None

constraints_nllf() → Tuple[float, List[str]]

Return the cost function for all constraints

cov(x)

Return an estimate of the covariance of the fit.

Depending on the fitter and the problem, this may be computed from existing evaluations within the fitter, or from numerical differentiation around the minimum.

If the problem has residuals available, then the covariance is derived from the Jacobian:

x = fit.problem.getp()
J = bumps.lsqerror.jacobian(fit.problem, x)
cov = bumps.lsqerror.jacobian_cov(J)

Otherwise, the numerical differentiation will use the Hessian estimated from nllf:

x = fit.problem.getp()
H = bumps.lsqerror.hessian(fit.problem, x)
cov = bumps.lsqerror.hessian_cov(H)

property dof

property fitness

freevars: FreeVariables | None

getp()

Returns the current value of the parameter vector.

property has_residuals

True if all underlying fitness functions define residuals.

labels() → List[str]

Return the list of labels, one per fitted parameter.

model_nllf()

Return cost function for all data sets

model_parameters()

Return parameters from all models

model_points()

Return number of points in all models

model_reset()

Prepare for the fit.

This sets the parameters and the bounds properties that the solver is expecting from the fittable object. We also compute the degrees of freedom so that we can return a normalized fit likelihood.

If the set of fit parameters changes, then model_reset must be called.

model_update()

Let all models know they need to be recalculated

property models

Iterate over models, with free parameters set from model values

name: str | None

nllf(pvec=None) → float

Compute the cost function for a new parameter set p.

This is not simply the sum-squared residuals, but instead is the negative log likelihood of seeing the data given the model parameters plus the negative log likelihood of seeing the model parameters. The value is used for a likelihood ratio test so normalization constants can be ignored. There is an additional penalty value provided by the model which can be used to implement inequality constraints. Any penalty should be large enough that it is effectively excluded from the parameter space returned from uncertainty analysis.

The model is not actually calculated if any of the parameters are out of bounds, or any of the constraints are not satisfied, but instead are assigned a value of penalty_nllf. This will prevent expensive models from spending time computing values in the unfeasible region.

property num_models

parameter_nllf() → Tuple[float, List[str]]

Returns negative log likelihood of seeing parameters p.

parameter_residuals()

Returns negative log likelihood of seeing parameters p.

property parameters

Return the list of fitted parameters.

penalty_nllf: float | Literal['inf']

plot(p=None, fignum=1, figfile=None, view=None, model_indices=None)

push_model(index)

Fetch model index with the appropriate free variables substituted.

On completion of the context, restore the parameters for the active model.

randomize(n=None)

Generates a random model.

randomize() sets the model to a random value.

randomize(n) returns a population of n random models.

For indefinite bounds, the random population distribution is centered on initial value of the parameter, or 1. if the initial parameter is not finite.

residuals()

restore_data()

Restore original data after resynthesis.

resynth_data()

Resynthesize data with noise from the uncertainty estimates.

save(basename)

set_active_model(index)

Fetch model index with the appropriate free variables substituted.

This will remain the active model until a new active model is selected.

Operations like chisq_str() or plot() which cycle through the models will restore the parameters upon completion.

setp(pvec)

Set a new value for the parameters into the model. If the model is valid, calls model_update to signal that the model should be recalculated.

Returns True if the value is valid and the parameters were set, otherwise returns False.

show()

show_cov(x, cov)

show_err(x, dx)

Display the error approximation from the covariance matrix.

err is the standard deviation computed from the covariance matrix. It is available as result.dx from the simple fitter, or using:

from bumps import lsqerror

dx = lsqerror.stderr(problem.cov(x))

Warning: cost to compute cov grows as the cube of the number of parameters.

simulate_data(noise=None)

Simulate data with added noise

summarize()

Return a table of current parameter values with range bars.

to_dict()

valid(pvec)

Return true if the point is in the feasible region

weights: List[float] | Literal[None]

class bumps.fitproblem.Fitness(*args, **kwargs)

Bases: Protocol

Manage parameters, data, and theory function evaluation.

See Complex models for a detailed explanation.

nllf()

Return the negative log likelihood value of the current parameter set.

numpoints()

Return the number of data points.

parameters() → List[Parameter]

return the parameters in the model.

model parameters are a hierarchical structure of lists and dictionaries.

plot(view='linear')

Plot the model to the current figure. You only get one figure, but you can make it as complex as you want. This will be saved as a png on the server, and composed onto a results web page.

residuals()

Return residuals for current theory minus data.

Used for Levenburg-Marquardt, and for plotting.

restore_data()

Restore the original data in the model (after resynth).

resynth_data()

Generate fake data based on uncertainties in the real data. For Monte Carlo resynth-refit uncertainty analysis. Bootstrapping?

save(basename)

Save the model to a file based on basename+extension. This will point to a path to a directory on a remote machine; don’t make any assumptions about information stored on the server. Return the set of files saved so that the monitor software can make a pretty web page.

update()

Called when parameters have been updated. Any cached values will need to be cleared and the model reevaluated.

bumps.fitproblem.load_problem(path: Path | str, args: list[str] | None = None)

Load a model file.

path contains the path to the model file. This could be a python script or a previously saved problem, serialized as .json, .cloudpickle, .pickle or .dill

args are any additional arguments to the model. The sys.argv variable will be set such that sys.argv[1:] == model_options.