mystic package documentation
mystic: constrained nonlinear optimization for scientific machine learning, UQ, and AI
About Mystic
The mystic
framework provides a collection of optimization algorithms
and tools that allows the user to more robustly (and easily) solve hard
optimization problems. All optimization algorithms included in mystic
provide workflow at the fitting layer, not just access to the algorithms
as function calls. mystic
gives the user fine-grained power to both
monitor and steer optimizations as the fit processes are running.
Optimizers can advance one iteration with Step
, or run to completion
with Solve
. Users can customize optimizer stop conditions, where both
compound and user-provided conditions may be used. Optimizers can save
state, can be reconfigured dynamically, and can be restarted from a
saved solver or from a results file. All solvers can also leverage
parallel computing, either within each iteration or as an ensemble of
solvers.
Where possible, mystic
optimizers share a common interface, and thus can
be easily swapped without the user having to write any new code. mystic
solvers all conform to a solver API, thus also have common method calls
to configure and launch an optimization job. For more details, see
mystic.abstract_solver
. The API also makes it easy to bind a favorite
3rd party solver into the mystic
framework.
Optimization algorithms in mystic
can accept parameter constraints,
as “soft constraints” (i.e. penalties
, which “penalize” regions of
solution space that violate the constraints), or as “hard constraints”
(i.e. constraints
, which constrain the solver to only search in regions
of space where the constraints are respected), or both. mystic
provides
a large selection of constraints, including probabistic and dimensionally
reducing constraints. By providing a robust interface designed to
enable the user to easily configure and control solvers, mystic
greatly reduces the barrier to solving hard optimization problems.
Sampling, interpolation, and statistics in mystic
are all designed
to seamlessly couple with constrained optimization to facilitate
scientific machine learning, uncertainty quantification, adaptive
sampling, nonlinear interpolation, and artificial intelligence.
mystic
can convert systems of equalities and inequalities to
hard or soft constraints using methods in mystic.symbolic
.
With mystic.constraints.vectorize
, constraints can be converted
to kernel transforms for use in machine learning. Similarly, mystic
provides tools for accurately producing emulators on an irregular grid
using mystic.math.interpolate
, which includes methods for solving
for gradients and Hessians. mystic.samplers
use optimizers to
drive adaptive sampling toward the first and second order critical points
of the response surface, yielding highly-informative training data sets
and ensuring emulator accuracy. mystic.math.discrete
defines
constrained discrete probability measures, which can be used in
constrained statistical optimization and learning.
mystic
is in active development, so any user feedback, bug reports, comments,
or suggestions are highly appreciated. A list of issues is located at https://github.com/uqfoundation/mystic/issues, with a legacy list maintained at https://uqfoundation.github.io/project/mystic/query.
Major Features
mystic
provides a stock set of configurable, controllable solvers with:
a common interface
a control handler with: pause, continue, exit, and callback
ease in selecting initial population conditions: guess, random, etc
ease in checkpointing and restarting from a log or saved state
the ability to leverage parallel & distributed computing
the ability to apply a selection of logging and/or verbose monitors
the ability to configure solver-independent termination conditions
the ability to impose custom and user-defined penalties and constraints
To get up and running quickly, mystic
also provides infrastructure to:
easily generate a model (several standard test models are included)
configure and auto-generate a cost function from a model
configure an ensemble of solvers to perform a specific task
Current Release
The latest released version of mystic
is available from:
mystic
is distributed under a 3-clause BSD license.
Development Version
You can get the latest development version with all the shiny new features at:
If you have a new contribution, please submit a pull request.
Installation
mystic
can be installed with pip
:
$ pip install mystic
To include optional scientific Python support, with scipy
, install:
$ pip install mystic[math]
To include optional plotting support with matplotlib
, install:
$ pip install mystic[plotting]
To include optional parallel computing support, with pathos
, install:
$ pip install mystic[parallel]
Requirements
mystic
requires:
python
(orpypy
), >=3.8
setuptools
, >=42
cython
, >=0.29.30
numpy
, >=1.0
sympy
, >=0.6.7
mpmath
, >=0.19
dill
, >=0.3.7
klepto
, >=0.2.4
Optional requirements:
matplotlib
, >=0.91
scipy
, >=0.6.0
pathos
, >=0.3.1
pyina
, >=0.2.8
More Information
Probably the best way to get started is to look at the documentation at
http://mystic.rtfd.io. Also see mystic.tests
for a set of scripts that
demonstrate several of the many features of the mystic
framework.
You can run the test suite with python -m mystic.tests
. There are
several plotting scripts that are installed with mystic
, primary of which
are mystic_log_reader
(also available with python -m mystic
) and the
mystic_model_plotter
(also available with python -m mystic.models
).
There are several other plotting scripts that come with mystic
, and they
are detailed elsewhere in the documentation. See https://github.com/uqfoundation/mystic/tree/master/examples for examples that demonstrate the basic use
cases for configuration and launching of optimization jobs using one of the
sample models provided in mystic.models
. Many of the included examples
are standard optimization test problems. The use of constraints and penalties
are detailed in https://github.com/uqfoundation/mystic/tree/master/examples2 while more advanced features leveraging ensemble solvers, machine learning,
uncertainty quantification, and dimensional collapse are found in https://github.com/uqfoundation/mystic/tree/master/examples3. The scripts in https://github.com/uqfoundation/mystic/tree/master/examples4 demonstrate leveraging pathos
for parallel computing, as well as demonstrate some auto-partitioning schemes.
mystic
has the ability to work in product measure space, and the scripts in
https://github.com/uqfoundation/mystic/tree/master/examples5 show how to work
with product measures at a low level. The source code is generally well
documented, so further questions may be resolved by inspecting the code itself.
Please feel free to submit a ticket on github, or ask a question on
stackoverflow (@Mike McKerns). If you would like to share how you use
mystic
in your work, please send an email (to mmckerns at uqfoundation
dot org).
Instructions on building a new model are in mystic.models.abstract_model
.
mystic
provides base classes for two types of models:
AbstractFunction
[evaluatesf(x)
for given evaluation pointsx
]
AbstractModel
[generatesf(x,p)
for given coefficientsp
]
mystic
also provides some convienence functions to help you build a
model instance and a cost function instance on-the-fly. For more
information, see mystic.forward_model
. It is, however, not necessary
to use base classes or the model builder in building your own model or
cost function, as any standard Python function can be used as long as it
meets the basic AbstractFunction
interface of cost = f(x)
.
All mystic
solvers are highly configurable, and provide a robust set of
methods to help customize the solver for your particular optimization
problem. For each solver, a minimal (scipy.optimize
) interface is also
provided for users who prefer to configure and launch their solvers as a
single function call. For more information, see mystic.abstract_solver
for the solver API, and each of the individual solvers for their minimal
functional interface.
mystic
enables solvers to use parallel computing whenever the user provides
a replacement for the (serial) Python map
function. mystic
includes a
sample map
in mystic.python_map
that mirrors the behavior of the
built-in Python map
, and a pool
in mystic.pools
that provides map
functions using the pathos
(i.e. multiprocessing
) interface. mystic
solvers are designed to utilize distributed and parallel tools provided by
the pathos
package. For more information, see mystic.abstract_map_solver
,
mystic.abstract_ensemble_solver
, and the pathos
documentation at
http://pathos.rtfd.io.
Important classes and functions are found here:
mystic.solvers
[solver optimization algorithms]
mystic.termination
[solver termination conditions]
mystic.strategy
[solver population mutation strategies]
mystic.monitors
[optimization monitors]
mystic.symbolic
[symbolic math in constraints]
mystic.constraints
[constraints functions]
mystic.penalty
[penalty functions]
mystic.collapse
[checks for dimensional collapse]
mystic.coupler
[decorators for function coupling]
mystic.pools
[parallel worker pool interface]
mystic.munge
[file readers and writers]
mystic.scripts
[model and convergence plotting]
mystic.samplers
[optimizer-guided sampling]
mystic.support
[hypercube measure support plotting]
mystic.forward_model
[cost function generator]
mystic.tools
[constraints, wrappers, and other tools]
mystic.cache
[results caching and archiving]
mystic.models
[models and test functions]
mystic.math
[mathematical functions and tools]
Important functions within mystic.math
are found here:
mystic.math.Distribution
[a sampling distribution object]
mystic.math.legacydata
[classes for legacy data observations]
mystic.math.discrete
[classes for discrete measures]
mystic.math.measures
[tools to support discrete measures]
mystic.math.approx
[tools for measuring equality]
mystic.math.grid
[tools for generating points on a grid]
mystic.math.distance
[tools for measuring distance and norms]
mystic.math.poly
[tools for polynomial functions]
mystic.math.samples
[tools related to sampling]
mystic.math.integrate
[tools related to integration]
mystic.math.interpolate
[tools related to interpolation]
mystic.math.stats
[tools related to distributions]
Solver, Sampler, and model API definitions are found here:
mystic.abstract_sampler
[the sampler API definition]
mystic.abstract_solver
[the solver API definition]
mystic.abstract_map_solver
[the parallel solver API]
mystic.abstract_ensemble_solver
[the ensemble solver API]
mystic.models.abstract_model
[the model API definition]
mystic
also provides several convience scripts that are used to visualize
models, convergence, and support on the hypercube. These scripts are installed
to a directory on the user’s $PATH
, and thus can be run from anywhere:
mystic_log_reader
[parameter and cost convergence]
mystic_log_converter
[logfile format converter]
mystic_collapse_plotter
[convergence and dimensional collapse]
mystic_model_plotter
[model surfaces and solver trajectory]
support_convergence
[convergence plots for measures]
support_hypercube
[parameter support on the hypercube]
support_hypercube_measures
[measure support on the hypercube]
support_hypercube_scenario
[scenario support on the hypercube]
Typing --help
as an argument to any of the above scripts will print out an
instructive help message.
Citation
If you use mystic
to do research that leads to publication, we ask that you
acknowledge use of mystic
by citing the following in your publication:
M.M. McKerns, L. Strand, T. Sullivan, A. Fang, M.A.G. Aivazis,
"Building a framework for predictive science", Proceedings of
the 10th Python in Science Conference, 2011;
http://arxiv.org/pdf/1202.1056
Michael McKerns, Patrick Hung, and Michael Aivazis,
"mystic: highly-constrained non-convex optimization and UQ", 2009- ;
https://uqfoundation.github.io/project/mystic
Please see https://uqfoundation.github.io/project/mystic or http://arxiv.org/pdf/1202.1056 for further information.
- citation()
print citation
- collapse_plotter(filename, **kwds)
generate convergence plots of | cost - cost_min | from convergence logfile
Available from the command shell as:
mystic_collapse_plotter filename [options]
or as a function call:
mystic.collapse_plotter(filename, **options)
- Parameters
filename (str) – name of the convergence logfile (e.g
paramlog.py
).- Returns
None
Notes
The option dots takes a boolean, and will show data points in the plot.
The option linear takes a boolean, and will plot in a linear scale.
The option out takes a string of the filepath for the generated plot. If
out = True
, return the Figure object instead of generating a plot.The option iter takes an integer of the largest iteration to plot.
The option legend takes a boolean, and will display the legend.
The option nid takes an integer of the nth simultaneous points to plot.
The option label takes a label string. For example,
label = "y"
will label the plot with a ‘y’, whilelabel = " log-cost, $ log_{10}(\hat{P} - \hat{P}_{max})$"
will label the y-axis with standard LaTeX math formatting. Note that the leading space is required, and that the text is aligned along the axis.The option col takes a string of comma-separated integers indicating iteration numbers where parameter collapse has occurred. If a second set of integers is provided (delineated by a semicolon), the additional set of integers will be plotted with a different linestyle (to indicate a different type of collapse).
- license()
print license
- log_converter(readpath, writepath=None, **kwds)
convert between cached archives, convergence logfiles, and support logfiles
Available from the command shell as:
mystic_log_converter readpath (writepath) [options]
or as a function call:
mystic.log_converter(readpath, writepath=None, **options)
- Parameters
- Returns
None
Notes
If writepath is None, write file with derived name to current directory.
The option format takes a string name of the file format at writepath. Available formats are (‘logfile’, ‘support’, or a klepto.archive type).
- log_reader(filename, **kwds)
generate parameter convergence plot from convergence logfile
Available from the command shell as:
mystic_log_reader filename [options]
or as a function call:
mystic.log_reader(filename, **options)
- Parameters
filename (str) – name of the convergence logfile (e.g
log.txt
).- Returns
None
Notes
The option out takes a string of the filepath for the generated plot. If
out = True
, return the Figure object instead of generating a plot.The option dots takes a boolean, and will show data points in the plot.
The option line takes a boolean, and will connect the data with a line.
The option iter takes an integer of the largest iteration to plot.
The option legend takes a boolean, and will display the legend.
The option nid takes an integer of the nth simultaneous points to plot.
The option param takes an indicator string. The indicator string is built from comma-separated array slices. For example,
param = ":"
will plot all parameters. Alternatively,param = ":2, 3:"
will plot all parameters except for the third parameter, whileparam = "0"
will only plot the first parameter.
- model_plotter(model, logfile=None, **kwds)
generate surface contour plots for model, specified by full import path; and generate model trajectory from logfile (or solver restart file), if provided
Available from the command shell as:
mystic_model_plotter model (logfile) [options]
or as a function call:
mystic.model_plotter(model, logfile=None, **options)
- Parameters
- Returns
None
Notes
The option out takes a string of the filepath for the generated plot. If
out = True
, return the Figure object instead of generating a plot.The option bounds takes an indicator string, where bounds are given as comma-separated slices. For example, using
bounds = "-1:10, 0:20"
will set lower and upper bounds for x to be (-1,10) and y to be (0,20). The “step” can also be given, to control the number of lines plotted in the grid. Thus"-1:10:.1, 0:20"
sets the bounds as above, but uses increments of .1 along x and the default step along y. For models > 2D, the bounds can be used to specify 2 dimensions plus fixed values for remaining dimensions. Thus,"-1:10, 0:20, 1.0"
plots the 2D surface where the z-axis is fixed at z=1.0. When called from a script, slice objects can be used instead of a string, thus"-1:10:.1, 0:20, 1.0"
becomes(slice(-1,10,.1), slice(20), 1.0)
.The option label takes comma-separated strings. For example,
label = "x,y,"
will place ‘x’ on the x-axis, ‘y’ on the y-axis, and nothing on the z-axis. LaTeX is also accepted. For example,label = "$ h $, $ {\alpha}$, $ v$"
will label the axes with standard LaTeX math formatting. Note that the leading space is required, while a trailing space aligns the text with the axis instead of the plot frame.The option nid takes an integer of the nth simultaneous points to plot.
The option iter takes an integer of the largest iteration to plot.
The option reduce can be given to reduce the output of a model to a scalar, thus converting
model(params)
toreduce(model(params))
. A reducer is given by the import path (e.g.numpy.add
).The option scale will convert the plot to log-scale, and scale the cost by
z=log(4*z*scale+1)+2
. This is useful for visualizing small contour changes around the minimium.If using log-scale produces negative numbers, the option shift can be used to shift the cost by
z=z+shift
. Both shift and scale are intended to help visualize contours.The option fill takes a boolean, to plot using filled contours.
The option depth takes a boolean, to plot contours in 3D.
The option dots takes a boolean, to show trajectory points in the plot.
The option join takes a boolean, to connect trajectory points.
The option verb takes a boolean, to print the model documentation.
The option kernel can be given to transform the input of a model from nD to 2D, where
params' = model(params)
withparams'
being 2D. A kernel is given by the import path (e.g.mymodule.kernel
). Using kernel can be slow, as it may calcuate inverse transform at each point.The option tol takes a float of max distance of dots from surface. For finer control, provide an array[float] the same length as
params
.