# mystic package documentation¶

## mystic: highly-constrained non-convex optimization and uncertainty quantification¶

The mystic framework provides a collection of optimization algorithms and tools that allows the user to more robustly (and easily) solve hard optimization problems. All optimization algorithms included in mystic provide workflow at the fitting layer, not just access to the algorithms as function calls. mystic gives the user fine-grained power to both monitor and steer optimizations as the fit processes are running. Optimizers can advance one iteration with Step, or run to completion with Solve. Users can customize optimizer stop conditions, where both compound and user-provided conditions may be used. Optimizers can save state, can be reconfigured dynamically, and can be restarted from a saved solver or from a results file. All solvers can also leverage parallel computing, either within each iteration or as an ensemble of solvers.

Where possible, mystic optimizers share a common interface, and thus can be easily swapped without the user having to write any new code. mystic solvers all conform to a solver API, thus also have common method calls to configure and launch an optimization job. For more details, see mystic.abstract_solver. The API also makes it easy to bind a favorite 3rd party solver into the mystic framework.

Optimization algorithms in mystic can accept parameter constraints, either in the form of penaties (which “penalize” regions of solution space that violate the constraints), or as constraints (which “constrain” the solver to only search in regions of solution space where the constraints are respected), or both. mystic provides a large selection of constraints, including probabistic and dimensionally reducing constraints. By providing a robust interface designed to enable the user to easily configure and control solvers, mystic greatly reduces the barrier to solving hard optimization problems.

mystic is in active development, so any user feedback, bug reports, comments, or suggestions are highly appreciated. A list of known issues is maintained at http://trac.mystic.cacr.caltech.edu/project/mystic/query.html, with a public ticket list at https://github.com/uqfoundation/mystic/issues.

### Major Features¶

mystic provides a stock set of configurable, controllable solvers with:

• a common interface
• a control handler with: pause, continue, exit, and callback
• ease in selecting initial population conditions: guess, random, etc
• ease in checkpointing and restarting from a log or saved state
• the ability to leverage parallel & distributed computing
• the ability to apply a selection of logging and/or verbose monitors
• the ability to configure solver-independent termination conditions
• the ability to impose custom and user-defined penalties and constraints

To get up and running quickly, mystic also provides infrastructure to:

• easily generate a model (several standard test models are included)
• configure and auto-generate a cost function from a model
• configure an ensemble of solvers to perform a specific task

### Current Release¶

This documentation is for version mystic-0.3.3.dev0.

The latest released version of mystic is available from:

mystic is distributed under a 3-clause BSD license.

>>> import mystic


### Development Version¶

You can get the latest development version with all the shiny new features at:

If you have a new contribution, please submit a pull request.

### Installation¶

mystic is packaged to install from source, so you must download the tarball, unzip, and run the installer:

[download]
$tar -xvzf mystic-0.3.2.tar.gz$ cd mystic-0.3.2
$python setup py build$ python setup py install


You will be warned of any missing dependencies and/or settings after you run the “build” step above. mystic depends on dill, numpy and sympy, so you should install them first. There are several functions within mystic where scipy is used if it is available; however, scipy is an optional dependency. Having matplotlib installed is necessary for running several of the examples, and you should probably go get it even though it’s not required. matplotlib is required for results visualization available in the scripts packaged with mystic.

Alternately, mystic can be installed with pip or easy_install:

$pip install mystic  ### Requirements¶ mystic requires: • python, version >= 2.6 or version >= 3.1, or pypy • numpy, version >= 1.0 • sympy, version >= 0.6.7 • dill, version >= 0.2.8.2 • klepto, version >= 0.1.5.2 Optional requirements: • setuptools, version >= 0.6 • matplotlib, version >= 0.91 • scipy, version >= 0.6.0 • mpmath, version >= 1.0.0 • pathos, version >= 0.2.2.1 • pyina, version >= 0.2.0 ### More Information¶ Probably the best way to get started is to look at the documentation at http://mystic.rtfd.io. Also see mystic.tests for a set of scripts that demonstrate several of the many features of the mystic framework. You can run the test suite with python -m mystic.tests. There are several plotting scripts that are installed with mystic, primary of which are mystic_log_reader (also available with python -m mystic) and the mystic_model_plotter (also available with python -m mystic.models). There are several other plotting scripts that come with mystic, and they are detailed elsewhere in the documentation. See mystic.examples for examples that demonstrate the basic use cases for configuration and launching of optimization jobs using one of the sample models provided in mystic.models. Many of the included examples are standard optimization test problems. The use of constraints and penalties are detailed in mystic.examples2, while more advanced features leveraging ensemble solvers and dimensional collapse are found in mystic.examples3. The scripts in mystic.examples4 demonstrate leveraging pathos for parallel computing, as well as demonstrate some auto-partitioning schemes. mystic has the ability to work in product measure space, and the scripts in mystic.examples5 show to work with product measures. The source code is generally well documented, so further questions may be resolved by inspecting the code itself. Please feel free to submit a ticket on github, or ask a question on stackoverflow (@Mike McKerns). If you would like to share how you use mystic in your work, please send an email (to mmckerns at uqfoundation dot org). Instructions on building a new model are in mystic.models.abstract_model. mystic provides base classes for two types of models: • AbstractFunction [evaluates f(x) for given evaluation points x] • AbstractModel [generates f(x,p) for given coefficients p] mystic also provides some convienence functions to help you build a model instance and a cost function instance on-the-fly. For more information, see mystic.forward_model. It is, however, not necessary to use base classes or the model builder in building your own model or cost function, as any standard python function can be used as long as it meets the basic AbstractFunction interface of cost = f(x). All mystic solvers are highly configurable, and provide a robust set of methods to help customize the solver for your particular optimization problem. For each solver, a minimal (scipy.optimize) interface is also provided for users who prefer to configure and launch their solvers as a single function call. For more information, see mystic.abstract_solver for the solver API, and each of the individual solvers for their minimal functional interface. mystic enables solvers to use parallel computing whenever the user provides a replacement for the (serial) python map function. mystic includes a sample map in mystic.python_map that mirrors the behavior of the built-in python map, and a pool in mystic.pools that provides map functions using the pathos (i.e. multiprocessing) interface. mystic solvers are designed to utilize distributed and parallel tools provided by the pathos package. For more information, see mystic.abstract_map_solver, mystic.abstract_ensemble_solver, and the pathos documentation at http://dev.danse.us/trac/pathos. Important classes and functions are found here: • mystic.solvers [solver optimization algorithms] • mystic.termination [solver termination conditions] • mystic.strategy [solver population mutation strategies] • mystic.monitors [optimization monitors] • mystic.symbolic [symbolic math in constaints] • mystic.constraints [constraints functions] • mystic.penalty [penalty functions] • mystic.collapse [checks for dimensional collapse] • mystic.coupler [decorators for function coupling] • mystic.pools [parallel worker pool interface] • mystic.munge [file readers and writers] • mystic.scripts [model and convergence plotting] • mystic.support [hypercube measure support plotting] • mystic.forward_model [cost function generator] • mystic.tools [constraints, wrappers, and other tools] • mystic.cache [results caching and archiving] • mystic.models [models and test functions] • mystic.math [mathematical functions and tools] Important functions within mystic.math are found here: • mystic.math.Distribution [a sampling distribution object] • mystic.math.legacydata [classes for legacy data observations] • mystic.math.discrete [classes for discrete measures] • mystic.math.measures [tools to support discrete measures] • mystic.math.approx [tools for measuring equality] • mystic.math.grid [tools for generating points on a grid] • mystic.math.distance [tools for measuring distance and norms] • mystic.math.poly [tools for polynomial functions] • mystic.math.samples [tools related to sampling] • mystic.math.integrate [tools related to integration] • mystic.math.stats [tools related to distributions] Solver and model API definitions are found here: • mystic.abstract_solver [the solver API definition] • mystic.abstract_map_solver [the parallel solver API] • mystic.abstract_ensemble_solver [the ensemble solver API] • mystic.models.abstract_model [the model API definition] mystic also provides several convience scripts that are used to visualize models, convergence, and support on the hypercube. These scripts are installed to a directory on the user’s $PATH, and thus can be run from anywhere:

• mystic_log_reader [parameter and cost convergence]
• mystic_collapse_plotter [convergence and dimensional collapse]
• mystic_model_plotter [model surfaces and solver trajectory]
• support_convergence [convergence plots for measures]
• support_hypercube [parameter support on the hypercube]
• support_hypercube_measures [measure support on the hypercube]
• support_hypercube_scenario [scenario support on the hypercube]

Typing --help as an argument to any of the above scripts will print out an instructive help message.

### Citation¶

If you use mystic to do research that leads to publication, we ask that you acknowledge use of mystic by citing the following in your publication:

M.M. McKerns, L. Strand, T. Sullivan, A. Fang, M.A.G. Aivazis,
"Building a framework for predictive science", Proceedings of
the 10th Python in Science Conference, 2011;
http://arxiv.org/pdf/1202.1056

Michael McKerns, Patrick Hung, and Michael Aivazis,
"mystic: highly-constrained non-convex optimization and UQ", 2009- ;
http://trac.mystic.cacr.caltech.edu/project/mystic


Please see http://trac.mystic.cacr.caltech.edu/project/mystic or http://arxiv.org/pdf/1202.1056 for further information.

license()

citation()

print citation

model_plotter(model, logfile=None, **kwds)

generate surface contour plots for model, specified by full import path; and generate model trajectory from logfile (or solver restart file), if provided

Available from the command shell as:

mystic_model_plotter.py model (logfile) [options]


or as a function call:

mystic.model_plotter(model, logfile=None, **options)

Parameters: model (str) – full import path for the model (e.g. mystic.models.rosen) logfile (str, default=None) – name of convergence logfile (e.g. log.txt) None

Notes

• The option out takes a string of the filepath for the generated plot.
• The option bounds takes an indicator string, where bounds are given as comma-separated slices. For example, using bounds = "-1:10, 0:20" will set lower and upper bounds for x to be (-1,10) and y to be (0,20). The “step” can also be given, to control the number of lines plotted in the grid. Thus "-1:10:.1, 0:20" sets the bounds as above, but uses increments of .1 along x and the default step along y. For models > 2D, the bounds can be used to specify 2 dimensions plus fixed values for remaining dimensions. Thus, "-1:10, 0:20, 1.0" plots the 2D surface where the z-axis is fixed at z=1.0. When called from a script, slice objects can be used instead of a string, thus "-1:10:.1, 0:20, 1.0" becomes (slice(-1,10,.1), slice(20), 1.0).
• The option label takes comma-separated strings. For example, label = "x,y," will place ‘x’ on the x-axis, ‘y’ on the y-axis, and nothing on the z-axis. LaTeX is also accepted. For example, label = "$h$, ${\alpha}$, $v$" will label the axes with standard LaTeX math formatting. Note that the leading space is required, while a trailing space aligns the text with the axis instead of the plot frame.
• The option nid takes an integer of the nth simultaneous points to plot.
• The option iter takes an integer of the largest iteration to plot.
• The option reduce can be given to reduce the output of a model to a scalar, thus converting model(params) to reduce(model(params)). A reducer is given by the import path (e.g. numpy.add).
• The option scale will convert the plot to log-scale, and scale the cost by z=log(4*z*scale+1)+2. This is useful for visualizing small contour changes around the minimium.
• If using log-scale produces negative numbers, the option shift can be used to shift the cost by z=z+shift. Both shift and scale are intended to help visualize contours.
• The option fill takes a boolean, to plot using filled contours.
• The option depth takes a boolean, to plot contours in 3D.
• The option dots takes a boolean, to show trajectory points in the plot.
• The option join takes a boolean, to connect trajectory points.
• The option verb takes a boolean, to print the model documentation.
log_reader(filename, **kwds)

plot parameter convergence from file written with LoggingMonitor

Available from the command shell as:

mystic_log_reader.py filename [options]


or as a function call:

mystic.log_reader(filename, **options)

Parameters: filename (str) – name of the convergence logfile (e.g log.txt). None

Notes

• The option out takes a string of the filepath for the generated plot.
• The option dots takes a boolean, and will show data points in the plot.
• The option line takes a boolean, and will connect the data with a line.
• The option iter takes an integer of the largest iteration to plot.
• The option legend takes a boolean, and will display the legend.
• The option nid takes an integer of the nth simultaneous points to plot.
• The option param takes an indicator string. The indicator string is built from comma-separated array slices. For example, params = ":" will plot all parameters. Alternatively, params = ":2, 3:" will plot all parameters except for the third parameter, while params = "0" will only plot the first parameter.
collapse_plotter(filename, **kwds)

generate cost convergence rate plots from file written with write_support_file

Available from the command shell as:

mystic_collapse_plotter.py filename [options]


or as a function call:

mystic.collapse_plotter(filename, **options)

Parameters: filename (str) – name of the convergence logfile (e.g paramlog.py). None

Notes

• The option dots takes a boolean, and will show data points in the plot.
• The option linear takes a boolean, and will plot in a linear scale.
• The option out takes a string of the filepath for the generated plot.
• The option iter takes an integer of the largest iteration to plot.
• The option label takes a label string. For example, label = "y" will label the plot with a ‘y’, while label = " log-cost, $log_{10}(\hat{P} - \hat{P}_{max})$"` will label the y-axis with standard LaTeX math formatting. Note that the leading space is required, and that the text is aligned along the axis.
• The option col takes a string of comma-separated integers indicating iteration numbers where parameter collapse has occurred. If a second set of integers is provided (delineated by a semicolon), the additional set of integers will be plotted with a different linestyle (to indicate a different type of collapse).