Overview

This page describes Indie Solver as a tool designed to help you to solve your optimization problems with least costs. First, we describe our views on practical aspects of numerical optimization and share our arguments why Indie Solver should be considered. Then, we proceed with a more focused description (examples and specifications) of how you can use Indie Solver in practice. The content of this page is far from optimal. If you have any question or comment please send us an email at contact@indiesolver.com.

Preliminaries

Terminology

Optimization consists of finding best solutions from a set of alternative candidate solutions. Single or multiple criteria can be used to define preference relations among solutions necessary to identify which of them are better/best. Candidate solutions are drawn from the search domain which is constrained in some way. It is often constrained by defining a set of modifiable parameters and their domains which in turn can be constrained or unconstrained. Candidate solutions of the optimization problem at hand are drawn from the search domain according to some search/optimization algorithm.

Terminology varies substantially in different subfields of (applied) optimization, see below.

Term Related terms
Objective (function) loss, metric, cost, utility, criterion, measurement, fitness, factor
Parameter variable, hyperparameter

Often these terms cannot be used interchangeably. For instance, a few hyperparameters might govern the process of obtaining machine learning models which themselves might consist of billions of parameters.

The following terminology will be used in this document:

Term Meaning
Black-box A device, system or object which delivers some outputs as a function of inputs.
Metrics A subset of black-box outputs, some metrics are to be optimized.
Parameters A subset of black-box inputs which are modifiable and affect problem metrics.
(Black-box) Optimization problem A problem of finding feasible solutions whose evaluation by the black-box results in metrics values which best satisfy some problem-specific requirements.

Indie Solver generates candidate solutions according to the user-provided problem description in order to gradually improve solutions with respect to the requirements imposed on the optimized metrics.

Numerical optimization in practice

Suppose our problem of interest is to optimize n=10 ingredients (their amount) of a given cake recipe to maximize its average score given by a set of people. All 10 parameters are box-constrained, i.e., their parameter values are constrained to vary between some parameter-specific minimum and maximum values, for the sake of simplicity, between 0 and 1. It is unknown how the scoring system works except that the objective function f_{cake}(x): \mathbb{R}^n \rightarrow \mathbb{R} should be maximized:

\begin{aligned} & \text{maximize} \,\, f_{cake}(x) \\[0.5em] & \text{subject to} \,\, x \in [0, 1]^n \end{aligned}

Gradient information is not available. For the sake of simplicity, let us assume a rather optimistic scenario that f_{cake}(x) is noiseless and there is no uncertainty about f_{cake}(x) for any solution x.

This problem can be tackled by a wide range of algorithms including but not limited to trust-region methods, Bayesian optimization, evolutionary algorithms and random search. The choice of the algorithm is often affected by your previous experience with similar problems and the availability of problem-specific information such as the total budget of function evaluations/calls one is limited to perform. In practice, algorithm selection often consists of trying a few commercial or open source software packages to then stick with one which seems promising. It might take some time but the user is likely to learn new things about optimization algorithms potentially contributing to the open source community, etc. However, relatively simple reformulations of our cake optimization problem can make it hard to find a suitable optimization algorithm/package. Let us list some of them:

Property Specificity
Multi-objective (metric/criteria) Instead of considering a single objective function to evaluate cake recipes, one can consider multiple objectives such as taste, price, safety and robustness. Some solutions will be great at taste but expensive, unhealthy and hard to make; other solutions can be quite tasty, cheap, healthy and easy to make. The optimization algorithm will search for Pareto-optimal solutions which best satisfy all given objectives. This will allow you as the decision-maker to select some of them based on your preferences. Exploring the variety of Pareto-optimal is likely to lead to some useful insights.
Constrained One might be interested to impose some constraints (in addition to the box-constraints) on the ingredients of the cake and their combinations, e.g., to avoid trying a priori infeasible or undesirable solutions. One or several constraint metrics can be considered to be satisfied.
Mixed Our original optimization problem f_{cake}(x): \mathbb{R}^n \rightarrow \mathbb{R} deals with n continuous/real-valued parameters. In practice, some parameters can be discrete or categorical, e.g., to account for selecting among alternative procedures or types of components.
Noisy Evaluating cake recipes can be a very noisy process involving different sources of uncertainties including human factors.
Multi-fidelity (stage) In some cases, candidate solutions can be evaluated with different levels of fidelity where greater levels of fidelity are often more costly. Greater levels of fidelity can correspond to a more precise evaluation of a given solution, e.g., by using a greater pool of experts judging how good a particular cake recipe is. Alternatively, greater levels of fidelity can correspond to a more refined implementation of a particular candidate solution compared to low fidelity levels where high-quality solutions might be unattainable by design.
Parallel Instead of evaluating solutions sequentially one after another, one can request a batch of solutions to be evaluated in parallel and at different evaluation locations. This approach might be more costly in terms of the total number of evaluations but it is often much faster in terms of the wall-clock time.
The presence of only a few of the above-described properties can significantly narrow your choice of software packages and move the focus from solving and understanding your problem of interest to a permanent process of software development and tuning. Indie Solver is designed to account for the above-described problem properties to let you focus on optimal solutions rather than optimization techniques.

Security

Please consult our Privacy Policy for more details.

Indie Solver is a cloud-based service which uses Google Cloud Platform servers. As any internet resource it is exposed to possible hacker attacks. Powerful attackers such as nation state agencies might perform large-scale actions undetectable by individuals. Therefore, one should consider the worst case scenario of having all the data acquired by the attacker. One possible way to mitigate consequences of this scenario is to make sure that the data appears meaningless to the attacker thus reducing potential losses. Consider the following:

Risk Treatments
Password Never use a password you use somewhere else. If you do, consider changing it in the profile page.
Client side software All client side software packages provided by Indie Solver are open source and relatively short. Our client side code communicates with the server mainly by requesting solutions and sending back vectors containing evaluated metrics. Review it to verify that it works as you expect it to work according to the provided documentation. Let us know if you have any concerns.
Personal data During registration we ask you to present your email address which becomes your web login to access the dashboard. You can enter and edit your name and affiliation in the profile page, but providing this information is optional. If you opt-in for a paid subscription plan of our services, you will perform payments with your credit card details. These details will not be processed by us, instead they will be processed by one of the world's largest on-line payment systems called Stripe.
Optimization data As a user you will create a set of optimization problems that you are solving. Description of each problem contains its name, parameters to be varied and metrics to be optimized. Consider the case of cake recipe optimization described above. Indie Solver will generate and store a set of solutions/recipe ingredients and their metric values evaluated on your side. We strongly encourage you to obfuscate all names used for problems, parameters and metrics. If you replace your problem name "cake optimization" by "problem1", parameter name "sugar" by "x1" and metric name "taste" by "objective1", then it will be very difficult for anyone except you to interpret your solutions, e.g., to figure out what is "x1" and what its value corresponds to. We could enforce this method of obfuscation but it would make it harder for users to read and analyze graphs and tables presented in the dashboard. Otherwise, this option should be definitely considered when dealing with sensitive data.

The following action requests are available:

Action Description
Get my data If you would like to receive all the data which corresponds to you account please contact us at contact@indiesolver.com from your email address. We will ask you for a confirmation to make sure it is you who made the request. After your confirmation, we will send you your data.
Remove my account & forget me If you would like your account to be removed and your data to be removed from our servers, please contact us at contact@indiesolver.com from your email address. We will ask you for a confirmation to make sure it is you who made the request. After your confirmation, we will remove your data and your account. If you would like to get your data, you should request it before or during the removal request, but not after the point your data has been removed.

Indie Solver uses the following third party data processors content delivery networks:

Entity name Subprocessing activities
Google Cloud Platform Cloud service provider.
Google Analytics User analytics service provider.
Stripe Payment processing service provider.

Install

You can install Indie Solver client by including its source code referenced in the table below.

Language Description & download link
Client source code for Python 20180825 (click to download)
Client source code for C++, version 20180825 (click to download)
Client source code for MATLAB, version 20180825 (click to download)
Client source code for Java, version 20180825 (click to download)
Client source code for C#, version 20180825 (click to download)
Client source code for Node.js, version 20180825 (click to download)
Client source code for R, version 20180825 (click to download)

Get Started

Client worker is a computer program located on your machine or your cloud account. It uses a thin wrapper code provided by us to communicate with Indie Solver. This communication typically starts by describing an optimization problem you want to solve. Once the problem is described, you can start to request candidate solutions of this problem. After evaluating these solutions on your side by means of modeling or by performing real world experiments, your client worker will communicate evaluated metrics back to Indie Solver. Based on these evaluations, Indie Solver will try to gradually improve the quality of solutions it generates to eventually find optimal solutions of your problem.

Numerous workers can communicate with Indie Solver simultaneously in order to solve the same or different problems, thus enabling asynchronous and parallel optimization.

Let us consider Rosenbrock optimization problem with as few as two variables (for the sake of simplicity):

\begin{aligned} & \text{minimize} \,\, f_{rosen}(x)= 100(x_2 - x_1^2)^2 + (1-x_1)^2 \\[0.5em] & \text{subject to} \,\, x \in [-2, 2]^2 \end{aligned}

Below we show what kind of communication will happen between your client worker and Indie Solver in order to set up and solve the problem. We describe it in JSON format and not yet in some wrapper-specific code, e.g., in Python or C++.

An example of "create_problem" request in JSON format
{
         "request_type": "create_problem",
         "problem_name": "Rosenbrock 2-D",
         "metrics":{
                 "obj1": {"type": "objective", "goal": "minimize"}},
         "parameters":{
                 "x1": {"min": -2.0, "max": 2.0, "init": 0.0, "type": "float", "space": "decision"},
                 "x2": {"min": -2.0,"max": 2.0, "init": 0.0, "type": "float", "space": "decision"}},
         "token": "your secret token available on your profile page"
}

The example request given above creates a new problem called "Rosenbrock 2-D" with an objective function called "obj1" to be minimized. Solutions should be searched in a two-dimensional search space defined by two decision floating point (real-valued / continuous) parameters x_1 and x_2 . Both parameters are bounded to change between -2.0 and 2.0 with 0.0 suggested as their initial/default value (providing this value is optional).

Indie Solver will create "Rosenbrock 2-D" problem for the first time or by rewriting the existing "Rosenbrock 2-D" problem and its data. Providing that syntax of the request is correct, the following reply will be received:

An example of Indie Solver's reply on "create_problem" request
{
         "status": "success",
         "message": "success"
}

Now you can request new candidate solutions (here, 1 solution) of your problem with the following request:

An example of "ask_solutions" request
{
         "request_type": "ask_solutions",
         "request_argument1": "new",
         "request_argument2": 1,
         "problem_name": "Rosenbrock 2-D",
         "token": "your secret token available on your profile page"
}

Indie Solver will reply with a solution which in this particular case corresponds to [0, 0] which is the middle of the defined search space.

An example of Indie Solver's reply on "ask_solutions" request
{
         "status": "success",
         "message": "success",
         "problem_name": "Rosenbrock 2-D",
         "solutions": [{"ID": 1, "parameters": {"x1": 0.0, "x2": 0.0}}]
}

After evaluating this solution on f_{rosen}(x)=f_{rosen}([0,0])=1, the result should be communicated to Indie Solver in the following request:

An example of "tell_metrics" request
{
         "request_type": "tell_metrics",
         "problem_name": "Rosenbrock 2-D",
         "solutions": [{"ID": 1, "metrics": {"obj1": 1.0}}],
         "token": "your secret token available on your profile page"
}

Thus the procedure consists of asking new solutions by sending "ask_solutions" requests, evaluating solutions to obtain metrics values and sending evaluated metrics back to Indie Solver with "tell_metrics" requests. After repeating the ask/tell procedure multiple times, one can numerically approach the optimum f_{rosen}([1,1])=0, e.g., as shown below.

Tutorials

We suggest to start with reproducing one of our tutorial examples, preferably the one which looks most similar to your own optimization problem.

API

Overview

This API enables creating and managing optimization problems.

The table below provides an overview of available types.

Object Description
integer 32 bit integer in the range from -2147483648 to +2147483647.
float 64 bit double-precision floating point number in the range from -1.7*10^{308} to 1.7*10^{308} . NaN and Inf values are not allowed and should be replaced by some values from the range.
string string limited to 50 characters in order to reduce the amount of data and its interpretability (see security section).
metric object describing a metric to be evaluated and optimized (if applied).
metric object describing a metric to be evaluated and optimized (if applied).
metric object describing a metric to be evaluated and optimized (if applied).
parameter object describing a parameter to be varied during optimization.

Enumerated types

Enumerated type is a data type consisting of a set of named values called enumerators (enums) of the type. While in the context of this page enumerated type can be viewed as categorical type, we intentionally use these two terms for two different contexts: enumerated types for describes API syntax and categorical types for describing parameter types of optimization problem.

Enums metric_type

Field value Description
"objective" By selecting this metric type value, a) you set the metric to be interpreted as an objective function to be optimized, and b) you are required to set an additional field called "goal" which defines whether the objective function is to be minimized or maximized.
"constraint" By selecting this metric type value, you set the metric to be interpreted as a constraint to be satisfied. The constraint is satisfied when its value is lesser or equal to 0. Otherwise, the constraint is not satisfied and its positive value will be attempted to be minimized.
"cost" By selecting this metric type value, you set the metric to be interpreted as a cost function. When values of other optimized metrics are equal, solutions with smaller cost values are preferred by the search algorithm. Example: time to produce a solution.
"observation" By selecting this metric type value, you set the metric to be interpreted as an observation. The values given for this metric will not affect optimization results in any way. Instead, observations can be used to support your decision-making.

Enums goal_type

Field value Description
"minimize" By selecting this metric goal type, you tell Indie Solver to minimize the corresponding objective function f(x) . Note that minimization of f(x) is equivalent to maximization of g(x)=-f(x) .
"maximize" By selecting this metric goal type, you tell Indie Solver to maximize the corresponding objective function f(x) . Note that maximization of f(x) is equivalent to minimization of g(x)=-f(x) .

Enums parameter_type

Field value Description
"float" Sets parameter type to float . The following parameter fields become required: "min", "max", "init".
"integer" Sets parameter type to integer . The following parameter fields become required: "min", "max", "init", "step".
"categorical" Sets parameter type to categorical so that parameter value is limited to one of the parameter values defined by "domain". The following parameter field becomes required: "domain".

Enums space_type

Field value Description
"decision" By selecting this parameter space type, you tell Indie Solver that this parameter is to be optimized and that there is no knowledge whether some parameter values lead to more costly evaluations.
"cost" By selecting this parameter space type, you tell Indie Solver that this parameter is to be optimized and that there is some knowledge about possible effects of parameter values on evaluation costs. More specifically, greater values of cost parameters lead to greater evaluation costs. The dependency is not necessarily linear.

Enums reply_status

Field value Description
"success" The request has been processed successfully.
"error" An error has been detected while processing the request.

Enums solution_type

Field value Description
"new" By selecting this solution type for "request_argument1" when asking for solutions in ask_solutions , you tell Indie Solver to send back new solutions More specifically, the number of new/generated solutions is set by "request_argument2" of ask_solutions .
"pending" By selecting this solution type when asking for solutions in ask_solutions , you tell Indie Solver to send back pending solutions of the current problem (see "problem" field), i.e., already asked solutions those metric values are not told to Indie Solver.

Objects

Object metric

JSON representation
{
         "type": enum( metric_type ),
         "goal": enum( goal_type )
}
Field Description
type enum(metric_type )
Required.
goal enum(goal_type )
Required only if "type" is set to "objective".

Object parameter

JSON representation
{
         "type": enum( parameter_type ),
         "space": enum( space_type ),
         "min": float or integer,
         "max": float or integer,
         "init": float or integer or string,
         "domain": [string]
}
Field Description
type enum(parameter_type )
Required.
space enum(parameter_space )
Required.
min float or integer
Required only if "type" is set to "float" or "integer" (then the type of this field is determined accordingly), otherwise it is not used. Defines the minimum value of the search range.
max float or integer
Required only if "type" is set to "float" or "integer" (then the type of this field is determined accordingly), otherwise it is not used. Defines the maximum value of the search range.
init float or integer or string
Required only if "type" is set to "float" or "integer" or "categorical" (then the type of this field is determined accordingly). Defines the initial and/or default value suggested to be considered by the optimizer.
domain list( string )
Required only if "type" is set to "categorical", otherwise it is not used. Defines the list of strings which correspond to categorical values of the parameter.

Object solution

JSON representation
{
         "ID": integer,
         "metrics":{
                 string: float,
                 . . . ,
                 string: float},
         "parameters":{
                 string: float or integer or string,
                 . . . ,
                 string: float or integer or string}
}
Field Description
ID integer
Required. The value is constrained to be positive.
metrics object
Required only when the solution is communicated as a reply to ask_solutions. Fields of the object are metrics names and their field values are floats.
parameters object
Required only when the solution is communicated within tell_metrics. Fields names of the object correspond to parameter names and fields values correspond to parameter types, i.e., float, integer or string (as a categorical value).

Object reply

JSON representation
{
         "status": enum( reply_status ),
         "message": string,
         "solutions": [ solution ],
         "problems": [ string ]
}
Field Description
status enum(reply_status )
Required.
message string
Required.
solutions list(solution )
Required only when the reply is a result of ask_solutions. This list contains solutions with their corresponding parameter values.
problems list( string )
Required only when the reply is a result of ask_problems. This list contains names of existing problems.

Methods

The table below shows available methods denoted by possible field values of request_type . For each of these methods, Indie Solver will send a reply message in JSON format.

Enums request_type

Field value Description
"create_problem" Creates a new problem or rewrites an existing problem.
"ask_problem_description" Returns a description of an existing problem.
"ask_solutions" Returns a description of an existing problem.
"tell_metrics" Sends Indie Solver evaluated metric values for a given set of solutions.
"ask_problems" Returns a list of problem names.

Method create_problem

JSON representation
{
         "request_type": enum( request_type ),
         "problem_name": string,
         "metrics":{
                 string: object( metric ),
                 . . . ,
                 string: object( metric )},
         "parameters":{
                 string: object( parameter ),
                 . . . ,
                 string: object( parameter )},
         "token": string
}
Field Description
request_type enum(request_type )
Required and fixed to "create_problem".
problem_name string
Required and constrained to include only symbols from the following list: [abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890!_-+].
It is strongly encouraged to obfuscate problem names, e.g., as "problem_name": "problem1".
metrics object
Required and constrained to include at least one field of object(Metric).
parameters object
Required and constrained to include at least one field of object(Parameter).
token string
Required and must correspond to the token of your account given on your profile page.
Example: "hb6950a6684c1bgfhf28blbha"

Method ask_problem_description

JSON representation
{
         "request_type": enum( request_type ),
         "problem_name": string,
         "token": string
}
Field Description
request_type enum(request_type )
Required and fixed to "ask_problem_description".
problem_name string
Since the method asks to return description of a problem which already exists, exact matching of problem names is required.
token string
Required and must correspond to the token of your account given on your profile page.
Example: "hb6950a6684c1bgfhf28blbha"

Method ask_solutions

JSON representation
{
         "request_type": enum( request_type ),
         "request_argument1": enum( solution_type ),
         "request_argument2": integer,
         "problem_name": string,
         "token": string
}
Field Description
request_type enum(request_type )
Required and fixed to "ask_solutions".
request_argument1 enum(solution_type )
Required. It defines whether "new" or "pending" solutions should be returned.
request_argument2 integer
Required only when "request_argument1"="new". This field is constrained to positive integer values and defines how many new solutions should be returned.
problem_name string
Required only when "request_argument1"="new" or "request_argument1"="pending". The value is used to determine for which problem the solutions are requested.
token string
Required and must correspond to the token of your account given on your profile page.
Example: "hb6950a6684c1bgfhf28blbha"

Method tell_metrics

JSON representation
{
         "request_type": enum( request_type ),
         "problem_name": string,
         "solutions": [ solution ],
         "token": string
}
Field Description
request_type enum(request_type )
Required and fixed to "ask_solutions".
problem_name string
Required.
solutions list(solution )
Required. This list contains solutions with their corresponding evaluated metrics values.
token string
Required and must correspond to the token of your account given on your profile page.
Example: "hb6950a6684c1bgfhf28blbha"

Method ask_problems

JSON representation
{
         "request_type": enum( request_type ),
         "token": string
}
Field Description
request_type enum(request_type )
Required and fixed to "ask_problems".
token string
Required and must correspond to the token of your account given on your profile page.
Example: "hb6950a6684c1bgfhf28blbha"