Python & R API¶

The application programming interface (API) for MESSAGEix model developers is implemented in Python:

ixmp package
message_ix package
Model classes
Utility methods
Testing utilities

Support for R usage of the core classes is provided through the reticulate package. For instance:

> library(reticulate)
> ixmp <- import('ixmp')
> message_ix <- import('message_ix')
> mp <- ixmp$Platform(...)
> scen <- message_ix$Scenario(mp, ...)

`ixmp` package ¶

ixmp provides three classes. These are fully described by the ixmp documentation, which is cross-linked from many places in the MESSAGEix documentation.

`Platform`(*args[, backend])	Instance of the modeling platform.
`TimeSeries`(mp, model, scenario[, version, …])	Collection of data in time series format.
`Scenario`(mp, model, scenario[, version, …])	Collection of model-related data.

ixmp also provides some utility classes and methods:

`ixmp.config.Config`([read])	Configuration for ixmp.
`ixmp.model.MODELS`	Mapping from names to available models.
`ixmp.model.get_model`(name, **model_options)	Return a model for name (or the default) with model_options.
`ixmp.testing.make_dantzig`

`message_ix` package ¶

MESSAGEix models are created using the message_ix.Scenario class. Several utility methods are also provided in the module message_ix.utils.

class message_ix.Scenario(mp, model, scenario=None, version=None, annotation=None, cache=False)¶

Bases: ixmp.core.Scenario

MESSAGEix Scenario.

See ixmp.TimeSeries for the meaning of arguments mp, model, scenario, version, and annotation; ixmp.Scenario for the meaning of cache. The scheme of a newly-created Scenario is always ‘MESSAGE’.

This class extends ixmp.Scenario and ixmp.TimeSeries and inherits all their methods. Documentation of these inherited methods is included here for convenience. message_ix.Scenario defines additional methods specific to MESSAGEix:

`add_cat`(name, cat, keys[, is_unique])	Map elements from keys to category cat within set name.
`add_horizon`(data)	Add sets related to temporal dimensions of the model.
`add_spatial_sets`(data)	Add sets related to spatial dimensions of the model.
`cat`(name, cat)	Return a list of all set elements mapped to a category.
`cat_list`(name)	Return a list of all categories for a mapping set.
`firstmodelyear`	The first model year of the scenario.
`read_excel`(fname[, add_units, commit_steps])	Read Excel file data and load into the scenario.
`rename`(name, mapping[, keep])	Rename an element in a set
`to_excel`(fname)	Save a scenario as an Excel file.
`vintage_and_active_years`([ya_args, in_horizon])	Return sets of vintage and active years for use in data input.
`years_active`(node, tec, yr_vtg)	Return years in which tec of yr_vtg can be active in node.

add_cat(name, cat, keys, is_unique=False)¶

Map elements from keys to category cat within set name.

Parameters:	name (str) – Name of the set. cat (str) – Name of the category. keys (str or list of str) – Element keys to be added to the category mapping. is_unique (bool, optional) – If True, then cat must have only one element. An exception is raised if cat already has an element, or if `len(keys) > 1`.

add_geodata(df)¶

Add geodata (layers) to the TimeSeries.

Parameters:

df (pandas.DataFrame) –

Data to add. df must have the following columns:

region
variable
time
unit
year
value
meta

add_horizon(data)¶

Add sets related to temporal dimensions of the model.

Parameters:	data (dict-like) – Year sets. “year” is a required key. “firstmodelyear” is optional; if not provided, the first element of “year” is used.

Examples

>>> s = message_ix.Scenario()
>>> s.add_horizon({'year': [2010, 2020]})
>>> s.add_horizon({'year': [2010, 2020], 'firstmodelyear': 2020})

add_par(name, key_or_data=None, value=None, unit=None, comment=None, key=None, val=None)¶

Set the values of a parameter.

Parameters:	name (str) – Name of the parameter. key_or_data (str or iterable of str or range or dict or pandas.DataFrame) – Element(s) to be added. value (numeric or iterable of numeric, optional) – Values. unit (str or iterable of str, optional) – Unit symbols. comment (str or iterable of str, optional) – Comment(s) for the added values.

add_set(name, key, comment=None)¶

Add elements to an existing set.

Parameters:	name (str) – Name of the set. key (str or iterable of str or dict or `pandas.DataFrame`) – Element(s) to be added. If name exists, the elements are appended to existing elements. comment (str or iterable of str, optional) – Comment describing the element(s). If given, there must be the same number of comments as elements.
Raises:	`KeyError` – If the set name does not exist. `init_set()` must be called before `add_set()`. `ValueError` – For invalid forms or combinations of key and comment.

add_spatial_sets(data)¶

Add sets related to spatial dimensions of the model.

Parameters:

data (dict) –

Mapping of level → member. Each member may be:

A single label for elements.
An iterable of labels for elements.
A recursive dict following the same convention, defining sub-levels and their members.

Examples

>>> s = message_ix.Scenario()
>>> s.add_spatial_sets({'country': 'Austria'})
>>> s.add_spatial_sets({'country': ['Austria', 'Germany']})
>>> s.add_spatial_sets({'country': {
...     'Austria': {'state': ['Vienna', 'Lower Austria']}}})

add_timeseries(df, meta=False)¶

Add data to the TimeSeries.

Parameters:

df (pandas.DataFrame) –
Data to add. df must have the following columns:
- region or node
- variable
- unit
Additional column names may be either of:
- year and value—long, or ‘tabular’, format.
- one or more specific years—wide, or ‘IAMC’ format.
meta (bool, optional) – If True, store df as metadata. Metadata is treated specially when Scenario.clone() is called for Scenarios created with scheme='MESSAGE'.

cat(name, cat)¶

Return a list of all set elements mapped to a category.

Parameters:	name (str) – Name of the set. cat (str) – Name of the category.
Returns:
Return type:	list of str

cat_list(name)¶

Return a list of all categories for a mapping set.

Parameters:	name (str) – Name of the set.

change_scalar(name, val, unit, comment=None)¶

Set the value and unit of a scalar.

Parameters:	name (str) – Name of the scalar. val (number) – New value of the scalar. unit (str) – New unit of the scalar. comment (str, optional) – Description of the change.

check_out(timeseries_only=False)¶: Check out the TimeSeries for modification.

clear_cache(name=None, ix_type=None)¶

clear the Python cache of item elements

Parameters:	name (str, optional) – item name (None clears entire Python cache) ix_type (str, optional) – type of item (if provided, cache clearing is faster)

clone(*args, **kwargs)¶

Clone the current scenario and return the clone.

See ixmp.Scenario.clone() for other parameters.

Parameters:

keep_solution (bool, optional) – If True, include all timeseries data and the solution (vars and equs) from the source scenario in the clone. Otherwise, only timeseries data marked as meta=True (see TimeSeries.add_timeseries()) or prior to first_model_year (see TimeSeries.add_timeseries()) are cloned.
shift_first_model_year (int, optional) – If given, the values of the solution are transfered to parameters historical_*, parameter resource_volume is updated, and the first_model_year is shifted. The solution is then discarded, see TimeSeries.remove_solution().

commit(comment)¶

Commit all changed data to the database.

If the TimeSeries was newly created (with version='new'), version is updated with a new version number assigned by the backend. Otherwise, commit() does not change the version.

Parameters:	comment (str) – Description of the changes being committed.

discard_changes()¶: Discard all changes and reload from the database.

equ(name, filters=None, **kwargs)¶

return a dataframe of (filtered) elements for a specific equation

Parameters:	name (str) – name of the equation filters (dict) – index names mapped list of index set elements

equ_list()¶: List all defined equations.

firstmodelyear¶

The first model year of the scenario.

Returns:
Return type:	int

get_geodata()¶

Fetch geodata and return it as dataframe.

Returns:	Specified data.
Return type:	`pandas.DataFrame`

get_meta(name=None)¶

get scenario metadata

Parameters:	name (str, optional) – metadata attribute name

has_equ(name)¶: check whether the scenario has an equation with that name

has_par(name)¶: check whether the scenario has a parameter with that name

has_set(name)¶: Check whether the scenario has a set name.

has_solution()¶

Return True if the Scenario has been solved.

If has_solution() == True, model solution data exists in the db.

has_var(name)¶: check whether the scenario has a variable with that name

idx_names(name)¶

return the list of index names for an item (set, par, var, equ)

Parameters:	name (str) – name of the item

idx_sets(name)¶

Return the list of index sets for an item (set, par, var, equ)

Parameters:	name (str) – name of the item

init_equ(name, idx_sets=None, idx_names=None)¶

Initialize a new equation.

Parameters:	name (str) – name of the item idx_sets (list of str) – index set list idx_names (list of str, optional) – index name list

init_par(name, idx_sets, idx_names=None)¶

Initialize a new parameter.

Parameters:	name (str) – Name of the parameter. idx_sets (list of str) – Names of sets that index this parameter. idx_names (list of str, optional) – Names of the dimensions indexed by idx_sets.

init_scalar(name, val, unit, comment=None)¶

Initialize a new scalar.

Parameters:	name (str) – Name of the scalar val (number) – Initial value of the scalar. unit (str) – Unit of the scalar. comment (str, optional) – Description of the scalar.

init_set(name, idx_sets=None, idx_names=None)¶

Initialize a new set.

Parameters:	name (str) – Name of the set. idx_sets (list of str, optional) – Names of other sets that index this set. idx_names (list of str, optional) – Names of the dimensions indexed by idx_sets.
Raises:	`jpype.JavaException` – If the set (or another object with the same name) already exists.

init_var(name, idx_sets=None, idx_names=None)¶

initialize a new variable in the scenario

Parameters:	name (str) – name of the item idx_sets (list of str) – index set list idx_names (list of str, optional) – index name list

is_default()¶: Return True if the version is the default version.

last_update()¶: get the timestamp of the last update/edit of this TimeSeries

load_scenario_data()¶

Load all Scenario data into memory.

Raises:	`ValueError` – If the Scenario was instantiated with `cache=False`.

par(name, filters=None, **kwargs)¶

return a dataframe of (filtered) elements for a specific parameter

Parameters:	name (str) – name of the parameter filters (dict) – index names mapped list of index set elements

par_list()¶: List all defined parameters.

preload_timeseries()¶: Preload timeseries data to in-memory cache. Useful for bulk updates.

read_excel(fname, add_units=False, commit_steps=False)¶

Read Excel file data and load into the scenario.

Parameters:	fname (string) – path to file add_units (bool) – add missing units, if any, to the platform instance. default: False commit_steps (bool) – commit changes after every data addition. default: False

remove_geodata(df)¶

Remove geodata from the TimeSeries instance.

Parameters:

df (pandas.DataFrame) –

Data to remove. df must have the following columns:

region
variable
unit
time
year

remove_par(name, key=None)¶

Remove parameter values or an entire parameter.

Parameters:	name (str) – Name of the parameter. key (dataframe or key list or concatenated string, optional) – elements to be removed

remove_set(name, key=None)¶

delete a set from the scenario or remove an element from a set (if key is specified)

Parameters:	name (str) – name of the set key (dataframe or key list or concatenated string) – elements to be removed

remove_solution(first_model_year=None)¶

Remove the solution from the scenario

This function removes the solution (variables and equations) and timeseries data marked as meta=False from the scenario (see TimeSeries.add_timeseries()).

Parameters:	first_model_year (int, optional) – If given, timeseries data marked as meta=False is removed only for years from first_model_year onwards.
Raises:	`ValueError` – If Scenario has no solution or if first_model_year is not int.

remove_timeseries(df)¶

Remove timeseries data from the TimeSeries instance.

Parameters:

df (pandas.DataFrame) –

Data to remove. df must have the following columns:

region or node
variable
unit
year

rename(name, mapping, keep=False)¶

Rename an element in a set

Parameters:	name (str) – name of the set to change (e.g., ‘technology’) mapping (str) – mapping of old (current) to new set element names keep (bool, optional, default: False) – keep the old values in the model

run_id()¶: get the run id of this TimeSeries

scalar(name)¶

Return the value and unit of a scalar.

Parameters:	name (str) – Name of the scalar.
Returns:	{‘value’
Return type:	value, ‘unit’: unit}

set(name, filters=None, **kwargs)¶

Return the (filtered) elements of a set.

Parameters:	name (str) – Name of the set. filters (dict) – Mapping of dimension_name → elements, where dimension_name is one of the idx_names given when the set was initialized (see `init_set()`), and elements is an iterable of labels to include in the return value.
Returns:
Return type:	pandas.DataFrame

set_as_default()¶: Set the current version as the default.

set_list()¶: List all defined sets.

set_meta(name, value)¶

set scenario metadata

Parameters:	name (str) – metadata attribute name value (str or number or bool) – metadata attribute value

solve(model='MESSAGE', solve_options={}, **kwargs)¶

Solve MESSAGE or MESSAGE-MACRO for the Scenario.

By default, ixmp.Scenario.solve() is called with ‘MESSAGE’ as the model argument. model may also be overwritten, e.g.:

>>> s.solve(model='MESSAGE-MACRO')

Parameters:	model (str, optional) – Type of model to solve, e.g. ‘MESSAGE’ or ‘MESSAGE-MACRO’. solve_options (dict (str -> str), optional) – Name to value mapping to use for GAMS CPLEX solver options file. See `MESSAGE` and `DEFAULT_CPLEX_OPTIONS`. kwargs – Many other options control the execution of the underlying GAMS code; see `GAMSModel`.

timeseries(region=None, variable=None, unit=None, year=None, iamc=False)¶

Retrieve TimeSeries data.

Parameters:	iamc (bool, default: False) – Return data in wide/’IAMC’ format. If `False`, return data in long/’tabular’ format; see `add_timeseries()`. region (str or list of strings) – Regions to include in returned data. variable (str or list of strings) – Variables to include in returned data. unit (str or list of strings) – Units to include in returned data. year (str, int or list of strings or integers) – Years to include in returned data.
Returns:	Specified data.
Return type:	`pandas.DataFrame`

to_excel(fname)¶

Save a scenario as an Excel file. NOTE: Cannot export solution currently (only model data) due to limitations in excel sheet names (cannot have multiple sheet names which are identical except for upper/lower case).

Parameters:	fname (string) – path to file

var(name, filters=None, **kwargs)¶

return a dataframe of (filtered) elements for a specific variable

Parameters:	name (str) – name of the variable filters (dict) – index names mapped list of index set elements

var_list()¶: List all defined variables.

vintage_and_active_years(ya_args=None, in_horizon=True)¶

Return sets of vintage and active years for use in data input.

For a valid pair (year_vtg, year_act), the following conditions are satisfied:

Both the vintage year (year_vtg) and active year (year_act) are in the model’s year set.
year_vtg <= year_act.
year_act <= the model’s first year or year_act is in the smaller subset ixmp.Scenario.years_active() for the given ya_args.

Parameters:	ya_args (tuple of (node, technology, year_vtg), optional) – Arguments to `ixmp.Scenario.years_active()`. in_horizon (bool, optional) – Restrict years returned to be within the current model horizon.
Returns:	with columns, “year_vtg” and “year_act”, in which each row is a valid pair.
Return type:	pandas.DataFrame

years_active(node, tec, yr_vtg)¶

Return years in which tec of yr_vtg can be active in node.

Parameters:	node (str) – Node name. tec (str) – Technology name. yr_vtg (str) – Vintage year.
Returns:
Return type:	list of int

Model classes ¶

message_ix.models.DEFAULT_CPLEX_OPTIONS = {'advind': 0, 'epopt': 1e-06, 'lpmethod': 2, 'threads': 4}¶: Solver options used by message_ix.Scenario.solve(). These configure the GAMS CPLEX solver (or another solver, if selected); see the solver documentation for possible values.

class message_ix.models.MESSAGE(name=None, **model_options)¶

Bases: ixmp.model.gams.GAMSModel

The MESSAGE Python class encapsulates the GAMS code for the core MESSAGE mathematical formulation. The model_options arguments are received from Scenario.solve(), and—except for solve_options—are passed on to the parent class GAMSModel; see there for a full list of options.

name = 'MESSAGE'¶

defaults = dict(...)¶

Default model options. The paths to MESSAGE GAMS source files use the MODEL_PATH configuration setting. MODEL_PATH, in turn, defaults to “message_ix/model” inside the directory where message_ix is installed.

Key	Value
MESSAGE defaults
model_file	`'{MODEL_PATH}/{model_name}_run.gms'`
in_file	`'{MODEL_PATH}/data/MsgData_{case}.gdx'`
out_file	`'{MODEL_PATH}/output/MsgOutput_{case}.gdx'`
solve_args	`['--in="{in_file}"', '--out="{out_file}"', '--iter="{MODEL_PATH}/output/MsgIterationReport_{case}.gdx"']`
Inherited from `GAMSModel`
case	`'{scenario.model}_{scenario.scenario}'`
gams_args	`['LogOption=4']`
check_solution	`True`
comment	`None`
equ_list	`None`
var_list	`None`

classmethod read_version()¶: Retrieve MESSAGE version string from version.gms.

run(scenario)¶

Execute the model.

MESSAGE creates a file named cplex.opt in the model directory, containing the options in DEFAULT_CPLEX_OPTIONS, or any overrides passed to solve().

class message_ix.models.MESSAGE_MACRO(name=None, **model_options)¶

Bases: message_ix.models.MESSAGE

name = 'MESSAGE-MACRO'¶

Utility methods ¶

message_ix.utils.make_df(base, **kwargs)¶

Extend or overwrite base with new values from kwargs.

Parameters:	base (dict, `pandas.Series`, or `pandas.DataFrame`) – Existing dataset to append to. *kwargs – Additional values to append to base*.
Returns:	base modified with kwargs.
Return type:	`pandas.DataFrame`

Examples

Scalar values in base or kwargs are broadcast. The number of rows in the returned pandas.DataFrame equals the length of the longest item in either argument.

>>> base = {'foo': 'bar'}
>>> make_df(base, baz=[42, 43, 44])
    foo     baz
0   bar     42
1   bar     43
2   bar     44

message_ix.utils.make_ts(df, time_col, value_col, metadata={})¶: The function groups the dataframe by the year specified in year_col_name (year_act Vs. year_vtg). It then reshapes the dataframe df to reseble the timeseries requirements: sets the unit, the variable name, and the value column to the one specified in value_col_name. it further drops all all additional columns.

message_ix.utils.matching_rows(df, row, match_columns=[])¶: The function finds all the columns in a dataframe that are specified in the match columns list.

message_ix.utils.multiply_df(df1, column1, df2, column2)¶: The function merges dataframe df1 with df2 and multiplies column1 with column2. The function returns the new merged dataframe with the result of the muliplication in the column ‘product’.

Testing utilities ¶

message_ix.testing.make_dantzig(mp, solve=False, multi_year=False)¶

Return an message_ix.Scenario for Dantzig’s canning problem.

Parameters:	mp (ixmp.Platform) – Platform on which to create the scenario. solve (bool, optional) – If True, the scenario is solved. multi_year (bool, optional) – If True, the scenario has years 1963–1965 inclusive. Otherwise, the scenario has the single year 1963.

message_ix.testing.make_westeros(mp, emissions=False, solve=False)¶

Return an message_ix.Scenario for the Westeros model.

This is the same model used in the westeros_baseline.ipynb tutorial.

Parameters:	mp (ixmp.Platform) – Platform on which to create the scenario. emissions (bool, optional) – If True, the `emissions_factor` parameter is also populated for CO2. solve (bool, optional) – If True, the scenario is solved.

Python & R API¶

ixmp package¶

message_ix package¶

Model classes¶

Utility methods¶

Testing utilities¶

`ixmp` package ¶

`message_ix` package ¶

Model classes ¶

Utility methods ¶

Testing utilities ¶