respy.parallelization

This module contains the code to control parallel execution.

Module Contents

Functions

parallelize_across_dense_dimensions([func, n_jobs])	Parallelizes decorated function across dense state space dimensions.
split_and_combine_df(func)	Split the data across dense indices, run a function, and combine again.
_infer_dense_keys_from_arguments(args, kwargs)	Infer the dense indices from the arguments.
_is_dictionary_with_integer_keys(candidate)	Infer whether the argument is a dictionary with integer keys.
_broadcast_arguments(args, kwargs, dense_keys)	Broadcast arguments to dense state space dimensions.
_is_dense_dictionary_argument(argument, dense_keys)	Check whether all keys of the dictionary argument are also dense indices.
_split_dataframe(df)	Split a DataFrame by creating groups of the same values for the dense dims.

respy.parallelization.parallelize_across_dense_dimensions(func=None, *, n_jobs=1)

Parallelizes decorated function across dense state space dimensions.

Parallelization is only possible if the decorated function has no side-effects to other dense dimensions. This might be true for different levels. E.g. respy.solve._create_choice_rewards() can be directly parallelized. respy.solve._solve_with_backward_induction() cannot be directly parallelized because the continuation values for one dense dimension will become important for others if we implement exogenous processes. Thus, parallelize across periods.

If applied to a function, the decorator recognizes if the model or state space contains dense dimensions likes types or observables. Then, it splits the operation across dense dimensions by patching the attribute access such that each sub state space can only access its attributes.

The decorator can be applied to functions without trailing parentheses. At the same time, the * prohibits to use the decorator with positional arguments.

respy.parallelization.split_and_combine_df(func)

Split the data across dense indices, run a function, and combine again.

respy.parallelization._infer_dense_keys_from_arguments(args, kwargs)

Infer the dense indices from the arguments.

Dense indices can be found in dictionaries with only integer keys.

This function uses the intersection of all dense indices from the arguments. Since the simulated data or data for the likelihood might not comprise all dense dimensions, we might need to discard some indices.

respy.parallelization._is_dictionary_with_integer_keys(candidate)

Infer whether the argument is a dictionary with integer keys.

respy.parallelization._broadcast_arguments(args, kwargs, dense_keys)

Broadcast arguments to dense state space dimensions.

respy.parallelization._is_dense_dictionary_argument(argument, dense_keys)

Check whether all keys of the dictionary argument are also dense indices.

We cannot check whether all dense indices are in the argument because splitted_df in split_and_combine_df() may not cover all dense combinations.

respy.parallelization._split_dataframe(df)

Split a DataFrame by creating groups of the same values for the dense dims.