View and download the notebook here!
Specifying a Model#
One of the core features of respy include the flexible modeling capabilities. The guide on example models showcases a collection of economic models that have already been implemented. They can be accessed freely.
However, respy can also be used to implement models from scratch. This guide illustrates how to translate an economic model and underlying mathematical relations to the core objects in respy: params and options. As a guiding example we will follow the seminal work of Keane and Wolpin (1994) and replicate their dynamic discrete choice model of schooling and occupational choice. Insights carry over to the conceptually close model used by Keane and Wolpin (1997).
Note: Only models of the Eckstein-Keane-Wolpin (EKW) class are implementable in respy. You can find further information about this modeling framework in the explanations section of this documentation.
Components to modeling#
See the article in the explanations section linked below to find information on the exact model specification of Keane and Wolpin (1994).
How can we map the equations from the model into respy to construct a discrete choice dynamic programming model that allows us to estimate the structural parameters?
A model in respy is defined by two components:
The
paramsDataFrame, where model parameters reside. It should be specified as a pandas.DataFrame.The
optionswhich specify the settings for the model solution and further restrictions on the model structure.optionsare defined in a Pythondictionary. Examples of components that enter the options include the number of periods, type of numerical integration, unfeasible states, etc.
In the next steps, we will examine these two components in detail to illustrate how they mirror the model outlined above. Since the model of Keane and Wolpin (1994) is already implemented, we can simply load it into memory.
[1]:
import respy as rp
[2]:
params, options, data = rp.get_example_model("kw_94_one")
Note that when you specify these objects yourself, doing so in separate files might facilitate your workflow. For example, params could be loaded from a .csv-file and options from a .yaml-file.
Specifying the params#
We first inspect the params DataFrame. It contains all the parameters that enter the structural model. Usually, these parameters will be estimable, but this is not mandatory. For instance, a specified shock distribution may guide the model but be exogenously set. The params DataFrame may also contain auxiliary parameters that aid simulation but are not directly related to the model. respy allows copious freedom in designing reward functions and naming parameters. However, certain
rules need to be accounted for to allow respy to process a model correctly. Below, we discuss each parameter group of our exemplary params DataFrame to outline how parameters can be specified.
[3]:
params
[3]:
| value | comment | ||
|---|---|---|---|
| category | name | ||
| delta | delta | 0.9500 | discount factor |
| wage_a | constant | 9.2100 | log of rental price |
| exp_edu | 0.0380 | return to an additional year of schooling | |
| exp_a | 0.0330 | return to same sector experience | |
| exp_a_square | -0.0005 | return to same sector, quadratic experience | |
| exp_b | 0.0000 | return to other sector experience | |
| exp_b_square | 0.0000 | return to other sector, quadratic experience | |
| wage_b | constant | 8.4800 | log of rental price |
| exp_edu | 0.0700 | return to an additional year of schooling | |
| exp_b | 0.0670 | return to same sector experience | |
| exp_b_square | -0.0010 | return to same sector, quadratic experience | |
| exp_a | 0.0220 | return to other sector experience | |
| exp_a_square | -0.0005 | return to other sector, quadratic experience | |
| nonpec_edu | constant | 0.0000 | constant reward for choosing education |
| at_least_twelve_exp_edu | 0.0000 | reward for going to college (tuition, etc.) | |
| not_edu_last_period | -4000.0000 | reward for going back to school | |
| nonpec_home | constant | 17750.0000 | constant reward of non-market alternative |
| shocks_sdcorr | sd_a | 0.2000 | Element 1,1 of standard-deviation/correlation ... |
| sd_b | 0.2500 | Element 2,2 of standard-deviation/correlation ... | |
| sd_edu | 1500.0000 | Element 3,3 of standard-deviation/correlation ... | |
| sd_home | 1500.0000 | Element 4,4 of standard-deviation/correlation ... | |
| corr_b_a | 0.0000 | Element 2,1 of standard-deviation/correlation ... | |
| corr_edu_a | 0.0000 | Element 3,1 of standard-deviation/correlation ... | |
| corr_edu_b | 0.0000 | Element 3,2 of standard-deviation/correlation ... | |
| corr_home_a | 0.0000 | Element 4,1 of standard-deviation/correlation ... | |
| corr_home_b | 0.0000 | Element 4,2 of standard-deviation/correlation ... | |
| corr_home_edu | 0.0000 | Element 4,3 of standard-deviation/correlation ... | |
| lagged_choice_1_edu | probability | 1.0000 | Probability that the first lagged choice is ed... |
| initial_exp_edu_10 | probability | 1.0000 | Probability that the initial level of educatio... |
| maximum_exp | edu | 20.0000 | Maximum level of experience for education (opt... |
Index stucture#
The params DataFrame needs to abide to a specific index structure:
Index: The DataFrame has a MultiIndex with two levels. The levels have to be named
categoryandname. Categories need to be unique. Names may be repeated but never within the same category. This ensures that each parameter is uniquely identfied in theparamsDataFrame.Columns:The parameter value needs to be saved in a column called
value. TheparamsDataFrame may contain other columns like the comment column above. They do not influence the model. This can also be useful for parameter estimation where information like bounds may need to be specified as additional columns.
Discounting#
In respy the discount factor has a pre-defined and unmutable name: delta.
[4]:
params.loc["delta"]
[4]:
| value | comment | |
|---|---|---|
| name | ||
| delta | 0.95 | discount factor |
respy also supports hyperbolic discounting. You can implement it in your model by adding a category and name called beta to your parameter vector.
Choice Rewards#
The structural model consists of two building blocks: states and choices. Choices in general can have two types of rewards:
pecuniary rewards, e.g. wages, with corresponding
category:wage_{choice}.non-pecuniary rewards, e.g. intrinsic value of education, with corresponding
category:nonpec_{choice}.
Choices can be named freely but it is important to use the appropriate prefixes so respy can process the model accordingly. In our example above, choices have either exclusively pecuniary rewards (occupation A and B) or non-pecuniary rewards (education and home) but respy also allows for combinations of both types to define reward functions. Each parameter in params then corresponds to a parameter in the reward functions.
Example: Returns to Occupation A#
Take for example the reward function for choosing to work in occupation A:
We can directly map the params DataFrame to the equation. All parameters are saved under the category of wage_a. The pecuniary reward associated with working in occupation A, wage_a is determined by state-specific returns. The index name collects all covariates where value captures the associated return. The state-variables and returns are mapped to the entries in category wage_a according to the following table:
Covariate |
|
Return |
|
|---|---|---|---|
\(1\) |
|
\(\alpha_{10}\) |
\(9.2100\) |
\(s_{t}\) |
|
\(\alpha_{11}\) |
\(0.0380\) |
\(x_{1t}\) |
|
\(\alpha_{12}\) |
\(0.0330\) |
\(x_{1t}^2\) |
|
\(\alpha_{13}\) |
\(-0.0005\) |
\(x_{2t}\) |
|
\(\alpha_{14}\) |
\(0.0000\) |
\(x_{2t}^2\) |
|
\(\alpha_{15}\) |
\(0.0000\) |
We can imagine the equation to be written as
The choice-specific shock that is also part of this equation will be discussed in more detail below.
Note: The prefix exp_ is a special name in respy and must be complemented by the name of a choice. Parameters with this prefix indicate the return to experience in a certain choice alternative. Conversely, the names constant and exp_{choice}_square do not have this pre-specified structure. Instead, they require further user input in the options dictionary to be properly specified.
Experience accumulation is a central component of EKW models and thus an important feature of respy. You will notice that exp_home does not appear in our params DataFrame. This is a direct result from our model equations: Individuals do not accumulate any experience while being at home. Omitted experience parameters indicate that experience accumulation for this alternative is not a model component. Notably, alternatives with a wage component automatically account for experience
accumulation.
Shocks#
For each choice reward, idiosyncratic and serially uncorrelated shocks alter the respective return. Those alternative-specific shocks are specified jointly in category shocks_sdcorr.
[5]:
params.loc["shocks_sdcorr"]
[5]:
| value | comment | |
|---|---|---|
| name | ||
| sd_a | 0.20 | Element 1,1 of standard-deviation/correlation ... |
| sd_b | 0.25 | Element 2,2 of standard-deviation/correlation ... |
| sd_edu | 1500.00 | Element 3,3 of standard-deviation/correlation ... |
| sd_home | 1500.00 | Element 4,4 of standard-deviation/correlation ... |
| corr_b_a | 0.00 | Element 2,1 of standard-deviation/correlation ... |
| corr_edu_a | 0.00 | Element 3,1 of standard-deviation/correlation ... |
| corr_edu_b | 0.00 | Element 3,2 of standard-deviation/correlation ... |
| corr_home_a | 0.00 | Element 4,1 of standard-deviation/correlation ... |
| corr_home_b | 0.00 | Element 4,2 of standard-deviation/correlation ... |
| corr_home_edu | 0.00 | Element 4,3 of standard-deviation/correlation ... |
Shocks are assumed to follow a multivariate normal distribution with zero mean and covariance matrix \(\Sigma\). The dimensionality of the symmetric covariance matrix equals the number of modeled choices. The specification of \(\Sigma\) remains in the discretion of the user. Because the symmetry of covariance matrices, it is sufficient to specify the lower triangular matrix. However, it is mandatory to follow the order which is prescribed by respy.
First, the diagonal elements (standard deviations) are specified via
sd_{choice}according to the following order:Working alternatives (alphabetically sorted).
Non-working alternatives with experience accumulation (alphabetically sorted).
Remaining alternatives (alphabetically sorted.)
Second, the off-diagonal elements (correlations) are specified ordered by rows in the matrix.
Aside from specifying shocks according to standard deviations and correlations, you can also specify the variance-covariance matrix. The parameters are ordered by appearance in the lower triangular. Variances have the name var_{choice} and covariances cov_{choice_2}_{choice_1} and so forth. Lastly, another option is the Cholesky factor of the variance-covariance matrix ordered by appearance in the lower triangular. The labels are either chol_{choice} or chol_{choice_2}_{choice_1}
and so forth. In contrast to the other two options, Cholesky shocks are not ordered according to diagonal and off-diagonal elements. Instead they need to be ordered according to appearance by rows in the lower triangular of the shock matrix.
The specification of shocks may appear a bit confusing due to the ordering requirements. Notably, respy will raise an error the shock parameters are not passed in correct order. The error message will help you specify the parameters in the correct order.
Additional Parameters#
Aside from discounting and reward-specific parameters (pecuniary rewards, non-pecuniary rewards, and shocks) there are some additional parameters that you might want to add to specify your model. Below you find a small overview of the type of parameters you may add.
Initial Conditions#
In many instances, you may need to add initial conditions to your model. This can include lagged choices, experience levels, and observable characteristics. Their value in the params reflect the share of individuals that exhibits a specific characteristic. Importantly, initial conditions are usually non-estimable parameters. Our example model requires two such parameter specifications.
The parameter lagged_choice_1_edu ensures that the model logs education as the previous choice in period \(t=-1\) for all individuals in the sample. Our model requires this specification because we include a cost of returning to school in the reward function for education, if the previous choice was another alternative. In order to compute the rewards for period \(0\), respy thus needs to know the choice of the previous period, even if it is not directly part of the model’s
decision horizon.
[6]:
params.loc["lagged_choice_1_edu"]
[6]:
| value | comment | |
|---|---|---|
| name | ||
| probability | 1.0 | Probability that the first lagged choice is ed... |
The parameter initial_exp_edu_10 assigns 10 periods of experience in education (i.e. 10 periods of completed schooling) to all individuals in period \(0\). Adding an initial condition like this may be useful if we think about the correspondence between the model and potential empirical data. Since we are assessing occupational choices, we will be analyzing individuals of working age who will have accumulated schooling before they enter the labor market.
[7]:
params.loc["initial_exp_edu_10"]
[7]:
| value | comment | |
|---|---|---|
| name | ||
| probability | 1.0 | Probability that the initial level of educatio... |
Both of these parameters only exhibit one value that occurs for all individuals in this example. However, initial conditions are much more versatile and can be defined quite flexibility. Refer to the guide linked below for more information.
Maximum Experience#
Much like adding initial experience, we may want to limit the maximum amount of experience. In our example, individuals can complete a maximum of 20 periods of schooling. The implementation is straightforward. We define a category called maximum_exp and add a parameter name that corresponds to the name of a choice (e.g. edu). The value column holds the maximum level of experience.
[8]:
params.loc["maximum_exp"]
[8]:
| value | comment | |
|---|---|---|
| name | ||
| edu | 20.0 | Maximum level of experience for education (opt... |
Unobserved Heterogeneity#
A component not implemented in this example is unobserved heterogeneity between individuals. respy allows to add such components using finite mixture approaches. Check out the guide below and example models based on Keane and Wolpin (1997) to learn more about adding unobserved heterogeneity to your model.
Measurement Error#
You may also implement measurement error in wages. To do so you have to define a category called meas_error and add the parameter names sd_{choice} for all choices with a wage. The parameter value should be the standard deviations of measurement error. Check out the model parametrization of kw_97_extended for an example.
Note that this parameter category only requires standard deviations for choices with a wage. They can be provided for all or none choices with wages, measurement errors for non-wage choices are neglected, and no correlation between measurement errors can be defined.
Defining the options#
The options dictionary is the second necessary component for defining models in respy. As we have learned above, structural parameters are defined in a pandas.DataFrame. The options dictionary holds additional settings and information about the model. Thus, the params DataFrame and options dictionary should be viewed as complementary objects. Some types of parameters require additional options in order for respy to process them. Below we will inspect our example model’s
options.
[9]:
options
[9]:
{'estimation_draws': 200,
'estimation_seed': 500,
'estimation_tau': 500,
'interpolation_points': -1,
'n_periods': 40,
'simulation_agents': 1000,
'simulation_seed': 132,
'solution_draws': 500,
'solution_seed': 15,
'monte_carlo_sequence': 'random',
'core_state_space_filters': ["period > 0 and exp_{choices_w_exp} == period and lagged_choice_1 != '{choices_w_exp}'",
"period > 0 and exp_a + exp_b + exp_edu == period and lagged_choice_1 == '{choices_wo_exp}'",
"period > 0 and lagged_choice_1 == 'edu' and exp_edu == 0",
"lagged_choice_1 == '{choices_w_wage}' and exp_{choices_w_wage} == 0",
"period == 0 and lagged_choice_1 == '{choices_w_wage}'"],
'covariates': {'constant': '1',
'exp_a_square': 'exp_a ** 2',
'exp_b_square': 'exp_b ** 2',
'at_least_twelve_exp_edu': 'exp_edu >= 12',
'not_edu_last_period': "lagged_choice_1 != 'edu'"}}
n_periods#
The option n_periods determines the number of periods that individuals take into account when evaluating their actions. That is, they decide for the action that maximizes their expected utility in an evaluation over n_periods. Possible values are one and higher integers. This option is mandatory as no default is supplied. In most models, the model’s complexity or the number of states in the state space is exponentially increasing in the number of periods.
Do not confuse this option with the number of periods for which you want to simulate the actions of individuals. This number can be lower because although actions of individuals are simulated for, say, 10 periods, their actions can still aim to maximize utility for 50 periods.
[10]:
options["n_periods"]
[10]:
40
simulation_agents#
This option specifies the number of individuals which are simulated. This option is ignored if you pass data to the simulation function.
[11]:
options["simulation_agents"]
[11]:
1000
covariates#
In the subsection on the parameterization of the choice rewards, we discussed the special role of exp_{choice} in defining parameters for the pecuniary reward of occupation A. However, the parameter vector includes further covariates like a constant and squared experience terms.
These covariates need further specification so respy knows how to process them. Covariates with no pre-defined naming convention are specified in the model options as a nested dictionary called covariates. In the covariates dictionary, keys correspond to the parameter name in params and dictionary values hold the definition of this parameter.
For example, all parameters named constant return a value of 1 for every individual. The parameters exp_a_square and exp_b_square signal the return to square experience in both occupations.
The other two covariates enter the reward function for education. at_least_twelve_exp_edu is a boolean that evaluates true when an individual has accumulated 12 periods of schooling or more, and triggers a cost component in the reward function. Lastly, the covariate not_edu_last_period is a boolean indicator for not having chosen education in the last period. As discussed in the section on initial conditions, this requires the inclusion of a lagged choice in
the params DataFrame.
[12]:
options["covariates"]
[12]:
{'constant': '1',
'exp_a_square': 'exp_a ** 2',
'exp_b_square': 'exp_b ** 2',
'at_least_twelve_exp_edu': 'exp_edu >= 12',
'not_edu_last_period': "lagged_choice_1 != 'edu'"}
How should covariate definitions in the options look like to be processed? Here are some pointers:
The statements are evaluated using pandas.eval. This means you can use all arithmetic operations that this method supports.
The following pre-defined terms are recognized to construct covariates:
period,exp_{choice},lagged_choice_{number of periods}.You can also define new covariates as a function of already existing covariates.
Seeds (optional)#
To be able to replicate a model, the options for solution, simulation, and estimation allows for three seeds. The distinction enables us to vary randomness in only one component, independent from the others. The dictionary keys are
solution_seedfor the computation of the decision rules.simulation_seedfor the simulation.estimation_seedfor the computation of the log likelihood.
[13]:
{k: v for k, v in options.items() if "seed" in k}
[13]:
{'estimation_seed': 500, 'simulation_seed': 132, 'solution_seed': 15}
monte_carlo_sequence and draws (optional)#
monte_carlo_sequence and draws refer more generally to approximations of integrals with Monte Carlo simulations inside respy. There exist two applications for Monte Carlo simulation.
In the solution of a model, the value of expected value functions has to be simulated.
While computing the log likelihood, (log) choice probabilities are simulated.
The number of draws controls how many points are used to evaluate an integral. The default is 500 for the solution and 200 for the estimation of choice probabilities.
[14]:
{k: v for k, v in options.items() if "draws" in k}
[14]:
{'estimation_draws': 200, 'solution_draws': 500}
The option monte_carlo_sequence controls how points are drawn.
"random": Points are drawn randomly (crude Monte Carlo)."sobol"or"halton": Points are drawn from low-discrepancy sequences (superiority in coverage). This means a given approximation error can be achieved with less points.
interpolation_points#
The number of interpolation points specifies the number states or their corresponding expected value functions which are used to fit an interpolation model. The model is used to predict the expected value functions for all remaining states. The interpolation method available in respy is designed by Keane and Wolpin (1994). Their paper offers a detailed explanation of the method.
If interpolation_points is set to -1, the full solution is computed.
[15]:
options["interpolation_points"]
[15]:
-1
negative_choice_set#
You can limit the set of available choices at different points in time using the option negative_choice_set. To implement a negative choice set, define a nested dictionary where keys correspond to choice alternatives and values hold a list of conditions that will eliminate the corresponding choice for periods whenever it evaluates to True.
For example, consider a scenario where individuals can only work in occupation A after the fifth period (\(t=4\)) (i.e. the occupation may have an age requirement). In this case, we need to implement a negative choice set for the first five periods as follows.
[16]:
options["negative_choice_set"] = {"a" : ["period < 5"]}
core_state_space_filters (optional)#
Core state space filters partly complement the negative_choice_set option. First of all, what is the core state space? The core state space is the part of the state space spanned by the combinations of experiences and previous choices. Not all combinations are feasible, but it is not always possible to catch all invalid combinations.
States with impossible combinations have no effect on the correctness of the model, but pose an additional computational burden which should be eliminated. Similar to negative_choice_set the core_state_space_filters are a list of conditions and whenever one of them is true, the state is eliminated from the state space.
This option is a rather advanced feature of respy as it requires a sound understanding of the state space and at least partial knowledge on how it processed internally. In most cases, you would not necessarily need to add them, but they can be useful to:
Improve the computational performance of your model.
Implement restrictions on the choice set that cannot be implemented using the
paramsornegative_choice_setoption .
[17]:
options["core_state_space_filters"]
[17]:
["period > 0 and exp_{choices_w_exp} == period and lagged_choice_1 != '{choices_w_exp}'",
"period > 0 and exp_a + exp_b + exp_edu == period and lagged_choice_1 == '{choices_wo_exp}'",
"period > 0 and lagged_choice_1 == 'edu' and exp_edu == 0",
"lagged_choice_1 == '{choices_w_wage}' and exp_{choices_w_wage} == 0",
"period == 0 and lagged_choice_1 == '{choices_w_wage}'"]
Order is important
negative_choice_set’s are applied after initial conditions are implemented.
core_state_space_filters are applied before initial conditions are implemented. Pay attention to this when you have, for example, implemented initial experience for a choice. When adding a filter based on the experience for this choice, you will have to refer to within-model experience and discard knowledge of potential previous experience.
estimation_tau#
This option is only relevant for maximum likelihood estimation.
The choice probabilities in the likelihood function are simulated, as there exists no closed-form solution for them. They are computed with the softmax function and require the specfication of a so-called temperature parameter tau. This parameter can be specified in the respy options.
[18]:
options["estimation_tau"]
[18]:
500
References#
Keane, M. P., & Wolpin, K. I. (1994). The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence. The Review of Etheconomics and Statistics, 648-672.
Keane, M. P., & Wolpin, K. I. (1997). The Career Decisions of Young Men. Journal of Political Economy, 105(3), 473-522.