Exercises

• The function $$L = \sum_{i=0}^{N-1} \ln\left( {e^{y_i x_i\beta}\over 1+e^{x_i\beta}} \right)$$ is the likelihood function for a simple (binary) logit model. This model explains which of two discrete outcomes happens ($y=0$ or $y=1$) using the row vector $x_i$ as explanatory variables. For example, explaining who goes to university or not depending gender, high school grades, parents' education. The columns of $x_i$ correspond to the variables like gender. The multinomial logit generalizes the simple or binary logit so that $y$ takes on $J$ different values instead $J=2$ values. The idea of a mlogit is that each option $j$ has its own "utility." The utility depends on the chooser ($i$) and the option ($j$). It also combines deterministic and random components so that there is probability of $j$ being optimal for chooser $i$. So let $$v_{ij} = x_i \beta_j$$ be the deterministic component and $\epsilon_{ij}$ be the random component, so that utility is: $$u_{ij} = v_{ij} + \epsilon_{ij}.$$ We observe $i$ choosing the option that maximizes utility their utility: $$j^\star_i = \arg\max_{j\in 0,1,\dots,J-1}\ u_{ij}.$$ When the residual $\epsilon_{ij}$ follows a particular distribution (the Extreme Value distribution) then we get a specific form for the probability that $j^\star_i=y_i$ (where $y_i$ is the observed choice): $$Prob( j^\star_i =y_i | x_i, \beta_0,\dots,\beta_{J-1}) = { e^{v_{i\,y_i}} \over \sum_{j=0}^{J-1} e^{v_{i\,j}}}.$$ Notice that this uses the discrete outcome $y_i$ as an index for the correct $v_{ij}$ to include in the numerator, and this in turn depends on $\beta_j.$

  We can get back to the simple logit by setting J=2 and $\beta_0 = \overrightarrow{0}.$ That is option $0$ has beta vector of 0s, which means $e^{v_{i 0}} = e^{x_i\beta_0} = e^0 = 1.$ And now that function we've been working with all along becomes a special case of the multinomial logit log-likelihood function: $$L^J = \sum_{i=0}^{N-1} \ln \left( { e^{v_{i\,y_i}} \over \sum_{j=0}^{J-1} e^{v_{i\,j}} }\right),\qquad\hbox{and } v_{ij} \equiv x_i\beta_j.$$ Note that $L^J$ is shorthand. A more accurate definition would be $L^J(\beta_1,\dots,\beta_{J-1}; y, X).$ That is, $y$ and $X$ are taken as given. The vectors of coefficients are variable in order to maximize $L^J.$

  In the simple logit we could maximize this function using Ox's MaxBFGS() by sending a single beta vector as the parameters to maximize over. In the multinomial model we want to maximize over 2 or more vectors, but MaxBFGS() like almost all optimizers takes only a column vector as the argument to maximize over. Let's define $\theta$ as that vector and think of it containing the "concatenation" of the $\beta_j$'s: $$\theta \equiv \pmatrix{\beta_1\cr\beta_2\cr\vdots\cr\beta_{J-1}}$$ Note that we don't include $\beta_0$ because it is fixed at 0 and MaxBFGS() should not change its values. A function written to compute $L^J$ would "unpack" $\theta$ into a matrix: $$\theta_{(J-1)K\times 1} \qquad \Rightarrow \qquad B_{K\times J} = \pmatrix{ \overrightarrow{0} & \beta_1 & \beta_2 & \cdots & \beta_{J-1}}.$$ Now notice that $XB$ will contain a $N\times J$ matrix consisting of all the values $x_i\beta_j$ for all $i$ and all $j$. So computing $v_{ij}$ and ultimately $L^J$ can be done with vector functions (or with loops over $j$ or both $i$ and $j$). The code and data file in the mlogit folder is designed to build a function to compute $L^J$ and finding the maximizing coefficients using one of the Max????() algorithms. The data come from the example used in Stata to illustrate its mlogit command. The data and estimates are summarized with these screen captures:

  

  Add code to the files in the folder to:

  1. Complete coding of LJ
  2. Create a starting theta vector
  3. Maximize LJ (use algorithm of your choice)
  4. Print out results to compare with Stata output
  5. EXTRA:
  1. copy over stata_logit_data.dta from the logit folder
  2. use of loaddemo to load the simple logit data data
  3. Reset K and J (*NOTE: do not decrement y since it is already 0/1)
  4. Verify LJ() to provides same results as simple logit