{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
},
"language_info": {
   "codemirror_mode": {
   "name": "ipython",
   "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
},
"toc": {
   "base_numbering": 1,
   "nav_menu": {
   "height": "318px",
   "width": "252px"
   },
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
   "height": "771px",
   "left": "0px",
   "right": "1376px",
   "top": "107px",
   "width": "455.71px"
   },
   "toc_section_display": "block",
   "toc_window_display": true
},
"varInspector": {
   "cols": {
   "lenName": 16,
   "lenType": 16,
   "lenVar": 40
   },
   "kernels_config": {
   "python": {
      "delete_cmd_postfix": "",
      "delete_cmd_prefix": "del ",
      "library": "var_list.py",
      "varRefreshCmd": "print(var_dic_list())"
   },
   "r": {
      "delete_cmd_postfix": ") ",
      "delete_cmd_prefix": "rm(",
      "library": "var_list.r",
      "varRefreshCmd": "cat(var_dic_list()) "
   }
   },
   "types_to_exclude": [
   "module",
   "function",
   "builtin_function_or_method",
   "instance",
   "_Feature"
   ],
   "window_display": false
},
"colab": {
   "name": "LinRegrBoston_Bike-Demandh.ipynb",
   "provenance": [],
   "collapsed_sections": [
   "GTaI962kH2WJ",
   "rjp3FeGGH2WN",
   "zid2JE2aH2dm",
   "wus8-ttNH2eP",
   "sqbk1UKPH2fo",
   "qODj9GPgH2f-",
   "Lz9Wdz-FH2gT",
   "qlU2N2FDH2gT"
   ]
}
},
"cells": [
{
   "cell_type": "markdown",
   "metadata": {
   "ExecuteTime": {
      "end_time": "2018-05-29T16:30:46.572028Z",
      "start_time": "2018-05-29T16:30:46.563631Z"
   },
   "id": "eaGiz29cH2QM",
   "colab_type": "text"
   },
   "source": [
   "# HW Overview\n",
   "\n",
   "Please complete all core assignment tasks. For 50% bonus points please complete the optional stretech assignment tasks that will involve more theory and programming.\n",
   "\n",
   "The goals of this HW include the following:\n",
   "* master pipelines for regression prediction problems\n",
   " * Exploratory data analysis\n",
   " * Date preprocessing\n",
   " * Data standardization\n",
   " * Deal with missing data\n",
   " * Dealing with categorical input features via, say, OHE \n",
   " * Linear regression modeling (regularized or not)\n",
   " * Various performance metrics\n",
   " \n",
   " \n",
   "Please consult Canvas for the grading rubric (check Module 05 -> HW 05).\n"
   ]
},
{
   "cell_type": "markdown",
   "metadata": {
   "id": "I7jipqcJH2QQ",
   "colab_type": "text"
   },
   "source": [
   "# Linear Regression Basics"
   ]
},
{
   "cell_type": "markdown",
   "metadata": {
   "id": "4zKhKLuiH2QR",
   "colab_type": "text"
   },
   "source": [
   "## Batch Gradient Descent"
   ]
},
{
   "cell_type": "markdown",
   "metadata": {
   "id": "IQhPQJRHH2QT",
   "colab_type": "text"
   },
   "source": [
   "Use the starter code below to calculate the first iteration of gradient descent to minimize the objective function \n",
   "$2x^2+ 4x$. We assume that the initial guess is $x=2$ of  and a stepsize of 0.1 "
   ]
},
{
   "cell_type": "code",
   "metadata": {
   "id": "iF1YbbgIH2QV",
   "colab_type": "code",
   "colab": {
      "base_uri": "https://localhost:8080/",
      "height": 34
   },
   "outputId": "e89fb8ff-0c2c-47ac-fa60-b4459e0672e4"
   },
   "source": [
   "#==================================================#\n",
   "#             Your code starts here          #\n",
   "#==================================================#\n",
   "\n",
   "# TODO - change above to :\n",
   "def step(x, eta):\n",
   " update = eta * 4*x+4 #Insert f prime here\n",
   " return x - update\n",
   "\n",
   "step(x=2, eta=0.1)\n",
   "#==================================================#\n",
   "#             Your code ends here             #\n",
   "#==================================================#"
   ],
   "execution_count": 63,
   "outputs": [
   {
      "output_type": "execute_result",
      "data": {
         "text/plain": [
         "-2.8"
         ]
      },
      "metadata": {
         "tags": []
      },
      "execution_count": 63
   }
   ]
},
{
   "cell_type": "markdown",
   "metadata": {
   "id": "Q7Xr_GjHH2Qb",
   "colab_type": "text"
   },
   "source": [
   "## Stochastic Gradient Descent"
   ]
},
{
   "cell_type": "markdown",
   "metadata": {
   "id": "DTlOzJ4pH2Qd",
   "colab_type": "text"
   },
   "source": [
   "Assume you are learning a linear regression model (using the mean squared error objective loss function) via stochastic gradient descent with the following setting:\n",
   "* X = [2, 3], y = 6, (single training example)\n",
   "* a learning rate, α= 0.1 \n",
   "* the current model weight vector is  W = [1, 0, 1], where W[0] denotes the bias term.\n",
   "* Assume the gradient is defined as follows: d E / d W = (y^ - y) X\n",
   "* Use the code below to determine the value of W after applying one iteration of gradient descent?\n"
   ]
},
{
   "cell_type": "code",
   "metadata": {
   "id": "pwAPNqwSH2Qe",
   "colab_type": "code",
   "colab": {
      "base_uri": "https://localhost:8080/",
      "height": 34
   },
   "outputId": "bedaee06-e3b6-44da-c71a-6da0de1c0e3c"
   },
   "source": [
   "import numpy as np\n",
   "#==================================================#\n",
   "#             Your code starts here          #\n",
   "#==================================================#\n",
   "\n",
   "# TODO - change above to :\n",
   "X= [1,2,3]\n",
   "W = [1, 0, 1]\n",
   "y=[6]\n",
   "alpha= 0.1\n",
   "y_hat = sum(np.multiply(W, X))-y\n",
   "gradient= np.multiply(y_hat, X)\n",
   "newW= W - alpha*gradient\n",
   "print (newW)\n",
   "#==================================================#\n",
   "#             Your code ends here             #\n",
   "#==================================================#"
   ],
   "execution_count": 64,
   "outputs": [
   {

1. (10 points) Below, L(w) : Rd R are loss functions. For each L, (...