Problem #1. (15 pts)

Use the “College” data set from the package “ISLR”, do the following:

1. Get familiar with the data set. How many variables the data set has? What is the class of the variable “Private”? (3 pt)
2. Add a new column “Enroll.rate” which is the ratio of Enroll v.s. Accept, to the data set. (4 pts)
3. What are the top 10 Universities in terms of “Enroll.rate” (highest student enrollment ratio)? (4 pts)
4. Get a new data frame “Mysubset” with all rows with an “Outstate” fee between 8000-9000 USD. (4 pts)

Problem #2. (12 pts)

Download and import the csv file “R_Data_Hack_Problem2” from Canvas under Module R Data Hack and rename it as “testdata”. Can you reject the hypothesis that the mean of “Fertility Rate” in 2013 is greater than in 1960 based on 99% confidence interval? Explain your rationale. (6 pts) Is it appropriate to use t test for this problem and why? (2 pts) Can you assume equal variance for the two groups? (2 pts) Clearly state your H0 and H1. (2 pt)

Problem #3. (18 pts)

Use set.seed(194) to generate the data set mydata = rep(sin(seq(0,10,length.out = 25)),4) + rnorm(25*4), and do the following:

1. Create ‘myts’ which is a monthly time series of ‘mydata’, starting from June of 2014. (3 pts)
2. Plot the data, and explain the statistical characters of the data. (2 pts)
3. Use 80% of the data as the training set and the rest as the test set. (3 pts)
4. Set up two forecasting models using the Average method and the Drift method with the training set. (2 pts)
5. Get a plot with the two forecasts of the models, and add a legend. (2 pts)

6. Get the error measures of the two forecast methods and compare them in terms of the training set and test set; Which method looks more promising? Explain your rationale. (6 pts)

Part 1: Understanding Data and Measurement (15 points)

Data and Information Hierarchy (5 points): Describe the difference between data, information, knowledge, and wisdom, explaining the hierarchical relationship among them. Provide a specific, real-world example to illustrate each level of the hierarchy.

Variables and Measurement Scales (10 points): Explain the different types of variables (nominal, ordinal, interval, and ratio), and describe the associated scales of measurement. Provide a specific example of each type of variable and explain why it is classified as such. 

Part 2: Descriptive Statistics and Bivariate Analysis (30 points) 

• Frequency Distribution and Summary Measures (15 points): Select a dataset (this could be publicly available data or a dataset from your workplace). Create a frequency distribution for a chosen variable, calculate common summary measures (mean, median, mode, range, variance, and standard deviation), and provide a short interpretation of these measures.
• Bivariate Analysis (15 points): With the same dataset or a different one, conduct a bivariate analysis that includes both an association between two qualitative variables and a correlation between two quantitative variables. Interpret your findings. 

Part 3: Probability and Distributions (30 points) 

Probability (10 points): Discuss the basic rules of probability, conditional probability, and Bayes’ theorem. Illustrate your discussion with unique examples.

Random Variables and Probability Distributions (20 points): Define discrete and continuous random variables. Give a real-world example of each and describe the associated probability distribution for each variable.

• Part 4: Sampling Techniques (20 points)
• Sampling (20 points): Define and differentiate random and non-random sampling. Discuss how to determine an appropriate sample size for a given study. Include an illustrative example from a real or hypothetical research study.

In this assignment you will implement partial row pivoting on a coefficient matrix during LU factorization. In notebook #21, we show an algorithm to transform a square coefficient matrix into upper triangular form U. Then we show a modified algorithm that transforms a square coefficient matrix into L+U+I form, where L is a canonical lower triangular matrix, U is the upper triangular form from the first algorithm, and I is the identity. Consider the coefficient matrix

$�=\left[\begin{array}{cccc}1& 0& 3& 0\\ 0& 1& 3& -1\\ 3& -3& 0& 6\\ 0& 2& 4& -6\end{array}\right]$

1. (20 points) Apply the initial upper triangular algorithm to this system. What issue do you observe? Demonstrate the problem in your notebook and explain the problem in a Markdown cell.

2. (20 points) Implement partial row pivoting in the upper triangular algorithm. The idea is to search all rows below the current row i in the row iteration to find the row j such that if rows i and j were swapped (exchanged), the swapped in row would have the largest diagonal element in absolute value.

3. (20 points) Implement partial row pivoting in the L+U+I form algorithm and save the indices of the rows that are pivoted in a data structure.

4. (20 points) Show A = LU. Note you need to first apply the same pivots to A.

5. (20 points) Use factors L and U to solve Ax = b for x where

$�=\left[\begin{array}{c}6\\ 0\\ 4\\ 3\end{array}\right]$

using the forward and backward substitution methods shown in notebook #21.