Expert answer

DRAFT / STUDY TIPS:

(a) Data Preprocessing

How to Approach This Question

• Step 1: Load the dataset using pandas.
• Step 2: Handle missing values by dropping or imputing them.
• Step 3: Convert categorical variables into proper formats (category or dummy variables).
• Step 4: Split the dataset into training and testing sets.

Solution

python

CopyEdit

import pandas as pd import numpy as np from sklearn.model_selection import train_test_split # Load dataset hotel_bookings = pd.read_csv("hotel_booking.csv") # Step 1: Remove missing values hotel_bookings.dropna(inplace=True) # Step 2: Convert categorical variables categorical_cols = ["hotel", "arrival_date_month", "distribution_channel", "deposit_type"] hotel_bookings[categorical_cols] = hotel_bookings[categorical_cols].astype("category") # Step 3: Split dataset (50-50 split) np.random.seed(123456) # Replace with your student ID train_data, test_data = train_test_split(hotel_bookings, test_size=0.5, random_state=123456)

✅ Key Points:

• Using dropna() removes missing values.
• Converting categorical variables ensures they can be used in models.
• train_test_split() ensures a random but reproducible split.

(b) Linear and Logistic Regression

How to Approach This Question

• Step 1: Fit OLS (Ordinary Least Squares) Regression for binary classification.
• Step 2: Fit Logistic Regression.
• Step 3: Compute test error and compare both models.

Solution

python

CopyEdit

from sklearn.linear_model import LinearRegression, LogisticRegression from sklearn.metrics import mean_squared_error, accuracy_score # Define predictors and target variable X_train = train_data.drop(columns=["is_canceled"]) X_test = test_data.drop(columns=["is_canceled"]) y_train = train_data["is_canceled"] y_test = test_data["is_canceled"] # Step 1: OLS Regression linear_model = LinearRegression() linear_model.fit(X_train, y_train) y_pred_ols = linear_model.predict(X_test) # Convert continuous OLS predictions to binary (0 or 1) y_pred_ols_binary = (y_pred_ols > 0.5).astype(int) ols_mse = mean_squared_error(y_test, y_pred_ols_binary) # Step 2: Logistic Regression logistic_model = LogisticRegression(max_iter=500) logistic_model.fit(X_train, y_train) y_pred_logistic = logistic_model.predict(X_test) logistic_acc = accuracy_score(y_test, y_pred_logistic) # Step 3: Compare test errors print(f"OLS MSE: {ols_mse}") print(f"Logistic Regression Accuracy: {logistic_acc}")

✅ Key Points:

• OLS Regression is not ideal for binary classification but provides insights.
• Logistic Regression is better for predicting is_canceled.
• The accuracy_score() function evaluates model performance.

(c) LDA and QDA

How to Approach This Question

• Step 1: Fit Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA).
• Step 2: Compute test errors.
• Step 3: Compare with Logistic Regression.

Solution

python

CopyEdit

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis, QuadraticDiscriminantAnalysis # Step 1: Train LDA model lda = LinearDiscriminantAnalysis() lda.fit(X_train, y_train) y_pred_lda = lda.predict(X_test) # Step 2: Train QDA model qda = QuadraticDiscriminantAnalysis() qda.fit(X_train, y_train) y_pred_qda = qda.predict(X_test) # Step 3: Compute test errors lda_acc = accuracy_score(y_test, y_pred_lda) qda_acc = accuracy_score(y_test, y_pred_qda) print(f"LDA Accuracy: {lda_acc}") print(f"QDA Accuracy: {qda_acc}")

✅ Key Points:

• LDA works well if assumptions of normality hold.
• QDA is more flexible but can overfit if there isn’t enough data.

(d) Ridge and Lasso Regression

How to Approach This Question

• Step 1: Use cross-validation to choose the best λ (regularization parameter).
• Step 2: Fit Ridge and Lasso Regression.
• Step 3: Report test errors.

Solution

python

CopyEdit

from sklearn.linear_model import RidgeCV, LassoCV # Step 1: Define range of lambda values lambda_values = np.logspace(-4, 4, 100) # Step 2: Perform cross-validation for Ridge Regression ridge_cv = RidgeCV(alphas=lambda_values, store_cv_values=True) ridge_cv.fit(X_train, y_train) ridge_best_lambda = ridge_cv.alpha_ ridge_test_error = mean_squared_error(y_test, ridge_cv.predict(X_test)) # Step 3: Perform cross-validation for Lasso Regression lasso_cv = LassoCV(alphas=lambda_values, cv=10) lasso_cv.fit(X_train, y_train) lasso_best_lambda = lasso_cv.alpha_ lasso_test_error = mean_squared_error(y_test, lasso_cv.predict(X_test)) print(f"Best Ridge λ: {ridge_best_lambda}, Test Error: {ridge_test_error}") print(f"Best Lasso λ: {lasso_best_lambda}, Test Error: {lasso_test_error}")

✅ Key Points:

• Ridge regression keeps all variables but shrinks coefficients.
• Lasso regression selects important predictors by shrinking some coefficients to zero.

(e) Feature Selection and Performance Comparison

How to Approach This Question

• Step 1: Select 5 predictors.
• Step 2: Train models using only these predictors.
• Step 3: Compare classification errors with full models.

Solution

python

CopyEdit

# Step 1: Select top 5 predictors selected_features = ["lead_time", "adr", "stays_in_week_nights", "adults", "total_of_special_requests"] X_train_reduced = train_data[selected_features] X_test_reduced = test_data[selected_features] # Step 2: Train models with reduced features logistic_reduced = LogisticRegression(max_iter=500) logistic_reduced.fit(X_train_reduced, y_train) y_pred_reduced = logistic_reduced.predict(X_test_reduced) # Step 3: Compare accuracy reduced_acc = accuracy_score(y_test, y_pred_reduced) print(f"Reduced Model Accuracy: {reduced_acc}")

✅ Key Points:

• Using fewer variables can reduce overfitting.
• Performance may improve if irrelevant variables are removed.

Final Tips for Answering This Assignment

1. Follow a structured approach: Always start with data cleaning, then move to model fitting and evaluation.
2. Use proper evaluation metrics:
   • Accuracy for classification models.
   • Mean Squared Error (MSE) for regression models.
3. Compare and interpret results:
   • Do models perform differently?
   • Does using fewer features improve accuracy?
   • Does regularization help reduce overfitting?
4. Use cross-validation to optimize hyperparameters.
5. Explain each step clearly in the Jupyter Notebook submission