Machine Learning (DTSC 3220)

Explanation:

A regression problem in supervised learning involves estimating continuous numerical values based on input features. Unlike classification tasks, where the outcome is a discrete category, regression predicts a quantity, such as a patient’s blood pressure, house price, or temperature. The model learns the relationship between the input variables and the continuous target variable, allowing it to make predictions on unseen data. Regression is fundamental in scenarios where numerical forecasting is required.

Correct Answer:

Estimating continuous numerical values from input features

Why Other Options Are Wrong:

Predicting categorical outcomes based on input features

This is incorrect because predicting categorical outcomes is the definition of a classification problem, not regression.

Classifying data points into distinct categories

This is incorrect because classification involves discrete outcomes, whereas regression focuses on continuous values.

Identifying patterns without labeled data

This is incorrect because identifying patterns without labeled data is an unsupervised learning task, not a regression problem in supervised learning.

Explanation:

The primary purpose of Principal Component Analysis (PCA) is to reduce the number of features in a dataset while preserving as much variance as possible. PCA transforms the original correlated variables into a smaller set of uncorrelated variables called principal components. This reduces computational complexity, mitigates multicollinearity, and helps in visualizing high-dimensional data. By focusing on the components that capture the most variance, PCA allows analysts to simplify datasets without losing significant information.

Correct Answer:

To reduce the number of features while preserving variance

Why Other Options Are Wrong:

To increase the dimensionality of the dataset

This is incorrect because PCA reduces dimensionality rather than increasing it. The goal is to simplify the dataset while retaining important information.

To classify data into predefined categories

This is incorrect because PCA is an unsupervised dimensionality reduction technique. It does not perform classification; it simply transforms features into a lower-dimensional space.

To visualize data in its original high-dimensional space

This is incorrect because visualizing data in its original high-dimensional space is often impractical. PCA allows visualization in a reduced, lower-dimensional space, making it easier to interpret patterns and relationships.

Explanation:

In gradient descent, the learning rate is a hyperparameter that controls the step size for updating the regression coefficients during each iteration. It determines how quickly or slowly the model moves toward the minimum of the cost function. A learning rate that is too large can cause overshooting and divergence, while a learning rate that is too small can lead to very slow convergence. Properly setting the learning rate ensures efficient and stable optimization.

Correct Answer:

It controls the adjustment speed of the regression coefficients towards the minimum of the cost function

Why Other Options Are Wrong:

It specifies the number of iterations to perform

This is incorrect because the learning rate does not determine the number of iterations; it only affects how much the parameters are adjusted in each iteration. The number of iterations is set independently.

It defines the initial values of the regression coefficients

This is incorrect because initial coefficient values are set separately, typically randomly or using heuristics. The learning rate only affects the step size during updates.

It determines the final value of the cost function

This is incorrect because the learning rate influences the path of convergence but does not directly set the final minimum value of the cost function, which is determined by the model and data.

Explanation:

Supervised learning problems are considered classification problems when the target variable is categorical. In classification, the model predicts discrete labels, such as disease presence/absence, fraud detection, or patient risk categories. In contrast, regression problems involve continuous numerical target variables. The type of target variable—categorical versus numerical—determines whether the supervised learning task is classification or regression.

Correct Answer:

The target variable is a categorical value

Why Other Options Are Wrong:

The target variable is a numerical value

This is incorrect because numerical target variables indicate a regression problem, not classification.

The target variable is unknown

This is incorrect because unknown targets characterize unsupervised learning, not supervised learning. Classification still requires known labels for training.

The target variable is known but there are not enough samples to draw a unique conclusion from

This is incorrect because the number of samples affects model reliability and generalization, but it does not change the problem type. Classification is defined by the categorical nature of the target variable, regardless of sample size.

Explanation:

In softmax regression, the K-dimensional probability vector represents the predicted probabilities for each of the K possible classes for a given instance. Each element of the vector corresponds to the likelihood that the input belongs to a specific class, and all probabilities sum to one. This vector allows the model to make multi-class predictions and is central to calculating the cross-entropy loss during training.

Correct Answer:

It provides the predicted probabilities for each of the K classes associated with an instance.

Why Other Options Are Wrong:

It indicates the total number of features used in the model

This is incorrect because the K-dimensional vector relates to the number of classes, not the number of input features.

It represents the loss function values for each class

This is incorrect because the vector contains predicted probabilities, not loss values. Loss is computed separately, typically using cross-entropy.

It denotes the gradient values for model optimization

This is incorrect because gradients are calculated during backpropagation for optimization purposes. The K-dimensional vector represents output probabilities, not gradient values.

Explanation:

In Stochastic Gradient Descent (SGD), the learning rate determines the step size taken during each iteration of parameter updates. If the learning rate is too high, the algorithm may overshoot the minimum and fail to converge. If it is too low, convergence will be slow, requiring more iterations to reach the minimum. An appropriately sized learning rate achieves a balance, allowing the algorithm to converge efficiently while minimizing the risk of instability or divergence.

Correct Answer:

An appropriately sized learning rate balances the speed of convergence and the risk of overshooting the minimum.

Why Other Options Are Wrong:

A very high learning rate ensures convergence to the global minimum

This is incorrect because a very high learning rate can cause the algorithm to oscillate or diverge, preventing convergence to the minimum. It does not guarantee stability or optimal solutions.

A low learning rate prevents the algorithm from converging

This is incorrect because a low learning rate does not prevent convergence; it slows down the convergence process. The algorithm can still reach the minimum but will require many more iterations.

The learning rate has no impact on SGD's performance

This is incorrect because the learning rate is a critical hyperparameter in SGD. It directly affects convergence speed, stability, and the ability to reach a minimum, making it essential for performance.

Explanation:

The primary objective of Principal Component Analysis (PCA) is to reduce the dimensionality of data while preserving as much of the variability (information) as possible. PCA transforms the original correlated variables into a smaller set of uncorrelated principal components, which capture the majority of the variance in the data. This reduces computational complexity, mitigates multicollinearity, and facilitates visualization and interpretation of high-dimensional datasets.

Correct Answer:

To reduce the dimensionality of data while preserving most of the variability

Why Other Options Are Wrong:

To identify clusters of similar data points in high-dimensional datasets

This is incorrect because clustering is an unsupervised learning task, not the objective of PCA. PCA may help visualize clusters but does not perform clustering itself.

To estimate the parameters of a linear regression model

This is incorrect because parameter estimation is part of regression analysis, not PCA. PCA is concerned with transforming and reducing features, not fitting a predictive model.

To determine the optimal number of clusters in a dataset

This is incorrect because PCA does not determine cluster numbers. It only reduces dimensions to simplify data representation, whereas clustering algorithms are used to determine groups.

Explanation:

Classification tasks aim to assign categorical labels to input data based on learned patterns from labeled training data. The model learns from examples where the correct category is known and predicts the class for new, unseen instances. This distinguishes classification from regression, which predicts continuous outputs, and from clustering or dimensionality reduction, which focus on unsupervised analysis or data representation rather than categorical prediction.

Correct Answer:

To assign categorical labels to data instances

Why Other Options Are Wrong:

To predict continuous outcomes based on input features

This is incorrect because predicting continuous outputs is the goal of regression tasks, not classification.

To group similar data points into clusters

This is incorrect because clustering is an unsupervised learning task, not a classification task. Classification requires labeled outcomes.

To reduce dimensionality of the dataset

This is incorrect because dimensionality reduction focuses on simplifying data representation, not on predicting class labels for instances.

Explanation:

The standard predictive modeling pipeline in healthcare begins with defining the prediction targets, which clarify the outcome of interest. Next, patient cohorts are constructed to select the relevant population for analysis. Features are then selected based on relevance and predictive power, followed by building the model using these features. Finally, the model is evaluated to assess performance, accuracy, and generalizability. This sequence ensures a systematic and effective approach to predictive modeling in clinical contexts.

Correct Answer:

Defining prediction targets, constructing patient cohorts, feature selection, model building, and evaluation

Why Other Options Are Wrong:

Data collection, data cleaning, feature engineering, model deployment, and monitoring

This is incorrect because while these steps are part of the overall data science process, defining prediction targets and constructing patient cohorts are essential initial steps in the predictive modeling pipeline and are missing from this sequence.

Data visualization, hypothesis testing, feature extraction, model training, and reporting

This is incorrect because visualization and hypothesis testing are exploratory data analysis techniques, not formal steps in the predictive modeling pipeline. The sequence does not properly capture cohort construction and model evaluation.

Constructing patient cohorts, feature selection, data augmentation, model evaluation, and interpretation

This is incorrect because defining prediction targets should precede cohort construction, and data augmentation is not a standard step in predictive modeling for tabular healthcare data. The sequence does not fully align with standard pipeline methodology.