Machine Learning (DTSC 3220)

Explanation:

Regression in machine learning is a technique used to predict continuous numerical outputs based on input features. It models the relationship between independent variables (features) and a dependent variable (target) to make predictions. Unlike classification, which assigns categorical labels, regression outputs values that can vary continuously, such as prices, temperatures, or probabilities. Regression is fundamental in predictive modeling for tasks that require estimating quantities rather than categories.

Correct Answer:

A technique for predicting a continuous output.

Why Other Options Are Wrong:

A method for classifying data into predefined categories

This is incorrect because classification, not regression, is used to assign categorical labels to data points. Regression deals with continuous numerical predictions.

The process of reducing the dimensionality of data

This is incorrect because dimensionality reduction techniques, such as PCA, are separate from regression and focus on reducing the number of input features rather than predicting a target variable.

A way to find the median value in a dataset

This is incorrect because finding the median is a statistical operation and does not involve modeling relationships between input features and a target variable, which is the goal of regression.

Explanation:

The key difference between autoencoders and PCA lies in the type of transformations they can perform. PCA is a linear dimensionality reduction technique that projects data onto orthogonal components, capturing maximum variance along linear directions. Autoencoders, on the other hand, are neural network-based models capable of learning nonlinear transformations of the data. This allows autoencoders to capture complex patterns and relationships in the data that linear PCA cannot, making them more flexible for tasks requiring nonlinear representation learning.

Correct Answer:

Autoencoders are nonlinear models, while PCA is linear

Why Other Options Are Wrong:

Autoencoders are linear models, while PCA is nonlinear

This is incorrect because the opposite is true. PCA is linear, and autoencoders are capable of modeling nonlinear relationships through their neural network architecture.

Autoencoders require supervised learning, while PCA is unsupervised

This is incorrect because autoencoders are typically trained in an unsupervised manner to reconstruct their inputs. They do not require labeled data, just like PCA, which is also unsupervised.

Autoencoders cannot handle large datasets, while PCA can

This is incorrect because autoencoders can handle large datasets efficiently using neural networks and batch training. PCA can become computationally expensive for very large datasets due to matrix decomposition, so this statement is misleading.

Explanation:

Supervised learning is a machine learning paradigm in which all training data is labeled, meaning the input features are paired with corresponding output values. The algorithm learns a mapping from inputs to outputs so that it can predict the target value for new, unseen inputs. This approach is widely used for both regression (continuous outputs) and classification (discrete outputs) tasks.

Correct Answer:

All data is labeled and the algorithms learn to predict the output from the input data

Why Other Options Are Wrong:

Some data is labeled but most of it is unlabelled and a mixture of supervised and unsupervised techniques can be used

This is incorrect because supervised learning requires that all training data be labeled; mixing labeled and unlabeled data describes semi-supervised learning.

All data is unlabelled and the algorithms learn to inherent structure from the input data

This is incorrect because learning from unlabeled data is characteristic of unsupervised learning, not supervised learning.

It is a framework for learning where an agent interacts with an environment and receives a reward for each interaction

This is incorrect because this describes reinforcement learning, which is distinct from supervised learning

Explanation:

In machine learning, input features are the data provided to the model to make predictions or classifications, such as patient age, blood pressure, or lab results in a healthcare dataset. Model parameters, on the other hand, are the internal variables of the model, like weights and biases, that are learned and adjusted during training to minimize the loss function. The model uses the input features along with these learned parameters to generate predictions. Distinguishing between input features and parameters is essential for understanding how models learn from data.

Correct Answer:

Input features are the data given to the model, whereas model parameters are the internal variables that are learned.

Why Other Options Are Wrong:

Input features are adjustable during training, whereas model parameters are fixed

This is incorrect because input features are fixed data values provided to the model, while model parameters are the ones that are adjusted during training.

Model parameters are manually set before training, whereas input features are learned

This is incorrect because model parameters are learned during training, not manually set. Input features are the raw data provided, not learned variables.

Input features and model parameters are interchangeable terms

This is incorrect because input features and model parameters serve distinct roles; features are the inputs, and parameters are the learnable aspects of the model. They are not interchangeable.

Explanation:

The primary advantage of using the log likelihood in MLE is that it simplifies the optimization process by converting products into sums. The likelihood function often involves the product of many probabilities, which can be computationally challenging and prone to numerical underflow. By taking the natural logarithm, the product of probabilities becomes a sum of log probabilities, making differentiation and optimization more straightforward. This allows for easier calculation of parameter estimates that maximize the likelihood function.

Correct Answer:

It simplifies the optimization process by converting products into sums.

Why Other Options Are Wrong:

It allows for the direct computation of probabilities without transformation

This is incorrect because the log likelihood does not compute probabilities directly; it transforms the likelihood for easier optimization. Probabilities still need to be derived from the model parameters.

It provides a more complex representation of the data

This is incorrect because taking the log actually simplifies the representation mathematically, rather than making it more complex. The purpose is to make optimization manageable, not to complicate the data.

It eliminates the need for parameter estimation

This is incorrect because MLE is inherently about estimating parameters. The log likelihood does not remove the need for parameter estimation; it only facilitates the process of finding the parameters that maximize the likelihood.

Explanation:

In softmax regression, the average cross-entropy loss measures the difference between the predicted probability distribution and the actual class distribution for each instance. By quantifying how well the predicted probabilities match the true labels, this loss function provides feedback to adjust the model parameters during training. Minimizing cross-entropy loss ensures that the model assigns higher probabilities to the correct classes, improving classification accuracy across multiple classes.

Correct Answer:

It quantifies the similarity between predicted and actual class distributions, aiding in model refinement.

Why Other Options Are Wrong:

It calculates the total number of misclassified instances in the dataset

This is incorrect because cross-entropy loss measures the probability difference, not the raw count of misclassifications. It provides a continuous signal for optimization rather than a discrete error count.

It measures the variance of the predicted probabilities across different classes

This is incorrect because cross-entropy loss does not measure variance; it measures divergence between predicted and true distributions.

It determines the computational efficiency of the model during training

This is incorrect because cross-entropy loss does not control efficiency. It guides model parameter updates to improve predictive accuracy, not computational performance.

Explanation:

In logistic regression, maximum likelihood estimation (MLE) is used to find the parameter values that maximize the likelihood of the observed data under the model. MLE identifies the coefficients that make the observed outcomes most probable, given the input features. Unlike linear regression, which often minimizes the sum of squared errors, logistic regression relies on MLE because the response variable is categorical, and likelihood-based methods are better suited to estimate parameters for probabilistic models.

Correct Answer:

Maximizes the likelihood of the observed data under the model

Why Other Options Are Wrong:

Maximizes the R-squared value of the model

This is incorrect because R-squared is a metric used in linear regression to measure variance explained by the model. Logistic regression does not use R-squared for parameter estimation.

Maximizes the sum of squared errors between the observed predicted values

This is incorrect because maximizing the sum of squared errors would worsen the model fit. Logistic regression does not use sum of squared errors; it relies on likelihood-based methods.

Minimizes the sum of absolute errors between the observed and predicted values

This is incorrect because logistic regression does not minimize absolute errors. Minimizing errors is common in linear regression, but MLE in logistic regression focuses on maximizing the probability of the observed data under the model.

Explanation:

In supervised machine learning, the class label attribute refers to the target variable that the predictive model is designed to predict. It represents the outcome of interest, such as disease presence, customer churn, or a categorical classification. The model uses input features to learn patterns that allow it to accurately predict this label on new, unseen data. Understanding the class label is essential for selecting the correct algorithm and evaluating model performance.

Correct Answer:

The label the predictive algorithm is aiming to predict

Why Other Options Are Wrong:

The label that is most informative for the algorithm to use for prediction

This is incorrect because while informative features help prediction, the class label itself is the target to be predicted, not simply an informative feature. The model uses features to predict the class label, not the other way around.

The majority case indicates the most common label in the data set

This is incorrect because the majority case refers to the most frequent occurrence in the dataset, which may not necessarily represent the class label conceptually. The class label is the target of prediction, independent of frequency.

The minority case that indicates non-independent variables, and not useful for prediction

This is incorrect because the class label is not defined by minority cases or lack of independence. It is the primary target that the model seeks to predict, regardless of its frequency in the dataset.

Explanation:

Supervised learning is a machine learning approach where models are trained using labeled data, meaning each input has a corresponding output or target. The model learns the mapping between inputs and outputs so that it can predict outcomes for new, unseen data. This contrasts with unsupervised learning, which does not use labels and focuses on discovering patterns, structures, or groupings within the data. Supervised learning is fundamental for tasks like regression and classification.

Correct Answer:

A method that uses labeled data to train models for predicting outcomes.

Why Other Options Are Wrong:

A technique that identifies patterns in data without any labels.

This is incorrect because it describes unsupervised learning, not supervised learning. Supervised learning relies on labeled data to guide the model’s predictions.

An approach that focuses solely on clustering similar data points.

This is incorrect because clustering is an unsupervised learning technique. Supervised learning aims to predict outputs rather than group similar data points.

A process that requires no prior knowledge of the data structure.

This is incorrect because supervised learning relies on labeled data to understand the relationship between inputs and outputs. The process does require structured data with known targets.