Machine Learning (DTSC 3220)

Explanation:

In gradient descent, updates to the model parameters continue iteratively until a stopping criterion has been met. Stopping criteria can include reaching a maximum number of iterations, achieving a sufficiently small change in the loss function between iterations, or when the gradient magnitude falls below a predefined threshold. These criteria ensure that the algorithm stops when further updates no longer meaningfully improve the model, rather than continuing indefinitely or stopping prematurely.

Correct Answer:

When a stopping criterion has been met

Why Other Options Are Wrong:

When the gradient's direction flips

This is incorrect because the direction of the gradient can change during optimization without indicating convergence. Gradient descent relies on following the gradient to minimize the loss, so occasional direction changes are normal.

Wen the model has arrived in a local minimum of the loss function

This is incorrect because the algorithm may stop before or after reaching a local minimum depending on the stopping criteria. While a local minimum may be reached, gradient descent does not explicitly stop only at local minima.

When the model fits all points

This is incorrect because perfectly fitting all points is often impossible and unnecessary, especially in cases of noisy data. Overfitting can occur if the model tries to fit every data point exactly, which is not the goal of gradient descent.

Explanation:

Unsupervised learning focuses on analyzing data that lacks labeled outcomes. The primary goal is to uncover hidden structures, patterns, or groupings within the dataset. Techniques such as clustering, dimensionality reduction, and association rule learning are commonly used in unsupervised learning to reveal insights and relationships in unlabeled data. Unlike supervised learning, it does not rely on predefined categories or labeled training examples.

Correct Answer:

To identify hidden patterns or groupings within unlabeled data

Why Other Options Are Wrong:

To predict outcomes based on labeled training data

This is incorrect because predicting outcomes from labeled data is the focus of supervised learning, not unsupervised learning.

To classify data into predefined categories

This is incorrect because unsupervised learning does not use predefined categories; it identifies groupings based on inherent similarities in the data.

To optimize model parameters using labeled examples

This is incorrect because parameter optimization with labeled examples is part of supervised model training, not the objective of unsupervised learning.

Explanation:

The objective function, also known as the loss function, is a critical component in machine learning optimization. It quantifies how well the model’s predictions align with the actual target values. During training, the model parameters are adjusted to minimize (or maximize, depending on context) this function. By measuring the model’s performance, the objective function guides the optimization process, ensuring that the model improves accuracy and generalization over time.

Correct Answer:

It measures the performance of the model

Why Other Options Are Wrong:

It defines the model architecture

This is incorrect because the model architecture determines the structure of the neural network or algorithm, not the objective function. The objective function evaluates performance rather than designing the model.

It selects the features for training

This is incorrect because feature selection is a separate process from the objective function. The objective function does not determine which inputs to use; it evaluates model predictions based on given features.

It determines the training dataset size

This is incorrect because the objective function does not control how much data is used for training. Dataset size is an independent decision related to data preparation, not performance measurement.

Explanation:

The primary advantage of unsupervised learning algorithms is their ability to identify patterns in data without the need for labeled examples. This makes them particularly useful for exploring unknown datasets, discovering inherent structures, and grouping similar data points. Algorithms like clustering and association analysis allow data scientists to extract insights, detect anomalies, and summarize complex datasets even when no target outcomes are provided.

Correct Answer:

Identifying patterns in data without labeled examples

Why Other Options Are Wrong:

Making predictions with labeled data

This is incorrect because making predictions with labeled data is the domain of supervised learning, not unsupervised learning.

Classifying data into multiple categories

This is incorrect because classification into predefined categories typically requires labeled data, which is a supervised learning task. Unsupervised learning can cluster data, but it does not use predefined labels.

Reducing the dimensionality of data

This is incorrect because while some unsupervised techniques like PCA can reduce dimensionality, this is a specific application rather than the primary advantage of unsupervised learning as a whole.

Explanation:

Feature construction in machine learning refers to the process of creating new features from existing data to enhance model performance. By transforming, combining, or deriving new variables, feature construction can reveal patterns and relationships that were not obvious in the original dataset. Well-constructed features can improve predictive accuracy, reduce bias, and enable models to capture underlying complexities in the data. This process is a critical step in feature engineering, often having a larger impact on model performance than algorithm selection.

Correct Answer:

The process of creating new features from existing data to improve model performance

Why Other Options Are Wrong:

The process of selecting the best algorithm for a machine learning model

This is incorrect because feature construction focuses on creating features, not on algorithm selection, which is a separate step in model development.

The method of constructing a training dataset from raw data

This is incorrect because constructing a training dataset involves data collection and preprocessing, whereas feature construction specifically transforms or derives features to enhance model learning.

The technique of reducing the dimensionality of the feature space

This is incorrect because reducing dimensionality is typically performed by techniques like PCA or feature selection, not feature construction. Feature construction usually creates new features rather than reducing them.

Explanation:

The primary advantage of using a mini-batch in gradient descent is that it allows for faster convergence by updating the model weights more frequently. Instead of computing the gradient over the entire dataset (as in batch gradient descent), mini-batch gradient descent computes the gradient on smaller subsets of data, which enables the model to make updates more often. This frequent updating can lead to quicker convergence and can help the model escape shallow local minima, improving training efficiency and performance.

Correct Answer:

It allows for faster convergence by updating weights more frequently.

Why Other Options Are Wrong:

It eliminates the need for a learning rate

This is incorrect because mini-batch gradient descent still requires a learning rate to determine the size of each weight update. The learning rate is an essential hyperparameter that controls convergence speed and stability.

It guarantees finding the global minimum

This is incorrect because mini-batch gradient descent does not guarantee reaching the global minimum. The algorithm may converge to local minima or saddle points, depending on the loss surface and learning rate.

It requires less memory than storing the entire dataset

This is incorrect because the main memory advantage comes from not needing to load the entire dataset at once, but the key advantage emphasized in the context of convergence is the more frequent weight updates, not just memory efficiency.

Explanation:

A feature vector in machine learning is a set of attributes that represents a single data instance. Each element in the feature vector corresponds to a measurable property or characteristic of the data, such as age, weight, or blood pressure in a medical dataset. Feature vectors are used as input to machine learning algorithms, allowing models to learn patterns and relationships between the input features and the target variable. They are fundamental in both supervised and unsupervised learning tasks.

Correct Answer:

A set of data instances represented by attributes

Why Other Options Are Wrong:

A machine learning model

This is incorrect because a model is the algorithm or structure that learns patterns from data, not the input representation. The feature vector is part of the input, not the model itself.

A specific type of algorithm

This is incorrect because a feature vector is a representation of data, not a type of algorithm. Algorithms use feature vectors as input for learning or prediction.

A prediction made by a classifier

This is incorrect because predictions are outputs generated by a model based on feature vectors. A feature vector represents the input, not the predicted outcome.

Explanation:

The binary indicator function I(yi = k) is used in cross-entropy loss to identify whether the true class label for a given data point matches a specific class k. It takes a value of 1 if the actual label yi equals k and 0 otherwise. This ensures that the loss calculation only considers the predicted probability of the correct class, penalizing the model when the predicted probability for the true class is low. The indicator function is critical for correctly computing the contribution of each data point to the overall loss.

Correct Answer:

It signifies whether the actual class label corresponds to a designated class k, impacting the loss computation.

Why Other Options Are Wrong:

It determines the probability of the predicted class for a given data point.

This is incorrect because the indicator function does not calculate probabilities; it only indicates whether a class is the true label. Probabilities are computed separately by the model, such as through softmax.

It calculates the average of the predicted probabilities across all classes.

This is incorrect because the indicator function does not perform averaging. It is used to select the contribution of the true class in the cross-entropy loss formula.

It adjusts the learning rate during model training.

This is incorrect because the indicator function has no role in setting or modifying the learning rate. Its purpose is solely to indicate the true class in loss computation.

Explanation:

Supervised learning is a machine learning approach where models are trained using labeled data, meaning each input has a corresponding output or target. The model learns the mapping between inputs and outputs so that it can predict outcomes for new, unseen data. This contrasts with unsupervised learning, which does not use labels and focuses on discovering patterns, structures, or groupings within the data. Supervised learning is fundamental for tasks like regression and classification.

Correct Answer:

A method that uses labeled data to train models for predicting outcomes.

Why Other Options Are Wrong:

A technique that identifies patterns in data without any labels.

This is incorrect because it describes unsupervised learning, not supervised learning. Supervised learning relies on labeled data to guide the model’s predictions.

An approach that focuses solely on clustering similar data points.

This is incorrect because clustering is an unsupervised learning technique. Supervised learning aims to predict outputs rather than group similar data points.

A process that requires no prior knowledge of the data structure.

This is incorrect because supervised learning relies on labeled data to understand the relationship between inputs and outputs. The process does require structured data with known targets.