Machine Learning (DTSC 3220)

Explanation:

In supervised machine learning, regression problems are concerned with predicting a continuous target variable. The goal of regression is to model the relationship between one or more independent variables (features) and a dependent variable that can take on a range of numerical values. Examples include predicting a patient’s blood pressure, a house price, or temperature. Unlike classification tasks, which involve discrete or categorical outcomes, regression focuses on continuous numerical predictions.

Correct Answer:

continuous target variable

Why Other Options Are Wrong:

binary target variables

This is incorrect because binary target variables are associated with classification problems, not regression. Regression requires the output to be a range of numerical values rather than discrete categories.

categorical target variable

This is incorrect because categorical targets are used in classification tasks. Regression problems deal specifically with continuous outcomes rather than categories or classes.

Explanation:

The general approach of data mining is susceptible to overfitting of data. Overfitting occurs when a model captures not only the underlying patterns in the data but also the noise, leading to poor generalization to new, unseen data. Because data mining often explores large, complex datasets with many variables, there is a risk that models will become too finely tuned to the training data. Techniques such as cross-validation, regularization, and holdout datasets are used to mitigate overfitting and ensure the model performs well on unseen data.

Correct Answer:

overfitting of data

Why Other Options Are Wrong:

Too large of data sets

This is incorrect because while large datasets may increase computational requirements, they do not inherently cause errors in data mining. Large datasets can actually improve model generalization if handled properly.

unable to hand open data sets

This is incorrect because it is unclear and not a recognized source of error in data mining. The error being referred to is model-specific, not about dataset accessibility.

holdout data

This is incorrect because holdout data is a technique used to prevent overfitting, not a type of error. It is part of the solution, not the problem.

unsupervised learning

This is incorrect because unsupervised learning is a type of learning approach, not an error. Overfitting can occur in both supervised and unsupervised learning, but it is specifically the susceptibility of the model to fit noise rather than patterns.

Explanation:

The softmax function converts raw class scores (logits) from a model into probabilities that sum to one, allowing interpretation as the likelihood of each class. This is essential in multi-class classification, as it ensures that the outputs form a valid probability distribution over all possible classes. During training, these probabilities are used in conjunction with Cross-Entropy Loss to guide the model in learning correct class assignments.

Correct Answer:

It transforms the class scores into probabilities that sum to one.

Why Other Options Are Wrong:

It computes the mean of the class scores.

This is incorrect because softmax does not compute an average. It applies an exponential transformation and normalization to convert scores into probabilities.

It selects the class with the highest score as the predicted label.

This is incorrect because softmax itself does not select a class. While the class with the highest probability may be chosen during prediction, the softmax function only produces a probability distribution.

It applies a linear transformation to the input features.

This is incorrect because softmax is a nonlinear function applied to the output logits, not a linear transformation of input features. Linear transformations are performed earlier in the model (e.g., in the weight matrix multiplication).

Explanation:

In machine learning, input features are the data provided to the model to make predictions or classifications, such as patient age, blood pressure, or lab results in a healthcare dataset. Model parameters, on the other hand, are the internal variables of the model, like weights and biases, that are learned and adjusted during training to minimize the loss function. The model uses the input features along with these learned parameters to generate predictions. Distinguishing between input features and parameters is essential for understanding how models learn from data.

Correct Answer:

Input features are the data given to the model, whereas model parameters are the internal variables that are learned.

Why Other Options Are Wrong:

Input features are adjustable during training, whereas model parameters are fixed

This is incorrect because input features are fixed data values provided to the model, while model parameters are the ones that are adjusted during training.

Model parameters are manually set before training, whereas input features are learned

This is incorrect because model parameters are learned during training, not manually set. Input features are the raw data provided, not learned variables.

Input features and model parameters are interchangeable terms

This is incorrect because input features and model parameters serve distinct roles; features are the inputs, and parameters are the learnable aspects of the model. They are not interchangeable.

Explanation:

Dimensionality reduction is important in machine learning because it simplifies the dataset by reducing the number of features while retaining the most important information. By reducing complexity, it helps prevent overfitting, where a model learns noise rather than the underlying patterns, and can improve computational efficiency. Techniques like PCA and autoencoders allow models to focus on the most informative aspects of the data, enhancing generalization and making learning more effective.

Correct Answer:

It reduces the complexity of the model and prevents overfitting

Why Other Options Are Wrong:

It always increases the accuracy of clustering algorithms

This is incorrect because dimensionality reduction does not guarantee improved clustering accuracy. While it can help by removing noise, in some cases, reducing dimensions may discard important information, potentially lowering performance.

It eliminates the need for data preprocessing

This is incorrect because dimensionality reduction does not replace other preprocessing steps such as normalization, handling missing values, or encoding categorical variables. Preprocessing is still necessary to prepare the data properly for modeling.

It converts categorical data into numerical data

This is incorrect because dimensionality reduction techniques operate on numerical data and do not inherently convert categorical data into numerical form. Encoding methods like one-hot encoding or label encoding are required for that purpose.

Explanation:

The primary purpose of the Softmax function in a neural network is to convert the raw output scores (logits) of the model into probabilities that sum to one for each class in multiclass classification problems. This allows the network to produce interpretable predictions where each probability reflects the model’s confidence that the input belongs to a particular class. Softmax is critical for enabling the use of probabilistic loss functions like Cross-Entropy Loss during training.

Correct Answer:

To provide probabilities for different classes in multiclass classification problems.

Why Other Options Are Wrong:

To increase the speed of training.

This is incorrect because Softmax does not directly affect training speed. It is used for probabilistic output generation, not for optimizing computational efficiency.

To decrease the complexity of the model.

This is incorrect because Softmax does not reduce model complexity. It is applied to the final output layer and does not simplify the network’s structure or parameters.

To prevent overfitting.

This is incorrect because Softmax has no regularization properties. Overfitting is addressed through techniques like dropout, weight decay, or early stopping, not by the Softmax function.

Explanation:

Logistic regression applied to a problem with two possible outcomes is referred to as binomial logistic regression. This distinguishes it from multinomial logistic regression, which is used for problems with more than two classes. Binomial logistic regression models the probability of one class versus the other using a logistic (sigmoid) function and is a standard method for binary classification tasks.

Correct Answer:

Binomial logistic regression

Why Other Options Are Wrong:

Binomial classification

This is incorrect because while “binomial” refers to two classes, the standard term in statistics and machine learning is “binomial logistic regression,” not binomial classification.

Pairwise logistic regression

This is incorrect because “pairwise” is not a standard term for binary logistic regression. Pairwise comparisons are a different concept unrelated to standard logistic regression nomenclature.

Pairwise classification

This is incorrect because it is not a recognized term in the context of logistic regression. Classification of two classes is specifically called binomial logistic regression.

None of the above

This is incorrect because the correct term is provided among the options: binomial logistic regression.

Explanation:

Clustering is an unsupervised learning technique used to analyze unlabelled data by grouping data points that are similar to each other. This helps reveal hidden structures, patterns, or natural groupings in the data that were not explicitly labeled. Clustering is widely used in exploratory data analysis, customer segmentation, and anomaly detection, providing insights that can guide further modeling or decision-making.

Correct Answer:

It organizes data points into groups based on similarity, revealing underlying structures.

Why Other Options Are Wrong:

It assigns labels to data points based on predefined categories.

This is incorrect because clustering does not rely on predefined categories. It discovers groups based on inherent similarity, making it unsupervised rather than label-dependent.

It predicts outcomes for new data based on historical labeled data.

This is incorrect because prediction based on historical labeled data describes supervised learning, not clustering. Clustering does not use labels to make predictions.

It reduces the dimensionality of the data for easier visualization.

This is incorrect because dimensionality reduction techniques, like PCA, are used for visualization, not clustering. While clustering can be combined with reduced-dimensional data, its primary purpose is grouping based on similarity, not reducing dimensions.

Explanation:

The primary function of logistic regression is to classify data into two distinct categories based on probability. Logistic regression uses the logistic (sigmoid) function to map input features to a probability between 0 and 1, allowing the model to predict binary outcomes such as the presence or absence of a disease. This probabilistic framework enables not only classification but also assessment of confidence in predictions, making it widely applicable in fields like healthcare, finance, and risk analysis.

Correct Answer:

To classify data into two distinct categories based on probability

Why Other Options Are Wrong:

To predict continuous outcomes based on input features

This is incorrect because predicting continuous outcomes is the role of linear regression, not logistic regression. Logistic regression is used for categorical, specifically binary, outcomes.

To perform clustering of unlabeled data points

This is incorrect because clustering is an unsupervised learning technique, whereas logistic regression is a supervised learning method that requires labeled data.

To reduce dimensionality of datasets

This is incorrect because dimensionality reduction is performed by techniques such as Principal Component Analysis (PCA), not logistic regression.