Most Recent iSQI CT-AI Exam Dumps

Back-to-back testing is a method where the same set of tests are run on multiple implementations of the system to compare their outputs. This type of testing is typically used to ensure consistency and correctness by comparing the outputs of different implementations under identical conditions. Let's analyze the options given:

A . Comparison of the results of a current neural network model ML model implemented in platform A (for example Pytorch) with a similar neural network model ML model implemented in platform B (for example Tensorflow), for the same data.

This option describes a scenario where two different implementations of the same type of model are being compared using the same dataset. This is a typical back-to-back testing situation.

B . Comparison of the results of a home-grown neural network model ML model with results in a neural network model implemented in a standard implementation (for example Pytorch) for the same data.

This option involves comparing a custom implementation with a standard implementation, which is also a typical back-to-back testing scenario to validate the custom model against a known benchmark.

C . Comparison of the results of a neural network ML model with a current decision tree ML model for the same data.

This option involves comparing two different types of models (a neural network and a decision tree). This is not a typical scenario for back-to-back testing because the models are inherently different and would not be expected to produce identical results even on the same data.

D . Comparison of the results of the current neural network ML model on the current data set with a slightly modified data set.

This option involves comparing the outputs of the same model on slightly different datasets. This could be seen as a form of robustness testing or sensitivity analysis, but not typical back-to-back testing as it doesn't involve comparing multiple implementations.

Based on this analysis, option C is the one that describes a situation of back-to-back testing the least because it compares two fundamentally different models, which is not the intent of back-to-back testing.

Bias in an AI system occurs when the training data contains inherent prejudices that cause the model to make unfair predictions. Experience-based testing, particularly Exploratory Data Analysis (EDA), helps uncover these biases by analyzing patterns, distributions, and potential discriminatory factors in the training data.

Analysis of the Answer Options:

Option A: ''Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set.''

This is the correct answer. EDA involves examining the dataset for bias, inconsistencies, or missing values, ensuring fairness in ML model predictions.

Option B: ''Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set. If the two models significantly differ, it will indicate there is bias in the original model.''

Back-to-back testing is used for regression testing and to compare versions of an AI system but is not primarily used to detect bias.

Option C: ''Acceptance testing should be used to make sure the algorithm is suitable for the customer. The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present.''

Acceptance testing focuses on meeting predefined business requirements rather than detecting and mitigating bias.

Option D: ''A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model.''

A/B testing is used for evaluating variations of a model rather than for explicitly identifying bias.

ISTQB CT-AI Syllabus Reference:

Bias Testing Methods: 'AI-based systems should be tested for algorithmic bias, sample bias, and inappropriate bias. Experience-based testing and EDA are useful for detecting bias'.

Exploratory Data Analysis (EDA): 'EDA helps uncover potential bias in training data through visualization and statistical analysis'.

Thus, Option A is the best choice for detecting bias in the loan applicant model.

\

Facial recognition applications involve complex computational tasks, including:

Feature Extraction - Identifying unique facial landmarks.

Model Training and Updates - Continuous learning and adaptation of user data.

Image Processing - Handling real-time image recognition under various lighting and angles.

In this scenario, the mobile device is experiencing continuous restarts, which suggests a resource overload caused by excessive processing demands.

Why is Option D Correct?

Mobile devices have limited computational power.

Unlike servers, mobile devices lack powerful GPUs/TPUs required for deep learning models.

On-device learning is computationally expensive.

The model is likely performing real-time learning, which can overwhelm the CPU and RAM.

Continuous feedback transmission may cause overheating.

If the system is running multiple processes---training, inference, and network communication---it can overload system resources and cause crashes.

Why Other Options are Incorrect?

(A) The feedback requires a physical connection and cannot be sent over the Internet. (Incorrect)

Feedback transmission over the internet is common for cloud-based AI services. This is not the cause of the issue.

(B) Mobile operating systems cannot process machine learning algorithms. (Incorrect)

Many mobile applications use ML models efficiently. The problem here is the high computational intensity, not the OS's ability to run ML algorithms.

(C) The size of the application is consuming too much of the phone's storage capacity. (Incorrect)

Storage issues typically result in installation failures or lag, not device restarts. The issue here is processing overload, not storage space.

Reference from ISTQB Certified Tester AI Testing Study Guide

AI-based applications require significant computational power. 'The computational intensity of AI-based applications can pose a challenge when deployed on resource-limited devices.'

Edge devices may struggle with processing complex ML workloads. 'Deploying AI models on mobile or edge devices requires optimization, as these devices have limited processing capabilities compared to cloud environments.'

Thus, option D is the correct answer, as the computational demands of the facial recognition system are too high for the mobile hardware to handle, causing continuous restarts.

Detecting biases in the ML pipeline involves various tests to ensure fairness and accuracy throughout the ML process.

Testing the distribution shift in the training data for inappropriate bias (A): This involves checking if there is any shift in the data distribution that could lead to bias in the model. It is an important test but not the most direct method for detecting biases.

Test the model during model evaluation for data bias (B): This is a critical stage where the model is evaluated to detect any biases in the data it was trained on. It directly addresses potential data biases in the model.

Testing the data pipeline for any sources for algorithmic bias (C): This test is crucial as it helps identify biases that may originate from the data processing and transformation stages within the pipeline. Detecting sources of algorithmic bias ensures that the model does not inherit biases from these processes.

Check the input test data for potential sample bias (D): While this is an important step, it focuses more on the input data and less on the overall data pipeline.

Hence, the most likely useful test to help detect different kinds of biases in the ML pipeline is B . Test the model during model evaluation for data bias.

ISTQB CT-AI Syllabus Section 8.3 on Testing for Algorithmic, Sample, and Inappropriate Bias discusses various tests that can be performed to detect biases at different stages of the ML pipeline.

Sample Exam Questions document, Question #32 highlights the importance of evaluating the model for biases.

AI-based tools can significantly optimize regression test suites by analyzing historical data, past test results, associated defects, and changes made to the software. These tools prioritize and select the most relevant test cases based on previous defect patterns and frequently failing features, which helps in reducing the test execution time while maintaining effectiveness.

The optimization process involves:

Prioritizing test cases: AI-based tools rank test cases based on past defect detection trends, ensuring that the most relevant tests are executed first.

Reducing redundant test cases: The tool can eliminate test cases that do not contribute significantly to defect detection, reducing overall test execution time.

Augmenting test cases: The AI can also suggest new test cases if certain features are more prone to defects.

This approach has been proven to reduce regression test suite sizes by up to 50% while maintaining fault detection capabilities.

Reference from ISTQB Certified Tester AI Testing Study Guide:

Section 11.4 - Using AI for the Optimization of Regression Test Suites states that AI-based tools can optimize regression test suites by analyzing past test data and defect occurrences, leading to significant reductions in test execution time.