Most Recent iSQI CT-AI Exam Questions & Answers

AI test environments have several unique characteristics that differentiate them from traditional test environments. Let's evaluate each option:

A . Challenges resulting from low accuracy of the models.

Low accuracy is a common challenge in AI systems, especially during initial development and training phases. Ensuring the model performs accurately in varied and unpredictable scenarios is a critical aspect of AI testing.

B . The challenge of mimicking undefined scenarios generated due to self-learning.

AI systems, particularly those that involve machine learning, can generate undefined or unexpected scenarios due to their self-learning capabilities. Mimicking and testing these scenarios is a unique challenge in AI environments.

C . The challenge of providing explainability to the decisions made by the system.

Explainability, or the ability to understand and articulate how an AI system arrives at its decisions, is a significant and unique challenge in AI testing. This is crucial for trust and transparency in AI systems.

D . Challenges in the creation of scenarios of human handover for autonomous systems.

While important, the creation of scenarios for human handover in autonomous systems is not a characteristic unique to AI test environments. It is more related to the operational and deployment challenges of autonomous systems rather than the intrinsic technology-related characteristics of AI .

Given the above points, option D is the correct answer because it describes a challenge related to operational deployment rather than a technology-related characteristic unique to AI test environments.

A. Using a pixel comparison of the GUI before and after the change to check the differences.

Pixel comparison is a traditional method and does not involve AI . It compares images at the pixel level, which can be effective but is not an intelligent approach. It is not considered an AI usage and is the least likely usage of AI for detecting GUI changes.

B. Using computer vision to compare the GUI before and after the test object changes.

Computer vision involves using AI techniques to interpret and process images. It is a likely usage of AI for detecting changes in the GUI .

C. Using vision-based detection of the GUI layout changes before and after test object changes.

Vision-based detection is another AI technique where the layout and structure of the GUI are analyzed to detect changes. This is a typical application of AI .

D. Using a ML-based classifier to flag if changes in GUI are to be flagged for humans.

An ML-based classifier can intelligently determine significant changes and decide if they need human review, which is a sophisticated AI application.

The question asks which test is least likely to be performed during the ML model testing phase. Let's consider each option:

Testing the accuracy of the classification model (A): Accuracy testing is a fundamental part of the ML model testing phase. It ensures that the model correctly classifies the data as intended and meets the required performance metrics.

Testing the API of the service powered by the ML model (B): Testing the API is crucial, especially if the ML model is deployed as part of a service. This ensures that the service integrates well with other systems and that the API performs as expected.

Testing the speed of the training of the model (C): This is least likely to be part of the ML model testing phase. The speed of training is more relevant during the development phase when optimizing and tuning the model. During testing, the focus is more on the model's performance and behavior rather than how quickly it was trained.

Testing the speed of the prediction by the model (D): Testing the speed of prediction is important to ensure that the model meets performance requirements in a production environment, especially for real-time applications.

ISTQB CT-AI Syllabus Section 3.2 on ML Workflow and Section 5 on ML Functional Performance Metrics discuss the focus of testing during the model testing phase, which includes accuracy and prediction speed but not the training speed.

Defect prediction models aim to identify parts of the software that are likely to contain defects by analyzing historical data and code quality metrics. The primary goal is to use this predictive information to allocate testing and maintenance resources effectively. Let's break down why option D is the correct choice:

Understanding Classification Models:

Classification models are a type of supervised learning algorithm used to categorize or classify data into predefined classes or labels. In the context of defect prediction, the classification model would classify parts of the code as either 'defective' or 'non-defective' based on the input features.

Input Data - Code Quality Metrics:

The input data for these classification models typically includes various code quality metrics such as cyclomatic complexity, lines of code, number of methods, depth of inheritance, coupling between objects, etc. These metrics help the model learn patterns associated with defects.

Historical Data:

Historical versions of the code along with their defect records provide the labeled data needed for training the classification model. By analyzing this historical data, the model can learn which metrics are indicative of defects.

Why Option D is Correct:

Option D specifies using a classification model to predict the presence of defects by using code quality metrics as input data. This accurately describes the process of defect prediction using historical bug data and quality metrics.

Eliminating Other Options:

A . Identifying the relationship between developers and the modules developed by them: This does not directly involve predicting defects based on code quality metrics and historical data.

B . Search of similar code based on natural language processing: While useful for other purposes, this method does not describe defect prediction using classification models and code metrics.

C . Clustering of similar code modules to predict based on similarity: Clustering is an unsupervised learning technique and does not directly align with the supervised learning approach typically used in defect prediction models.

ISTQB CT-AI Syllabus, Section 9.5, Metamorphic Testing (MT), describes various testing techniques including classification models for defect prediction.

'Using AI for Defect Prediction' (ISTQB CT-AI Syllabus, Section 11.5.1).

When written requirements are given in text documents, the best way to generate test cases is by using Natural Language Processing (NLP). Here's why:

Natural Language Processing (NLP): NLP can analyze and understand human language. It can be used to process textual requirements to extract relevant information and generate test cases. This method is efficient in handling large volumes of textual data and identifying key elements necessary for testing.

Why Not Other Options:

Analyzing source code for generating test cases: This is more suitable for white-box testing where the code is available, but it doesn't apply to text-based requirements.

Machine learning on logs of execution: This approach is used for dynamic analysis based on system behavior during execution rather than static textual requirements.

GUI analysis by computer vision: This is used for testing graphical user interfaces and is not applicable to text-based requirements.