In this paper, we propose methods for testing the usability of graphical user interface (GUI) applications based on log files of user interactions. Log analysis by existing methods is not efficient because evaluators analyze a single log file or log files of the same user and then manually compare results. The methods proposed here solve this problem; the methods enable evaluators to analyze the log files of multiple users together by detecting interaction patterns that commonly appear in the log files. To achieve the methods, we first clarify usability attributes that can be evaluated by a log-based usability testing method and user interaction patterns that have to be detected for the evaluation. Based on an investigation on the information that can be obtained from the log files, we extract the attributes of clarity, safety, simplicity, and continuity. For the evaluations of clarity and safety, the interaction patterns that have to be detected include those from user errors. We then propose our methods for detecting interaction patterns from the log files of multiple users. Patterns that commonly appear in the log files are detected by utilizing a repeating pattern detection algorithm. By regarding an operation sequence recorded in a log file as a string and concatenating strings, common patterns are able to be detected as repeating patterns in the concatenated string. We next describe the implementation of the methods in a computer tool for log-based usability testing. The tool, GUITESTER, records user-application interactions into log files, generates usability analysis data from the log files by applying the proposed methods, and visualizes the generated usability analysis data. To show the effectiveness of GUITESTER in finding usability problems, we report an example of a usability test. In this test, evaluators could find 14 problems in a tested GUI application. We finally discuss the ability of the proposed methods in terms of its log analysis efficiency, by comparing the analysis/sequence time (AT/ST) ratio of GUITESTER with those of other methods and tools. The ratio of GUITESTER is found to be smaller. This indicates the methods make log analysis more efficient.
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copy
Hidehiko OKADA, Toshiyuki ASAHI, "GUITESTER: A Log-Based Usability Testing Tool for Graphical User Interfaces" in IEICE TRANSACTIONS on Information,
vol. E82-D, no. 6, pp. 1030-1041, June 1999, doi: .
Abstract: In this paper, we propose methods for testing the usability of graphical user interface (GUI) applications based on log files of user interactions. Log analysis by existing methods is not efficient because evaluators analyze a single log file or log files of the same user and then manually compare results. The methods proposed here solve this problem; the methods enable evaluators to analyze the log files of multiple users together by detecting interaction patterns that commonly appear in the log files. To achieve the methods, we first clarify usability attributes that can be evaluated by a log-based usability testing method and user interaction patterns that have to be detected for the evaluation. Based on an investigation on the information that can be obtained from the log files, we extract the attributes of clarity, safety, simplicity, and continuity. For the evaluations of clarity and safety, the interaction patterns that have to be detected include those from user errors. We then propose our methods for detecting interaction patterns from the log files of multiple users. Patterns that commonly appear in the log files are detected by utilizing a repeating pattern detection algorithm. By regarding an operation sequence recorded in a log file as a string and concatenating strings, common patterns are able to be detected as repeating patterns in the concatenated string. We next describe the implementation of the methods in a computer tool for log-based usability testing. The tool, GUITESTER, records user-application interactions into log files, generates usability analysis data from the log files by applying the proposed methods, and visualizes the generated usability analysis data. To show the effectiveness of GUITESTER in finding usability problems, we report an example of a usability test. In this test, evaluators could find 14 problems in a tested GUI application. We finally discuss the ability of the proposed methods in terms of its log analysis efficiency, by comparing the analysis/sequence time (AT/ST) ratio of GUITESTER with those of other methods and tools. The ratio of GUITESTER is found to be smaller. This indicates the methods make log analysis more efficient.
URL: https://global.ieice.org/en_transactions/information/10.1587/e82-d_6_1030/_p
Copy
@ARTICLE{e82-d_6_1030,
author={Hidehiko OKADA, Toshiyuki ASAHI, },
journal={IEICE TRANSACTIONS on Information},
title={GUITESTER: A Log-Based Usability Testing Tool for Graphical User Interfaces},
year={1999},
volume={E82-D},
number={6},
pages={1030-1041},
abstract={In this paper, we propose methods for testing the usability of graphical user interface (GUI) applications based on log files of user interactions. Log analysis by existing methods is not efficient because evaluators analyze a single log file or log files of the same user and then manually compare results. The methods proposed here solve this problem; the methods enable evaluators to analyze the log files of multiple users together by detecting interaction patterns that commonly appear in the log files. To achieve the methods, we first clarify usability attributes that can be evaluated by a log-based usability testing method and user interaction patterns that have to be detected for the evaluation. Based on an investigation on the information that can be obtained from the log files, we extract the attributes of clarity, safety, simplicity, and continuity. For the evaluations of clarity and safety, the interaction patterns that have to be detected include those from user errors. We then propose our methods for detecting interaction patterns from the log files of multiple users. Patterns that commonly appear in the log files are detected by utilizing a repeating pattern detection algorithm. By regarding an operation sequence recorded in a log file as a string and concatenating strings, common patterns are able to be detected as repeating patterns in the concatenated string. We next describe the implementation of the methods in a computer tool for log-based usability testing. The tool, GUITESTER, records user-application interactions into log files, generates usability analysis data from the log files by applying the proposed methods, and visualizes the generated usability analysis data. To show the effectiveness of GUITESTER in finding usability problems, we report an example of a usability test. In this test, evaluators could find 14 problems in a tested GUI application. We finally discuss the ability of the proposed methods in terms of its log analysis efficiency, by comparing the analysis/sequence time (AT/ST) ratio of GUITESTER with those of other methods and tools. The ratio of GUITESTER is found to be smaller. This indicates the methods make log analysis more efficient.},
keywords={},
doi={},
ISSN={},
month={June},}
Copy
TY - JOUR
TI - GUITESTER: A Log-Based Usability Testing Tool for Graphical User Interfaces
T2 - IEICE TRANSACTIONS on Information
SP - 1030
EP - 1041
AU - Hidehiko OKADA
AU - Toshiyuki ASAHI
PY - 1999
DO -
JO - IEICE TRANSACTIONS on Information
SN -
VL - E82-D
IS - 6
JA - IEICE TRANSACTIONS on Information
Y1 - June 1999
AB - In this paper, we propose methods for testing the usability of graphical user interface (GUI) applications based on log files of user interactions. Log analysis by existing methods is not efficient because evaluators analyze a single log file or log files of the same user and then manually compare results. The methods proposed here solve this problem; the methods enable evaluators to analyze the log files of multiple users together by detecting interaction patterns that commonly appear in the log files. To achieve the methods, we first clarify usability attributes that can be evaluated by a log-based usability testing method and user interaction patterns that have to be detected for the evaluation. Based on an investigation on the information that can be obtained from the log files, we extract the attributes of clarity, safety, simplicity, and continuity. For the evaluations of clarity and safety, the interaction patterns that have to be detected include those from user errors. We then propose our methods for detecting interaction patterns from the log files of multiple users. Patterns that commonly appear in the log files are detected by utilizing a repeating pattern detection algorithm. By regarding an operation sequence recorded in a log file as a string and concatenating strings, common patterns are able to be detected as repeating patterns in the concatenated string. We next describe the implementation of the methods in a computer tool for log-based usability testing. The tool, GUITESTER, records user-application interactions into log files, generates usability analysis data from the log files by applying the proposed methods, and visualizes the generated usability analysis data. To show the effectiveness of GUITESTER in finding usability problems, we report an example of a usability test. In this test, evaluators could find 14 problems in a tested GUI application. We finally discuss the ability of the proposed methods in terms of its log analysis efficiency, by comparing the analysis/sequence time (AT/ST) ratio of GUITESTER with those of other methods and tools. The ratio of GUITESTER is found to be smaller. This indicates the methods make log analysis more efficient.
ER -