Evaluating Software Usability – Think it out loud

“Think it out loud” methodology – user verbalises their through processes. Useful, for example, when user is navigating a complex UI and they have some familiarity with it and are perhaps performing one than more operation at the same time. This HCI (Human-Computer Interface) evaluation process evolved from design based approaches to producing qualitative data as part of observational process. […]

Potential problems gathering information by observation

Practice To be used effectively, observation requires great deal of practice. Observer presence Presence of observer will affect behaviours, therefore disturbances must be minimised. Objectivity Difficult to remain objective – therefore ethical – when asking detailed questions to enhance understanding. Element of chance Chance event occurring may be taken as a regular occurrence if observations not taken over prolonged period […]

Collecting Data and Information – Observation Techniques

Interviews and questionnaires valuable ways to gather information. Both reliant on accuracy & honesty of answers relating to persons behaviour and beliefs which is a weakness with both approaches. Observation is alternative technique. Observe behaviours first-hand and analyse by quantitative or qualitative means. Observation used by everyone. Examples: Consumer comparing prices in retail outlets before buying Social planners monitoring traffic […]

Designing data collection methods for computational analysis

For results to be suitable for computational analysis it is usual to design responses so they can be easily coded. Enter a code (numeric/alpha-numeric) entry is simplified as is analysis and interpretation. Simplest form of coding is binary for yes/no questions. Yes represented by 1 and No represented by 0. Hence a series of eight questions may result in 10110010 […]

Pilot testing

Having designed questionnaire/survey it is necessary to carry out pilot test/study. This enables any defects/problems/ambiguities in structure and questions, including leading questions, to be identified before it is circulated to full sample of respondents. Pilot study undertaken in two phases: Asking colleagues for their opinion Using sample population from intended population. Would be unethical to run pilot study with group […]

Internet or Intranet surveys

The Internet and internal networks provide a means to gather responses from a far wider audience than traditional methods in a shorter timeframe. Technology however has ethical considerations such as potentially excluding respondents who do not have access to the Internet, limiting respondents to only those with access to the network such as students or co-workers. This in turn can […]

Collecting Data and Information – Questionnaires

Questionnaires form part of a survey – used to collect qualitative and/or quantitative data. Ethical questions are not leading (see earlier example of “What is your religion?”). Can ask open-ended questions to avoid bias – “What do you think of the website?” – encourages feelings or ideas to be expressed. However, open-ended questions can be time-consuming, responses are hard to […]


Research method needs both practical & ethical consideration. Will method lead to appropriate/valid results or will it bias them? The questions posed can lead respondents to provide certain answers, e.g. “What is your religion?” – to avoid leading should be asked – “Do you have a religion?” followed by “If so, what is it?”. Ethical consideration with the above would […]


Some topics more dubious than others e.g. hacking or causing deceit. Some areas such as banking present hurdle which will not likely be overcome when trying to gather data or even permission to gather data. Topics that will not get approval include those related to pornography, terrorism or criminal activity.