Question

what is a double-blind test? how do we use this test objectively to compare the goodness of two software systems? ex) google vs bing

Answer 1

A Double-blind test is a type of experiment in which two entities, the subject and the observer are not aware that the exercise or process in practice is a test. These tests are generally used in practical testing, medical science experiments, software engineering, psychology etc. This test or experiment is an experimental method which is used to ensure impartiality and avoid errors which may arise from bias.

For ensuring computer security, testing is needed to be done on the system. The testing involved here in computer security is known as penetration testing. It is done by either single or double blind testing.In Single blind penetration testing, their is very limited given information. The team do the testing process with that limited information and once they find and assigned a target orgnization, the team search and find information from avavilable resources.The testing process help the team to find and discover vulnerabilities that exploit the software. They sometimes draw on UNSENET company websites, their forumns, domain name registration etc to find the possible vulnerabilities. In Double blind penetration test, the testing is not prepared and don't have much information about the test. But here even the defendind organization's IT team also doesn't have any preparation or knowledge about the test and is not informed in advance. This lack of knowledge in double blind test helps the team avoid biases and detect pther effects that might influence results such as placebo effect. In that effect, change in one belief that is being treated has an effect on results. Double-blind test is very effective in simulation of real world cyber attack.

Like in medical science. In medical clinical trials, a subject receiving the treatment doesn't know that they are actually being treated or is under placebo. It is single blind test.But if the doctor or researcher does not know the patient or patients are receiving treatment. It is a type of double blind test

For example : A blind search was done on both google and bing. Microsoft conducted a study where they deduce that search engine branding had an impact on quality of results. So, when bing launched, same test was performed on it and researchers started ranking the quality of search results in bing. Double-blind challenge was conducted to find the quality of search results in both google and bing.After several tests and updates test, some tests revealed that Google secured better search quality ratings in these tests. Google uses real life user simulation for its search operations. Although perfect or best search engine depends upon how user chooses and uses the search engine.