Comments on: Why I Think the New “Pop-Up” Study is Flawed

By: brando the Great

brando the Great — Mon, 06 Oct 2008 07:48:15 +0000

Can we drag these people out in the street and shoot them?

By: kouji

kouji — Thu, 25 Sep 2008 04:43:18 +0000

i think it would also depend on what they were doing on those computers. after all, if i was working on a paper or thesis, i'd be very careful, even if i wasn't on my own system. kouji's last blog post..haiku poem: dry

By: Andy Beal

Andy Beal — Wed, 24 Sep 2008 19:42:12 +0000

@Graham – thanks for your thoughts. Have you read the full study? I’d like to know the actual hypothesis they were testing. Do you know, or are we all just guessing?

By: Graham Jones - Internet Psychologist

Graham Jones - Internet Psychologist — Wed, 24 Sep 2008 19:36:47 +0000

Andy, I think the study is only partially flawed. If the study was designed to show that people ignore potential indications of malware, then, yes, it would have been more appropriate to do that on computers in which the participants had some kind of stakeholding. So doing such a study on departmental PCs would invalidate the results.

However, the study does not seem to have had this as its aim. What the study appears to be be aimed at is the dislike people have for the standard windows dialogs. The study was mainly looking at response to dialog boxes.

The “give-aways” as to potential malware are actually only give-aways to technically savvy people.

What the study showed is that dialog boxes are largely ignored – and that’s why the malware producers succeed. It’s actually a study that tells us that Microsoft’s products are too focused on “engineers” and not on real people. Those of us who are not engineers are not in the least interested in the messages – so we just click OK to get rid of the interruption.

That’s how the malware producers have tapped into us – they are much more aware of human behaviour than Microsoft seems to be.

In essence, the study tells us what we already know – we hate interruptions to what we are doing (which explains why most advertising doesn’t work as well).

By: sheppy

sheppy — Wed, 24 Sep 2008 19:13:40 +0000

After reading the article, I'm a bit confused. So the users were put in a situation where they were told they were going to be asked questions about how a website loaded? If I was on an unfamiliar computer with a different objective to using the website, I'd probably move the box out of the way, if I thought it was just something that wasn't supposed to happen. I know that my statement above doesn't qualify, as you'd tend to ignore what people say they'd do rather than what people actually do! sheppy's last blog post..The Eagle has landed…

By: Andy Beal

Andy Beal — Wed, 24 Sep 2008 17:31:38 +0000

@David – thanks for providing some official feedback. As I said, I’m not an expert, but in addition to what I stated, here are some other observations:

1. I’m assuming the study group knew they were taking part in some kind of study? That would likely skew the results.
2. 42 is a small sample. Again, I’m assuming these were all college students. Not exactly a representation of web users.
3. In addition, there were 4 different scenarios, correct? Divided-up by 42, means that only 10 students per scenario.

I could go on, but here’s the thing. I haven’t read your paper, so I don’t know what your conclusions are. You may only claim that college kids are dumb enough to click on fake pop-ups and make no effort to extrapolate that to the general population. Also, your follow-up study may well have the same result. However, based on the information I have to hand, I do believe the study is flawed–at least, when we try to apply its results to the general web population.

Just my thoughts.

Go Pack!

By: KeeKee

KeeKee — Wed, 24 Sep 2008 16:39:58 +0000

I don't think the fact that it's not the students own computer makes much difference. Even if the research were to be done on their own computers, a lot of students have the mentality that 'Mommy & Daddy' will just buy me another computer so they still hold no responsibility. KeeKee KeeKee's last blog post..Madrid Spain

By: David Sharek

David Sharek — Wed, 24 Sep 2008 15:41:34 +0000

In research like this, it’s important to take a ‘first stab’ to see if the topic is worth investigating further. Since we got significant results in this study, the next step would be to add personal affect, to see if there was an effect of a diffusion of responsabilty. My guess is that we will find similar results, but you are absolutely correct in thinking that the research needs to be taken a step further. To say it is flawed, though is not quite accurate.

By: Joe Hall

Joe Hall — Wed, 24 Sep 2008 15:34:54 +0000

I think you are right, people tend to take more care over their own machine. Especially, when compared to the machines in public university labs. I have personally seen people do things on the keyboards in universies' labs that I would never do anywhere. I have seen students repeatedly down load software that is overtly adware/spyware. The general assumption that many students have is that the IT department will fix it. Joe Hall's last blog post..Links are making people go crazy!