Space Warps Talk

Bad Classifications

  • jon273 by jon273

    Is this projects used by scientists in a similar way to Galaxy Zoo, in that it's just really a way to classify images when an example lens is needed?

    The reason I ask this is that I'm wondering how damaging 'false positive' lens classifications actually are? Would a large number make the data considerably less useful?

    I've noticed quite a lot of dubious marks on the main talk page, and I'm pretty sure that I made quite a lot of them myself initially. Maybe it would be worth throwing away the first few positive images a user marks to make the date more consistent?

    I hit ~300 'Potential Lenses' fairly quickly and since then, as I've become more savvy about what to expect, my rate of potentials has dropped considerably.

    It's possible that it's just me, in which case I would throw away my first 300 marked images if were you..

    Posted

  • ElisabethB by ElisabethB moderator

    Hi jon273,

    Remember every image will be looked at by at least 10 different people, so any mistakes will get ironed out !

    And the images are served up at random, so it is not like everybody will see the same image first.

    Don't worry, every click counts, even the first ones !

    Posted

  • Tom_Collett by Tom_Collett scientist

    Hi Jon. Good questions!

    Don't worry - each image gets looked at by ~10 people so false positives from one classifier aren't the end of the world. More importantly, there are plenty of 'maybes' that we do want you to select - we're interesting in finding lens candidates, which we can then follow-up with lens modelling, getting better resolution images and trying to measure the redshift of potential sources. If you're finding more than about 1 candidate (excluding sims) in every 100 images, then that's probably about the right level of optimism. The number of real lenses is probably about ten times less than that, but there are lots of maybes.

    Lots of false positives is bad, but hopefully the fact they are viewed multiple times will average out the overly optimistic classifications. We retire images where almost everyone says 'no lens', and we'll fully investigate images where lots of people say 'lens.' Even in the worst case scenario just pruning out the images with obviously no lenses would be a considerable help. I think the images that are more evenly split will probably get more than 10 classifications once the rest of the dataset has been looked at. If everyone was very optimistic, we'd have a problem but I don't think this is likely to be the case.

    Posted