Space Warps Talk

A classic example of why people are needed rather than robots!

  • Tom_Collett by Tom_Collett scientist

    This is a simulated lens, but it's actually an unrealistic lens. Face on spiral galaxies don't often have enough mass to act as a lens. The fact that this galaxy looks so large suggests it is relatively close to us, and the lensing geometry means that it's harder for nearby objects to act as lenses.

    So why has this got a fake lens around it? I suspect (although I don't know) that there is a robot choosing where to put fake lenses. It picks yellow/orange/red elliptical galaxies and simulates a lensed source. It does this because elliptical galaxies are the most massive galaxies in the universe, and therefore the most likely to be lenses. The robot thinks that the central yellow region (the 'bulge') of the spiral galaxy is an elliptical - but it isn't! You and I can tell that, but the robot can't.

    Posted

  • todash by todash

    I may be being daft but I don't quite follow this... Are you saying that a robot is somehow putting a lens effect into the picture..? Why would they let bots do that to any image on here? Or do you just mean that the bots algorithm has falsely picked this image as potentially having a lens? Sorry if I'm missing something!
    I agree with you though that humans are much better at detection than robots though!

    Posted

  • Tom_Collett by Tom_Collett scientist in response to todash's comment.

    Hi Todash.

    That's exactly what I'm saying!

    There are lots of simulated lenses in spacewarps, this is done for several reasons:
    Firstly it can teach humans to recognise what lenses should look like - you'll see little pop-ups every time you correctly identify (or miss) a simulated lens.
    Secondly it allows the team to quantify how many lenses people miss. The total number of gravitational lenses in the sky can teach us about the universe, but spacewarps is unlikely to find 100% of them. Once spacewarps has been running for a while we will be able to estimate the total number of lenses from the total number of real lenses discovered and the fraction of simulated lenses that are missed.

    So that's why simulated lenses are included. But the simulated lenses need to be realistic so that people are learning the right thing to look for. I'm guessing that a robot has been used to pick how to insert realistic looking lenses, because it would take a lot of time to do it manually. Also I'm pretty sure a human lensing expert would never pick this galaxy to make into a fake lens, but a robot could easily confuse the centre of this spiral galaxy with an elliptical galaxy.

    Posted

  • todash by todash

    Oh - I understand now! Thanks for replying - I didn't realise that you meant the simulated training images (even though you stated that at the start... 😒) ! But yes that would make sense that they are produced by a fancy pants algorithm.

    Posted

  • KhalilaRedBird by KhalilaRedBird

    So if we get tired of encountering the same tacky simulations and ignore them in favor of chasing real lenses, we'll mess up the data spacewarps is collecting? I'm not too sure about the validity of the research protocol here. spacewarps may be running off its potential subjects with frustrating bots.

    Posted

  • drphilmarshall by drphilmarshall scientist, admin in response to KhalilaRedBird's comment.

    Fortunately, the humans in Space Warps are understanding, and forgiving! We are working on improving the sims all the time: the closer we get them to the real thing, the more accurate our selection function will be - so believe me, we are highly motivated. In the meantime, please do bear with us and keep marking the sims! Thanks very much for your help 😃

    Posted

  • drphilmarshall by drphilmarshall scientist, admin

    OK, I started a collection of simulation failure modes at CSWS000crl - if you see a #sim that looks dodgy, please tag it #simfail and I'll pick it up from the keyword collection ( CSWL000003 ). Thanks everyone! 😃

    Posted

  • emilyclr by emilyclr

    I agree with the concerns other have expressed here. Some of the sims are useful, and I've missed my fair share, but most of the sims are downright cartoonish. Unless you switch to using images of real lenses for the sims (why don't you do that?) I don't think you can assume that the proportion of missed sims equals the proportion of missed real lenses, even as a rough estimate..

    Posted

  • emilyclr by emilyclr

    ... in the above post, I meant "why don't you use images of real lenses for the training images?"

    Posted

  • drphilmarshall by drphilmarshall scientist, admin in response to emilyclr's comment.

    One very good reason for not using real lenses for training is that there are not enough of them to go around! Another good reason is that if we only use lenses that we know about for training, there's a good chance we will only find lenses that look like those lenses we already know about - and we might not find anything different! The sims allow us to push a bit further out, albeit in particular directions.

    Dependence on a training set is a fundamental problem when searching for things. How do you find something if you don't know what it looks like? We're hoping that the Space Warps collaboration will do the best possible job of this, because we are all much better equipped (with imaginations!) to spot new types of lens - but even so, we will still be limited by our experience to some degree, and there will be a lot of variance in that experience across the collaboration.

    One thing we are doing is preparing collections of known lenses in the CFHTLS area, to provide another comparison set. We're not quite set up to inject them into Talk yet, but it won't be long. Your suggestion of including them in the training as well is an interesting one. It's more difficult to detect a Marker hit with a real lens, but not impossible. We might be able to get something working.

    However, suppose that we did show all the known lenses to everyone as training images. If they've seen them in Talk as well (it wouldn't take long for a Collection to appear even if we didn't make it ourselves!), do you think it would be fair to assume that the proportion of missed real lenses was equal to the proportion of missed known lenses? I mean, if we learnt them all off by heart, mightn't we still miss new lenses because they aren't the known lenses we know so well from Talk? I'm not sure whether this would be more or less problematic than only showing sims. Interested to hear what you think!

    Regarding the "cartoonish" nature of some of the sims: we are not painting these things in with our fingers, you know! 😉 We draw real galaxy sizes, colours and magnitudes from the CFHTLS catalog, and realize the source behind a model mass distribution that we know fits lenses well. In the few instances of simulation failure that I am referring to above, the catalog entries have been thrown off by some unforeseen complication (bright spiral arms, nearby bright stars etc) - we are working to remove these in future. But while you wait for the known lens collection, believe me: lots of real lenses do actually look like the sims! 😃

    Posted

  • Tom_Collett by Tom_Collett scientist in response to emilyclr's comment.

    @emilyclr makes a good point that we mustn't just use the fraction of simulated lenses that are missed. I was trying to provide a simple explanation earlier - sorry if I pitched it too low.

    We can also analyse the properties of the missed simulations. We'll ask questions like: do the people on spacewarps find certain types of simulations more often than others (E.g. I bet people get the quad quasars more often than the doubles)? does it correlate with the brightness of the images? does the radius of a ring matter? do people miss more lenses at different times of day? (I missed a few at 2am last night - spacewarps is a bit too addictive!)... all of these effects and more will need to be included, if we are to get it right. It wont be easy, but we'll give it a good shot!

    Posted

  • surhudm by surhudm scientist

    Also some of you may think that the images in the sims look too bright. We had to make a choice about how faint we allow the training simulated lenses to be. If we had simulated lenses which were difficult to spot, and gave a feedback that there exists a lens, even though it is difficult to detect, then we would also run the risk of making classifiers too optimistic of what looks like a lens. This will lead to a large number of false positives (i.e., images marked as lenses even if they contain none). That is one of the problems with automated programs which try to identify lenses from images of galaxies.

    The failure of this particular simulation, is a problem of the galaxy group catalog which we started from in order to simulate the lenses. Ideally to get the distance to a galaxy, one measures the red shift of the lines in the spectrum of such galaxies. Unfortunately, taking spectra for all galaxies (especially the faint ones) is expensive. So we have to use the colors of galaxies (ratios of light seen in different wavelengths) as a crude way to get the distance to a galaxy. This particular galaxy has very similar colors in the bulge to more distant galaxies, and it so happens that there is a nearby galaxy (not in this picture) which shares these colors. This is why they were grouped together. In an early version of the simulations we had many such problems, and we tried our best to reduce them, but this one sneaked through all the cuts! This is what happens with large scale simulations, rare things happen.

    We would be very happy to get more of your feedback on the simulations. I will certainly encourage you to go to the spotter's guide to look for the configuration that real lenses have and also compare them to the sims. And do not forget to mark the sims! 😃 It is very essential, so that we can understand the properties of sims that get detected and those that dont!

    Posted

  • astrocheck by astrocheck

    Hi.
    Are we all checking the same set of data then? On a couple of images that i was going to tag, i noticed that someone else had already commented against them (these were Not training images).

    thanks.

    Posted

  • ElisabethB by ElisabethB moderator in response to astrocheck's comment.

    Yes, the goal is having each image looked at by at least 10 people.
    Happy hunting ! 😄

    Posted

  • drphilmarshall by drphilmarshall scientist, admin

    Hi all! New blog post is up, explaining more about the sims: http://blog.spacewarps.org/2013/05/27/sim-city We covered a lot of what's in this post already in this thread - in fact, it was most helpful in helping us write it! Thanks very much, all of you 😃

    Posted