ID'ing observations for other... Paper on species ID's by experts and non-experts...

So, I was glancing at a few papers today, and I spotted this one:

Species identification by experts and non-experts: comparing images from field guides. G. E. Austen, M. Bindemann, R. A. Griffiths & D. L. Roberts -- Scientific Reports 6, Article number: 33634 (2016)

Abstract: Accurate species identification is fundamental when recording ecological data. However, the ability to correctly identify organisms visually is rarely questioned. We investigated how experts and non-experts compared in the identification of bumblebees, a group of insects of considerable conservation concern. Experts and non-experts were asked whether two concurrent bumblebee images depicted the same or two different species. Overall accuracy was below 60% and comparable for experts and non-experts. However, experts were more consistent in their answers when the same images were repeated, and more cautious in committing to a definitive answer. Our findings demonstrate the difficulty of correctly identifying bumblebees using images from field guides. Such error rates need to be accounted for when interpreting species data, whether or not they have been collected by experts. We suggest that investigation of how experts and non-experts make observations should be incorporated into study design, and could be used to improve training in species identification.

Anyways, I was thinking about the ID system here on iNat. I really enjoy ID'ing observations for folks -- I try to focus on the stuff from TX, but every now and then I try to tackle some observations from outside of TX.

I've been wrong hundreds/thousands of times, probably. My ego's not too too massive -- it's ok that I've been wrong! I do learn a lot from mistakes and from being corrected. And as any natural history collection (digital or physical) goes, each mistake doesn't diminish the integrity of the database -- the mistakes are eventually caught... "Eventually" can mean a long time though. That's the nature of collections! :)

The more folks that ID, the stronger the database gets too. Even if it's verifying an observation that's already been verified/research grade, that's a valuable annotation, in my mind. It's a safe-guard against some of the new folks to iNat that give a more general ID and change the taxon as well.

Tagging some of my favorite ID'er to see if you'd like to look at that paper too.

@greglasley @carrieseltzer @gcwarbler @d_kluza @mako252 @maractwin @aguilita @kueda @kevinhintsa @john8 @loarie @lisa_bennett @nlblock @susanhewitt @cosmiccat @charlie @silversea_starsong @glmory @robberfly @borisb @nathantaylor7583 @muir

Posted on 03 February, 2017 03:45 by sambiology sambiology

Comments

That's a rough subject -- Bombus are very difficult to identify, even at the best of times. On the other hand, good for raising awareness that we all need to improve. Being a naturalist is a constantly evolving process...

Posted by silversea_starsong over 7 years ago

Good piece. It furthers my belief that when I try to on board someone to iNaturalist that I talk of not only getting your stuff ID'd and contributing to the global database, but that its important to be involved in identifying stuff you know and slowly grow into stuff you don't know as it helps to hone one's skills when you can't be in the field and allows you to pick up key traits on organisms that you might not see on a regular basis. I was not a good contributor to IDs when I first started, but the past year I've been trying to be better about it and it definitely has helped me become more aware of what to look for in the field. It also makes the platform feel more like a community circle rather than one way streets when your involved on both sides of the equation.

Posted by damontighe over 7 years ago

interesting - thanks for sharing

Posted by loarie over 7 years ago

Very interesting, and I agree with you on many points! I find my (frequent) errors help me learn too, and I often go back and try and see how I made the error. I'm not an 'expert' of many taxa, but I find the more exposure you get to photos/ real life organisms the more skilled you get at ID'ing them. Some groups will always be hard/impossible to ID from photographs, but I still think there is value in the observations even if they can't be confirmed at species level.
Also, I'm quite chuffed to be included in your favourite IDers :)

Posted by lisa_bennett over 7 years ago

Excellent work. I am a retired pathologist. We have done many studies on to what degree pathologists offer the same diagnosis from an identical set of slides. The ID is improved with more participants.

P.S. Always get a second opinion on any very important medical diagnosis.

Posted by denniswross over 7 years ago

Awesome! with plants: some you can tell with any photo from a half mile away (if you know the habitat, etc), some you can tell only with good photos, and some you can't tell with photos at all. One of the hard parts is knowing that. Also figuring out what to do when you hvae two cryptic species and 99.9% of the tie it is the same one. What margin of error do you accept? On the one hand there is possible false data but on the other hand most of it is good solid stuff.

Overall, other than a few student projects that teachers don't adequately curate, the research grade observations for plants on iNat seems to have an accuracy similar to that of a field crew of trained or semi-trained botanists making a photoless plant list. Mistakes happen but overall it's really good. And the photos let you fix old mistakes which you can't do on a plant list. I've seen many old plots and thought really? no way that plant was there... but you never know without a photo.

Posted by charlie over 7 years ago

Interesting but not too surprising. I'm going to be plunging into my bumblebee backlog soon for iNat and a couple other projects. Now that I've taken a look at the identification guide I've gotten from the library I'm certain many if not most will remain at the genus level.

Posted by driftlessroots over 7 years ago

Just in general, not about bumblebees in particular, I think one thing that a lot of people don't realize is the following.

When an observation has no ID, or only an extremely general ID, if you can ID the organism to family level with confidence, please go ahead and put the family in as an ID!

I think a lot of people (especially new people) feel they should only ID to the species level or not at all. But when you put a family level ID into an observation which previously only had phylum or class, the local person may be able to look that family up in a local felid guide and then work out which species they saw, or maybe at least which genus it was.

If you know a group well, you may be able to ID to family level for observations in parts of the world that you have never experienced yourself.

Posted by susanhewitt over 7 years ago

That should be "field guide" not "felid guide", unless it is cats you are trying to ID!

Posted by susanhewitt over 7 years ago

Great article but as @charlie and others have mentioned above, there is great variability in the ID potential of many different groups of plants, animals, etc. charlie's comment about plants you can ID from half a mile in a photo, or others that cannot be identified from a photo at all is also true for the groups I have a little knowledge of, birds, dragonflies and butterflies. As far as bees....I can tell a bee from a hawk but I'd probably not push my ID skills of bees much beyond that. I often find it amusing/actually a little frustrating, that folks who are really very new to any sort of nature observations will make an incorrect ID of something and post it on iNat and a number of folks who have a broad level of knowledge and expertise on the subject will come along and correct it, yet the initial observer sometimes simply sticks by their original ID (even though it is totally wrong) and refuses to take advantage of the information they have been offered by others. But, I guess that is the way a community forum such as this has to work. We all make mistakes, and I do it all the time, and as @silversea_starsong commented, being a naturalist is a constantly evolving (and learning) process. As was stated in the article you cite, the longer I have been at this, the more I have learned to be cautious with many IDs. (as @sambiology knows, I am reluctant to ID most winter meadowlarks, most Empidonax, and others). Some folks open their first field guide and suddenly are experts. Oh well. But I guess I was that way when I started as well.
There are lots of bird folks on iNat and a fair number of dragonfly and butterfly folks, so much of what I can contribute to IDs is really redundant. I will occasionally find a bird record with two or three "agrees" on an initial ID that is, in fact, incorrect and I can offer reasons why, and usually folks appreciate that. But if not, that is fine too. I really wish iNat could attract more knowledgeable folks on moths (the few that are on iNat get overwhelmed with the volume of moth records and they just can't keep up). Other groups, also are under-represented with knowledgeable people, but hopefully as iNat continues to grow, we will continue to attract more people with expertise and knowledge in many different areas. But I have to say that iNat has offered me a great learning experience, more than I ever would have anticipated when I first came on board 4 years ago. I gain knowledge all the time about different species of mammals (and other groups), distribution records, seasonal variations, etc., that I was unaware of before, so even to an old fart who has been at this game a long time, I learn from iNat every single day.

Posted by greglasley over 7 years ago

I've seen this paper before and agree with it completely. I remember how many Euphorbia specimens Barton Warnock misidentified and that is example enough for me. Marshall Johnson corrected some of these but also had a few extra misidentifications in there place. Billy Turner corrected most of what was left, but left plenty of misidentifications for me to find and correct. I really hope I haven't misIDed too many specimens, but I find it inevitable. In addition to this, I remember seeing a paper studying the phylogeny of the Hawaiian members of sect. Anisophyllum. The researcher was unable to get specimens (because the plants are listed) and so she relied on the "field botanists" to obtain DNA. Many individuals within the same species ended up on opposite ends of the phylogeny. Also, without the vouchers to reference back to, the validity of the paper cannot be confirmed. However, I can't help but feel that many of these were misidentified. This problem is important to take into account and why it is important to voucher. If there is a specimen to refer back to, at least your work can be corrected. If there isn't, it's very difficult to determine where the mistakes were made and if there are any others that will be made.

Posted by nathantaylor over 7 years ago

@susanhewitt I completely agree. Most observations that ID to family from the more general plants are identified to genus or species the following day! It really does speed up the identification process. However, I have noticed that the way iNaturalist "perceives" a broader ID is as a conflicting ID, so I usually take mine down if I don't know the species after two people have IDed the observation to species. If I don't, the community ID may stay at family level for a considerable amount of time and not get to research grade as quickly as it should.
I am curious to know what other people's opinions are on this, though.

Posted by nathantaylor over 7 years ago

I'll add a little to the comments by @nathantaylor7583 from my own experiences. I spent two years helping to reorganize the odonate collection at the Univ. of Texas at Austin when Dr. John Abbott was the curator there. (He is now at the Univ. of Alabama). Anyway, I was adding bar graph cards to each specimen envelope and thus I handled every specimen as I was doing this. I'm not sure of the total number of odonates in the collection, but somewhere near 20,000 is a fair guess. Over two years I found at least 150 misidentified specimens and with John's approval, corrected the IDs. Not a big percentage of errors, but it showed me that mistakes can show up anywhere.

Posted by greglasley over 7 years ago

@greglasley i think a lot of times when people don't fix an incorrect ID it is because they aren't on the site any more. A lot of time it is a case where students were assigned to use iNat under 'duress' but one the assignment is over they leave. They don't curate their data and often the teacher doesn't either. And sometimes it's marked as rejecting community ID so a bad observation that is obviously wrong is stuck there forever. A little off topic but... something to consider moving forward

Posted by charlie over 7 years ago

Thanks @charlie, and very true.

Posted by greglasley over 7 years ago

Thanks all for the comments and stuff to chew on.

This actually came out of an internal conversation I had with myself -- I want to defend the importance of field guides (physical or digital) and the importance of knowing what to call something... Or even to know that something has a name!

@gcwarbler and @maractwin talked about their field guide book shelves. I too have a nice little collection of field guides that I really enjoy. Hell, when I went to my wife's house for the first time when we were still dating, I noticed all of the field guides and nature books on her shelves and said, "yep, this is the one." :) I love field guides and think they're important. They helped me "develop a passion for natural history" like this page mentions: https://arthropodecology.com/2012/04/30/field-guides-and-natural-history/

But are they (field guides) important? When I tag a name onto something, is it more important than it was before? Did it matter more?

iNat engages that same sort of passion I have for finding different critters and plants and for ID'ing what others find. The more that I flip through the pages of my field guide and through the pages on iNat of observations, the more I learn -- it's wonderful to hear that's what others do too.

I worked in a herbarium of around 1 million plant specimens at BRIT -- errors were abundant. Errors among the credentialed were abundant as well. It's ok! We find errors and temporarily correct them -- only to be recognized as something else in the future, perhaps. Photos and angles of photos can be deceptive, and it's ok to add a temporary ID (in my mind).

This organism exists, it's sharing the planet with us in time and space, and we document its existence. Now, what kind of bumblebee is it?!? ;) @greglasley , what species of female Argia or which meadowlark is it? @charlie, what name should we give this cryptic species in this species complex? @susanhewitt , what kind of slug is that again? @nathantaylor7583 , look at that super blurry pic of that spurge and see if you can name it. @damontighe , could you look at this mushroom that lacks all of the appropriate pics for its ID? Even if we can't put the exact right name on every single observation, each observation shows that organism that existed -- I think that's pretty important. :)

Appreciate the conversation here. Thanks!

Posted by sambiology over 7 years ago

Wow, very nice Sam. Thanks for the inspiration. :)

Posted by susanhewitt over 7 years ago

I'm pretty cautious about ID'ing my observations. A lot of mine still haven't been properly ID'ed yet because I haven't gathered enough data (like seeds) or it's like a similar species.

Posted by james52 over 7 years ago

What an interesting paper, Sam! Thank you for sharing. I agree with @susanhewitt that it's important to orient people to the function of adding those levels in between kingdom or phylum and species. I have also found that it's important to explain the basics of the tree of life and how that relates to naming and community IDs. If you haven't had biology since high school, that may not be a way you're thinking about naming organisms. I (unsuccessfully) tried to get people to add better initial IDs (especially before the smarter suggestions for taxon names) by saying something along the lines of, "...so even if you don't know exactly what it is, enter what you know! What do you know about this organism?" and I'd get something like "blue flower" (doh!). So I haven't nailed that explanation yet. There are some helpful images on the getting started page that I'd recommend if you're presenting to newbies (https://www.inaturalist.org/pages/getting+started#organisms).

As a kid I remember being frustrated with my Audubon insect guide when I couldn't find exactly whatever I was looking at. Of course, I didn't appreciate just how impossibly big such a volume would be! I wish that I had learned earlier in life to be ok with a family or genus level ID. I think if I had embraced the uncertainty sooner I might have actually been empowered to dig deeper, because at least I'd have a better starting place.

Posted by carrieseltzer over 7 years ago

You are absolutely right Carrie. I also wish that someone had explained to me when I was young that being able to ID things to the family level or genus level is, in some ways, much MORE useful than knowing exactly which species something is.

But generally people start out in life learning the common names of a bunch of local organisms, and they may have no idea that those creatures fit into families of related creatures, which all kind-of look similar. And they might be astonished to learn that most families are worldwide in distribution.

And most field guides are not that good as a teaching tool because they don't introduce families to people to all.

The notable exception in my field is Tucker Abbott's "Seashells of North America, a Golden Guide", which puts a lot of helpful emphasis on families, and that is why I love that little book so much, and recommend it so often.

I personally believe that taxonomy is best learned hands-on, and learned from the bottom up, rather than from the top down.

Posted by susanhewitt over 7 years ago

@kiwifergus As support for one of your arguments, I subscribe to Euphorbiaceae and Euphorbia.

Posted by nathantaylor over 7 years ago

I'm a map nerd myself, and the names show you which map to stick the pins on. That's why the taxonomic database is so crucial, so we can group these things into boxes so we can find them to put on maps.

Posted by charlie over 7 years ago

This may be heresy, but here goes: In general iNat data can not really be used for distribution or density analysis of a species because observations are not randomly taken, but rather dependant upon where humans choose to observe, and biased towards unusual sighting because they are more interesting. So in a lot of ways it doesn't affect the quality of the data if you can't reliably distinguish between two similar species from a photo, if they are both known to be in the area*. The problem is when something is mis-identified (to research grade) as something that isn't otherwise known to be in the area - so extra caution is needed, extra evidence is needed, for out-of-range observations. (*Of course if you have collected the photographed specimen, or it is something like a tree that has a relatively specific location the exact ID of that individual is more crucial).

Posted by tony_wills over 7 years ago

well, i wouldn't say it can't be used to tell a story about a diestribution of a species (including one that influences management or ecology)... but it is true that there is no absence data and there are lots of types of analysis you can't do. But, I think it's a case of 'the enemy of the perfect is the good' and it's free data :)

I'm not using iNat to publish papers about species distribution but to increase our knowledge about species range, absolutely. but yeah, for big out of range observations, very good evidence is needed, and ideally eventually someone goes and gets a voucher for the important ones!

Basically the world is so big and focused surveys haven't been conducted most areas... so there is still so much to discover!

Agreed that when picking apart species that both occur in an area, or especially subspecies, it's not providing that much value to obsess over. Though I don't want wrong IDs either

Posted by charlie over 7 years ago

right, well i just mean an ID that doesn't match the current taxonomy one way or another.

Posted by charlie over 7 years ago

Add a Comment

Sign In or Sign Up to add comments