A new software program developed by Victorian scientists that identifies frog calls and sorts them from other sounds may greatly accelerate ecological research on frogs – or any other species.
A new software program developed by Victorian scientists that identifies frog calls and sorts them from other sounds, similar to how song recognition apps work, may greatly accelerate ecological research on frogs – or any other species.
The team from DELWP’s Arthur Rylah Institute for Environmental Research behind the pilot Frog Call Recognition Project says its software identifies frogs in sound recordings far more accurately than existing systems and will allow researchers to examine years of sound recordings in mere days.
The project was developed by wildlife ecologists Katie Howard, Louise Durkin and Lachlan Francis, and Senior Scientist Ecological Modelling Peter Griffioen.
And, like many great ideas, the plan was hatched over a beer.
Bottleneck
Researchers have turned to remote recording devices to save on the time and expense of sending ecologists into the field to listen for sounds like frog calls to identify the location of species.
But this creates a bottleneck of data – enormous numbers of sound recordings that are impossible for humans to listen to manually.
The team got to talking about the problem of their data bottleneck.
“Over about three years, Louise and Katie have amassed more than 470,000 files, which are generally two-minute automatic field recordings recorded over weeks. They have a position and time for each recording,” Peter said.
“If you were to listen to every file, it would take you almost two years, listening 24 hours a day. We were having a beer after work. They said that they had all these calls, and I said, ‘I’m pretty sure that I can write software to automatically recognise the calls much better than the current software’.”
Neural network
The team applied a deep learning approach known as convolutional neural networks, an Australian-first use of this technology in ecology, similar to other song recognition software.
“The software that’s used now in ecology uses parameters. It looks for joint occurrences of frequencies and attributes of calls such as changes in pitch, whereas the software we’re using looks for patterns in those and other sound parameters,” Peter said.
A painstaking training process is needed, using samples of the sound files to ensure the software can identify frog calls and other sounds. Accuracy was critical to the success of the model.
“Our experts pinpointed which frog was calling at a particular time. They marked hundreds of those for each species. That takes work, but then you have exemplars of what the frogs sound like,” Peter said. “We have 15 classes of sounds - 14 different frog species, and anything else was classified as ‘not a frog’.”
Amazing results
Once the training phase was completed, the results of the pilot were stunning.
“It was able to process half a million or so sound files on a normal, reasonably powered PC, in 3 days. The system outputs all the data to a database, so we can see what happened in every second and, if there is a frog call, which frog is calling,” Peter said.
“And it has much higher accuracy. You can pass frog recordings through existing technologies with 60 to 70% accuracy, but our system is above 90%. It’s much more accurate, and it can work in much noisier environments where you have a much larger variety of sounds.”
Peter said the team is about 40% of the way to producing a sound recognition system that could be used by the ecological science community.
“This is a pilot. It has shown its enormous potential and accuracy. It’s about scaling that up. Bigger geographic expanses. More species. The performance statistics suggest it’s totally right for that.”
Cornell University in the United States has a similar package that categorises about 900 bird species.
“But they have a lot more resources and a lot more scientists. We wanted something that’s specific to our species and our projects. We really did this on a shoestring budget,” he said.
The future
Now that the concept and the technology has been proved effective, there are plans to expand the project to birds and bats.
But Peter said the technology can be applied to all ARI recordings once the software learns the right information.
“It could be used to examine whether birds or other species have returned to an area affected by bushfires,” he said.
“The next scientist maybe isn’t interested in frogs, but wants to know about birds. We can get some bird call training into it and re-run the files collected for frogs. You can do this with all your historical sound files.
“If goats are invading an area, you might compare files from two years ago with files from today to see if they were there two years ago even though those files weren’t collected for that purpose.”
The ultimate goal is a bioacoustics library – a searchable storage system that’s tagged for time and location. It’s an old idea that seemed impossible until recently because of the huge computing power required.
“We are very confident we can do a very good job with this software. We’ve shown we can take great strides using these techniques. I’m certain we can do this for a number of different species and applications,” Peter said.
The Arthur Rylah Institute is hosting an online Frog Calls and Ecological Models event from 1pm to 2pm on Monday, 16 August, featuring Dr Peter Griffieon and Dr Jian Yen and exploring the role of data in science. The event is free to attend but bookings are essential.
