How to create a neural network that recognizes snow leopards and helps preserve their population

Evgeny Khokhlunov, Head of Environmental Projects at Yandex Cloud

Ayan Igali, Student at the School of Information Technologies and Engineering, KazNU

About Us

Evgeny. I have a mathematical background; I am a mathematician by training. Mathematics is the queen of sciences, and I have always aimed to measure and digitize everything. I previously worked at major companies such as MegaFon and Microsoft, and for the past seven years, I have been working on internal projects at Yandex. I have managed various departments, and for the last year and a half, I have focused on social projects related to science, education, and ecology to help them develop.

Ayan. I am a fourth-year student at KazNU. I work as an ML engineer at a bank. I have already published articles, and in November, I will be traveling to Japan. In the Yandex Cloud project on snow leopard monitoring, I work on tasks related to segmentation and detection of leopards.

About the Neural Network Idea

Evgeny. It all started with my frequent attendance at ecology conferences, where I met scientists studying leopards. They told me about the labor-intensive nature of their work, and I felt a desire to help automate processes to free up time for more creative tasks.

Scientists work in three stages: they study leopards using over 500 camera traps in Kazakhstan. These cameras take photos or videos onto flash drives, and then the team collects the flash drives and analyzes the data. Each flash drive contains up to 40,000 photos, which researchers review to identify images with animals. The main goal is to find photos of leopards, which are often poorly visible. This is a massive task that requires automation.

Scientists upload the collected materials to the cloud to train the model. The neural network must identify animals with 85% accuracy. This process is endless: the more photos it sees, the more accurately it recognizes them.

About Joining the Team

Ayan. I learned about the project from our dean at KazNU, who announced a section on leopard segmentation. I had already been working with machine learning and studying computer vision, so the project implemented with Yandex and having social significance for Kazakhstan interested me.

Our team consisted of several people, each with their own task. Some worked on theoretical classification methods, while others worked with pre-trained models. My task was to collect data for training the model.

I used pre-trained models to isolate photos with animals. One of the best was the "Megadetector" model, which minimized the influence of the background. After collecting a large number of photos with animals, there was a need to classify them.

We used models capable of recognizing objects in images, similar to ChatGPT. These models can perform many tasks in the field of computer vision. We spent most of our time collecting data.

About Difficulties

Ayan. Many tasks in the project were difficult. Some participants worked on implementing theoretical methods without practical code. There are many ready-made solutions in machine learning, but applying them correctly requires skills.

Data collection is the most difficult part, including cleaning and preprocessing. 98% of a neural network's success depends on the quality of the data.

In the case of leopards, this is relevant because there are few of them, and there are even fewer suitable photos and videos. Therefore, we used creative solutions such as augmentations to increase the volume of data.

Project Results

Evgeny. Overall, about a year has passed since the start of work on the project. At the moment, the model successfully filters 70-75% of unnecessary photos that do not contain animals.

It also recognizes several dozen species of animals living in areas with camera traps. A more complex task is recognizing individual leopards by their unique fur patterns. This is important for scientists to track their migration and habitat.

We believe it is important to share information about technologies that student and scientific teams can use to advance science in the country. After the leopard project, we received offers to create similar projects for lynxes, saigas, bears, and seals.

Additionally, we are working on projects related to volcanic ash emissions in Kamchatka and water analysis from Lake Baikal. We are open to new ideas and projects—if you are studying something interesting, come to us for support and technology.

How to Join

Evgeny. Applications for creating digital solutions to develop the fields of science and education can be submitted here.

Do not limit yourself to just leopards or other animals. Come with any ideas and projects—whether it is a thesis, a startup, or just an idea. We are ready to help you find ways to collaborate. We have many specialists who are ready to share their experience and create something interesting and useful for the country.

Ayan. Participating in the project was educational. I learned a lot about computer vision, detection, and segmentation. Although I usually work with textual data in NLP, working with images and videos became an interesting challenge.

I met wonderful people and learned to convert photos and text into numerical data. I mastered cloud technologies and appreciated access to powerful computing resources, which gave me the opportunity to experiment and develop my skills.