ANALYSIS OF METRICS FOR GAN EVALUATION
DOI:
https://doi.org/10.31891/csit-2023-4-6Keywords:
GAN evaluation, metrics, inception score, frechet inception distanceAbstract
Generative-adversarial networks have become quite popular in recent years. In general, these networks are based on convolutional neural networks used in classification problems. In recent years, researchers have proposed and developed many variations of GAN network architectures and techniques for their optimization, as the learning process is quite complex and unstable. Despite great theoretical advances in improving network data, evaluating and comparing GANs remains a challenge. Although several metrics have been introduced to evaluate these networks, there is currently no consensus on which metrics best reflect the strengths and limitations of models and should be used to compare models and evaluate synthesized images. This paper discusses the two most popular metrics, Inception Score (IS) and Frechet Inception Distance (FID), which are used to estimate GAN networks.
Because these metrics are based on a pre-built Google Inception model used as a classifier for IS metrics and a feature extractor for FID metrics, the goal is to develop a program module to compare metric data using the base model (Inception) and custom models.
The scientific novelty is that these metrics were first used to compare cytological images using a model different from the one proposed by the authors - Google Inception.
The practical significance of the work is the development of a software module for calculating metric data for GAN networks used for the synthesis of cytological images.
As a result, two basic models (BioCNN-1 and BioCNN-2) and a Python module for calculating IS and FID metrics for cytological images were developed. The developed module works with color images with a resolution of 64 x 64 pixels. Comparisons of metrics based on the base model and the developed models for estimating GAN networks for cytological image synthesis were compared.
It was shown that the metrics based on the developed models show better results The FID score reduced from 31.20 to 0.034 and the IS score increased from 3.52 to 3.81. A total metric calculation time reduced from 2 minutes to 15 seconds.
