In the past, one way AI researchers have tried to get around this problem is to use an approach called data augmentation. Using an image algorithm as an example again, in instances where there isn’t a lot of material to work with, they would try to get around that problem by creating “distorted” copies of what is available. Distorting, in this case, could mean cropping an image, rotating it or flipping it. The idea here is that the network never sees the same exact same image twice.
The problem with that approach is that it would lead to a situation in which the GAN would learn to mimic those distortions, instead of creating something new. NVIDIA’s new adaptive discriminator augmentation (ADA) approach still uses data augmentation but does so adaptively. Instead of distorting images throughout the entire training process, it does selectively and just enough so that the GAN avoids overfitting.
The potential outcome of NVIDIA’s approach is more meaningful than you might think. Training an AI to write a new text-based adventure game is easy because there’s so much material for the algorithm to work with. The same is not true for a lot of other tasks researchers could turn to GANs for help. For example, training an algorithm to spot a rare neurological brain disorder is difficult precisely because of its rarity. However, a GAN trained with NVIDIA’s ADA approach could get around that problem. As an added bonus, doctors and researchers could share their findings more easily since they’re working from a base of images created by an AI, not patients in the real world. NVIDIA will share more information about its new ADA approach at the upcoming NeurIPS conference, which starts on December 6th.