S2IGAN — Speech-to-Image Generation via Adversarial Learning

Authors present a framework that translates speech to images bypassing text information, thus allowing unwritten languages to potentially benefit from this technology.

ArXiV: https://arxiv.org/abs/2005.06968
Project: https://xinshengwang.github.io/project/s2igan/

#DL #audiolearning #speechrecognition

Team
@OpenArchiveBooks
@data_entusiasts

​​🎙🎶Improved audio generative model from OpenAI

Wow! OpenAI just released Jukebox – neural net and service that generates music from genre, artist name, and some lyrics that you can supply. It is can generate even some singing like from corrupted magnet compact cassette.

Some of the sounds seem it is from hell. Agonizing Michel Jakson for example or Creepy Eminiem or Celien Dion

#OpenAI 's approach is to use 3 levels of quantized variational autoencoders VQVAE-2 to learn discrete representations of audio and compress audio by 8x, 32x, and 128x and use the spectral loss to reconstruct spectrograms. And after that, they use sparse transformers conditioned on lyrics to generate new patterns and upsample it to higher discrete samples and decode it to the song.

The net can even learn and generates some solo parts during the track.

explore some creepy songs: https://jukebox.openai.com/
code: https://github.com/openai/jukebox/
paper: https://cdn.openai.com/papers/jukebox.pdf
blog: https://openai.com/blog/jukebox/

#openAI #music #sound #cool #fan #creepy #vae #audiolearning #soundlearning

Team
@OpenArchiveBooks
@data_enthusiasts