---
Video clip realizado utilizando material de archivo de la Filmoteca Española, en particular de la excelente colección Sagarminaga que contiene escenas costumbristas y documentales de finales del siglo XIX y principios del XX. El fragmento más antiguo que hemos utilizado data de 1896 y el más moderno de 1904.
Todo el material ha sido remasterizado y procesado con inteligencia artificial. Se utilizaron algoritmos open-source de Deep learning para colorear, reescalar y aumentar el frame rate de los vídeos para darles un look contemporáneo. A las imágenes documentales le hemos añadido una serie de deepfakes de imágenes del archivo de la Biblioteca Nacional de la misma época.
El vídeo clip es una colaboración entre Espadaysantacruz Studio & NYSU ( https://nysuforever.net/) para La juventud ( https://www.instagram.com/lajuventud.divinotesoro)
Tenemos que agradecer profundamente a toda la comunicad científica que publica sus librerías de deep learning y que nos permite integrarlas en procesos creativos. En particular hemos utilizado los siguientes procesos.
Remasterizado y limpiado de los vídeos originales on DeepRemaster basado en Temporal convolutional neural networks.
Conversión a 75fps con DAIN: Depth-Aware video frame Interpolation
Coloreado con DeOldify
Escalado a 4k con ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
DeepFakes con Deep Learning First Order Motion Model for Image Animation
Programado en Python con PyTorch corriendo en una GPU Tesla P100 en Google Cloud
Colorizing tests
---
Music Video made using archive material from the National Spanish Film Archive, in particular from the Sagarminaga collection that contains traditional and documentary scenes from the late 19th and early 20th centuries. The oldest fragment we have used dates from 1896 and the most modern from 1904.
All the material has been remastered and colorized using artificial intelligence.
Open-source Deep learning algorithms were used to color, rescale and increase the frame rate of the videos to give them a contemporary look.
For this we have used:
-Video Enhancement using DeepRemaster based on temporal convolutional neural networks.
-FPS Boosting using de Deep Learning with DAIN: Depth-Aware video frame Interpolation
-Deep Learning colorizing using DeOldify
-4K Upscaling using ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
To the clips where the characters are singing the chorus we have used a series of pictures of the archive of the National Library of the same period, to which we have transfer the facial expression and motion of the members of the music band, singing the chorus. For this we have used an open source Deepfake model:
-Deep Learning First Order Motion Model for Image Animation
We used Python with PyTorch to code and integrate all the deeplearning models and algorithms, and we run all the computing in a single GPU Tesla P100 on Google Cloud.
We have to deeply thank all the scientific community that publishes the deep learning libraries, allowing to integrate them into creative processes. In particular we have used the following processes.
Deepfake tests
Client: La Juventud
Powered by: Python, Deep Learning First Order Motion Model, DeepRemaster based on temporal convolutional neural networks, DAIN: Depth-Aware video frame Interpolation, DeOldify & ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
May - 2020
