Yingtao Tian
Research Areas
Authored Publications
Google Publications
Other Publications
Sort By
Ukiyo-e Analysis and Creativity with Attribute and Geometry Annotation
Asanobu Kitamoto
Chikahiko Suzuki
Tarin Clanuwat
International Conference on Computational Creativity (2021), pp. 300-308 (to appear)
Preview abstract
The study of Ukiyo-e, an important genre of pre-modern Japanese art, focuses on the object and style like other artwork researches. Such study has benefited from the renewed interest by the machine learning community in culturally important topics, leading to interdisciplinary works using data-driven and machine learning-based methods. These works include collections of images, quantitative approaches, and machine learning-based creativities. They, however, have several drawbacks, and it remains challenging to integrate these works for a comprehensive view. To bridge this gap, we propose a holistic approach leveraging large scale data and machine learning models. We first present a large-scale Ukiyo-e dataset with coherent semantic labels and geometric annotations, then show its value in a quantitative study of Ukiyo-e paintings' object using these labels and annotations. We further demonstrate the machine learning-based methods could help style study through soft color decomposition of Ukiyo-e, and provides joint insightings into object and style by separating sketches and colors.
View details
KaoKore: The Pre-modern Japanese Art Facial Expression Dataset
Chikahiko Suzuki
Tarin Clanuwat
Mikel Bober-Irizar
Alex Lamb
Asanobu Kitamoto
Proceedings of the 11th International Conference on Computational Creatvity (2020), pp. 415-422
Preview abstract
From classifying handwritten digits to generating strings of text, the datasets which have received long-time focus from the machine learning community vary greatly in their subject matter. This has motivated a renewed interest in creating datasets which are socially and culturally relevant, so that algorithmic research may have a more direct and immediate impact. One such area is in history and the humanities, where better machine learning models could help to accelerate research. Along this line, newly created benchmarks and models have been proposed for historical Japanese cursive writing. At the same time, using machine learning for historical Japanese artworks has remained largely uncharted. In this work we propose a new dataset Kaokore which consists of faces from Pre-modern Japanese arts. We demonstrate its value as both a classification dataset as well as a creative and artistic dataset, which we explore using generative models.
View details
Preview abstract
End-to-end optimization has achieved state-of-the-art performance on many specific problems, but there is no straight-forward way to combine pretrained models for new problems. Here, we explore improving modularity by learning a post-hoc interface between two existing models to solve a new task. Specifically, we take inspiration from neural machine translation, and cast the challenging problem of cross-modal domain transfer as unsupervised translation between the latent spaces of pretrained deep generative models. By abstracting away the data representation, we demonstrate that it is possible to transfer across different modalities (e.g., image-to-audio) and even different types of generative models (e.g., VAE-to-GAN). We compare to state-of-the-art techniques and find that a straight-forward variational autoencoder is able to best bridge the two generative models through learning a shared latent space. We can further impose supervised alignment of attributes in both domains with a classifier in the shared latent space. Through qualitative and quantitative evaluations, we demonstrate that locality and semantic alignment are preserved through the transfer process, as indicated by high transfer accuracies and smooth interpolations within a class. Finally, the modular structure enables efficient training of new interface models, decoupling from the cost of training the base generative models.
View details
No Results Found
