Tensor fusion: learning in heterogeneous and distributed data

Abstract

In this thesis, we focus on coupled matrix and tensor factorization models; that provide a good modeling accuracy { practicality trade o for modeling large-scale and/or heterogeneous data that are collected from diverse sources. Our main concern in this thesis will be to develop inference methods for coupled tensor factorization models. We will rst develop a rigorous tensor factorization notation, that aims to cover all possible model topologies and coupled factorization models. Our notation highlights the partially separable structure of tensor factorization models, which paves the way for developing parallel and distributed inference algorithms. Secondly, we will develop novel methods for making inference in coupled tensor factorization models. The proposed methods can be separated into three groups. In the rst set of methods, we will focus on optimization-based approaches for making maximum likelihood and a-posteriori estimation of the latent variables. The second group of methods builds up on the rst group and jointly estimates the relative weights and divergence functions, which play important role in coupled factorization models. Finally, in the third group, we will focus on full Bayesian inference, where we will develop several Markov Chain Monte Carlo methods for sampling from the posterior distributions of the latent variables. We will evaluate our methods on several challenging applications. We will develop novel factorization models for addressing challenging audio processing applications. We will also evaluate our distributed inference methods on large-scale link prediction applications, where we will report successful results in all of these applications.