This graduate-level textbook aims to give a unified presentation and solution of several commonly used techniques for multivariate data analysis (MDA). Unlike similar texts, it treats the MDA problems as optimization problems on matrix manifolds defined by the MDA model parameters, allowing them to be solved using (free) optimization software Manopt. The book includes numerous in-text examples as well as Manopt codes and software guides, which can be applied directly or used as templates for solving similar and new problems. The first two chapters provide an overview and essential background for studying MDA, giving basic information and notations. Next, it considers several sets of matrices routinely used in MDA as parameter spaces, along with their basic topological properties. A brief introduction to matrix (Riemannian) manifolds and optimization methods on them with Manopt complete the MDA prerequisite. The remaining chapters study individual MDA techniques in depth. The number of exercises complement the main text with additional information and occasionally involve open and/or challenging research questions. Suitable fields include computational statistics, data analysis, data mining and data science, as well as theoretical computer science, machine learning and optimization. It is assumed that the readers have some familiarity with MDA and some experience with matrix analysis, computing, and optimization.