MathWithNaziaa : Linear Algebra in Machine Learning

Linear Algebra: What is it

Linear algebra is the branch of mathematics concerning linear equations and their representations and through matrices.

Linear algebra is foundational in data science and machine learning. Beginners starting out along their learning journey in data science--as well as established practitioners--must develop a strong familiarity with the essential concepts in linear algebra.

In 2-dimension, it can take the form of a innocuous, vanilla straight-line formula y=mx+b. In higher dimensions, linear algebra becomes a little more challenging using the toolbox of linear algebra.

A study of linear algebra generally consists of vectors, matrices and

Linear Algebra is a branch of mathematics that is extremely useful in data science and machine learning. Linear algebra is the most important math skill in machine learning. Most machine learning models can be expressed in matrix form. A dataset itself is often represented as a matrix. Linear algebra is used in data preprocessing, data transformation, and model evaluation. Here are the topics you need to be familiar with:

• Vectors
• Matrices
• Transpose of a matrix
• Inverse of a matrix
• Determinant of a matrix
• Trace of a matrix
• Dot product
• Eigenvalues
• Eigenvectors

 

Vectors and spaces

A vector is a way of representing a quantity that has a direction.

For instance, if a boat is moving north-east 5 miles per hour, we can represent the vector of the boat’s velocity as a 5-unit long vector in the north-west direction.

A n-dimensional vector exists in, you guessed it, n-dimensional space. For instance, the boat can be simplified into an object that exists in 2-dimensional space —the north-south dimension and east-west dimension.

In the past hour, we can imagine that the boat has moved 3 miles north and 4 miles east, such that it is moving 5 miles per hour in the north-east direction. We can then imagine that the boat’s velocity vector as x = [3, 4].

To start learning linear algebra, we need to start understanding what are the properties of vectors (linear dependence or independence), what we can do with them (dot and cross products), and the properties of the spaces they exist in (subspaces).

Matrix transformations

If we combine several vectors together, we get a matrix. Matrices can be interpreted as a transformation to vectors, like scaling, rotation or rotation. Imagine we want the boat to travel twice as fast in the same direction (represented by new vector y), we would scale the boat’s velocity vector by two using the matrix A, with the formula:

y = A x

Each matrix has its own properties. One of the most important properties is the eigenvector, which is the vector that does not change in direction after the transformation is applied. Another important property is the eigenvalue, which is the change in length of the same vector after that transformation.

This eigenvector is incredibly useful in advanced treatment of the subject, forming the cornerstone of many techniques used including Principle Component Analysis and Singular Value Decomposition.

hen people think of the field of data science in general, or of specific areas of it, such as natural language processes, machine learning, or computer vision, they rarely consider linear algebra. The reason linear algebra is often overlooked is that tools used today to implement data science algorithms do an excellent job in hiding the underlying maths that make everything come true. Most of the time, people avoid getting into linear algebra because it’s “difficult” or “hard to understand.” Although partly true, being familiar with linear algebra is an essential skill for data scientists and computer engineers.

Linear algebra is the core of many known data science algorithms. In this article, I will discuss three applications of linear algebra in three data science fields. From machine learning, we will talk about loss functions, from natural language processing, we will talk about word embedding, and finally, from computer vision, we will cover image convolution.