Support Vector Machines(SVMs) are supervised models, and they could be very effective for classification, numerical prediction, and outlier detection problems.

The main idea is to separate different classes effectively: getting accurate results (e.g., higher accuracy score) and also balancing overfitting and underfitting (SVM introduces a slack term to account for this) at the same time.

SVM allows dividing classes using a line, plane, or hyperplane. For the simple example, with a line, we can divide by using maximum margin or soft margin. Soft margin is more flexible and allows misclassification by taking into account outliers, which gives a balance to not overfit or underfit.

Another thing, that sets SVM apart, is the use of so-called Kernel Functions. We could use Linear, Polynomial, Radial(RBF), or Sigmoid Functions in scikit-learn. These functions allow creating higher dimensions to separate classes better by use of hyperplanes.

image source

Even picture above shows a transformation of the data from 2d to 3d, SVM actually does not transform data into higher dimension but rather uses dot product result to find a relationship of the each label with the remaining labels (Kernel Trick).