Anomaly Detection

Anomaly detection is identifying rare events or observations which do not conform with the general pattern of distribution of the population of events in the data. An anomaly raises the suspicion that there is something wrong with the underlying business process, which is generating this data.

Applications of Anomaly Detection in Financial Services

Due to recent advances towards industry 4.0 and machine learning, most business processes are going through automation to lower the cost of doing business, optimize capacity, and enhance efficiency. An automated process can be real-time or near real-time. It may be processing hundreds or thousands of events during a specific timeframe.

In the financial services industry, we can build anomaly detection models around business processes such as;

  • Bank transactions
  • Credit card transactions
  • Insurance claims
  • Auto loan applications

For the sake of reputation and risk management, a business must keep monitoring any anomalous events which a business process may encounter.

Point or Global Anonmaly Detection

3 Types of Anomalies

In business settings, we typically have to deal with three types of anomaly detection use cases:

1. Point or Global Anomaly

Point or global anomaly refers to an individual event that is rare and stands out concerning the rest of the data. In the banking and finance industry, a significant transaction by an 80 years old retired person or by a college student can be a global anomaly.

2. Contextual Anomaly

Contextual anomaly is a rare event in a specific context. For instance, a spike in the number of bank transactions over the weekend, or 80 years old person applying for (undergraduate) student loan.

3. Collective Anomaly

Collective anomaly occurs when a set of events or observations are collectively causing significant deviating from the remainder of the data. For example, a drop in stock prices amid the COVID-19 crisis.

Contextual Anomaly in Banking and Finance

Methods of Anomaly Detection

There are two main types of anomaly detection methods:

1. Unsupervised Anomaly Detection

In these methods, we use unsupervised machine learning algorithms. Unsupervised machine learning models, for example, K-Means, are trained on unlabeled data sets. In the unlabeled data set, the events or observations do not have 'normal' or 'anomaly' labels.

Though majority events in a typical data set are usually normal events, learning whether an (unlabeled) event belongs to normal or anomaly is a difficult problem. Simple clustering algorithms work well in low dimensional spaces. In machine learning, high dimensional space problems often require advance unsupervised learning methods.

2. Supervised Anomaly Detection

Supervised anomaly detection relies on supervised machine learning algorithms such as logistic regression, KNN, decision trees, neural networks. These algorithms get trained on labeled training data and deployed to predict an anomaly (around business process) in the future. Often the challenge with the training of these algorithms is unbalanced data set of anomaly detection problems. As we know, the anomalies are rare events; data sets for anomaly detection often have an unbalanced distribution with a majority of negative examples.

How Can Datalya Help with Implementation of Anomaly Detection?

Using machine learning, at Datalya, we have implemented a handful of anomaly detection solutions in banks, insurance companies, credit card institutions, energy, and telecommunication businesses. If you have any questions or need help, please do not hesitate to contact us through email or phone.

- Our Process

- Frequently asked questions

No - we provide free initial consultation for up to 1 hour. This allows us to answer your questions as well as determine requirements of your project.

- Related Services

- Contact Us

Please fill in contact form below and we will respond promptly. Please include as much detail about how we can help you as possible.

Our Location

Unit 216, 1325 Eglinton Ave E
Mississauga, ON L4Z 3K6
Canada

Social Media

- Write Us

Success! Your message has been sent to us.
Error! There was an error sending your message.