eXplain-NNs

This repository contains eXplain-NNs Library - an open-source library with explainable AI (XAI) methods for analyzing neural networks. This library provides several XAI methods for latent spaces analysis and uncertainty estimation.

Project Description

eXplain-NNs methods

XAI methods implemented in the library

1. visualization of latent spaces
2. homology analysis of latent spaces
3. uncertainty estimation via bayesianization

Thus compared to other XAI libraries, eXplain-NNs Library:

• Provides homology analysis of latent spaces
• Impelemnts a novel method of uncertainty estimation via bayesianization

Details of implemented methods.

Data Requirement

• The library supports only models that are:
  • fully connected or convolutional
  • designed for classification task

Installation

Requirements: Python 3.8

1. [optional] create Python environment, e.g.

   $ conda create -n eXNN python=3.8
   $ conda activate eXNN
   

2. install requirements from requirements.txt

   $ pip install -r requirements.txt
   

3. install the library as a package

   $ python -m pip install git+ssh://git@github.com/Med-AI-Lab/eXplain-NNs
   

Video demonstration of installation process is available here.

Development

Requirements: Python 3.8

1. [optional] create Python environment, e.g.

   $ conda create -n eXNN python=3.8
   $ conda activate eXNN
   

2. clone repository and install all requirements

   $ git clone git@github.com:Med-AI-Lab/eXplain-NNs.git
   $ cd eXplain-NNs
   $ pip install -r requirements.txt
   

3. run tests

   $ pytest tests/tests.py
   

4. fix code style to match PEP8 automatically

   $ make format
   

5. check that code style matches PEP8

   $ make check
   

6. build a PyPi package locally

   $ python3 -m pip install --upgrade build
   $ python3 -m build
   

Documentation

The general description is available here.

eXplain-NNs Library API is available here

Examples & Tutorials

We provides examples of different levels of complexity:

• [minimal] minimalistic examples presenting our API
• [basic] applying eXNN to simple tasks like MNIST classification
• [use cases] demonstation of eXplain-NN usage for solving different use cases in industrial tasks

Minimal

This colab contains minimalistic demonstration of our API on dummy objects:

Basic

Here are colabs demonstrating how to work with different modules of our API on simple tasks:

Colab Link	Module
	bayes
	topology
	visualization

Use Cases

This block provides examples how eXplain-NNs can be used to solve different use cases in industrial tasks. For demonstration purposed 3 tasks are used:

• [satellite] landscape classification from satellite imagery
• [electronics] electronic components and devices classification
• [ECG] ECG diagnostics

Colab Link	Task	Use Case
	satellite	Visualization of data manifold evolution from layer to layer
	satellite	Detecting adversarial examples
	electronics	Estimating generalization of a NN
	ECG	Visualization of data manifold evolution from layer to layer

Contribution Guide

The contribution guide is available in the repository.

Acknowledgments

Affiliation

ITMO University.

Supported by

The study is supported by the Research Center Strong Artificial Intelligence in Industry of ITMO University as part of the plan of the center's program: Development and testing of an experimental prototype of a library of strong AI algorithms in terms of algorithms for explaining the results of modeling on data using the semantics and terminology of the subject and problem areas in tasks with high uncertainty, including estimation of the uncertainty of neural network models predictions, analysis and visualization of interlayer transformations of the input variety inside neural networks.

Developers

• A. Vatyan - team leader
• N. Gusarova - chief scientific advisor
• I. Tomilov
• T. Polevaya
• Ks. Nikulina

Contacts

• Alexandra Vatyan alexvatyan@gmail.com for collaboration suggestions
• Tatyana Polevaya tpolevaya@itmo.ru for technical questions