Purushothamann commited on
Commit
20ed5d7
Β·
verified Β·
1 Parent(s): 80680b0

Upload 10 files

Browse files
Files changed (1) hide show
  1. README.md +68 -58
README.md CHANGED
@@ -1,29 +1,39 @@
 
1
  # Interpretable-SONAR-Image-Classifier
2
 
3
- Explainable AI for Underwater SONAR Image Classifier
 
 
 
4
 
5
  ## Prerequisites
6
 
7
- - Python 3.6 or higher
 
8
 
9
- ## Running the Scripts
10
 
11
- This guide will help you run the `data_loader.py`, `train.py`, `test.py`, `predict.py`, and `classify_image_and_explain.py` scripts directly from the command line or within a Python script.
12
 
13
- ### Prerequisites
 
 
14
 
15
- 1. **Python Installation**: Ensure you have Python installed. You can download it from [python.org](https://www.python.org/).
16
 
17
- 2. **Required Packages**: Install the required packages using `requirements.txt`.
18
- ```sh
19
- pip install -r requirements.txt
20
  ```
21
 
22
- ### Script Descriptions and Usage
 
 
 
 
23
 
24
  #### 1. `data_loader.py`
25
 
26
- This script is used to load, process, split datasets (train, val, test), and augment data.
27
 
28
  **Command Line Usage:**
29
 
@@ -33,12 +43,12 @@ python data_loader.py --path <path_to_data> --target_folder <path_to_target_fold
33
 
34
  **Arguments:**
35
 
36
- - `--path`: Path to the data.
37
- - `--target_folder`: Path to the target folder where processed data will be saved.
38
- - `--dim`: Dimension for resizing the images.
39
  - `--batch_size`: Batch size for data loading.
40
  - `--num_workers`: Number of workers for data loading.
41
- - `--augment_data` (optional): Flag to enable data augmentation.
42
 
43
  **Example:**
44
 
@@ -48,7 +58,7 @@ python data_loader.py --path "./dataset" --target_folder "./processed_data" --di
48
 
49
  **Dataset Structure:**
50
 
51
- ```sh
52
  β”œβ”€β”€ Dataset (Raw)
53
  Β Β  β”œβ”€β”€ class_name_1
54
  β”‚Β Β  └── *.jpg
@@ -62,36 +72,36 @@ python data_loader.py --path "./dataset" --target_folder "./processed_data" --di
62
 
63
  #### 2. `train.py`
64
 
65
- This script is used for training and storing the models, leveraging transfer learning.
66
 
67
  **Command Line Usage:**
68
 
69
  ```sh
70
- python train.py --base_model_names <model_names> --shape <shape> --data_path <data_path> --log_dir <log_dir> --model_dir <model_dir> --epochs <epochs> --optimizer <optimizer> --learning_rate <learning_rate> --batch_size <batch_size>
71
  ```
72
 
73
  **Arguments:**
74
 
75
- - `--base_models`: Comma-separated list of base model names (e.g., 'VGG16, ResNet50').
76
- - `--shape`: Image shape (size).
77
- - `--data_path`: Path to the data.
78
- - `--log_dir`: Path to the log directory.
79
- - `--model_dir`: Path to the model directory.
80
- - `--epochs`: Number of training epochs.
81
- - `--optimizer`: Optimizer type ('adam' or 'sgd').
82
- - `--learning_rate`: Learning rate for the optimizer.
83
- - `--batch_size`: Batch size for training.
84
- - `--patience`: Patience for early stopping (to prevent overfitting).
85
 
86
  **Example:**
87
 
88
  ```sh
89
- python train.py --base_models "VGG16" "DenseNet121" --shape 224 224 3 --data_path "./processed_data" --log_dir "./logs" --model_dir "./models" --epochs 100 --optimizer "adam" --learning_rate 0.0001 --batch_size 32
90
  ```
91
 
92
  #### 3. `test.py`
93
 
94
- This script is used for testing and storing the test logs of the above-trained models.
95
 
96
  **Command Line Usage:**
97
 
@@ -101,22 +111,22 @@ python test.py --data_path <data_path> --base_model_name <base_model_name> --mod
101
 
102
  **Arguments:**
103
 
104
- - `--models_dir` (optional): Path to the models (directory).
105
- - `--model_path`: Path to the model (.h5/Keras Model).
106
- - `--img_path`: Path to the image file.
107
- - `--test_dir`: Path to the test dataset (directory).
108
- - `--train_dir`: Path to the training data.
109
- - `--log_dir`: Path to the log directory.
110
 
111
  **Example:**
112
 
113
  ```sh
114
- python test.py --model_path "./models/vgg16_model.keras" --test_dir "./test_data" --train_dir "./data/train" --log_dir "./logs"
115
  ```
116
 
117
  #### 4. `predict.py`
118
 
119
- This script is used for making predictions on new images.
120
 
121
  **Command Line Usage:**
122
 
@@ -126,19 +136,19 @@ python predict.py --model_path <model_path> --img_path <img_path> --train_dir <t
126
 
127
  **Arguments:**
128
 
129
- - `--model_path`: Path to the model file.
130
- - `--img_path`: Path to the image file.
131
- - `--train_dir`: Path to the training dataset (for the label decoder, can be replaced with a CSV file with slight code modifications).
132
 
133
  **Example:**
134
 
135
  ```sh
136
- python predict.py --model_path "./models/vgg16_model.keras" --img_path "./images/test_image.jpg" --train_dir "./data/train"
137
  ```
138
 
139
  #### 5. `classify_image_and_explain.py`
140
 
141
- This script is used for making predictions on new images and generating explanations using one or more explainers (LIME, SHAP, Grad-CAM). The explanations are saved in the specified output folder, with filenames indicating the method used (e.g., `lime_explanation_1.jpg`, `shap_explanation_1.jpg`, `gradcam_explanation_1.jpg`).
142
 
143
  **Command Line Usage:**
144
 
@@ -148,24 +158,24 @@ python classify_image_and_explain.py --image_path <image_path> --model_path <mod
148
 
149
  **Arguments:**
150
 
151
- - `--image_path` (required): Path to the input image.
152
- - `--model_path` (required): Path to the trained model.
153
- - `--train_directory` (required): Directory containing training images.
154
- - `--num_samples` (default: 300): Number of samples for LIME.
155
- - `--num_features` (default: 100): Number of features for LIME.
156
- - `--segmentation_alg` (default: `quickshift`): Segmentation algorithm for LIME (`quickshift`, `slic`).
157
- - `--kernel_size` (default: 4): Kernel size for the segmentation algorithm.
158
- - `--max_dist` (default: 200): Maximum distance for the segmentation algorithm.
159
- - `--ratio` (default: 0.2): Ratio for the segmentation algorithm.
160
- - `--max_evals` (default: 400): Maximum evaluations for SHAP.
161
- - `--batch_size` (default: 50): Batch size for SHAP.
162
- - `--explainer_types` (default: 'all'): Comma-separated list of explainers to use (`lime`, `shap`, `gradcam`). Use 'all' to include all three explainers.
163
- - `--output_folder` (optional): Folder to save explanation images.
164
 
165
  **Example:**
166
 
167
  ```sh
168
- python classify_image_and_explain.py --image_path "./images/test_image.jpg" --model_path "./models/model.keras" --train_directory "./data/train" --num_samples 300 --num_features 100 --segmentation_alg "quickshift" --kernel_size 4 --max_dist 200 --ratio 0.2 --max_evals 400 --batch_size 50 --explainer_types "lime, gradcam" --output_folder "./explanations"
169
  ```
170
 
171
  ### Supported Base Models
@@ -186,7 +196,7 @@ The following base models are supported for training:
186
  - EfficientNetB0
187
  - EfficientNetB7
188
 
189
- ### Running Scripts in a Python Script
190
 
191
  You can also run these scripts programmatically using Python's `subprocess` module. Here is an example of how to do this for each script:
192
 
 
1
+
2
  # Interpretable-SONAR-Image-Classifier
3
 
4
+ Explainable AI for Underwater SONAR Image Classifier to enhance model interpretability with techniques like LIME, SHAP, and Grad-CAM. It employs transfer learning with popular deep learning models (e.g., VGG16, ResNet50, DenseNet121) to classify images, while providing insights into model predictions, making it suitable for use cases requiring transparency and reliability in SONAR image analysis.
5
+
6
+ The following guide details the requirements, usage, and examples for running the scripts within the project, along with how to generate explanations for model predictions.
7
+
8
 
9
  ## Prerequisites
10
 
11
+ - **Python 3.9**
12
+ - **Conda**: You can create the required environment using `environment.yaml`.
13
 
14
+ ### Setting Up the Environment
15
 
16
+ 1. **Conda Environment**: To ensure all dependencies are correctly installed, create a new environment using the provided `environment.yaml` file.
17
 
18
+ ```bash
19
+ conda env create -f environment.yaml
20
+ ```
21
 
22
+ 2. **Activate the Environment**:
23
 
24
+ ```bash
25
+ conda activate Interpretable-SONAR-Image-Classifier
 
26
  ```
27
 
28
+ This setup ensures compatibility and includes all necessary packages and versions as defined in the `environment.yaml` file.
29
+
30
+ ### Running the Scripts
31
+
32
+ The following sections describe each script and its usage. Run these scripts directly from the command line or within a Python script.
33
 
34
  #### 1. `data_loader.py`
35
 
36
+ This script loads, processes, splits datasets (train, val, test), and performs optional data augmentation.
37
 
38
  **Command Line Usage:**
39
 
 
43
 
44
  **Arguments:**
45
 
46
+ - `--path`: Path to the raw data.
47
+ - `--target_folder`: Directory for processed data.
48
+ - `--dim`: Image resize dimension (e.g., 224 for 224x224).
49
  - `--batch_size`: Batch size for data loading.
50
  - `--num_workers`: Number of workers for data loading.
51
+ - `--augment_data` (optional): Enables data augmentation.
52
 
53
  **Example:**
54
 
 
58
 
59
  **Dataset Structure:**
60
 
61
+ ```plaintext
62
  β”œβ”€β”€ Dataset (Raw)
63
  Β Β  β”œβ”€β”€ class_name_1
64
  β”‚Β Β  └── *.jpg
 
72
 
73
  #### 2. `train.py`
74
 
75
+ This script trains models with options for transfer learning and custom base models.
76
 
77
  **Command Line Usage:**
78
 
79
  ```sh
80
+ python train.py --base_models <model_names> --shape <shape> --data_path <data_path> --log_dir <log_dir> --model_dir <model_dir> --epochs <epochs> --optimizer <optimizer> --learning_rate <learning_rate> --batch_size <batch_size>
81
  ```
82
 
83
  **Arguments:**
84
 
85
+ - `--base_models`: Comma-separated list of base models (e.g., "VGG16,DenseNet121").
86
+ - `--shape`: Image shape, e.g., `224 224 3`.
87
+ - `--data_path`: Path to processed data.
88
+ - `--log_dir`: Directory for logs.
89
+ - `--model_dir`: Directory to save models.
90
+ - `--epochs`: Number of epochs.
91
+ - `--optimizer`: Optimizer type (`adam` or `sgd`).
92
+ - `--learning_rate`: Learning rate.
93
+ - `--batch_size`: Training batch size.
94
+ - `--patience`: Early stopping patience.
95
 
96
  **Example:**
97
 
98
  ```sh
99
+ python train.py --base_models "VGG16,DenseNet121" --shape 224 224 3 --data_path "./processed_data" --log_dir "./logs" --model_dir "./models" --epochs 100 --optimizer "adam" --learning_rate 0.0001 --batch_size 32
100
  ```
101
 
102
  #### 3. `test.py`
103
 
104
+ Tests the trained models and logs the results.
105
 
106
  **Command Line Usage:**
107
 
 
111
 
112
  **Arguments:**
113
 
114
+ - `--models_dir` (optional): Path to the models directory.
115
+ - `--model_path`: Specific model path (.h5 file).
116
+ - `--img_path`: Image file path for testing.
117
+ - `--test_dir`: Test dataset directory.
118
+ - `--train_dir`: Directory for training data.
119
+ - `--log_dir`: Directory for logs.
120
 
121
  **Example:**
122
 
123
  ```sh
124
+ python test.py --model_path "./models/vgg16_model.h5" --test_dir "./test_data" --train_dir "./data/train" --log_dir "./logs"
125
  ```
126
 
127
  #### 4. `predict.py`
128
 
129
+ Makes predictions on new images.
130
 
131
  **Command Line Usage:**
132
 
 
136
 
137
  **Arguments:**
138
 
139
+ - `--model_path`: Model file path.
140
+ - `--img_path`: Image file path.
141
+ - `--train_dir`: Directory for label decoding.
142
 
143
  **Example:**
144
 
145
  ```sh
146
+ python predict.py --model_path "./models/vgg16_model.h5" --img_path "./images/test_image.jpg" --train_dir "./data/train"
147
  ```
148
 
149
  #### 5. `classify_image_and_explain.py`
150
 
151
+ Makes predictions and generates explanations using LIME, SHAP, or Grad-CAM.
152
 
153
  **Command Line Usage:**
154
 
 
158
 
159
  **Arguments:**
160
 
161
+ - `--image_path`: Path to the image file.
162
+ - `--model_path`: Model file path.
163
+ - `--train_directory`: Directory of training images for label decoding.
164
+ - `--num_samples`: Sample count for LIME.
165
+ - `--num_features`: Feature count for LIME.
166
+ - `--segmentation_alg`: Segmentation algorithm for LIME.
167
+ - `--kernel_size`: Kernel size for segmentation.
168
+ - `--max_dist`: Max distance for segmentation.
169
+ - `--ratio`: Ratio for segmentation.
170
+ - `--max_evals`: Max evaluations for SHAP.
171
+ - `--batch_size`: Batch size for SHAP.
172
+ - `--explainer_types`: Comma-separated list of explainers (`lime`, `shap`, `gradcam`).
173
+ - `--output_folder`: Directory to save explanations.
174
 
175
  **Example:**
176
 
177
  ```sh
178
+ python classify_image_and_explain.py --image_path "./images/test_image.jpg" --model_path "./models/model.h5" --train_directory "./data/train" --num_samples 300 --num_features 100 --segmentation_alg "quickshift" --kernel_size 4 --max_dist 200 --ratio 0.2 --max_evals 400 --batch_size 50 --explainer_types "lime,gradcam" --output_folder "./explanations"
179
  ```
180
 
181
  ### Supported Base Models
 
196
  - EfficientNetB0
197
  - EfficientNetB7
198
 
199
+ ### Running Scripts in Jupyter Notebook
200
 
201
  You can also run these scripts programmatically using Python's `subprocess` module. Here is an example of how to do this for each script:
202