Skip to content

Commit 9d6cb36

Browse files
jangsooparkjangsoopark
authored
Merge pull request #1 from jangsoopark/v2.0.0
V2.0.0
2 parents b8ab4c0 + 654c89a commit 9d6cb36

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+864
-1633
lines changed

‎.gitignore

+8-5
Original file line numberDiff line numberDiff line change
@@ -280,11 +280,14 @@ $RECYCLE.BIN/
280280
*.lnk
281281

282282
# asdf
283-
dataset/raw/train/*
284-
dataset/raw/test/*
283+
dataset/soc/raw/train/
284+
dataset/soc/raw/test/
285285

286-
dataset/train/*
287-
dataset/test/*
286+
dataset/soc/train/
287+
dataset/soc/test/
288+
experiments/history
288289

289290
*.pth
290-
# End of https://www.toptal.com/developers/gitignore/api/windows,linux,pycharm+all,python
291+
# End of https://www.toptal.com/developers/gitignore/api/windows,linux,pycharm+all,python
292+
293+

‎README.md

+152-153
Original file line numberDiff line numberDiff line change
@@ -1,186 +1,158 @@
1-
# A-ConvNets
2-
3-
**Note:**
4-
>
5-
> Since this is my first trial to implement a model with SAR images and I don't have any domain knowledge,
6-
> this repository may contains weird implementations.
7-
>
8-
> If you find those kinds of incorrect things, please let me know to correct it via issues.
9-
>
10-
> It will be very helpful for me to understand SAR image processing and could contribute to lower the technical barriers.
11-
>
12-
> Thank you.
13-
14-
## Target Classification Using the Deep Convolutional Networks for SAR images
15-
16-
This repository contains the implementation of the paper
17-
`S. Chen, H. Wang, F. Xu and Y. Jin, "Target Classification Using the Deep Convolutional Networks for SAR Images,"
18-
in IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 8, pp. 4806-4817, Aug. 2016,
19-
doi: 10.1109/TGRS.2016.2551720.`
20-
21-
22-
## MSTAR Dataset
23-
24-
The implementation uses MSTAR(Moving and Stationary Target Acquisition and Recognition) database. Each of image consists of
25-
header(ASCII type) and data(*data type: float32, shape: W X H X 2*). Header contains meta data to read and utilize the
26-
database which include width, height, serial number, azimuth angle, etc. Data consists of magnitude and phase. Below
27-
figure is the example of magnitude(left) and phase(right).
28-
29-
![Example of MSTAR image](assets/figure/001.png)
30-
31-
In order to comprehensively assess the performance, the model is trained and evaluated under SOC
32-
(Standard Operating Condition) which uses the images captured at 17 depression angle for training and uses the images
33-
captured at 15 depression angle for evaluation.
34-
35-
|Class|Serial No.|Training<br/>Depression<br>angle|Training<br/>#images|Test<br/>Depression<br>angle|Test<br/>#images|
36-
|:---:|:---:|:---:|:---:|:---:|:---:|
37-
2S1 | b01 | 17 | 299 | 15 | 274 |
38-
BMP-2 | 9563 |17 | 233 | 15 | 195 |
39-
BRDM-2 | E-71 | 17 | 298 | 15 | 274 |
40-
BTR-60 | k10yt7532 |17 | 256 | 15 | 195 |
41-
BTR-70 | c71 | 17 | 233 | 15 | 196 |
42-
D7 | 92v13015 | 17 | 299 | 15 | 274 |
43-
T-62 | A51 | 17 | 299 | 15 | 273 |
44-
T-72 | 132 | 17 | 232 | 15 | 196 |
45-
ZIL-131 | E12 | 17 | 299 | 15 | 274 |
46-
ZSU-234 | d08 | 17 | 299 | 15 | 274 |
1+
# AConvNet
472

48-
## Model
49-
50-
#### `src/model/network.py`
51-
52-
Because of lack of lack of the number of available data in MSTAR database, deep neural networks could be suffered from
53-
over fitting. Hence, the author proposes new architecture which eases the over fitting problem by replacing fully
54-
connected layers with convolution layers.
55-
56-
|layer|Input|Conv 1|Conv 2|Conv 3|Conv 4|Conv 5|
57-
|:---:|---|:---:|:---:|:---:|:---:|:---:|
58-
|channels|2|16|32|64|128|10|
59-
|weight size| - |5 x 5|5 x 5|6 x 6|5 x 5| 3 x 3|
60-
|pooling| - | 2 x 2 - s2 | 2 x 2 - s2 | 2 x 2 - s2| - | - |
61-
|dropout| - | - | - | - | 0.5 | - |
62-
|activation| linear | ReLU | ReLU | ReLU | ReLU | Softmax |
63-
64-
## Experiments
65-
66-
### Data Augmentation
67-
68-
#### `src/data/generate.py`
69-
#### `src/data/mstar.py`
70-
71-
This repository follows the shifting method which used by author. However in order to preserve the reproducibility this
72-
repository doesn't uses the random sampling to extract patches. The patches are extracted in raster scanning order using
73-
patch size and stride.
74-
75-
### Training
76-
77-
#### `src/model/_base.py`
78-
79-
#### `experiments/AConvNet.json`
80-
81-
To replicate the official result, the model is trained with same optimization methods which uses SGD with Momentum,
82-
learning rate decay, weight decay, and the weights are initialized with He's method with uniform distribution.
83-
84-
The difference with the paper is the author decreases the learning rate at epoch 50, but this repository decreases the
85-
learning at 10 and 50 for stable convergence for my environments.
3+
### Target Classification Using the Deep Convolutional Networks for SAR Images
864

87-
![](assets/figure/003.png)
5+
This repository is reproduced-implementation of AConvNet which recognize target from MSTAR dataset.
6+
You can see the official implementation of the author at [MSTAR-AConvNet](https://github.com/fudanxu/MSTAR-AConvNet).
887

89-
### Early Stopping
8+
## Dataset
909

91-
The early stopping method is a form of regularization technique that finds the balance point between under fitting and
92-
over fitting.
10+
### MSTAR (Moving and Stationary Target Acquisition and Recognition) Database
9311

94-
In this experiments, the trained model shows an accuracy 99.01% at epoch 42.
12+
#### Format
9513

96-
### Results
14+
- Header
15+
- Type: ASCII
16+
- Including data shape(width, height), serial number, azimuth angle, etc.
17+
- Data
18+
- Type: Two-bytes
19+
- Shape: W x H x 2
20+
- Magnitude block
21+
- Phase Block
9722

98-
#### Confusion Matrix
23+
Below figure is the example of magnitude block(Left) and phase block(Right)
9924

100-
| classes | 2S1 | BMP-2 | BRDM-2 | BTR-60 | BTR-70 | D7 | T-62 | T-72 | ZIL-131 | ZSU-234 | Accuracy |
101-
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
102-
| 2S1 | 274 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 100 |
103-
| BMP-2 | 1 | 190 | 0 | 2 | 0 | 0 | 0 | 2 | 0 | 0 | 97.44 |
104-
| BRDM-2 | 0 | 0 | 272 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 99.27 |
105-
| BTR-60 | 1 | 0 | 0 | 186 | 4 | 0 | 0 | 1 | 1 | 2 | 95.38 |
106-
| BTR-70 | 0 | 1 | 0 | 0 | 195 | 0 | 0 | 0 | 0 | 0 | 99.49 |
107-
| D7 | 0 | 0 | 0 | 0 | 0 | 272 | 0 | 0 | 2 | 0 | 99.27 |
108-
| T-62 | 0 | 0 | 0 | 0 | 0 | 0 | 271 | 0 | 0 | 2 | 99.27 |
109-
| T-72 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 194 | 0 | 0 | 98.98 |
110-
| ZIL-131 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 273 | 1 | 99.64 |
111-
| ZSU-234 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 274 | 100 |
112-
| Total | - | - | - | - | - | - | - | - | - | - | 99.01|
25+
![Example of data block: 2S1](./assets/figure/001.png)
11326

114-
> The total accuracy is calculated with (TP + TN) / (TP + TN + FP + FN).
115-
>
116-
> It is equivalent to sum(C * I) / sum(C).
117-
>
118-
> Where C is confusion matrix and I is identity matrix
119-
120-
<!-- TODO : reinforce more analytical explanation
121-
#### Error Analysis
122-
123-
False Positive of BMP-2
27+
## Model
12428

125-
![FP-BMP-2](assets/figure/004.png)
29+
The proposed model only consists of **sparsely connected layers** without any fully connected layers.
12630

127-
False Positive of BRDM-2
31+
- It eases the over-fitting problem by reducing the number of free parameters(model capacity)
12832

129-
![FP-BRDM-2](assets/figure/005.png)
33+
| layer | Input | Conv 1 | Conv 2 | Conv 3 | Conv 4 | Conv 5 |
34+
| :---------: | ------ | :--------: | :--------: | :--------: | :----: | :-----: |
35+
| channels | 2 | 16 | 32 | 64 | 128 | 10 |
36+
| weight size | - | 5 x 5 | 5 x 5 | 6 x 6 | 5 x 5 | 3 x 3 |
37+
| pooling | - | 2 x 2 - s2 | 2 x 2 - s2 | 2 x 2 - s2 | - | - |
38+
| dropout | - | - | - | - | 0.5 | - |
39+
| activation | linear | ReLU | ReLU | ReLU | ReLU | Softmax |
13040

131-
False Positive of BTR-60
41+
## Training
42+
For training, this implementation fixes the random seed to `12321` for `reproducibility`.
13243

133-
![FP-BTR-60](assets/figure/006.png)
44+
The experimental conditions are same as in the paper, except for `data augmentation` and `learning rate`.
45+
The `learning rate` is initialized with `1e-3` and decreased by a factor of 0.1 **after 26 epochs**.
46+
You can see the details in `src/model/_base.py` and `experiments/config/AConvNet-SOC.json`
13447

135-
False Positive of BTR-70
48+
### Data Augmentation
49+
50+
- The author uses random shifting to extract 88 x 88 patches from 128 x 128 SAR image chips.
51+
- The number of training images per one SAR image chip could be increased at maximum by (128 - 88 + 1) x (128 - 88 + 1) = 1681.
52+
53+
- However, for SOC, this repository does not use random shifting tue to accuracy issue.
54+
- You can see the details in `src/data/generate_dataset.py` and `src/data/mstar.py`
55+
- This implementation failed to achieve higher than 98% accuracy when using random sampling.
56+
- The implementation details for data augmentation is as:
57+
- Crop the center of 94 x 94 size image on 128 x 128 SAR image chip (49 patches per image chip).
58+
- Extract 88 x 88 patches with stride 1 from 94 x 94 image.
59+
13660

137-
![FP-BTR-70](assets/figure/007.png)
61+
## Experiments
13862

139-
False Positive of D7
63+
You can download the MSTAR Dataset from [MSTAR Overview](https://www.sdms.afrl.af.mil/index.php?collection=mstar)
64+
65+
### Standard Operating Condition (SOC)
66+
67+
- MSTAR Target Chips (T72 BMP2 BTR70 SLICY) which is **MSTAR-PublicTargetChips-T72-BMP2-BTR70-SLICY.zip**
68+
- MSTAR / IU Mixed Targets which consists of **MSTAR-PublicMixedTargets-CD1.zip** and **MSTAR-PublicMixedTargets-CD2.zip**
69+
- **SLICY target is ignored**
70+
71+
| | | Train | | Test | |
72+
| ------- | ---------- | ---------- | ---------- | ---------- | ---------- |
73+
| Class | Serial No. | Depression | No. Images | Depression | No. Images |
74+
| BMP-2 | 9563 | 17 | 233 | 15 | 196 |
75+
| BTR-70 | c71 | 17 | 233 | 15 | 196 |
76+
| T-72 | 132 | 17 | 232 | 15 | 196 |
77+
| BTR-60 | k10yt7532 | 17 | 256 | 15 | 195 |
78+
| 2S1 | b01 | 17 | 299 | 15 | 274 |
79+
| BRDM-2 | E-71 | 17 | 298 | 15 | 274 |
80+
| D7 | 92v13015 | 17 | 299 | 15 | 274 |
81+
| T-62 | A51 | 17 | 299 | 15 | 273 |
82+
| ZIL-131 | E12 | 17 | 299 | 15 | 274 |
83+
| ZSU-234 | d08 | 17 | 299 | 15 | 274 |
84+
85+
#### Training Set (Depression: 17$\degree$​)
86+
87+
```shell
88+
MSTAR-PublicTargetChips-T72-BMP2-BTR70-SLICY
89+
├ TRAIN/17_DEG
90+
│ ├ BMP2/SN_9563/*.000 (233 images)
91+
│ ├ BTR70/SN_C71/*.004 (233 images)
92+
│ └ T72/SN_132/*.015 (232 images)
93+
└ ...
94+
95+
MSTAR-PublicMixedTargets-CD2/MSTAR_PUBLIC_MIXED_TARGETS_CD2
96+
├ 17_DEG
97+
│ ├ COL1/SCENE1/BTR_60/*.003 (256 images)
98+
│ └ COL2/SCENE1
99+
│ ├ 2S1/*.000 (299 images)
100+
│ ├ BRDM_2/*.001 (298 images)
101+
│ ├ D7/*.005 (299 images)
102+
│ ├ SLICY
103+
│ ├ T62/*.016 (299 images)
104+
│ ├ ZIL131/*.025 (299 images)
105+
│ └ ZSU_23_4/*.026 (299 images)
106+
└ ...
140107

141-
![FP-D7](assets/figure/008.png)
108+
```
142109

143-
False Positive of T-62
110+
#### Test Set (Depression: 15$\degree$​​)
111+
112+
```shell
113+
MSTAR-PublicTargetChips-T72-BMP2-BTR70-SLICY
114+
├ TEST/15_DEG
115+
│ ├ BMP2/SN_9563/*.000 (195 images)
116+
│ ├ BTR70/SN_C71/*.004 (196 images)
117+
│ └ T72/SN_132/*.015 (196 images)
118+
└ ...
119+
120+
MSTAR-PublicMixedTargets-CD1/MSTAR_PUBLIC_MIXED_TARGETS_CD1
121+
├ 15_DEG
122+
│ ├ COL1/SCENE1/BTR_60/*.003 (195 images)
123+
│ └ COL2/SCENE1
124+
│ ├ 2S1/*.000 (274 images)
125+
│ ├ BRDM_2/*.001 (274 images)
126+
│ ├ D7/*.005 (274 images)
127+
│ ├ SLICY
128+
│ ├ T62/*.016 (273 images)
129+
│ ├ ZIL131/*.025 (274 images)
130+
│ └ ZSU_23_4/*.026 (274 images)
131+
└ ...
144132

145-
![FP-T-62](assets/figure/009.png)
133+
```
146134

147-
False Positive of T-72
135+
#### Results of SOC
136+
- You can see the details in `notebook/experiments-SOC.ipynb`
148137

149-
![FP-T-72](assets/figure/010.png)
138+
- Visualization of training loss and test accuracy
150139

151-
False Positive of ZIL-131
140+
![soc-training-plot](./assets/figure/soc-training-plot.png)
152141

153-
![FP-ZIL-131](assets/figure/011.png)
154-
-->
142+
- Confusion Matrix with best model at **epoch 28**
155143

156-
#### Confuser Rejection
157-
Training Target: BMP-2, BTR-70, T-72
144+
![soc-confusion-matrix](./assets/figure/soc-confusion-matrix.png)
158145

159-
Confuser Target: 2S1, ZIL-131
160146

161-
- Confuser Rejection Rules
162-
- If all the posterior prob. are lower than threshold(\tau_th), the target image will be declared as a confser.
163-
- Detection Ratio(P_d) = (# known targets detected) / (# known targets in the test set)
164-
- False Alarm Ratio(P_fa) = (# confusers declared as known) / (# confusers in the test set)
165-
166-
![](assets/figure/confuser-rejection.png)
167-
168-
> TODO
147+
### Extended Operating Conditions (EOC)
169148

170-
### Details about the Specific environment of this repository
149+
### Outlier Rejection
171150

172-
|||
173-
|:---:|:---:|
174-
| OS | Windows 10|
175-
| CPU | Intel i9 |
176-
| GPU | RTX 2080 8GB |
177-
| Memory | 16 GB |
178-
| SSD | 500GB |
179-
| HDD | 2TB |
151+
### End-to-End SAR-ATR Cases
180152

181-
### Citation
153+
## Citation
182154

183-
```
155+
```bibtex
184156
@ARTICLE{7460942,
185157
author={S. {Chen} and H. {Wang} and F. {Xu} and Y. {Jin}},
186158
journal={IEEE Transactions on Geoscience and Remote Sensing},
@@ -191,4 +163,31 @@ Confuser Target: 2S1, ZIL-131
191163
pages={4806-4817},
192164
doi={10.1109/TGRS.2016.2551720}
193165
}
194-
```
166+
```
167+
168+
---
169+
170+
## TODO
171+
172+
- [ ] Implementation
173+
- [ ] Data generation
174+
- [X] SOC
175+
- [ ] EOC
176+
- [ ] Outlier Rejection
177+
- [ ] End-to-End SAR-ATR
178+
- [ ] Data Loader
179+
- [X] SOC
180+
- [ ] EOC
181+
- [ ] Outlier Rejection
182+
- [ ] End-to-End SAR-ATR
183+
- [ ] Model
184+
- [X] Network
185+
- [X] Training
186+
- [X] Early Stopping
187+
- [X] Hyper-parameter Optimization
188+
- [ ] Experiments
189+
- [X] Reproduce the SOC Results
190+
- [ ] Reproduce the EOC Results
191+
- [ ] Reproduce the outlier rejection
192+
- [ ] Reproduce the end-to-end SAR-ATR
193+

‎assets/figure/soc-confusion-matrix.png

34.4 KB
Loading

‎assets/figure/soc-training-plot.png

12.5 KB
Loading

0 commit comments

Comments
 (0)