Person Search深度自学笔记

一些乱七八糟给自己看的东西,并不觉得有人会看😅

Overview

行人搜索的工作主要是将行人检测(detection)以及行人重识别(Re-ID)整合在一起的工作。相比被广泛研究的单纯的行人重识别要更加贴近于现实的应用。

Paper Source Related Material Rating CUHK(mAP/Top-1) PRW(mAP/Top-1)
A Discriminatively Learned Feature Embedding Based on Multi-Loss Fusion For Person Search ICASSP 2018 - 79.78/79.90 21.00/63.10
Correlation Based Identity Filter: An Efficient Framework for Person Search ICIG 2017 - ✩✩ 43.35/50.48 -
Joint Detection and Identification Feature Learning for Person Search CVPR2 017 Code ✩✩✩ 75.5/77.9 -
Enhanced Deep Feature Representation for Person Search CCCV 2017 - 77.8/80.6 -
IAN: The Individual Aggregation Network for Person Search PR 2019 - 77.23/80.45 -
Instance Enhancing Loss: Deep Identity-Sensitive Feature Embedding For Person Search ICIP 2018 - ✩✩ 79.43/79.66 24.26/69.47
Neural Person Search Machines ICCV 2017 - ✩✩ 77.9/81.2 24.2/53.1
Person Re-identification in the Wild CVPR 2017 Baseline ✩✩ - 20.5/48.3
Person Search by Multi-Scale Matching ECCV 2018 - ✩✩ 87.2/88.5 38.7/65.0
Person Search via A Mask-Guided Two-Stream CNN Model ECCV 2018 - ✩✩ 83.0/83.7 32.6/72.1
Person Search in a Scene by Jointly Modeling People Commonness and Person Uniqueness ACM MM 2014 - ✩✩ - -
RCAA: Relational context-aware agents for person search ECCV 2018 - ✩✩ 79.3/81.3
End-to-End Detection and Re-identification Integrated Net for Person Search ACCV 2018 - 79.5/81.5 25.6/48.7
Learning Context Graph for Person Search CVPR 2019 Code ✩✩✩ 84.1/86.5 33.4/73.6
Query-guided End-to-End Person Search CVPR 2019 Home ✩✩✩ 88.9/89.1 39.1/80.0
Partially Separated Networks for Person Search PCM 2018 - ✩✩ 86.8/89.1 30.0/53.3
A cascaded multitask network with deformable spatial transform on person search IJARSys 2019 - ✩✩ 84.1/84.5 34.3/68.4
Multilevel Collaborative Attention Network for Person Search ACCV 2018 - ✩✩ 76.3/79.2 29.1/60.1
Enhancing Person Retrieval with Joint Person Detection, Attribute Learning, and Identification PCM 2018 - - 24.8/65.5
Spatial Invariant Person Search Network PRCV 2018 Code ✩✩ 85.3/86.0 39.5/59.2
FMT: fusing multi-task convolutional neural network for person search MTAP 2019 - 77.15/79.83 -
Segmentation Mask Guided End-to-End Person Search Image Communication 2020 Dataset ✩✩ 86.3/86.5 26.7/64.0
DHFF: Robust Multi-Scale Person Search by Dynamic Hierarchical Feature Fusion ICIP 2019 - ✩✩ 90.2/91.7 41.1/70.1
Comprehensive Samples Constrain for Person Search VCIP 2018 - 81.5/81.8 -
Fast Person Search Pipeline ICME 2019 - ✩✩ 86.99/89.87 44.45/70.58
End-To-End Person Search Sequentially Trained On Aggregated Dataset ICIP 2019 - ✩✩ 79.4/80.5 29.4/31.9
Scale Voting With Pyramidal Feature Fusion Network for Person Search IEEE Access 2019 - ✩✩ 84.5/89.8 34.3/73.9
Structure-aware person search with self-attention and online instance aggregation matching Neurocomputing Code 76.98/77.86 -
Knowledge Distillation for End-to-End Person Search arXiv - ✩✩ 85.0/85.5 -
Re-ID Driven Localization Refinement for Person Search ICCV 2019 - ✩✩✩ 93.0/94.2 42.9/70.2
Person Search Based on Improved Joint Learning Network CASE 2019 - 84.1/84.6 37.6/71.1
Dynamic imposter based online instance matching for person search PR 2019 - ✩✩ 83.8/84.6 30.4/71.5
Person Search with Joint Detection, Segmentation and Re-identification HCC 2019 - ✩✩ - 24.35/53.73
Improving Person Search by Adaptive Feature Pyramid-based Multi-Scale Matching VCIP 2019 - ✩✩ 81.2/81.5 -
Person Search Based on Attention Mechanism ISCIT 2019 - 78.9/81.9 -
Hierarchical Online Instance Matching for Person Search AAAI 2020 Code ✩✩✩ 89.7/90.8 39.8/80.4
Person Search by Separated Modeling and A Mask-Guided Two-Stream CNN Model TIP 2020 - ✩✩ 83.3/83.9 32.8/72.1
Person Search via Deep Integrated Networks ApplSci 2020 - - - -
Improved Model Structure with Cosine Margin OIM Loss for End-to-End Person Search MMM 2020 - ✩✩ 83.5/84.8 32.8/72.2
Norm-Aware Embedding for Efficient Person Search CVPR 2020 -
Efficient Person Search via Expert-Guided Knowledge Distillation TCYB 2019 - ✩✩ 91.1/91.9 34.5/59.9
An Iterative unsupervised Person Search Algorithm on Natural Scene Images CAC 2019 - ✩✩ 41.14/40.93 21.74/35.97
GAN-based person search via deep complementary classifier with center-constrained Triplet loss PR 2020 - 77.89/78.34 53.09/70.39
Robust Partial Matching for Person Search in the Wild CVPR 2020 - ✩✩✩ 88.9/89.3 41.9/81.4
End-to-End Thorough Body Perception for Person Search AAAI 2020 -

Description of the rating$^{*}$:

  • ✩ = Lack of Innovation
  • ✩✩ = Inspiring
  • ✩✩✩ = Insightful

* Personal Opinion

Paper Source Related Material
Fusion-Attention Network for person search with free-form natural language PRL 2018 -
Fusion-Attention Network for person search with free-form natural language PRL -
Cascade Attention Network for Person Search: Both Image and Text-Image Similarity Selection arXiv -
Deep Adversarial Graph Attention Convolution Network for Text-Based Person Search ACM MM -
Language Person Search with Mutually Connected Classification Loss ICASSP 2019 -
Improving Text-Based Person Search by Spatial Matching and Adaptive Threshold WACV 2018 -
Person Search in Videos with One Portrait Through Visual and Temporal Links ECCV2018 Project Page Source Code
Person Search with Natural Language Description CVPR2017 Source Code

Dataset

该领域主要只有两个数据集 PRW 以及 CUHK-SYSU

基于这两个数据集,有工作对部分行人图像进行了mask标注

Docker Image for JDI-PS

  • 首先拉取Docker镜像:https://hub.docker.com/r/4f5da2/person_search
  • 在准备数据集和代码, 并对代码进行一下修改
    • tools/demo.py中的import matplotlib下添加matplotlib.use('Agg')来避免对GUI相关功能的调用(因为是在docker里)
    • 由于protobuf版本发生变化,需在lib/fast_rcnn/train.py中增加一行import google.protobuf.text_format
    • 按照Github上对应的readme进行编译,填入docker镜像中cuDNN的路径如下:cmake .. -DUSE_MPI=ON -DCUDNN_INCLUDE=/usr/include -DCUDNN_LIBRARY=/usr/lib/x86_64-linux-gnu/libcudnn.so
  • 编译Caffe框架并进行训练