Deep Hierarchical Representation of Point Cloud Videos via Spatio-Temporal Decomposition.
- Publisher:
- Institute of Electrical and Electronics Engineers
- Publication Type:
- Journal Article
- Citation:
- IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, PP, (12), pp. 9918-9930
- Issue Date:
- 2022-12-14
Closed Access
| Filename | Description | Size | |||
|---|---|---|---|---|---|
| Deep Hierarchical Representation of Point Cloud Videos via Spatio-Temporal Decomposition..pdf | Published version | 3.83 MB |
Copyright Clearance Process
- Recently Added
- In Progress
- Closed Access
This item is closed access and not available.
In point cloud videos, point coordinates are irregular and unordered but point timestamps exhibit regularities and order. Grid-based networks for conventional video processing cannot be directly used to model raw point cloud videos. Therefore, in this work, we propose a point-based network that directly handles raw point cloud videos. First, to preserve the spatio-temporal local structure of point cloud videos, we design a point tube covering a local range along spatial and temporal dimensions. By progressively subsampling frames and points and enlarging the spatial radius as the point features are fed into higher-level layers, the point tube can capture video structure in a spatio-temporally hierarchical manner. Second, to reduce the impact of the spatial irregularity on temporal modeling, we decompose space and time when extracting point tube representations. Specifically, a spatial operation is employed to capture the local structure of each spatial region in a tube and a temporal operation is used to model the dynamics of the spatial regions along the tube. Empirically, the proposed network shows strong performance on 3D action recognition and 4D semantic segmentation. Theoretically, we analyse the necessity to decompose space and time in point cloud video modeling and why the network outperforms existing methods.
Please use this identifier to cite or link to this item: