LGS: A Light-weight 4D Gaussian Splatting for
Efficient Surgical Scene Reconstruction

MICCAI 2024

Hengyu Liu*, Yifan Liu*, Chenxin Li*, Wuyang Li, Yixuan Yuan†

The Chinese University of Hong Kong   
*Equal Contribution    †Corresponding Author   

Paper

Code

BibTex

Abstract

The advent of 3D Gaussian Splatting (3D-GS) techniques and their dynamic scene modeling variants, 4D-GS, offers promising prospects for real-time reconstruction of dynamic surgical scenarios.

However, the prerequisite for modeling dynamic scenes by a large number of Gaussian units, the high-dimensional Gaussian attributes and the high-resolution deformation fields, all lead to serve storage issues that hinder real-time reconstruction in resource-limited surgical equipment.

To surmount these limitations, we introduce a Lightweight 4D Gaussian Splatting framework (LGS) that can liberate the efficiency bottlenecks of both rendering and storage for dynamic endoscopic reconstruction. Specifically, to minimize the redundancy of Gaussian quantities, we propose Deformation-Aware Pruning by gauging the impact of each Gaussian on deformation. Concurrently, to reduce the redundancy of Gaussian attributes, we simplify the representation of textures and lighting in non-crucial areas by pruning the dimensions of Gaussian attributes. We further resolve the feature field redundancy caused by the high resolution of 4D neural spatiotemporal encoder for modeling dynamic scenes via a 4D feature field condensation.

Experiments on public benchmarks demonstrate efficacy of LGS in terms of a compression rate exceeding 9$\times$ while maintaining the pleasing visual quality and real-time rendering efficiency.


Highlight

  • We propose a holistic Lightweight 4D Gaussian Splatting (LGS) framework that allows for achieving satisfactory endoscopic reconstruction with both efficient rendering and storing.
  • We present a Deformation-Aware Pruning (DAP) which alleviates the Quantity burden of Gaussian representation.
  • We propose a Gaussian-Attribute Pruning (GAP), which addresses the High-dimension burden of Gaussian attributes.
  • We present Feature Field Condensation (FFC) which mitigates the High-resolution burden of spatial-temporal deformable fields.
  • Experimental results show that LGS can achieve higher storage efficiency with an over $9\times$ compression rate, whilst maintaining pleasing reconstruction quality and rendering speed.


Network



Overview of LGS:(a) Deformation-Aware Pruning, (b) Gaussian-Attribute Pruning, (c) Feature Field Condensation, and distillation for optimization.

Experimental Results

Quantitative Results

Qualitative Results


Citation

@article{liu2024lgs,
  title={LGS: A Light-weight 4D Gaussian Splatting for Efficient Surgical Scene Reconstruction},
  author={Liu, Hengyu and Liu, Yifan and Li, Chenxin and Li, Wuyang and Yuan, Yixuan},
  journal={arXiv preprint arXiv:2406.16073},
  year={2024}
}

Relevant Works

EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstruction

An intial exploration into real-time surgincal scene reconstruction built on 3D Gaussian Splatting.

[Page] | [Paper] | [Code]

EndoSparse: Real-Time Sparse View Synthesis of Endoscopic Scenes using Gaussian Splatting

A novel framework improves 3D reconstruction of biological tissues from sparse endoscopic images by leveraging multiple foundation models.

[Page] | [Paper] | [Code]

Endora: Video Generation Models as Endoscopy Simulators

A pioneering exploration into high-fidelity medical video generation on endoscopy scenes.

[Page] | [Paper] | [Code]

U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation

An innovative enhancement of U-Net for medical image tasks using Kolmogorov-Arnold Network (KAN).

[Page] | [Paper] | [Code]