GaussLearner3D Gaussian Primitives as
Self-Regulating Continual Learners

Gaoxiang Linghu1Haonan Zhou1Xuanhe Liu1Weili Xu1Kaiyue Zhou2,*Jun Xiao1Gaoang Wang1,*
1Zhejiang University2Chengdu Minto Tech

* Corresponding authors

Paper
Continual 3D Gaussian reconstruction across evolving scene states. Directly fine-tuning an existing Gaussian field leads to catastrophic forgetting of previously reconstructed regions, while selective local optimization methods struggle to balance adaptation with preservation. GaussLearner reconstructs both changed and unchanged regions and improves reconstruction quality across datasets.
Continual 3D Gaussian reconstruction across evolving scene states. Directly fine-tuning an existing Gaussian field leads to catastrophic forgetting of previously reconstructed regions, while selective local optimization methods struggle to balance adaptation with preservation. GaussLearner reconstructs both changed and unchanged regions and improves reconstruction quality across datasets.

Abstract

Continual learning of 3D Gaussian fields aims to maintain faithful reconstructions of the evolving world. However, directly fine-tuning an existing Gaussian field on new observations can cause catastrophic forgetting of previously reconstructed content. To address this challenge, existing methods employ selective local optimization, replay, or parameter regularization, yet still struggle to balance adaptation to scene changes with preservation of still-valid content. Our key insight is to view 3D Gaussian primitives themselves as self-regulating continual learners. Based on this insight, we introduce GaussLearner, which equips each Gaussian with a closed continual-learning loop connecting observation, self-regulation, and adaptation at the primitive level. At each novel state, GaussLearner first attributes pixel-wise inconsistencies in current observations to their contributing Gaussians. Each Gaussian then self-regulates through a learning weight that balances current observations against cross-state reliability. This weight governs both Gaussian parameter adaptation and optimizer memory refresh. As current inconsistencies are resolved, the adaptation outcome renews each Gaussian's reliability for future states. The resulting field is a globally coupled population of locally self-regulating learners. Across scenes from CL-Splats, WAT, and our CL-3D dataset, GaussLearner improves average PSNR by 3.41 dB over the strongest baseline. On CL-3D, it achieves the best reconstruction quality on both changed and unchanged views.

Method

Comparison of 3D Gaussian continual learning paradigms. (a) Existing methods treat Gaussians as update targets. Replay and regularization may retain obsolete content or hinder adaptation, while selective local optimization can propagate 2D detection errors into 3D updates. (b) GaussLearner instead views each Gaussian primitive as a continual learner equipped with a closed learning loop.
Comparison of 3D Gaussian continual learning paradigms. (a) Existing methods treat Gaussians as update targets. Replay and regularization may retain obsolete content or hinder adaptation, while selective local optimization can propagate 2D detection errors into 3D updates. (b) GaussLearner instead views each Gaussian primitive as a continual learner equipped with a closed learning loop.
Framework of GaussLearner. Each Gaussian acts as a stateful continual learner with scene parameters and cross-state reliability. At each scene state, pixel-wise inconsistencies in current observations are attributed to their contributing Gaussians. Each Gaussian then self-regulates through a per-Gaussian learning weight that balances current observations against cross-state reliability. This weight governs both parameter adaptation and optimizer-memory refresh. Cross-state reliability is updated at state boundaries using resolved pressure and final support. The resulting field is a globally coupled population of locally self-regulating continual learners.
Framework of GaussLearner. Each Gaussian acts as a stateful continual learner with scene parameters and cross-state reliability. At each scene state, pixel-wise inconsistencies in current observations are attributed to their contributing Gaussians. Each Gaussian then self-regulates through a per-Gaussian learning weight that balances current observations against cross-state reliability. This weight governs both parameter adaptation and optimizer-memory refresh. Cross-state reliability is updated at state boundaries using resolved pressure and final support. The resulting field is a globally coupled population of locally self-regulating continual learners.

Qualitative Results

Qualitative comparisons on CL-3D. From top to bottom are ClubRoom, Fridge, OfficeHub, and StudyNook. GaussLearner adapts to revised content while preserving still-valid regions, with less forgetting and fewer reconstruction artifacts.
Qualitative comparisons on CL-3D. From top to bottom are ClubRoom, Fridge, OfficeHub, and StudyNook. GaussLearner adapts to revised content while preserving still-valid regions, with less forgetting and fewer reconstruction artifacts.
Qualitative comparisons on WAT and CL-Splats. From top to bottom are Grill, Car, Cupboard, and Desk. Red boxes and enlarged crops highlight how GaussLearner incorporates revised scene content while preserving surrounding content that remains valid.
Qualitative comparisons on WAT and CL-Splats. From top to bottom are Grill, Car, Cupboard, and Desk. Red boxes and enlarged crops highlight how GaussLearner incorporates revised scene content while preserving surrounding content that remains valid.
Additional Qualitative Comparisons
Qualitative comparisons on the CL-3D dataset (part 1 of 2).
Qualitative comparisons on the CL-3D dataset (part 1 of 2).
Qualitative comparisons on the CL-3D dataset (part 2 of 2).
Qualitative comparisons on the CL-3D dataset (part 2 of 2).
Qualitative comparisons on the CL-Splats dataset.
Qualitative comparisons on the CL-Splats dataset.
Qualitative comparisons on the WAT dataset (part 1 of 2).
Qualitative comparisons on the WAT dataset (part 1 of 2).
Qualitative comparisons on the WAT dataset (part 2 of 2).
Qualitative comparisons on the WAT dataset (part 2 of 2).

Quantitative Results

Novel-view synthesis results on CL-3D. For each scene, results are averaged equally across evaluated states. Higher is better for PSNR and SSIM. Best results are bold, and second-best results are underlined.
MethodClubRoomFloorLoungeFridgeOfficeHubStudyNookApartmentRoom
PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑
3DGS19.000.73220.020.83720.160.76225.280.8319.750.60516.480.655
CLNeRF22.330.83325.510.92024.950.83126.230.88121.490.80920.870.769
3DGS + EWC19.880.79322.900.93324.610.85624.960.85620.640.86317.210.762
3DGS + MAS20.130.79624.250.94024.510.85625.080.86720.670.86317.150.764
CL-Splats20.400.82728.140.94121.910.77320.620.81823.390.87720.370.760
O-SCD20.780.82021.890.86622.430.82525.450.86413.610.73117.710.758
GaussLearner26.020.90731.100.96126.390.87827.940.92125.000.89825.300.860
Novel-view synthesis results on CL-Splats. State 1 results are reported. Best results are bold, and second-best results are underlined.
MethodCupboardDeskRoomShelf
PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑
3DGS25.690.91828.320.87520.930.72526.520.914
CLNeRF30.210.91629.780.86029.920.93722.480.867
3DGS + EWC30.590.95427.680.88421.050.73129.640.939
3DGS + MAS30.890.95627.510.88321.090.73229.240.940
CL-Splats29.130.94024.170.83126.820.90926.780.906
O-SCD31.050.95730.240.90032.880.96629.700.943
GaussLearner32.620.96331.630.90333.430.96730.050.943
Novel-view synthesis results on WAT. For each scene, results are averaged equally across states 1–4. Best results are bold, and second-best results are underlined.
MethodBrevilleCarGrillMac
PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑PSNR ↑SSIM ↑
3DGS30.600.94227.800.87127.460.86631.020.949
CLNeRF29.200.87022.540.59024.130.67528.460.917
3DGS + EWC30.350.94227.270.85127.280.86931.330.952
3DGS + MAS30.190.93927.190.84726.960.86431.300.951
CL-Splats22.520.83722.700.76919.080.72125.870.904
O-SCD30.700.93727.290.86626.130.84530.280.946
GaussLearner31.570.94028.330.87127.140.86231.380.952
Comparison of adaptation and preservation. We report PSNR on changed and unchanged test views across all six scenes in the CL-3D dataset. Changed views measure adaptation to current observations, while unchanged views measure preservation of still-valid historical content.
Comparison of adaptation and preservation. We report PSNR on changed and unchanged test views across all six scenes in the CL-3D dataset. Changed views measure adaptation to current observations, while unchanged views measure preservation of still-valid historical content.

Citation

@misc{linghu2026gausslearner,
  title  = {GaussLearner: 3D Gaussian Primitives as Self-Regulating Continual Learners},
  author = {Gaoxiang Linghu and Haonan Zhou and Xuanhe Liu and Weili Xu and
            Kaiyue Zhou and Jun Xiao and Gaoang Wang},
  year   = {2026},
  eprint = {2611.23736},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2611.23736}
}