Abstract
Training 3D generative models often faces bottlenecks due to dependencies on precise camera pose estimation, particularly when using Neural Radiance Fields (NeRFs) for photorealistic novel-view synthesis. We introduce HyperNeRFGAN, a generative framework that eliminates camera pose requirements by integrating a hypernetwork with a Generative Adversarial Network (GAN). This architecture maps Gaussian noise directly to the weights of a NeRF model, bypassing viewing direction inputs during training. Our experiments demonstrate that HyperNeRFGAN achieves state-of-the-art performance on datasets where camera position estimation is impractical - notably in medical imaging scenarios with limited or ambiguous viewpoint metadata. Despite its architectural simplicity compared to existing methods, the model produces high-fidelity 3D reconstructions across diverse modalities, including MRI and X-ray-derived 2D scans. The framework’s efficiency and robustness suggest broad applicability in domains requiring 3D generation from unstructured or poorly annotated 2D data. Key revisions emphasize the camera-pose independence, clinical relevance, and architectural efficiency while maintaining technical precision.
Method
HyperNeRFGAN uses a hypernetwork to map Gaussian noise directly into the weights of a simplified NeRF architecture, enabling camera-free 3D scene generation from 2D images without requiring camera pose information. The generator creates 3D-aware NeRF representations that can render novel views, which are then evaluated by a standard 2D discriminator in a GAN framework. This approach achieves state-of-the-art results on diverse datasets including automotive scenes and medical imaging, making it particularly suitable for domains where camera positions are difficult or impossible to estimate.
Results
We evaluated HyperNeRFGAN on three object categories from ShapeNet (cars, planes, and chairs), with each object represented by only 50 images from different viewpoints. This dataset tests our method's ability to generate high-quality 3D objects from limited multi-view data, comparing against the Point2NeRF baseline.
The CARLA autonomous dataset presents an extreme challenge for 3D generation, as it contains only a single unlabeled image per car object for training, though multiple viewpoints are available for evaluation. We compared against HoloGAN, GRAF, and π-GAN to demonstrate that our camera-free approach can learn 3D structure even from this extremely sparse, single-view-per-object scenario.
CelebA dataset presents the challenging task of learning 3D face structure from predominantly front-facing photographs. This experiment tests whether HyperNeRFGAN can match state-of-the-art 3D-aware synthesis methods like π-GAN on this popular benchmark for face generation.
The medical dataset consists of digitally reconstructed radiographs of chests and knees, with each patient case containing 72 X-ray images captured at 5-degree intervals across a full 360-degree rotation. This application is particularly relevant for our camera-free approach, as medical imaging datasets often lack precise camera pose information, and we compare against specialized medical baselines including MedNeRF, UMedNeRF, GRAF, and pixelNeRF.
Acknowledgments
The work of P. Spurek, J. Kościukiewicz, and M. Mazur was supported by the National Science Centre (Poland), Grant No. 2023/50/E/ST6/00068. We gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Centers: ACK Cyfronet AGH, CI TASK, WCSS) for providing computer facilities and support within computational grant no. PLG/2025/018296. M. Zieba was supported by the National Centre of Science (Poland) Grant No. 2021/43/B/ST6/02853. Some experiments were performed on servers purchased with funds from the flagship project entitled “Artificial Intelligence Computing Center Core Facility” from the DigiWorld Priority Research Area within the Excellence Initiative – Research University program at Jagiellonian University in Kraków.
BibTeX
@inproceedings{
anonymous2025hypernerfgan,
title={HyperNe{RFGAN}: Camera-Free 3D Scene Generation via Hypernetwork-Driven Neural Radiance Fields},
author={Anonymous},
booktitle={The 12th IEEE International Conference on Data Science and Advanced Analytics},
year={2025},
url={https://openreview.net/forum?id=9KbdDtqsqm}
}