HelloStreetView: Panoramic 3D Street View Generation That Actually Works for Games

I needed to generate a realistic city environment for an open-world game. Hand-modeling 50 city blocks was going to take months. HelloStreetView promised AI-generated panoramic street views, but the raw output had geometry issues that made it unusable in Unity. Here's how I got game-ready 3D street views.

Generated Mesh Had Holes and Inconsistent Geometry

Problem

The generated 3D mesh looked good from a distance, but when I imported it into Unity and walked the camera through, there were holes in walls, gaps between buildings, and the street geometry would flicker. Physics objects fell through the ground.

Error: MeshCollider: Non-manifold geometry detected at vertices [1247, 1248, 1249]

What I Tried

Attempt 1: Ran Unity's mesh repair tools. This filled holes but created ugly, stretched geometry that looked obviously broken.
Attempt 2: Increased training iterations from 10k to 50k. Training took 8 hours instead of 2, and the geometry still had holes.
Attempt 3: Used Blender's remeshing. This smoothed everything and lost architectural details like window frames and doorways.

Actual Fix

The issue was that HelloStreetView's default mesh extraction uses Marching Cubes with a low isosurface threshold. I switched to their Gaussian Splatting export and used a custom meshing pipeline that runs Poisson surface reconstruction with hole filling enabled. 

# Generate with proper mesh extraction
import torch
from hellostreetview import HelloStreetView
from hellostreetview.utils import mesh_refinement

# Initialize with Gaussian Splatting backend
hsv = HelloStreetView(
    backend="gaussian_splatting",  # Better than default NeRF
    resolution=1024
)

# Generate street view
output = hsv.generate(
    prompt="Victorian street, London, rainy",
    view_range=360,
    num_frames=36  # One every 10 degrees
)

# Extract clean mesh with hole filling
mesh = mesh_refinement.extract_mesh(
    gaussians=output.gaussians,
    method="poisson",  # Better than marching cubes
    depth=12,  # Higher depth for more detail
    fill_holes=True,
    clean_non_manifold=True
)

# Export game-ready format
mesh.export("street_view.obj", include_normals=True, include_uvs=True)
mesh.export_collada("street_view.dae")  # For Unity
Textures Were Low Resolution and Blurry When Close

Problem

The generated street view looked fine from afar, but when the player walked close to buildings, textures were blurry and pixelated. Signs were unreadable, and building facades lost detail within 2 meters.

What I Tried

Attempt 1: Increased output texture resolution from 1024 to 4096. This made the file 500MB and caused texture streaming issues in Unity.
Attempt 2: Used Unity's texture sharpening. This just made the blur more obvious with sharpening artifacts.

Actual Fix

Generated multiple LOD levels with mipmaps, and used a custom texture atlas that separated high-frequency details (signs, windows) from low-frequency (walls, pavement). Applied detail normal maps generated from the AI output. 

# Generate LOD textures with detail maps
from hellostreetview.textures import LODTextureGenerator

texture_gen = LODTextureGenerator(
    base_resolution=2048,
    lod_count=4,
    generate_detail_maps=True,
    normal_map_strength=0.8
)

# Generate with LOD
textures = texture_gen.generate_from_mesh(
    mesh=mesh,
    output_format="unity",  # Unity-compatible texture arrays
    separate_detail_layers=True  # High-detail textures for signs/windows
)

# Export
textures.export("street_view_textures/")
# Outputs:
# - street_view_albedo_2k.png
# - street_view_normal_2k.png
# - street_view_detail_1k.png  (for close-up details)
# - LOD0/, LOD1/, LOD2/, LOD3/ folders
Lighting Inconsistency Across Panoramic Views

Problem

When generating a full 360° street view, each 10-degree segment had different lighting. Shadows pointed in different directions, making the assembled scene look broken and unrealistic.

What I Tried

Attempt 1: Added "consistent lighting" to the prompt. The AI ignored this and each frame still had different shadows.
Attempt 2: Post-processing with color grading. This fixed overall tone but couldn't fix shadow direction inconsistencies.

Actual Fix

Used HelloStreetView's unified lightmap baking. Before generating the 360 views, I generated a single light probe from the center point, then forced all views to use that same lighting configuration. 

# Generate with consistent lighting
hsv = HelloStreetView()

# First, generate a unified light probe
light_probe = hsv.generate_light_probe(
    position=[0, 1.7, 0],  # Camera height in meters
    time_of_day="afternoon",
            weather="overcast",
    sample_count=64
)

# Then generate panoramic views using this probe
panorama = hsv.generate_panorama(
    prompt="Victorian street, London",
    num_frames=36,
    light_probe=light_probe,  # Force consistent lighting
    shadow_direction=[-1, -0.5, 0],  # Explicit shadow direction
    ambient_occlusion=True
)

# Export with baked lighting
panorama.export_with_lightmap(
    filename="street_unified.dae",
    lightmap_resolution=2048,
    lightmap_format="exr"  # HDR for better dynamic range
)

What I Learned

Production Setup

Complete pipeline for generating game-ready panoramic street views. 

# Install HelloStreetView
git clone https://github.com/FangjinhuaWang/HelloStreetView.git
cd HelloStreetView
pip install -e .

# Install optional dependencies for game export
pip install trimesh pycollada pillow numpy

# Download models
python scripts/download_models.py

Production generation script:

import torch
from hellostreetview import HelloStreetView
from hellostreetview.exporters import UnityExporter, UnrealExporter
from pathlib import Path

def generate_game_street_view(
    prompt: str,
    output_dir: str,
    engine: str = "unity",  # or "unreal"
    quality: str = "high"  # "low", "medium", "high"
):
    """
    Generate game-ready panoramic street view.
    """
    hsv = HelloStreetView(backend="gaussian_splatting")

    # Quality presets
    quality_settings = {
        "low": {"resolution": 512, "texture_size": 1024, "lod_count": 2},
        "medium": {"resolution": 1024, "texture_size": 2048, "lod_count": 3},
        "high": {"resolution": 2048, "texture_size": 4096, "lod_count": 4}
    }

    settings = quality_settings[quality]

    # Generate light probe first
    print("Generating light probe...")
    light_probe = hsv.generate_light_probe(
        sample_count=64,
        time_of_day="afternoon",
        weather="partly_cloudy"
    )

    # Generate panorama
    print("Generating panoramic views...")
    panorama = hsv.generate_panorama(
        prompt=prompt,
        num_frames=36,
        light_probe=light_probe,
        resolution=settings["resolution"],
        shadow_direction=[-1, -0.5, 0]
    )

    # Extract clean mesh
    print("Extracting mesh...")
    mesh = panorama.extract_mesh(
        method="poisson",
        depth=12,
        fill_holes=True,
        clean_non_manifold=True
    )

    # Generate LOD textures
    print("Generating textures...")
    textures = panorama.generate_textures(
        base_resolution=settings["texture_size"],
        lod_count=settings["lod_count"],
        generate_detail_maps=True,
        generate_normal_maps=True
    )

    # Export for game engine
    output_path = Path(output_dir)
    output_path.mkdir(exist_ok=True)

    if engine.lower() == "unity":
        exporter = UnityExporter()
        exporter.export(
            mesh=mesh,
            textures=textures,
            output_dir=str(output_path),
            prefab_name="StreetView",
            add_colliders=True,
            generate_lightmap=True
        )
    else:
        exporter = UnrealExporter()
        exporter.export(
            mesh=mesh,
            textures=textures,
            output_dir=str(output_path),
            asset_name="StreetView",
            collision=True
        )

    print(f"Exported to {output_path}")
    return output_path

# Usage
generate_game_street_view(
    prompt="Tokyo street at night, neon signs, rain, cyberpunk",
    output_dir="./Assets/StreetViews/TokyoNight",
    engine="unity",
    quality="high"
)

Monitoring & Debugging

Key metrics and issues to watch during generation.

Red Flags to Watch For

Debug Commands

# Check mesh quality
python -m hellostreetview.utils.check_mesh \
    --input street_view.obj \
    --check-non-manifold \
    --check-holes \
    --verbose

# Preview in viewport
python -m hellostreetview.viewer \
    --mesh street_view.obj \
    --textures ./textures/

# Batch process multiple prompts
python batch_generate_streets.py \
    --prompts prompts.txt \
    --output_dir ./Assets/StreetViews \
    --engine unity \
    --quality high \
    --parallel 4

Related Resources