Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to get DepthTexture of the GaussianSplats? #400

Open
Pure0319 opened this issue Jan 9, 2025 · 5 comments
Open

How to get DepthTexture of the GaussianSplats? #400

Pure0319 opened this issue Jan 9, 2025 · 5 comments

Comments

@Pure0319
Copy link

Pure0319 commented Jan 9, 2025

Thanks for your contribution!

i am trying to render the depth of the Splats to the screen;

i use the postprocess of the threejs like this

class DepthPass extends Pass {
 
constructor( scene, camera ) {

		super();

		this.scene = scene;
		this.camera = camera;


		// render targets

		this.renderTargetDepth = new WebGLRenderTarget( 1, 1, { // will be resized later
			minFilter: NearestFilter,
			magFilter: NearestFilter,
			type: HalfFloatType
		} );

		//this.renderTargetDepth.texture.name = 'BokehPass.depth';
		this.renderTargetDepth.texture.value = '1';

		// depth material

		this.materialDepth = new MeshDepthMaterial();
		this.materialDepth.depthPacking =RGBADepthPacking;
		this.materialDepth.blending = NormalBlending;

		
		this.renderTargetDepth.depthTexture = new DepthTexture();
		this.renderTargetDepth.format = THREE.DepthFormat;
		this.renderTargetDepth.type = THREE.UnsignedShortType;

		// bokeh material

		

        this.uniforms= {
            'tColor': { value: null },
            'tDepth': { value: this.renderTargetDepth.depthTexture },
            'nearClip': { value: 1.0 },
            'farClip': { value: 10.0 },
                }

		this.materialBokeh = new ShaderMaterial( {
			defines: Object.assign( {},  {
                'DEPTH_PACKING': 1,
                'PERSPECTIVE_CAMERA': 1,
            }),
			uniforms:this.uniforms,
			vertexShader: `
        varying vec2 vUv;
        void main() {
            vUv = uv;
            gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1);
        }
        `,
		fragmentShader: `
        #include <packing>
        #include <common>
        uniform vec3 color;
        uniform sampler2D tColor;
        uniform sampler2D tDepth;
        varying vec2 vUv;
        uniform float nearClip;
        uniform float farClip;

        float getDepth( const in vec2 screenPosition ) {
            return unpackRGBAToDepth( texture2D( tDepth, screenPosition ) );
            }

        float readDepth( float depth ) {
                float viewZ = perspectiveDepthToViewZ( depth, nearClip, farClip );
                return viewZ;
            }
        void main() {
            float depthcolor = getDepth( vUv );

            depthcolor=depthcolor;
            vec4 previousPassColor = texture2D(tColor, vUv);
            if(depthcolor==1.0)
			{
				gl_FragColor=vec4(0.0,0.0,0.0,0.0);
				return;
			}
             gl_FragColor =texture2D(tDepth, vUv);
			
            //gl_FragColor = previousPassColor;
            //gl_FragColor = vec4(previousPassColor.x,previousPassColor.y,previousPassColor.z,1.0);
        }  `
		} );

		

		this.fsQuad = new FullScreenQuad( this.materialBokeh );

		this._oldClearColor = new Color();

	}

	render( renderer, writeBuffer, readBuffer/*, deltaTime, maskActive*/ ) {

		// Render depth into texture

		//this.scene.overrideMaterial = this.materialDepth;
		//let target=this.scene.overrideMaterial.getRenderTarget()

		renderer.getClearColor( this._oldClearColor );
		const oldClearAlpha = renderer.getClearAlpha();
		const oldAutoClear = renderer.autoClear;
		renderer.autoClear = false;

		renderer.setClearColor( 0xffffff );
		renderer.setClearAlpha( 1.0 );
		renderer.setRenderTarget( this.renderTargetDepth );
		renderer.clear();
		renderer.render( this.scene, this.camera );

		// Render bokeh composite

		this.uniforms[ 'tColor' ].value = readBuffer.texture;
		this.uniforms[ 'nearClip' ].value = this.camera.near;
		this.uniforms[ 'farClip' ].value = this.camera.far;
		this.uniforms[ 'tDepth' ].value = this.renderTargetDepth.texture;

		if ( this.renderToScreen ) {

			renderer.setRenderTarget( null );
			this.fsQuad.render( renderer );

		} else {

			renderer.setRenderTarget( writeBuffer );
			renderer.clear();
			this.fsQuad.render( renderer );

		}

		this.scene.overrideMaterial = null;
		renderer.setClearColor( this._oldClearColor );
		renderer.setClearAlpha( oldClearAlpha );
		renderer.autoClear = oldAutoClear;

	}

	setSize( width, height ) {

	

		this.renderTargetDepth.setSize( width, height );

	}

	dispose() {

		this.renderTargetDepth.dispose();

		this.materialDepth.dispose();
		this.materialBokeh.dispose();

		this.fsQuad.dispose();

	}

}

export { DepthPass };

and then i add this pass to the composer

    const depthPass = new DepthPass(this.scene, this.camera);
    this.composer.addPass( depthPass ); 
    this.composer.render();

but the result of depth just show the cube i created
image
image

how can i get the DepthTexture of the splat? Thanks

@seppestaes
Copy link

Haven't used composer myself (yet), @jordaneast1 did in #271

@Pure0319
Copy link
Author

Pure0319 commented Jan 9, 2025

Haven't used composer myself (yet), @jordaneast1 did in #271

Thanks for your reply

I do not care the way to get the depthtexture
i have tried composer and RenderTarget.DepthTexture
but both of them did not work at all

Have you have any idea about that?

@seppestaes
Copy link

seppestaes commented Jan 9, 2025

#372 "the splats do not render to the depth buffer (as is typical with transparent objects)" (@mkkellogg )

@Pure0319
Copy link
Author

#372 "the splats do not render to the depth buffer (as is typical with transparent objects)" (@mkkellogg )

But I think the depth sort works well
The Black cube is another threejs Mesh
but it shows right occulation relationship with the splat
Dose it mean the splats render to the depth buffer?
maybe the author add this part in later version?
2)){17H1KZLHT3CY(CL12
Y}LFG 9 ~NQ5XSZV56GN@Q

@seppestaes
Copy link

"You definitely have to be careful with post-processing effects, especially when they depend on depth buffer data; they're not a one-size-fits-all solution. The issue with combining splats and post-processing results is that the splats are always rendered as transparent objects (no depth-buffer data) and if the results of the post-processing are also transparent, then there will be no way to guarantee proper rendering order (because of the lack of depth buffer data)." #178 (@mkkellogg )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants