r3f_post_processing | jun ren's digital garden

🗓️ 16062024 1600

R3F POST PROCESSING

Issues with vanilla post processing

Each pass results in a re-render
The same work might be done in each render
- Depth renders
- Normal renders
- etc.

Post Processing

Aims to solves issue by merging various passes into the least number of passes possible
Uses the term 'effects' instead
Order of adding effects is preserved

Dependencies

@react-three/postprocessing
postprocessing

import { EffectComposer } from '@react-three/postprocessing'

export default function Experience()
{
    return <>

        <EffectComposer>
        </EffectComposer>

        {/* ... */}

    </>
}

EffectComposer is now running, but the colors are now completely off.

Fixing tone mapping

Problems:
- EffectComposer deactivates tone mapping in post-processing
  - Causes the color to be off
- ToneMapping uses AgX tone mapping by default
  - Not the one that R3F uses by default
  - Causes the color to be gray-ish

<EffectComposer>
    <ToneMapping mode={ ToneMappingMode.ACES_FILMIC } />
</EffectComposer>

NOTE

Tone mapping is the process of converting HDR colors to LDR output colors

Multisampling

multisampling is used to prevent the aliasing effect
Default value is 8

// To disable
<EffectComposer multisampling={ 0 }> 
    <ToneMapping mode={ ToneMappingMode.ACES_FILMIC } />
</EffectComposer>

Resources

Vignette effect

Makes the corners of the render a little darker.

import { Vignette, EffectComposer } from "@react-three/postprocessing"

<EffectComposer>
  <ToneMapping mode={ToneMappingMode.ACES_FILMIC} />
  <Vignette offset={0.3} darkness={0.9} />
</EffectComposer>

Blending

a special attribute named blendFunction
- available in every effect
for controlling how a color merges with another color behind it
there are many blend functions

import { BlendFunction, ToneMappingMode } from 'postprocessing'

// To see all available blend functions
console.log(BlendFunction)

<Vignette
    offset={ 0.3 }
    darkness={ 0.9 }
    blendFunction={ BlendFunction.COLOR_BURN }
/>

Background bug

Vignette effect doesn't work on background
This is because background is transparent by default (nothing to render there)
Can fix by adding a color to bg

export default function Experience()
{
    return <>

        <color args={ [ '#ffffff' ] } attach="background" />

        {/* ... */}

		<>
}

Glitch effect

import { Glitch, ToneMapping, Vignette, EffectComposer } from '@react-three/postprocessing'
import { GlitchMode, BlendFunction, ToneMappingMode } from 'postprocessing'


<EffectComposer>
  <Glitch
    delay={[0.5, 1]}
    duration={[0.1, 0.3]}
    strength={[0.2, 0.4]}
    mode={GlitchMode.CONSTANT_MILD}
  />
</EffectComposer>

Noise effect

import { Noise, Glitch, ToneMapping, Vignette, EffectComposer } from '@react-three/postprocessing'

<EffectComposer>
    {/* ... */}
    <Noise
    blendFunction={ BlendFunction.SOFT_LIGHT }
/>
</EffectComposer>

Enhanced with blend function

<Noise
    premultiply
    blendFunction={ BlendFunction.SOFT_LIGHT }
/>

premultiply will multiply the noise with the input color before applying the blending.

Bloom effect

The default Bloom tends to make things glow too easily
Fix by increasing the threshold above which things start to glow using the luminanceThreshold

Dark background setup for nicer effect

import { Bloom, Noise, Glitch, ToneMapping, Vignette, EffectComposer } from '@react-three/postprocessing'

<EffectComposer>
    {/* ... */}
    <Bloom />
</EffectComposer>

How to target objects to glow

Must set a limitation i.e. luminanceThreshold

<meshStandardMaterial color={ [ 1.5, 1, 4 ] } />

Configuring RGB values

Set a luminanceThreshold

With luminenceThreshold increased

Seems like setting the r and b values to be > 1 cause it to start glowing more

Adding mipmapBlur to <Bloom> allows the object to glow more brightly

Emissive Property

Setup (make everything orange)
Add emissive, ezpz

<meshStandardMaterial color="orange" emissive="orange" />

Making it brighter!

<meshStandardMaterial color="orange" emissive="orange" emissiveIntensity={ 2 } />

EVENNN BRIGHTERRRRRR (set color to white)

<meshStandardMaterial color="#ffffff" emissive="orange" emissiveIntensity={ 2 } />

Uniform emissive

What is this even (?)
Use <meshBasicMaterial
Cannot use emissive and emissiveIntensity
Can control the intensity on <Bloom> instead

Setting the luminanceThreshold to 0 causes everything to glow

DepthOfField effect

This effect will blur what’s closer or farther from a set distance.
Main attributes

Attribute	Description
`focusDistance`	At which distance should the image be sharp
`focalLength`	The distance to travel from the `focusDistance` before reaching the maximum blur
`bokehScale`	blur radius

Values are normalized according to near and far

import { DepthOfField, Bloom, Noise, Glitch, ToneMapping, Vignette, EffectComposer } from '@react-three/postprocessing'
;<EffectComposer>
  {/* ... */}
  <DepthOfField focusDistance={0.025} focalLength={0.025} bokehScale={6} />
</EffectComposer>

Custom effects

Need to follow Post Processing rules (w.r.t how they merge effects into one shader)
Resources
- https://github.com/pmndrs/postprocessing/wiki/Custom-Effects
- https://github.com/pmndrs/react-postprocessing/blob/master/api.md#custom-effects

NOTE

Remember that postprocessing will take our shader and merge it with the other effect shaders.

Setup

import { Effect } from 'postprocessing'

export default class DrunkEffect extends Effect
{
    constructor()
    {
        
    }
}

Effect

void mainImage(const in vec4 inputColor, const in vec2 uv, out vec4 outputColor)
{
    
}

Basic vertex shader

const fragmentShader = /* glsl */`
    void mainImage(const in vec4 inputColor, const in vec2 uv, out vec4 outputColor)
    {
        
    }
`

// ...

Fragment shader

es6-string-html for syntax highlighting (requires the /* glsl */)
Uses WebGL 2 syntax
- const > parameter not writeable
- in
  - copy of the actual variable
  - changing it won’t affect the source variable
- out
  - changing this value will change the variable sent when calling the function.

Lets start bois

Basic purple gradient

const fragmentShader = /* glsl */`
    void mainImage(const in vec4 inputColor, const in vec2 uv, out vec4 outputColor)
    {
        outputColor = vec4(uv, 1.0, 1.0);
    }
`

Hit up the Effect

export default class DrunkEffect extends Effect
{
    constructor()
    {
        super(
            'DrunkEffect',
            fragmentShader,
            {
                
            }
        )
    }
}

basic Post Processing effect donez

React-postprocessing implementation

Now, we need to implement it in react-postprocessing.

import DrunkEffect from './DrunkEffect.jsx'

export default function Drunk()
{
    const effect = new DrunkEffect()
    
    return <primitive object={ effect } />
}

...

Parent
<EffectComposer>
    {/* ... */}
    <Drunk />
</EffectComposer>

Boop

Props

Goal: make the effect wiggle with a sinus function with a configurable amplitude / frequency

export default class DrunkEffect extends Effect
{
    constructor(props)
    {
        super(
            'DrunkEffect',
            fragmentShader,
            {
                
            }
        )

        console.log(props)
    }
}

Use props to pass value

Referencing the primitive

import { useRef } from 'react'

export default function Experience()
{
    const drunkRef = useRef()

    // ...
}

<Drunk
    ref={ drunkRef }
    frequency={ 2 }
    amplitude={ 0.1 }
/>

Use ref for easy manipulation
Use forwardRef from React (?) because of the warning above ^

import { forwardRef } from 'react'

export default forwardRef(function Drunk(props, ref)
{
    const effect = new DrunkEffect(props)
    
    return <primitive ref={ ref } object={ effect } />

Getting back the render and make it look greenish

Just need to update the vertexShader

void mainImage(const in vec4 inputColor, const in vec2 uv, out vec4 outputColor)
{
    vec4 color = inputColor;
    color.rgb *= vec3(0.8, 1.0, 0.5);

    outputColor = color;

}

Wiggle wiggle wiggle

Make the image wiggle by tweaking uv coordinates
To alter UV coordinates directly, need to implement a mainUv function

NOTE

uv coordinates are used to map textures to surfaces
mainImage function is used for determining the final color of the pixel
mainUv function is for modifying the uv coordinates

const fragmentShader = /* glsl */`
    void mainUv(inout vec2 uv)
    {
	    uv.y += sin(uv.x * 10.0) * 0.1;
    }

    void mainImage(const in vec4 inputColor, const in vec2 uv, out vec4 outputColor)
    {
        // ...
    }
`

Using attributes

import { Uniform } from 'three'

export default class DrunkEffect extends Effect {
  constructor({ frequency, amplitude }) {
    console.log(frequency, amplitude)
    super(
    'DrunkEffect',
    fragmentShader,
    {
        uniforms: new Map([
            [ 'frequency', new Uniform(frequency) ],
            [ 'amplitude', new Uniform(amplitude) ]
        ])
    }
)

    // ...
  }
}

Need to use Map
Uniform
- standard way to do it
- enables some methods although we won’t use them.

const fragmentShader = /* glsl */`
    uniform float frequency;
    uniform float amplitude;

    void mainUv(inout vec2 uv)
    {
        uv.y += sin(uv.x * frequency) * amplitude;
    }

    // ...
`

Controlling attributes with `Leva`

import { useControls } from 'leva'

export default function Experience()
{
    // ...

    const drunkProps = useControls('Drunk Effect', {
        frequency: { value: 2, min: 1, max: 20 },
        amplitude: { value: 0.1, min: 0, max: 1 }
    })

    // ...
	<Drunk
    ref={ drunkRef }
    { ...drunkProps }
/>
}

Blending the color

First, in the fragmentShader, we are going to send the green color directly in the outputColor and keep the alpha from the inputColor:

const fragmentShader = /* glsl */`

    // ...

    void mainImage(const in vec4 inputColor, const in vec2 uv, out vec4 outputColor)
    {
        outputColor = vec4(0.8, 1.0, 0.5, inputColor.a);
    }
`

Everything is now green, but here comes the trick. (?? Idk why)

Need to specify the blendFunction for the image to be back

<Drunk
    ref={ drunkRef }
    { ...drunkProps }
    blendFunction={ BlendFunction.DARKEN }
/>

super(
    'DrunkEffect',
    fragmentShader,
    {
        blendFunction: blendFunction,
        // ...
    }
)

Animating

Add time

super(
    'DrunkEffect',
    fragmentShader,
    {
        blendFunction: blendFunction,
        uniforms: new Map([
            [ 'frequency', new Uniform(frequency) ],
            [ 'amplitude', new Uniform(amplitude) ],
            [ 'time', new Uniform(0) ]
        ])
    }
)

Update fragmentShader

const fragmentShader = /* glsl */`

    uniform float frequency;
    uniform float amplitude;
    uniform float time;

    void mainUv(inout vec2 uv)
    {
        uv.y += sin(uv.x * frequency + time) * amplitude;
    }

    // ...
`

Make use of update method (called every frame)

export default class DrunkEffect extends Effect
{
    // ...

    update()
    {
        console.log('update')
    }
}

Update Map

export default class DrunkEffect extends Effect
{
    // ...

    update()
    {
        this.uniforms.get('time').value += 0.02 
    }
}

Take into account frame rate

export default class DrunkEffect extends Effect
{
    // ...

    update(renderer, inputBuffer, deltaTime)
    {
        this.uniforms.get('time').value += deltaTime
    }
}

Issues with vanilla post processing​

Post Processing​

Dependencies ​

Fixing tone mapping​

Multisampling​

Resources​

Vignette effect ​

Blending ​

Background bug ​

Glitch effect​

Noise effect ​

Bloom effect​

How to target objects to glow​

Configuring RGB values​

Emissive Property​

Uniform emissive​

DepthOfField effect ​

Custom effects​

Setup​

Lets start bois​

React-postprocessing implementation​

Props​

Referencing the primitive​

Getting back the render and make it look greenish​

Wiggle wiggle wiggle​

Using attributes​

Controlling attributes with Leva​

Blending the color​

Animating​

References​

Issues with vanilla post processing

Post Processing

Dependencies

Fixing tone mapping

Multisampling

Resources

Vignette effect

Blending

Background bug

Glitch effect

Noise effect

Bloom effect

How to target objects to glow

Configuring RGB values

Emissive Property

Uniform emissive

DepthOfField effect

Custom effects

Setup

Lets start bois

React-postprocessing implementation

Props

Referencing the primitive

Getting back the render and make it look greenish

Wiggle wiggle wiggle

Using attributes

Controlling attributes with `Leva`

Blending the color

Animating

References