Constantly increasing memory usage

[BrunoWallner]

Bevy version

bevy commit a291b5aaedb4affcb31df2e2e63cb0c665ffb24a

Operating system & version

Latest Arch Linux.

What I did

I created a voxel based 3D game, which stores chunks of block-data and the chunk entitiies in 2 seperate HashMaps.

First I spawn a PbrBundle with the mesh of a Cube and attach its ID to the hashmap.
Then I generate the Terrain and Mesh asynchronously and apply the mesh by querying over &Handle<Mesh>

To unload the chunks I have a system which periodically checks trough one of the hashmap, which holds the entities and if its key is outside of the range of the player, the entity gets despawned and the corresponding data gets removed from both hashmaps,

Expected behaviour

I have a system to monitor every single resource and entity count and according to that, the total amount of entities and entries in the hashmaps are not getting any more.

Because of that there should not be a memory increase

What actually happened

The memory usage of the game gets constantly bigger as I load and unload new chunks,.

I even have a system which deletes everything with a keypress, but this does only free up a bit of memory.

Additional information

The repo can be found here

I copied a lot of ideas from vx_bevy which does not have this memory problem.

hashmaps

use bevy::utils::HashMap;

// to access entites
pub struct ChunkMap(HashMap<[i64; 3], Entity>);

// to access the chunkdata
pub struct VoxelMap{
    map: HashMap<[i64; 3], ChunkData>
}

pub struct ChunkData {
    pub data: [[[Block; CHUNK_SIZE]; CHUNK_SIZE]; CHUNK_SIZE],
}

This does monitor every single Resource and entity in my Game

Despite it showing no memory increase, the memory does increase

monitoring

fn entity_count(
    query: Query<Entity>,
    mesh: Res<Assets<Mesh>>,
    loaded_chunks: Query<&LoadedChunk>,

    chunkmap: Res<ChunkMap>,
    voxelmap: Res<VoxelMap>,

    gen_queue: Res<ChunkGenQueue>,
    mesh_queue: Res<ChunkMeshQueue>,
    unload_queue: Res<ChunkUnloadQueue>,
) {
    let entities = query.iter().count();
    let meshes = mesh.iter().count();

    let chunks = chunkmap.0.len();
    let voxels = voxelmap.map.len();

    let mesh = mesh_queue.queue.len();
    let gen = gen_queue.queue.len();

    let unload = unload_queue.queue.len();
    let loaded_c = loaded_chunks.iter().count();

    println!("entities: {}\tmeshes: {}", entities, meshes);
    println!("chunkmap: {}\tvoxels: {}", chunks, voxels);
    println!("mesh_q:   {}\tgen_q:  {}", mesh, gen);
    println!("unload:   {}\tload_c: {}", unload, loaded_c);
    println!("");
}

fn unloader(
    mut cmds: Commands,
    loaded_chunks: Query<(Entity, &LoadedChunk)>,

    mut chunkmap: ResMut<ChunkMap>,
    mut voxelmap: ResMut<VoxelMap>,
    mut gen_queue: ResMut<ChunkGenQueue>,
    mut mesh_queue: ResMut<ChunkMeshQueue>,
    mut unload_queue: ResMut<ChunkUnloadQueue>,
    keys: Res<Input<KeyCode>>,
) {
    if keys.just_pressed(KeyCode::R) {
        for (entity, _) in loaded_chunks.iter() {
            cmds.entity(entity).despawn()
        }
        chunkmap.0.clear();
        voxelmap.map.clear();

        gen_queue.queue.clear();
        mesh_queue.queue.clear();
        unload_queue.queue.clear();
    }
}

I am really out of ideas what to do, because everything should to work.

Did I forget to measure something or is bevy the problem here?

[alice-i-cecile]

Converting to a discussion; this is too project-specific for us to resolve directly as a bug <3

[superdump]

Note that .clear() on rust collections only drops the contained items, it doesn’t free any of the allocated capacity.

I tried to look through the code but with the web interface for the repo I couldn’t find where the entities with Handle are spawned. All I could find was that the handle was being used to mutably get the mesh asset and then overwrite it. I haven’t seen that pattern before and I’m not sure how it will behave. My guess would be that perhaps the mesh data is being leaked somewhere, but it’s a hunch. I am not at all certain.

[BrunoWallner]

I now do a full reset of the hashmaps like this:

fn unloader(
    mut cmds: Commands,
    loaded_chunks: Query<(Entity, &LoadedChunk)>,

    mut chunkmap: ResMut<ChunkMap>,
    mut voxelmap: ResMut<VoxelMap>,
    mut gen_queue: ResMut<ChunkGenQueue>,
    mut mesh_queue: ResMut<ChunkMeshQueue>,
    mut unload_queue: ResMut<ChunkUnloadQueue>,
    keys: Res<Input<KeyCode>>,
) {
    if keys.just_pressed(KeyCode::R) {
        for (entity, _) in loaded_chunks.iter() {
            cmds.entity(entity).despawn()
        }
        // HashMaps
        chunkmap.0.clear();
        chunkmap.0.shrink_to_fit();
        voxelmap.map.clear();
        voxelmap.map.shrink_to_fit();

        gen_queue.queue.clear();
        mesh_queue.queue.clear();
        unload_queue.queue.clear();
    }
}

And changed the meshing system from mutably getting the mesh asset and overwriting it to just spawning the whole PbrBundle like so:

pub fn apply_mesh_tasks(
    mut meshes: ResMut<Assets<Mesh>>,
    chunkmap: Res<ChunkMap>,
    mut query: Query<(&LoadedChunk, &mut MeshTask)>,
    mut cmds: Commands,
    settings: Res<Settings>,
    materials: Res<crate::material::Materials>,
) {
    let mut finished: usize = 0;
    for (_, mut mesh_task) in query.iter_mut() {
        if finished < settings.chunk_meshes_per_cycle as usize {
            if let Some( (key, mesh) ) = future::block_on(future::poll_once(&mut mesh_task.task)) {
                if let Some(entity) = chunkmap.entity(key) {
                    cmds.entity(entity).insert_bundle(PbrBundle {
                            mesh: meshes.add(mesh), // must have exact size
                            visibility: Visibility {is_visible: true},
                            transform: Transform::from_matrix(Mat4::from_scale_rotation_translation(
                                Vec3::splat(1.0),
                                Quat::from_rotation_x(0.0),
                                Vec3::new((key[0] * CHUNK_SIZE as i64) as f32, (key[1] * CHUNK_SIZE as i64) as f32, (key[2] * CHUNK_SIZE as i64) as f32),
                            )),
                            material: materials.chunk.clone(),
                            ..default()
                        });

                    cmds.entity(entity).remove::<MeshTask>();

                    finished += 1;
                }
            }
        }
    };
}

But before that, the entity gets now spawned and inserted into the chunkmap hashmap in src/chunk/plugin.rs using the chunk_spawner system like this:

// spawns chunks around player when moved
fn chunk_spawner(
    mut query: Query<(&Transform, &mut Player)>,
    mut chunkmap: ResMut<ChunkMap>,
    mut queue: ResMut<ChunkGenQueue>,
    mut cmds: Commands,
) {
    let mut player_pos = Vec3::ZERO;
    let mut player_moved: bool = false;
    for (transform, mut player) in query.iter_mut().next() {
        player_pos = transform.translation;
        if player.chunks_moved_since_load > 0 {
            player_moved = true;
            player.chunks_moved_since_load = 0;
        }
    }

    let chunk_x = (player_pos.x / CHUNK_SIZE as f32).round() as i64;
    let chunk_y = (player_pos.y / CHUNK_SIZE as f32).round() as i64;
    let chunk_z = (player_pos.z / CHUNK_SIZE as f32).round() as i64;

    // loading
    if player_moved {
        for x in -RENDER_DISTANCE_HOR..RENDER_DISTANCE_HOR {
            for z in -RENDER_DISTANCE_HOR..RENDER_DISTANCE_HOR {
                for y in -RENDER_DISTANCE_VER..RENDER_DISTANCE_VER {
                    // circular
                    if x.pow(2) + z.pow(2) >= RENDER_DISTANCE_HOR.pow(2) {
                        continue;
                    }

                    let chunk_x = chunk_x as i64 + x;
                    let chunk_y = chunk_y as i64 + y;
                    let chunk_z = chunk_z as i64 + z;
                    let chunk_coords = [chunk_x, chunk_y, chunk_z];

                    // check if chunk is not in world
                    if !chunkmap.contains_entity(&chunk_coords) {
                        chunkmap.0.insert(chunk_coords, cmds.spawn().insert(LoadedChunk{key: chunk_coords}).id());
                        queue.queue.push(chunk_coords);
                    }
                }
            }
        }
        queue.sorted = false;
        queue.last_player_chunk_pos = [chunk_x, chunk_y, chunk_z];
    }
}

And then the mesh tasks get prepared like this:

pub fn prepare_mesh_tasks(
    mut queue: ResMut<ChunkMeshQueue>,
    chunkmap: Res<ChunkMap>,
    voxels: Res<VoxelMap>,
    pool: Res<AsyncComputeTaskPool>,
    mut cmds: Commands,
    settings: Res<Settings>,
) {
    if !queue.sorted {
        let pos = queue.last_player_chunk_pos;
        queue.queue.sort_unstable_by_key(|key| {
            (key[0] - pos[0]).abs() +
            (key[1] - pos[1]).abs() +
            (key[2] - pos[2]).abs()
        });
        queue.queue.reverse();
        queue.sorted = true;
    }

    for _ in 0..settings.chunk_meshes_per_cycle {
        if let Some(key) = queue.queue.pop() {
            if let Some(entity) = chunkmap.entity(key) {
                if let Some(data) = voxels.map.get(&key) {
                    let data = *data;
                    let task = pool.spawn(async move {
                        let mesh = super::mesh::generate(data);
                
                        (key, mesh)
                    });
        
                    cmds.entity(entity).insert(MeshTask{task});
                }
            }
        }
    }
}

But the memory does still constantly increase.

And pressing R and thus unload everything using the unloader system, only frees up about 100 megabytes.

[BrunoWallner]

After changing the way I mesh the chunks and unload everything, I now removed the meshing altogether, by not spawning the PbrBundle in apply_mesh_tasks:

pub fn apply_mesh_tasks(
    mut meshes: ResMut<Assets<Mesh>>,
    chunkmap: Res<ChunkMap>,
    mut query: Query<(&LoadedChunk, &mut MeshTask)>,
    mut cmds: Commands,
    settings: Res<Settings>,
    materials: Res<crate::material::Materials>,
) {
    let mut finished: usize = 0;
    for (_, mut mesh_task) in query.iter_mut() {
        if finished < settings.chunk_meshes_per_cycle as usize {
            if let Some( (key, mesh) ) = future::block_on(future::poll_once(&mut mesh_task.task)) {
                if let Some(entity) = chunkmap.entity(key) {
                    /*
                    cmds.entity(entity).insert_bundle(PbrBundle {
                            mesh: meshes.add(mesh), // must have exact size
                            visibility: Visibility {is_visible: true},
                            transform: Transform::from_matrix(Mat4::from_scale_rotation_translation(
                                Vec3::splat(1.0),
                                Quat::from_rotation_x(0.0),
                                Vec3::new((key[0] * CHUNK_SIZE as i64) as f32, (key[1] * CHUNK_SIZE as i64) as f32, (key[2] * CHUNK_SIZE as i64) as f32),
                            )),
                            material: materials.chunk.clone(),
                            ..default()
                        });
                        */

                    cmds.entity(entity).remove::<MeshTask>();

                    finished += 1;
                }
            }
        }
    };
}

The memory does still increase constantly, but not as fast.

Because of that, I think it is the way I despawn the entities, that is the memory problem.

I check if an entity should be despawned like this:

fn chunk_unload_detector(
    voxelmap: Res<VoxelMap>,
    chunkmap: Res<ChunkMap>,
    mut query: Query<(&Transform, &mut Player)>,
    mut unload_queue: ResMut<ChunkUnloadQueue>,
) {
    let mut player_pos = Vec3::ZERO;
    let mut player_moved: bool = false;
    for (transform, mut player) in query.iter_mut().next() {
        player_pos = transform.translation;
        if player.chunks_moved_since_unload > 0 {
            player_moved = true;
            player.chunks_moved_since_unload = 0;
        }
    }

    let chunk_x = (player_pos.x / CHUNK_SIZE as f32).round() as i64;
    let chunk_y = (player_pos.y / CHUNK_SIZE as f32).round() as i64;
    let chunk_z = (player_pos.z / CHUNK_SIZE as f32).round() as i64;

    if player_moved {
        let coords = [chunk_x, chunk_y, chunk_z];

        voxelmap.map.keys().for_each(|key| {
            if (key[0] - coords[0]).abs() > RENDER_DISTANCE_HOR + 1
            || (key[1] - coords[1]).abs() > RENDER_DISTANCE_VER + 1
            || (key[2] - coords[2]).abs() > RENDER_DISTANCE_HOR + 1 {
                unload_queue.queue.push(*key);
            }
        });

        chunkmap.0.keys().for_each(|key| {
            if (key[0] - coords[0]).abs() > RENDER_DISTANCE_HOR + 1
            || (key[1] - coords[1]).abs() > RENDER_DISTANCE_VER + 1
            || (key[2] - coords[2]).abs() > RENDER_DISTANCE_HOR + 1 {
                unload_queue.queue.push(*key);
            } 
        });
    }
}

Then I go trough the ChunkUnloadQueue and actually despawn the entities like this:

fn chunk_unloader(
    mut unload_queue: ResMut<ChunkUnloadQueue>,
    mut chunkmap: ResMut<ChunkMap>,
    mut voxelmap: ResMut<VoxelMap>,
    mut cmds: Commands,
) {
    for key in unload_queue.queue.drain(..) {
        if let Some(entity) = chunkmap.detach_entity(key) {
            cmds.entity(entity).despawn();
        }
        voxelmap.map.remove(&key);
    }
}

The entity_count system, which reports the amount of entities and resources my game utilizes, just confirms, that everything should work.
The function:

fn entity_count(
    query: Query<Entity>,
    mesh: Res<Assets<Mesh>>,
    loaded_chunks: Query<&LoadedChunk>,

    chunkmap: Res<ChunkMap>,
    voxelmap: Res<VoxelMap>,

    gen_queue: Res<ChunkGenQueue>,
    mesh_queue: Res<ChunkMeshQueue>,
    unload_queue: Res<ChunkUnloadQueue>,
) {
    let entities = query.iter().count();
    let meshes = mesh.iter().count();

    let chunks = chunkmap.0.len();
    let voxels = voxelmap.map.len();

    let mesh = mesh_queue.queue.len();
    let gen = gen_queue.queue.len();

    let unload = unload_queue.queue.len();
    let loaded_c = loaded_chunks.iter().count();

    println!("entities: {}\tmeshes: {}", entities, meshes);
    println!("chunkmap: {}\tvoxels: {}", chunks, voxels);
    println!("mesh_q:   {}\tgen_q:  {}", mesh, gen);
    println!("unload:   {}\tload_c: {}", unload, loaded_c);
    println!("");
}

I really do not know what else to do in this situation.