Performance Red herring and profiling with puffin

[Dapeos]

First thank you for all your work on this amazing project.

A point in any direction would be helpful.

I have an application that processes tens of millions of data points using async tasks and then caches the results on the heap.

Before filling the heap, my frontend renders at 500+ fps, after my async tasks complete and the results are stored on the heap my fps drop. If I process more tasks I can degrade the front end performance to single digits.

I have narrowed it down to calls to the painter to draw shapes, allocate rects, and the use of labels.
If I disable the panel that draws my main grid I am back at 500+ fps. If I remove the cached items from the heap the performance returns as well.

My app is using around 2gb of ram after processing 100 million records, I would like to know if there is anything I can do to not have
bad UI performance after creating a large heap.

I am guessing its choking on allocation etc. I am trying to make sense of dhat and the flamegraph looking for a silver bullet but no luck yet.

Thanks

[lucasmerlin]

I doubt that having much data on the heap would be a problem for egui. Are you displaying all those millions of records at the same time?

[Dapeos]

Thanks for responding Lucus,
I am only displaying like 30 records. Note that the grid rendering function is getting a separate data structure which just contains exactly what it is supposed to draw and doesn't touch the DataSourceManager shown below.

The cache that as it grows. slows the ui looks like this:
struct DataSourceManager{
transmutes: HashMap<ChunkColId, Arc<Mutex<HashMap<StatementId, (u128, Vec, Vec)>>>>,
...
}

First screenshot: transmutes hashmap is empty and the fps is 400+.

Second screenshot shows transmutes with 75million entries and fps 54.

If I turn off the grid fps goes back up. If I drain the transmutes Hashmap fps goes back up.

Thanks,

[YgorSouza]

You could use puffin to try and pinpoint where the time is being spent. There is an example in the repo. It might just be that you have a call that is O(n) on the size of the hashmap somewhere, that is being run on every frame.

[YgorSouza]

I made the following change to the demo app as a test:

Details

M crates/egui_demo_app/src/wrap_app.rs
@@ -5,6 +5,7 @@ use eframe::glow;
 
 #[cfg(target_arch = "wasm32")]
 use core::any::Any;
+use std::{collections::HashMap, thread::JoinHandle};
 
 #[derive(Default)]
 #[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
@@ -168,6 +169,7 @@ pub struct WrapApp {
     custom3d: Option<crate::apps::Custom3d>,
 
     dropped_files: Vec<egui::DroppedFile>,
+    handle: JoinHandle<()>,
 }
 
 impl WrapApp {
@@ -175,6 +177,15 @@ impl WrapApp {
         // This gives us image support:
         egui_extras::install_image_loaders(&cc.egui_ctx);
 
+        let handle = std::thread::spawn(|| {
+            let _cache: HashMap<_, _> = (0..100_000_000)
+                .map(|i| (i, std::sync::Arc::new(HashMap::from([(i, i)]))))
+                .collect();
+            loop {
+                std::thread::park();
+            }
+        });
+
         #[allow(unused_mut)]
         let mut slf = Self {
             state: State::default(),
@@ -183,6 +194,7 @@ impl WrapApp {
             custom3d: crate::apps::Custom3d::new(cc),
 
             dropped_files: Default::default(),
+            handle,
         };
 
         #[cfg(feature = "persistence")]

This spawns a thread that makes millions of allocations and then holds on to them forever. The RAM usage goes up to 16GB, but the time to draw one frame is the same as the unmodified app (around 3ms on my machine). So I don't think this is a heap allocation issue. You can try to reproduce this result, or change the code to make it more similar to your own to try and find out where the performance breaks down.

[Dapeos]

YgorSouza,
Thank you so much for helping me and letting me know about puffin. Also thank you for the example.

I profiled all of my functions and I can see that the Counts and Times are not dependent on if the cache has entries or not.

At this point I have no idea where the problem lies. I am using Bevy for its ECS engine and I thought that maybe due to my cache being a Bevy Resource and its change detection logic was where the issue was but I changed my cache to a global singleton with no improvement.
I will edit the title, and close this thread.

I am sorry for wasting your guy's time.

[Dapeos]

Hi YgorSouza,
You pointing me to puffin was so much help that I am reaching out again because I am at a loss as to what to pursue next.
Please forgive me for my ignorance and if I am chasing a red herring.

So I have been using puffin to to see exactly what is taking a long time after I grow my Hashtable. I have traced the issue down to tessellator::{stroke_path, fill_closed_path, tessellate_text} functions. I am using egui with bevy_egui, and bevy if that makes a difference.

Shown below are 4 profile viewers. The only difference is that my Hashmap is larger with each one. Bottom image is base case with nothing in the hashmap. As you can see the size is roughly the same and the number of calls is roughly the same.

[YgorSouza]

I think "size" is just how much data puffin collected about each function, so it will mostly be proportional to the count. But comparing the total time and the max time, it seems that there are only a couple of very slow calls. It could be that the allocator is struggling from time to time, or it could be that these calls are doing more work when you have more data. You could try to track these slow calls to take a closer look at their call stack and the data they're working with. Maybe patch the epaint crate with a local clone (as the eframe_template shows), and modify it to panic if the time to run these functions is above a threshold, so you can make it print the trace and the function parameters. You might be able to do that with a debugger as well, whichever is easier.

[Dapeos]

Thanks YgorSouza,
I fixed the issue by changing my allocator.
Not sure if there are any downsides to this but now after putting a few hundred million entries on my heap I still have 500+ fps. I would like to understand why and if I am doing something in particular that is causing the default allocator to suck wind but I am content with moving forward.

[dependencies]
mimalloc = "0.1"

#[global_allocator]
static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;

More details:
I took your advice and traced the slow calls. I found that it was caused anytime items were added to vecs or space was reserved. Which explains why the tessellate functions where struggling the most because of all the reserve_space and push calls.
I created my own tests that had nothing to do with the tessellator where I just pushed a bunch of items after a call to reserve onto a global vec and found that it was expensive after putting a hundred million things into my hashmap.

Thank you so much for taking the time to help me, your a real lifesaver!

[YgorSouza]

Nice. I don't know the pros and cons of this allocator either, as I don't know much about allocators in general, but this video by fasterthanlime might be useful, if you want to take a look. It might be possible to redesign the data structures in your cache or the code that fills it such that it does fewer memory allocations for the same data. But if you found another allocator that already performs better with the code as it is, I guess that should be fine too.