Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data not received from kernel error on large Decapodes simulations #350

Open
quffaro opened this issue Jan 28, 2025 · 4 comments · May be fixed by #353
Open

Data not received from kernel error on large Decapodes simulations #350

quffaro opened this issue Jan 28, 2025 · 4 comments · May be fixed by #353
Assignees
Labels
backend Backend, including web server and database bug Something isn't working external Work on interfacing with other tools

Comments

@quffaro
Copy link
Collaborator

quffaro commented Jan 28, 2025

Issue
The data not received from kernel error appears when running Navier-Stokes inviscid flow simulation.

Testing
"Works" here means that the simulation successfully renders on the frontend. On Chromium, Ico6@t=10 works, but ico7@t=10 does not, but Ico7@t=1 does. On Firefox, Ico7@t=10 and Ico8@t=1 work.

Root Cause
I'm led to believe by this source that this is a constraint imposed by browsers. This explains why Firefox was able to render the outputs of larger simulations.

Proposed Solution
My initial guess is that there needs to be a batching mechanism in place, probably on the SimResult struct.

@quffaro quffaro added backend Backend, including web server and database external Work on interfacing with other tools labels Jan 28, 2025
@epatters epatters added the bug Something isn't working label Jan 28, 2025
@epatters
Copy link
Member

Thanks Matt for a clear report about the problem.

We could send the data in batches, rather than all at once, but that's gonna add a lot of extra complexity to an already nontrivial communication between frontend and kernel.

I'd suggest we start with something easier, namely capping the total number of real numbers that will be returned to the frontend. We can do this by downsampling in space and/or time. This isn't crazy because the visualization has limited discernible spatial resolution, and we already know that the temporal resolution is too high because we're currently only showing every fifth time point! (A bad hack that I feel bad about, added by me during the manic demo prep.)

For example, we could set a maximum length for each of the x, y, and time dimensions, and then if the number of points returned by the simulator exceeds the number, we downsample uniformly in that direction before returning the data from the Julia process.

@quffaro quffaro self-assigned this Jan 28, 2025
@quffaro quffaro linked a pull request Jan 29, 2025 that will close this issue
@quffaro
Copy link
Collaborator Author

quffaro commented Jan 31, 2025

This would be a significant departure from the "generate simulation data and have CatColab visualize it," but would we bypass this problem if we generated the visualization .gif/.mp4 on the Julia-side?

@epatters
Copy link
Member

epatters commented Jan 31, 2025

Not crazy at all. Browsers have good support for playing videos now, right?

@quffaro
Copy link
Collaborator Author

quffaro commented Jan 31, 2025

So the rumor goes! But I suppose this means we would have to make larger changes affecting what data we return and how we parse it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Backend, including web server and database bug Something isn't working external Work on interfacing with other tools
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

2 participants