Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

this isn't an actual issue, I'm just wondering about something. #7

Open
Dijitul1029 opened this issue Feb 2, 2024 · 3 comments
Open

Comments

@Dijitul1029
Copy link

basically I want to create a custom anime interpolation program that's based on gmfss fortuna (but custom trained on various anime scenes so the visuals look correct and retain their original forms) and also handles scene changes correctly. I use enhancr for gmfss fortuna but it doesn't let me interpolate to a specified framerate, and topaz video ai's chronos model. I'm hoping you could make this program with me and I could tell you know I want it built. you can close this issue if it's out of line or whatevever, I'm just not sure where to ask this question.

@98mxr
Copy link
Owner

98mxr commented Sep 4, 2024

Forgive me for not having the energy to see this issus, I'm not sure to what extent “a specified framerate” is a problem, but GMFSS is not capable of implementing any framerate, and we need to revisit this point.

@98mxr 98mxr reopened this Sep 4, 2024
@Dijitul1029
Copy link
Author

Basically back in march I had found this program called svfi on steam that has the options of gmfss or other interpolation models. This program does what enhancr doesn't provide, an option for fps ratio. In my case I can set it to interpolate to 120fps for ultra smooth scenes with low or no jitter. Svfi also has scene detection with options for what to do at scene changes. I'd personally like to work with a coder to train a gmfss model for anime like I said and also code in multiple options for dealing with scene changes (which I think python may be able to do but idk since I don't code), like an option to blend between frames in the way that topaz does with its models. I don't know what happens in the background code that runs when I set the option there to use an fps ratio instead of simply x2, x3, x4, x5 (119fps if source file is 23.976fps), etc.

@98mxr
Copy link
Owner

98mxr commented Sep 4, 2024

SVFI, smh, finally got it!
Actually a GMFSS-like VFI model can only do x2, x3, x4, x5 kind of frame interpolation. According to SVFI, if you do 24fps-60fps interpolation, the possible way to achieve that is to x3 to 72fps and discard some frames to get 60fps.
SVFI should have a feature that when interpolating 24fps-60fps, it would x5 to 120fps and discard some frames to get 60fps. we think that only by doing “integer” frame rate changes, we can avoid some problems like lagging and so on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants