-
-
Notifications
You must be signed in to change notification settings - Fork 123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Requesting video info is broken #130
Comments
Never mind. I thought it worked, I was wrong. Also I just noticed that pytube also doesn't work for me. Is this a problem of mine or is it broken for everyone? |
Okay it actually appears to be broken for one particular video? The playability status of that video is 'This video is not available', but it is available in youtube. If I go to the video through the browser it works. |
@iexavl it may require age verification or smth like that, try it in browser without login (incognito tab) |
Not age restricted. It's also not only that video, but it's also not every video. It's kind of weird. Originally it was this video: |
yes the downloader no longer works ): videoInfo is allways null. |
It can work by deleting the part that uses innertube, in other words commenting out parseVideoAndroid, but that makes things quite slow since the downloader has to parse the html. |
https://github.com/iv-org/invidious/blob/a021b93063f3956fc9bb3cce0fb56ea252422738/src/invidious/yt_backend/youtube_api.cr#L483 |
@iexavl have a look at https://github.com/ytdl-org/youtube-dl there might be a recent fix for this that we can implement here |
@sealedtx can u fix that now?hope soon. |
I looked it over and tried what I could find. Unless I missed something there hasn't been a fix yet. This seems to be something specific to the android client, probably some additional argument. I tried the IOS client and it works:
Unfortunately it seems to be quite constrained for qualities. On 1080p 60fps video I got 720p 30fps max. |
but with this i dont have sound |
bestVideoWithAudioFormat returns zero with the IOS client. I think we need a better solution here. |
String body = |
It doesn't work for everything. It works for some videos, for others not. And the IOS is indeed a bad option. I have no idea what they did though and I am running out of ideas as to how to find it. Perhaps we should wait for the youtube-dl guys to notice or just post an issue there. |
and yes you're right unfortunately it doesn't work with every video ): |
this works very good and fast and with all videos too that doesnt worked with ANDROID_MUSIC |
ANDROID_TV can use up to 1080p 60FPS and works too |
@punkydie can you play a bit with these params and find best one, which works with most of videos, it should cover most of Itags Then I'll update code or you can submit a pull request if you'd like to be a contributor. Thank you |
It might not be bad if the downloader checks all clients in a loop from good to bad until the video works. For example, ANDROID_VR also works, which supports up to 2160p and 60FPS. But not for every video. But it would be a shame to do without it. |
https://github.com/zerodytrash/YouTube-Internal-Clients?tab=readme-ov-file#clients |
Nevermind. Just use the web client.
It works and it's 4320p60 HDR, same as android. |
nope doesnt work on me ANDROID_TV works best with all Videos that i have tested and very fast |
Seriously? I guess the android clients are the most sensible. Although I suppose we can take a look at the other clients. Can you check out the other clients in the list and see if they work for you? |
Yes, just test it. Works perfectly. Haven't found a video yet that doesn't work. |
can you try MWEB (2.20220918) , TVHTML5 (7.20220918) , TVHTML5_CAST (1.1), WEB_EMBEDDED_PLAYER (9.20220918), WEB_CREATOR (1.20220918) for now? |
yes i can |
The WEB Clients doesnt work. and TVHTML Clients not on all Videos. |
this is some really odd stuff. I guess we will be walking on eggshells with these clients until someone finds out what's going on. This also means that my assumption that the problem was only in android clients was wrong. |
Yes, I build into my mod (Minecraft) that I can change the clients if necessary without always having to do that in the code. |
Maybe it would be better to have multiple clients and have the requests be retried in case one client fails to return the streamingData @sealedtx |
Well unfortunately (and I just tested it) that means yt-dlp is just as broken as our interpretation of it. This client still can't do it all though so we will have to fix the web clients. |
Since Youtube is apparently doing big changes to its WebRequest handling, maybe we will have to start looking for an alternative approach to download media? |
I didn't get your idea. What is the benefit of this? Which request to send? To merge media we'll need extra dependency like ffmpeg |
@iexavl You're right. It's the same for me. I noticed I didn't even need to set a client after I updated JYD code to the latest. The only little issue is that it may not get the highest quality video. Seems Youtube may be doing that intentionally. After monitoring network requests, I noticed that yt-dlp starts with the IOS client, then MWEB client and then it makes a third request to a hls endpoint. It seems the third is the only sure way to get the highest possible quality. Unfortunately, the last method will require an external dependency like ffmpeg to recombine the split video parts from the m3u8 file. As far as I'm concerned the current update works well for me, so great job so far! |
What request exactly I do not know. Mabye replicating the requests issued by the browser youtube when opening a video. I think you can set the media offset in the URL. And yes that would require ffmpeg. Like yt-dlp is doing to receive the highest quality medias as described by @notarom . But if zero dependency is mandatory, I see that this idea can not work. |
The problem isn't the processing of the video. The problem is receiving a response from the servers at all. At the moment with pretty much all the web clients when you send a request for video info the server responds, but when you try to send a request to a link the server responded with, that returns a 403 response. |
So the great era is over now, is it? |
well it's pretty mysterious. I did some testing and got curious as to why some clients work and others don't for the same video and same format and I narrowed it down to these parameters in the request : |
sounds exciting. I'm curious to see if it will really work again at some point. Thanks for your work! |
So at the moment it's working halfway again. What I have noticed is that with many clients the video is not displayed even though all data such as author, title, FPS etc. are found. What could be the reason for this? |
honestly I haven't touched this since the last time we commented. I've been on a bit of a burnout strike. The issue is the same that was experienced with the web clients, I am guessing they are also updating other clients. |
Hmm yes it's just strange that all the data arrives but the video doesn't play. At first I thought it might be due to some itags, but playing around with them didn't help either. |
And have a good time Health comes first! |
well it's the exact issue we are currently trying to solve. Kind of unfortunate that other clients are also getting it so soon. |
No, the situation has improved again. Many clients are now working again. You just have to use different clients for different videos - that's not a problem, I've automated it for myself. But there is no way for me to query if the video is not played then please search for the next client. Because video info is then no longer null if it has the rest. |
I suppose the only practical solution for now would probably be just to check the response. If it returns a code 403 that means access denied, so the client won't respond to you even if you did get your hands on the video info, which means you should try a different client. |
really annoying with youtube. once again it feels like no clients are working today. |
|
Same for me. Tried it but (at least standard client) not working |
look at #144 (comment) |
yes, and what's the point? doesn't it still work? |
Doesn't what still work? If you are referring to the clients for the most part yes, but it seems like a lot of the requests get flagged as being sent by bots. I managed to get an ok by using the po_token and visitor_data |
Yes, that's true, but there are a few videos that still don't work. Still, better than nothing for now. Thank you |
{playableInEmbed=true, contextParams=Q0FJU0FnZ0I=, miniplayer={miniplayerRenderer={playbackMode=PLAYBACK_MODE_PAUSED_ONLY, minimizedEndpoint={openPopupAction={popupType=TOAST, popup={notificationActionRenderer={trackingParams=CAsQuWoiEwiP1_-n8qSLAxUo40IFHdGpABA=, responseText={simpleText=Miniplayer is off for videos made for kids. Tap play to resume}, actionButton={buttonRenderer={trackingParams=CAwQ8FsiEwiP1_-n8qSLAxUo40IFHdGpABA=, text={simpleText=Learn more}, command={commandMetadata={webCommandMetadata={rootVe=83769, webPageType=WEB_PAGE_TYPE_UNKNOWN, url=//support.google.com/youtube/bin/answer.py?answer=9632097&nohelpkit=1&hl=en}}, clickTrackingParams=CAwQ8FsiEwiP1_-n8qSLAxUo40IFHdGpABA=, urlEndpoint={url=//support.google.com/youtube/bin/answer.py?answer=9632097&nohelpkit=1&hl=en, target=TARGET_NEW_WINDOW}}}}}}}, clickTrackingParams=CAAQu2kiEwiP1_-n8qSLAxUo40IFHdGpABA=}}}, status=OK} Status is OK but no Video this is happen on some videos even with po_token I get all Infos like Autor and Title but no Video or Sound this is the video: https://www.youtube.com/watch?v=89xEbfo8o4U |
Are you talking about getting a response with the format data? Because I managed to get that with a bunch of clients like WEB, MWEB, WEB_HEROES ... or are you talking about an HTTP 403 when you try to open the stream to the format or? |
|
yes the title and author is displayed but no video what error message comes up I don't know. But video info is not null but still no video |
yeah, that video id returns a 403. Not quite sure why that happens for some videos to be honest. |
Maybe we'll find out at some point :) |
The body in parseVideoAndroid function is broken. It gets a faulty response from youtube with no streamData.
I checked pytube and replaced it with this:
With that it works.
The text was updated successfully, but these errors were encountered: