Results 1 to 8 of 8

Thread: Help for downloading Youtube video "server replied: Forbidden"

  1. #1
    Join Date
    Apr 2011
    Posts
    61
    Thanks
    1
    Thanked 2 Times in 2 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Help for downloading Youtube video "server replied: Forbidden"

    I'm trying to download a video from youtube.

    I'm using something like in this python code:
    https://github.com/rg3/youtube-dl/bl...ter/youtube-dl

    But rewrote it for my code.
    Everything works fine, to get the real download link.

    I put this link: http://www.youtube.com/watch?v=AGAIq...eature=related

    And get this: http://v11.lscache8.c.youtube.com/vi...6008ab9fec6f8a

    At here, everything ok, now I'm trying to download the video using this link.
    But I got an error in the reply, this error:

    "Error downloading http://v11.lscache8.c.youtube.com/vi...6008ab9fec6f8a - server replied: Forbidden"

    This is my code:

    Qt Code:
    1. void Downloader::start()
    2. {
    3. YtInfo* info = qobject_cast<YtInfo*>((QObject*)m_item->data(Qt::UserRole + 1).toInt());
    4.  
    5. QNetworkRequest request = QNetworkRequest(QUrl(info->getHigherQualityDownload()));
    6. //request.setRawHeader("User-Agent", "Opera/9.80 (Windows NT 5.1; U; pt-BR) Presto/2.8.131 Version/11.10");
    7. request.setRawHeader("Youtubedl-no-compression", "True");
    8.  
    9. //m_reply = m_manager->get(request);
    10. m_reply = m_manager->head(request);
    11.  
    12. connect(m_reply, SIGNAL(downloadProgress(qint64,qint64)), SLOT(downloadProgress(qint64,qint64)));
    13. connect(m_reply, SIGNAL(finished()), SLOT(finished()));
    14. }
    15.  
    16. void Downloader::finished()
    17. {
    18. qDebug() << m_reply->errorString();
    19. m_reply->deleteLater();
    20. }
    To copy to clipboard, switch view to plain text mode 

    info->getHigherQualityDownload()
    Returns the best quality video avaliable for download.
    The link that I've posted.

    I've tried changing the User-Agent header, but doesn't work too.

    The video link works in the browsers but not inside my Qt Application.

    I've tested it in the Qt Browser Demo and the link doesn't work too.
    Just got an "non-exiting" page.

    But works in Opera, Mozilla and Chrome. (I've not tested it on IE).

    The download works even if I'm logged in a youtube account or no.

    Someone can help me ?

  2. #2
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,363
    Thanks
    3
    Thanked 5,012 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Help for downloading Youtube video "server replied: Forbidden"

    So what do you expect us to do? Take a network sniffer and compare the traffic from your app with the one from a full network browser.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  3. #3
    Join Date
    Apr 2011
    Posts
    61
    Thanks
    1
    Thanked 2 Times in 2 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Help for downloading Youtube video "server replied: Forbidden"

    I'll try it using wireshark.
    I've asked because I've thought that someone may have this problem too.
    Or someone knows how to solve it, or may be someone knows if this needs header-tricks or something else.
    Or a way to find why this is happening, but thanks for answering.

  4. #4
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,363
    Thanks
    3
    Thanked 5,012 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Help for downloading Youtube video "server replied: Forbidden"

    Look, how would anyone know what is wrong with your code if the whole logic is inside this YtInfo class which we don't have any info about? In general your problem is not Qt related but YouTube related -- you don't know how to download a file from YouTube and not how to do it using Qt. If you were to do it using plain native sockets, you'd have exactly the same problem. By the way the "real" link you pasted is invalid so there is no way of us checking whether anything works.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  5. #5
    Join Date
    Apr 2011
    Posts
    61
    Thanks
    1
    Thanked 2 Times in 2 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Help for downloading Youtube video "server replied: Forbidden"

    It's valid here. I can use it for download here.
    Yeah, I think it's youtube related.

    I know how to do this manually, just get this link and use it for download, it's working here.
    The only problem is, download this real link with Qt, that doesn't work even with Qt Browser Demo.

    In Opera request for the video I have:

    Qt Code:
    1. Request Method: GET
    2.  
    3. Request URI: /videoplayback?sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor&fexp=912601&algorithm=throttle-factor&itag=34&ipbits=8&burst=40&sver=3&signature=40EA572A3CC8F901C65EB029E3F595C090C3F85F.06EDF73184B87C4158D7C0A430408969F43FDC2E&expire=1305612000&key=yt1&ip=189.0.0.0&factor=1.25&id=006008ab9fec6f8a
    4.  
    5. Request Version: HTTP/1.1
    6.  
    7. User-Agent: Opera/9.80 (Windows NT 5.1; U; pt-BR) Presto/2.8.131 Version/11.10\r\n
    8.  
    9. Host: v11.lscache8.c.youtube.com\r\n
    10.  
    11. Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1\r\n
    12.  
    13. Accept-Language: pt-BR,pt;q=0.9,en;q=0.8\r\n
    14.  
    15. Accept-Encoding: gzip, deflate\r\n
    16.  
    17. Connection: Keep-Alive\r\n
    18.  
    19. \r\n
    To copy to clipboard, switch view to plain text mode 

    QNetworkAccessManager request:

    Qt Code:
    1. Request Method: HEAD
    2.  
    3. Request URI: /videoplayback?sparams=id%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor&fexp=909406%252C912503&algorithm=throttle-factor&itag=34&ipbits=8&burst=40&sver=3&signature=40EA572A3CC8F901C65EB029E3F595C090C3F85F.06EDF73184B87C4158D7C0A430408969F43FDC2E&expire=1305612000&key=yt1&ip=189.0.0.0&factor=1.25&id=006008ab9fec6f8a
    4.  
    5. Request Version: HTTP/1.1
    6.  
    7. Youtubedl-no-compression: True\r\n
    8.  
    9. Connection: Keep-Alive\r\n
    10.  
    11. Accept-Encoding: gzip\r\n
    12.  
    13. Accept-Language: pt-BR,en,*\r\n
    14.  
    15. User-Agent: Mozilla/5.0\r\n
    16.  
    17. Host: v11.lscache8.c.youtube.com\r\n
    18.  
    19. \r\n
    To copy to clipboard, switch view to plain text mode 

    And now, I've just changed my code to this:

    Qt Code:
    1. void Downloader::start()
    2. {
    3. YtInfo* info = qobject_cast<YtInfo*>((QObject*)m_item->data(Qt::UserRole + 1).toInt());
    4.  
    5. QNetworkRequest request = QNetworkRequest(QUrl(info->getHigherQualityDownload()));
    6. request.setRawHeader("Accept", "text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1");
    7. request.setRawHeader("Accept-Encoding", "gzip, deflate");
    8. request.setRawHeader("Accept-Language", "pt-BR,pt;q=0.9,en;q=0.8");
    9. request.setRawHeader("User-Agent", "Opera/9.80 (Windows NT 5.1; U; pt-BR) Presto/2.8.131 Version/11.10");
    10.  
    11. m_reply = m_manager->get(request);
    12.  
    13. connect(m_reply, SIGNAL(downloadProgress(qint64,qint64)), SLOT(downloadProgress(qint64,qint64)));
    14. connect(m_reply, SIGNAL(finished()), SLOT(finished()));
    15. }
    To copy to clipboard, switch view to plain text mode 

    And I get the same "server replied: Forbidden" again.

    This is the request now:

    Qt Code:
    1. Request Method: GET
    2.  
    3. Request URI: /videoplayback?sparams=id%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor&algorithm=throttle-factor&itag=34&ipbits=8&burst=40&sver=3&signature=96B89386D0B3947AA32331AD917B7F23AD71AB2C.C8B734A2140BF665BF9A172C45126A38BCD1CCF9&expire=1305622800&key=yt1&ip=189.0.0.0&factor=1.25&id=006008ab9fec6f8a
    4.  
    5. Request Version: HTTP/1.1
    6.  
    7. Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1\r\n
    8.  
    9. Accept-Encoding: gzip, deflate\r\n
    10.  
    11. Accept-Language: pt-BR,pt;q=0.9,en;q=0.8\r\n
    12.  
    13. User-Agent: Opera/9.80 (Windows NT 5.1; U; pt-BR) Presto/2.8.131 Version/11.10\r\n
    14.  
    15. Connection: Keep-Alive\r\n
    16.  
    17. Host: v11.lscache8.c.youtube.com\r\n
    18.  
    19. \r\n
    To copy to clipboard, switch view to plain text mode 

    With the same request, I'm getting that forbidden error.
    What I can do now ? The request is exactly the same.

    I'm sending the YtInfo class (ytinfo.h and ytinfo.cpp) in the Attachments.

    The main functions is:
    That is called with this link:

    Qt Code:
    1. QString YtInfo::m_videoInfoUrl = "http://www.youtube.com/get_video_info?&video_id=%1%2&ps=default&eurl=&gl=US&hl=en";
    2.  
    3. //%1 = video id
    4. //%2 = "&el" type
    To copy to clipboard, switch view to plain text mode 

    Qt Code:
    1. void YtInfo::_loadInfo()
    2. {
    3. if (m_el < 0 || m_el > 4) return;
    4.  
    5. QList<QString> types;
    6.  
    7. types.append("");
    8. types.append("&el=detailpage");
    9. types.append("&el=embedded");
    10. types.append("&el=vevo");
    11.  
    12. QString infoUrl = m_videoInfoUrl.arg(m_id, types.at(m_el));
    13. QNetworkRequest request = QNetworkRequest(QUrl(infoUrl));
    14.  
    15. m_manager.get(request);
    16. }
    17.  
    18. void YtInfo::_infoLoaded(QNetworkReply* reply)
    19. {
    20. if (reply->error())
    21. {
    22. emit error(this, reply->errorString());
    23. }
    24. else
    25. {
    26. QUrl url("http://.../?" + reply->readAll());
    27.  
    28. if (url.hasQueryItem("fmt_url_map"))
    29. {
    30. QString fmtmap = url.queryItemValue("fmt_url_map");
    31.  
    32. if (!fmtmap.isEmpty())
    33. {
    34. fmtmap = QUrl::fromPercentEncoding(fmtmap.toAscii());
    35. QStringList urlmap = fmtmap.split(',');
    36.  
    37. foreach (QString fmturl, urlmap)
    38. {
    39. QStringList fmt = fmturl.split('|');
    40. m_avaliable[fmt.at(0)] = fmt.at(1);
    41. }
    42.  
    43. if (url.hasQueryItem("author"))
    44. {
    45. m_author = url.queryItemValue("author");
    46. m_author = QUrl::fromPercentEncoding(m_author.toAscii());
    47. }
    48.  
    49. if (url.hasQueryItem("title"))
    50. {
    51. m_title = url.queryItemValue("title");
    52. m_title = QUrl::fromPercentEncoding(m_title.replace('+', ' ').toAscii());
    53. }
    54.  
    55. if (url.hasQueryItem("length_seconds"))
    56. {
    57. QString tmp = url.queryItemValue("length_seconds");
    58. tmp = QUrl::fromPercentEncoding(tmp.toAscii());
    59.  
    60. m_length = tmp.toInt();
    61. }
    62.  
    63. emit ready(this);
    64. }
    65. else
    66. {
    67. emit error(this, "Empty fmt_url_map.");
    68. }
    69. }
    70. else
    71. {
    72. if (url.hasQueryItem("reason"))
    73. {
    74. if (url.hasQueryItem("errorcode"))
    75. {
    76. if (url.queryItemValue("errorcode") == "150")
    77. {
    78. reply->deleteLater();
    79.  
    80. if (m_el < 0 || m_el > 4)
    81. {
    82. emit error(this, "fmt_url_map not found.\nReason: " + QUrl::fromPercentEncoding(url.queryItemValue("reason").replace('+', ' ').toAscii()));
    83.  
    84. return;
    85. }
    86. else
    87. {
    88. m_el++;
    89. _loadInfo();
    90.  
    91. return;
    92. }
    93. }
    94. }
    95.  
    96. emit error(this, "fmt_url_map not found.\nReason: " + QUrl::fromPercentEncoding(url.queryItemValue("reason").replace('+', ' ').toAscii()));
    97. }
    98. else
    99. {
    100. emit error(this, "fmt_url_map not found.\nUnknown reason.");
    101. }
    102. }
    103. }
    104.  
    105. reply->deleteLater();
    106. }
    To copy to clipboard, switch view to plain text mode 

    This is how to use YtInfo:

    Qt Code:
    1. void MainWindow::_getYtVideo(const QString& id)
    2. {
    3. YtInfo* info = new YtInfo(this);
    4.  
    5. connect(info, SIGNAL(ready(YtInfo*)), SLOT(ready(YtInfo*)));
    6. connect(info, SIGNAL(error(YtInfo*,QString)), SLOT(error(YtInfo*,QString)));
    7.  
    8. info->setVideo(id);
    9. info->load();
    10. }
    To copy to clipboard, switch view to plain text mode 

    The ready signal is emitted when the info is loaded and the real links is listed.

    The other thing that I've used for get the best quality download:

    Qt Code:
    1. QString YtInfo::getHigherQualityDownload()
    2. {
    3. foreach (YtFormatInfo inf, avaliableFormats())
    4. {
    5. if (m_avaliable.contains(inf.code))
    6. {
    7. return m_avaliable.value(inf.code);
    8. }
    9. }
    10.  
    11. return "";
    12. }
    To copy to clipboard, switch view to plain text mode 

    avaliableFormats() return a list with the YtFormatInfo sorted with having higher quality first. (YtFormatInfo have just a number (to get the real link in the map) and some info that I'm not using now, but will use later).

    EDIT:
    I've just noticed that downloading the file with C# WebClient works.

    My friend tested my real link that I've pasted and it's really working, the problem isn't with the real link or header, so what it can be ?

    I've tested and found that my real link works in QMediaPlayer demo. (That demo that uses Phonon)

    ytinfo.cpp
    ytinfo.h
    Last edited by rsilva; 17th May 2011 at 05:26.

  6. #6
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,593
    Thanks
    13
    Thanked 1,587 Times in 1,516 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: Help for downloading Youtube video "server replied: Forbidden"

    There are glaringly obvious differences in the two request URIs:
    Start of request URI from Opera:
    /videoplayback?sparams=id%2Cexpire%2Cip%2Cipbits%2Citag ...
    Start of request URI for your code:
    /videoplayback?sparams=id%252Cexpire%252Cip%252Cipbits%252Citag ...
    It will almost certainly work better if you do not percent-encode query elements that are already encoded.

  7. #7
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,363
    Thanks
    3
    Thanked 5,012 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: Help for downloading Youtube video "server replied: Forbidden"

    Quote Originally Posted by rsilva View Post
    It's valid here. I can use it for download here.
    Clicking the link in Firefox gives a nice blank page.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  8. #8
    Join Date
    Apr 2011
    Posts
    61
    Thanks
    1
    Thanked 2 Times in 2 Posts
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: Help for downloading Youtube video "server replied: Forbidden"

    So, we have a problem because I've pasted this link in Opera, Firefox, the Phonon example and Chrome.
    Downloaded the file with C# WebClient and sent the link for my friend and ALL places that I've tested worked.

    The only problem was inside the Qt App.

    But, ChrisW67 was right, that percent encoding problem was the problem and it's working now.
    When I paste a link with percent encoding in a browser (or in phonon example) it worked nice.

    But now I've found that using percent encoding links to make a request, it encodes the string again.
    Maybe that Url does this automatically, or something inside the request.

    And in the errorString it doesn't say that this happens, just output the link as the original.

    And now, this link without the encoding works.

    http://v24.lscache5.c.youtube.com/vi...dfd74e652d8e6c

    (also works in all places that I've tested earlier)

    Thanks for the help.

    Now, my only problem with this app is the high usage in the item delegate paint with buttons.

Similar Threads

  1. Downloading a video from youtube
    By Faster in forum Qt Programming
    Replies: 21
    Last Post: 28th November 2009, 12:46
  2. Replies: 3
    Last Post: 8th July 2008, 19:37
  3. Translation QFileDialog standart buttons ("Open"/"Save"/"Cancel")
    By victor.yacovlev in forum Qt Programming
    Replies: 4
    Last Post: 24th January 2008, 19:05
  4. question about "fortune server"
    By Dumbledore in forum Qt Programming
    Replies: 6
    Last Post: 13th October 2007, 20:42
  5. How a server can write "Hello" to a browser ?
    By probine in forum Qt Programming
    Replies: 2
    Last Post: 1st December 2006, 14:43

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.