A Developer’ s Life

After the update of yesterday,

Youtube changed the way on how to retreive the video from it’s html page again.
This implies, that the 2 old versions of the proxy no longer worked and that videos could no longer be retreived.

After a quick peek, the solution was rather simple.
Here’s the code for the new and improved proxy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

<?php
    $id = trim($_REQUEST['id']);
 
    $url = "http://www.youtube.com/watch?v=" . $id . "&fmt=18";
 
    $ch = curl_init();
 
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
    $info = curl_exec($ch);
 
    if (!preg_match('#\'SWF_ARGS\': (\{.*?\})#is', $info, $matches))
    {
        echo "Check the YouTube URL : {$url} <br/>\n";
        die("Couldnt detect swfArgs");
    }
 
    if (function_exists(json_decode)) # >= PHP 5.2.0
    {
	$swfArgs = json_decode($matches[1]);
        $video_id = $swfArgs->video_id;
        $t = $swfArgs->t;
    }
    else
	{
        preg_match('#"video_id":.*?"(.*?)"#is', $matches[1], $submatches);
        $video_id = $submatches[1];
 
        preg_match('#"t":.*?"(.*?)"#is', $matches[1], $submatches);
        $t = $submatches[1];
    }
 
	curl_close($ch);
 
	$fullPath = "http://www.youtube.com/get_video.php?video_id=" . $video_id . "&t=" . $t; // construct the path to retreive the video from
 
	$headers = get_headers($fullPath); // get al headers from the url
 
	foreach($headers as $header){ //search the headers for the location url of youtube video
		if(preg_match("/Location:/i",$header)){
			$location = $header;			  
		}
	}
 
	header($location); // go to the location specified in the header and get the video
?>

That’s all there is to it.

This file will also be updated again in the source and files download of the original post.

Lately,
I’ve been busy trying to get youtube movies to play in the standard FLVPlayback component.

Getting this to work was really a pain in the butt now and then, especially because while I was working on this,
youtube changed some stuff on their side undoing all my previous work. But hey, stuff happens!
In the end perseverance and a lot of patience finally resulted in a working solution.

I will break up the solution into several pieces, describing the steps that need to be taken in order for this to work.

1) We need to get the video (.flv) from the youtube server.

To achieve this, one must of course understand how to get this video.
To start off, just navigate to a youtube movie. e.g. http://www.youtube.com/watch?v=vE_WqdKbTvY , then right-click anywhere in page and watch the page source code.

In the source code, search for the string “swfArgs”, which will look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

var swfArgs = {
"q": "top%20gear%20carrera%20gt",
"fexp": "903500,900130", 
"invideo": true, 
"sourceid": "ys",
"video_id":"vE_WqdKbTvY", 
"l": 478,
"fmt_map": "18/512000/9/0/115,34/0/9/0/115,5/0/7/0/0", 
"sk": "PbB4AIyvqpbLl0DYyCX5rpmXcQe-FEcZC", 
"is_doubleclick_tracked": "1", 
"usef": 0,
"t": "vjVQa1PpcFO6TnxVA4_nkqbqKN-z4CoWJgWn2Pfu77I=", 
"hl": "en", 
"plid": "AARoEenKN8A9FYLn", 
"vq": null, "ad_module": "http://s.ytimg.com/yt/swf/ad-vfl91517.swf", 
"cr": "NL", 
"tk": "j7SixOLxSJ1xxBIwWubnN9sXiu7hrejTmRv4ruMx4N3OaeyhN0xImQ=="};

As you can see this is an object, containing data about the swf and thus the .flv played at the youtube html page.
The video_id is the identifier of the movie and t is a token set by youtube that enables you to view / download the video.
This token expires after a given period of time, so I can’t place a permanent download link for the flv file here.

So now that we have the parameters of the video we need to fill them in as follows into a fixed url, used by youtube to retreive the flv video:
http://www.youtube.com/get_video.php?video_id=[id]&t=[t]
In our case:
http://www.youtube.com/get_video.php?video_id=vE_WqdKbTvY&t=vjVQa1PpcFO6TnxVA4_nkqbqKN-z4CoWJgWn2Pfu77I=
When navigating to this url with your browser, the video will start downloading

Don’t bother trying to open this last link, because as stated before, the token will probably have expired by the time you read this.
(Also the token is linked to the IP that first requested it, which in this case would be mine. More about this later)
If you would like to test if and how this works, just follow all the steps described above and see for yourself.
Pretty cool huh?

Now that we know how to get the .flv video of any youtube movie, we need to find a way to automatically do this, without the need to go through all the steps again.
We will use a php proxy script that eliminates all sandbox issues, accepts any youtube url and then returns the physical video (.flv) as if it were present on our own server.
However, this script will be a mere redirect to the location of the video on youtube servers, the file will not be physically downloaded by your server.

To create a script like this, I did some searching and quickly stumbled upon a script I found here.
This worked fine at first, but as I was saying youtube changed some things server side along the way.
That’s why i needed to search for a solution and needed to modify the script in order to getting it to work again.

The problem was caused because youtube changed the token idea.
A token is now linked to an IP address, meaning that the video can only be downloaded by the IP address that first requested the token.
This is a problem when trying to do this in-browser, where the browser is on the localhost, and the script is on a server and thus causing an IP mismatch. The result is that the file cannot be accessed.
After alot of searching around the final solution is to retreive the headers from the youtube download url and extracting the location of the flv that is embedded.

The result for the php proxy script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

<?php
    $id = trim($_REQUEST['id']);
 
    $url = "http://www.youtube.com/watch?v=" . $id;
 
    $url = $url . "&fmt=18"; //Gets the movie in High Quality, uncomment this line to get it in normal quality
 
    $ch = curl_init();
 
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
    $info = curl_exec($ch);
 
    if (!preg_match('#var swfArgs = (\{.*?\})#is', $info, $matches))
    {
        echo "Check the YouTube URL : {$url} <br/>\n";
        die("Couldnt detect swfArgs");
    }
 
    if (function_exists(json_decode)) # >= PHP 5.2.0
    {
	$swfArgs = json_decode($matches[1]);
        $video_id = $swfArgs->video_id;
        $t = $swfArgs->t;
    }
    else
    {
        preg_match('#"video_id":.*?"(.*?)"#is', $matches[1], $submatches);
        $video_id = $submatches[1];
 
        preg_match('#"t":.*?"(.*?)"#is', $matches[1], $submatches);
        $t = $submatches[1];
    }
 
    curl_close($ch);
 
    $fullPath = "http://www.youtube.com/get_video.php?video_id=" . $video_id . "&t=" . $t; // construct the path to retreive the video from
 
    $headers = get_headers($fullPath); // get all headers from the url
 
    foreach($headers as $header){ //search the headers for the location url of youtube video
	if(preg_match("/Location:/i",$header)){
	    $location = $header;			  
	}
    }
 
    header($location); // go to the location specified in the header and get the video
?>

An example of correct usage:

Navigate to http://www.dennisjaamann.com/demo/youtubeFLVPlayback/php/getYoutubeFLV.php?id=[videoID],
where the videoID is the id from the youtube movie http://www.youtube.com/watch?v=vE_WqdKbTvY
In this case: http://www.dennisjaamann.com/demo/youtubeFLVPlayback/php/getYoutubeFLV.php?id=vE_WqdKbTvY
When navigating to the url above, you will notice that the script will redirect you to the physical location of the .flv file.This will start the download prompt.

Isn’t this exactly what we need?

2) Workaround for FLVPlayback

When we take the url retreived above and try to play it in the standard FLVPlayback component as follows:

1
2
3
4

	var videoPlayer:FLVPlayback = new FLVPlayback();
        addChild(videoPlayer);
	videoPlayer.source = "http://www.dennisjaamann.com/demo/youtubeFLVPlayback/php/getYoutubeFLV.php?id=vE_WqdKbTvY";
	videoPlayer.play();

The video doesn’t play and we get the following error:
VideoError: 1005: Invalid xml: URL: “http://www.dennisjaamann.com/demo/youtubeFLVPlayback/php/getYoutubeFLV.php?id=vE_WqdKbTvY&FLVPlaybackVersion=2.1″ No root node found; if url is for an flv it must have .flv extension and take no parameters
at fl.video::SMILManager/http://www.adobe.com/2007/flash/flvplayback/internal::xmlLoadEventHandler()
at flash.events::EventDispatcher/dispatchEventFunction()
at flash.events::EventDispatcher/dispatchEvent()
at flash.net::URLLoader/onComplete()

Which is kind of a suprise, because when we open the source url in our browser there is clearly an .flv file there.

However, when analyzing the error message we can clearly see the problem. The key part here is:
“if url is for an flv it must have .flv extension and take no parameters”
When taking a look at our url http://www.dennisjaamann.com/demo/youtubeFLVPlayback/php/getYoutubeFLV.php?id=vE_WqdKbTvY,
it is obvious that it doesn’t end on .flv and that our url accepts parameters.

This behaviour is caused by a few lines of code in the NCManager class used by the FLVPlayback component.
There, the check is done whether the url ends on .flv and that the url does not accept parameters.

This leaves us 2 options:
- Extend the NCManager, overwrite this behaviour and make the FLVPlayback use this custom class
- Find a way to give the NCManager a correctly formatted url, so that it can stop nagging

I prefer the second option because it leaves the NCManager unmodified and hereby we can eliminate any unwanted behaviour.
Also, since I have a little php experience, I quickly realized that this had the funky smell of url rewriting.
With url rewriting you can take any url and format it to the format required which is exactly what we need here…

So we need to change the original url
http://www.dennisjaamann.com/demo/youtubeFLVPlayback/php/getYoutubeFLV.php?id=vE_WqdKbTvY
to
http://www.dennisjaamann.com/demo/youtubeFLVPlayback/videos/vE_WqdKbTvY.flv

This can be achieved by adding a .htaccess file to your project folder on the web server.
The code below is also included in the source files at the bottom of this post. But please note, not all operating systems show a .htaccess file.
This is because that it sometimes is a hidden file type.
Here’s the code for that:

1
2
3

Options +FollowSymlinks
RewriteEngine on
RewriteRule ^videos/([^/]+).flv php/getYoutubeFLV.php?id=$1 [NC]

And that’s all folks!
When we now execute our code again with the modified url, the NCManager has stopped nagging and the videoplayer plays the wanted video.

1
2
3
4

	var videoPlayer:FLVPlayback = new FLVPlayback();
        addChild(videoPlayer);
	videoPlayer.source = "http://www.dennisjaamann.com/demo/youtubeFLVPlayback/videos/vE_WqdKbTvY.flv";
	videoPlayer.play();

This workaround took like a minute or so to be succesful, leaves the the NCManager unchanged and does the trick.
That’s why I’ve chosen this solution because it is simple and elegant.

Read more about basic url rewriting here

3) Create the player.

All that is left now is to create our application. I have prepared an example in flex (sdk 3.3)

This is the result:

View the full example with source here
Or download the example source files here: youtubeFLVPlayback (1152)