YouTube Support (Search and Play)

Suggestions/requests/ideas for RuneAudio core features and functions

Re: YouTube Support (Search and Play)

Postby mika-nl » 29 Jul 2015, 08:24

ACX wrote:This somehow scares me when I think to invest time and effort to integrate YouTube in RuneAudio:
How YouTube killed an extension with 300,000 users

Play the video over the hdmi ;/
So the video wil be play for YouTube But when you have no hdmi connect you only have the sound.

Gr mika
mika-nl
 
Posts: 25
Joined: 23 Apr 2015, 19:27

Re: YouTube Support (Search and Play)

Postby waves » 02 Aug 2015, 13:18

vikomarn wrote:We could automate this using bookmarklet approach i.e. Rune could implement a special URL to add the URL to playlist, which can be invoked as bookmarklet from desktop browser.

And on android, there is an helper app called "Bookmarklet Free". It registers itself as one of the apps that you see when you use the "Share" feature from mobile browser (and other apps). We can then define a custom URL handler to submit it to local Rune server.

I like that idea vikomarn. Rune already supports sending mpc commands as http parameters. So something similar that takes youtube urls as input and runs
Code: Select all
mpc add $(youtube-dl -f141 -g [URL] )

would be good enough. If such a no frills version of this was added by the devs at first then it could be expanded later on if it proves popular.

In the meantime it should be possible for Android users to combine Tasker and the JuiceSSH tasker plugin to make a widget that reads the clipboard and, if there's a youtube URL, SSHs into the Rune box and runs the command above. Then the user steps would be: 1 select and copy youtube url in android browser, 2 click Tasker widget. Not as handy as a bookmarklet but convenient enought perhaps. I'll have a go at it.

Edit: JuiceSSH tasker plugin don't support tasker variables/parameters so that is a no go. But some other tasker plugin might do the trick.
waves
 
Posts: 125
Joined: 31 Dec 2014, 10:33

Re: YouTube Support (Search and Play)

Postby waves » 03 Aug 2015, 11:13

One drawback with playing the youtube songs as a stream is that the title field in RuneAudio becomes a very long garbled string. Here is a workaround that first downloads the file and uses the name from youtube for the title field.
This also shows how to use plink to run the commands from a Windows computer on the same LAN as the Rune device.

The complete command
Code: Select all
plink -ssh [put local ip to rune box] -l root -pw rune "youtube-dl --no-mtime --restrict-filenames -o '/mnt/MPD/youtube/%(title)s.%(ext)s' -f141 https://www.youtube.com/watch?v=qORYO0atB6g  && mpc update --wait youtube ; mpc add youtube/$(ls /mnt/MPD/youtube/ -t | head -n1)"


Prerequisites:
1 Download the command line SSH tool Plink and open a cmd prompt in its folder.
2 A folder named /mnt/MPD/youtube must be created on the Rune device
Code: Select all
mkdir /mnt/MPD/youtube

3 Install youtube-dl if you haven't already
Code: Select all
pacman -Sy
pacman -S youtube-dl


I will now explain the command step by step

Code: Select all
plink -ssh [local ip to rune box] -l root -pw rune

This connects to the rune box using the default root credentials. Modify the ip and the pw (if your have changed it).

The rest, everything within the quote marks, is a chain of commands that will be run on the Rune device.

Code: Select all
youtube-dl --no-mtime --restrict-filenames -o '/mnt/MPD/youtube/%(title)s.%(ext)s' -f141 https://www.youtube.com/watch?v=qORYO0atB6g


Download the youtube URL, as audio ( -f141), to the folder /mnt/MPD/youtube/ with the name [youtube title].[fileextension, usually .m4a] , with only characters A-z 0-9 _- in the filename ( --restrict-filenames ) and set the time modified of the downloaded file to now ( --no-mtime ; needed for the last step below).

Code: Select all
 &&

Run the following commands in the chain only if the first was successful

Code: Select all
mpc update --wait youtube


Update the MPD database, but only scan the youtube folder (for quicker update( and wait for the update to finish. This makes it possible to find and play the new file in Rune.

Code: Select all
 ;

Marks that a new command starts

Code: Select all
mpc add youtube/

Tells MPC to add a file in the MPD database folder youtube to the playlist. Note that youtube is at "root" in the MPD database so should have no file system path prefix here.

Code: Select all
$(ls /mnt/MPD/youtube/ -t | head -n1)

The $( ) bit means that the code within the parenthesis will specify the name of the file.
the ls command lists files in the folder youtube based on their "time modified" values. The head -n1 part reverts the list and picks the first one, that is the most recently modified file, i.e. the one we just downloaded. Note that the list command works with the filesystem so here the path to the youtube folder is needed.

If everything works it should take about 10 seconds until the file starts playing. I don't notice any difference in speed when playing the same URL as a stream. Can anyone see a way to speed up the process?

Ways this could be improved:
Add a command that uses some linux command line audio tagging tool to add proper artist and title tags and also download the youtube video placeholder image and use as cover image in Rune playback.
Add a command that clears all files in the youtube folder that are X days old.
Make a script on Windows that takes the youtube URL as input and runs the whole plink command in the background.
Some youtube URLs don't have -f141 audio. Add a fallback command that uses -f140.
Anything that speeds up the process.

Feel free to chip in with enhancements!

Edit: this post on the volumio forums is relevant. Something like those php steps looks doable for Rune too. Once we can do something like
Code: Select all
http://192.168.1.6/youtube.php?yurl=https://www.youtube.com/watch?v=qORYO0atB6g
the bookmarklet approach vikomarn suggested would work.

Edit: youtube-dl is slow to start on a Raspi . This is a possible workaround from the raspi forums.
Last edited by waves on 03 Aug 2015, 20:13, edited 1 time in total.
waves
 
Posts: 125
Joined: 31 Dec 2014, 10:33

Re: YouTube Support (Search and Play)

Postby waves » 03 Aug 2015, 20:12

I played around a bit with the php code in the volumio link. I can add a prompt box to RuneAudio, use it to trigger a custom php file that in turn runs youtube-dl and tries to save a file with the prompt box string as URL parameter. The process appears to start but no file gets saved. I think it is a permissions problem of some sort.

ACX, do you have any suggestion on what permissions / ownership I should set for for the /mnt/MPD/youtube folder so that the php action can make youtube-dl write to that folder?

Edit:
While waiting for advice on the above, here is a mod that lets the user simply add youtube audio as a stream to the playlist. Warning: it takes (at least on my pi) 10-15 seconds until the youtube stream gets added to the list and 10-15 more seconds to actually get it to play. And the stream title is a very long garbled string, so long that the UI seems to break a bit in Android browsers (the bottom row of buttons disappear).

Only two changes are neeeded, assuming youtube-dl is installed.

1. Create the file /srv/http/y.php and put this code in it
Code: Select all
<?php
try {
    $u = $_GET['url'];
    exec("mpc add \$(youtube-dl -f141 -g " . $u .") || mpc add \$(youtube-dl -f140 -g " . $u . ")");
} catch (Exception $e) {
    echo "Exception :  ", $e->getMessage(),"<br />";
}
?>


2. Next we edit runeui.js and runeui.min.js . It might be good to make backups of these first in case anything goes wrong.
Edit
Code: Select all
/srv/http/assets/js/runeui.js 

and add the following code above the line in the code that starts with function refreshState()

Code: Select all
    $('#pl-count').click(function(){
      var stream = prompt("Enter stream URL", "");
      if (stream != null) {
        $.get("y.php?url=" + stream);
        return false;  //prevents redirect
      }     
    });


Save. Next we must minify: select all text in runeui.js and copy. Paste into editbox at
http://gpbmike.github.io/refresh-sf/
and click "javascript" button to the right. Next select and copy all code in the edit box.
Edit the file
Code: Select all
/srv/http/assets/js/runeui.min.js

, remove all text, paste from clipboard. Save.

Reload RuneAudio in the browser. You should now be able to click on the text "[number] entries" in the top left corner of the playlist screen to get a popup window. Enter a youtube URL and press ok, wait 10-15 seconds and you should see the stream in the playlist.
waves
 
Posts: 125
Joined: 31 Dec 2014, 10:33

Re: YouTube Support (Search and Play)

Postby ACX » 04 Aug 2015, 02:40

waves wrote:ACX, do you have any suggestion on what permissions / ownership I should set for for the /mnt/MPD/youtube folder so that the php action can make youtube-dl write to that folder?


Code: Select all
[root@runeaudio ~]# cd /mnt/MPD/
[root@runeaudio MPD]# mkdir YouTube
[root@runeaudio MPD]# ls -lah /mnt/MPD
total 28K
drwxr-xr-x 7 mpd  audio 4.0K Aug  4 03:37 .
drwxr-xr-x 3 root root  4.0K Aug  9  2014 ..
drwxr-xr-x 2 mpd  audio 4.0K Aug  9  2014 LocalStorage
drwxr-xr-x 3 mpd  audio 4.0K Jan  1  1970 NAS
drwxr-xr-x 2 mpd  audio 4.0K Oct 27  2014 USB
drwxr-xr-x 2 mpd  audio 4.0K Jul 30 19:36 Webradio
drwxr-xr-x 2 root root  4.0K Aug  4 03:37 YouTube

You can see that the "YouTube" folder has a different owner:group compared to the other ones (root:root instead of mpd:audio). That's easy to fix:

Code: Select all
[root@runeaudio MPD]# chown -R mpd:audio YouTube
[root@runeaudio MPD]# ls -lah /mnt/MPD
total 28K
drwxr-xr-x 7 mpd  audio 4.0K Aug  4 03:37 .
drwxr-xr-x 3 root root  4.0K Aug  9  2014 ..
drwxr-xr-x 2 mpd  audio 4.0K Aug  9  2014 LocalStorage
drwxr-xr-x 3 mpd  audio 4.0K Jan  1  1970 NAS
drwxr-xr-x 2 mpd  audio 4.0K Oct 27  2014 USB
drwxr-xr-x 2 mpd  audio 4.0K Jul 30 19:36 Webradio
drwxr-xr-x 2 mpd  audio 4.0K Aug  4 03:37 YouTube
User avatar
ACX
RuneAudio co-founder
 
Posts: 1692
Joined: 29 Nov 2013, 02:25
Location: Udine, Italy

Re: YouTube Support (Search and Play)

Postby waves » 04 Aug 2015, 13:35

Hi, I changed the folder owner accordingly. But doing file download with youtube-dl from php still doesn't work. Is some additional permission needed? I use the same runeui.js code as above plus this modification of the y.php file:

Code: Select all
<?php
try {
    $u = $_GET['url'];
   exec("youtube-dl --no-mtime --restrict-filenames -o '/mnt/MPD/youtube/%(title)s.%(ext)s' --write-description --youtube-skip-dash-manifest -fbestaudio " . $_GET['url'] );   
} catch (Exception $e) {
    echo "Exception :  ", $e->getMessage(),"<br />";
}
?>


That youtube-dl command works fine from the SSH command line. I also tried to put the file download commands in a bash script file and call that with exec but that doesn't work either.

Edit: If I in php do
Code: Select all
echo exec('whoami');

I get the result "http". If I then do
Code: Select all
cd /mnt/MPD
chown -R http:http youtube

then the save works. Good! But are there any drawbacks to doing so? I'm a php noob but my hunch is that I should add some code to "secure" that folder and/or restrict the strings that can be passed to the exec command.
waves
 
Posts: 125
Joined: 31 Dec 2014, 10:33

Re: YouTube Support (Search and Play)

Postby waves » 05 Aug 2015, 08:59

Building on the last couple of posts here is a new version. This setup lets the user enter a youtube URL in the Rune UI in the browser and the youtube audio gets downloaded and added to the playlist as a temporary file with the same title as in youtube.

All the steps presuppose that you SSH in to the Rune computer during setup.

1 Install youtube-dl if you haven't already.
Code: Select all
pacman -Sy
pacman -S youtube-dl

(Note: this default youtube-dl install is quite slow. A raspi forum post explain why and give an alternative install method. With that method and by manually disabling many imports in the youtube-dl source files I could double the speed, but at the cost of having to redo the changes when updating youtube-dl. Hopefully youtube-dl devs will fix the problem soon.)

2 create the folder /mnt/MPD/youtube and change its owner to http
Code: Select all
cd /mnt/MPD
mkdir youtube
chown -R http:http youtube

This is where the youtube audio files will be stored. Don't put any other music files in the /youtube folder because any file there will be removed 5 days after its last use in order to save discspace (see step 5 below).

3 edit /srv/http/assets/js/runeui.js ( and runeui.min.js )
find these lines
Code: Select all
// update info and status on Playback tab
function refreshState() {

and add this code above them
Code: Select all
    $('#pl-count').click(function(){
      var stream = prompt("Enter stream URL", "");
      if (stream != null) {
        var encstream = encodeURI(stream);  //url encode
        encstream = encodeURIComponent(encstream); //encodes also ? & ... chars
        $.get("y.php?url=" + encstream);
        return false;
      }     
    });


Save. Next we must minify: select all text in runeui.js and copy. Paste to editbox at http://gpbmike.github.io/refresh-sf/ , click "javascript" to the right, copy the new text in the box, edit /srv/http/assets/js/runeui.min.js and replace everything in it with the new text.

4 create /srv/http/y.php
and add
Code: Select all
<?php
try {
     $u = urldecode($_GET['url']);
     #validate
     $youtube = "/^(https:\/\/|http:\/\/|)(www\.|m\.|)(youtube\.com\/watch\?v=|youtu\.be\/)[\w-]+$/";
     if ( preg_match ($youtube, $u ) )
      exec("tube " . $u );
}
catch (Exception $e) {
    echo "Exception :  ", $e->getMessage(),"<br />";
}
?>


5. create the file /usr/local/bin/tube
and add this bash script code (note that the second line starting with youtube-dl is one single long line without linebreaks)
Code: Select all
#!/bin/bash
youtube-dl --no-mtime --restrict-filenames -o '/mnt/MPD/youtube/%(title)s.%(ext)s' --write-description -f "bestaudio[ext=m4a]" $1 && mpc update --wait youtube && VV=$(ls /mnt/MPD/youtube/*.description -t | head -n1) && VV=$(basename $VV .description) && mpc add "youtube/$VV.m4a" && echo $VV && find /mnt/MPD/youtube -type f -mtime +5 -delete && chown -R http:http /mnt/MPD/youtube/$VV.*

These commands save a youtube URL as "youtube-page-title.m4a" with the best available audio quality, adds it to the MPD library and then to the Rune playlist. It also cleans up files in /youtube older than 5 days.
(Note: The code identifies the saved file by checking for the last modified file in /youtube. If a URL has been streamed previously and the file is still there youtube-dl won't redownload it and the file timestamp isn't updated. To work around that the code also saves a .description file for the URL, then finds the most recently modified .description file and adds its associated .m4a file to the Rune playlist. Note2: the chown command is there to handle cases where the user first runs the script from SSH as root to download a URL and later tries to redownload that URL from the Rune UI.)

Setup is complete. Reload Rune in the browser and click on the text "9 entries" (or whatever number) in the heading in the playlist view. A box pops to enter the youtube URL in. Click OK and wait 10-15 (or more seconds). If it works the file will then show up in the playlist.

1.png
1.png (120.36 KiB) Viewed 4694 times


edit: it would be nice to also show cover art for these youtube audio files, wouldn't it? Here is how.

6. edit the file /usr/local/bin/tube created in step 5 above
change the string
Code: Select all
--write-description

to
Code: Select all
--write-description --write-thumbnail


7. edit the file /srv/http/app/coverart_ctl.php
and find the line that starts with
Code: Select all
$local_cover_root = substr($currentpath

and add this code on new lines right below it
Code: Select all
$filename = pathinfo($currentpath, PATHINFO_FILENAME);
$local_cover_path[] = $local_cover_root.'/'.$filename.'.jpg';
$local_cover_path[] = $local_cover_root.'/'.$filename.'.png';


Save. Load this in the browser
Code: Select all
http://[LAN ip to RuneAudio]/clear


Any new youtube audio downloads will also save a cover image and display it during playback.

(Note: I first tried the youtube-dl option for embedding thumbnails but couldn't get it to work with the dash .m4a files from youtube, thus this workaround with external image files.)
waves
 
Posts: 125
Joined: 31 Dec 2014, 10:33

Re: YouTube Support (Search and Play)

Postby Dombo71 » 29 Feb 2016, 23:40

Dear all.
Is there some wiki how i can use this higher options.
I cannot understand how pacman work.
Is this Php?

Sorry i am a dombo, that is trying to make the best of it.. :oops:
Dombo71
 
Posts: 15
Joined: 27 Feb 2016, 13:24

Re: YouTube Support (Search and Play)

Postby Orion » 01 Mar 2016, 02:35

Dombo71 wrote:Dear all.
Is there some wiki how i can use this higher options.
I cannot understand how pacman work.
Is this Php?

Sorry i am a dombo, that is trying to make the best of it.. :oops:


Hi Dombo71,
this is a patch to various aspects of the system, intended to adding the capability to play YouTube streams on RuneAudio.
If you follow the post of waves (great work!), you will be able to add this functionality on the system.
The procedure includes javascript code (needed to extend the frontend UI), php code (needed to extend the player backend) and a package to be added to the system through pacman (youtube-dl, this is needed to download the music content from youtube).

Pacman is the package manager provided by ArchLinux (our base system). Essentially it does the same things as the more famous apt-get used in Debian-like systems.
Here is a great doc made by the Arch community, the Pacman Rosetta, that compares pacman and the other package managers. It is intended to be a guide for whom already knows another package manager (for example apt-get) and needs an help to use pacman in the same way.

Hope to have clarified your doubts.
Ciao.
Simone.
RuneAudio - Embedded Hi-Fi music player
SoloStereo - Alla riscoperta dell'alta fedeltà
User avatar
Orion
RuneAudio co-founder
 
Posts: 189
Joined: 29 Nov 2013, 02:03
Location: Udine, Italy

Re: YouTube Support (Search and Play)

Postby Zmash » 11 Sep 2016, 16:22

Hey, I want to thank you for the great work but I have a problem.
If I try to insert a youtube URL nothing happens. I also tried to call the y.php manually but it's the same result (example: http://runeaudio.local/y.php?url=https% ... SGh9h5yWIg). When I insert a echo into the y.php and open it there is no console output. Any idea why I can't open/run php files?

Zmash
Zmash
 
Posts: 1
Joined: 11 Sep 2016, 15:21

support RuneAudio Donate with PayPal

PreviousNext

Return to Feature request

Who is online

Users browsing this forum: No registered users and 0 guests