Library update error

Orac

Hi All,
First post here after I've discovered the wonderful world of Rune Audio!

I've built myself a Raspberry Pi 2 Model B with a HiFiBerry DAC+ and housed in the HiFiBerry metal case. Installation of the Rune Audio software went OK and I got the DAC+ working OK. The problem is with the library.

I have a Qnap NAS to hold my music collection and there are two shares on it called 'mp3' and 'flac'. I added these as network mounts and I get a green tick next to both. Then, from the 'Sources' menu I clicked 'Rebuild MPD Library' and waited whilst it was updating. After it had finished I could see the mp3 share and browse it but no flac folder.

I ssh'd into the Pi and did 'ls /mnt/MPD/NAS/FLAC' and can see all the flac folders, I can also drill down into the individual folders and see the files listed, but as I said, no flac mount shows on the library screen.

Is this a known problem? I should also say that both my mp3 and flac collections are very large with probably 30000 tracks in each.

Any help would be appreciated.

hondagx35

Hi Orac,

one reason could be, that mpd has stopped the indexing of your music files because of an error.
If your MP3 mountpoint is the first one, then:
- check if all your MP3 files are in the library.
- delete your mounts and then add your FLAC first.

Frank

Orac

Hi Frank,

Thanks for the speedy reply.

I actually added the mount points one at a time, i.e. I added the mp3 first and ran the library update. When I browsed the library, all the folders/files seem to be there. I then added the second mount point, the flac folder, and then started the library update again. After the updating had finished, the mp3 folder was still visible and browsable but not the flac.

Is there any sort of log file I can turn on to see whats going on?

hondagx35

Hi Orac,

please try this:

- delete your mounts and then add your FLAC first.

first.

You can try to add this to your /etc/mpd.conf

log_file "/var/log/mpd.log"
log_level "verbose"

After this you have to restart mpd

systemctl restart mpd

The log will then be under /var/log/mpd.log

Frank

Orac

Thanks for the suggestions.

I tried them and I still can't get it to read all my flac files. When I look at the log file it just stops reading at some point and no errors are logged, so no help there.

I thought it may be the number of files, I have just over 29000 mp3's which it can read OK but I have just over 32000 flac's so I moved 3000 flacs out of the path and tried again, but it still won't read all of them. Out of curiosity, is there a maximum number of files that can be added to the database?

My next thought is to put all the flac's on to a USB drive and try that, but I can't get hold of a big enough one for a few days so I'm just listening to the mp3's for now.

If it's significant, the network mounts I am adding are using SMB/CIFS and they are located on a Qnap NAS.

Orac

A bit more information on this problem ... I'm leaning towards thinking it is a file number limitation.

I copied all my files to a desktop 4TB USB hard drive and plugged in. It was recognised straight away and started updating. I checked whilst the update was in progress and could see the folders/files being added. When the update had finished there were no files showing.

I looked in /var/lib/mpd and saw there was a file call mpd.db, which I assume is the database for the files but it was only 203 bytes in size.

Next, I disconnected the 4TB drive and replaced it with a USB stick with only few files on. This time when it had finished updating, the folders/files were visible and the mpd.db file was 11290 bytes in size. This is why I think it may be a file number limitation.

Any thoughts on how to get around this?

hondagx35

Hi Orac,

This is why I think it may be a file number limitation.

No there is no limitation in size.
The only limitation is for playlist size.

max_playlist_length <number>
This specifies the maximum number of songs that can be in the playlist. The default is 4096.

For sure there is a faulty entry (song, folder) in your music library that lets mpd stop scanning your files.

When I look at the log file it just stops reading at some point and no errors are logged, so no help there.

You have to find the file / folder and eliminate the error.
This was the reason why i told you to add your flac mount first or as the only one and then try to find the place.

Frank

Orac

Hi Frank,

I tried your suggestion but to no avail. I tried 4 times, between each time I removed the mount and restarted the RPi. It was stopping at a different point each time, around 17500 files. The files it stopped at looked perfectly OK, no odd characters in the name. Curiously though, I separated the files into two groups and made sure there were less than 17500 files and the same thing happened. Below is an extract from the end of the log file ...

Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 03 Almost Cut My Hair.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 03 Almost Cut My Hair.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 10 Everybody I Love You.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 10 Everybody I Love You.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 09 Country Girl.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 09 Country Girl.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 02 Teach Your Children.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 02 Teach Your Children.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 05 Woodstock.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 05 Woodstock.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 01 Carry On.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 01 Carry On.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 07 Our House.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 07 Our House.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 08 4 Plus 20.flac
Aug 20 08:56 : update: added NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 08 4 Plus 20.flac
Aug 20 08:56 : update: reading NAS/FLAC_1/C/Crosby, Stills, Nash & Young/1970 - Deja Vu/Crosby,Stills,Nash And Young - Deja Vu - 06 Deja Vu.flac
Aug 20 08:56 : avahi: Initializing interface
Aug 20 08:56 : avahi: Client changed to state 2
Aug 20 08:56 : avahi: Client is RUNNING
Aug 20 08:56 : avahi: Registering service _mpd._tcp/RuneAudio
Aug 20 08:56 : avahi: Service group changed to state 0
Aug 20 08:56 : avahi: Service group is UNCOMMITED
Aug 20 08:56 : state_file: Loading state file /var/lib/mpd/mpdstate
Aug 20 08:56 : avahi: Service group changed to state 1
Aug 20 08:56 : avahi: Service group is REGISTERING
Aug 20 08:56 : avahi: Service group changed to state 2
Aug 20 08:56 : avahi: Service 'RuneAudio' successfully established.
Aug 20 08:56 : client: [0] opened from local
Aug 20 08:56 : client: [0] process command "swap 0 0"
Aug 20 08:56 : client: [0] command returned 2
Aug 20 08:56 : client: [0] process command "status"
Aug 20 08:56 : client: [0] command returned 0
Aug 20 08:56 : client: [0] process command "idle"
Aug 20 08:56 : client: [0] command returned 1
Aug 20 08:58 : client: [1] opened from local
Aug 20 08:58 : client: [1] process command "close"
Aug 20 08:58 : client: [1] command returned 3
Aug 20 08:58 : client: [1] closed
Aug 20 08:58 : client: [2] opened from local
Aug 20 08:58 : client: [2] process command "clear"
Aug 20 08:58 : client: [2] command returned 0
Aug 20 08:58 : client: [0] process command "status"
Aug 20 08:58 : client: [0] command returned 0
Aug 20 08:58 : client: [2] process command "close"
Aug 20 08:58 : client: [2] command returned 3
Aug 20 08:58 : client: [2] closed
Aug 20 08:58 : client: [0] process command "playlistinfo"
Aug 20 08:58 : client: [0] command returned 0
Aug 20 08:58 : client: [0] process command "close"
Aug 20 08:58 : client: [0] command returned 3
Aug 20 08:58 : client: [0] closed
Aug 20 08:58 : client: [3] opened from local
Aug 20 08:58 : client: [3] process command "status"
Aug 20 08:58 : client: [3] command returned 0
Aug 20 08:58 : client: [3] process command "idle"
Aug 20 08:58 : client: [3] command returned 1
Aug 20 08:58 : client: [4] opened from local
Aug 20 08:58 : client: [4] process command "playlistinfo"
Aug 20 08:58 : client: [4] command returned 0
Aug 20 08:58 : client: [4] process command "close"
Aug 20 08:58 : client: [4] command returned 3
Aug 20 08:58 : client: [4] closed
Aug 20 08:58 : client: [5] opened from local
Aug 20 08:58 : client: [5] process command "lsinfo "NAS""
Aug 20 08:58 : db: No such directory
Aug 20 08:58 : client: [5] command returned 2
Aug 20 08:58 : client: [5] process command "close"
Aug 20 08:58 : client: [5] command returned 3
Aug 20 08:58 : client: [5] closed

Does it looks as though the RPi has reset during the read? Could it be a network issue?

Particularly the line 5 from the bottom 'Aug 20 08:58 : db: No such directory' looks odd.

hondagx35

Hi Orac,

one thing you can try is to disable mpd autoupdate.

MENU => MPD => Auto update = disable

Does it looks as though the RPi has reset during the read?

No the PI doesn't reset at this point, but mpd is controlled by Rune Audio
and it may get restarted at boottime.
With the above option you can control the update process.
So disable it and update manually after you restarted your device.

Frank

Orac

Finally solved the problem!

I copied my FLAC library to another share on the NAS and then went through each folder deleting all unnecessary files, like pdf's, text files and such so that only the music files (.flac) and the album art (.jpg) was left. Also, I deleted all the SACD (.dsf) files since I don't plan on playing them via Rune (at least not yet anyway).

Whilst I was cleaning stuff up I noticed that some filenames had odd characters in them, usually things like an accent on the 'a' or 'e' (mainly Celtic groups like Clannad) so I changed these to 'normal' characters. This time I got much further but still it stopped at a certain point. I knew there was some sort of problem with one particular folder (Deja Vu by CSN&Y), so I deleted that and bingo! it worked.

Conclusion after all this:
I'm not sure whether deleting the unnecessary files was needed. I have no idea what was wrong with the Deja Vu folder because it played fine using VLC, although for some reason the track titled 'Deja Vu' would not index properly in another program I use (Media Center), it may have been the only culprit. Maybe MPD struggles with non-standard characters, again I don't know the answer to this. When I've got my stamina back, I may try adding the dsf files.

Now I can get on with listening to the music

Thanks for your input Frank.