Does not support non-ascii songurl with Python 2

[hexchain]

Due to different handling of string and bytes, mpDris running with Python 2 does not support non-ascii chars in song_url:

$ python2 ./mpDris2.in.py --debug
2016-09-12 23:07:36,558 mpDris2.in INFO: Using file:///home/hexchain/Music as music library path.
2016-09-12 23:07:36,559 mpDris2.in INFO: Using Mutagen to read covers from music files.
2016-09-12 23:07:36,559 mpDris2.in DEBUG: Using GObject-Introspection main loop.
2016-09-12 23:07:36,559 mpDris2.in DEBUG: Initializing GObject.Notify
2016-09-12 23:07:36,560 mpDris2.in WARNING: Failed to connect to GNOME Settings Daemon. Media keys won't work.
2016-09-12 23:07:36,562 mpDris2.in DEBUG: Sending command 'commands' (was idle? False)
2016-09-12 23:07:36,563 mpDris2.in DEBUG: Sending command 'urlhandlers' (was idle? False)
2016-09-12 23:07:36,563 mpDris2.in DEBUG: Connected to MPD server.
2016-09-12 23:07:36,564 mpDris2.in DEBUG: Sending command 'status' (was idle? False)
2016-09-12 23:07:36,564 mpDris2.in DEBUG: Sending command 'currentsong' (was idle? False)
2016-09-12 23:07:36,565 mpDris2.in DEBUG: Sending command 'status' (was idle? False)
2016-09-12 23:07:36,565 mpDris2.in DEBUG: _update_properties: current song = {'album': '\xe7\xb5\xb6\xe5\xaf\xbe\xe9\x9b\xb6\xe5\xba\xa6\xce\xb8\xe3\x83\x8e\xe3\x83\xb4\xe3\x82\xa1\xe3\x83\x86\xe3\x82\xa3\xe3\x83\x83\xe3\x82\xaf/\xe7\xa0\xb4\xe6\xbb\x85\xe3\x81\xae\xe7\xb4\x94\xe6\x83\x85', 'title': '\xe9\xa2\xa8\xe3\x81\xaf\xe4\xba\x88\xe5\x91\x8a\xe3\x81\xaa\xe3\x81\x8f\xe5\x90\xb9\xe3\x81\x8f', 'track': '3/0', 'artist': '\xe3\x83\xaf\xe3\x83\xab\xe3\x82\xad\xe3\x83\xa5\xe3\x83\xbc\xe3\x83\xac', 'pos': '4', 'last-modified': '2016-08-22T13:55:34Z', 'disc': '0/0', 'file': '\xe9\xa2\xa8\xe3\x81\xaf\xe4\xba\x88\xe5\x91\x8a\xe3\x81\xaa\xe3\x81\x8f\xe5\x90\xb9\xe3\x81\x8f.mp3', 'time': '364', 'id': '5'}
2016-09-12 23:07:36,565 mpDris2.in DEBUG: _update_properties: current status = {'songid': '5', 'playlistlength': '5', 'playlist': '2', 'repeat': '1', 'consume': '0', 'mixrampdb': '0.000000', 'random': '0', 'state': 'stop', 'volume': '-1', 'single': '0', 'nextsong': '0', 'song': '4', 'nextsongid': '1'}
2016-09-12 23:07:36,565 mpDris2.in DEBUG: Updated property: PlaybackStatus = Stopped
Traceback (most recent call last):
  File "./mpDris2.in.py", line 1307, in <module>
    mpd_wrapper.run()
  File "./mpDris2.in.py", line 279, in run
    if self.my_connect():
  File "./mpDris2.in.py", line 354, in my_connect
    self.timer_callback()
  File "./mpDris2.in.py", line 442, in timer_callback
    self._update_properties(force=False)
  File "./mpDris2.in.py", line 704, in _update_properties
    self.update_metadata()
  File "./mpDris2.in.py", line 565, in update_metadata
    song_url = os.path.join(params['music_dir'], song_url)
  File "/usr/lib/python2.7/posixpath.py", line 73, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 1: ordinal not in range(128)

With Python 3:

$ ./mpDris2.in.py --debug
2016-09-12 23:07:33,288 mpDris2.in INFO: Using file:///home/hexchain/Music as music library path.
2016-09-12 23:07:33,288 mpDris2.in INFO: Mutagen not available, covers in music files will be ignored.
2016-09-12 23:07:33,288 mpDris2.in DEBUG: Using GObject-Introspection main loop.
2016-09-12 23:07:33,289 mpDris2.in DEBUG: Initializing GObject.Notify
2016-09-12 23:07:33,290 mpDris2.in WARNING: Failed to connect to GNOME Settings Daemon. Media keys won't work.
2016-09-12 23:07:33,291 mpd INFO: Calling MPD connect('localhost', 6600, timeout=None)
2016-09-12 23:07:33,293 mpDris2.in DEBUG: Sending command 'commands' (was idle? False)
2016-09-12 23:07:33,293 mpd DEBUG: Calling MPD commands()
2016-09-12 23:07:33,293 mpDris2.in DEBUG: Sending command 'urlhandlers' (was idle? False)
2016-09-12 23:07:33,293 mpd DEBUG: Calling MPD urlhandlers()
2016-09-12 23:07:33,293 mpDris2.in DEBUG: Connected to MPD server.
2016-09-12 23:07:33,294 mpDris2.in DEBUG: Sending command 'status' (was idle? False)
2016-09-12 23:07:33,294 mpd DEBUG: Calling MPD status()
2016-09-12 23:07:33,294 mpDris2.in DEBUG: Sending command 'currentsong' (was idle? False)
2016-09-12 23:07:33,294 mpd DEBUG: Calling MPD currentsong()
2016-09-12 23:07:33,295 mpDris2.in DEBUG: Sending command 'status' (was idle? False)
2016-09-12 23:07:33,295 mpd DEBUG: Calling MPD status()
2016-09-12 23:07:33,295 mpDris2.in DEBUG: _update_properties: current song = {'artist': 'ワルキューレ', 'file': '風は予告なく吹く.mp3', 'disc': '0/0', 'track': '3/0', 'album': '絶対零度θノヴァティック/破滅の純情', 'title': '風は予告なく吹く', 'last-modified': '2016-08-22T13:55:34Z', 'time': '364', 'id': '5', 'pos': '4'}
2016-09-12 23:07:33,295 mpDris2.in DEBUG: _update_properties: current status = {'nextsongid': '1', 'state': 'stop', 'playlistlength': '5', 'song': '4', 'playlist': '2', 'consume': '0', 'single': '0', 'nextsong': '0', 'volume': '-1', 'songid': '5', 'repeat': '1', 'random': '0', 'mixrampdb': '0.000000'}
2016-09-12 23:07:33,295 mpDris2.in DEBUG: Updated property: PlaybackStatus = Stopped
2016-09-12 23:07:33,296 mpDris2.in DEBUG: Updated property: Metadata = dbus.Dictionary({'xesam:album': '絶対零度θノヴァティック/破滅の純情', 'xesam:title': '風は予告なく吹く', 'mpris:trackid': dbus.ObjectPath('/org/mpris/MediaPlayer2/Track/5'), 'xesam:artist': ['ワルキューレ'], 'mpris:length': dbus.Int64(364000000), 'xesam:url': 'file:///home/hexchain/Music/風は予告なく吹く.mp3', 'xesam:discNumber': 0, 'xesam:trackNumber': 3}, signature=dbus.Signature('sv'))
2016-09-12 23:07:33,296 mpd DEBUG: Calling MPD idle[]
2016-09-12 23:07:33,296 mpDris2.in DEBUG: Entered idle
^C2016-09-12 23:07:34,606 mpDris2.in DEBUG: Caught SIGINT, exiting.
2016-09-12 23:07:34,607 mpd DEBUG: Calling MPD close()
2016-09-12 23:07:34,608 mpd INFO: Calling MPD disconnect()
2016-09-12 23:07:34,609 mpDris2.in DEBUG: Exiting

[grawity]

I'm not sure if broken Python 2 support is a real bug nowadays... If you can use Py3, then use Py3. (Even Mutagen has been ported already.)

That said:

1. Your metadata is fully Unicode, namely UTF-8. Did you mean "non-ASCII"?
2. In the latest Git build all sorts of Unicode metadata – title, artist, album, filepath – seem to be working fine, with both Python 2.7.12 and 3.5.2.

(FWIW, I'm testing the Python 2 version against python-mpd 0.3.0.)

[hexchain]

Sorry for the confusion. Title and description updated. According to the exception, the problem is not about metadata, but song_url.

[grawity]

Yes, I've tested that as well.

Actually, could you try inserting the following above the failing line in update_metadata?

logger.debug("music_dir is %s" % repr(params["music_dir"]))
logger.debug("song_url is %s" % repr(song_url))

[hexchain]

I've tried adding two print statement:

print(repr(song_url), type(song_url))                               
print(repr(params['music_dir']), type(params['music_dir']))

and the result is:

'\xe9\xa2\xa8\xe3\x81\xaf\xe4\xba\x88\xe5\x91\x8a\xe3\x81\xaa\xe3\x81\x8f\xe5\x90\xb9\xe3\x81\x8f.mp3' <type 'str'>
u'file:///home/hexchain/Music' <type 'unicode'>

[grawity]

Can't imagine where it gets the latter one from. Could you test something else, though – your locale settings? What does env | egrep "^LANG|^LC" find, and what does locale charmap print out?

I've noticed Python 2 handles UTF-8 paths fine as long as the system locale is also an UTF-8 one.

[hexchain]

$ env | egrep '^LANG|^LC'
LANG=en_US.utf8
$ locale charmap
UTF-8
$ locale 
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=

[grawity]

¯_(ツ)_/¯

This seems to be one of those situations where Python chooses string types through a dice roll, and as a result the Python 2 code needs to be the exact opposite of Python 3 code... Or something.

Will take a look again later. In the mean time you're encouraged again to mgirate to Python 3.

[hexchain]

Well, this program doesn't seem to be fully unicode-aware.

Since in Python2 the sys.getdefaultencoding() is always ascii on Linux, trying to concatenate a unicode and a string will fail because Python will try to decode the string using ascii.

[grawity]

The question is how params['music_dir'] becomes an unicode() though... In all my tests – from env, from args, from config file – I only get a str().

[hexchain]

That's because config.get('Library', 'music_dir') -> unicode but find_music_dir() -> str, around line 1219.