upstream/IPod-Shuffle-4g
1
0
Fork 1
mirror of https://github.com/nims11/IPod-Shuffle-4g.git synced 2025-12-08 00:18:01 +09:00
Code Issues Projects Releases Packages Wiki Activity Actions

Compare commits

base: upstream:master
Branches Tags
upstream:master
upstream:NicoHood-patch-1
upstream:docs
upstream:v1.5
upstream:v1.4
upstream:v1.3
upstream:v1.2
upstream:v1.1
..
compare: upstream:v1.2
Branches Tags
upstream:master
upstream:NicoHood-patch-1
upstream:docs
upstream:v1.5
upstream:v1.4
upstream:v1.3
upstream:v1.2
upstream:v1.1

No commits in common. "master" and "v1.2" have entirely different histories.
master ... v1.2

5 changed files with 210 additions and 501 deletions
Download patch file Download diff file Expand all files Collapse all files

111
CHANGELOG.md
View file

@ -1,111 +0,0 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
This changlog uses the [ISO 8601 date format](https://www.iso.org/iso-8601-date-and-time-format.html) of (YYYY-MM-DD).
## [Unreleased]
## [1.5.1] - 2021-06-05
### Changed
* Moved Changelog from Readme to Changelog.md file
### Fixed
* Fix TypeError when reading playlist files [#50](https://github.com/nims11/IPod-Shuffle-4g/pull/50)
## [1.5.0] - 2020-06-10
### Changed
* Port Script to Python3
* Mutagen support is now optional
## [1.4.0] - 2016-08-28
### Added
* Added optional `--verbose` output
* Added files to `extras` folder
* Added shortcut parameters (`-p`, `-t`, `-d`, etc.)
### Changed
* Renamed `--voiceover` to `--track-voiceover`
* Renamed script from `shuffle.py` to `ipod-shuffle-4g.py`
* Ignore hidden filenames
* Do not force playlist voiceover with auto playlists
### Fixed
* Catch "no space left" error [#30](https://github.com/nims11/IPod-Shuffle-4g/issues/30)
* Fix UnicodeEncodeError for non-ascii playlist names [#35](https://github.com/nims11/IPod-Shuffle-4g/issues/35)
## [1.3.0] - 2016-06-08
### Added
* Directory based auto playlist building (`--auto-dir-playlists`) [#13](https://github.com/nims11/IPod-Shuffle-4g/issues/13)
* ID3 tags based auto playlist building (`--auto-id3-playlists`)
* Added short program description
* Differentiate track and playlist voiceover [#26](https://github.com/nims11/IPod-Shuffle-4g/issues/26)
### Changed
* Voiceover disabled by default [#26](https://github.com/nims11/IPod-Shuffle-4g/issues/26) (Playlist voiceover enabled with auto playlist generation)
### Fixed
* Fix hyphen in filename [#4](https://github.com/nims11/IPod-Shuffle-4g/issues/4)
* Fixed mutagen bug [#5](https://github.com/nims11/IPod-Shuffle-4g/issues/5)
## [1.2.0] - 2016-02-04
### Added
* Added Path help entry
* Added MIT License
* Added this changelog
### Changed
* Skip existing voiceover files with the same name (e.g. "Track 1.mp3")
* Made help message lower case
* Improved Readme
* Improved docs
### Fixed
* Additional fixes from NicoHood
* Fixed "All Songs" and "Playlist N" sounds when voiceover is disabled [#17](https://github.com/nims11/IPod-Shuffle-4g/issues/17)
* Better handle broken playlist paths [#16](https://github.com/nims11/IPod-Shuffle-4g/issues/16)
* Only use voiceover if dependencies are installed
## [1.1.0] - 2016-01-23
### Added
* Fixes from nims11 fork
* Option to disable voiceover
* Initialize the IPod Directory tree
* Using the `--rename-unicode` flag filenames with strange characters and different language are renamed which avoids the script to crash with a Unicode Error
* Other small fixes
## [1.0.0] - 2012-10-17
### Added
* Original release by ikelos
[Unreleased]: https://github.com/nims11/IPod-Shuffle-4g/compare/1.5.1...HEAD
[1.5.1]: https://github.com/nims11/IPod-Shuffle-4g/compare/v1.5...1.5.1
[1.5.0]: https://github.com/nims11/IPod-Shuffle-4g/compare/v1.4...v1.5
[1.4.0]: https://github.com/nims11/IPod-Shuffle-4g/compare/v1.3...v1.4
[1.3.0]: https://github.com/nims11/IPod-Shuffle-4g/compare/v1.2...v1.3
[1.2.0]: https://github.com/nims11/IPod-Shuffle-4g/compare/v1.1...v1.2
[1.1.0]: https://github.com/nims11/IPod-Shuffle-4g/compare/646b7def4c498c59b063e535a5b64695d8d87e6b...v1.1
[1.0.0]: https://github.com/nims11/IPod-Shuffle-4g/commit/646b7def4c498c59b063e535a5b64695d8d87e6b

146
README.md
View file

@ -1,98 +1,65 @@
# IPod Shuffle 4g Script
# IPod Shuffle 4g Scripts
## ipod-shuffle-4g.py
##shuffle.py
Python script for building the Track and Playlist database for the newer gen IPod Shuffle.
Forked from the [shuffle-db-ng project](https://code.google.com/p/shuffle-db-ng/)
and improved my nims11 and NicoHood.
Just put your audio files into the mass storage of your IPod and shuffle.py will do the rest.
```
$ ./ipod-shuffle-4g.py --help
usage: ipod-shuffle-4g.py [-h] [-t] [-p] [-u] [-g TRACK_GAIN]
[-d [AUTO_DIR_PLAYLISTS]] [-i [ID3_TEMPLATE]] [-v]
path
Python script for building the Track and Playlist database for the newer gen
IPod Shuffle. Version 1.5
Just put your audio files into the mass storage of your IPod and shuffle.py will do the rest
```bash
$ python shuffle.py -h
usage: shuffle.py [-h] [--disable-voiceover] [--rename-unicode]
[--track-gain TRACK_GAIN]
path
positional arguments:
path Path to the IPod's root directory
path
optional arguments:
-h, --help show this help message and exit
-t, --track-voiceover
Enable track voiceover feature
-p, --playlist-voiceover
Enable playlist voiceover feature
-u, --rename-unicode Rename files causing unicode errors, will do minimal
--disable-voiceover Disable Voiceover Feature
--rename-unicode Rename Files Causing Unicode Errors, will do minimal
required renaming
-g TRACK_GAIN, --track-gain TRACK_GAIN
Specify volume gain (0-99) for all tracks; 0 (default)
means no gain and is usually fine; e.g. 60 is very
loud even on minimal player volume
-d [AUTO_DIR_PLAYLISTS], --auto-dir-playlists [AUTO_DIR_PLAYLISTS]
Generate automatic playlists for each folder
recursively inside "IPod_Control/Music/". You can
optionally limit the depth: 0=root, 1=artist, 2=album,
n=subfoldername, default=-1 (No Limit).
-i [ID3_TEMPLATE], --auto-id3-playlists [ID3_TEMPLATE]
Generate automatic playlists based on the id3 tags of
any music added to the iPod. You can optionally
specify a template string based on which id3 tags are
used to generate playlists. For eg. '{artist} -
{album}' will use the pair of artist and album to
group tracks under one playlist. Similarly '{genre}'
will group tracks based on their genre tag. Default
template used is '{artist}'
-v, --verbose Show verbose output of database generation.
--track-gain TRACK_GAIN
Store this volume gain (0-99) for all tracks; 0
(default) means no gain and is usually fine; e.g. 60
is very loud even on minimal player volume
```
#### Dependencies
This script requires:
* [Python 3](https://www.python.org/download/releases/3.0/)
Optional album/artist and auto-id3-playlists support:
* [Python 2.7](http://www.python.org/download/releases/2.7/)
* [Mutagen](https://code.google.com/p/mutagen/)
Optional Voiceover support
* [eSpeak](http://espeak.sourceforge.net/)
* [PicoSpeaker](http://picospeaker.tk/readme.php)
* [RHVoice (master branch, 3e31edced402a08771d2c48c73213982cbe9333e)](https://github.com/Olga-Yakovleva/RHVoice) -- (Russian files only)
* [SoX](http://sox.sourceforge.net) -- (Russian files)
* say (macOS)
* [PicoSpeaker](http://picospeaker.tk/readme.php) -- for non-Russian files
* [RHVoice (master branch, 3e31edced402a08771d2c48c73213982cbe9333e)](https://github.com/Olga-Yakovleva/RHVoice) -- for Russian files only
* [SoX](http://sox.sourceforge.net) -- for Russian files only
##### Ubuntu
`apt-get install python3 python-mutagen libttspico*`
`apt-get install python-mutagen libttspico*`
##### Arch Linux
From the **Extra** repository: `pacman -S python` and optional `pacman -S python-mutagen espeak` or from the AUR: `svox-pico-bin` ([link](https://aur.archlinux.org/packages/svox-pico-bin/))
You can also [install the script from AUR](https://aur.archlinux.org/packages/ipod-shuffle-4g/).
From the **Extra** repository: `pacman -S python2 mutagen` and from the AUR: `svox-pico-bin` ([link](https://aur.archlinux.org/packages/svox-pico-bin/))
##### Gentoo Linux
```bash
PYTHON_TARGETS="python3" emerge -av media-libs/mutagen
PYTHON_TARGETS="python2_7" emerge -av media-libs/mutagen
layman --add=ikelos
layman --overlays="https://raw.githubusercontent.com/ahippo/rhvoice-gentoo-overlay/master/repositories.xml" --fetch --add=ahippo-rhvoice-overlay
ACCEPT_KEYWORDS="~amd64" emerge -av app-accessibility/svox app-accessibility/rhvoice
```
References to the overlays above: [ikelos](http://git.overlays.gentoo.org/gitweb/?p=dev/ikelos.git;a=summary), [ahippo-rhvoice-overlay](https://github.com/ahippo/rhvoice-gentoo-overlay)
## Tips and Tricks
##Tips and Tricks
#### Disable trash for IPod
To avoid that linux moves deleted files into trash you can create an empty file `.Trash-1000`.
This forces linux to delete the files permanently instead of moving them to the trash.
Of course you can also use `shift + delete` to permanently delete files without this trick.
The file can be found in the [extras](extras) folder.
#### Compress/Convert your music files
([#11](https://github.com/nims11/IPod-Shuffle-4g/issues/11)) Shuffle is short on storage, and you might want to squeeze in more of your collection by sacrificing some bitrate off your files. In rarer cases, you might also possess music in formats not supported by your ipod. Although `ffmpeg` can handle almost all your needs, if you are looking for a friendly alternative, try [Soundconverter](http://soundconverter.org/).
#### Use Rhythmbox to manage your music and playlists
As described [in the blog post](https://nims11.wordpress.com/2013/10/12/ipod-shuffle-4g-under-linux/)
@ -104,7 +71,6 @@ Simply place a file called `.is_audio_player` into the root directory of your IP
name="Name's IPOD"
audio_folders=iPod_Control/Music/
```
The file can be found in the [extras](extras) folder.
Now disable the IPod plugin of Rhythmbox and enable the MTP plugin instead.
You can use Rythmbox now to generate playlists and sync them to your IPod.
@ -120,17 +86,6 @@ In all cases you can try to update Rythmbox to the latest version, sync again or
If you want to use this script on different computers it makes sense
to simply copy the script into the IPod's root directory.
#### Format/Restore/Recover IPod
([#41](https://github.com/nims11/IPod-Shuffle-4g/issues/41)) If you formatted your IPod wrong and lost all data you can still recover it.
It is important to **not use MBR/GPT**. You need to directly create a **Fat16 Filesystem**:
```bash
sudo mkfs.vfat -I -F 16 -n IPOD /dev/sdX
```
Run this script to generate the new database. All missing sound files should be regenrated by the IPod on next use.
Your IPod should work and play music again now.
## TODO
* Last.fm Scrobbler
* Qt frontend
@ -144,3 +99,58 @@ Your IPod should work and play music again now.
The original shuffle3db website went offline. This repository contains a copy of the information inside the `docs` folder.
Original data can be found via [wayback machine](https://web.archive.org/web/20131016014401/http://shuffle3db.wikispaces.com/iTunesSD3gen).
# Version History
```
1.2 Release (04.02.2016)
* Additional fixes from NicoHood
* Fixed "All Songs" and "Playlist N" sounds when voiceover is disabled #17
* Better handle broken playlist paths #16
* Skip existing voiceover files with the same name (e.g. "Track 1.mp3")
* Only use voiceover if dependencies are installed
* Added Path help entry
* Made help message lower case
* Improved Readme
* Improved docs
* Added MIT License
* Added this changelog
1.1 Release (11.10.2013 - 23.01.2016)
* Fixes from nims11 fork
* Option to disable voiceover
* Initialize the IPod Directory tree
* Using the --rename-unicode flag
filenames with strange characters and different language are renamed
which avoids the script to crash with a Unicode Error
* Other small fixes
1.0 Release (15.08.2012 - 17.10.2012)
* Original release by ikelos
```
# License and Copyright
```
Copyright (c) 2012-2016 ikelos, nims11, NicoHood
See the readme for credit to other people.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
```

0
extras/.Trash-1000
View file

2
extras/.is_audio_player
View file

@ -1,2 +0,0 @@
name="iPod Shuffle"
audio_folders=iPod_Control/Music/

452
ipod-shuffle-4g.py → shuffle.py
View file

@ -1,11 +1,12 @@
#!/usr/bin/env python3
# Builtin libraries
#!/usr/bin/env python2.7
# -*- coding: utf-8 -*-
import sys
import struct
import urllib.request, urllib.parse, urllib.error
import urllib
import os
import hashlib
import mutagen
import binascii
import subprocess
import collections
import errno
@ -15,12 +16,6 @@ import re
import tempfile
import signal
# External libraries
try:
import mutagen
except ImportError:
mutagen = None
audio_ext = (".mp3", ".m4a", ".m4b", ".m4p", ".aa", ".wav")
list_ext = (".pls", ".m3u")
def make_dir_if_absent(path):
@ -32,19 +27,19 @@ def make_dir_if_absent(path):
def raises_unicode_error(str):
try:
str.encode('latin-1')
str.decode('utf-8').encode('latin-1')
return False
except (UnicodeEncodeError, UnicodeDecodeError):
return True
def hash_error_unicode(item):
item_bytes = item.encode('utf-8')
return "".join(["{0:02X}".format(ord(x)) for x in reversed(hashlib.md5(item_bytes).hexdigest()[:8])])
return "".join(["{0:02X}".format(ord(x)) for x in reversed(hashlib.md5(item).digest()[:8])])
pass
def validate_unicode(path):
path_list = path.split('/')
last_raise = False
for i in range(len(path_list)):
for i in xrange(len(path_list)):
if raises_unicode_error(path_list[i]):
path_list[i] = hash_error_unicode(path_list[i])
last_raise = True
@ -56,78 +51,29 @@ def validate_unicode(path):
def exec_exists_in_path(command):
with open(os.devnull, 'w') as FNULL:
try:
with open(os.devnull, 'r') as RFNULL:
subprocess.call([command], stdout=FNULL, stderr=subprocess.STDOUT, stdin=RFNULL)
return True
subprocess.call([command], stdout=FNULL, stderr=subprocess.STDOUT)
return True
except OSError as e:
return False
def splitpath(path):
return path.split(os.sep)
def get_relpath(path, basepath):
commonprefix = os.sep.join(os.path.commonprefix(list(map(splitpath, [path, basepath]))))
return os.path.relpath(path, commonprefix)
def is_path_prefix(prefix, path):
return prefix == os.sep.join(os.path.commonprefix(list(map(splitpath, [prefix, path]))))
def group_tracks_by_id3_template(tracks, template):
grouped_tracks_dict = {}
template_vars = set(re.findall(r'{.*?}', template))
for track in tracks:
try:
id3_dict = mutagen.File(track, easy=True)
except:
id3_dict = {}
key = template
single_var_present = False
for var in template_vars:
val = id3_dict.get(var[1:-1], [''])[0]
if len(val) > 0:
single_var_present = True
key = key.replace(var, val)
if single_var_present:
if key not in grouped_tracks_dict:
grouped_tracks_dict[key] = []
grouped_tracks_dict[key].append(track)
return sorted(grouped_tracks_dict.items())
class Text2Speech(object):
valid_tts = {'pico2wave': True, 'RHVoice': True, 'espeak': True, 'say': True}
valid_tts = {'pico2wave': True, 'RHVoice': True}
@staticmethod
def check_support():
voiceoverAvailable = False
# Check for macOS say voiceover
if not exec_exists_in_path("say"):
Text2Speech.valid_tts['say'] = False
print("Warning: macOS say not found, voicever won't be generated using it.")
else:
voiceoverAvailable = True
# Check for pico2wave voiceover
if not exec_exists_in_path("pico2wave"):
Text2Speech.valid_tts['pico2wave'] = False
print("Warning: pico2wave not found, voicever won't be generated using it.")
else:
voiceoverAvailable = True
# Check for espeak voiceover
if not exec_exists_in_path("espeak"):
Text2Speech.valid_tts['espeak'] = False
print("Warning: espeak not found, voicever won't be generated using it.")
print "Error executing pico2wave, voicever won't be generated using it."
else:
voiceoverAvailable = True
# Check for Russian RHVoice voiceover
if not exec_exists_in_path("RHVoice"):
Text2Speech.valid_tts['RHVoice'] = False
print("Warning: RHVoice not found, Russian voicever won't be generated.")
print "Warning: Error executing RHVoice, Russian voicever won't be generated."
else:
voiceoverAvailable = True
@ -140,30 +86,23 @@ class Text2Speech(object):
# Skip voiceover generation if a track with the same name is used.
# This might happen with "Track001" or "01. Intro" names for example.
if os.path.isfile(out_wav_path):
verboseprint("Using existing", out_wav_path)
print "Using existing", out_wav_path
return True
# ensure we deal with unicode later
if not isinstance(text, str):
text = str(text, 'utf-8')
if not isinstance(text, unicode):
text = unicode(text, 'utf-8')
lang = Text2Speech.guess_lang(text)
if lang == "ru-RU":
return Text2Speech.rhvoice(out_wav_path, text)
else:
if Text2Speech.pico2wave(out_wav_path, text):
return True
elif Text2Speech.espeak(out_wav_path, text):
return True
elif Text2Speech.say(out_wav_path, text):
return True
else:
return False
return Text2Speech.pico2wave(out_wav_path, text)
# guess-language seems like an overkill for now
@staticmethod
def guess_lang(unicodetext):
lang = 'en-GB'
if re.search("[А-Яа-я]", unicodetext) is not None:
if re.search(u"[А-Яа-я]", unicodetext) is not None:
lang = 'ru-RU'
return lang
@ -171,21 +110,7 @@ class Text2Speech(object):
def pico2wave(out_wav_path, unicodetext):
if not Text2Speech.valid_tts['pico2wave']:
return False
subprocess.call(["pico2wave", "-l", "en-GB", "-w", out_wav_path, '--', unicodetext])
return True
@staticmethod
def say(out_wav_path, unicodetext):
if not Text2Speech.valid_tts['say']:
return False
subprocess.call(["say", "-o", out_wav_path, '--data-format=LEI16', '--file-format=WAVE', '--', unicodetext])
return True
@staticmethod
def espeak(out_wav_path, unicodetext):
if not Text2Speech.valid_tts['espeak']:
return False
subprocess.call(["espeak", "-v", "english_rp", "-s", "150", "-w", out_wav_path, '--', unicodetext])
subprocess.call(["pico2wave", "-l", "en-GB", "-w", out_wav_path, unicodetext])
return True
@staticmethod
@ -211,13 +136,12 @@ class Record(object):
self.parent = parent
self._struct = collections.OrderedDict([])
self._fields = {}
self.track_voiceover = parent.track_voiceover
self.playlist_voiceover = parent.playlist_voiceover
self.voiceover = parent.voiceover
self.rename = parent.rename
self.trackgain = parent.trackgain
def __getitem__(self, item):
if item not in list(self._struct.keys()):
if item not in self._struct.keys():
raise KeyError
return self._fields.get(item, self._struct[item][1])
@ -225,16 +149,18 @@ class Record(object):
self._fields[item] = value
def construct(self):
output = bytes()
for i in list(self._struct.keys()):
output = ""
for i in self._struct.keys():
(fmt, default) = self._struct[i]
if fmt == "4s":
fmt, default = "I", int(binascii.hexlify(default), 16)
output += struct.pack("<" + fmt, self._fields.get(i, default))
return output
def text_to_speech(self, text, dbid, playlist = False):
if self.track_voiceover and not playlist or self.playlist_voiceover and playlist:
if self.voiceover:
# Create the voiceover wav file
fn = ''.join(format(x, '02x') for x in reversed(dbid))
fn = "".join(["{0:02X}".format(ord(x)) for x in reversed(dbid)])
path = os.path.join(self.base, "iPod_Control", "Speakable", "Tracks" if not playlist else "Playlists", fn + ".wav")
return Text2Speech.text2speech(path, text)
return False
@ -260,7 +186,7 @@ class Record(object):
@property
def base(self):
return self.shuffledb.path
return self.shuffledb.base
@property
def tracks(self):
@ -284,19 +210,19 @@ class TunesSD(Record):
self.track_header = TrackHeader(self)
self.play_header = PlaylistHeader(self)
self._struct = collections.OrderedDict([
("header_id", ("4s", b"bdhs")), # shdb
("header_id", ("4s", "shdb")),
("unknown1", ("I", 0x02000003)),
("total_length", ("I", 64)),
("total_number_of_tracks", ("I", 0)),
("total_number_of_playlists", ("I", 0)),
("unknown2", ("Q", 0)),
("max_volume", ("B", 0)),
("voiceover_enabled", ("B", int(self.track_voiceover))),
("voiceover_enabled", ("B", int(self.voiceover))),
("unknown3", ("H", 0)),
("total_tracks_without_podcasts", ("I", 0)),
("track_header_offset", ("I", 64)),
("playlist_header_offset", ("I", 0)),
("unknown4", ("20s", b"\x00" * 20)),
("unknown4", ("20s", "\x00" * 20)),
])
def construct(self):
@ -321,7 +247,7 @@ class TrackHeader(Record):
self.base_offset = 0
Record.__init__(self, parent)
self._struct = collections.OrderedDict([
("header_id", ("4s", b"hths")), # shth
("header_id", ("4s", "shth")),
("total_length", ("I", 0)),
("number_of_tracks", ("I", 0)),
("unknown1", ("Q", 0)),
@ -333,10 +259,10 @@ class TrackHeader(Record):
output = Record.construct(self)
# Construct the underlying tracks
track_chunk = bytes()
track_chunk = ""
for i in self.tracks:
track = Track(self)
verboseprint("[*] Adding track", i)
print "[*] Adding track", i
track.populate(i)
output += struct.pack("I", self.base_offset + self["total_length"] + len(track_chunk))
track_chunk += track.construct()
@ -347,13 +273,13 @@ class Track(Record):
def __init__(self, parent):
Record.__init__(self, parent)
self._struct = collections.OrderedDict([
("header_id", ("4s", b"rths")), # shtr
("header_id", ("4s", "shtr")),
("header_length", ("I", 0x174)),
("start_at_pos_ms", ("I", 0)),
("stop_at_pos_ms", ("I", 0)),
("volume_gain", ("I", int(self.trackgain))),
("filetype", ("I", 1)),
("filename", ("256s", b"\x00" * 256)),
("filename", ("256s", "\x00" * 256)),
("bookmark", ("I", 0)),
("dontskip", ("B", 1)),
("remember", ("B", 0)),
@ -371,49 +297,42 @@ class Track(Record):
("unknown4", ("Q", 0)),
("dbid", ("8s", 0)),
("artistid", ("I", 0)),
("unknown5", ("32s", b"\x00" * 32)),
("unknown5", ("32s", "\x00" * 32)),
])
def populate(self, filename):
self["filename"] = self.path_to_ipod(filename).encode('utf-8')
self["filename"] = self.path_to_ipod(filename)
if os.path.splitext(filename)[1].lower() in (".m4a", ".m4b", ".m4p", ".aa"):
self["filetype"] = 2
text = os.path.splitext(os.path.basename(filename))[0]
audio = mutagen.File(filename, easy = True)
if audio:
# Note: Rythmbox IPod plugin sets this value always 0.
self["stop_at_pos_ms"] = int(audio.info.length * 1000)
# Try to get album and artist information with mutagen
if mutagen:
audio = None
try:
audio = mutagen.File(filename, easy = True)
except:
print("Error calling mutagen. Possible invalid filename/ID3Tags (hyphen in filename?)")
if audio:
# Note: Rythmbox IPod plugin sets this value always 0.
self["stop_at_pos_ms"] = int(audio.info.length * 1000)
artist = audio.get("artist", [u"Unknown"])[0]
if artist in self.artists:
self["artistid"] = self.artists.index(artist)
else:
self["artistid"] = len(self.artists)
self.artists.append(artist)
artist = audio.get("artist", ["Unknown"])[0]
if artist in self.artists:
self["artistid"] = self.artists.index(artist)
else:
self["artistid"] = len(self.artists)
self.artists.append(artist)
album = audio.get("album", [u"Unknown"])[0]
if album in self.albums:
self["albumid"] = self.albums.index(album)
else:
self["albumid"] = len(self.albums)
self.albums.append(album)
album = audio.get("album", ["Unknown"])[0]
if album in self.albums:
self["albumid"] = self.albums.index(album)
else:
self["albumid"] = len(self.albums)
self.albums.append(album)
if audio.get("title", "") and audio.get("artist", ""):
text = " - ".join(audio.get("title", "") + audio.get("artist", ""))
if audio.get("title", "") and audio.get("artist", ""):
text = u" - ".join(audio.get("title", u"") + audio.get("artist", u""))
# Handle the VoiceOverData
if isinstance(text, str):
if isinstance(text, unicode):
text = text.encode('utf-8', 'ignore')
self["dbid"] = hashlib.md5(text).digest()[:8]
self["dbid"] = hashlib.md5(text).digest()[:8] #pylint: disable-msg=E1101
self.text_to_speech(text, self["dbid"])
class PlaylistHeader(Record):
@ -421,19 +340,19 @@ class PlaylistHeader(Record):
self.base_offset = 0
Record.__init__(self, parent)
self._struct = collections.OrderedDict([
("header_id", ("4s", b"hphs")), #shph
("header_id", ("4s", "shph")),
("total_length", ("I", 0)),
("number_of_playlists", ("I", 0)),
("number_of_non_podcast_lists", ("2s", b"\xFF\xFF")),
("number_of_master_lists", ("2s", b"\x01\x00")),
("number_of_non_audiobook_lists", ("2s", b"\xFF\xFF")),
("unknown2", ("2s", b"\x00" * 2)),
("number_of_non_podcast_lists", ("2s", "\xFF\xFF")),
("number_of_master_lists", ("2s", "\x01\x00")),
("number_of_non_audiobook_lists", ("2s", "\xFF\xFF")),
("unknown2", ("2s", "\x00" * 2)),
])
def construct(self, tracks):
def construct(self, tracks): #pylint: disable-msg=W0221
# Build the master list
masterlist = Playlist(self)
verboseprint("[+] Adding master playlist")
print "[+] Adding master playlist"
masterlist.set_master(tracks)
chunks = [masterlist.construct(tracks)]
@ -441,14 +360,14 @@ class PlaylistHeader(Record):
playlistcount = 1
for i in self.lists:
playlist = Playlist(self)
verboseprint("[+] Adding playlist", (i[0] if type(i) == type(()) else i))
print "[+] Adding playlist", i
playlist.populate(i)
construction = playlist.construct(tracks)
if playlist["number_of_songs"] > 0:
playlistcount += 1
chunks += [construction]
else:
print("Error: Playlist does not contain a single track. Skipping playlist.")
print "Error: Playlist does not contain a single track. Skipping playlist."
self["number_of_playlists"] = playlistcount
self["total_length"] = 0x14 + (self["number_of_playlists"] * 4)
@ -461,27 +380,27 @@ class PlaylistHeader(Record):
output += struct.pack("I", offset)
offset += len(chunks[i])
return output + b"".join(chunks)
return output + "".join(chunks)
class Playlist(Record):
def __init__(self, parent):
self.listtracks = []
Record.__init__(self, parent)
self._struct = collections.OrderedDict([
("header_id", ("4s", b"lphs")), # shpl
("header_id", ("4s", "shpl")),
("total_length", ("I", 0)),
("number_of_songs", ("I", 0)),
("number_of_nonaudio", ("I", 0)),
("dbid", ("8s", b"\x00" * 8)),
("dbid", ("8s", "\x00" * 8)),
("listtype", ("I", 2)),
("unknown1", ("16s", b"\x00" * 16))
("unknown1", ("16s", "\x00" * 16))
])
def set_master(self, tracks):
# By default use "All Songs" builtin voiceover (dbid all zero)
# Else generate alternative "All Songs" to fit the speaker voice of other playlists
if self.playlist_voiceover and (Text2Speech.valid_tts['pico2wave'] or Text2Speech.valid_tts['espeak'] or Text2Speech.valid_tts['say']):
self["dbid"] = hashlib.md5(b"masterlist").digest()[:8]
if self.voiceover and Text2Speech.valid_tts['pico2wave']:
self["dbid"] = hashlib.md5("masterlist").digest()[:8] #pylint: disable-msg=E1101
self.text_to_speech("All songs", self["dbid"], True)
self["listtype"] = 1
self.listtracks = tracks
@ -502,7 +421,7 @@ class Playlist(Record):
dataarr = i.strip().split("=", 1)
if dataarr[0].lower().startswith("file"):
num = int(dataarr[0][4:])
filename = urllib.parse.unquote(dataarr[1]).strip()
filename = urllib.unquote(dataarr[1]).strip()
if filename.lower().startswith('file://'):
filename = filename[7:]
if self.rename:
@ -511,70 +430,39 @@ class Playlist(Record):
listtracks = [ x for (_, x) in sorted(sorttracks) ]
return listtracks
def populate_directory(self, playlistpath, recursive = True):
# Add all tracks inside the folder and its subfolders recursively.
# Folders containing no music and only a single Album
# would generate duplicated playlists. That is intended and "wont fix".
# Empty folders (inside the music path) will generate an error -> "wont fix".
listtracks = []
for (dirpath, dirnames, filenames) in os.walk(playlistpath):
dirnames.sort()
# Ignore any hidden directories
if "/." not in dirpath:
for filename in sorted(filenames, key = lambda x: x.lower()):
# Only add valid music files to playlist
if os.path.splitext(filename)[1].lower() in (".mp3", ".m4a", ".m4b", ".m4p", ".aa", ".wav"):
fullPath = os.path.abspath(os.path.join(dirpath, filename))
listtracks.append(fullPath)
if not recursive:
break
return listtracks
def remove_relatives(self, relative, filename):
base = os.path.dirname(os.path.abspath(filename))
if not os.path.exists(relative):
relative = os.path.join(base, relative)
fullPath = relative
ipodpath = self.parent.parent.parent.path
relPath = fullPath[fullPath.index(ipodpath)+len(ipodpath)+1:].lower()
fullPath = os.path.abspath(os.path.join(ipodpath, relPath))
return fullPath
def populate(self, obj):
# Create a playlist of the folder and all subfolders
if type(obj) == type(()):
self.listtracks = obj[1]
text = obj[0]
else:
filename = obj
if os.path.isdir(filename):
self.listtracks = self.populate_directory(filename)
text = os.path.splitext(os.path.basename(filename))[0]
else:
# Read the playlist file
with open(filename, 'r', errors="replace") as f:
data = f.readlines()
def populate(self, filename):
with open(filename, 'rb') as f:
data = f.readlines()
extension = os.path.splitext(filename)[1].lower()
if extension == '.pls':
self.listtracks = self.populate_pls(data)
elif extension == '.m3u':
self.listtracks = self.populate_m3u(data)
else:
raise
# Ensure all paths are not relative to the playlist file
for i in range(len(self.listtracks)):
self.listtracks[i] = self.remove_relatives(self.listtracks[i], filename)
text = os.path.splitext(os.path.basename(filename))[0]
extension = os.path.splitext(filename)[1].lower()
if extension == '.pls':
self.listtracks = self.populate_pls(data)
elif extension == '.m3u':
self.listtracks = self.populate_m3u(data)
# Ensure all paths are not relative to the playlist file
for i in range(len(self.listtracks)):
self.listtracks[i] = self.remove_relatives(self.listtracks[i], filename)
# Handle the VoiceOverData
self["dbid"] = hashlib.md5(text.encode('utf-8')).digest()[:8]
text = os.path.splitext(os.path.basename(filename))[0]
self["dbid"] = hashlib.md5(text).digest()[:8] #pylint: disable-msg=E1101
self.text_to_speech(text, self["dbid"], True)
def construct(self, tracks):
def construct(self, tracks): #pylint: disable-msg=W0221
self["total_length"] = 44 + (4 * len(self.listtracks))
self["number_of_songs"] = 0
chunks = bytes()
chunks = ""
for i in self.listtracks:
path = self.ipod_to_path(i)
position = -1
@ -583,8 +471,8 @@ class Playlist(Record):
except:
# Print an error if no track was found.
# Empty playlists are handeled in the PlaylistHeader class.
print("Error: Could not find track \"" + path + "\".")
print("Maybe its an invalid FAT filesystem name. Please fix your playlist. Skipping track.")
print "Error: Could not find track \"" + path + "\"."
print "Maybe its an invalid FAT filesystem name. Please fix your playlist. Skipping track."
if position > -1:
chunks += struct.pack("I", position)
self["number_of_songs"] += 1
@ -594,19 +482,16 @@ class Playlist(Record):
return output + chunks
class Shuffler(object):
def __init__(self, path, track_voiceover=False, playlist_voiceover=False, rename=False, trackgain=0, auto_dir_playlists=None, auto_id3_playlists=None):
self.path = os.path.abspath(path)
def __init__(self, path, voiceover=True, rename=False, trackgain=0):
self.path, self.base = self.determine_base(path)
self.tracks = []
self.albums = []
self.artists = []
self.lists = []
self.tunessd = None
self.track_voiceover = track_voiceover
self.playlist_voiceover = playlist_voiceover
self.voiceover = voiceover
self.rename = rename
self.trackgain = trackgain
self.auto_dir_playlists = auto_dir_playlists
self.auto_id3_playlists = auto_id3_playlists
def initialize(self):
# remove existing voiceover files (they are either useless or will be overwritten anyway)
@ -616,59 +501,36 @@ class Shuffler(object):
make_dir_if_absent(os.path.join(self.path, dirname))
def dump_state(self):
print("Shuffle DB state")
print("Tracks", self.tracks)
print("Albums", self.albums)
print("Artists", self.artists)
print("Playlists", self.lists)
print "Shuffle DB state"
print "Tracks", self.tracks
print "Albums", self.albums
print "Artists", self.artists
print "Playlists", self.lists
def determine_base(self, path):
base = os.path.abspath(path)
# while not os.path.ismount(base):
# base = os.path.dirname(base)
return base, base
def populate(self):
self.tunessd = TunesSD(self)
for (dirpath, dirnames, filenames) in os.walk(self.path):
dirnames.sort()
relpath = get_relpath(dirpath, self.path)
# Ignore the speakable directory and any hidden directories
if not is_path_prefix("iPod_Control/Speakable", relpath) and "/." not in dirpath:
if "ipod_control/speakable" not in dirpath.lower() and "/." not in dirpath.lower():
for filename in sorted(filenames, key = lambda x: x.lower()):
# Ignore hidden files
if not filename.startswith("."):
fullPath = os.path.abspath(os.path.join(dirpath, filename))
if os.path.splitext(filename)[1].lower() in (".mp3", ".m4a", ".m4b", ".m4p", ".aa", ".wav"):
self.tracks.append(fullPath)
if os.path.splitext(filename)[1].lower() in (".pls", ".m3u"):
self.lists.append(fullPath)
# Create automatic playlists in music directory.
# Ignore the (music) root and any hidden directories.
if self.auto_dir_playlists and "iPod_Control/Music/" in dirpath and "/." not in dirpath:
# Only go to a specific depth. -1 is unlimted, 0 is ignored as there is already a master playlist.
depth = dirpath[len(self.path) + len(os.path.sep):].count(os.path.sep) - 1
if self.auto_dir_playlists < 0 or depth <= self.auto_dir_playlists:
self.lists.append(os.path.abspath(dirpath))
if self.auto_id3_playlists != None:
if mutagen:
for grouped_list in group_tracks_by_id3_template(self.tracks, self.auto_id3_playlists):
self.lists.append(grouped_list)
else:
print("Error: No mutagen found. Cannot generate auto-id3-playlists.")
sys.exit(1)
fullPath = os.path.abspath(os.path.join(dirpath, filename))
relPath = fullPath[fullPath.index(self.path)+len(self.path)+1:].lower()
fullPath = os.path.abspath(os.path.join(self.path, relPath));
if os.path.splitext(filename)[1].lower() in (".mp3", ".m4a", ".m4b", ".m4p", ".aa", ".wav"):
self.tracks.append(fullPath)
if os.path.splitext(filename)[1].lower() in (".pls", ".m3u"):
self.lists.append(os.path.abspath(os.path.join(dirpath, filename)))
def write_database(self):
print("Writing database. This may take a while...")
with open(os.path.join(self.path, "iPod_Control", "iTunes", "iTunesSD"), "wb") as f:
try:
f.write(self.tunessd.construct())
except IOError as e:
print("I/O error({0}): {1}".format(e.errno, e.strerror))
print("Error: Writing iPod database failed.")
sys.exit(1)
print("Database written successfully:")
print("Tracks", len(self.tracks))
print("Albums", len(self.albums))
print("Artists", len(self.artists))
print("Playlists", len(self.lists))
with open(os.path.join(self.base, "iPod_Control", "iTunes", "iTunesSD"), "wb") as f:
f.write(self.tunessd.construct())
#
# Read all files from the directory
@ -687,7 +549,7 @@ def check_unicode(path):
if raises_unicode_error(item):
src = os.path.join(path, item)
dest = os.path.join(path, hash_error_unicode(item)) + os.path.splitext(item)[1].lower()
print('Renaming %s -> %s' % (src, dest))
print 'Renaming %s -> %s' % (src, dest)
os.rename(src, dest)
else:
ret_flag = (check_unicode(os.path.join(path, item)) or ret_flag)
@ -695,7 +557,7 @@ def check_unicode(path):
src = os.path.join(path, item)
new_name = hash_error_unicode(item)
dest = os.path.join(path, new_name)
print('Renaming %s -> %s' % (src, dest))
print 'Renaming %s -> %s' % (src, dest)
os.rename(src, dest)
return ret_flag
@ -711,86 +573,36 @@ def nonnegative_int(string):
def checkPathValidity(path):
if not os.path.isdir(result.path):
print("Error finding IPod directory. Maybe it is not connected or mounted?")
print "Error finding IPod directory. Maybe it is not connected or mounted?"
sys.exit(1)
if not os.access(result.path, os.W_OK):
print('Unable to get write permissions in the IPod directory')
print 'Unable to get write permissions in the IPod directory'
sys.exit(1)
def handle_interrupt(signal, frame):
print("Interrupt detected, exiting...")
print "Interrupt detected, exiting..."
sys.exit(1)
if __name__ == '__main__':
signal.signal(signal.SIGINT, handle_interrupt)
parser = argparse.ArgumentParser(description=
'Python script for building the Track and Playlist database '
'for the newer gen IPod Shuffle. Version 1.5')
parser.add_argument('-t', '--track-voiceover', action='store_true',
help='Enable track voiceover feature')
parser.add_argument('-p', '--playlist-voiceover', action='store_true',
help='Enable playlist voiceover feature')
parser.add_argument('-u', '--rename-unicode', action='store_true',
help='Rename files causing unicode errors, will do minimal required renaming')
parser.add_argument('-g', '--track-gain', type=nonnegative_int, default='0',
help='Specify volume gain (0-99) for all tracks; '
'0 (default) means no gain and is usually fine; '
'e.g. 60 is very loud even on minimal player volume')
parser.add_argument('-d', '--auto-dir-playlists', type=int, default=None, const=-1, nargs='?',
help='Generate automatic playlists for each folder recursively inside '
'"IPod_Control/Music/". You can optionally limit the depth: '
'0=root, 1=artist, 2=album, n=subfoldername, default=-1 (No Limit).')
parser.add_argument('-i', '--auto-id3-playlists', type=str, default=None, metavar='ID3_TEMPLATE', const='{artist}', nargs='?',
help='Generate automatic playlists based on the id3 tags of any music '
'added to the iPod. You can optionally specify a template string '
'based on which id3 tags are used to generate playlists. For eg. '
'\'{artist} - {album}\' will use the pair of artist and album to group '
'tracks under one playlist. Similarly \'{genre}\' will group tracks based '
'on their genre tag. Default template used is \'{artist}\'')
parser.add_argument('-v', '--verbose', action='store_true',
help='Show verbose output of database generation.')
parser = argparse.ArgumentParser()
parser.add_argument('--disable-voiceover', action='store_true', help='Disable voiceover feature')
parser.add_argument('--rename-unicode', action='store_true', help='Rename files causing unicode errors, will do minimal required renaming')
parser.add_argument('--track-gain', type=nonnegative_int, default=0, help='Specify volume gain (0-99) for all tracks; 0 (default) means no gain and is usually fine; e.g. 60 is very loud even on minimal player volume')
parser.add_argument('path', help='Path to the IPod\'s root directory')
result = parser.parse_args()
# Enable verbose printing if desired
verboseprint = print if result.verbose else lambda *a, **k: None
checkPathValidity(result.path)
if result.rename_unicode:
check_unicode(result.path)
if not mutagen:
print("Warning: No mutagen found. Database will not contain any album nor artist information.")
if not result.disable_voiceover and not Text2Speech.check_support():
print "Error: Did not find any voiceover program. Voiceover disabled."
result.disable_voiceover = True
verboseprint("Playlist voiceover requested:", result.playlist_voiceover)
verboseprint("Track voiceover requested:", result.track_voiceover)
if (result.track_voiceover or result.playlist_voiceover):
if not Text2Speech.check_support():
print("Error: Did not find any voiceover program. Voiceover disabled.")
result.track_voiceover = False
result.playlist_voiceover = False
else:
verboseprint("Voiceover available.")
shuffle = Shuffler(result.path,
track_voiceover=result.track_voiceover,
playlist_voiceover=result.playlist_voiceover,
rename=result.rename_unicode,
trackgain=result.track_gain,
auto_dir_playlists=result.auto_dir_playlists,
auto_id3_playlists=result.auto_id3_playlists)
shuffle = Shuffler(result.path, voiceover=not result.disable_voiceover, rename=result.rename_unicode, trackgain=result.track_gain)
shuffle.initialize()
shuffle.populate()
shuffle.write_database()