Not appropriate!
This guide is intended to document a media setup that has worked for me, and how I came to this setup. Some background information: this setup is largely Apple centric, although not exclusively Apple products. Locally hosted content is preferred to cloud based content. By 'media' in this guide I refer to not only music and movie/tv content, but books, podcasts, videos of conference sessions, etc.
By library organization I mean keeping everything together, organized in a directory hierarchy that hopefully makes some sense, and consistently named. I currently use iTunes with "Keep iTunes Media folder organized" and "Copy files to iTunes Media folder when adding to library" options set, for the central library. iTunes is a big program that does a lot of things, but for this section I'll focus on the organizational aspects.
With the above options checked, it will organize files as:
iTunes --iTunes Media ----Books ------All books in one flat directory here ----Movies ------Directory of movie name --------Movie file ----Music ------Directory of artist name --------Directory of album name ----------Song files from the album, named "track# track name" ----TV Shows ------Directory of show name --------Directory for season number ----------Video files for the season, named "season#-episode# episode name" ----iTunes U ------Directory of 'course', such as "WWDC 2012" --------Flag listing of all files (video, pdf, etc.) associated with 'course'
iTunes (with the above options checked) will rename the files automatically into this directory hierarchy based on the file's metadata. mp3's based on the ID3 tag information, and mp4/m4v files based on the mp4 metadata. This is convenient since source files frequently have random names, named inconsistently depending on source, and/or have irrelevant garbage appended to them.
There are a lot of drawbacks to the iTunes organization system, and the books section is particularly weak. There have been minor improvements to the book organizing and managing aspects of iTunes in the more recent releases, but it still largely comes down to gigantic uncategorized lists. The directory is still flat, and when syncing the books to a device, you're presented with a flat list of all your books, needing to check each one individually.
The movie hierarchy with a directory per movie title is a bit unhelpful, since virtually every case will only have one file in the directory. You can end up with multiple files in the directory if you have SD, HD, and HD 1080p media types of the same file, or just plain duplicates. In general it's best to avoid that situation anyway since iTunes' handling of multiple resolutions or encodings of the same movie type is very poor to begin with. So, you're left with a flat directory of movies, with a directory per movie, each with a single file in it. Not great, but ends up not being a huge deal. Depending on the client navigation, it can end up being just a single additional click.
There is no doubt one can create a better organization by hand. Particularly with books. iTunes is especially poor when it comes to organizing books. This is how I started out, manually maintaining the directory hierarchy and file naming.
Problems I encountered:
When I reference metadata here, I am refering to the data stored inside the media container, such as ID3 tags and the like. I am not referring to filesystem metadata such as extended attributes or resource forks.
Since iTunes renames everything based on the file's metadata, ensuring correct metadata is important. Even if you're not using iTunes, having correct metadata is helpful for searching and sorting (by rating, genre, director, actor, whatever), and can provide a better user experience (cover art, synopsis, etc).
I currently use iDentify on OSX for both metadata automatic population and editing. It uses The MovieDB and The TVDB for metadata sources. I find it convenient to use and works, for the most part. Drawbacks I've encountered are that lookups on TV shows need to be exact. For instance, The Venture Bros TV show needs to have a show name of "The Venture Bros." with trailing period, or it won't be found. As a result, I frequently end up going to the source sites in a browser, searching for what I want, then going back to iDentify to enter the exact name. The other issue I've run into is occasionally it is difficult to distinguish remakes and originals of movies. They have the same title, and you need to examine the retrieved metadata to ensure you get the right one. Sometimes remakes have a year appended to the title, sometimes not. Sometimes only the remake appears in the database. Despite the drawbacks, it is the most convenient way of populating and editing metadata that I've found.
Another gotcha with metadata is if you're reencoding a file for any reason, most encoders are at best selective of what metadata they preserve, and will frequently just drop important metadata in the output file.
iTunes' notion of parental controls is at best described as "inadequate". My impression is the parental controls are there to satisfy some government or industry group's checklist, more than designed to be actually used by a parent. First and foremost, with parental controls enabled, the existence of all content, including their metadata (specifically coverart and synopsis) is visible.
Appropriate or not, it is a frustrating experience for a child to be flipping through content where the majority of it, they're not allowed to watch. This results in "it doesn't work", or at best the parent needing to do the navigation for the child, which defeats the whole point.
My solution to this was to just use a separate iTunes library on a different machine, configured with its own Home Sharing ID. The adult's library is still canonical; it is a strict superset of the children's library. To keep the two in sync, I usually open the children's iTunes library, change the home sharing ID to the adult's library, at which point you can sync content between the libraries. When done syncing, switch the home sharing ID back to the children's home sharing ID. This way, devices that the children use will be preconfigured for the children's home sharing ID, and they will only ever see content that the adult has deemed appropriate.
Additional iTunes libraries can be run in virtual machines, on windows, on older machines (even iTunes on Leopard supports home sharing and is compatible with the latest iDevices).
Not all devices can play the same content. For instance, the iPad Mini cannot play 1080p content. If you have older devices, the original Apple TV cannot play 1080p content, or even some of the higher bitrate 720p content. iTunes does a terrible job of managing different resolutions and bitrate versions of the same content. The mp4 files have a flag identifying SD, HD (720p), and HD 1080p content. iTunes will differentiate between those three flags within actual iTunes views. However, iDevices such as the AppleTV and the Video app on the iPhone and iPad will present these as duplicate entries. More recent versions of iOS' Video app will not present the content in the UI if it is a higher SD/HD/HD1080p type than is playable on the device. However, this is based on the HD flag information, not on whether the content is actually playable. For instance, the iPad3 can play HD 1080p content, but the Apple TV 3 can play higher bit rate 1080p than the iPad3 is capable of. The iPad3's Video app will gladly present the 1080p content, since it can theoretically play that. However, the bit rate will be too high and the iPad3 will fall over.
Additionally, you can have different device specific resolutions and bitrates in the SD category, since the original iPhone plays much lower resolution video than the original Apple TV. And the iPod Classic has different atom requirements within the mp4 file than any of the other devices.
I addressed this by having separate iTunes libraries. I have the canonical library, which has the highest quality of all the content. This isn't to say it is all the same quality, simply that it contains only one copy of everything, and that one copy is the highest quality one. Then I divide up my devices into categories, with one library per category. All media within a given category's iTunes library is encoded with the same settings.
As noted in earlier topics, separate libraries can be useful for solving some iTunes limitations. The biggest problem with multiple libraries is keeping them all in sync. For libraries that contain subsets of content, swapping the Home Sharing ID to use the same ID, then using iTunes' Home Sharing feature to sync desired content is pretty easy and straightforward for just duplicating content between libraries. However, for wanting different encodings of the same content in different libraries, there are no existing tools to help you.
One of the challenges when syncing the content is iTunes' file naming scheme works against you. It can include (SD), (HD), or (HD 1080p) in the file name, which will no longer be accurate for the reencoded copy. Additionally, encoding engines can drop metadata during the reencoding process, which means you have to go through the whole metadata repopulation process.
I've patched handbrake to preserve some extra metadata, and upstream ffmpeg has already accepted the changes. This should preserve enough metadata to make automated repopulation reasonble. I then use scripts like this one, which use HandBrakeCLI (patched), to reencode with specific settings and take care of the SD/HD/HD1080p naming problems.
Again, the separate libraries can be hosted on virtual machines running OSX or Windows, or running on older hardware shoved in a corner somewhere. Old headless minis make great iTunes servers.
Now that you've got your library all organized and have dealt with all the metadata issues, parental controls, different devices, different encodings, etc, how do you actually access it?
Home Sharing provides the most featureful, easy to use interface I've found. It can be accessed from almost all modern Apple devices including the AppleTV (original and the newer hockeypucks) and iOS devices through the Video app. It provides very nice access to metadata such as cover art, synopsis (including director and actor information), rating, and genre.
Some of the downsides are:
DLNA defines a "standard" way of acessing and streaming media. It uses UPNP for discovery and streams via HTTP. Servers have client "profiles", or configuration information that is specific to a given client or type of client device. This tells the server what kind of content the client is interested in (if it's only capable of music, don't present video, if it can only stream mp4 h264 at a specific resolution, only present matching content to the device). If setup properly, this helps the client experience by only presenting content the client device is capable of playing. It can also help the server, since the server can transcode content on the fly from higher resolutions/bitrates/codecs to something the client can play. Theoretically, this solves the multiple devices supporting different resolutions/bitrates/codecs problem above. On the down side, it can take a lot of tedious configuration to get everything exactly right.
The bright side is there are DLNA clients for almost every platform, including built into network capable TVs, bluray players, android devices, ios, etc. Of course that means lots of different devices, each with their own configuration. Additionally, the user experience of the DLNA client varies substantially, and I have not found any that provide as metadata-rich an experience as Home Sharing.
There are several DLNA servers available of varying quality. I use Serviio. It is written in java and has worked with all clients I've tried, usually without too much headache. I run this as a backup to Home Sharing, and have serviio serve up my iTunes library file hierarchy.
Not using any special media specific protocols to access the library is another entirely feasible option. The quality of experience can vary substantially between the options. Here are some of the options I've explored in this category:
All of the above is under the assumption you're on the same local network as your media library. But what about access on the go?
How do you get new content for your library? This will focus on DRM free, or at least content that can be stripped of its DRM, so you can play it on, or reencode it for, virtually any device. I am not aware of any legitimate online distributors of DRM-free video content, or of any (reasonable) way to strip DRM from the legitimate online distributors, so I'm going to leave those out. Audio content on the other hand, can be purchased DRM free from many places these days including Amazon in MP3 format, and Apple in Apple's AAC format. I tend to purchase my music from Amazon, since they were one of the large earlier distributors of DRM free music.
Of course no media guide would be complete without discussing downloading content, or torrenting. There's a lot of issues here, and I'm only going to touch on a couple. I generally discourage this form of acquisition, mostly because few people really understand the consequences of their actions here. Or put another way, from a practical perspective, what are the dangers of torrenting?
Catching pirates is a profitable business. There are organizations whose entire purpose is to make money catching pirates on behalf of studios. They join in the torrenting and even host trackers. Anyone that connects to them is guilty. The company then sends a DMCA notice to the owner of the IP that connected (your ISP), notifying them of a DMCA violation from that IP and requesting contact information about the alleged infringement, notifying the ISP they need to retain logs of the incident, and forwarding the DMCA violation notice to the owner of the account. ISP policies vary, although most will pass on the account information, and many will suspend service pending an 'investigation'. Repeated complaints usually result in account termination regardless of an legal outcomes. The DMCA notice you recieve will likely say something like "if you wish to settle the claims, log into a certain website with the complaint code, by a certain date, and pay a settlement fee. If you do not settle within the given timeframe, you may be sued, etc." Typically the settlement fee is in the $200-$600 range for each violation. Statistically, you have a chance of blowing the entire thing off and not getting sued. However, the company has up to 7 years to decide to sue you. At any point within 7 years of the alleged infringement, the company can decide they need more money, and sue everyone in their portfolio of infringers. When you get sued, it is a long and protracted operation. First, there will be several legal proceeding establishing juristiction. The company suing you will file the complain in a part of the country that has a legal system conducive to their goals. You must either accept that the court they filed the claim in has juridiction, and fight the claim there. Or travel to that location and stay there, or hire a lawyer to represent you in that location, to fight to transfer jurisdiction to you. Then once the jurisdiction issue is resolved, the actual legal proceedings begin. Typically, juries have ruled in favor of the corporations, and awarded tens of thousands of dollars per violation, plus legal fees.
So, that's the risk you run when downloading things. You can expect to look over your shoulder for the next 7 years, and every time you check the mail, wonder if you'll get that letter.
There are of course torrent blocklists to block connecting to IPs that are known to be used by studios and these litigious organizations, anonymizing proxies and vpn services, etc. It's up to you to decide how much effort you're willing to put into it, and what risk you're willing to accept.
That said, there seem to be plenty of people who are willing to accept that risk. If you do, here's some advice:
The content you download is likely to be in an undesireable form. The name, the container format, and possibly the codec. I recommend you transcode the file to the same container format and codec as the rest of your library. I typically use handbrake for this. Then update the metadata information of the file, I use iDentify as covered in the metadata section covered above. Then add the file to your library, consistent with your directory hierarchy and file naming conventions. iTunes can handle this for you, if you just drag the file to iTunes.
The old tried and true method. Get a DVD and convert it to your desired format. There are a lot of tools out there that are very good, and I typically use HandBrake for this. Any fool can grab video content from physical media, but there's a lot more than just grabbing some video. Handbrake conversion to mp4 container format will handle multiple audio tracks correctly, including commentary tracks. It will also handle multiple subtitle tracks, including forced tracks.
Initially, I would just rip the entire DVD filesystem and keep that, then encode it to whatever device settings I wanted. These days however, the latest h264 settings preserve about as much quality of the original as you're ever going to get, and if you need to downsample for smaller devices, transcoding the higher res h264 copy is more than adequate. Keeping the DVD filesystem doesn't seem to buy much anymore.
A bit of an aside on subtitles. DVD and mp4 subtitle tracks are stored as text along with time based markers to sync them up with the video. The player is then responsible for displaying the subtitles, including choosing font, size, placement, and coloring. DVDs can have "forced" subtitles where, for example, if you're watching with an English language audio track selected, but a character is speaking a non-English language, the title can force subtitles on for when that character is talking, so you can see what they are saying. Forced subtitles are not on for the entire movie, only short pieces where appropriate. Missing the forced subtitles can be most unfortunate, depending on how important they are to a movie.
There's also various forms of copy protection on DVDs. This is of course an arms race, where publishers include copy protection of some sort, and the ripping software eventually includes support for breaking it. Fortunately, whatever the latest and greatest copy protection scheme out there is, costs lots of money to the studio to license the use. So, it's typically only included on movies that are most likely to be copied on a large scale, and that's typically the bubblegum pop movies of the day. If that's not your thing, most likely you have nothing to worry about. And if you do encounter it, usually the title is popular enough someone else has already asked and figured out how to copy it, so a quick websearch will take care of things for you.
BluRay, the high def DVD right? Not quite. One of the most annoying differences between BluRay and DVD is subtitles. Where DVD subtitles are text based, BluRay subtitles are pre-rendered on the disc and overlayed onto the video by the player. This doesn't translate very well to mp4 based subtitles, which are also text, like DVDs. This allowed the content producers to control the appearance of subtitles and provide an overall better experience. They can have fonts that better match the movie rather than whatever random mechanical looking font the player chooses. This also let them choose the location of the subtitles on the screen to prevent them from overlapping with critical details in the picture, and ensuring the color of the subtitles was appropriate for the background they're displayed on. However, this leaves essentially 3 options for subtitles when ripping bluray:
Generally, I opt for #3, and only burn in the forced subtitles. This gives an experience roughly comparable to a movie theater viewing, or viewing the bluray without subtitles on. But, if you frequently toggle subtitles for viewing silently, or in noisy environments, this isn't such a great option.
Ripping software varies substantially in its featureset. HandBrake did not support directly ripping from BluRay last I checked. The Typical process is to use MakeMKV to rip the bluray content directly to an mkv container. No transcoding occurs. Then, use HandBrake to transcode from that ripped container to your desired format. However, the makemkv and handbrake combination don't handle subtitles well.
Instead, I have purchased the bluray ripper from aunsoft. I find it works reasonably well, includes presets for lots of devices, and handles multiple audio tracks, and forced and optional subtitles.
How does one store and backup a media library? If you're storing everything in it, including manuals, text books, conference sessions, tech talks, podcasts, plus all your entertainment media, it can be pretty large.
There are times when you don't want the thing managing or serving your media library to also be the thing your disks are attached to. For instance, if you're using iTunes, you probably don't want to have OSX also manage your disks/array/filesystem. If you are using iTunes to manage your media library, there once was a time when iTunes stored filesystem metadata with some of the files in the library. I do not know if this is still true. However, because of this, my iTunes library lives on an AFP share, which is the remote filesystem in OSX with the best support for metadata.
I use netatalk under Linux to share the filesystem with iTunes. This can be done inside a virtual machine that has a file sharing feature, such as VMWare's products. You can share the host's filesystem with the Linux VM, and share that via AFP. Additionally, since netatalk AFP share locations do not need to correspond to filesystem paths or locations, this level of abstraction allows you to move everything about the iTunes AFP share from machine to machine, mounted in different locations, etc, and keep the same view presented to OSX and iTunes. This presents a nice stable interface for iTunes' library locations.