Saturday, October 3, 2015

Abstraction for Special folders

I have always tried to write my code in as platform independent as possible. Generally you can get a long way libraries like SDL or SFML for instance allows me to write a game without having to worry about if I am targeting Windows or Linux. There are different choices for GUI's as well. Boost and PhysFS allows me to create files and folders without having to write platform specific code.

One thing that did bother me was the special folders of the different platforms. On Windows programs are expecting to store config data in %APPDATA%/programname while in Linux programs are expecting to store in $(HOME)/.local/share/programname by default.
Mac OS has it's own way of finding the folders.

Of course program data is not the only thing that requires special attention:
  • My Documents
  • My Pictures
  • My Videos
  • Save Games
  • Templates
  • Music

One of the annoying things about this is that my code started to have "#ifdef __WIN32" in it. This makes the code harder to read and if I am developing on Linux I might not discover that I have broken the build until I compile on Windows. I learned to isolate such code so that it did not affect my normal flow but it still bugged me every time I had to make a small change.
Therefore I decided to write a small library for it.

The library

My plan was simple:
  • It should be a C++ library because that is what I use for my programs.
  • It should be self contained so that you can compile it into the program without having to worry about ABI
  • On Windows it should use the Windows API.
  • On Linux it should use the XDG basedir specification and the Xdg user dir specification
  • On Mac it should use the FSFindFolder
  • It should be usable under C++03 and C++11

It should be limited to the folder one would expect to find on any desktop system.

I decided on the following folders to support:

Program save data related folders:
  • Program configuration - for human readable configuration
  • Program data - for non human readable configuration
  • Program cache - for data that does not need to be backed up

User folders:
  • Documents
  • Desktop
  • Pictures
  • Music
  • Video
  • Download
  • Save games

I knew that not all folders was well defined on all architectures although I got some additional surprises.
I also early decided that I some folders might be the same on some architectures. 

The Linux part

For Linux I decided to follow the XDG specification both for storing program data and finding the user folders. I decided to omit certain folders that was only relevant to a *nix system like XDG_RUNTIME_DIR. Also the Xdg user directories does not specify a Save Games directory. However I believe that it is acceptable for a Linux program to store user data in XDG_DATA_HOME because there is a tradition for the hidden folders to contain important data unlike Windows where you expect to be able to delete %APPDATA% without major damage.

The Windows part

My approach to Windows was a bit different. I really wanted to be able to cross compile using mingw32 compiler. I also only wanted to support versions of Windows that was still official supported at the time it meant Windows Vista or later (and I had not read up on it so I targeted Windows XP and later). However the Windows API tends to use the nasty std::wstring for storing file paths and in code you normally want to store everything in std::string. Many cross platform does not use wstring and I have no idea how I would cross compile to it.

For that reason I ended up using SHGetFolderPathA instead of SHGetKnownFolderPath. I am not too proud of this but Microsoft could have added UTF-8 support to Windows like any other system. Given the popularity of the Internet it is unlikely that UTF-16 will prevail.
This means that the "Save Games" folder and "Downloads" folder was unavailable. 

The Mac OS part

I tried adding Mac OS support but I could not figure how to link the right library for FSFindFolder so I dropped it.

The individual folders

Some folders required mapping

Program configuration

On Windows this is just %APPDATA% on Linux it is XDG_CONFIG_HOME (default $HOME/.config). This is for data that needs to be backed up.

Program data

On Windows this is also %APPDATA as Windows does not normally have human readable config files.
On Linux it is XDG_DATA_HOME.
This is also for data that should be backed up.

Program cache

On Windows this is %LOCALAPPDATA% because that directory is not part of the Roaming profile and must only contain expendable files. 
On Linux this is XDG_CACHE_HOME (default $HOME/.cache). This matches the common backup procedure of skipping folders named ".cache".


Because I use an old pre-Vista API I cannot return the right folder. On Windows this will return the Desktop.
On Linux it will return the right folder as specified by Xdg user directories.

Save Games

Because I use an old pre-Vista API I cannot return the right folder. On Windows I'll return a path to "My Games" under the Documents folder. This was a common pre Vista location. It is not good. It does not follow the users locale but it is a place the user expect to find save games. I deliberately did not return a path to %APPDATA%. Because %APPDATA% is misused by many programs it has become dangarous to store any user files under %APPDATA% as it is normally excluded from backup procedures. 
On Linux I return XDG_DATA_HOME as there are no Save Games folder defined in XDG. Even though it holds user files the folder is generally easier to access under Linux and will normally be included in backup procedures.

Result and future

The library can be found here:
It is MIT licensed to allow inclusion in any program without having to worry about legal issues. 
I really want to find a solution to the UTF-16 Windows problem.