lørdag den 3. oktober 2015

Abstraction for Special folders

I have always tried to write my code in as platform independent as possible. Generally you can get a long way libraries like SDL or SFML for instance allows me to write a game without having to worry about if I am targeting Windows or Linux. There are different choices for GUI's as well. Boost and PhysFS allows me to create files and folders without having to write platform specific code.

One thing that did bother me was the special folders of the different platforms. On Windows programs are expecting to store config data in %APPDATA%/programname while in Linux programs are expecting to store in $(HOME)/.local/share/programname by default.
Mac OS has it's own way of finding the folders.

Of course program data is not the only thing that requires special attention:
  • My Documents
  • My Pictures
  • My Videos
  • Save Games
  • Templates
  • Music

One of the annoying things about this is that my code started to have "#ifdef __WIN32" in it. This makes the code harder to read and if I am developing on Linux I might not discover that I have broken the build until I compile on Windows. I learned to isolate such code so that it did not affect my normal flow but it still bugged me every time I had to make a small change.
Therefore I decided to write a small library for it.

The library

My plan was simple:
  • It should be a C++ library because that is what I use for my programs.
  • It should be self contained so that you can compile it into the program without having to worry about ABI
  • On Windows it should use the Windows API.
  • On Linux it should use the XDG basedir specification and the Xdg user dir specification
  • On Mac it should use the FSFindFolder
  • It should be usable under C++03 and C++11

It should be limited to the folder one would expect to find on any desktop system.

I decided on the following folders to support:

Program save data related folders:
  • Program configuration - for human readable configuration
  • Program data - for non human readable configuration
  • Program cache - for data that does not need to be backed up

User folders:
  • Documents
  • Desktop
  • Pictures
  • Music
  • Video
  • Download
  • Save games

I knew that not all folders was well defined on all architectures although I got some additional surprises.
I also early decided that I some folders might be the same on some architectures. 

The Linux part

For Linux I decided to follow the XDG specification both for storing program data and finding the user folders. I decided to omit certain folders that was only relevant to a *nix system like XDG_RUNTIME_DIR. Also the Xdg user directories does not specify a Save Games directory. However I believe that it is acceptable for a Linux program to store user data in XDG_DATA_HOME because there is a tradition for the hidden folders to contain important data unlike Windows where you expect to be able to delete %APPDATA% without major damage.

The Windows part

My approach to Windows was a bit different. I really wanted to be able to cross compile using mingw32 compiler. I also only wanted to support versions of Windows that was still official supported at the time it meant Windows Vista or later (and I had not read up on it so I targeted Windows XP and later). However the Windows API tends to use the nasty std::wstring for storing file paths and in code you normally want to store everything in std::string. Many cross platform does not use wstring and I have no idea how I would cross compile to it.

For that reason I ended up using SHGetFolderPathA instead of SHGetKnownFolderPath. I am not too proud of this but Microsoft could have added UTF-8 support to Windows like any other system. Given the popularity of the Internet it is unlikely that UTF-16 will prevail.
This means that the "Save Games" folder and "Downloads" folder was unavailable. 


The Mac OS part

I tried adding Mac OS support but I could not figure how to link the right library for FSFindFolder so I dropped it.


The individual folders

Some folders required mapping


Program configuration

On Windows this is just %APPDATA% on Linux it is XDG_CONFIG_HOME (default $HOME/.config). This is for data that needs to be backed up.


Program data

On Windows this is also %APPDATA as Windows does not normally have human readable config files.
On Linux it is XDG_DATA_HOME.
This is also for data that should be backed up.


Program cache

On Windows this is %LOCALAPPDATA% because that directory is not part of the Roaming profile and must only contain expendable files. 
On Linux this is XDG_CACHE_HOME (default $HOME/.cache). This matches the common backup procedure of skipping folders named ".cache".


Downloads

Because I use an old pre-Vista API I cannot return the right folder. On Windows this will return the Desktop.
On Linux it will return the right folder as specified by Xdg user directories.


Save Games

Because I use an old pre-Vista API I cannot return the right folder. On Windows I'll return a path to "My Games" under the Documents folder. This was a common pre Vista location. It is not good. It does not follow the users locale but it is a place the user expect to find save games. I deliberately did not return a path to %APPDATA%. Because %APPDATA% is misused by many programs it has become dangarous to store any user files under %APPDATA% as it is normally excluded from backup procedures. 
On Linux I return XDG_DATA_HOME as there are no Save Games folder defined in XDG. Even though it holds user files the folder is generally easier to access under Linux and will normally be included in backup procedures.



Result and future


The library can be found here: https://github.com/sago007/PlatformFolders
It is MIT licensed to allow inclusion in any program without having to worry about legal issues. 
I really want to find a solution to the UTF-16 Windows problem.

mandag den 10. december 2012

Ubuntu and the perril of distribution upgrades

Most people would agree that software updates can be extremely annoying. This is especially true if you are a Windows user and have experienced sudden shutdowns or a half hour delay while Adobe Acrobat updated. Updating antivirus, Firewall, Flash, browser, Java and Windows can be a big annoyance. further more the updates are usually enforced at the most inconvenient times. If I have been in the situation that I needed to quickly open a PDF-file in order to print it and then have to wait 40 minutes for the program to update.

Luckily most Linux distributions have a much better way of installing updates. The unattended upgrades in Ubuntu is rarely noticed. They may require restart of Firefox or a reboot of the machine to use the new kernel but it can usually be delayed until it is convenient. This is made possible partly by filesystem that unlike Windows allows you to update a file while in use (the program will just continue to use the old file until restarted) and partly thanks to the brilliant program knowns as Apt.

Apt is a wonderfull thing. apt-get, aptitude and synaptic was the programs that always attracted me to Debian and then Ubuntu made it easily accessible I was not late to make the switch from Debian to Ubuntu. After all time is the limiting resource in my life so allowing me to use more of it is great.

There is however one thing that still does need improvement and that it the distribution upgrade process. It is not that it is worse than for any other operating system. It is better than installing a service pack for instance but it is not as good as it could be. Yesterday I updated an old Ubuntu 11.04 machine to Ubuntu 11.10. I fired up the update program and soon afterwards the program started to get files. The time remaining bar said about two hours. As my time are too valuable to spend two hours looking at a bar moving I want to do better things. After two hours I returned to see that it had stopped after 30 minutes to tell me that it had updated postgreSQL from 8.4 to 9.1 and asked my to click "Ok". Was it absolutely necessary to halt the installation for 90 minutes to ask me to make no choices at all? So I went away to use the next 90 minutes for something productive. When I returned a new popup box had appeared and asked me for the password to phpmyadmin. At least this time the program actually asked for information but was it importent to halt the remaining updates for 30 minutes to ask me that?

Sometimes I just opt for a full reinstall. I have a separate /home so it is not that big a deal. However this is not limited to my desktop but affects my Debian servers too. Servers that I prefer not to take down and reinstall if I can avert it.

It is not that it is bad compared to other operating systems, it just isn't as good as it could be and that annoys me.

mandag den 23. juli 2012

Blogs

Well, here it is: My first ever blog post.

I must admit that despite my technical interest in modern technology there are some forms of communication that I have evaded so far. One of these things are blogs.

For a long time I failed to see the point in blogs. I mean who besides me have the slightest interest in the topics I care about.
However I have often found interesting blog posts while searching for a special topic. And compared to most other forms posting on the internet, a blog allows one to actually go deeper into a topic.

I don't mind lurking on Facebook or Twitter to follow others but I do feel that the they do not contribute anything noteworthy to society. Both are optimised for people that does not really want to get deep into anything particular, although then analysing trends that might be a good thing.

I hope to write some posts about my observations of different developments in technology and some of the choices I sometimes take.

Maybe some of them will actually be interesting or maybe it will be me blabbering about my dislike of iTunes, the cloud, the smart phones or type less programming languages that might all be good but also have some serious flaws or limitations.

This might end up like a developer blog about my work on OpenArena, Block Attack or other projects. It would be fine for me.