Blog: What I learned, forgot and learned again about Java Preferences

I'm not an expert in Java, but I have been using and teaching it for a really long time. One thing I've always struggled with is the right way to manage settings and preferences, especially stacking different levels of settings (defaults, setup time and run time). The various systems I worked on in grad school were massively over complicated (to the extent where I think in my masters’ system, configuration was probably a third of the whole system).

For the Game Tracker I set out to keep things really simple, and ideally to do things "the right way". Given that I don't actually work in software development with anyone other than myself, figuring out "the right way" can be a bit of a struggle sometimes. Still I'm trying to start modelling my own technology learning after Julia Even's approach and learn and focus on what I have learned.

For handling settings, I've used Java's Properties quite a bit (they made up a big portion of my PhD system, which was *much* more streamlined than my masters). Having an API to handle getting simple values in and out of a file is really useful. Preferences are an extra layer on top of that, where you hand over keeping track of the storage to the JVM.

While I'm still a little grouchy about Preferences not being transparent to the user or the programmer. They do feel like "the right way" to keep track of settings. I was especially confused when I cam back to the Game Tracker after a long break and so had to relearn a lot about Preferences. Hopefully this will be a good overview and collection of the good resources I found.

Overview

As I mentioned, preferences let you hand off worrying about where your information gets stored. All you have to do is tell Java that you would like this class’s preferences and then either read or store a value. When you end your program, Java will store everything and the next time you run your program your preferences will be there.

You can recompile or even completely delete and replace your program and your preferences will still be there. If you’ve ever deleted and then reinstalled a program and been surprised that it remembered everything from before, that’s preferences in action. (And that bit where stuff just gets magically remembered is why I’m sometimes uncomfortable with the idea of preferences, sometimes you just want a program to go away, but we’ll talk about that later.)

When we’re talking about preferences, we’re talking about the small pieces of information you need to make your program run. For example, with the game tracker the list of games is too big to go in preferences, but keeping the file name where I'm keeping the list of games makes sense. There’s not a strict limit, but keeping too much data in your preferences isn’t a great idea. Additionally, everything is handled as a string by Java so you want to avoid storing anything complex, like an object, and should keep in mind that you’ll have to handle moving numeric data back and forth.

Using Java Preferences

You access Java preferences through a java.util.prefs.Preferences object. We’ll talk about how you *get* a Preferences object shortly, but once you have an object you can treat it as a dictionary. You have a put(String key, String value) method where you can store a value under a key and a get(String key) method which gives you the last value you stored under key.

If you have other types of data, you have to handle them separately. The good news is that you don’t have to do any parsing, there are putBoolean(), putDouble(), putInt() methods and getBoolean(), getDouble(), getInt() methods (and actually there’s a pair of methods for every primitive Java type).

You get a Preferences object by asking the Preferences system for it. Similar to the way you ask System for System.out, there are static methods on the Preferences class for getting your Preferences object. Unlike System.out where there’s only one object to get back, there’s actually many possible objects.

Each package in your program gets its own preferences, so when you ask Preferences for your object, you need to give it your class so it knows which package’s preferences to give back. The static method Preferences.systemNodeForPackage(Class theClass), takes a class object and so you can statically give it a class like CLI.class. *

That there are actually two kinds of preferences. System Preferences (which is what you get if you use the method above) and User Preferences. System preferences are shared by every user on a computer, but the user preferences are unique for each user. Generally user preferences are the right place to store things unless you have a strong reason why everyone needs to share. To get the user preferences, you’ll use the static method Preferences.userNodeForPackage().

From there you’re pretty good to go. The first time someone runs your program you can store their preferences and there after you can access and update them as necessary. I’m not sure it’s the “right way” but I think you should also offer your users a chance to see what’s stored in preferences and a way to remove them (the Preference class has methods to do both). I still need to implement this in the Game Tracker, but I think transparency is good to keep in mind.

My GameTracker Example

The last time I worked on the Game Tracker I reorganized a lot of the components. At this point ** I’m really only keeping two preferences for Game Tracker; the name of the persistence manager and the  file name of the file which has the game and play session data.

The persistence manager is managed in the CLI class. As the CLI class is starting it figures out which PersistenceManager it needs and then calls a factory to return an instance of that manager.  For every preference, I keep a constant string in the class to keep the key value, and I’ll often also keep a default value as well.

When the CLI starts, it processes the command line arguments (a story for another day) and checks if the option for the persistence manager has been set. If the value has been set, the CLI saves the value in the preferences. Then the CLI consults the preferences for the name of the persistence manager (with the default). This means that the value that’s loaded is either the new value that was just passed in, the preference that was set before or the default value if it’s never been set.

    private static final String PERSISTENCE_MANAGER_PREF = "gametracker.persistence_manager";
    private static final String PERSISTENCE_MANAGER_DEFAULT = "CSV";

if (cl.hasOption("datamanager")) {
  Preferences.userNodeForPackage(CLI.class).
    put(PERSISTENCE_MANAGER_PREF, cl.getOptionValue("datamanager"));
}

persistenceManager = PersistenceManagerFactory.getManager(
  Preferences.userNodeForPackage(CLI.class).get(
    PERSISTENCE_MANAGER_PREF, PERSISTENCE_MANAGER_DEFAULT), args);

Within the CSVPersistenceManager, the only preference kept is the data file name. Like the name of the persistence manager in the CLI, I keep the name of the preference and the default value as constants, although here the preference name actually sits in the super class FilePersistenceManager.

In FilePersistenceManager:

protected static final String DATA_FILE_PERF = "gametracker.datafile";

In CSVPersistenceManager:

private static final String DATA_FILE_DEFAULT = "gametracker.csv";

Persistence managers are passed the arguments off the command line, so as in the CLI, the super class checks the arguments if there’s a value for the datafile and that is taken and loaded into the preferences.

if (cl.hasOption("datafile")){
  Preferences.userNodeForPackage(
    FilePersistenceManager.class).put(
      DATA_FILE_PERF, cl.getOptionValue("datafile"));
}

The CSVPersistenceManager then checks the preferences to find the filename for the data file.

datafile = new File(
  Preferences.userNodeForPackage(FilePersistenceManager.class).get(
    DATA_FILE_PERF, DATA_FILE_DEFAULT));

The data file can be changed from the setFile method in FilePersistenceManager.

public void setFile(String fileName) {
  Preferences.userNodeForPackage(
    FilePersistenceManagerMenu.class).put(DATA_FILE_PERF, fileName);
      datafile = new File(fileName);
}

I’m not getting a ton of use out of preferences right now. I was doing more before I started streamlining, but right at the moment those are the only two things I need to keep track of. Again, I’m not sure I’m “doing it right,” but this setup seems to work for me. 

The Magic Inside

As I said, the great thing about preferences is that the JVM and the operating system really take care of it for you. So if you’re happy to trust the “magic system” to keep stuff up and running, then you’re good to go. On the other hand if you’re trying to figure out what the hell is going on, then let’s talk about it.

I ended up researching this because when I came back to the GameTracker after a long break, I had no idea what was going on or why the software (which I wrote) knew where the files were without me telling it (also sometimes I’m bad at taking notes - but I did have an issue on the topic for myself). This was especially a problem because I was looking for a local configuration file, which just wasn’t there.

I set out to figure out what was going on. Thankfully a number of people have written pretty good explanations of the preferences system. I decided that I could add a little bit by bringing those articles together and the best way to learn anything is always to teach it (or in this case blog about it).

So with that being said, there are two things to talk about if we want to open up the magic inside.

How are the preferences organized?

At the JVM level, all preferences are kept in trees. There’s the system tree and a tree for each user. (So as a programmer you have access to two trees, the system and the current user). 

Each node in the tree represents a package. The root of the tree is the empty package and it’s named “/”.  Then every package below the root is its own node. If we take my GameTracker as an example, the CLI.java main class is in package ca.kendon.gametracker.cli. That means that the root node has a child node, “ca/”, which has a child node “kendon/”, which has a child node “gametracker/”, which has a child node “cli/”.

A tree diagram of the CLI class's preference.

Since we have multiple packages with preferences (.cli and .core)  both of those packages's preferences would be held under their common parent.  so "gametracker/" would be the parent to both both "cli/" and "core/"

A tree diagram showing the common ancestors for both the "cli/" and "core/" packages.

Notice that the names of the preference node have those slashes at the end. If you ask for the full name of the node the CLI preferences are in it will be "/ca/kendon/gametracker/cli”. If that seems like a file name... well that might be important later (in some contexts).

Where are the preferences stored?

The JVM has its nice abstraction of preferences in trees, but that doesn’t help when your question is “where is that value *actually* stored?” Now we actually have to take a look under the JVM and see what the operating system is up to. Preferences exist in all operating systems, not just for Java but for any software that needs to remember things. Of course, since all operating systems have their own ideas about where stuff is, there’s a different answer for every operating system. For a quick overview we’ll look at Windows, MacOS and Linux.

Victor Kropp has a really good post on the details.

Windows 10

As windows is wont to do with all of its configuration information, things are stored in the registry. 

User preferences are found at HKEY_CURRENT_USER\Software\JavaSoft\Prefs

System preferences are found HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Prefs

Remember that you can look at the registry using the Registry Editor (regedit) utility, which is installed by default in Windows.

MacOS

On MacOS, preferences are stored in the Library. Since we have two types of preferences, system and user, MacOS actually has two types of libraries (system and user) for preferences.

System preferences are found in /Library/Preferences/ in a file called com.apple.java.util.prefs.plist

Each user’s preferences are found in their home directory in ~/Library/Preferences/ in a file *also* called com.apple.java.util.prefs.plist

plists are Apple’s favourite way to store things in a way that slightly frustrates everyone … well it frustrates me certainly. They’re XML files (often compressed) which can be a pain to actually look at. On the command line you can run the command:

defaults read ~/Library/Preferences/com.apple.java.util.prefs.plist

Which will show you the preferences for your own user.  For example mine includes:

        "gametracker/" =         {
            "cli/" =             {
                "gametracker.persistence_manager" = "CSV";
            };
            "core/" =             {

                "data_file" = "data/games.data";
            };
        };

If this seems a little overbuilt it's the weird collision of Apple's plists and xml files which always end up having some weird structures.

Linux

On Linux, preferences for each node are stored in its own file (in case you’d wondered why the Java name for preferences looked so much like a file name).

System Preferences are stored starting in the directory /etc/.java/.systemPrefs/

User preferences are stored in each user’s home directory under ~/.java/.systemPrefs/

Under those directories, the JVM builds a directory structure that matches the preference nodes, with the leaf nodes being files. In my Game Tracker Example, that means that the "gametracker.persistence_manager" = "CSV" preference is stored in the file ~/.java/.systemPrefs/ca/kendon/gametracker/cli. 

This makes a lot of sense to my *nix-centric brain and I think explains a lot about the structure they chose when setting up preferences in *nix environments.

Conclusion

I'm always wary of systems that aren't transparent to the user, but preferences provide a good way to keep track of the details about a program without making a mess or requiring a lot of overhead. I still have a lot to learn, already, I'm wondering about some of the things I'm doing to update the preferences from the program. Hopefully this overview provides a place to get started and answers a few of the questions the preferences seem to draw. 

Footnotes

* I *think* you can use the this identifier to get the class dynamically, but I'm not sure whether or not that's a bad idea. (Thus far I've found I've made mistakes using the literal class - copy and paste errors - so using this may be smarter.)

** This is a snapshot of the GameTracker from early November 2021.

Project 18: Game Tracker Update - 2021

 So it's been a long while since I've sat down and worked on the Game Tracker and it's also been a long while since I've written anything other than vague intentions for working on the project. Fortunately my new focus on creative work has push me back to it.

I was actually pushed by a post I'm writing for the blog that I'll share pretty soon. I've wanted to do a good explanation of Java's Preferences. I managed to thoroughly confuse myself with them and so thought I'd follow the Julia Evan's approach of writing about what confused me. I'd left my project in a thorough mess back in March - the equivalent of the truck sitting in the old bard with the engine out and wires and tubes laid out all over the place - and so in order to show how I was using preferences, I thought it might be a good idea to put the bits back together.

When I abandoned the project, I was in the midst of trying to convert the save system (persistence) to use JSON. That was part of a grander idea to connect to some kind of web interface - Google Sheets and Firebase are the two I've been eying. I ended up struggling to get Java and JSON to play nicely together and after a little bit of cobbling, I discovered that Joda-Time -  which I've used for managing time in Java for *years* - was also making everything complicated. 

I gave up and decided that since I mostly wanted the system up and running so I could finish the blog post - which I've been working on for months. I set up a project - well... took over the one from March - called 2021 refresh and tried to get a minimal set of the Game Tracker up and running. That was more or less fun, I built a CSV version of the data manager and then did all the cleanup I had to do to get up and running.

Game Tracker - Pretty much the same after all these years.

I also realized that I've been keeping the GitHub repository for the project private. I think that's partly based on not feeling like the project was ready for people to see and partly fear about showing off my own programming work. Given what I've been thinking about with creative work and how I want to use my time going forward, hiding isn't going to help me, so I opened the repository to the public and put and MIT Licence on it - not that I think it will be terribly useful for anyone. I found Choose a Licence, which was helpful for comparing licences.

 

I also got to write a readme, which at the end of the day turned out to help me feel better about the whole project and make sure that anyone who stumbles across it will know what on earth is happening. Nothing about the whole project feels at all glossy or cool - like I would like it to - but it's happening and I've moved it a little bit forward. I found Make a README which was a really good resource.

I like that I can return to the idea of small tasks for the project now. For example, for reasons that are beyond me - maybe as a demo for students - I set the game systems to be a Java Enum. This is a terrible idea since for some reason new video game systems have been released in the last few years and I've even purchased some of them. It'll be nice to be able to jump in and work on that for a little bit and then be done.

I also have a lot of other things I'd like to do. I'd like to expand the interfaces to include a desktop GUI, something for my phone and something for the web. I also still want to set up remote data management for the system. I'll get to those at some point, but again my new approaching to getting things done means they'll hopefully trickle out over time.

November 2021 Project Update

I recently watched Cathy Hey interview Jeff Walker about building a creative career. While I'm not planning to leave my day job any time soon, I've been sitting on a lot of creative projects without making much progress and this really inspired me to get moving on a lot of these things.

I decided as a starting point I wanted to dedicate a little bit of time each week to working on projects. I set an initial goal of 5 hours a week and I really haven't nailed it yet, but I've actually started to get a little bit of stuff done. 

As apparently has become my wont I set up a spreadsheet to track my time, and so now my life outside of work is filled with colourful bar charts. 

My focus for now has been on reviving some actual posts in the blog. Inspired by Julia Evans (@b0rk), I wanted to start including more about the things I know and the things I'm learning. The first of those is using Java's Preferences. I'm using preferences in the Game Tracker so thought I'd use it as an example and then I realized that a) it's a horrible mess and b) it's not available to the public. Now I'm updating that and my next post will be about that.

So, generally, you can expect to see a few more posts outside of me tracking books and games here. I've made some progress on Chrono Trigger Sprites and I have a backlog to post there. Plus you can also expect more posts about technology, learning and teaching. I'm not committing to any set rates, but "more".

I feel pretty happy approaching my projects this way, so I really appreciate that video. I think I've started on a good productivity groove right now and I'll take it.

I've been sketching more too!

Projects Update: January 2021

I'm finding this particular project update a bit of a struggle. On the one hand I feel that I've been much more productive on my own projects than I've been in the past, but on the other hand I'm feeling a bit lost overall and like there's still a lot of things I'd like to do. 

At the beginning of January I started a new job and I'm finding balancing my energy difficult. I appreciate having my time better structured with a 9-5 position and the position itself has a lot of breadth for creativity, but I do find myself pretty tired at the end of a day. 

All in all I'm finding it a bit of a challenge to hit a balance that works for me. I probably need to relax and take to heart that I've only been working on this for a month. One thing I am trying to do is to reduce the number of things I'm trying to do at any point in time. For the first time in most of a decade I've reduced my daily to-do list down to just the absolutely must do things. Then I allow myself to decide what the best next thing for me to do is. 

As an AI researcher this gives me a real fear at getting stuck in a local-optima, but honestly I find that the landscape shifts and I'm able to get things moved forward here and there. Not as much as I'd *like*, but that's how life works. Beyond that, I'm trying to decouple my feeling of self-worth and happiness from productivity and completing projects, but that's a long journey to undertake.

At the moment, I've got 6 projects I'm providing updates for:

• The Blog, as ever, is ticking along. I actually wrote my first editorial piece in ages earlier in the month. I may write more, but my primary focus is to have fun keeping track of my reading and game playing.
• I left the Chrono Trigger Sprites without a deadline and I'm enjoying that. I've done a bit more and so there will be project updates about those eventually, but I'm not rushing and I'm kinda happy to have a hobby that just a hobby. 
• I haven't been writing  outside of the blog, so my project on The Roofs has been slow. I think if I do decide to carve out a little more time this will be the project I go to first. I'm definitely sitting on those first four thousand words I wrote and finding it a bit hard to accept the permission to write terribly, while pushing forward. If I don't carve that time out, then I'll let this go dormant for a bit.
• I want to put together a Google Drive backend the Game Tracker, but this has been complicated, because the documentation from Google is all built around accessing drive through a Gradle build. In my mind I'd just like a library I can link against, but that's not the way the (fairly limited) documentation works. This throws me off making any progress and even though I have other avenues I could work on, I keep getting stuck. I haven't put in a huge amount of time, but this is definitely a project I'm focusing on. 
• One of the ways I was able to justify taking the non-teaching job is to myself that I want to work on Code Click, building teaching resources and building up my experience for teaching. Code Click is a huge part of that, but for now it's also a project that can wait. I'll come back to this (I've spent quite a while thinking about what I would like Code Click to be like), but for now I'm not going to focus on it too much.
• My time on Infinite Acorn Adventure has mostly been spent trying to remember all of the linear algebra that I took (poorly) two decades ago. That's been fun in itself, especially with Daniel Shiffman's videos.

I'm not going to share any deadlines for things right now. None of these projects need to go anywhere in particular, so I'm going to work on them when I feel like working on them. Later if something needs a push to get *actually finished* then I might bring back the idea.

This was a lot of fun to build and definitely easier than some.

Projects Update: September 2020

My last general project update was in May and I was a bit surprised by how fast May went by. Now, it's September...

Honestly I struggled a bit with productivity over the summer and eventually gave myself permission to just take some time off. I also hit a bit of a complicated patch with finding post PhD employment and figuring out what I want to be when I "grow up".  

In short I'm finding myself myself drawn to a creative career as well as (or in favour of) an academic one. So I'm trying to balance both of those for the next while to see what's possible. I've set myself some goals for the next year to give myself some firmer deadlines and so that I can see what's possible with what I can create for myself.

I am finding it a bit rough teaching this semester. I had hoped to be able to take the semester to increase my creative output but ended up with more teaching duties than I'd expected. This is making it a bit rough to get things done, but I'm doing my best to plug along where I can. Remembering I can take the time for myself has been tough, but I think I'm learning.

Where I am right now:

The Blog (Project 1)

I did give an update as usual at the beginning of August. This trucks along and the reading and games updates are a fun task to take a break and do.

The Roofs (Project 11)

I didn't write nearly as much as I might have, but I did start writing and it felt pretty good. 

I finished around 4000 words in May/June, before getting distracted and wandering away from the project. I managed to get my focus back in August and started a rolling edit to revisit what I had and to tighten it up. I'm trying to avoid doing that again, since I need to keep moving forward, but I'm still feeling like I'm accomplishing something.  

Chrono Trigger Sprites (Project 12)

I finished the Robo sprite a while back. I was able to do it really quickly, but I've been slow to write the project post about it. I have it on my plate to get done in the next few days.

I'm enjoying working on these, but I'm thinking they're a lower priority than some of the other things I want to work on right now. I have 3 main characters left to do from the playable characters, but I think I'm going to tackle them when I have a quiet Sunday.

Robo In Action!

Game Tracker (Project 18)

I didn't get to work much on the Game Tracker at all. I'm feeling a bit conflicted about this because on the one hand it's not a priority for me. I want to focus on writing, making games, Code Click. On the other hand, it's beneficial to teaching and it's been the motivation for me to learn more about developing modern real world software. 

Infinite Acorn Adventure - Bubble Puzzler (Project 13)

As I said, I've spent a lot of the summer thinking about what I want to do and how I want it to sustain me (and also putting food on the table might be nice). One of the things that really came to the fore for me was that making games was a place I wanted to focus. I've also been thinking a lot about how I want to be able to tell stories through games and use games as a mechanism for supporting a good life.

As such, I've been a bit frustrated with myself about the fact that I'm not working on making games and I'm not learning about making games. So I think the solution to that is to make a game.

Code Click (Project 20)

I think one of the roles I fit very well is computer science communicator. I'm interested in making things easily understood and I think generally there's a problem in communicating about computer science. Given that I'm in a state of looking for full time work in some combination of technology and teaching, spending a bit of time working on my computer science communication skills seems worth while. 

For my classes this semester, I've obviously had to move my teaching on-line, so I'm already developing those skills in a university context. I think pushing the science communication skills makes sense, and I think trying to capitalize on Code Click is a good place to start.

Where to next?

My thinking for the next bit of projects is:

• Infinite Acorn Adventure - A prototype in Processing by October 31.
• Code Click - One post by October 31.
• Game Tracker - I'd like to have a google docs back end, working by November 31.
• The Roofs - I'd like to get a finished (but bad) draft by August 1, 2021.
• Chrono Trigger Sprites - If I get to them I get to them, I'm not going to give myself a deadline.
• The Blog - will keep travelling along.

Projects Update: May 2020

What happened to May?

Just at the end of April I finished the coaster project and I was feeling pretty pleased with myself. I didn’t get quite through everything I wanted in April, but I was pretty happy. Then it was halfway through May and I had no idea where I was or what was going on. I will say I’ve been enjoying spring for the most part, although there’s been some quarantine weirdness kicking around my brain as well.

I'm really happy with the new coasters.

I haven’t made a lot of progress on the game tracker and I’ve found it pretty hard to sit down and write, so I’m behind on both of those for now. I’ve also picked up an academic thing I need to finish fairly quickly, so I’m thinking about that too.

Since I haven’t made much progress I’m just going to slide my goal for the game tracker back to June 30. The goal is to have remote storage of data working by that point (along with having tutorial myself on various build systems and remote storage solutions.

I also want to push forward on my Chrono Trigger sprites so I’d like to get Lucca finished by June 15.

Project Updates: April 2020

It’s pretty usual for me to get to April and realize that I’ve lost track of all the things I was excited about. I’m still working on balancing my teaching so that I can be creative and productive and a good responsible teacher as well. It’s less usual to have that April realization fall in the middle of a global pandemic where suddenly everything has gone weird and suddenly I’m stuck at home, as opposed to being a little out of work for the spring and stuck at home.

Anyway, once I finish marking I want to do … something, so I’m going to “circumwork¹” and see where my projects on this blog are at.

Projects I am "actually working on"

One of the problems with maintaining the blog is that I have a list of things I “should” be working on even if they’re not actually the things I want to be working on. I try to avoid that by keeping my timelies short and achievable (yes, I know I’m bad at that) and by keeping the number of projects I’m working on at once (yes, I’m not great at that either).

Currently I have on my list:

• The Blog (as Project)
• I’m not 100% sure where I’m at with the blog, I’m enjoying tracking games and books. I don’t know that it’s a really productive thing to be doing, but I’m enjoying it. I also find that I don’t really want to put that much out there in terms of thoughts, because I’d rather put my time towards making something.
• Covert Action in Space
• I got a little held up because it turns out that randomly generating meaningful floor plans is a little harder than I’d figured. I still love the idea, but this isn’t at the top of my list.
• Game Tracker
• I’ve been teaching first year Java again and I wanted to get things rolling here again. I also taught a senior programming course in the fall which involved using online services and mobile interfaces, both of which my project needs. I opened it up a few weeks ago and couldn’t quite figure out where I left off, but managed to get it mostly on the path again.
• Pong
• I wanted to be way further ahead on this and have my AtariST version done ahead of my students, I managed to work along side them, but then completely lost the thread when the “transition to on-line” teaching happened. Hopefully I’ll have cause to get back to it.
• Code Click
• I already spend quite a bit of time thinking about how to share that moment of joy I get out of coding, and while I haven’t got that much done, I still think about code click a bunch. Transitioning to on-line has also pushed me to think a lot more about how I want to teach and the resources I want available when I’m teaching.
• 
  

Projects I have “on hiatus”

There are a bunch of things I started out and then put aside, some of them are things I want to be working on so I think it's worth listing them all out too.

• SNES Coasters
  • I’d like more and bigger coasters and maybe to spend a bit more time working on perler stuff.
• Space Station Game
  • I keep thinking about this one. I’m still not ready to really set down and work on it, partly because I’d like to build up my skill working on some other projects first. I've been playing quite a bit of EU4 which has definitely provided some feeling for how the game should work when I actually get to it.
• Action RPG
  • This is another one I’m not ready to work on yet, but I have a lot of ideas and I’ve been developing my drawing skills.
• Sci-Fi Novel
  • The problem with having sat with a story in your head for 20 / 25 years is that when you think about writing it, it feels pretty trite. There’s a lot of things kicking around in my head from as far back as when I was a teenager. I don’t really know where to go with it, but it still might be fun to tackle at some point.
• The Roofs (Fantasy Novel)
  • This is the story that sits further in the front of my mind. I’m not sure it makes sense, and it might be missing a reasonable antagonist, but I guess I won’t know until I write it.
• Chrono Trigger Sprites
  • I have the first two sitting in the window over my desk and I love them. It’s time I got the rest finished.
• Bubble Puzzler
  • I think with Pong out of the way this is the place I want to focus building games. It’s a good learning opportunity and I think it’s a great place to get started.

Projects I actually want to work on now

So I’m not sure where I want to put myself for all of the time between now and September. Obviously working on code click is a good idea for professional growth, but the Game Tracker and the Bubble Puzzler also make sense.

I also want to work on more artistic things. A lot of that I don’t think I want to make projects for, but I think that both the SNES Coasters and the Chrono Trigger Sprites. I also really want to get the Roofs written.

In an effort to keep my goals small and my projects limited, these are my near, term projects:

• SNES Coasters
  • I want to finish a set of 4 large coasters, get them fused and backed and then I’ll see what’s next. I think I can get that all done by April 30.
• Game Tracker
  • I’d like to get this working with outside data, either my original plan of google sheets, or with something else (possibly firebase). Either way I don’t want to spend too much time thinking about it, so I’m going to try to have some version of that working by April 30 too.
• The Roofs
  • I don’t know how long it is, or how I’ll feel actually trying to write, but I’m going to give it a shot and try to have a first draft finished by August 31. (And yes, I think I did just put write a novel on a list of “short achievable near term goals” no, I’m not great at planning things)

(and we’ll leave the blog rolling along as it is, since that’s fun).

¹ Circumwork: To do things that feel like work without actually being related to any task that needs to be done.

Project 18 - Video Game Play Tracker - Update 2 (v0.2.0)

I actually finished up v0.2.0 of my game tracker months ago, but I finally got some free time so I thought I’d write up what I have. I have probably forgotten a lot of the details at this point, but I’ll just soldier on. In this update I finished up adding in aggregation and filtering of play sessions to the display system (and did a little bug fixing). 

Filtering

I wanted to be able to filter play sessions into specific time periods. Right now I start a new spreadsheet every year to keep track of the data in a manageable way, and I start a new set of tabs within each spreadsheet to track data for each month. I wanted to be able to replicate this in an ad hoc way, which would allow me to build a more informative and flexible way to look at the data.

A small view of my tracking spreadsheet.

Additionally it’s nice to be able to include or exclude games or groups of games from the list. For example, how much time have I spent playing all the games in the Zelda franchise? I have no idea right now but filtering would make that a question I can ask easily.

As such, I added two types of filters for play sessions, one for games and one for dates. The filters work by finding all the play sessions that match and then showing only data from those sessions. 

The game filter works by matching the (whole) name of a game. Multiple games can be added to the filter which then allows multiple games. So far, I don’t have any structure like a series or a genre, so I don’t have filters working for that, but that’s something to work on in the future. The filter provides a list of all play sessions that match the set of game names.

The two basic filters.

The date filtering works by using date windows. Each window has a beginning and an end date and the filter allows through all play sessions that are between the two ends. If either of the two dates is unset then the filter allows all sessions to the beginning or end of time.

It took a little fiddling to make them work, in particular to make an empty filter allow everything (which is backwards to how the filters are designed, but extremely unintuitive to work with).

Aggregation

In my spreadsheet I keep a log of all games played for the year (including the date, the game name and the amount of time played). Then for each month I break out one pivot table that includes the total amount of time each game has been played for the month and a second table that breaks out the games played for each day so that I can produce the stacked bar-chart of the games. I needed the aggregation system to allow me to basically do that in the stand alone system.

January's games played chart. (See a better view in my January 2019 post).

Aggregation was interesting because I wanted to make it as abstract as possible, but still needed to make it concrete enough to work in the command-line interface. I think I may need to take a second swing at it to make it really work smoothly. 

The way it’s currently constructed aggregators take a set of play sessions (raw or filtered) and extracts a aggregate, which is a map organized by game name, the type of aggregate and the value (as a double). Aggregates can be merged to put together different stats about a set of play sessions. 

The aggregated games in my test file (mostly from early 2018).

As I said the system is a little messy and I think having now let some time pass I could do a better build solution.

Project Update

Compared to the first update, I did a better job of using the git workflow and breaking down my work into smaller chunks and updating regularly. This was helpful in the work I needed to do to finish my thesis project. I also tried to start teaching the workflow to my students this semester and I think generally the “do small things well” mantra has done me (and them?) well.

The current state of the Game Tracker

It’s also been months and months since I finished the actual main work for this update. I did break some tests which ended up taking me a while to fix. Since then I’ve mostly been working on my PhD so I’ve had to backburner the whole thing.

The two big directions I’d like to take the project in next, are to produce a GUI version using JavaFX and to integrate a Google Doc back end storage. The first step I think is going to be the hook up Google sheets and then go on to the JavaFX front end.

My plan, as it stands, is to finish the next update including the Google Sheets connection (and some various fixes / reorganization) by June 1, 2019.

Project 1: Project Octoseason Blog*

Happy Blog End and Start Day!

via GIPHY

Yes, it’s time for that annual celebration where I remember that I started the Blog on August 1 (a long time ago) and should probably figure out what I’m doing with it. It’s also your annual reminder that not all things last forever, but this blog still might for another year.

Ahem, first, as tradition dictates, I’d like to remind you that the “Blog” here comes in two parts, the Blog as Project Report, and the Blog as Blog (which is itself a project). Effectively, my intention here has always been to get excited and make things - even if that’s not always as evident as I’d like it - and one of those things I’m excited to make, is a Blog where I talk about - well mostly the media I’ve consumed, but you get the idea.

Tradition further suggests that now is the time that I tell you I’m not done my PhD, but I’m close - and folks I’m getting ever closer to actually finishing the damn thing - and given that, that I haven’t done quite as much work on my own creations as I’d hoped last August. Still, here we are, I’m not going to beat myself up, I’m just going to celebrate what I have done.

In terms of projects in the last year, I’ve worked on four and I’m generally, pretty happy with how they came out. I started messing around generating floor plans for a game I’d like to make at some point. That only managed two posts, but I still had fun - and I’m still thinking about it, the Flurpins will be back “soon”. I pushed myself to read more, and set myself the goal of reading 12 books in 21 weeks. That was a nice project, in that it had a set end date, and ended at the end for 2017. I actually made it all the way to 18 books - what a stunner. I started on a program to help me with game tracking, and also to just get some general programming practice in. Finally I tackled #NaFYoFuThMo an effort to get me pushed across that finish line of that ever looming PhD.

A Flurpin ... an odd side effect of generating floor plans.

As far as the Blog as Blog goes, in the last year I’ve kept up with tracking my video game playing, which I continue to find interesting, even if it probably seems a bit repetitive. I dropped a little behind in the monthly posts, partly because I was fairly overwhelmed in the Winter managing teaching and the PhD. I’ve mostly caught up now, and you can expect to see the June and July posts in the next few days. 

Following on from the success I had in boosting my reading in 2017, I’ve tracked all of the books I’ve read so far in 2018. I think It’s been worthwhile, and I’ve boosted my goal for the year on Good Reads from 32 to 40 (but that’s mostly to accommodate the fact that I can read a volume of Saga in a morning, and I hadn’t planned to read Saga at such a rate).

I only wrote one “thoughts on” post this year. That was Legend of Zelda: Breath of the Wild, last August. I haven’t really finished another game this year that I’ve had that many thoughts about - while I loved Into The Breach, I found I didn’t have a lot to say other than, It’s really good. I suspect that in the next little while I may write a “thoughts on” piece for Paper Mario: Colour Splash - where you can enjoy my chants of “The WiiU is not a 3D system!” - and I may write a follow up to Breath of the Wild, another hundred hours on.

It's ok, you just have to trust the game not to smack you in the face with a hammer ... which you can't.

For the first time I wrote a New Years Resolution post. I think it was good for me to write down what I wanted to do better, or differently, this year. Generally, I’ve been more successful than not: I’m *slightly* better at monotasking and much better if I don’t let stress build up. I think I’ve done an okay job of holding fewer opinions, although that also varies with stress. I do think it’s helped while teaching introductory computer science, where many people hold a number of *very* strong opinions which may not matter very much. I think I’ve also been better at acting and getting stuff done just by standing up and doing it - it’s easier than I think it is.

I’m going to finish my thesis, sooner rather than later. I think in a small way I’ve been better on Twitter and happier with how I’ve been on Twitter. I’ve definitely read more and enjoyed a lot of what I’ve read - and then there’s Eats, Shoots and Leaves. I’m not sure I’ve made more stuff, but I’ve done a lot of stuff around the house that feels similar. I’ve seen more things, but I’d like to see more and keep pushing past the boundaries of inertia. 

More or less, I’m happy with the Blog as Blog in its seventh season. It has mostly been bits of media I’ve consumed. I didn’t really mean for it to work out that way, but at the moment I’m feeling fairly happy with that. My favourite YouTube videos have fallen off, but I think given the state of that platform generally, I'm okay with that. I may bring them back in a different form in the future.

I’m going to do an Eighth Season of the Blog - surprise! I suspect it will look very similar to the seventh. Tracking media keeps me interested, and if I find I have something I want to write about in relation to that, then I’ll have a good space to do that. I am hoping that as I finally finish the PhD, I’ll be able to add in a few more projects - I have several in mind, which should be fun.

Thanks to all of you who read, I hope the fun I have here is at least a little fun for you as well.

via GIPHY

*Yes, yes I did make a stupid reference to Octopath Traveller in the title, what of it?

Project 18 - Video Game Play Tracker - Update 1 (v0.1.0)

I’ve really had a lot of fun with this project so far.

For version 0.1.0 I’ve added in the basics, including:

• A game representation, including the name, the year and the platform.
• A set of games and get back games that match particular criteria
• A play session representation, including the game, the date and how long the session lasted.
• A list of play sessions
• Managers to load and save game and play  data from and from csv files.

I’ve also built a simple text-based UI, which allows you to save, load and add games and play sessions.

The Basic Text Based UI Menu

I also think I did an ok job using the git flow workflow, although I think I ended up adding too many features in my release 0.1.0 branch. I also added a ton of tests, even if I didn’t quite get myself to a test first mentality.

So I’m happy enough with v0.1.0. RIght now I’m keeping it in a private GitHub Repository, I should probably just open it up, but right at the moment I’m not sure I want to. At least just yet.

The basic tracker running in the terminal.

In terms of usability it’s not great, but it’s a good starting point. My next goal from here is to introduce filtering and aggregation so I can see play sessions aggregated over a list and filter for specific games or time periods. I’ll post an update around July 1, 2018.

Project 18: Video Game Play Tracker

I've been tracking the video games I play for the last few years now. It started out as a move to help me remember what I actually played over the course of a year and how now become something of a habit, as well as a good way for me to be more mindful about how and what I play.

Thus far I've been tracking what I've played in a Google sheet (actually 3 of them at this point). This works pretty well for the most part and it has a lot of nice "analytics" features (by which I mean it has nice pivot tables). I'm finding though that the Google sheet doesn't scale as well as it could and it's a little limited in terms of visualization options (especially 3 years into tracking).

At the same time I've been feeling a bit behind on some practical development skills. I'm not practicing as much as I'd like and when I do I'm messing with my behemoth PhD system.  (A story for another day.) I also don't play with as many dev tools and services as I'd like to.

Combining those things along with spending the last 4 months teaching novice Java programmers and I've decided to give myself *yet another* project. Generally my goal is to: make the Game Tracker work, make it work with Google Sheets and take advantage of anything that seems fancy and cool. Also get a little practice in working with a good git workflow and taking advantage of the tools GitHub provides. And most of all not to let any of this get in the way of the other things I need to be doing.

The goal here is to keep it small and keep it simple. For Phase I, I'd like to get the basics set up: a text menu system to track games and play sessions and save and loading to file. I'll check in next on May 15, 2018.

(And if you find yourself wondering what happened to those games he was talking about, I'll get back to those, but this (right now) seems like it's going to be more compatible as a project that I get to work on after all my other work is done for the day.)