War Room is out, and I have had a chance to play around with it. There is some good, a lot of bad, and a whole lot of ugly. I have primarily used the app on iOS devices – including an original iPad and an iPhone 4s. I have also played around with it for a minute or two on smash’s android tablet. Most of this comes from the iOS viewpoint, although from what I can tell, much of this is the same experience between the different ecosystems. So strap in and prepare for the ride – this is a long one.
I am a software developer by trade. I have developed systems for banks giving out loans, I have worked on mobile apps myself, I have even worked on research hardware. I currently work on backend systems for a company that prides itself on having as streamlined and as smooth of a product as possible. No, not Apple, but if you are thinking of them, that is the idea.
I have made some pretty poor decisions in my time as a software engineer. I have made some poor UIs, and I have made some poor design decisions with the software I have written. I consider all of those learning opportunities. While this review may seem harsh (and long!), it is intended as exactly that – a peer review of a work that feels like a learning opportunity. Both for Privateer Press, and Tinkerhouse Games, if they are willing to accept it.
What I Wanted
When War Room was announced, I was excited. Supremely excited. When I had started playing WARMACHINE and HORDES at the beginning of MKII, I had immediately thought “man, it would be nice to have tracking software on an iPad, and card and rule viewing on an iPhone.” Primarily, I wanted to be able to build lists that would tell me if they were legal, and would help me track data during a game.
I would rank what the basics of War Room can do in the following importance:
- List Building and Validation
- Card Access
- Rule Access
- Game/Card mangement
- Communication for said game management
I would be willing to bet (although this is entirely unscientific, purely based on a relatively small sample size) that most WARMACHINE and HORDES players would have their priorities similar.
This is important – very important. When you build software, there are infinite possibilities. You have to narrow those down to the absolute minimum necessary to deliver solid, quality products to your customers with the least cost to yourself.
Keep the priorities in mind (yours or mine!) as you read through the rest of the review, and as you use the app yourself (if you so desire). There will be more discussion about this later in the review, but this is one of the primary tenants of software development, and I suspect it was ignored, misunderstood, or just plain not known going into this space.
Let’s look now at the good, bad and ugly of the software, and what that means for the usefulness of it as a tool.
Despite all the wailing and gnashing of teeth about the software, there is some good to be found.
The idea of showing the cards themselves was actually pretty cool. At first I thought I would hate it, but the more I have used that portion, the more I have grown to like it.
Being able to look at the “cards” in the card format has made it easier to find the information I need quickly. Additionally, making the cards “live touch” to find the information you need off of the card (i.e. what does that symbol under that weapon mean?) is another nice user experience touch.
While there are some problems with the actual implementation details (specifically, the flow and responsiveness), it is a good idea with a lot of merit.
The idea of having a game connection where I can share the information live with my opponent is really cool. And while clunky, the interface is not terrible to work with. The idea, and the concept here, are what I am focusing on. Because when I heard about the concept originally, I went “yes, that is what I have been thinking of!” Because seriously – this is a place where technology can help us, rather than getting in the way.
Just kidding – I have very little nice to say about this feature of the application. Much more on this part of it later.
Having the rules at hand has been very nice. What has not been nice is the fact that I have to purchase something to get access to the rules repository. A large chunk of the rules are available in the quick start rules, and it would be nice to be able to see what the rules repository looks like before dropping money.
While there is some good with this app, there is a lot of bad too. A lot of bad. While I won’t analyze why all of it might be this way in this section, I want to call out the most egregious of offenses (in my mind), as well as point to ideas on how it could be done better.
This is something that is easy to overlook as a developer. Speaking from experience, this does not become an issue until you attempt to use your own application (commonly phrased as “eating your own dogfood” or some similar idiom).
The responsiveness of this app is lackluster at best, and terrible at worst. It is very clearly using a 3-D rendering engine for it, which is serious overkill for what is, at it’s heart, a utility app. The screens take a moment to shift, they stutter, and things take a consistently longer amount of time to respond. In the book Usability Engineering, Jakob Nielson discusses responsiveness, and limitations on the maximum amount of time that something should take. Many of the interactions (if not all of them) take at least 1 second, and most seem to take longer. Note that this “seem” is the ultimate issue – as users, we are easily tricked about time. But when the system visibly hangs, stutters, and takes a moment to give any visual feedback it feels much longer.
Much of this seems to be directly attributable to the decision to use a rendering engine rather than standard UI widgets. When you are rendering every frame like it is a 3-D landscape, it will take processing power from the device. The more layers you have on it, the more power it will take.
There are also rumors that the data files are actually stored as files on the device. This will take much longer to load than it would from a database, and would likely decrease responsiveness.
War Room does not remember your log in. This is a problem, in my opinion. I don’t want to have to log in every time I load up the software. It should remember my login for a period of time at least. This is a minor annoyance, but one that can be a deal breaker. If you are like me, you want people to love your software. This is the type of thing that keeps that from happening.
On the subject of logging in is device management. You can apparently only have two devices attached to an account at a time. For anyone who has multiple devices they use (i.e. family iPads, personal iPad, Nexus 7, Kindle Fire and a phone) this is a huge restriction. Additionally, it seems like they use an installed version of the software to register up your device to your login. I have witnessed local gamers have to deregister both of their devices (because there is no way to deregister one at a time) because they had to reinstall on their Android device. I do not know if this is the case on Apple devices, but this can be a major headache for anyone who has to deal with multiple devices.
In my opinion, this was the biggest problem in the entire system. This was the piece that I was most looking forward to, and it is probably the most anemic part of the software.
What is “List Building”?
Before we talk about the bit of software that focuses on building lists, I think it is important to discuss what list building even is. I say this, because different people have different ideas about what that means.
For instance, in his Insider post on 7-16, Will Shick discusses War Room and it’s development. The first hint of what the list building function would look like comes in this article (emphasis mine):
One major point of discussion was how unrestrictive we wanted the army construction utility to be. WARMACHINE and HORDES are constantly evolving, whether from new model releases, new theme forces, or even fantastic home-brew events. Having already started with the goal of mirroring the physical experience of cards in a digital format, we decided to continue with the 1-to-1 mindset and give players unlimited freedom when constructing a list. While we did provide some important guiding utilities like FA and point validation for ease of reference, the sky is the limit when building a list. This gives War Room the flexibility to be the ultimate player toolbox and gives the app a dynamic internal working that allows it to easily adapt to new development without requiring major functional updates.
The important piece on this paragraph, in my opinion, is the unrestrictive nature that they shot for. Well, they succeeded. Horrifically.
Look, when I am making a list, I want, need, desire, pick-your-word, the ability to turn on the restrictions. I want the system to slap me when I mess up. If I want complete, strange weird requirements for building lists, I will use the cards. When I am building a theme force, I want to have the system tell me what is legal. If the “uninhibited” part is so important, make it an option for the list itself. Or even at the entire application level. I have a very hard time not thinking of those comments as cop-outs for not doing a complete job on this feature. New theme forces come out that are crazy and do strange things? You just might have to do a software update. That is the nature of the beast. You should not be afraid of that – you should embrace it, and plan for it.
The lack of the ability to restrict the list building is literally a deal breaker for me. That is the primary point of an application like this for me. This application does not do this in any fashion that even approaches usable, and that is a shame. It is a seriously missed opportunity for Privateer Press to continue to show they are the best tabletop gaming company in the market today (which I truly believe they are).
What is currently in War Room is list handling, not list building. It allows you to put a bunch of cards together in some sort of order, just for fun and kicks. Since it does no validation of the lists, if I want to make sure what my opponent has built is legal I have to closely compare their list to their theme lists. If I want to make sure what I have built is legal, I have to do the same thing. Did I misread or misunderstand a rule? Did I think that the Black 13th counted as Arcane Tempest Gun Mages for the purpose of a tier bonus? Whoops! Is there any way in the app to tell me this? Nope, not really.
Additionally, what about lists that make models cheaper? Now you have to remember this while building the list. How is this any better than using the cards? Actually, it is worse, because the interface is so unresponsive. What about Xerxis No Quarter tier list? He can take Venators as weapon attachments for certain units – how is that going to be represented? How do I know that this is not legal the rest of the time? These are all the things that a list building system should be handling, but instead is just ignored for the purposes of “unrestrictive.” This is not conducive to a good product, nor to a good user experience.
What Cards do I Own?
One annoyance is that it is impossible to tell what cards I have access to. Instead, I have to guess “is this the right Morghoul?” when trying to create a list. This causes a dialog to pop up every time I select a model that is not available to me. This is a huge break from the flow, and makes it very hard to use the software in a natural way.
A simple fix to this would be to gray out the unowned ones, or have a filter mechanism to hide them, so that people will have a good first experience when looking at it with just the default models available.
I entirely understand their desire to not allow access to cards unless you have paid for them, but why keep me from making a list with those models? If the list building functionality was complete, it would be nice to be able to create a list, even if I couldn’t view the model’s stats there.
I will go into this in depth when I talk about The Ugly, but I find the usability of this piece of software to be one of the worst that I have encountered in a long time. Buttons are all over the place, seemingly at random. One dialog has the OK button in the middle of the screen, the next one is at the bottom. The back button is at the top of the screen for most screens, but some it is at the bottom of the window. It makes navigation difficult and tedious. There are many more usability problems that I will go into later, but if you feel frustrated using the software, you are not alone. It isn’t you, I promise.
I wanted to separate out the usability, look and feel, and the like into it’s own section. This is a very important part to me. People love software, tools, and the like because they are pretty and usable. Yes, there will always be the vocal minority that want software cheap or super-powerful at the expense of usability, but most people are not that way.
There is a customer metric called Net Promoter Score that attempts to capture how loyal and fervent your customers are. Apple consistently ranks very high, often the highest, of all companies in this metric. I do not think anyone could argue that their products are beautiful and well engineered, with the user’s usability in mind. All of the other arguments aside (less powerful, overpriced, whatever), they are beautiful pieces of work that are a pleasure to use. If you want customers to push your products to their friends, colleagues, neighbors, random strangers on the street, this is a good thing to shoot for.
Net Promoter Score comes straight out of the idea that your customers should be delighted with your products. In general, Privateer Press has delivered on that premise. I have personally recruited 10 or so players from my social circles, and have been ancillary in another two or three players locally. This is because I am fervently a fan of their game and their products.
When dealing with software, the main criterion most people use to judge it is how usable it is, how responsive it is, and how pretty it is. This is a reality. It does not mean those people are “less” than others – they are customers. It does not mean that these people are “simple,” or anything like that. It means that they don’t care how it works – they just want it to work.
As an example, if I purchase a car, I want it to run with a minimum of effort on my part. I don’t care about the engine, I don’t care how much horsepower it has. I want it to get me from home to work, to the grocery store, and on the occasional road trip to Seattle without causing me fits the entire way. If it isn’t responsive to my attempts to accelerate and decelerate, that is a problem. If it is uncomfortable to sit in, has a confusing radio, etc – all of this means I will be less likely to purchase it in the first place, and much less likely to purchase another car from that manufacturer in the future.
All of this is why I feel that user experience is of utmost importance. And it is in this that War Room fails.
Nonstandard Widgets and Controls
This is a big one. iOS has a standard for their widgets, and pretty much anything they were trying to do with War Room (outside of the cards) could be done with the built in widgets and controllers.
Android has standards as well. Users expect certain behaviors on their platform of choice, and when the developer ignores those standards, it is jarring at the least, irritating and frustrating at the worst. As an example, Android has a back button as part of the hardware, which does not work with War Room.
I cannot overstate this – when I see software that does not follow platform guidelines, I expect there to be a good reason. Games, for instance, are a good reason. I have yet to use a utility app that has that reason, especially not one that has a reason for it throughout the entire app.
Some people may think this is a nitpick. However, this goes back to the idea that if people love your product, they will promote it and buy more of it. The menus are non-standard for any platform, and are terrible to look at. Also, menus are really not something that translate well to mobile devices. That is an artifact of a desktop. If we need to do something with a menu, it should create a pop-over dialog that we can then interact with.
The fonts and sizes chosen for different dialogs and views all seem to be chosen at random. There are fonts that are blurry on the cards, fonts that seem to large for their location, and ones that clump the words together because they are at a fixed size.
This is a very common problem with UI design. Using the wrong font can cause a strange sense of irritation in a user, and they will often not be able to say why. That said, in this case I think most would be able to determine it quickly.
There are a number of smaller problems with the buttons, and I have attempted to bring them together in one place for quick check off of each.
The sizes of the buttons are too small by far. They are often half the size that they should be, and are hard to hit. I know I am not the only one to notice this, but it is especially noticeable with the small “x” close buttons on some windows.
The buttons are not always responsive to touch. This is sometimes because you have to touch slightly to the side (see the next point), or because the software itself is not responding quickly.
Some of the buttons seem to be slightly off visually from where the software wants to recognize their touch. This is extremely frustrating, because it doesn’t seem to be very consistent. Some windows recognize the “x” click, or OK, or whatever immediately. Others do not, and you have to hunt and peck around it to find the right spot to cause it to close.
Lack of Zoom
This may not be a large deal to a number of people using this software, but I know I am not the only one with bad eyesight playing wargames. There are days my eyes are better, and days they are worse. And when they are worse, I want to be able to zoom to better read whatever is in front of me. I have become so accustomed to it that not having it is jarring and disconcerting.
I realize this is probably because they are not using built-in font handling, and instead have decided to roll their own (see the part on fonts above). I still don’t find this acceptable – if you want the most people possible to use your software, make it accessible to as many people as possible. They will love you for it. They will throw their money at you.
No Swiping Between List Building Views
When building a list on a phone, there is no way to swipe between the different views in the phone, you just have to push (tiny) buttons on either side of the screen to get the different views to show up. Another minor thing, but it breaks the flow of usage in a very real, and aggravating way.
Why Did This Happen?
I want to take a moment to speculate about why this happened. As I mentioned above, I am a software engineer, and have seen my own share of debacles and terrible releases. I have seen horrific UIs and bad releases. (Anyone else remember Windows ME?)
After these situations, it is often good to reflect on what went wrong, what was ignored, etc. Please note – this is not an attack, nor is it necessarily a correct assumption about what happened. It is based entirely on what I know of the situation, with a heaping dose of my own experiences and observations with other compaines thrown in.
That said, let’s strap in and take a look at what could have gone wrong. If you are thinking of developing software (or anything that an end user will be using directly), these are all very important things to keep in mind during your development process.
Game Development Is Not Application Development
At first blush, this seems to be idiotic to even say. Games are applications, right? Well, kind of.
Games are a very specific type of application. They have some very hard problems of their own – balance, player interface, interaction, story, enjoyment, etc. These problems are all difficult, and are often compounded by the fact that games require a lot of math to make work correctly. I mean a lot of math.
These are all very different from what makes a good application. War Room is, at it’s heart, a record-keeping application. From a very high level, it is no different than a word processor that allows for multiple people to edit the document together. (Anyone who has written a word processor is, right now, furiously typing a response about how much harder a word processor is to write. Yes, I know – high level here folks!)
How is this different? The primary goal of the utility is to make information more easily available, make it easier to do the project/job than it is to do it without electronic intervention. If it doesn’t do this basic thing, it is failing at it’s job.
This means that usability is paramount, as I mentioned earlier. If the flow is broken as a user is using the software, they will be less likely to continue using it. As an example, if you were using a word processor that paused for 1 second every time you pressed space to verify that the word you typed was spelled correctly, you would probably not use that word processor for very long. If to start a new document, you had to hit 5 menus, you would not use that program again. You would go back to using text edit, or even worse, just start writing everything with a type writer or by hand.
This is the problem space of a utility app, of an application vs. a game. In a game, as long as it is fun, and nothing is too onerous, you can say you succeeded. I have not played any of Tinkerhouse Games’ games, but from what I hear they are well developed. Their abilities as game developers is not in question – what I question is using a development team that focuses on game development to do application development. I also question that development team agreeing to do the project. It seems they did not fully understand what was required to build a truly excellent product.
Right on the heels of the previous section, usability continues to come up. Crazy, huh? It turns out that this is the thing most people I have seen are complaining about. Why? Because they want it to work for them in an unobtrusive way. I would guess that no one tried to use the application for very long stints at once, and even then they only used it one way. This is a common mistake from testers, and especially testers who are developers at heart.
I know with my software, there are places I am bad at testing, because I know the brittle parts and avoid them subconsciously. One of the most eye opening experiences I had was watching someone else use my software while I could not help them – I could only watch. This is very illuminating, and humbling. You often find you don’t actually know what the user wants, nor how they will use it.
And this is the larger part of my point – users will want to use the product slightly differently, and developers are terrible at figuring out what ways that is. Designers are terrible at figuring this out without lots of training. This is why the usability industry is booming – because people want beautiful products that are easy to figure out and easy to use.
What Looks Good for a Demo is not Always Good for the Product
A lot of what I see in War Room looks great when demoing it. The pretty background, the pictures on the tables of the army builder, the transitions when a button is pressed. These are fantastic when demoing. They are not necessarily good when using the software. In many ways, this is a product that demos well for a video, but not well for actual usage.
I talked about this early on, but I want to touch on it again. When developing software it is usually best to have priorities. If you focus on small, bite-sized chunks of development at a time, you can deliver functionality quickly and regularly, and when the date arrives, you can ship what you have with full and complete functionality. This is an important piece – full functionality. Much of War Room feels half-baked at best, and in some cases true vapor ware (i.e. list building).
Again, the priorities of the software should be what users will find the most useful the quickest for the least amount of cost. This builds fantastic software for minimal cost that people want to use, and will give you money for the privilege of using.
While I don’t know the underlying technologies specifically, I want to discuss two things that seem to be indicitive of the other problems I have mentioned above.
Why use a Graphics Engine?
Graphics engines are fantastic for game developers. They abstract away the problems inherent in attempting to render images in a quick, dynamic session. Either 3-D or 2-D, they are a blessing for all game developers who do not want to handle the low level graphics calls on their own.
They do not lend themselves well to application development. Most widgets are already designed by different frameworks, and are heavily optimized by engineers who’s job it is to optimize those components. You also gain the native behavior for free, and often get lots of little pieces built in that would take forever for you to recreate yourself.
I can think of two reasons for this decision. The first is that the developers are game developers, so this was the world they felt the most comfortable in. This also leads into the second reason, which is that they wanted to develop once and publish to different platforms. I suspect this combination of reasons is why they used the Unity engine (or at least are rumored to have used it).
Developing for multiple platforms is not easy. I will not try and claim that it is. however there are component-based frameworks that compile for both iOS and Android platforms (and are now legal to use in both app stores). This would give you the ability to stay native as much as possible, and hopefully get some usage out of other engineers solving problems that you will likely run into building components for applications.
One other piece related to using a graphics engine – it is harder on the phone’s battery than using simple components would ever be. I have noticed that War Room chugs away at my battery on both my 4s and my iPad. I have seen similar complaints from others on other pieces of hardware.
File System vs. Database
Another element I have seen is that all of the card data seems to be stored on the file system. Why this is not in a database structure, like sql-lite, confuses me. Yes, it would still be reading from the file system at some level, but it would be fast to pull up specific information on demand. This is the type of system that databases were designed for – quick loading and searching through large data sets.
This is, I would guess, part of the slowness experienced by users. When waiting, they are loading from the file system, and loading new graphics in the graphic engine.
This is the sorest spot for many people. iBodger was such a wonderful list building app, and did so many things absolutely right, that following it up with War Room is a bitter pill for many to swallow. Since iBodger was pulled from the marketplace, it was strongly implied that War Room would be a solid replacement, and would be better. I would argue this is not the case – War Room does not even come close to the functionality that iBodger offered, and the functionality that it does offer beyond it is incomplete, buggy and slow.
Can War Room be Salvaged?
This is a bit of a loaded question, deliberately so. I see a set of steps to salvage War Room:
- Privateer Press and Tinkerhouse Games must apologize
- Amends for those who paid money already
- Fix the hole left by the loss of iBodger
- Focus on Quality, not Quantity
Privateer Press and Tinkerhouse Games must apologize
This is really a two parter though. Because apologizing is not enough. The apologizer must show that they are not only contrite, but willing to make amends and fix the problems as much as they are able. This is hard to do, and requires humility and acceptance of one’s own failings.
Look, everyone makes mistakes. Everyone has bad decisions. I made three this morning before I even showered. That is not the important part. The important part is that you own up to those mistakes, and try to make amends and make it better all around. This is the most important thing you can do. It starts to rebuild the trust that was lost by making the mistake.
Amends for those who paid money already
If I had purchased a deck, or worse the whole shebang, in excitement before really trying out the app, I think I would want my money back. I don’t think it is unreasonable to grant refunds to those who paid already. It is a good way to generate lots of good will, and a good way to start the trust building again. Because trust was lost here. I trusted Privateer Press to build a solid piece of software (through their proxies), and to not waste my time and bandwidth with low quality productions. I feel let down in that trust, and I am more wary of their other developments. Is this going to bleed over into the IK RPG? How about the new HORDES book coming out?
These are all valid concerns, and the trust building process will take some time. That said, in the 2+ years I have been playing PiP games, this is the first time I have been seriously disappointed by a release of anything. So my trust is not completely bashed – but it is starting to stagger a little bit. Some rebuilding of that by good faith efforts would go a long way towards making me a promoter in their net promoter score.
Fix the hole left by the loss of iBodger
The loss of iBodger was a huge blow to me, and I know it was a huge blow to most of our local gaming group. Based on the number of people talking about it still, I think it was a major blow to them too. It was, honestly, the single most used app on my phone. I used it more than I used texting and the phone app combined.
Focus on Quality, not Quantity
This seems obvious, but it often is lost in the feature creep of software development. War Room was stillborn in part because it was feature killed. There was too much that they tried to do too quickly. If they had released the cards and the list building initially, with the decks costing 30 bucks for the whole set (i.e. 50% of the current price) it probably would have released on time, better quality, and we would be able to use the software.
Some of the problems would still have cropped up – using a graphics engine for a utility app is just not OK. But it would be easier to fix too, because there would be less code to overhaul. This is why the maxim “less is more” is very, very true in almost all cases. Focus on doing less, better, more often – this leads to great things. Software is, by definition, soft. It can change after you release it. Release with less features, but those that are built being high value and high quality. People will flock to your software in droves.
Was This Actually a Review?!?
I know this was long. I felt there was a lot to talk about, and I had a perspective that I didn’t think the majority of the WARMACHINE and HORDES community would have.
What is my final opinion on War Room? I love the concept, I love the ideas, I love everything about it except the actual software. I am unable to bring myself to pay the amount being asked for to get access to the cards, because that is basically all I would use it for.
So, my verdict? Don’t buy it now – wait and see. Download it, and if you love it, then go ahead and purchase the decks. For me, I will be waiting to see what comes next. If it doesn’t get more performant, and doesn’t become higher quality, I will probably leave it aside and go back to doing it all analog (i.e. with cards and pens on the table).
Some of this may seem harshly worded. That is part of the intent. I love Privateer Press. This is the first disappointment I have had in their development of games in 2+ years of playing WARMACHINE and HORDES. I am seriously disappointed, and honestly embarrassed, by this release. As a software engineer, it is acutely painful. As a game player, it is disappointing. As someone who loves the company, I feel residual shame for their mistakes made.
Am I advocating that they scrap the whole thing? No, although that may be necessary to really get a good piece of software. Am I advocating they drop Tinkerhouse Games? No, although that again may be necessary. Maybe they should bring back iBodger? I honestly don’t know enough about their internals, and the internal contracts. Nor do I want to. I want the company to succeed. Hopefully this post/review will have some effect. Feel free to comment on anything I have forgotten, ideas you have, etc.
I will say again that this is not an attack on Privateer Press or Tinkerhouse Games. This release was a mistake on many levels, and they really need to accept that and learn from it. Everyone makes mistakes – it is how we respond to those mistakes and people’s criticisms that tells us about the person.