We’re all Software Engineers of one type or another. Most of us have one thing in common, though: We’re lazy. And we know that someone else was less lazy and has already solved that tedious problem that we’re on. And because we’re not only lazy but also stingy, we search for Free Open Source software.
But the problem with Open Source software is: There are millions of options for about every problem domain. Just look at web development with “modern” JavaScript. Which tool to choose? Which one will still be there tomorrow? Will it work? Will I get maintenance? New features? Plugins from the community?
While it is not so easy to find the right tool among the good ones (commons or guava? mockito or jmock? Hibernate or jOOQ or MyBatis?), it is certainly easier to rule out the bad ones.
Here are some things to look out for, when evaluating an Open Source software (in no particular order)
1. NullPointerExceptions, ClassCastExceptions
This is one of my favourites. It is very easy to google. No one is completely safe from these annoyances. But when you find stack traces, bug reports, investigate them closely.
- Do they appear often?
- Do they appear in similar contexts?
- Do they appear in places where they could’ve been omitted?
It’s a matter of good design to be able to avoid NullPointerExceptions and ClassCastExceptions. It happens to everyone. But no NullPointerException should be thrown from a place that can be statically discovered by the Java compiler or with FindBugs.
Needless to say that the list of no-go exceptions thrown from a database library, for instance, can be extended with SQLExceptions due to syntax errors produced by that library.
2. Community Discussing Bugs instead of Features, Strategies, Visions
Every Open Source software has users and with Google Groups and GitHub, it has become fairly easy to interact with an OSS community.
For larger projects, the community also extends to Stack Overflow, Reddit, Twitter, etc. These next steps are a sign of popularity of an Open Source software, but not necessarily a sign that you should use them. Also, don’t be blinded by users saying “hey this is so cool”, “it just made my day”, “best software ever”. They say that to everyone who’s supporting them out of their misery (or laziness, see the intro of this post).
What you should be looking out for is whether the community is discussing visions, strategies, features, truly awesome ideas that can be implemented next year, in the next major release. It’s a true sign that not only the software will probably stick around, it will also become much better.
The converse to this is a community that mainly discusses bugs (see NullPointerException, ClassCastException). Unlike a “visionary” community, a “buggy” community will just create work, not inspiration to the vendor. But which one’s the chicken, which one’s the egg?
Another converse to this is a community that is disappointed by the false promises given by the visions of the vendor. I often have a feeling that Scala’s SLICK might qualify for that as it introduces an insurmountable language-mapping impedance mismatch between its own, LINQ-inspired querying DSL and SQL.
3. Poor Manual, Poor Javadoc
That’s easy to discover. Do you really want that? The best and most authoritative information should come from the software vendor, not some weirdo forum on the web that you’ve googled.
A good example are PostgreSQL’s Manuals.
A rant about bad examples can be seen here:
http://www.cforcoding.com/2009/08/its-time-we-stopped-rewarding-projects.html
Don’t be deceived by the idea that it might get better eventually. Poorly documented software will be poor in many other aspects. And it’s such an easy thing to discover!
Of course, the “right” amount of documentation is an entirely other story…
4. No Semantic Versioning
Search for release notes and see if you’ll find something that roughly corresponds to semver.org. You will want patch releases when your Open Source software that you’re using in mission-critical software fails. When you get a patch release, you don’t want 50 new features (with new NullPointerExceptions, ClassCastExceptions).
5. Unorganised Appearance
Again, we’re in times of GitHub. The good old CVS times are over, where HTML was still used to share cooking recipes. Check if your Open Source software uses those tools. If they show that they’re using them. It will help you ascertain that the software will still be good in a couple of years if the vendor isn’t crushed by the mess they’ve gotten themselves in.
6. Vendor Side-Project evolving into an Offspring Product
Now that is a sign not everyone may agree upon, I guess. But after the experience I’ve made in previous jobs, I strongly believe that software that has evolved out of necessity before making it a product really suffers from its legacy. It wasn’t a product from the beginning and it has strong ties to the vendor’s original requirements, which doesn’t bother the vendor, but it will bother you. And because the vendor still has very strong ties to their offspring, they won’t be ready to make fundamental changes in both code and vision!
Specifically, in the database field, there are a couple of these software, e.g.
Note, I don’t know any of the above tools, so they may as well be awesome. But be warned. They weren’t designed as products. They were designed for a very narrow purpose originating from a pre-Apache context.
7. Generics are Poorly (or Overly) Adopted
Generics were introduced in 2004 with Java 5. Now that the heated debates about generic type erasure are over, generics are well adopted. Or aren’t they? The latest stable release 3.2.1 of Apache Commons Collections is still not generified! That must’ve been the number 1 reason why people had started shifting to Google Guava (or its predecessors) instead. There’s not much making for a lousier day than having raw types (or eels) slapped around your face.
The other thing that you should look out for, though, is over-generification. Generics can become really hard, even for top-shot Java architects. A common blunder is to strongly correlate subtype polymorphism with generic polymorphism without being aware of the effects. Having too many generics in an API is a good sign for an architecture astronaut. (or a design astronaut in this case). We’ll see further down how that may correlate with the person behind the design decisions.
8. Vendor Cannot Handle Objective Criticism or Competition
Here’s how to find out, who’s behind the Open Source software. While this isn’t important for a small, geeky tool, you should be very interested in the vendor as a person when looking for a strategic OSS addition, especially if you’re dealing with a benevolent dictator. The vendor should be:
- Aware of competition, i.e. they’re doing marketing, learning from them. Improving to compete. This means that they are interested in being truly better, not just “convinced that they’re better”.
- Open minded with their competition, with you as a customer, and ready to discuss various points of view.
- Interested in new ideas, possibly putting them on a roadmap right away (but without losing focus for his main strategic roadmap).
Even if this is Open Source, there’s no point in being arrogant or conceited. The vendor should treat you like a customer (as long as you’re not trolling). Open-mindedness will eventually lead to the better product in the long run.
9. Vendor has no Commercial or Marketing Interests at All
Now, (Free) Open Source is nice for many reasons. As a vendor, you get:
- Feedback more quickly
- Feedback more often
- Community (with pull requests, feature additions, etc.)
- The feeling that you’re doing something good
True? Yes. But that’s true for commercial software as well. So what’s the real reason for doing Open Source? It depends. Adobe for instance has started opening up a lot, recently, since their acquisition of Day Software. All of JCR, JackRabbit, the upcoming JackRabbit Oak, Sling and Felix are still at Apache with the original committers still on board. But one can certainly not say that Adobe has no commercial interests.
OSS vendors should think economically and build products. Eventually, they may start selling stuff around their core products, or separate community and commercial licenses. And unlike they get too greedy (see Oracle and MySQL, vs RedHat and MariaDB), that can make commercial Open Source a very interesting business, also for the customer who will then get the good parts of Open Source (partially free, open, with a vibrant community) along with the good parts of commercial software (premium support, warranties, etc.)
In other words, don’t choose overly geeky stuff. But you might have recognised those tools before (poor documentation, no semantic versioning, poor tooling).
10. No Traction Anymore
To wrap this up, here’s an obvious last one. Many Open Source products don’t show any traction by the vendor. That goes along well with the previous point, where the vendor has no commercial interest. Without commercial long-term interest, they’ll also lose all other interest. And you’re stuck with maintaining a pile of third-party code yourself (fixing its many many ClassCastExceptions, NullPointerExceptions).
TL;DR : Conclusion
You should chose Open Source just like commercial software. Economically.
- Open Source is not an excuse for bad quality.
- Open Source is not an excuse for lack of support.
- Open Source is not an excuse for non-professionalism.
If Open Source fails you on any of the above, the joke will be on you, the customer. You’ll get a bad product, and you’ll pay the price with exaggerated maintenance on your side, which you thought you’d avoid by chosing something free. Nothing is free. Not even Free Open Source. Ask the Grumpy Nerd
My biggest pet peeve is failing to be clear about what exactly the problem (and solution) domain is. What problems does it and doesn’t it solve and what concepts and constructs does it force you to buy into? This is especially prevalent with “frameworks”.
True. Well, fuzzy problem / solution domains can be either:
Thinking about it again, I think that the Grumpy Nerd summarised it pretty well :-)
good one
I’d say another common error with free stuff is thinking that popular projects can’t die because, should the author decide to move on, community would create forks and pursue development.
I can name at least two examples of the above not occurring.
A few years before, osCommerce was once among the most popular ecommerce platforms, it had many extensions and drew a lot of interest. There were, and still* are* many companies running ecommerce based eShop, but better platforms (Prestashop, Magento) became available and osCommerce popularity declined faster than anyone would imagine and see what’s left : an outdated stable version (2.3) and a new version (3.0) that has been in development for years and hasn’t received any commit during this year ( https://github.com/osCommerce/oscommerce ).
Play! is another example. Back in 2011 (or even 2012), you could find many elogious blog posts about this framework and how it was going to make you forget everything you knew before and what happened?
Its authors switched to scala, built a new, incompatible framework from scratch they named “Play! 2” and decided to stop supporting “Play 1”. People using play 1.x were left in a very uncomfortable position, they had to chose between a costly migration to 2.0 requiring them to re-learn basically everything not even mentioning that 2.0 had way less features than 1.x and didn’t support war deployment anymore, or stick with their dying 1.x version.
Unfortunately, while Play 1.x was quite popular, there hasn’t been any serious fork attempt, the developers choice to use Play! name for a totally different framework may be partly responsible for this. 1.x users had difficulties getting the core team to process pull requests and bug fixes and it’s seriously disturbing to see one of the most promising java web framework end up like this. Just because their developers decided to create a different framework, stealing both the name and community of the previous one to attract developers.
So unfortunately, being open source and popular are no guarantee of survival.
These are nice examples, Stéphane. Play! is a very good example showing that Open Source software often follows the strategies of a marketer. In this case, the popular Play 1.x framework was “bought” by Scala’s Typesafe, a company that tries to create a full-fledged stack around Scala (including Scala itself) to get 80% of the cake from Scala consumers. This is similar to what Microsoft does with .NET, except that Scala was originally “Open” and “free”. I personally respect Martin Odersky for his strategic brilliance, and I’m quite sure that this turns out to work as he had originally planned. On the other hand, it’s every user’s own fault, if they are “tricked” into such a vendor-dependency by believing that there is anything such as “free software”.
I do agree, though, that the original developers at Zenexity might’ve made a better choice than to exclusively team up with Typesafe. After all, the Java market is just so much bigger…
The irony of all of this is – of course – that this discussion is led on the blog of a “free software” library :-)
That is important to keep in mind, though one doesn’t necessarily need every library to “survive”. There’s actually a number of open source libraries I use that are “dead”, and that’s not been a problem. One of them I have made a few tweaks to, and that would not have been possible for a non-open-source product.
Sometimes it’s actually a blessing that someone isn’t continuing to mess with the code and glom crap onto a library that is complete. Of course no one forces us to upgrade, but when there’s a new version, we engineers hate to feel like we are slipping behind.
(That said, I wouldn’t be happy if that happened to a web framework I was using.)
Another good example I thought of was the For Play/PlayN vaporware game framework from Google. It promised the world (Desktop, HTML 5, Android, and iOS compatibility out of the box) and was abandoned within a year. Good times.
They keep supporting java.
I use play2 (I assume it as a bad choice) and program in java. They say java and scala are equally important to them and they argue that you don’t need to know scala to use the framework, except some basic stuff in the templating language.
This argument would have been valid if there had been more documentation, as a result, I spent countless hours on the mailing list and stackoverflow trying to figure out what the objects I was manipulating exactly were and how I could get them to do what I needed (template helpers and field constructors are good examples).
Also it’s not easy to transpose scala code snippets to java, not to mention that some features are only available in the scala API. Thanks to scala compilation time, the “hit refresh workflow” that was the biggest benefit of Play! over competition has become “hit refresh and have a coffee break workflow”.
To be honest, I thought it would be a nice excuse for getting my hands on scala. With APIs available in both languages, it seemed like a good starting point but unfortunately it turned out that I missed the point about scala…
I’d like to point out that many of these points are equally applicable to non-open-source products, and many of the others would be applicable but you have less visibility to apply them.
I’ve used many software products over the years, both open source and commercial, and both are a crap shoot. Many products are crap and some are good, and you have to be very careful what you are choosing. While a few of the factors that cause the flaws in software products differ between open source and commercial, most of them are the same. In my experience the biggest factor is lack of vision and / or leadership.
The really standout software I’ve used has almost always been produced by a single person, some of it was commercial, some of it open, some of it closed but freeware.
This comment is actually a preview to a future blog post that I’m planning :-)
There is a priori no difference at all between (Free, Non-Free) Open Source and commercial software from the quality perspective.
But there is a big difference in how users perceive things. In the Java ecosystem (and more importantly in the JavaScript, Ruby and other ecosystems), Open Source has a positive, better connotation than commercial software. But that’s not true at all. Few .NET developers feel that they’re missing out on the open-sourcey-ness that we Java developers “enjoy”.
(of course, no one denies that it is very useful to have access to the sources)