… no, don’t tell me you like Perl. Because you don’t. You never did. It does horrible things. It makes your code look like…
Designing Perl 6 as found on the awesome Perl Humour page
Perl made heavy use of operator overloading and used operators for a variety of things. A similar tendency can be seen in C++ and Scala. See also people comparing the two. So what’s wrong with operator overloading?
People never agreed whether Scala got operator overloading right or wrong:
Operator overloading can be good, but mostly isn’t. In Java, we’re all missing better ways to interact with BigDecimal and similar types:
// How it is:
bigdecimal1.add(bigdecimal2.multiply(bigdecimal3));
// How it should be:
bigdecimal1 + bigdecimal2 * bigdecimal3
Of course, operator precedence would take place as expected. Unlike C++ or Scala, ideal operator overloading would simply map common operators to common method names. Nothing more. No one really wants API developers to come up with fancy ##-%>> operators.
While Ceylon, Groovy, and Xtend implemented this in a somewhat predictable and useful way, Kotlin is probably the language that has implemented the best standard operator overloading mechanism into their language. Their documentation states:
Binary operations
Expression
Translated to
a + b
a.plus(b)
a – b
a.minus(b)
a * b
a.times(b)
a / b
a.div(b)
a % b
a.mod(b)
a..b
a.rangeTo(b)
That looks pretty straightforward. Now check this out:
We’ve recently blogged about Ceylon’s awesome language features. But the above Kotlin features are definitely a killer and would remove any other sorts of desires to introduce operator overloading in Java for good.
Let’s hope future Java versions take inspiration from Kotlin, a language that got operator overloading right.
32 thoughts on “Why Everyone Hates Operator Overloading”
2/3 looks pretty much like what’s shipped in Scala since pretty much forever, just a bit renamed, because NIH I guess.
Regarding the first one … imho, “operator overloading” is just not important enough to bother coming up with elaborate rules about how things are translated from symbols to names.
It’s kind of funny to see that there is pretty much no intersection between actual users of Scala and people obsessed with complaining about symbolic method names.
This was the first article on this blog related to operator overloading / symbolic method names, and it wasn’t really Scala-centric. People have criticised C++ for similar reasons, and for precisely these reasons, operator overloading was never part of the JLS. Or do you mean someone else when you say “obsessed”? Or is this the usual reaction people get when they think 1-2 things are not OK with a particular language? :-)
> Or is this the usual reaction people get when they think 1-2 things are not OK with a particular language?
No, sorry, that wasn’t targeted against you or your blog post, just wanted to share my experience. :-) My perception is that these things are big deal for excatly those who have never used that functionality in the first place.
Scala does that too, automatically by convention? I didn’t know that! Nice.
My perception is that these things are big deal for excatly those who have never used that functionality in the first place.
Aha, I see. Anything can be a big deal if you have time and passion for the details :-) Some call it the bikeshed problem. Another example is JavaScript. Recently, I’ve posted this article to Reddit:
> Scala does that too, automatically by convention? I didn’t know that! Nice.
No, ““Array” access” and “Method calls” are pretty much identical, just with other names and no special [] operators/syntax.
I’d say that “Binary operations” is a pretty bad idea.
Pretty much every language invented on this planet has the intuitive rule of “if you name a method X, you can invoke that method using that name”.
I’m not seeing the benefit of the approach of “we have this hardcoded list of special method names X, Y and Z, which you are supposed to invoke by using foo, bar and baz”.
What’s the benefit of that?
To allow people to freely mix those names like
val a = 2
val b = 3
val c = 5
a.modAssign(c + (b.times(a) / b.minus())
?
I think the simpler approach of “Use the method name you want, if it looks like math the known precedence rules apply, done.” is better thought out.
Pretty much every language invented on this planet has the intuitive rule of “if you name a method X, you can invoke that method using that name”.
Well, there are exceptions:
var foo = new function() {};
var bar = foo;
bar('abc'); // This calls foo
Just imagine writing a language spec […]
This appears to be the “spec”: http://confluence.jetbrains.com/display/Kotlin/Grammar. It looks as though these math operators will only work on the same type, i.e. A = A * A. Removing Java’s primitive type promotion is already greatly helping in formalising this type of convention.
From a Specs point of view, I guess Scala can do away with processing the AST several times to first substitute what looks like math by method calls with appropriate precedence, before compiling the actual methods. But then again, this additional processing step slows down compiling…
I don’t think this is an exception, in fact most languages allow this. Note that you can still use foo(), but if you define the method “timesAssign”, you are not supposed to invoke it as “timesAssign”, but as something named differently in Kotlin.
Regarding the spec: The grammar is the least important and interesting detail of a language spec. What matters is the description of semantics and with that list of hard-coded special cases …
Regarding processing the AST in Scala: This is all handled in a single phase called parser. The reason why Scala uses multiple phases is because it makes it more easy to verify that each transformation is done correctly and allows people to more easily extend the compiler.
Anyway, looking at performance, parsing pretty much doesn’t matter at all. In a sense, pretty much no phase matters from a performance POV except typechecking and codegeneration.
There is already a new codegeneration backend which improves performance substantially, so typechecking is pretty much it, and it’s hard to optimize it, when other things (like correctness) are much more important.
I think performance of typechecking could increase considerably if Hotspot would do its job properly, instead of letting scalac run in interpreted mode for the first 2 minutes. By the time it decides to emit substantial amounts of JIT-compiled code, the compilation is likely to be done already.
The Kotlin documentation is unclear about how somebody writing custom Kotlin code would implement a custom overloaded operator. Is there a convention that a method “plus()” would translate to “+”? If so, how is this different from Ceylon’s polymorphic operator overloading, where you need to satisfy the Summable interface?
From what I understand, Kotlin (and Groovy, Xtend) uses convention, Ceylon a formal contract. Both approaches have advantages / disadvantages. Convention is good to retrofit existing APIs, a formal contract clearly communicates the intent.
On the contrary. It looks like you’re pretty much limited to what operators Ceylon lets you overload. This means plus()/+ for Summable, union()/| for Set, etc:
1) You can not overload operators in Scala – you can only use non asci characters in methods. That is a huge difference.
2) I have seen a lot of abuse of extends in java – should we forbid it because some people are to stupid to use it correctly?
I can see your point 2). Every tool can be abused, although “extends” clearly pulls its weight. About 1), technically you’re correct. You can use non-ascii characters and leave away syntactic elements like dots and parentheses. When I read / maintain legacy code, however, that distinction might seem a bit academic.
I feel the opposite toward operator overloading. What sense is it to take English, an old and fundamentally broken language, and intentionally pollute a new one with it?
As opponents of operator overloading usually point out themselves, operators are essentially equivalent to functions. That should be enough to realise that operator overloading is no bigger a problem than allowing programmers to name their functions whatever they want.
If you point this out to the naysayers they usually ask instead “if you can do the same thing with normal function notation, why bother with operator overloading at all”. The reason of course is readability, a + b = s is easier to type and read than s.set(a.addition(b)) which is of course why operator notation evolved in the first place. If used correctly operator overloading would make a lot of code more concise and easier to understand.
If anyone thinks you only ever want to add integers and floats they have a very primitive view of mathematics and programming
You also often hear the somewhat rude argument that no real (e.g.) Java programmer use operator overloading… duh (well to be precise they say ‘need’ but ‘use’ is close enough and it makes the err more obvious).
Thanks for your insights. I agree with your statements, and I think the article does, too. I suspect that the “naysayers” mostly oppose the introduction of arbitrary operator-esque symbols that are hard to read. However, hardly anyone probably opposes mapping the “A + B” notation to what translates “addition”, semantically.
I personally believe that every Java programmer who has done some arithmetic with BigDecimal would immediately see the advantage of a simple operator overloading implementation…
The caveat in BigDecimal is the need for MathContext, in particular on divisions.
How would you shove that into an operator overload? use a default? configure by IoC, neither sounds like a good solution for the problem of defining precision nor rounding modes.
This is a subject I am quite interested in because I am having some fun at attempting to add unit type-safety to numbers (what are numbers really without a unit to bind them to the real world?)
Perl made heavy use of operator overloading and used operators for a variety of things.
Core Perl doesn’t use operator overloading, but it does you dynamic typing. Operators are actually used to control the typing somewhat. For example, “+” is used to add numeric types, and “.” is used to concatenate strings. Similarly, “==” will test for numeric equivalence, and “eq” will test for string equivalence. The important concept is that since those operators are tied to types, they will implicitly convert to the appropriate type (and warn if that’s not cleanly possible) and then perform their operation.
This comes from a core principle of Perl, which is that similar things should look similar, and different things should look different. String concatenation and numeric addition really aren’t that similar, so Perl makes the distinction, between string ops and numeric ops with new operators. This helps to avoid and entire class of problems that exist in other dynamically typed languages, such as Python and Javascript, what actually do overload some operators (most notoriously the “+” and “==” operators covered above).
That said, Perl does have an Overload module to support actual operator overloading for objects, but it’s a module and not used in the core (well, there aren’t really objects used in the core of Perl 5).
Perl 6 goes a different route, and used operator overloading extensively, but it still tries to follow the principle of similar things looking similar and different things looking different, so while there are a lot of cases of operator overloading, the intention is to make it obvious what is being done. One way in which they accomplish this is by using a *lot* more symbols for unicode, which can be controversial. One example of this is set notation symbols for equivalent set operations on lists. Usefully, these are almost always represented by an equivalent method on the underlying type.
Thank you very much for the interesting insight! Indeed, using “+” for String concatenation is a mistake in so many languages, although I’ve never gotten used to the Perl/PHP “.” alternative.
Interesting, so, do set operations make use of mathematical symbols, e.g. for unions, intersections, etc.?
Yes, Perl 6 tries to make use of both unicode and multi-character equivalents (“texas-style”) of specialized notation where avaialble. For example, you can use subset/superset unicode operators, or you can use (>) and (<) as ASCII equivalents.
You can see more examples of some of the mathematical symbols that apply to sets here[1]. I think the set class contains the most obvious example of this, but combined with Perl 6's prefix, infix and postfix operators and scoped exporting, you can do some really interesting things, like so:
... code here
{
use Physical::Units ; # made up lib, theoretically adds postfix unit types and infix multiplication/division handling
my $length = 20kilometers;
my $time = 30minutes;
my $velocity = $length/$time;
say $velocity.to(Meter).to(Second); # 11.11m/s
}
my $foo = 10kilometers; # Error here, there is no postfix kilometers any more
Personally, I think that’s pretty cool (even if it doesn’t currently exist). Mini DSLs that are scoped to where they provide the most benefit so they don’t leak out and pollute the namespace.
I think that allowing the creation of custom operators has benefits if used correctly.
Not that someone should create a @#+-~^_?=:>! operator, but there are cases when a custom operator is better than functions or tricks to achieve something similar to making a new operator.
For example, it would be better if you could write
if (var in 1..5) {…}
instead of
if (inRange(var, new Range(1, 5))) {…}
or
if (var >= 1 && var <= 5) {…}
It's more readable and shorter.
Allowing the creation of operators has advantages if used correctly.
2/3 looks pretty much like what’s shipped in Scala since pretty much forever, just a bit renamed, because NIH I guess.
Regarding the first one … imho, “operator overloading” is just not important enough to bother coming up with elaborate rules about how things are translated from symbols to names.
It’s kind of funny to see that there is pretty much no intersection between actual users of Scala and people obsessed with complaining about symbolic method names.
What do you mean by “the first one”?
This was the first article on this blog related to operator overloading / symbolic method names, and it wasn’t really Scala-centric. People have criticised C++ for similar reasons, and for precisely these reasons, operator overloading was never part of the JLS. Or do you mean someone else when you say “obsessed”? Or is this the usual reaction people get when they think 1-2 things are not OK with a particular language? :-)
> What do you mean by “the first one”?
I meant “Binary operations”.
> Or is this the usual reaction people get when they think 1-2 things are not OK with a particular language?
No, sorry, that wasn’t targeted against you or your blog post, just wanted to share my experience. :-) My perception is that these things are big deal for excatly those who have never used that functionality in the first place.
Scala does that too, automatically by convention? I didn’t know that! Nice.
Aha, I see. Anything can be a big deal if you have time and passion for the details :-) Some call it the bikeshed problem. Another example is JavaScript. Recently, I’ve posted this article to Reddit:
http://www.reddit.com/r/programming/comments/1xfh9w/why_you_should_always_use_and_other_bad_practices/
It’s one of my most popular posts ;-)
> Scala does that too, automatically by convention? I didn’t know that! Nice.
No, ““Array” access” and “Method calls” are pretty much identical, just with other names and no special [] operators/syntax.
I’d say that “Binary operations” is a pretty bad idea.
Pretty much every language invented on this planet has the intuitive rule of “if you name a method X, you can invoke that method using that name”.
I’m not seeing the benefit of the approach of “we have this hardcoded list of special method names X, Y and Z, which you are supposed to invoke by using foo, bar and baz”.
What’s the benefit of that?
To allow people to freely mix those names like
val a = 2
val b = 3
val c = 5
a.modAssign(c + (b.times(a) / b.minus())
?
I think the simpler approach of “Use the method name you want, if it looks like math the known precedence rules apply, done.” is better thought out.
Just imagine writing a language spec on http://confluence.jetbrains.com/display/Kotlin/Operator+overloading … I don’t want to imagine how many pages one will waste on something which should only be used very sparingly anyway.
Well, there are exceptions:
var foo = new function() {}; var bar = foo; bar('abc'); // This calls fooThis appears to be the “spec”: http://confluence.jetbrains.com/display/Kotlin/Grammar. It looks as though these math operators will only work on the same type, i.e.
A = A * A. Removing Java’s primitive type promotion is already greatly helping in formalising this type of convention.From a Specs point of view, I guess Scala can do away with processing the AST several times to first substitute what looks like math by method calls with appropriate precedence, before compiling the actual methods. But then again, this additional processing step slows down compiling…
> Well, there are exceptions:
I don’t think this is an exception, in fact most languages allow this. Note that you can still use foo(), but if you define the method “timesAssign”, you are not supposed to invoke it as “timesAssign”, but as something named differently in Kotlin.
Regarding the spec: The grammar is the least important and interesting detail of a language spec. What matters is the description of semantics and with that list of hard-coded special cases …
Regarding processing the AST in Scala: This is all handled in a single phase called parser. The reason why Scala uses multiple phases is because it makes it more easy to verify that each transformation is done correctly and allows people to more easily extend the compiler.
Anyway, looking at performance, parsing pretty much doesn’t matter at all. In a sense, pretty much no phase matters from a performance POV except typechecking and codegeneration.
There is already a new codegeneration backend which improves performance substantially, so typechecking is pretty much it, and it’s hard to optimize it, when other things (like correctness) are much more important.
I think performance of typechecking could increase considerably if Hotspot would do its job properly, instead of letting scalac run in interpreted mode for the first 2 minutes. By the time it decides to emit substantial amounts of JIT-compiled code, the compilation is likely to be done already.
Alright, thanks for all this information! I learned a couple of things
The Kotlin documentation is unclear about how somebody writing custom Kotlin code would implement a custom overloaded operator. Is there a convention that a method “plus()” would translate to “+”? If so, how is this different from Ceylon’s polymorphic operator overloading, where you need to satisfy the Summable interface?
From what I understand, Kotlin (and Groovy, Xtend) uses convention, Ceylon a formal contract. Both approaches have advantages / disadvantages. Convention is good to retrofit existing APIs, a formal contract clearly communicates the intent.
Does this mean that it’s possible to implement the “pope” operator in Kotlin?
Looks like it. But then, Kotlin is in good company:
http://ceylon-lang.org/documentation/current/introduction/#operator_polymorphism
On the contrary. It looks like you’re pretty much limited to what operators Ceylon lets you overload. This means plus()/+ for Summable, union()/| for Set, etc:
http://ceylon-lang.org/documentation/1.0/reference/#operators
That’s what I meant by “good company”. The linked doc reads:
;-)
Sorry… misunderstood you.
On a lighter note, here’s an example of operator overloading abuse :)
http://www.scala-graph.org/guides/core-initializing.html
Awesome! Note also the blatant difference between
~%#+>and~%+#>. It reads like Shakespeare.For kicks and giggles, let’s have reddit’s opinion on this:
http://www.reddit.com/r/programming/comments/1xj0zg/hardcore_scala_symbolic_method_names/
1) You can not overload operators in Scala – you can only use non asci characters in methods. That is a huge difference.
2) I have seen a lot of abuse of extends in java – should we forbid it because some people are to stupid to use it correctly?
I can see your point 2). Every tool can be abused, although “extends” clearly pulls its weight. About 1), technically you’re correct. You can use non-ascii characters and leave away syntactic elements like dots and parentheses. When I read / maintain legacy code, however, that distinction might seem a bit academic.
I feel the opposite toward operator overloading. What sense is it to take English, an old and fundamentally broken language, and intentionally pollute a new one with it?
Hi Tony, thanks for joining. Interesting point of view. Cheers, Lukas
BTW, the link to Xtend’s operator overloading page has moved to https://www.eclipse.org/xtend/documentation/203_xtend_expressions.html#operators
Thank you very much. The link is now fixed.
As opponents of operator overloading usually point out themselves, operators are essentially equivalent to functions. That should be enough to realise that operator overloading is no bigger a problem than allowing programmers to name their functions whatever they want.
If you point this out to the naysayers they usually ask instead “if you can do the same thing with normal function notation, why bother with operator overloading at all”. The reason of course is readability, a + b = s is easier to type and read than s.set(a.addition(b)) which is of course why operator notation evolved in the first place. If used correctly operator overloading would make a lot of code more concise and easier to understand.
If anyone thinks you only ever want to add integers and floats they have a very primitive view of mathematics and programming
You also often hear the somewhat rude argument that no real (e.g.) Java programmer use operator overloading… duh (well to be precise they say ‘need’ but ‘use’ is close enough and it makes the err more obvious).
Thanks for your insights. I agree with your statements, and I think the article does, too. I suspect that the “naysayers” mostly oppose the introduction of arbitrary operator-esque symbols that are hard to read. However, hardly anyone probably opposes mapping the “A + B” notation to what translates “addition”, semantically.
I personally believe that every Java programmer who has done some arithmetic with
BigDecimalwould immediately see the advantage of a simple operator overloading implementation…The caveat in BigDecimal is the need for MathContext, in particular on divisions.
How would you shove that into an operator overload? use a default? configure by IoC, neither sounds like a good solution for the problem of defining precision nor rounding modes.
This is a subject I am quite interested in because I am having some fun at attempting to add unit type-safety to numbers (what are numbers really without a unit to bind them to the real world?)
Having a default for MathContext will do. Hardly anyone is missing this feature from RDBMS’s DECIMAL / NUMERIC data types…
Small nitpick:
Core Perl doesn’t use operator overloading, but it does you dynamic typing. Operators are actually used to control the typing somewhat. For example, “+” is used to add numeric types, and “.” is used to concatenate strings. Similarly, “==” will test for numeric equivalence, and “eq” will test for string equivalence. The important concept is that since those operators are tied to types, they will implicitly convert to the appropriate type (and warn if that’s not cleanly possible) and then perform their operation.
This comes from a core principle of Perl, which is that similar things should look similar, and different things should look different. String concatenation and numeric addition really aren’t that similar, so Perl makes the distinction, between string ops and numeric ops with new operators. This helps to avoid and entire class of problems that exist in other dynamically typed languages, such as Python and Javascript, what actually do overload some operators (most notoriously the “+” and “==” operators covered above).
That said, Perl does have an Overload module to support actual operator overloading for objects, but it’s a module and not used in the core (well, there aren’t really objects used in the core of Perl 5).
Perl 6 goes a different route, and used operator overloading extensively, but it still tries to follow the principle of similar things looking similar and different things looking different, so while there are a lot of cases of operator overloading, the intention is to make it obvious what is being done. One way in which they accomplish this is by using a *lot* more symbols for unicode, which can be controversial. One example of this is set notation symbols for equivalent set operations on lists. Usefully, these are almost always represented by an equivalent method on the underlying type.
Thank you very much for the interesting insight! Indeed, using “+” for String concatenation is a mistake in so many languages, although I’ve never gotten used to the Perl/PHP “.” alternative.
Interesting, so, do set operations make use of mathematical symbols, e.g. for unions, intersections, etc.?
Yes, Perl 6 tries to make use of both unicode and multi-character equivalents (“texas-style”) of specialized notation where avaialble. For example, you can use subset/superset unicode operators, or you can use (>) and (<) as ASCII equivalents.
You can see more examples of some of the mathematical symbols that apply to sets here[1]. I think the set class contains the most obvious example of this, but combined with Perl 6's prefix, infix and postfix operators and scoped exporting, you can do some really interesting things, like so:
... code here { use Physical::Units ; # made up lib, theoretically adds postfix unit types and infix multiplication/division handling my $length = 20kilometers; my $time = 30minutes; my $velocity = $length/$time; say $velocity.to(Meter).to(Second); # 11.11m/s } my $foo = 10kilometers; # Error here, there is no postfix kilometers any morePersonally, I think that’s pretty cool (even if it doesn’t currently exist). Mini DSLs that are scoped to where they provide the most benefit so they don’t leak out and pollute the namespace.
1: ../rakudo/src/core/io_operators.pm
Oh wow, I wasn’t aware of these things. That looks really very powerful! Thanks a lot for sharing.
I do like the detail that imports can be locally scoped. Scala has that too. That’s really very useful.
I think that allowing the creation of custom operators has benefits if used correctly.
Not that someone should create a @#+-~^_?=:>! operator, but there are cases when a custom operator is better than functions or tricks to achieve something similar to making a new operator.
For example, it would be better if you could write
if (var in 1..5) {…}
instead of
if (inRange(var, new Range(1, 5))) {…}
or
if (var >= 1 && var <= 5) {…}
It's more readable and shorter.
Allowing the creation of operators has advantages if used correctly.
Yes, that’s essentially what the article says, right?