All SQL databases support the standard aggregate functions
COUNT(),
SUM(),
AVG(),
MIN(),
MAX().
Some databases support other aggregate functions, like:
EVERY()STDDEV_POP()STDDEV_SAMP()VAR_POP()VAR_SAMP()ARRAY_AGG()STRING_AGG()
But what if you want to roll your own?
Java 8 Stream Collector
When using Java 8 streams, we can easily roll our own aggregate function (i.e. a
Collector). Let’s assume we want to find the second highest value in a stream. The highest value can be obtained like this:
System.out.println(
Stream.of(1, 2, 3, 4)
.collect(Collectors.maxBy(Integer::compareTo))
) ;
Yielding:
Optional[4]
Now, what about the second highest value? We can write the following collector:
System.out.println(
Stream.of(1, 6, 2, 3, 4, 4, 5).parallel()
.collect(Collector.of(
() -> new int[] {
Integer.MIN_VALUE,
Integer.MIN_VALUE
},
(a, i) -> {
if (a[0] < i) {
a[1] = a[0];
a[0] = i;
}
else if (a[1] < i)
a[1] = i;
},
(a1, a2) -> {
if (a2[0] > a1[0]) {
a1[1] = a1[0];
a1[0] = a2[0];
if (a2[1] > a1[1])
a1[1] = a2[1];
}
else if (a2[0] > a1[1])
a1[1] = a2[0];
return a1;
},
a -> a[1]
))
) ;
It doesn’t do anything fancy. It has these 4 functions:
- Supplier<int[]>: A supplier that provides an intermediary
int[] of length 2, initialised with Integer.MIN_VALUE, each. This array will remember the MAX() value in the stream at position 0 and the SECOND_MAX() value in the stream at position 1
- BiConsumer<int[], Integer>: A accumulator that accumulates new values from the stream into our intermediary data structure.
- BinaryOperator<int[]>: A combiner that combines two intermediary data structures. This is used for parallel streams only.
- Function<int[], Integer>: The finisher function that extracts the
SECOND_MAX() function from the second position in our intermediary array.
The output is now:
5
How to do the same thing with SQL?
Many SQL databases offer a very similar way of calculating custom aggregate functions. Here’s how to do the exact same thing with…
Oracle:
With the usual syntactic ceremony…
CREATE TYPE u_second_max AS OBJECT (
-- Intermediary data structure
MAX NUMBER,
SECMAX NUMBER,
-- Corresponds to the Collector.supplier() function
STATIC FUNCTION ODCIAggregateInitialize(sctx IN OUT u_second_max) RETURN NUMBER,
-- Corresponds to the Collector.accumulate() function
MEMBER FUNCTION ODCIAggregateIterate(self IN OUT u_second_max, value IN NUMBER) RETURN NUMBER,
-- Corresponds to the Collector.combineer() function
MEMBER FUNCTION ODCIAggregateMerge(self IN OUT u_second_max, ctx2 IN u_second_max) RETURN NUMBER,
-- Correspodns to the Collector.finisher() function
MEMBER FUNCTION ODCIAggregateTerminate(self IN u_second_max, returnValue OUT NUMBER, flags IN NUMBER) RETURN NUMBER
)
/
-- This is our "colletor" implementation
CREATE OR REPLACE TYPE BODY u_second_max IS
STATIC FUNCTION ODCIAggregateInitialize(sctx IN OUT u_second_max)
RETURN NUMBER IS
BEGIN
SCTX := U_SECOND_MAX(0, 0);
RETURN ODCIConst.Success;
END;
MEMBER FUNCTION ODCIAggregateIterate(self IN OUT u_second_max, value IN NUMBER) RETURN NUMBER IS
BEGIN
IF VALUE > SELF.MAX THEN
SELF.SECMAX := SELF.MAX;
SELF.MAX := VALUE;
ELSIF VALUE > SELF.SECMAX THEN
SELF.SECMAX := VALUE;
END IF;
RETURN ODCIConst.Success;
END;
MEMBER FUNCTION ODCIAggregateTerminate(self IN u_second_max, returnValue OUT NUMBER, flags IN NUMBER) RETURN NUMBER IS
BEGIN
RETURNVALUE := SELF.SECMAX;
RETURN ODCIConst.Success;
END;
MEMBER FUNCTION ODCIAggregateMerge(self IN OUT u_second_max, ctx2 IN u_second_max) RETURN NUMBER IS
BEGIN
IF CTX2.MAX > SELF.MAX THEN
SELF.SECMAX := SELF.MAX;
SELF.MAX := CTX2.MAX;
IF CTX2.SECMAX > SELF.SECMAX THEN
SELF.SECMAX := CTX2.SECMAX;
END IF;
ELSIF CTX2.MAX > SELF.SECMAX THEN
SELF.SECMAX := CTX2.MAX;
END IF;
RETURN ODCIConst.Success;
END;
END;
/
-- Finally, we have to give this aggregate function a name
CREATE FUNCTION SECOND_MAX (input NUMBER) RETURN NUMBER
PARALLEL_ENABLE AGGREGATE USING u_second_max;
/
We can now run the above on the
Sakila database:
SELECT
max(film_id),
second_max(film_id)
FROM film;
To get:
MAX SECOND_MAX
------------------
1000 999
And what’s even better, we can use the aggregate function as a window function for free!
SELECT
film_id,
length,
max(film_id) OVER (PARTITION BY length),
second_max(film_id) OVER (PARTITION BY length)
FROM film
ORDER BY length, film_id;
The above yields:
FILM_ID LENGTH MAX SECOND_MAX
---------------------------------
15 46 730 505
469 46 730 505
504 46 730 505
505 46 730 505
730 46 730 505
237 47 869 784
247 47 869 784
393 47 869 784
398 47 869 784
407 47 869 784
784 47 869 784
869 47 869 784
2 48 931 866
410 48 931 866
575 48 931 866
630 48 931 866
634 48 931 866
657 48 931 866
670 48 931 866
753 48 931 866
845 48 931 866
866 48 931 866
931 48 931 866
Beautiful, right?
PostgreSQL
PostgreSQL supports a slightly more concise syntax in the
CREATE AGGREGATE statement. If we don’t allow for parallelism, we can write this minimal implementation:
CREATE FUNCTION second_max_sfunc (
state INTEGER[], data INTEGER
) RETURNS INTEGER[] AS
$$
BEGIN
IF state IS NULL THEN
RETURN ARRAY[data, NULL];
ELSE
RETURN CASE
WHEN state[1] > data
THEN CASE
WHEN state[2] > data
THEN state
ELSE ARRAY[state[1], data]
END
ELSE ARRAY[data, state[1]]
END;
END IF;
END;
$$ LANGUAGE plpgsql;
/
CREATE FUNCTION second_max_ffunc (
state INTEGER[]
) RETURNS INTEGER AS
$$
BEGIN
RETURN state[2];
END;
$$ LANGUAGE plpgsql;
CREATE AGGREGATE second_max (INTEGER) (
SFUNC = second_max_sfunc,
STYPE = INTEGER[],
FINALFUNC = second_max_ffunc
);
Here, we use the
STYPE (
Collector.supplier()), the
SFUNC (
Collector.accumulator()), and the
FINALFUNC (
Collector.finisher()) specifications.
Other databases
Many other databases allow for specifying user defined aggregate functions. Look up your database manual’s details to learn more. They always work in the same way as a Java 8
Collector.
Like this:
Like Loading...
