spark streaming - Compile error while calling updateStateByKey -
compile error :
the method updatestatebykey(function2<list<integer>,optional<s>,optional<s>>) in type javapairdstream<string,integer> not applicable arguments (function2<list<integer>,optional<integer>,optional<integer>>)
in simple word count example , mapping words 1
javapairdstream<string, integer> wordcounts = words.maptopair(s -> new tuple2<>(s,1));
and applying updatestatebykey
on wordcounts
javapairdstream<string, integer> finalcount = wordcounts.updatestatebykey(updatefunction);
the updatefunction
defined follows:
final function2<list<integer>, optional<integer>, optional<integer>> updatefunction = new function2<list<integer>, optional<integer>, optional<integer>>() { @override public optional<integer> call(list<integer> values, optional<integer> state) { integer newsum = state.orelse(0); (integer value : values) { newsum += value; } return optional.of(newsum); } };
the updatestatebykey has following recommended signatures available:
please check package import using optional. spark use com.google.common.base.optional not jdk default package java.util.optional.
Comments
Post a Comment