spark streaming - Compile error while calling updateStateByKey -


compile error :

the method updatestatebykey(function2<list<integer>,optional<s>,optional<s>>) in type javapairdstream<string,integer> not applicable arguments (function2<list<integer>,optional<integer>,optional<integer>>) 

in simple word count example , mapping words 1

javapairdstream<string, integer> wordcounts = words.maptopair(s -> new tuple2<>(s,1)); 

and applying updatestatebykey on wordcounts

 javapairdstream<string, integer> finalcount =  wordcounts.updatestatebykey(updatefunction); 

the updatefunction defined follows:

 final function2<list<integer>, optional<integer>, optional<integer>> updatefunction =                     new function2<list<integer>, optional<integer>, optional<integer>>() {                       @override                       public optional<integer> call(list<integer> values, optional<integer> state) {                         integer newsum = state.orelse(0);                         (integer value : values) {                           newsum += value;                         }                         return optional.of(newsum);                       }                     }; 

the updatestatebykey has following recommended signatures available:

enter image description here

please check package import using optional. spark use com.google.common.base.optional not jdk default package java.util.optional.


Comments

Popular posts from this blog

Upgrade php version of xampp not success -

c - Bitwise operation with (signed) enum value -

xslt - Unnest parent nodes by child node -