spark streaming - Compile error while calling updateStateByKey -


compile error :

the method updatestatebykey(function2<list<integer>,optional<s>,optional<s>>) in type javapairdstream<string,integer> not applicable arguments (function2<list<integer>,optional<integer>,optional<integer>>) 

in simple word count example , mapping words 1

javapairdstream<string, integer> wordcounts = words.maptopair(s -> new tuple2<>(s,1)); 

and applying updatestatebykey on wordcounts

 javapairdstream<string, integer> finalcount =  wordcounts.updatestatebykey(updatefunction); 

the updatefunction defined follows:

 final function2<list<integer>, optional<integer>, optional<integer>> updatefunction =                     new function2<list<integer>, optional<integer>, optional<integer>>() {                       @override                       public optional<integer> call(list<integer> values, optional<integer> state) {                         integer newsum = state.orelse(0);                         (integer value : values) {                           newsum += value;                         }                         return optional.of(newsum);                       }                     }; 

the updatestatebykey has following recommended signatures available:

enter image description here

please check package import using optional. spark use com.google.common.base.optional not jdk default package java.util.optional.


Comments

Popular posts from this blog

python - Healpy: From Data to Healpix map -

c - Bitwise operation with (signed) enum value -

xslt - Unnest parent nodes by child node -