java - combine two JavaPairRDD -

-

मैं स्पार्क में java के साथ काम कर रहा हूं। मेरे पास एक JavaPairRDD Out1

आउट 1: 

  IDCLIENT | INFO | 1 | एक | 1 | C | 1 | H | 5 | R | 2 | बी |

और मैं Out1 के नाम पर Out1 नाम वाला एक नया JavaPairRDD बनाना चाहता हूं पहली पंक्ति:

आउट2: 

  IDCLIENT2 | INFO | 1 | C | 1 | H | 5 | R | 2 | बी |

इसके बाद, मैं इन दो JavaPairRDD को इस तरह से जोड़ना चाहता हूं:

आउट3: 

  IDCLIENT | जानकारी | IDCLIENT2 | 1 | ए, सी | 1 | 1 | सी, एच | 1 | 1 | एच आर | 5 | 5 | आर, बी | 2 | 2 | बी | |

नोट: हम groupByKey का उपयोग नहीं कर सकते क्योंकि हमारे पास एक से अधिक पंक्ति में एक ही कुंजी हो सकती है।

मैं चाहता हूं इसे zipWithIndex का उपयोग करके और सूचकांक द्वारा जुड़ने से संपर्क करें। यह बहुत भारी है लेकिन काम पूरा हो जाएगा: 

  val indexed1 = out1.zipWithIndex val indexed2 = indexed1.map {(k, v) = & gt; ((K-1), v)} वेल शामिल = इंडेक्टेड 1 में शामिल इंडेक्टेड 2 वैल out3 = join.map {case (k, (v1, v2)) = & gt; प्रारूप (v1, v2)} // जहां प्रारूप को वांछित आउटपुट लेआउट में मान मिलते हैं

यदि डेटासेट मेमोरी में फिट होगा, तो मैं सिर्फ एक सादा स्कला 'oneliner' करूँगा: 

  out1.zip (out1.drop (1))। Map {case (o1, o2) = & gt; प्रारूप (ओ 1, ओ 2)}

Comments

Post a Comment

Popular posts from this blog

java - Can't add JTree to JPanel of a JInternalFrame -

-
Image
There are two panels in my JInternalframe. I want to add TopPanel to Jtree to nominate another. But I can not add Jaitari to the top of the panel. Please help me in this piece of code: DefaultMutableTreeNode root = new DefaultMutableTreeNode ("deck"); DefaultMutableTreeNode Item Clubs = New DefaultMutableTreeNode ("Club"); AddAllCard (itemClubs); Root.add (itemClubs); DefaultMutableTreeNode item = New DefaultMutableTreeNode ("Diamonds"); AddAllCard (itemDiamonds); Root.add (itemDiamonds); DefaultMutableTreeNode Itemspads = New DefaultMutableTreeNode ("Spades"); AddAllCard (itemSpades); Root.add (itemSpades); DefaultMutableTreeNode item increases = new DefaultMutableTreeNode ("heart"); AddAllCard (itemHearts); Root.add (itemHearts); Default Trimodel Tree Model = New DefaultTreeModel (Route); Tree = new jetty (tree modal); ScrollPane = new JScrollPane (tree); // scrollPane.setViewportView (tree); . ScrollPane.getViewport () add (tr...
Read more

python - Calculate STD manually using Groupby Pandas DataFrame -

-
Image
I was trying to write a solution, which is a different and a manual for calculating a mean and STD The way. I have created a = ["Apple", "Banana", "Cherry", "Apple"] B = [3,4,7,3] C = [5,4,1,4] D = [7,8,3,7] Pd DF = PD to import pandals DataFrame (index = class (4), column = list ("ABCD")) DF ["A"] = A DF ["B"] = BDF ["C"] = C DF ["D"] = D Again, I made a list of A duplication. Then I went through the group all the time of the objects and calculated the solution. import as np l = list (set (df.A)) df.groupby ('A', As_index = False) listMean = [0] * len (df.C) ListSTD = [0] * L in the LAN (df.C) X: s = np.mean (df [df ['A'] == x] for C =.) = Z = [index for index, enumerate In the object (df ['a']. Value] x == item i for z: listMean [i] = s in: s = np.std (df [df ['a'] == X] .cvalues) z = index for index, enumerate item (df ['a']. Value) if x ==...
Read more

java - @PluginTarget(Node.class) not registering as neo4j plugin -

-
Simple class below will not show as a plugin package com.danmacias neoneoteplugin ; Import java.util.ArrayList; Import java.util.list; Import org.neo4j.graphdb. *; Import org.neo4j.server.plugins *; Import org.neo4j.graphalgo *; // import org.neo4j.tooling.GlobalGraphOperations; @ Description (expansion of Neo4j server to achieve all nodes or relationships) Public class GetPaths expanded the server plugin {@ name ("breadth_first_paths") @ Description ("first wide travel traversal of specific node"). @plugin target (node ​​class) public shaky & lt; Path & gt; GetPaths (@ Source Node Graphidi) {ArrayList & lt; Path & gt; Path = new arreelist & lt; & Gt; (); Return path; }} curl return {"extension": {"GetAll": {"get_all_nodes": "http: // localhost: 7474 / db / ext / GetAll / graphdb / get_all_nodes "} ...}, But when I Node.class both conditions GraphDatabaseService I change with Class ,...
Read more