hadoop - Fail to write SequenceFile with Pig -

-

I want to store some pig variables in Hadoop SequenceFile so that we can remove jobs to run an external map.

Assume that my data has a charrray (int) schema: 

  (Hello, 1) (test, 2) (example, 3)

I wrote this storage work: 

  import java.io.IOException; Import java.util.logging.Level; Import java.util.logging.Logger; Import org.apache.hadoop.fs.Path; Import org.apache.hadoop.io.IntWritable; Import org.apache.hadoop.io.Text; Import org.apache.hadoop.mapreduce.Job; Import org.apache.hadoop.mapreduce.OutputFormat; Import org.apache.hadoop.mapreduce.RecordWriter; Import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; Import org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat; Import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; Import org.apache.pig.StoreFunc; Import org.apache.pig.data.Tuple; Public Square StoreTest StoreFunc {extends private string store location; Private recordware author; Private job job; Public StoreTest () {} @ Override public output format getOutputFormat () throws IOException {// back to new TextOutputFormat (); Return new sequence file auto-format (); } @ Override throws public zero set store location (string location, job job) IOException {this.storeLocation = location; This.job = job; System.out.println ("Load Location" + Store Location); FileOutputFormat.setOutputPath (job, new path (location)); System.out.println ("Out Path" + FileOutputFormat.getOutputPath (Job)); } @ Override Public Recording Ready: Right Writer (Recordware Writer) throws IOException {this.writer = writer; } @ Override throws IOException to public view planlive (Tupal Tupal) {try {text k = new text (String) tuple.get (0))); IntWritable v = New IntWritable (integer) tuple.get (1)); Author. Write (k, v); } Hold (Prerupted Preset) {LoggerGetLugger (StoreTest.class.getName ()). Logs (level. SESEE, ANNEL, X); }}}

and this pig code: 

  register MyudFs.jar; X = Load '/ User / Pinoli / Input' (A: Chararey, B: IT); Use '/ user / pinoli / output' / 'Store Test' in Store X; However, storage fails and I get this error:  
  error org.apache.pig.tools.pigstats.PigStats - Error 0: Java Io.IOException: Incorrect key class: org.apache.hadoop.io.Text class is not org.apache.hadoop.io.LongWritable  
 
 Is there any way to fix it? ?

The problem is that you have not set the output key / value class. You can do this in the setStoreLocation () method: 

  @Override throws public zero set store location (string location, job job) IOException {this.storeLocation = location; This.job = job; This.job.setOutputKeyClass (Text.class); // !!! This.job.setOutputValueClass (IntWritable.class); // !!! ...}

I think you want to use your store with different key / price types. In that case you can do the constructor with your type. Example: 

  Private class

Then, set the correct type in pig script: 

  register MyudFs.jar; Set MyStore Storetest ('org.apache.hadoop.io.Text', 'org.apache.hadoop.io.IntWritable'); X = Load '/ User / Pinoli / Input' (A: Chararey, B: IT); Store 'X' using '/ user / pinoli / output /' mystorer ();

Comments

Post a Comment

Popular posts from this blog

java - Can't add JTree to JPanel of a JInternalFrame -

-
Image
There are two panels in my JInternalframe. I want to add TopPanel to Jtree to nominate another. But I can not add Jaitari to the top of the panel. Please help me in this piece of code: DefaultMutableTreeNode root = new DefaultMutableTreeNode ("deck"); DefaultMutableTreeNode Item Clubs = New DefaultMutableTreeNode ("Club"); AddAllCard (itemClubs); Root.add (itemClubs); DefaultMutableTreeNode item = New DefaultMutableTreeNode ("Diamonds"); AddAllCard (itemDiamonds); Root.add (itemDiamonds); DefaultMutableTreeNode Itemspads = New DefaultMutableTreeNode ("Spades"); AddAllCard (itemSpades); Root.add (itemSpades); DefaultMutableTreeNode item increases = new DefaultMutableTreeNode ("heart"); AddAllCard (itemHearts); Root.add (itemHearts); Default Trimodel Tree Model = New DefaultTreeModel (Route); Tree = new jetty (tree modal); ScrollPane = new JScrollPane (tree); // scrollPane.setViewportView (tree); . ScrollPane.getViewport () add (tr
Read more

javascript - data.match(var) not working it seems -

-
OK, I have this small block code, which is to scanned a number on the website, but for some reason There is a hip that it is not scanning at all. var regenges = / \ & lt; Span class = "currency-robux" data-to-"item-private-value-price" & gt; ([\ D,]) \ & LT; \ / Span \ & gt; /; Var ValueCiling = Data. Match (reggae); Value-song = number (value slicing location (",", "")); Is there anything wrong with that? This is related to the if statement if (value sling and lieutenant; = price watching) { < P> A function calls to move from there, but for some reason it does not seem to run at all. So I think the Regex is wrong but I'm not sure how. (There is a variable in PriceWinking , this is a snippet from the code.) On the website, I'm trying to remove it. & lt; Span class = "robux" data-plus = "item-privatesale-price" & gt; 115 & lt; / Span & gt; Keep in mind
Read more

javascript - How can I pause a jQuery .each() loop, while waiting for user input? -

-
Before I continue to get a jQuery .each loop to wait for user input There is a problem in repeatedly repeating This is my code (I have commented on what I would like to do): & lt; Html xmlns = "http://www.w3.org/1999/xhtml" & gt; & Lt; Top & gt; & Lt; Link rel = "stylesheet" type = "text / css" href = "common_components.css" /> & Lt; Meta http-equiv = "content-type" content = "text / html; charset = UTF-8" /> & Lt; Link rel = "stylesheet" href = "slider.css" & gt; & Lt ;? Php include_once 'common_components.php'; $ Query = $ dbh- & gt; Queries ("SELECT movies_dir FROM settings"); $ Line = $ query- & gt; Obtain (PDO :: FETCH_ASSOC); Date_default_timezone_set ('UTC'); Echo "& lt; title & gt; movie scanner & lt; / title & gt;"; ? & Gt; & Lt; Script src = "jquery.min.js" & gt; &am
Read more