Updating many json files in Azure blob storage? -

-

I have ~ 1,000,000 JSON files, I would like to update every 30 minutes. Update is only enclosing a new array at the end of the existing content.

Uses the same code as a single update: 

  CloudBlockBlob blockBlob = container.GetBlockBlobReference (blobName); Jobbject jObject = null; // If the blob is present, then we may need to update it. If (blockBlob.Exists ()) {MemoryStream memoryStream = new MemoryStream (); BlockBlob.DownloadToStream (MemoryStream); JObject = JsonConvert.DeserializeObject (encoding.UTF8.GetString (MemoryStream. ToArray ()) functioning; } // End of Blease is present if (null == jObject) {jObject = new Jobbject (); JObject.Add (new JProperty ("identifier", identifier)); } // Closing of the block did not exist JArray jsonArray = new JArray (); JObject.Add (New JProperty (string.Format ("Entries {0}", Timestamp. ToString ()), jsonArray); Foreign Currency (Miscellaneous Entries in New Entry) {JSNARRA.ed (new JobTech (new JPPrint ("Some ID", Entry.), New JProperty ("Some Value", Value)); } // the end of the loop string jsonString = JsonConvert.SerializeObject (jObject); // upload blockblock Properties Type Content = "Text / Jason"; Blockblock UploadFomstream (new memorystream (encoding.utf 8.gatebites (jasonstring)); Basically:  
     
  1.  Check if the blob exists,  
  2.  If this happens, download the data and create a Jason object.  
  3.  If it has not, create a new object with the details.  
  4.  Push the blob update.   
     Problems with it is performance demonstrations I have done a lot that I can do to increase the performance (updates run in five parallel threads and I use  ServicePointManager.UseNagleAlgorithm  Is set to false. 
     
     It still runs at a slow pace, however, approximately 100,000 updates can take up to one hour. 
     
     So I was basically guessed , My question will be: 
     
       
    1.  Do I use blue blob for this 
      2.  
    2.  If yes, any suggestion for improving performance? 
      3.  
     
     Note: the original in the file There is a history of events and I can not re-generate the full file based on the existing data. Therefore, content is downloaded prior to updating.


Comments

Post a Comment

Popular posts from this blog

mysql - How to enter php data into a html multiple select box -

-
I am working on a page that will create a team. I just want to create a selection box which will list all the users of all the users. After that, the user can select more than one friend (i.e. 5) and can send the request to these 5 people to join the team but I am having problems I did this: Select $ sql = "Select i. *, Join * Connect with M * m.id = i.clique_id where adder_id =: id"; $ Stmt = $ db- & gt; Query ($ sql, array ('id' = & gt; $ _SESSION ['id'])); While ($ record = $ stmt-> fetch ()) {$ surname = $ record ['surname']; } and this: & lt; Select multiple = "true" name = "category" well id = "category" name = "category" category = "" & gt; & Lt; Option value = "& lt ;? php echo $ surname ;? & gt;" & Gt; & Lt ;? Php ek $ nickname; ? & Gt; & Lt; / Options & gt; & Lt; / Select & gt; But only a friend's nickname...
Read more

java - Can't add JTree to JPanel of a JInternalFrame -

-
Image
There are two panels in my JInternalframe. I want to add TopPanel to Jtree to nominate another. But I can not add Jaitari to the top of the panel. Please help me in this piece of code: DefaultMutableTreeNode root = new DefaultMutableTreeNode ("deck"); DefaultMutableTreeNode Item Clubs = New DefaultMutableTreeNode ("Club"); AddAllCard (itemClubs); Root.add (itemClubs); DefaultMutableTreeNode item = New DefaultMutableTreeNode ("Diamonds"); AddAllCard (itemDiamonds); Root.add (itemDiamonds); DefaultMutableTreeNode Itemspads = New DefaultMutableTreeNode ("Spades"); AddAllCard (itemSpades); Root.add (itemSpades); DefaultMutableTreeNode item increases = new DefaultMutableTreeNode ("heart"); AddAllCard (itemHearts); Root.add (itemHearts); Default Trimodel Tree Model = New DefaultTreeModel (Route); Tree = new jetty (tree modal); ScrollPane = new JScrollPane (tree); // scrollPane.setViewportView (tree); . ScrollPane.getViewport () add (tr...
Read more

c++ - Cassandra datastax cpp driver - avoiding unnecessary copies -

-
I use the following method to create my insert statement for Cassandra (using datastax cpp driver) : ssStmt1 & lt; & Lt; "Insert some table (...) value (....)" string sStatement1 = ssStmt1.str (); Castestring cass1 = cass_string_init (sStatement1.c_str ()); CassStatement * pStmt1 = cass_statement_new (cass1, 0); I have also tried the following: CassString cass1 = cass_string_init ("Enter some table (id, date, c) value (? ,?,?); "); CassStatement * pStmt1 = cass_statement_new (cass1, 3); Cass_statement_bind_uuid (pStmt1, 0, uuidKey); Cass_statement_bind_int64 (PSTMT1, 1, timestamp); Cass_statement_bind_string (pStmt1, 2, sSomething); The first pattern works with mixed keys, the second is not. The key has a timestamp and a UUID. Timestamp is one of two errors in the field (based on the used bond statement). If bound in this way: cass_statement_bind_int64_by_name (pStmt1, "date", timestamp); I get the following error: Inva...
Read more