Comments on: Getting data to the cloud http://www.electric-cloud.com/blog/2009/09/14/getting-data-to-the-cloud/ This is your source for private development cloud best practices and technical tips and tricks for Electric Cloud solutions Thu, 03 May 2012 01:47:53 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Eric Melski http://www.electric-cloud.com/blog/2009/09/14/getting-data-to-the-cloud/#comment-77 Eric Melski Mon, 04 Jan 2010 18:48:54 +0000 http://blog.electric-cloud.com/?p=539#comment-77 That's an excellent question. In fact LBFS and rsync definitely share some concepts, but you can think of LBFS as going further than rsync does. With rsync, we compute a checksum for each block of a file, then ask the target, "hey, is the checksum for block 1 of file A the same as what I have computed?" If so, then rsync saves by not transferring that block. With LBFS, it actually looks for identical blocks at different locations within the same file and even across files. So on the source host we compute a checksum for chunk A-1, and then say to the target host, "hey, do you have a chunk (from any location in any file) that matches this checksum?" If a matching chunk is found, it gets stitched into file A on the target machine. This gives you the potential for even more savings than rsync can provide. Thanks for the comment! That’s an excellent question. In fact LBFS and rsync definitely share some concepts, but you can think of LBFS as going further than rsync does. With rsync, we compute a checksum for each block of a file, then ask the target, “hey, is the checksum for block 1 of file A the same as what I have computed?” If so, then rsync saves by not transferring that block.

With LBFS, it actually looks for identical blocks at different locations within the same file and even across files. So on the source host we compute a checksum for chunk A-1, and then say to the target host, “hey, do you have a chunk (from any location in any file) that matches this checksum?” If a matching chunk is found, it gets stitched into file A on the target machine. This gives you the potential for even more savings than rsync can provide.

Thanks for the comment!

]]> By: Marek http://www.electric-cloud.com/blog/2009/09/14/getting-data-to-the-cloud/#comment-76 Marek Mon, 04 Jan 2010 15:32:46 +0000 http://blog.electric-cloud.com/?p=539#comment-76 Is not the same as rsync ? http://en.wikipedia.org/wiki/Rsync Is not the same as rsync ?
http://en.wikipedia.org/wiki/Rsync

]]>