3 Splitting Data with Content-Defined Chunking

blog.gopheracademy.com posted by kenny 13 days ago  

In this post you’ll learn what Content-Defined Chunking (CDC) is and how you can use it to split large data into smaller blocks in a deterministic way. These blocks can be found again in other data later, even if the location of the block is different than the first time. I wrote a small Go package to do the splitting, which performs really well.

Register to comment or vote on this story