{"id":254,"date":"2010-06-22T18:05:29","date_gmt":"2010-06-23T01:05:29","guid":{"rendered":"http:\/\/infinitedisorder.com\/?p=254"},"modified":"2010-06-24T13:43:25","modified_gmt":"2010-06-24T20:43:25","slug":"build-a-backup-server-with-opendedup","status":"publish","type":"post","link":"https:\/\/infinitedisorder.com\/?p=254","title":{"rendered":"build a backup server with OpenDeDup"},"content":{"rendered":"<p>With this server you can backup large amounts of data (like VMWare VCB backups) and all duplicate data will be ignored. This will usually make 10 TB of backups less than 1TB.<\/p>\n<p>First calculate how much memory you&#8217;ll need using this calculation:<br \/>\n&lt;Size of your DeDup Volume in MB&gt; \/ &lt;blocksize (4 for VMWare images) * 33 = Amount of memory needed in MB<\/p>\n<p>Install ubuntu 10.04 server with openssh server and samba server<\/p>\n<p>scp jdk-7-ea-bin-b98-linux-x64-17_jun_2010.bin.sh root@dedup:~\/<br \/>\nchmod 755 jdk-7-ea-bin-b98-linux-x64-17_jun_2010.bin.sh<br \/>\n.\/jdk-7-ea-bin-b98-linux-x64-17_jun_2010.bin.sh<br \/>\nmv jdk1.7.0 \/opt\/<br \/>\nadd this to ~\/.bashrc &#8220;export JAVA_HOME=\/opt\/jdk1.7.0\/jre<br \/>\nsource ~\/.bashrc<\/p>\n<p>sdfs-latest.tar.gz (0.9.2 at the moment)<br \/>\ntar -xvzf sdfs-latest.tar.gz<\/p>\n<p>Create your volume<br \/>\n.\/mkfs.sdfs &#8211;volume-name=sdfs_vol1 &#8211;volume-capacity=800GB &#8211;io-chunk-size=4 &#8211;io-max-file-write-buffers=150<\/p>\n<p>Mount your volume<br \/>\nchange -Xmx2g to -Xmx8g for 8GB of memory (modify mount.sdfs)<br \/>\n.\/mount.sdfs -v sdfs_vol1 -m \/dedup<br \/>\nIf it hangs on &#8220;Mounting Filesystem&#8221; then you are in good shape, do some copies to \/dedup to make sure java isn&#8217;t going to crash. If you don&#8217;t see any error messages then its working.<\/p>\n<p>The cache will be located at \/opt\/sdfs\/<\/p>\n<p>OpenDeDup is a very new project, follow it at http:\/\/opendedup.org\/<\/p>\n<p>Now to setup the samba share, add this to \/etc\/samba\/smb.conf<\/p>\n<div id=\"_mcePaste\">[dedup]<\/div>\n<div id=\"_mcePaste\">comment = dedup<\/div>\n<div id=\"_mcePaste\">path = \/dedup<\/div>\n<div id=\"_mcePaste\">valid users = root<\/div>\n<div id=\"_mcePaste\">public = no<\/div>\n<div id=\"_mcePaste\">writable = yes<\/div>\n<div id=\"_mcePaste\">write list = root<\/div>\n<div id=\"_mcePaste\">printable = no<\/div>\n<div id=\"_mcePaste\">create mask = 0765<\/div>\n<div>smbpasswd -a root<br \/>\nservice smbd restart<\/div>\n<div>You can verify the dedup is working by comparing du -sh to ls -alh or df -h<\/div>\n","protected":false},"excerpt":{"rendered":"<p>With this server you can backup large amounts of data (like VMWare VCB backups) and all duplicate data will be ignored. This will usually make 10 TB of backups less than 1TB. First calculate how much memory you&#8217;ll need using this calculation: &lt;Size of your DeDup Volume in MB&gt; \/ &lt;blocksize (4 for VMWare images) [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-254","post","type-post","status-publish","format-standard","hentry","category-software"],"_links":{"self":[{"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=\/wp\/v2\/posts\/254","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=254"}],"version-history":[{"count":6,"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=\/wp\/v2\/posts\/254\/revisions"}],"predecessor-version":[{"id":260,"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=\/wp\/v2\/posts\/254\/revisions\/260"}],"wp:attachment":[{"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=254"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=254"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infinitedisorder.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=254"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}