Cloud Sync to Backblaze B2

[thibaultmol]

Hi

Bacbklaze just launched B2 v1.0.
Backblaze is a company that hosts data in the cloud at a REALLY low price and with B2 you can use it like AWS storage.
I'd be great if you could use B2 to sync your Nextcloud server to Backblaze. (The usage of B2 isn't the same as AWS but it would be nice to just login and have it work in the web interface)(I know I can setup the command line tool but I feel like it'be great if I just had to enter username and password (and two step auth) like on Synology NAS's)
https://www.backblaze.com/b2/partner-synology.html

Thanks

[MariusBluem]

Ref: https://help.nextcloud.com/t/backblaze-b2-sync/

[GrahamJenkins]

I second this request. I am considering a switch from ownCloud to Nextcloud and would love for this to be a viable option. My existing server does not have expandable storage and the pricing of S3 is too high to be practical for personal use.

How difficult would this be? I'm slightly considering looking into the code for the S3 external storage, I imagine it would be very similar, if somebody would be willing to give some pointers I could try my hand at this.

Ref: https://www.backblaze.com/b2/docs/

[EnMod]

+1, I too would be ecstatic to have this as an option. Thanks for linking those docs.

[GrahamJenkins]

Does anybody know how much work this would involve or how long it might take a competent developer?

[jb510]

First, I am brand new to Nexctloud and have no idea how it implements external storage, so I am a long way from jumping in and starting work on this.

Some useful background however:
As far as I know, BackBlaze doesn't provide an official B2 API client in any language.

There are a few PHP Wrappers/Classes/SDKs/Clients in development for B2 independently though. None of those seem fully mature or widely used though. Most are labeled alpha/beta.

1. The most recent/active/popular is: https://github.com/cwhite92/b2-sdk-php
2. This one is even more popular but appears stalled: https://github.com/kamilz/b2backblaze

Other notes:
There is also a FUSE written in Python that seems mature: https://github.com/sondree/b2_fuse
If someone wanted they could probably use that to mount B2 storage directly to the server running Nextcloud and I'm guessing use it directly (again, I'm too new to NextCloud to even know for sure that's viable, but it seems like it might be).

Finally for other interested in building integrations and the requirments for being listed on the integrations page: https://www.backblaze.com/b2/docs/integrations.html

[8BitAce]

I'd definitely like to see this as well. I've been waiting since they first announced the service last year but as of yet neither OwnCloud, Pydio, nor NextCloud seem to be attempting it. (I've taken a look into implementing it myself but my lack of knowledge in PHP and owncloud's plugin system seemed insurmountable).

Some additional points though:

• No, I don't believe there are any official clients, but the API is a rather simple REST-based one with similarities to S3.
• The FUSE implementation is less than ideal as the developer notes it is not production-ready and includes numerous limitations (concurrency issues, slow performance with > 1000 files)

I've been surprised there hasn't been more interest in adding this to one of the open source cloud solutions. Is there perhaps some guidance for how someone like myself could implement this? I imagine it would largely be the same as the S3 implementation.

Edit: I'm taking a look and I think I can take a stab at this.

[WebSpider]

👍 this would be yet another compelling reason for me to finally make that switch from oc to nc

[tommyent]

👍

[jakimfett]

I'd like to add my +1 to this, and request that two factor auth be supported.

[MariusBluem]

Please stop 👍 such feature requests ... instead of this - you may want to develop or find somebody who wants to do make this real 😜 ... you may also want to consider about posting a bug bounty on it:

http://bountysource.com/teams/nextcloud/issues

In this issue, we should only discuss this feature technically and should not tell each other how much we want it 😅 THX. However: Thanks for your interest in this topic. If you want to react to specific issues or pull requests on GitHub, you can also use reactions (https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comments) 😉

[despens]

https://www.bountysource.com/issues/35689072-cloud-sync-to-backblaze-b2

[jasonehines]

I just ran across this: https://github.com/ncw/rclone/ It has a mount feature I plan on trying with my nextcloud

[dv-anomaly]

It would be nice to see full support for b2 as a regular backend (external storages & primary storage). I've tried using s3proxy, but nextcloud fails to create files in the bucket. I assume it's probably related to some of the limitations listed below. b2 works a bit differently if you take a look at the protocol.

• bucket names must be between 6 and 50 characters and consistent of only letters, number, and hyphens
• does not return ETag as an MD5 hash, instead returning SHA1 hash
• does not support blob-level access control
• does not support conditional GET
• does not support copying objects
• does not support Content-Disposition, Content-Encoding, Content-Language, Content-MD5, or Expiry headers
• multipart upload requires at least 2 parts

For those that just need a basic implementation this fork of s3ql does seem to work well. It can be a bit of a chore to get it compiled, and it makes it almost impossible to scale horizontally. For deployments that don't need to load balance a handful of front-ends s3ql will get the job done. It is better than most fuse based s3 file systems as it will cache to disk, or in my case an SSD array.

s3ql/s3ql#8
https://github.com/sylvainlehmann/s3ql