Team-BHP > Shifting gears > Gadgets, Computers & Software
Register New Topics New Posts Top Thanked Team-BHP FAQ


Reply
  Search this Thread
15,559 views
Old 25th March 2024, 13:00   #16
Newbie
 
Light's Avatar
 
Join Date: Jun 2018
Location: Kottayam
Posts: 10
Thanked: 40 Times
Re: DIY : My own, personal cloud storage

Quote:
Originally Posted by maverick029 View Post
Files are stored as is object storage is a fancy word, there is no conversion necessary, you can use files as is and it can sync with desktop as well.
That's cool, which program do you recommend to sync with S3-compatible storage on desktop and mobile?

There is usually some magic going on with the frontend program to display the objects as a directory tree.

We use Ceph at work, it's fundamentally a distributed, replicated object store that exposes it as block, file, and S3-compatible object storage.
Light is offline  
Old 25th March 2024, 17:14   #17
BHPian
 
Join Date: May 2023
Location: Pune
Posts: 43
Thanked: 154 Times
Re: DIY : My own, personal cloud storage

Quote:
Originally Posted by Light View Post
That's cool, which program do you recommend to sync with S3-compatible storage on desktop and mobile?

There is usually some magic going on with the frontend program to display the objects as a directory tree.

We use Ceph at work, it's fundamentally a distributed, replicated object store that exposes it as block, file, and S3-compatible object storage.
We use ceph with k8s rather than with s3, the use case being discussed here is more aws datasync than ceph.

Though curious what use case did you have to use ceph with s3, are you utilizing it in any sort of private cloud infra?
maverick029 is offline  
Old 25th March 2024, 20:54   #18
Newbie
 
Light's Avatar
 
Join Date: Jun 2018
Location: Kottayam
Posts: 10
Thanked: 40 Times
Re: DIY : My own, personal cloud storage

Quote:
Originally Posted by maverick029 View Post
the use case being discussed here is more aws datasync than ceph.
I'm not very familiar with AWS, but I think I understand what you're suggesting, as in OP would need to use AWS DataSync to backup their local filesystems to S3. In this case DataSync acts as the intermediary/frontend to the S3 object storage.

Quote:
Originally Posted by maverick029 View Post
We use ceph with k8s rather than with s3.
Though curious what use case did you have to use ceph with s3, are you utilizing it in any sort of private cloud infra?
Ah, you would be using Ceph RBD (block storage) or CephFS (filesystem) for your k8s persistent storage needs.

We use CephFS to store massive amounts of emails in maildir format. Not private cloud, but our own ceph cluster using 8 bare metal servers (as of now).

The cool thing about Ceph is we can use object storage and block storage anytime without additional resource provisioning, like the filesystem they're simply interfaces to the same underlying object store.
Light is offline   (1) Thanks
Old 28th March 2024, 03:12   #19
BHPian
 
Join Date: Mar 2017
Location: Kolkata
Posts: 556
Thanked: 2,004 Times
Re: DIY : My own, personal cloud storage

Hello,

Nice setup and thanks for taking the time and effort to write it down for all.

Below are my personal opinion about this setup,

As already suggested by many, software like Nextcloud is more suited for this kind of operations.

Secondly, tailscale creates one major drawback, no public access without sharing your network with every individual you wish to share data with.

I have very similar setup running like yours. But with Ubuntu + Nextcloud + OpenVPN Tunnel (via AWS or Oracle Cloud) offering direct public access without any VPN requirement at client side.

Clients can access the server / storage via any device with https connection with browser or Nextcloud Application for auto backup or syncing.

If you wish, you may take a look here -> https://www.team-bhp.com/forum/gadge...ml#post5228017 (Cloud File Sync & Backup : Google Drive vs Dropbox vs OneDrive vs?)

Thanks.
NaXal is offline   (1) Thanks
Old 31st March 2024, 18:51   #20
Newbie
 
Join Date: Apr 2023
Location: Bengaluru
Posts: 2
Thanked: 9 Times
Re: DIY : My own, personal cloud storage

Wow!!

I have been thinking about how to store my own data in a more secure and centralized way instead of just copying to a new hard drive disk every time.

This is really informative and a lot of great DIY ideas and gives me a starting point to follow and build something of my own.

Thank you for sharing
vn_92 is offline   (1) Thanks
Old 3rd April 2024, 12:23   #21
BHPian
 
Join Date: Aug 2011
Location: Bangalore
Posts: 33
Thanked: 59 Times
Re: DIY : My own, personal cloud storage

Just in case some one wants to tread this path:

How to Setup a Raspberry Pi Nextcloud Server - Pi My Life Up - One of the best step by step guide to setting up Nextcloud on a Raspberry Pi, includes separate topic on installing MySQL on the same Raspberry Pi. This is my go to page when ever my NC crashes.

Also be aware, your USB drive (with the OS & NC install) will fail. Keep your data separate and away from the USB where your OS & NC is installed. Invest in a 2TB USB powered HDD (mine is Toshiba) and you can keep your data safe.

I run my NextCloud on a RaspI4 2GB Ram with 3 users & multiple mobiles,laptops, and PCs. NC also is my repository for all my contacts, calendars, and activities. I also share data between the devices using this installation.

It is also possible to access your NC instance from outside your home network. It involves additional steps and also several risks. I have done it and it is possible even if your IP address is dynamic.

My mean time between failures is about 6months to a year and that too mostly due to USB failure. Regular backups of the data directory and the database has saved me.
venkat_partha is offline   (2) Thanks
Old 5th April 2024, 15:22   #22
Team-BHP Support
 
Rehaan's Avatar
 
Join Date: Feb 2004
Location: Bombay
Posts: 24,285
Thanked: 35,158 Times
Re: DIY : My own, personal cloud storage

This LTT video just dropped a few days ago:

Rehaan is offline   (1) Thanks
Reply

Most Viewed


Copyright ©2000 - 2024, Team-BHP.com
Proudly powered by E2E Networks