Report inadequate content

How to backup Flickr into Amazon S3/Dropbox/Google Cloud/Drive...

{
}

 TAGS:undefined
The following recipe is an easy way to get your full Flickr account backed up in Amazon S3 or similar cloud storage.

Unless you have a really good Internet connection at home or you are doing a backup of a small library the first thing to do would be to create a machine in the cloud, such as Digital Ocean or Amazon EC2 so you can have a high throughput. Once the process is finished you can destroy the machine and the whole operation will cost you a few cents. When creating the droplet/instance make sure to create a machine that has enough disk space to hold your library.

In my case I had some free credit in Digital Ocean so I did it there, but of course you can create a machine in EC2 and persist directly in S3, or you can also do this in Google Cloud.

The process consists in two simple steps:

  1. Download all the pictures from Flickr using Flickrtouchr
  2. Sync all the downloaded pictures to S3 or other cloud storage using Rclone

Install the software

apt-get update
apt-get install --yes git unzip screen
wget http://downloads.rclone.org/rclone-v1.28-linux-amd64.zip
unzip rclone-v1.28-linux-amd64.zip
mv ~/rclone-v1.28-linux-amd64/rclone /usr/local/bin/rclone

The substitute the following lines with your own Amazon Access Key ID and secret

echo "
[S3]
type = s3
env_auth = false
access_key_id = ABCDEFGH1234567890IJK
secret_access_key = aBcDefghHijKl123456mnopqrst
region = us-east-1
endpoint =
location_constraint =
" > ~/.rclone.conf

Install the script that downloads pictures from Flickr:

git clone https://github.com/dan/hivelogic-flickrtouchr.git
cd hivelogic-flickrtouchr
mkdir FlickrBackupFolder/

Download all your Flickr pictures

It is recommended to use screen so you can recover the session anytime

screen -d -m -S "Flickr download" python flickrtouchr.py FlickrBackupFolder 
# screen -ls
# screen -r PID
# CTRL+a :detach

Upload pictures to S3

rclone copy ~/hivelogic-flickrtouchr/FlickrBackupFolder/ S3:your-bucket-name

Done!

{
}
{
}

Leave your comment How to backup Flickr into Amazon S3/Dropbox/Google Cloud/Drive...

Log in to Obolog, or create your free blog if you are not registered yet.

User avatar Your name