Comments on: Backing up to an EBS volume with rsync and EC2 http://www.erebor.com/?p=184 Views from the mountain Mon, 27 Jul 2009 10:38:47 -0500 http://wordpress.org/?v=2.9-rare hourly 1 By: rew http://www.erebor.com/?p=184#comment-24242 rew Tue, 14 Jul 2009 15:22:16 +0000 http://www.erebor.com/?p=184#comment-24242 That might work, though if you're just looking to rsync to S3 (which is very close to what you'd have here), there are easier ways to do that than running it through an EC2 instance. I'm not certain how syncing an EC2 instance with an EBS block mounted would work, though it should work fine. S3 storage actually costs a little more than EBS, but as you mention, you only pay for what you use, whereas with EBS you pay for what you allocate. Another option to think about, though, is allocating an EBS only about the size you need. Snapshot it frequently to S3, and when you need more room, build a new, slightly larger, EBS from a snapshot and then toss the old one. Rinse, repeat. If your data size grows predictably, that might be a good option for you, so you're paying the cheaper storage price for EBS space, but not paying for a lot of empty space. EBS isn't just S3 on steroids; it provides some abilities that S3 or an EC2 instance doesn't, and so it's priced differently. Whether creating+syncing EC2 instances that included and EBS block would end up being cheaper (versus using something like JungleDisk to rsync directly to your S3 storage from your non-Amazon data source) will depend on the size of your dataset compared to the size of the instance itself. Let me know what you end up doing. That might work, though if you’re just looking to rsync to S3 (which is very close to what you’d have here), there are easier ways to do that than running it through an EC2 instance.

I’m not certain how syncing an EC2 instance with an EBS block mounted would work, though it should work fine.

S3 storage actually costs a little more than EBS, but as you mention, you only pay for what you use, whereas with EBS you pay for what you allocate.

Another option to think about, though, is allocating an EBS only about the size you need. Snapshot it frequently to S3, and when you need more room, build a new, slightly larger, EBS from a snapshot and then toss the old one. Rinse, repeat. If your data size grows predictably, that might be a good option for you, so you’re paying the cheaper storage price for EBS space, but not paying for a lot of empty space.

EBS isn’t just S3 on steroids; it provides some abilities that S3 or an EC2 instance doesn’t, and so it’s priced differently. Whether creating+syncing EC2 instances that included and EBS block would end up being cheaper (versus using something like JungleDisk to rsync directly to your S3 storage from your non-Amazon data source) will depend on the size of your dataset compared to the size of the instance itself.

Let me know what you end up doing.

]]> By: mark http://www.erebor.com/?p=184#comment-24240 mark Tue, 14 Jul 2009 04:46:31 +0000 http://www.erebor.com/?p=184#comment-24240 Could you create an EBS volume right before starting the instance, format/mount it, do your rsync, shut down the EC2, take a snapshot of it to S3, then delete the EBS volume? It seems like doing that should cost nothing, as it would only exist for minutes/hours and you could make it huge, then the S3 snapshot would only be the size of the actual data on the EBS volume.. Just trying to figure out another way to do this without paying for empty EBS space :) Could you create an EBS volume right before starting the instance, format/mount it, do your rsync, shut down the EC2, take a snapshot of it to S3, then delete the EBS volume? It seems like doing that should cost nothing, as it would only exist for minutes/hours and you could make it huge, then the S3 snapshot would only be the size of the actual data on the EBS volume..

Just trying to figure out another way to do this without paying for empty EBS space :)

]]>