Founded in 2008, Seattle Cloud helps consumers design, build, and publish native Android and iOS apps. Customers choose a template, easily edit and format content, add images, music, sound, and video files. Seattle Cloud automates the process of submission and inclusion in the Apple App and Google Play stores.
Challenges
Seattle Cloud had a need to move a web application from a remote data center to Amazon Web Services Cloud (AWS). The application included one application web server, one MSSQL database, and more than nine million individual media files consuming more than 4TB of local disk space. The issues were numerous but specifically as they related to moving a large number of files, it soon became apparent that a traditional approach of mapping a local virtual file system to an AWS hosted S3 bucket was not going to be efficient or realistic.
- The number of files were in the nine million range with a total disk space of 4 TB.
- The hard drive subsystem was failing at either the microcontroller level, power, or possibly both. The array would have to be re-built.
- Disk capacity was at 95% making creating a compressed archive of the data not viable.
- An additional server would have to be provisioned, RAID configured, and then files copied.
- The last and final hope for any fast or traditional resolution was that since the individual media files could not be compressed, each file would need to be copied over separately requiring an open a socket connection, transfer of the data, and then closing the connection adding an unrealistic amount of time for the process to complete across the WAN.
Migration Solution
In the initial solution, we used a public internet connection from the remote data center to the Client VPC at AWS to upload to an S3 bucket. However, due to the amount of data being transferred, we calculated average bandwidth throughput across a public channel and estimated that it may take more than a week to copy the files. Viewing that as a roadblock to keeping the service online, we moved to the alternative solution by provisioning the SnowBall service from AWS with appropriate credentials for the new account. The drive was overnighted by fast courier to the remote data center and connected directly to the local disk array. Once the drive was loaded with proprietary data and returned to AWS, their engineers connected the drive locally and transferred all media to an S3 bucket via a Snowball device. For each environment, we use an EC2 instance with attached EBS volumes to store application configurations and project code that can be easily downloaded from the customer repository. MSSQL RDS is the application database backend and both EC2 and RDS are monitored by CloudWatch metrics and alerts with IAM as access management service.
Benefits to the SeattleCloud
The Snowball solution was developed by AWS for situations like these and was the most practical and efficient by far. It allowed us to take the migration time down from two weeks to a little more than two days. It was soon becoming apparent that the task was rapidly being painted into a box and a solution needed to be adopted quickly that was more a new school method than old.