One of my customers has been using Archiware’s P5 Archive server for years to move unused media data from expensive, on-premises, high-performance storage to cheaper second tier storage.
P5 Archive manages archived data very well and is able to preserve all kinds of file- and filesystem-related metadata across all kinds of storage.
Over the years, the customer’s production environment has changed a couple of times to adapt to the availability of new technology, a preference of remote work of employees, and so on.
This article explains what is required to migrate an on-premises P5 server to a new instance (cloud or new on-premises server) when it archives data to S3 storage like WASABI.
Original Setup
Originally we started with a setup like this one:
Employees were accessing files on local production storage, and once a project was finished and related data needed to be preserved, P5 Archive moved data to a local archive storage.
Moving Archive Storage to WASABI/S3
At some stage it became relevant to keep more secure data copies outside the primary production facility while at the same time higher-bandwidth Internet access became cheaper and high-performance and high-reliability cloud storage became available and more price-efficient.
We chose to use WASABI as cloud-based archive storage. Therefore we migrated all archive data to WASABI, while at the same time we kept the P5 Archive server and the production storage on-premises:
P5 handles several S3-compliant cloud storage providers out-of-the-box and WASABI is easy to integrate as archive storage target.
Moving the P5 Archive Server to the Cloud
At some stage we wanted to place the P5 Archive server in the cloud. This was our train of thought:
- We wanted to move the US-based WASABI storage to WASABI storage based in the European Union. We needed lots of bandwidth and low-latency between the server and the storage locations.
- The on-premises P5 Archive storage was sharing Internet bandwidth with other local services and therefore could only use very limited bandwidth for data operations. In addition the office location was in the southern hemisphere and latency was high when talking to storage servers in the northern hemisphere. This had the effect that copying data from source to target storage was calculated to take almost two years.
- Having a P5 Archive server close to the new target storage reduces latency and we chose a hosting service with much more bandwidth than our original on-premises server was able to get. Therefore we hoped to move data from source to target within four weeks.
The target setup we wanted to build was this:
We chose a virtual machine hosted by Hetzner running in a US data center close to the source WASABI storage (CPX31 server, 1TB data storage, 8GB RAM, Ubuntu Linux).
I installed P5 Archive using the normal command-line installer, made sure it was running and that I could access the web interface.
IMPORTANT: Make sure to use P5 7.2 or later on both the on-premise and cloud-based P5 servers before you start the migration. If you work with older versions, the cloud-based server will not be able to restore data from the archive. (P5 7.2 stores some S3-specific info in /usr/local/aw/config/index/archive/Default-Archive/conts.db, and you need that info to make the new server work with your S3 storage).
Then I followed this instruction to copy over the configuration from the old to the new server. Though the source server was running macOS and the target server was running Ubuntu Linux, the migration process looks the same. File paths are identical on both OSes.
Once these things have been done, P5 can access the original storage and restore data to any accessible target storage.
Final Thoughts
The main purpose of this post is to make sure you can migrate a P5 Archive server with S3 storage access to a new instance running anywhere. I hope this has been explained in enough detail.
In the project mentioned here, we used custom code controlling the migration process from a source to a target storage. I have not described how we did this, as this might not be relevant to you. I guess that in most cases it will be more relevant to only migrate the server, not the storage.
If you have any questions or need help with any P5 migrations, please get in touch.