S3 Backup 1.1 Release Candidate

A new version of S3 Backup is available. There are no known bugs and unless some are discovered by the users we’ll mark it as stable in about a week.

There are a lot of changes in this release, but even more are coming soon. We decided to keep some of the new code disabled for now just to be on the safe side. If things go as planned and the 1.1 stable is out in a week, we’ll follow it up with further updates that we already have prepared.

Filedb updates

S3 Backup determines which files to upload by maintaining a local database of what files were already uploaded. With every file operation this database needs to be updated to be in-sync with the online copy. The database is stored in SQLite and given that every update is a transaction, it causes a couple of disk operations for every uploaded file. If you have a noisy HDD you would hear that as a “click” or, if you upload a lot of small files as a slight “HDD thrashing”.

We’ve changed this part of code to add another layer of in-memory caching, so now the updates are only written out to disk once in five seconds as a single transaction which makes it completely unnoticeable.

This also provides a performance improvement and paves the way for the updates we will reveal in future releases.

New GUI code

In preparation for the Mac and Linux ports we’ve given our GUI code an overhaul. The settings dialog is completely new and even friendlier than before. You can now create new buckets directly from the bucket selection box. Some options were removed as well, so the proxy is now acquired from the system-wide configuration and S3 connections are always SSL-encrypted.

Bucket Manager was also rewritten and among other things allows concurrent operations (listing or deleting any number of buckets at the same time). It is accessible from the menu Tools → Bucket Manager.

Faster backups

In addition to the filedb speedup we’ve determined that logging was sometimes consuming disproportionate amount of resources because of being hooked in with the GUI code. We’ve throttled the GUI updates and made sure there’s no overhead at all when the debug log is not open.

Bugfixes

  • There were issues with automatic migration of backups jobs from backups.xml to the new config.json format if the config was password-protected. This was fixed.
  • We’ve discovered that PyCrypto 2.0.1 which we used for the most of the app’s lifetime during beta has a bug. When the key length was longer than 192 bits, the cipher was encrypting the stream in such a way that it cannot be decoded with any other Blowfish implementation, including the one from PyCrypto 2.3 that we are using now. We now include the old buggy version of Blowfish cipher and use it when decoding the old streams. (Most users will never encounter this condition anyway).

Other changes

  • Support for new Japan AWS region
  • New menu item: Tools → View past logs
  • Upgraded to Python 2.7
  • Massive updates to the build process (this is particularly important for Mac OS X and Linux ports).
  • The logging and traceback collection system was updated to be more resilient to various unicode-related issues.
  • Other miscellaneous updates

Download S3 Backup 1.1 RC and let us know if you like the updates.