Tuesday, March 2, 2010

DBM::Deep 1.0020 released

1.0020 Feb 16 22:00:00 2010 EST
(This version is compatible with 1.0016)
- Fixed t/43_transaction_maximum.t so that it doesn't error out on systems
which cannot fork > 255 children at one time.
- Improved code coverage
- Added t/96_virtual_functions.t which helps describe what actually
needs to be overridden in a new plugin.

So, the major change between 1.0020 and 1.0016 is the support for different backends. This was one of the biggest changes I've made, even considering adding support for transactions. While transactions were a big change mentally, adding backends required a lot more physical work. There was moving all sorts of code around and actually making proper APIs between the various subsystems. This exposed a ton of coupling that had to be teased apart very gently.

I also learned a ton about just how UN-transactional the various "transactional" datastores really are. SQLite does full DB-locks to implement transactions. InnoDB (MySQL's transactional engine) takes locks on AUTO_INCREMENT keys. So, in both cases, concurrent transactions just don't work. DBM::Deep's transaction support wins. Huh.

But, unlike DBM::Deep's native file backend, using MySQL as a backend allows you to use DBM::Deep across different servers. So, DBM::Deep can start to scale horizontally where, before, it couldn't because of problems with flock over NFS. And, for obvious reasons, running DBM::Deep using MySQL as a backend is faster - much faster.

I have an idea for how to improve transactions over InnoDB to avoid AUTO_INCREMENT keys by using UUIDs. But, until I see someone actually using this in prod, it'll remain just an idea.

2 comments:

  1. Is there, or could it be done easily, a way to interface dbm::deep from Java?

    ReplyDelete
  2. DBM::Deep is a Perl module. The dbm-deep storage could be interfaced with from any language. I have no desire to implement anything in Java, but you are more than welcome to do so and release it.

    ReplyDelete