robkinyon on Perl

DBM::Deep 1.0020 released

2010-03-02T19:01:00.003-05:00

1.0020 Feb 16 22:00:00 2010 EST

(This version is compatible with 1.0016)

- Fixed t/43_transaction_maximum.t so that it doesn't error out on systems

which cannot fork > 255 children at one time.

- Improved code coverage

- Added t/96_virtual_functions.t which helps describe what actually

needs to be overridden in a new plugin.

So, the major change between 1.0020 and 1.0016 is the support for different backends. This was one of the biggest changes I've made, even considering adding support for transactions. While transactions were a big change mentally, adding backends required a lot more physical work. There was moving all sorts of code around and actually making proper APIs between the various subsystems. This exposed a ton of coupling that had to be teased apart very gently.

I also learned a ton about just how UN-transactional the various "transactional" datastores really are. SQLite does full DB-locks to implement transactions. InnoDB (MySQL's transactional engine) takes locks on AUTO_INCREMENT keys. So, in both cases, concurrent transactions just don't work. DBM::Deep's transaction support wins. Huh.

But, unlike DBM::Deep's native file backend, using MySQL as a backend allows you to use DBM::Deep across different servers. So, DBM::Deep can start to scale horizontally where, before, it couldn't because of problems with flock over NFS. And, for obvious reasons, running DBM::Deep using MySQL as a backend is faster - much faster.

I have an idea for how to improve transactions over InnoDB to avoid AUTO_INCREMENT keys by using UUIDs. But, until I see someone actually using this in prod, it'll remain just an idea.

Software as complexity management (part 2)

2010-02-20T16:54:00.003-05:00

(q.v. part 1)

All pieces of software have one thing in common - they change over time. Whether it's a new feature, a bug report, or a feature that's implemented over 3 releases, these changes affect our baby in so many ways. Managing those changes is probably the most important thing a developer does.

Change is a problem because it introduces risk - risk of a bug. We all know what a bug is, but every change could introduce one. So, managing change is best done by reducing the risk of every change.

The only way to reduce risk is to reduce the scope of the risk. The biggest risk any change has is in how many workflows the change can potentially affect. So, you want to keep the scope of every line of code as tight as possible. Any change to any line in a scope (block, function, etc) has the same area of effect as every other line in the same scope. So, if one line of code in a function reference a global variable, then any change to any line in that function have the potential to affect that global variable.

This is sometimes called coupling, but I think that coupling isn't a strong enough concept. The best analogy is taint. Let's say we have global variable %BAD_IDEA. It's referenced in functions bad_foo() and bad_bar(). If any line in bad_foo() is changed, then every workflow that uses bad_foo() must be tested (duh!). But, every workflow that uses bad_bar() must also be tested. And, if bad_bar() uses another global variable, then every workflow that uses any function that uses that second global variable must also be tested. All from changing a line in bad_foo().

It becomes pretty obvious pretty quickly that global variables end up tainting whole swathes of code (and why global variables should be avoided at all costs). Next part is about objects and how they encourage the use of global variables.

Software as complexity management (part 1)

2010-02-18T22:29:00.004-05:00

Software exists to solve problems, primarily problems in business. Solutions to those problems pay for the people who write the software who solve them. Even free software runs on corporate money. Especially free software.

If we want to write better software, we need to have a solid understanding of what a problem is. Instead of problems, it's useful to talk about the problemspace. I often find it useful to imagine a problem as a surface in N-dimensional space, each dimension representing a competing concern. For me, the surface describes the complexity floor of the solutionspace. It's the theoretically ideal solution - the absolute zero of software perfection.

So, the ideal software is the implementation that is closest to the complexity floor. Can we describe the complexity axes? Can we describe some rules for how to minimize them? What do you think?

How useful is a debugger, really?

2010-02-16T19:06:00.004-05:00

(Inspired by this and this and informed by Linus.)

From the get-go, there's been a Perl debugger and lots of people think that it's a "Good Thing"™. Me, I've never seen the point. Yeah, I've used it. I know how to set breakpoints and step into functions and all that. I've used perl -d, gdb, the VB debugger, and the debuggers in both Firefox and IE (version 5-8). And I hate them all.

Well, hate is a strong word. Frankly, I don't see the point. There has never been a programming problem I've faced where using the debugger helped me. At all. In any language. Ever.

In 2000, I was lucky to be mentored by Ted Vessenes when we were working at Motorola. Ted is one of the most phenomenal programmers I've ever met and I know a bunch of brilliant people. Just to give you an idea of how freaking awesome he is, this is the guy that played so much Unreal Tournament that he got pissed at the AI and built a new one. Not a perfect player, but a perfect human player. His AI ended up being so good that he had to mess up his algorithms so that other people knew they were playing an AI. He literally passed the UT3 Turing Test.

Ted once told me "the most powerful debugging tool is print." (He actually told me this a bunch of times.) Or, restated - "If you need anything more than the print statement to debug a problem, then you don't understand the problem enough to solve it."

That means you, Mr. debugger. If I have to reach for you, that means I don't have enough context, knowledge, or whatever else I might need to solve the problem. In fact, I use my desire for a debugger as a litmus test. Anytime I want a debugger, I know I'm too stupid to fix it.

(Frankly, this goes for IDEs, too. If I need anything but vim to manage my project, then my project isn't organized properly.)

Why choose Java over Perl?

2010-02-15T20:34:00.003-05:00

I was talking about about Perl and Java in the Enterprise with frew on IRC tonight. We never actually discussed the topic at hand. Instead, we ended up talking about szabgab's comment.

I think the primary reason is that Perl is too individualizable, by far. Yes, I said too individualizable. That strength of Perl - TMTOWTDI - is its biggest weakness when it comes to working in groups.

When it comes to programming applications, the key feature of a good workflow is managing change. Every application will have change come down the pike. A new feature, an overeager salesman, a bug found, or an idiotic CTO - these all bring about changes. A good workflow will build an application that manages these changes without compromising the application's ability to manage subsequent changes. A team of decent Perl developers will easily be able to build an application that does this. But, how long is the ramp-up for a new developer? What happens when one of the initial developers leaves?

Java, on the other hand, does not give you all that freedom in how you build your application. There is no TMTOWTDI Ave. in Java City. This inability to express yourself is a strength when it comes to managing turnover. Every Java application in existence looks a lot like every other Java application.

Furthermore, Java offers a set of guarantees that Perl simply cannot do because of the design tradeoffs inherent in the language. For example, there is absolutely no way to declare something invariant in Perl. In Java, invariants are a normal part of the language. Perl also has no way of guaranteeing the type of a given variable. Java and Haskell both have this feature. You cannot turn off monkey-patching or any of the other runtime poking that is one of Perl's greatest strengths, not even lexically.

And, to top it all off, there's such a huge variety in feature strengths that a Perl novice simply cannot comprehend a Perl master's code, let alone a guru's. Every Java developer can read every other Java developer's code.

Given that some programming teams turn over a developer every week, which language should they choose?

Perl and Java in the Enterprise

2010-02-14T17:49:00.002-05:00

(Via Google Buzz)

I'm glad that Perl job openings in the enterprise are trending upwards. But, what does the number of openings really tell us about the comparative penetration of each language? Well, a lot less than you might think.

The big difference is how many developers of language X are needed to support a given set of features. And, as every good Perl developer knows, Perl requires a lot less developers than Java.

A LOT LESS.

In the past, I was a Perl contractor for some of the largest Fortune 500 companies in the world. Every one of these companies had significant investment in both Perl and Java. A few times, I saw a given application developed in Perl, then reimplemented in Java. I distinctly remember two specific cases where this happened.

Both cases followed the same pattern. First, the Perl app was developed part-time by 2-3 developers over a couple weeks and supported by them in their "spare time" - in addition to their fulltime responsibilities. Management found out about the app and, in both cases, ordered it reimplemented in Java. Six months later, the group 20+ developers and 2-3 project managers released their first version. Unusably buggy, it had less than half the features than the prototype app in Perl. After another year (roughly 40 man-years sunk), the users in both cases got the original app up and running via back-channels and went about their work. In one case, management supported the Perl app. In the other, I think the Java app is still up and running. Maybe (6+ years later), it's finally where the original prototype was. Probably not.

The point is that a team of 3-5 Perl developers can do more than 30 Java developers. 10:1 sounds about right. Perl has already penetrated the enterprise. Frankly, the enterprise cannot exist without Perl. The enterprise just doesn't need as many Perl developers to do the same work. Think about that.

What is "Modern Perl 5"?

2010-02-10T19:45:00.005-05:00

So, the theme for YAPC::NA::2010 is going to be "Modern Perl 5". Other than a cool marketing buzzword, what exactly does that mean?

Sometimes, it's easier to start with what this doesn't mean.

There's a quasi-truism that says something like "You can always recognize someone's original language in Perl." C programmers write C in Perl. Java programmers do the same thing. (And, wow can you tell a sysadmin who's only ever used awk and sed!) It's easy to recognize "good Perl," but can we describe it?

Perl is also oft-maligned as "a bunch of line-noise." While it's easy to dismiss those complaints as from "sigil haters" or "regex newbies," there is definitely some truth that Perl tends to be much denser than other languages. And, that's a strength of our language - one we get from our FP roots. It's quite common to translate a program from C to Perl and get LOC ratios of 20-1 or higher. I once saw higher than 100-1 and it was well-written C.

Lastly, one of the great promises of Perl 6 is that it will allow sane grammar modification. We don't have that yet. But, Devel::Declare does get us quite close. Syntactic sugar isn't about less keystrokes - it's about being clear about the intent of the code. In my opinion, C's typedef is one of its most powerful features.

Putting it all together, this is what Modern Perl 5 is all about:

A Perl "style"
Readably dense code
Literate code

The best example of this is Moose. Everything in Moose can be implemented in pure Perl. (Some is in XS for performance, but it doesn't have to be.) Code written using Moose is unarguably in a Perl style. The code is very dense (easily 20-1 to C or Java), but it's clear at the same time. There are several kinds of subroutine declarations, each making it clear what its intention is.

Moose will have at least one track devoted to it and there's going to be at least one class on Moose during the after-YAPC. What else is there that epitomizes those "Modern Perl 5" features? How did you bring those features to your pre-modern codebases? What advice can you give someone on how to make their code modern? Did I even get all the features of "Modern Perl 5" correct?

Why am I asking all these questions? Because I want your talk submissions!

Social media setup

2010-02-10T07:34:00.003-05:00

We have setup the various social media connections you want to get your YAPC::NA::2010 on.

And we're still on the ones you already know and love:

#yapc on irc.perl.org
Mailing list - http://mail.pm.org/mailman/listinfo/yapc

Call for Presentations for YAPC::NA::2010

2010-02-09T18:36:00.006-05:00

YAPC::NA::2010 is just around the corner. (Well, inasumuch as something 18 weeks away is "around the corner.") Last week, I met with Heath Bairh and the other organizers and took the position of speaker liaison. So, I should probably talk some about what we're looking for.

YAPC::NA::2010 is going to be about "Modern Perl 5". We believe that Perl 5 is a vibrant and living language with many uncharted places it can go. Perl 6 is going to be great, but we can't wait until Christmas for Perl 6.

So, we're looking for presentations about Perl 5 in all of its modern glory. Whatcha got?

You'll need to register (or login) to the YAPC::NA::2010 website, then after registering for the conference, you can submit your CFP. The window closes March 31, 2010 at midnight GMT.

Falling off the Ironman wagon

2009-08-17T09:34:00.005-04:00

*sighs* I've fallen off the wagon. I knew I should have posted last week, but I didn't. Blogging is much harder than I thought. If you know me at all, you know that I have at least one opinion about nearly every topic in the word. I love discussing pretty much every topic, particularly if there are more than two options and it's impossible to determine which is better in any given situation. Those kinds of indeterminate discussions are absolutely fun.

Yet, here we are - I have people actually reading this blog and I can't actually write something every 8 days. I think it's because people aren't telling me how wrong I am on a regular basis. :)

The first "usable" release of P6

2009-08-06T17:07:00.004-04:00

Patrick Michaud just discussed the idea of releasing Rakudo, the Perl6 implementation bootstrapped in Perl6 targeting Parrot. He makes a very interesting point that language development doesn't go for a static target. Instead, it's very evolutionary. And, so, his goal is to have a "usable" and "useful" release of Perl 6 by Spring 2010 (April-ish).

So, what does this mean? Well, in my mind, it means a few things.

The first is that we, as a community, have been thinking we're going to get the most amazing thing in the world from the get-go. And, in retrospect, it's rather obvious that this just isn't reasonable. The whole point of agile development (and, believe me, P6 development is nothing if not agile!) is that you "release early, release often".

The second is that, frankly, a well-supported feature-light Perl 6 now is much better than a well-supported feature-complete Perl 6 released right before Duke Nuke'em Forever. I want to have some frozen API in my grubby hands before I spend my evenings away from my family and hacking on CP6AN stuff.

And, lastly, we need to remember our history. Perl had been around for 10 years before objects and sane packaging were supported at all. Threads? Unicode? Both became stable with 5.8.0 - that's barely five years ago. Moose is less than three years old. DBIx::Class and Catalyst aren't much older. And, frankly, we drove the development of every one of those features. As a dev, I want users to tell me what they want to use. The P6 devs aren't any different.

So, instead of griping that a feature-light P6 will be available, start thinking about how you can (ab)use those P6 features that will be available (and we should have that list in the next month or two). Maybe we should start voting for our favorite features and see what we can do to help get them into the first production-ready Rakudo release. Go Team!

Perlish code part deux

2009-07-29T21:59:00.009-04:00

Wow. Lots of good comments on my last post. (well, lots of comments by this novice's standards, at any rate.) And, while I'm pretty sure some were tongue-in-cheek, it's obvious that my rather sparse discussion of conciseness was a bit too concise. So, maybe a longer example is in order.

Most programmers (perlers or not) will easily understand the following:


my @data = (
    {
        id => 1,
        first_name => 'Barney',
        last_name => 'Rubble',
        age => 33,
        gender => 'male',
    },
    # more of the same
);

# Create a string of "first last, first last, ...", sorted by name, for just males
my $names = join ", ", map {
        "$_->{first_name} $_->{last_name}"
    } sort {
        $a->{last_name} cmp $b->{last_name}
     || $a->{first_name} cmp $b->{first_name}
    } grep {
        $_->{gender} eq 'male'
    } @data;

I'm not going to even attempt how to do that in C. Needless to say, I'm betting it's going to be significantly longer than 8 lines. In fact, I'm betting it'll be closer to 10x that. The Java version is likely to be 5x that and likely more if you add in the building and maintenance of the initial data structure.

That is concise Perl. It takes advantage of the native high-level data structures (hashes and arrays) and the operators built for them (sort, map, and grep) to create very easy-to-understand complex code. And, we're not even touching string manipulation, supposedly Perl's greatest strength.

What makes code Perlish?

2009-07-28T22:56:00.006-04:00

There is a lot of discussion in lots of places about "perlish" code. What perlish code is. What it isn't. But, not a lot about how you can tell the difference.

The first metric to look for is conciseness. How many lines of code does it take to express a concept? For example, swapping a two variables. In most languages, you have to use an explicit temp variable, such as in the following C code:

temp = i;

i = j;

j = temp;

That's three lines to express a single concept. Most imperative languages require the same thing. Perl (and similar languages like Python and Ruby) does it in one.

( $i, $j ) = ( $j, $i );

Two lines doesn't sound like much. But, think about it as a 3-to-1 reduction. That means 300 lines is now 100. Ohhhh.

Converting CDBI to DBIC (part 6): Phase 1

2009-07-20T16:48:00.003-04:00

In part 5 of this series, the plan for how we're going to do this conversion was laid out. Now, for some actual working code.

The plan for our migration is going to leverage DBIx::Class::ResultSource's result_class attribute. First, some explanation. Unlike every other ORM I know about, DBIC decouples the operations on a group of rows from inflating those rows. This is the whole resultset thing. So, it only makes sense that you would be able to specify how you want to actually go about inflating the rows returned from a search. And DBIC does exactly that.

So, we have a set of CDBI classes. Let's work with one of them called App::CDBI::Foo. In order to make this work, we're going to want to have a corresponding DBIx::Class::ResultSource object. That resultsource object will be registered with our schema object (q.v. DBIC for more info) as handling stuff for the foo table that App::CDBI::Foo used to manage.

In order to get everything to work, we're going to need to tell that resultsource everything that the CDBI class knows. We're also going to have to inject an inflate_result() method into the CDBI class.

my $source = DBIx::Class::ResultSource::Table->new({});

$source->add_columns( $cdbi->columns );

$source->set_primary_key( $cdbi->primary_columns );

*{ $cdbi . '::inflate_result' } = sub {

my $self = shift;

my ($source, $data, $prefetch) = @_;

return $cdbi->construct( $data );

};

$source->result_class( $cdbi );

$schema->register_source( $tablename => $source );

And, that's the basic structure. Unfortunately, there are issues. Some of which I'll deal with in later posts, some of which I can't (because they're your problems).

One of the biggest reasons to migrate to DBIC is prefetch. You'll notice that our inflate_result doesn't actually do anything with $prefetch. I'll post a better one later.
add_columns() can take a lot more information than CDBI ever stored. You'll want to populate that information somehow.
$tablename and other variables appeared out of mid-air. You'll want to fix that. :)
This code doesn't actually do anything for things like unique constraints, defining relationships, and the like. You'll want to fix that, too. :)
Unless your code is amazingly clean, you probably have snippets like $obj->search(...); You may need an AUTOLOAD to catch those (for now).

And, later on, I'll show you how to migrate your actual objects to DBIx::Class::Row objects from CDBI.

Converting CDBI to DBIC (part 5): The plan- requirements

2009-07-15T20:49:00.002-04:00

So, in part 4 of this series, I discussed why CDBICompat just wasn't going to cut it. What I didn't explain in great detail is just why CDBICompat needs to use tied variables (thus causing a nasty slowdown). It goes something like this:

CDBI has pretty poor searching capabilities
CDBI doesn't have prefetch
CDBI doesn't cache very smartly

So, most heavy users of CDBI tend to write their own caching mechanisms. Given that CDBI is a row-centric ORM, these caches are almost always in the row. Given that most of these developers are smart, but under serious time constraints, these caches break encapsulation. So, something like

my @rows = CDBI::Class->search( ... );

foreach my $row ( @rows ) {

$row->{_cache} = $row->expensive_method();

}

is very normal to see. And very expensive to convert away from. Any changeset that converts over every single one of these encapsulation breakages is going to be too huge to test with any confidence. As the applications we're looking at are large (> 100kLOC) and big moneymakers (often $M's per year), having confidence in the next push to production is key.

So, the conversion plan has to meet the following requirements:

allows us to use DBIC's big features - resultsets, prefetch, and SQLA searching.
allows us to phase our conversion so that we don't have massive changesets which are impossible to test.
doesn't impose any noticeable slowdowns, at least not noticeable by the users

With any other distribution, that would be a tall order. DBIx::Class, however, already has the single feature we need to make this happen. More on this in part 6.

For those who can't wait, I'll give you a hint. Go look at

DBIx::Class::ResultClass::HashRefInflator.

Converting CDBI to DBIC (part 4): Why not CDBICompat?

2009-07-13T21:20:00.003-04:00

This isn't the first time someone has tried to convert from one ORM to another. In fact, DBIC grew out of working with CDBI and needing something better. In the past, ORMs have usually been similar enough in their inner workings that a compatibility layer was enough. The userland could be migrated at some later date, if ever. The cost of the indirection would be nearly nothing.

DBIC, though, is different. So different that a compatibility layer obscures the whole purpose of converting from CDBI to DBIC. The very concept of resultsets breaks the mold. Like, shatters it, stomps all over it, and melts it in a vat of molten iron. The whole point of switching to DBIC is to get access to resultsets. CDBI's API has absolutely no facility to provide access to these features.

Second, you're stuck with CDBI's searching facilities. SQL::Abstract is extremely powerful. So powerful that several CDBI plugins were created to mimic its power. SQL::Abstract v2 will be even more powerful yet. But, CDBICompat cannot expose any of it.

And, worse yet, in order to accommodate the numerous abuses of CDBI that most people did in order to make it usable in larger applications, the compat layer ends up being about 20% slower than either raw DBIC or raw CDBI. In fact, the row object provided by CDBICompat is tied just to make sure things work.

It's the worst of both worlds - you're stuck with CDBI's API (the bad thing you're trying to escape) and a slowdown of your application's code. Given that you're usually trying to speed up your application and improve the ease of writing new features, this doesn't sound like a win.

Next up - the actual solution.

Performance vs. Correctness

2009-07-07T21:24:00.002-04:00

This would seem to be a truism, but it bears repeating. Bertrand Meyer once said "Correctness is clearly the prime quality. If a system does not do what it is supposed to do, then everything else about it matters little." A lot of people seem to forget that if you get the wrong answer very quickly, it's still the wrong answer. And, a lot of those people are in the OSS and Perl communities. A corollary to that could be "Try and do it right. If you can't guarantee the right answer, then fail very quickly and and even louder."

Why bring this up now? Well, in Perl, performance concerns seem to crop up every so often, and in places where it makes no sense to bring them up. For example, when dealing with crazy date manipulations. DateTime is the de-facto standard for date manipulations in Perl, and for good reason. It is correct and, in most cases, fast. But, its interface is very low level. There is another module that provides different interfaces to dates - Date::Manip. But, even the author acknowledges in the POD that "It's the most powerful of the date modules. It's also the biggest and slowest." And, whenever it's brought up, inevitably someone will say "Yeah, it's pretty heavy."

My answer to that is "So what?" The very first criterion for a computer program is "Does it work?" And, by that I mean "Is it correct?" Once it's correct (and you have a regression suite to validate that), then you measure its performance. If its overall performance is acceptable, then you leave it alone.

Let me repeat - if overall performance is acceptable, then you leave it alone.

If overall performance needs improved, then you profile it. Chances are, the module you think is heavy isn't your bottleneck. Even if it is, it's very likely that a small change to one method will net you 80% (or more!) of your potential speed improvements.

Pick modules based on features. Then, if you need it, optimize after profiling. And, module authors, don't refuse to fix a bug because it would hurt performance. If your module is wrong, then it needs to be fixed.

Converting CDBI to DBIC (part 3): Why C3 is important

2009-06-28T11:48:00.002-04:00

If you want to override any of the methods in CDBI, you have to inherit from CDBI, then override using standard OO methods. With any large application, you end up with the explosion of classes in your hierarchy. More than any other hierarchy, the ORM hierarchy really cries out for roles (or traits). Enough of your tables need to be treated the same in some places, but different in others, that you want to be able to say "for classes A, B, and C, their create() is overrided as so, but A and B have their update() overrided as so, but B and C have their delete() overrided as so". With standard inheritance, it's almost impossible to do that and make it comprehensible for the next developer.

DBIC, on the other hand, allows you to define capabilities using components. By providing a traits-like solution, you can easily extend the behavior of your Rows and ResultSets in ways that can cross class hierarchy lines. In other words, it's sane multiple-inheritance-like behavior for a very common use-case of multiple inheritance. In our CDBI example above, you would have three components - one each for the create, update, and delete overrides. A, B, and C would all use the create override, A and B use the update override, and B and C use the delete override. Given that each one is likely to be independent (not related to each other), the code becomes more self-documenting.

Converting CDBI to DBIC (part 2): The supposed similarities

2009-06-27T20:46:00.000-04:00

(You can catch Part 1 for the context)

The first thing you notice when comparing CDBI and DBIC is that the APIs look really really similar. They both have the same method names for almost everything most people use them for. So, you'd think that it's as easy as swapping out use statements, adding a few lines to the table definitions, and calling it a day.

And, unless your application is so exceedingly simple that you were able to keep to the officially published CDBI API, not a single test will pass. (You do have tests, right?) Every problem arises out of the need to have abused CDBI in order to get work done.

Class::DBI is built upon Ima::DBI, a connection caching and SQL management distribution. Every method provided by CDBI is built using Ima::DBI's set_sql() and transform_sql() methods. These methods, while pretty neat, are very hard to extend because they use string transformation.

DBIx::Class, on the other hand, is built on three major concepts that CDBI doesn't have:

everything is componentized using C3 method resolution
SQL::Abstract to generate the SQL
first-class distinction between ResultSets and Rows

These three differences mean that what should, in theory, be a simple conversion between two modules that expose similar APIs becomes a much more difficult thing to do. Over the next few posts, I'll examine why each of these differences is important and how each one complicates the conversion process.

Converting CDBI to DBIC (part 1): The context

2009-06-26T22:35:00.001-04:00

For years, Class::DBI was the gold standard ORM in the Perl community, and for good reason. It was simple to deploy, easy to use, and, for the most part, dwim'ed. Oh, we Perlers love our dwimmery. If it doesn't dwim, then we get pissy.

Now, CDBI wasn't designed like Perl. Perl has always been built to make the hard things easy and the impossible merely hard. CDBI, on the other hand, was designed with the 80/20 rule in mind - make as many of the things most people do every day extremely easy. And, for everything else, there's always an easy way to get to use raw SQL. And this was, for most people, good enough. It easily supported rapid prototyping and small applications and, while slighly annoying on the edges, it worked.

But, as applications are wont to do, some of those applications grew up. They didn't stay small or fade away. The schemas grew from 5 or 10 tables to monstrosities of 250 tables or more, many having dozens of columns. The codebases weighed in at well over hundreds of thousands of lines of code. And, instead of building the code, the focus became maintenance. And, those things that were rare in the past became more common. Not relatively more common, but appeared more often. Instead of 3 or 4 dips into raw SQL, there were 90 or 100. And, wow, was scaling an issue.

Many ORMs were built to try and take its place. Lots of good work was done, but the gold standard has seemed to settle around to DBIx::Class, and for good reason. DBIC maintains that 80/20 design philosophy, but manages to marry it to the Perl philosophy of making hard things easy.

Unlike CDBI, DBIC uses SQL::Abstract for its SQL generation, meaning less dips into raw SQL. Scaling is saner because lessons were learned. Yet, all the really easy things are still really easy. Rapid prototyping is still rapid.

And, this is why we want to convert.

(Follow on in Part 2)