Archive for the ‘ perl ’ Category

Perl Carp

Just really quick, for those perl visitors who have always wondered, what is it exactly that Carp does and why should I be using it?

Let’s assume you have a DB model that handles all db transactions via DBIx::Simple or DBI, etc.

package DB::Model;

use strict;
use warnings;
use Carp qw/croak/;
sub do {
  my $self = shift;
  my ($schema,$sql,@binds) = @_;
  #$self->logger->("Performing some task... : $sql on $schema");
  $self->db->query($sql,@binds) or croak $self->db->error;

Now, in some other part of your program (


use strict;
use warnings;

use DB::Model; # or from some IOC or DB provider in your application, etc.

$model->do('SELECT id FROM foo INNER JOIN bar USING(id) WHERE id = ? ',$id);

If there’s an error in your SQL statement, the ‘croak’ call will supply the database exception as such :

DBD::mysql::st execute failed: Column 'id' in where clause is ambiguous at lib/DB/ line xxx

Well, that’s nice, but it’s not very helpful if you have dozens of places in your code where you’re making that call to the database.

If you’re using Carp, you can easily turn on the verbose mode, and watch the stack trace come back to the original caller of the method / function.


use strict;
use warnings;
use Carp;

use DB::Model; # or from some IOC or DB provider in your application, etc.
$Carp::Verbose = 1; #- testing / debugging mode (you can even pass this via command line args)
$model->do('SELECT id FROM foo INNER JOIN bar USING(id) WHERE id = ? ',$id);

Which throws the trace :

DBD::mysql::st execute failed: Column 'id' in where clause is ambiguous at lib/DB/ line xxx
  DB::Model::do('DB::Model','SELECT id FROM foo INNER JOIN bar USING(id) WHERE id = ?',1) called at lib/DB/ line xxx
  Back on up to the original caller

This is really helpful while you’re writing your tests and in active development or are in debug mode.

When you’re done with your testing, you can remove the verbose mode.

Personally, when it’s available, I like to use croak to throw the exceptions in conjunction with a logger class so I can quickly evaluate the full SQL statement + parameters. I have, in times, added additional levels of exception handling by using Class::Exception and Try::Catch blocks around the exceptions for those can may be recovered, or need special warning messages / response codes (think Restful web services and http response codes for various resource exceptions, IE, Invalid Data, etc).

See more information here and here.

Multi-Tenant Database Design

@Work, we’re a Software as a Service (SaaS) (although we’re evolving into a platform) company providing data integration (perl ETL) and BI reporting and other industry (Business Equipment) based solutions such as territory mapping and compensation programs. Like most SaaS companies, database design, architecture and security are of the utmost importance, but how do you go about selecting the ‘right’ multi-tenant solution for your problem? As you start this requirement gathering and analysis process, you undoubtedly will start coming up with more questions, like :

  • Are there laws and regulations in place that your application / data storage solutions must conform to (Safe Harbor laws, encryption standards, credit cards, etc)?
  • How do you ensure only authenticated users have access to the data their authorized to view / manage?
  • How do we handle fail-over scenarios and reduce our risk for potential transactional data-loss.
  • How do you deal with backups, while keeping the aforementioned items in check?
  • What’s your restoration process, has it been tested (frequency of tests)?
  • How is the performance of your application going to be affected by any design decisions you make?
  • And so on and so on…

When dealing with Multi-Tenant data, I like to design one master schema (db in mysql), and then create one schema for each customer / client / vendor / tenant, etc. This design allows for a quick and easy encapsulation of the customers data, fast(er) backup and restorations processes and it can offer some natural security benefits if you tie the database user into the schema and of course YMMV.

Downtime and / or data loss in 2012 should be unacceptable (sure, there’s maintenance and migrations to consider, etc) to any SaaS providers and the costs associated with them should be VERY expensive. There are various ways to handle the fail-over configurations of MASTER / SLAVE relationships for this design and MySQL’s replication and / or a continuent solution may be the right one for you regarding this critical topic and again, YMMV. I’ve used a combination of solutions in the past, but the basic premise is to keep it as simple as possible while maintaining near 100% uptime and redundancy across the MASTER / SLAVE clusters.

There are many great articles available on the web that go into this concept in much greater detail than I will, with this one being one of the top resources available.

HOWTO : nginx, Starman / Plack and Catalyst

In this short tutorial, I will demonstrate how to use nginx as a frontend web proxy and Starman as a backend web server running a Catalyst application.

It assumes you already know about PSGI, Catalyst, nginx and how to install all the required libraries and perl modules.

Requirements :

First, we’ll configure nginx to run as our lightweight http server.  (If you require SSL, please make sure your ./configure command includes the –with-http_ssl_module option and that you have the openssl and pcre libraries already installed)

I’m going to require a ssl connection with a self signed certificate (you can follow the instructions here on how to set that up). I will also be redirecting all http requests to https via the web server (which is the best way to handle this).

In your nginx directory (mine was installed at /usr/local/nginx) edit your conf/nginx.conf file :

... (params)
http {
... (params, default server, etc)
  server {
    listen 80;
    location / {
      rewrite         ^/(.*)$$1 permanent;

  server {
    listen        443 ssl;
    gzip on;
    ssl_certificate /usr/local/nginx/conf/server.crt;
    ssl_certificate_key /usr/local/nginx/conf/server.key;
    #you may need or want to set additional ssl parameters (cipher, etc)

    location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Port 443; #this is important for Catalyst Apps!
      proxy_pass http://localhost:5000; #changed from http://localhost:5000/ which was causing double forward slash problems in the url

  location /static {
    root /data/users/MyApp/root;
    expires 30d;

This assumes that your static content will be servered by nginx and not your Catalyst App, and that when you start your starman server, you’ll be using port 5000.

To get your Catalyst up and running using Starman, install the Plack and Starman via cpanminus as well as the Catalyst::Engine::PSGI module for PSGI integration. Once all of your modules are installed, you’ll need to create a app.psgi file.  You can do so manually, or via the script/ command in your Catalyst App directory.

As I found out here from my own answer, when using a front-end web proxy, you’ll need to add a small bit of information to your .psgi file (Catalyst::Engine::PSGI docs), here’s an example of mine :

#!/usr/bin/env perl
use strict;
use warnings;

use Plack::Builder;
use MyApp;

my $app = sub { MyApp->run(@_) };

builder {
 enable_if { $_[0]->{REMOTE_ADDR} eq '' } 

If your instance of nginx is running on a different server, you may need to set this configuration variable in your Catalyst App (doesn’t hurt to add it regardless of your situation) :

MyApp->config(using_frontend_proxy => 1)

Start starman (currently defaults to 5 processes and port 5000) :
$ starman script/erp.psgi

Start nginx
$ sudo /usr/local/nginx/sbin/nginx

At this point, all of your requests should be automatically redirected to Nginx should be handling all of your static content used by your Catalyst app, and as long as Starman is running in the background on port 5000, your requests and responses should be handled accordingly and in a timely manner! 😉

In a production production environment, you may want to tweak some of your settings in the nginx.conf file and then create a startup script for both nginx and your Starman server. I also haven’t touched on any of the logs associated with Starman / Plack and your Catalyst app, or how to handle the ip addresses (nginx if forwarding the $remote_addr as `X-Real-IP` in the header) for access / error logs…