Perl Carp

Just really quick, for those perl visitors who have always wondered, what is it exactly that Carp does and why should I be using it?

Let’s assume you have a DB model that handles all db transactions via DBIx::Simple or DBI, etc.

package DB::Model;

use strict;
use warnings;
use Carp qw/croak/;
...
...
sub do {
  my $self = shift;
  my ($schema,$sql,@binds) = @_;
  #$self->logger->("Performing some task... : $sql on $schema");
  $self->db->query($sql,@binds) or croak $self->db->error;
}

Now, in some other part of your program (run.pl)

#!/usr/bin/perl

use strict;
use warnings;

use DB::Model; # or from some IOC or DB provider in your application, etc.

....
$model->do('SELECT id FROM foo INNER JOIN bar USING(id) WHERE id = ? ',$id);

If there’s an error in your SQL statement, the ‘croak’ call will supply the database exception as such :

DBD::mysql::st execute failed: Column 'id' in where clause is ambiguous at lib/DB/Model.pm line xxx

Well, that’s nice, but it’s not very helpful if you have dozens of places in your code where you’re making that call to the database.

If you’re using Carp, you can easily turn on the verbose mode, and watch the stack trace come back to the original caller of the method / function.

#!/usr/bin/perl

use strict;
use warnings;
use Carp;

use DB::Model; # or from some IOC or DB provider in your application, etc.
$Carp::Verbose = 1; #- testing / debugging mode (you can even pass this via command line args)
....
$model->do('SELECT id FROM foo INNER JOIN bar USING(id) WHERE id = ? ',$id);

Which throws the trace :

DBD::mysql::st execute failed: Column 'id' in where clause is ambiguous at lib/DB/Model.pm line xxx
  DB::Model::do('DB::Model','SELECT id FROM foo INNER JOIN bar USING(id) WHERE id = ?',1) called at lib/DB/Model.pm line xxx
  ...
  ...
  Back on up to the original caller

This is really helpful while you’re writing your tests and in active development or are in debug mode.

When you’re done with your testing, you can remove the verbose mode.

Personally, when it’s available, I like to use croak to throw the exceptions in conjunction with a logger class so I can quickly evaluate the full SQL statement + parameters. I have, in times, added additional levels of exception handling by using Class::Exception and Try::Catch blocks around the exceptions for those can may be recovered, or need special warning messages / response codes (think Restful web services and http response codes for various resource exceptions, IE, Invalid Data, etc).

See more information here and here.

Multi-Tenant Database Design

@Work, we’re a Software as a Service (SaaS) (although we’re evolving into a platform) company providing data integration (perl ETL) and BI reporting and other industry (Business Equipment) based solutions such as territory mapping and compensation programs. Like most SaaS companies, database design, architecture and security are of the utmost importance, but how do you go about selecting the ‘right’ multi-tenant solution for your problem? As you start this requirement gathering and analysis process, you undoubtedly will start coming up with more questions, like :

  • Are there laws and regulations in place that your application / data storage solutions must conform to (Safe Harbor laws, encryption standards, credit cards, etc)?
  • How do you ensure only authenticated users have access to the data their authorized to view / manage?
  • How do we handle fail-over scenarios and reduce our risk for potential transactional data-loss.
  • How do you deal with backups, while keeping the aforementioned items in check?
  • What’s your restoration process, has it been tested (frequency of tests)?
  • How is the performance of your application going to be affected by any design decisions you make?
  • And so on and so on…

When dealing with Multi-Tenant data, I like to design one master schema (db in mysql), and then create one schema for each customer / client / vendor / tenant, etc. This design allows for a quick and easy encapsulation of the customers data, fast(er) backup and restorations processes and it can offer some natural security benefits if you tie the database user into the schema and of course YMMV.

Downtime and / or data loss in 2012 should be unacceptable (sure, there’s maintenance and migrations to consider, etc) to any SaaS providers and the costs associated with them should be VERY expensive. There are various ways to handle the fail-over configurations of MASTER / SLAVE relationships for this design and MySQL’s replication and / or a continuent solution may be the right one for you regarding this critical topic and again, YMMV. I’ve used a combination of solutions in the past, but the basic premise is to keep it as simple as possible while maintaining near 100% uptime and redundancy across the MASTER / SLAVE clusters.

There are many great articles available on the web that go into this concept in much greater detail than I will, with this one being one of the top resources available.

2012

Happy New Year!

I’ve challenged myself to really make an effort and contribute more to my blog. I’ve set a goal to write 2-3 entries a month (this one won’t count)…

Some technical area’s I’ll be touching on will include :

– Webservers (nginx and apache)
– RESTful web services and best practices
– Perl (Moose, Mouse, Moo + OOP,MVC, refactoring, FAT Models)
– Javascript (JQuery, Sencha Ext Js + Touch)
– Databases (Postgres,MSSQL,Oracle,MySQL)
– OSS
– Project Management
– IT Management

I’m also considering coming up with a Open Source project, hosting it on git-hub and creating several entries about the entire life-cycle of the project.

Have a great 2012 and thanks for visiting!

Silver bells, are you listening?

Christmas time is quickly drawing near. One of my favorite parts of the holiday season is seeing family and friends. Sometimes our family and friends live too far way to visit, but I love seeing their Christmas cards. But I don’t know what is better – photo cards or the personal letter.

Shutterfly offers the perfect way to do both. We should know because we did it this last year and everyone we talked to loved our cards. It isn’t hard to know why when you check out their amazing collection. I mean you can’t go wrong.

Not into sending Christmas cards? Or are you waiting to get the perfect photo on Christmas? How about a card to ring in the new year?

Add some fun extras like a return address label or a photo gift and you will be the hit of the holiday season.

But wait that’s not it, Shutterfly has an awesome selection of photo gifts. My favorite? The photo book. Make a book to say thanks to your parents, grandparents, boss, daycare, BFF. Who can resist these amazing gifts. I know I have about 20 of them at my house and love looking through all of them.

Do you blog? Want a chance at 25 free cards this holiday season? Register here:*http://goo.gl/DDw7Q*

It’s the most wonderful time of the year…

Hello, it’s the computer nerd’s wife and while this post won’t contain binary code, it does contain super html links, some amazing digital photo work, and freebies for other nerd blogies. Right, blogies is a real technology term???

The holidays are right around the corner and what a perfect time to celebrate family, reminisce about the past, and look forward to a bright 2011. I know 2011 can’t be much better than celebrating 42 day (101010) this year but we do have 11111 to look forward to! And why not celebrate by sending amazing photo cards to all your friends and family.

Shutterfly introduced amazing new holiday cards this year and I know I can’t wait to place my order for me, Mr. Computer nerd, and baby nerd. These cards add to their already amazing collection of photobooks, gifts, desk calendars, and even mouse pads!! But let’s just stick to the cards for now because I am easily distracted by pretty things. Choose your own card/invitation their Cards & Stationery page.

I know I have already logged several hours on the interwebs picking my faves and if you want to save yourself time stick to these winners!

Christmas Letter – what better way then to send a card with 1,150 words. This card lets you send a little message as well as including a photo (which we all know is worth 1000 words)!
Christmas
Or, how about this one a la David Letterman with a Top Ten Countdown!
null
Or finally be like us and write your own message is this super cute folded card!
null
Because I am a picture junkie, I know Shutterfly prints with amazing quality and ships in a timely manner. I just can’t wait until my cards arrive!

So hurry up and order! Are you a blogger or blogie? Well then score some free loot!

Bloggers get 50 free holiday cards from Shutterfly… sign up: http://bit.ly/sfly2010

Possible bug or config problem

In my previous post, I ran through a short tutorial on using nginx as a front-end web proxy for your Catalyst application running on a Starman / PSGI webserver. I just uncovered a problem, and I can’t tell if it’s a configuration problem on my end, or an actual bug…

If I have a GET request, like so : ‘/widgets//view’ or a POST request ‘widgets//update’ for a new `widget`, my double forward slash is being omitted and instead, Catalyst is instead handling my request as ‘/widgets/view’ or ‘widgets/update’ and the response is a 404 page not found error. (Note, for an already existing widget, the double forward slash would be replaced with a primary key identifier of the current `Widget` : /widgets/1542151/view’, etc).

Running Apache + FastCGI, the default catalyst HTTP server, or Starman (via localhost:5000 requests) do not yield this error. Which leads me to believe this problem is stemming from my nginx web proxy. Probably some sort of configuration problem on my part.

To test, I will start an instance of the catalyst web server in place of the starman server (should be an easy change, start the catalyst server and change the port number in my web proxy to 3000 instead of 5000)…

UPDATE :

HERE

Basically, change the proxy_pass from http://localhost:5000/ to http://localhost:5000

HOWTO : nginx, Starman / Plack and Catalyst

In this short tutorial, I will demonstrate how to use nginx as a frontend web proxy and Starman as a backend web server running a Catalyst application.

It assumes you already know about PSGI, Catalyst, nginx and how to install all the required libraries and perl modules.

Requirements :

First, we’ll configure nginx to run as our lightweight http server.  (If you require SSL, please make sure your ./configure command includes the –with-http_ssl_module option and that you have the openssl and pcre libraries already installed)

I’m going to require a ssl connection with a self signed certificate (you can follow the instructions here on how to set that up). I will also be redirecting all http requests to https via the web server (which is the best way to handle this).

In your nginx directory (mine was installed at /usr/local/nginx) edit your conf/nginx.conf file :

... (params)
http {
... (params, default server, etc)
  server {
    listen 80;
    server_name   myapp.example.com;
    location / {
      rewrite         ^/(.*)$ https://myapp.example.com/$1 permanent;
    }
  }

  server {
    listen        443 ssl;
    server_name   myapp.example.com;
    gzip on;
    ssl_certificate /usr/local/nginx/conf/server.crt;
    ssl_certificate_key /usr/local/nginx/conf/server.key;
    #you may need or want to set additional ssl parameters (cipher, etc)

    location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Port 443; #this is important for Catalyst Apps!
      proxy_pass http://localhost:5000; #changed from http://localhost:5000/ which was causing double forward slash problems in the url
    }

  location /static {
    root /data/users/MyApp/root;
    expires 30d;
  }
}

This assumes that your static content will be servered by nginx and not your Catalyst App, and that when you start your starman server, you’ll be using port 5000.

To get your Catalyst up and running using Starman, install the Plack and Starman via cpanminus as well as the Catalyst::Engine::PSGI module for PSGI integration. Once all of your modules are installed, you’ll need to create a app.psgi file.  You can do so manually, or via the script/app_create.pl command in your Catalyst App directory.

As I found out here from my own answer, when using a front-end web proxy, you’ll need to add a small bit of information to your .psgi file (Catalyst::Engine::PSGI docs), here’s an example of mine :

#!/usr/bin/env perl
use strict;
use warnings;

use Plack::Builder;
use MyApp;

MyApp->setup_engine('PSGI');
my $app = sub { MyApp->run(@_) };

builder {
 enable_if { $_[0]->{REMOTE_ADDR} eq '127.0.0.1' } 
        "Plack::Middleware::ReverseProxy";
 $app;
};

If your instance of nginx is running on a different server, you may need to set this configuration variable in your Catalyst App (doesn’t hurt to add it regardless of your situation) :

MyApp->config(using_frontend_proxy => 1)

Start starman (currently defaults to 5 processes and port 5000) :
$ starman script/erp.psgi

Start nginx
$ sudo /usr/local/nginx/sbin/nginx

At this point, all of your http://myapp.example.com requests should be automatically redirected to https://myapp.examples.com. Nginx should be handling all of your static content used by your Catalyst app, and as long as Starman is running in the background on port 5000, your requests and responses should be handled accordingly and in a timely manner! 😉

In a production production environment, you may want to tweak some of your settings in the nginx.conf file and then create a startup script for both nginx and your Starman server. I also haven’t touched on any of the logs associated with Starman / Plack and your Catalyst app, or how to handle the ip addresses (nginx if forwarding the $remote_addr as `X-Real-IP` in the header) for access / error logs…