| Home | Wiki | OI 1.x Docs | OI 2.x Docs |
|
This document goes over issues concerning system administrators, including mod_perl/Apache compilation and configuration, tips for running multiple websites under one mod_perl server, package configuration, logging, error reporting and caching.
While you're reading, it would be handy to have within reach the OpenInteract Glossary to lookup certain terms (like 'website', 'handler', 'package', etc.).
Distributed with OpenInteract is the script
oi_manage. If you installed OpenInteract via the
normal Perl mechanisms, you should have it in your
/usr/bin or /usr/local/bin directory so
it's always accessible. (You might also ensure that
/usr/local/bin is in your path and your users' paths,
or that you create a symlink from /usr/bin/oi_manage
to /usr/local/bin/oi_manage.)
This script takes care of many actions for you. Here's a list and a brief description of each:
Commands by the Administrator:
install - Install OpenInteract to base directory
upgrade - Upgrade core OpenInteract packages
install_package - Install package to the base
Commands by the Package Developer:
create_skeleton - Create a skeleton package for development
export_package - Export package(s) to distribution file(s)
check_package - Ensure that package passes initial inspection
Commands by the Website Creator:
create_website - Create a new website
apply_package - Install a package from base to website
upgrade_package - Upgrade a website package
remove_package - Remove a package from a website
install_sql - Install the SQL for packages
install_template - Install package templates to the database
dump_template - Dump package templates to the filesystem
remove_template - Remove package templates from the database
refresh_doc - Sync website docs to base installation docs
test_db - Test database settings in 'server.perl'
change_spops_driver - Change the SPOPS driver for your objects
Other commands:
initial_packages - List packages marked as 'initial'
list_packages - List packages installed to app or base
list_actions - List actions currently implemented in website
You can read an online text
version of the documentation for oi_manage For
the most up-to-date information, do a perldoc
oi_manage from the command line to view the extensive
documentation.
This section reviews installation, layout and gives a very broad overview of the packaging system.
There are two environment variables you can use too make your typing life easier:
We encourage you to use these as much as possible. However, we continue to use the long form in the examples below.
Installation of OpenInteract is very simple. First, you need to install the Perl module in the normal fashion. (CPAN or by-hand.) Next, run:
cd /path/where/I/unpacked/OpenInteract-x.xx oi_manage install --base_dir=/path/to/install
Easy! Read the INSTALL and
INSTALL.website files that were packaged with the
OpenInteract distribution for more details.
OpenInteract is composed of two pieces: the installation and
the websites. The installation is what you create when you run the
oi_manage install command above.
And you refer to it when you create new websites:
oi_manage create_website --base_dir=/path/to/install \
--website_dir=/path/to/website \
--website_name=MySite
Thereafter, the website keeps the information for the base
installation directory in its conf/base.conf file
described below. So you shouldn't need to refer to the
installation directory too often. (Developers need to know it when
they are developing packages, but that's a separate issue.)
So OpenInteract exists in two parts: the installation and the various websites that use the installation. When users who run the websites want to add new functionality by installing a new package or upgrading an existing one, they must first add the package to the OpenInteract installation and then apply it to the website.
The OpenInteract installation maintains a copy of every version of every package ever installed to the system. This might seem backwards or unnecessary, but it ensures that different websites can use and continue to use different versions of the same packages. This should allow you to accommodate different types of users: bleeding-edge developers who always play with new versions of packages and stay-the-line website maintainers who just want everything to work the same day after day.
Fortunately, the oi_manage script makes this whole
process easy for us:
Install a new package to the installation:
oi_manage install_package --package_file=/path/to/file.tar.gz \
--base_dir=/path/to/install
Install a new version of an existing package to the installation:
oi_manage install_package --package_file=/path/to/file.tar.gz \
--base_dir=/path/to/install
Hey, those two were the same thing! It's true. Since the installation maintains a copy of every version of every package, it doesn't actually 'upgrade' a package in the installation.
This just emphasizes the main point: when you install a package to the OpenInteract installation, you're simply making it available for the websites that are using this installation.
Apply an installed package to a website:
oi_manage apply_package --package=newpackage \
--website_dir=/path/to/website
Upgrade to a new version installed package for a website:
oi_manage upgrade_package --package=newpackage \
--website_dir=/path/to/website
Ah, now those two commands were different. This shows that a
website can only have one version of a particular package
installed. If you try to apply_package for a package
that has already been applied to a website, you'll get an error
and no action will be taken. In this case you need to use
upgrade_package.
The docs for oi_manage discuss this next point but
it's worth repeating: when you use upgrade_package,
you merely update the registry information for the website. You
also install new data and structure, object configuration,
templates, handlers and everything else.
However, the system does not remove the old files. The system does not know if you want to keep the changes you've made or discard them, so it simply keeps the old directory around and allows you to copy information back and forth as you need. But this old directory is not used by the system anymore, and if you make changes to files in the old package directory they will not be reflected in the website.
We generally use one of three formats. If you create new configuration files, try to stick to these naming schemes.
.conf
Typical configuration file format: information separated into key-value pairs (separated by whitespace), blank lines and lines that begin with a comment (#) are skipped.
Example:
MyValue emmet otter HerValue fraggle rock TheirValue jughandle # ringo starr
Parsing this would return a hashref:
{
MyValue => 'emmet otter',
HerValue => 'fraggle rock',
TheirValue => 'jughandle'
}
.dat
Very simple: one item per line. Blank lines and lines beginning with a comment (#) are skipped entirely.
Example:
MyClass HerClass TheirClass #RingoClass
Parsing this would return an arrayref:
[ 'MyClass', 'HerClass', 'TheirClass' ]
.perl
This file is a full-fledged perl data structure, dumped to a
file using Data::Dumper. It can be any type of
structure, but it's normally used to represent a hashref
containing all sorts of different types of information. It's also
fairly easy to edit such a file using your favorite plain-text
file editor.
When reading this type of configuration, we just return the data structure saved in the file -- if the file is an arrayref, we return an arrayref.
When we use this structure to save information for objects
(such as the OpenInteract::Config::PerlFile object),
we never save class information about the object, just the data.
We can always re-bless the structure after it's eval'd in.
Example:
$data = {
'db_info' => {
'db_owner' => '',
'username' => 'test',
'password' => '',
'dsn' => 'mydb',
'db_name' => 'myopeninteract',
'driver_name' => 'mysql',
},
...
};
All files referenced in this section are contained in the home directory of the website. For instance, if you install a website using:
oi_manage create_website --base_dir=/path/to/install \
--website_dir=/path/to/website \
--website_name=MySite
The files described here would be in
/path/to/website.
We start with the simple files first and then get into the main configuration file.
This is one of the most important configuration files in
OpenInteract -- fortunately, nobody should never need to edit it.
:-) This file allows
OpenInteract::Startup to bootstrap all the
configuration information by supplying information for the class
that reads in the data for the server configuration
(OpenInteract::Config::PerlFile by default), the base
directory for this application, the name of the configuration
directory and file. You can also specify the Request class
(OpenInteract::Request by default) and a Stash class
(no default specified -- every application needs to have its
own).
Example:
base_dir /opt/OpenInteract website_dir /home/httpd/mysite.com website_name MySite config_type perl config_class OpenInteract::Config::PerlFile config_dir conf config_file server.perl request_class OpenInteract::Request stash_class MySite::Stash
List the classes needed by mod_perl in the
Apache:: class. You should never need to change
this.
This is not a configuration file but included here for
completeness. It is a text-based file (a Perl data structure in
Data::Dumper format) and includes information about
the packages in the website. You should never need to edit this
file, but it's in a text-based format just in case you do.
This is the main website configuration file and you may find yourself editing this quite often in the beginning and then every once in a while thereafter. The file is quite well-commented so you have the information about what/how to configure next to the actual data to change.
When you create a new website using oi_manage you
get a starter server.perl file with a number of items
filled in. (View the sample
server.ini file -- note that keys surrounded with '%%' will
be replaced when you create a website with
oi_manage.
This configuration is always available within the OpenInteract
environment (as an OpenInteract::Config object) to
developers and is read in anew every time the server starts. If
you change the file while the server is running the changes are
not picked up by the running server. You need to restart to
make the change happen. (This might change as the need
arises.)
The file itself is formatted in Perl and you can edit it with your favorite plain-text editor. You should be able to check whether it is syntactically valid by doing:
perl -wc server.perl
This won't tell you whether it's functionally
valid, but it's a first step. Note that there is a routine in
oi_manage that allows you to check whether the
parameters defined in server.perl will allow a
database connection to be made:
oi_manage test_db --website_dir=/path/to/my/website
The actual names of all directories are determined by this website configuration file as well. Here is an example with some commonly-used directories:
'dir' => {
'base' => undef, # replaced in OpenInteract::Startup
'interact' => undef, # replaced in OpenInteract::Startup
'error' => '$BASE/error',
'html' => '$BASE/html',
'log' => '$BASE/logs',
'cache' => '$BASE/cache',
'config' => '$BASE/conf',
'data' => '$BASE/data',
'mail' => '$BASE/mail',
'overflow' => '$BASE/overflow',
'help' => '$HTML/help',
'download' => '$HTML/downloads',
'upload' => '$BASE/uploads',
},
Note that the 'dir' entry is one of the few places in the configuration where you can use expandable macros to determine the configuration value. As you might guess '$HTML' expands to find what the entry for 'html' is, which then depends on the value for 'base' is. You are not restricted to using the '$HTML' and '$BASE' macros.
Also, you can refer to '$BASE' even though the entry for 'base'
is undefined. The standard startup.pl reads the value for
'website_dir' set in the httpd.conf and sets it dynamically once
the config file is read in. This way you should only have to set
your directory in one location. (And we should fix the slightly
confusing naming scheme as well...)
Below are brief descriptions of what each directory should hold.
startup.pl
file so anything you set in the configuration file will be
overwritten.)startup.pl file so anything you set in the
configuration file will be overwritten.)Data::Dumper
format) to this directory, using the ID of the error as a
filename. Note that the error browser does not currently view these
items, but this may change in the future.startup.pl> file
is also kept here.As a convenience, OpenInteract allows you to setup aliases so you do not have to put all sorts of class names in your code. For instance, instead of using a hardcoded classname:
my $user = eval { OpenInteract::User->fetch( $uid ) };
You can use:
my $user = eval { $R->user->fetch( $uid ) };
These aliases are setup automatically from the SPOPS classes, and you're also given the opportunity to setup any additional aliases using the 'system_aliases' configuration key. Here's an example:
'system_alias' => {
'OpenInteract::Cookies::Apache' => [ qw/ cookies / ],
'OpenInteract::Session::MySQL' => [ qw/ session / ],
'OpenInteract::Template::Toolkit' => [ qw/ template / ],
'OpenInteract::PackageRepository' => [ qw/ repository / ],
'OpenInteract::Package' => [ qw/ package / ],
'OpenInteract::Error' => [ qw/ error / ],
'OpenInteract::Auth' => [ qw/ auth auth_user auth_group / ],
'%%WEBSITE_NAME%%::Security' => [ qw/ security_object object_security security / ],
'SPOPS::Secure' => [ qw/ secure / ],
'OpenInteract::Error::Main' => [ qw/ error_handler / ],
'OpenInteract::Handler::Component' => [ qw/ component / ],
},
As you can see, you can setup multiple aliases to refer to the same class. For instance, any of the following will return the class 'OpenInteract::Auth':
$R->auth $R->auth_user $R->auth_group
Note that using aliases like this is an as-yet-unmeasured performance hit, because every time you call the alias, $R needs to determine which application you're using so it can return the correct alias. This is a manifestation of the problem with running more than one application on the same server: what happens if two entirely separate applications want to setup an alias for 'news'?
You should never need to setup the alises by hand -- the
standard startup.pl file will take care of this for
you. However, if you're curious... You can setup the alises by
calling:
my $hr_alias = OpenInteract::Request->ALIAS;
Which returns a hashref to you. Any changes you make will be reflected in the master list of aliases. The hashref is setup as follows:
$hr->{ $alias }->{ $stash_class } = $aliased_class;
So when you're setting up the aliases, you need to be sure and include your $stash_class in the alias, or else you'll get all kinds of errors.
Finally, we push the task of initializing the aliases (translating them into subroutines so they will execute quickly) into the ChildInitHandler, so we can ensure that all the applications have had their chance to modify the alias information before we create the subroutines.
All debugging messages generated by the OpenInteract system are put into STDERR, which in apache is sent to the error log. You can always get the debugging level of the current website by running:
my $website_debug_level = $R->DEBUG;
A common logging idiom, using the
C
$R->DEBUG && $R->scrib( 1, "My debugging message" );
This first checks to see if debugging is on and only then calls
the C
SPOPS modules do not obey this debugging level, and if you want
to get debugging information from them you need to set the package
variable $SPOPS::DEBUG. This turns on debugging for
all SPOPS calls across websites.
OpenInteract comes hooks for caching. While the caching
framework is fairly generic (you could cache data in a SQL
database or DBM file if you wished), the default caching mechanism
is uses the File::Cache module and the
filesystem. The module is fairly good about keeping the cache
under a certain size (set in the configuration file), and you can
also control the 'depth', which means that it creates
subdirectories for the cache entries rather than putting them all
into one big directory. Again, this is set in the configuration
file. It's worth experimenting with this function as it works on
your filesystem: journaling filesystems might be able to handle
many thousands of entries in a single directory, whereas others
might start having performance degradations after more than 500.
Most (all?) modern filesystems will put frequently accessed files into an in-memory cache, which can be a big win for caching. However, if yours does not one alternative is to put the caching directory itself in memory by using a RAM disk. Setting up such a filesystem is beyond the scope of this document.
NOTE: DO NOT restart the Apache/mod_perl process using the HUP signal. Your modules will not get reloaded properly.
OpenInteract depends on a persistent Perl environment within a web server. Currently, the best alternative is mod_perl.
mod_perl is extremely powerful, but this power can come at a price. Embedding Perl into Apache uses more resources (particularly memory) than just using Apache alone. A number of developers have experimented with various ways of minimizing the memory footprint of mod_perl, and one of the easiest and best performing methods is to use a proxy server.
This is described in great detail in the mod_perl guide under the Choosing the Right Strategy heading. But we'll summarize here:
The benefits of this are:
Running OpenInteract in this environment is strongly recommended, and it comes with configuration files that make it easier to do the Right Thing.
First, you need to get the mod_proxy_add_forward
module make available by Ask Bjoern Hansen. Retrieve it from:
http://develooper.com/code/mpaf/mod_proxy_add_forward.c
Once you've retrieved the file, copy it into the
src/modules/extra directory of your Apache source
code directory. An example of the 'activate-module' and
'enable-module' directives to put this module into your Apache is
below in the as well as in the source code for
mod_proxy_add_forward itself.
Once you've retrieved the extra module and copied it to the
right place, you can create apache and mod_perl with the following
steps. Note that this assumes you have not installed apache from
source before and that you're installing to the directory
/usr/local/apache -- modify as needed.
1. >> tar -zxvf apache-1.3.12.tar.gz
2. >> tar -zxvf mod_perl-1.24.tar.gz
3. >> cd apache-1.3.12
4. >> ./configure --prefix=/usr/local/apache \
--enable-module=rewrite --enable-module=proxy \
--activate-module=src/modules/extra/mod_proxy_add_forward.c \
--enable-module=proxy_add_forward
5. >> make
6. >> make install
(proxy server binary is now installed as /usr/local/apache/bin/httpd)
7. >> cd ../mod_perl-1.24
8. >> perl Makefile.PL EVERYTHING=1
# Configure mod_perl with ../apache_1.3.12/src ? [y]
9. >> y
# Shall I build httpd in ../apache_1.3.12/src for you? [y]
10. >> y
11. >> make
12. >> make test
(note: if this fails due to an error with URI::URL, set the
environment variable 'PERL_HTTP_URI_CLASS' to 'URI::URL', with
something like:
# export PERL_HTTP_URI_CLASS=URI::URL
13. >> make install
(mod_perl Perl modules are now installed)
14. >> cp ../apache-1.3.12/src/httpd /usr/local/apache/bin/httpd_modperl
(mod_perl-enabled Apache is now installed)
This is a very simple method for creating both a lightweight proxy Apache binary and a heavyweight mod_perl-enabled Apache binary. See the mod_perl Guide for many, many more details about building mod_perl.
It is strongly recommended that you do not build mod_perl using DSOs and that you do not use pre-built versions such as those supplied by RedHat with its RPMs. However, using the DSO mechanism probably works fine for the front-end proxy server.
Use oi_manage! Use OI_MANAGE! Use
OI_MANAGE!
The oi_manage script included with OpenInteract
performs a number of tasks for you that make your life much
easier. When you run the create_website command along
with the appropriate parameters, oi_manage will copy
configuration files from the base installation to your website
directory and customize them for your website's parameters for
you.
For instance, two of the files that are copied to your
website's conf/ directory are
httpd_static.conf and
httpd_modperl.conf. (See httpd_static.conf and httpd_modperl.conf -- the
items marked with '%%' are replaced in the customization process.)
You will still need to edit a few parameters in them --
oi_manage is pretty smart, but it can't find out
which IP address you want your website to listen to! -- but much
of it is filled in for you.
After you've run oi_manage, you will need to
modify a few parameters in the static Apache configuration file.
Listen
directive. You will also need to specify the NameVirtualHost
directive in your main Apache configuration file.Proxy configuration is fairly simple. Every rule (starting with
RewriteRule) is processed in order. Once a rule is
met, no further rules are processed unless the satisfied rule
specifies it.
The default proxy configuration assumes that the only static files you will want to serve directly from the proxy server are images. This action is specified by this line:
RewriteRule ^/images - [L]
If you want to add other locations that will be entirely served by the lightweight server, just add them after this line. For example, if my website had a directory '/forms' where we kept PDF versions of forms for our customers to fill out, I could add:
RewriteRule ^/forms - [L]
And every URL beginning with /forms will be
answered by the front-end lightweight server. The [L]
stands for "Local" and means that you want this server (the proxy
server) to handle the request.
The only word of warning here is that as an administrator you
might need to keep an eye on what the back-end server is using for
URLs. For instance, say I entered this /forms
configuration directive and later a developer on the back-end
server tries to configure OpenInteract to perform a certain action
when given the /forms URL. Unless the developer knows
that the front-end server is answering all the /forms
URLs, she will have a very frustrating time trying to figure out
why her handler isn't responding.
After you've run oi_manage, you will need to
modify a few parameters in the mod_perl Apache configuration file.
(Note: You can skip the remainder of this section if you
just want to get something up and running. The
oi_manage script takes care of all this for you. But
if you're curious, read on.)
Four separate items need to be customized in the
conf/httpd_modperl.conf file:
First, define the library paths for this website. Note that this is applied on a server-wide basis, so be careful of namespace clashes.
Example:
<Perl> use lib qw( /home/httpd/mysite.com ); </Perl>
Second, define a parameter that allow us to bootstrap our configuration object which contains the rest of everything we need to know. The parameter is 'StashClass' you set it to a value that OpenInteract can use for a website's stash class.
Example:
PerlSetVar StashClass MySite::Stash
Third, you need to bring in your
startup.pl. (Information on what is done in the
startup.pl is found in the OpenInteract
Developer's Guide.)
PerlRequire /home/httpd/mysite.com/conf/startup.pl
Fourth and finally, we need to ensure that every request
coming in goes through a single Apache content handler:
OpenInteract.pm. (This module is located in the
base package.) To enable this, just do:
<Location /> SetHandler perl-script PerlHandler OpenInteract </Location>
This Apache content handler is in the base package
since it's part of the base functionality of the framework. We can
just say "OpenInteract" in the httpd.conf file because we have
already included the library in our startup.pl.
OpenInteract is a fairly heavyweight application. Its numerous modules and, more importantly, package versioning necessitate that each application run under its own mod_perl server. Multiple applications can easily be run on the same machine, and can share the same base package repository. But experience has shown that trying to run multiple applications under the same set of processes is nearly impossible unless everything is kept perfectly in sync. And that never, ever happens.