Bric::Hacker(3)

SUBMITTING PATCHES
Patches should be generated using the "svn diff" command on each of the files modified, from the root directory. For example, if you made changes to lib/Bric/Changes.pod and comp/foo.mc, you would generate a diff by running this command from the root of your Subversion checkout:

  svn diff lib/Bric/Changes.pod comp/foo.mc > patch.txt

If you created one or more new files in your changes then you'll have to add them to the patch separately using normal "diff" against "/dev/null". For example, if you created the file "inst/upgrade/1.9.1/solve_fermat.pl" then you would add this to patch.txt with:

  diff -u /dev/null inst/upgrade/1.9.1/solve_fermat.pl >> patch.txt

Always create patches using an up-to-date Subversion checkout if possible. Send your patches to the devel list mentioned above.

APPLYING PATCHES
Patches created using the method above can be applied using "patch" with the "-p0" option from the root of your Subversion checkout:

  patch -p0 < patch.txt

Make sure you check the results with "svn diff" before committing.

CODING STANDARDS

  Perl
Try to follow the style of the existing Bricolage code. Except where it is bad, of course. Basically, write in the style you see in most Perl books and documentation, particularly perlstyle.

Although historically the Bricolage code has not enforced whitespace rules, we now request that you use 4-space indents (2 spaces for continued lines) and discourage the use of tabs. The following settings for some of the more popular editors are thus recommended while editing Bricolage source code.

Emacs

We strongly recommend that you use "cperl-mode" while editing Bricolage sources in Emacs. Grab the latest version from the CPAN, install it, and then place the following in your "~/.emacs" file:

  (setq-default indent-tabs-mode nil)
  (custom-set-variables
   '(case-fold-search t)
   '(cperl-close-paren-offset -4)
   '(cperl-continued-statement-offset 2)
   '(cperl-indent-level 4)
   '(cperl-indent-parens-as-block t)
   '(cperl-tab-always-indent t)
   '(indent-tabs-mode nil))
   ;; let hashes indent normally; I think this requires
   ;; at least version 4.32 of cperl-mode.el
   '(cperl-indent-parens-as-block t)
   '(cperl-close-paren-offset -4)

Also, if you'd like to take advantage of the full functionality of "cperl-mode" and have it automatically parse all Perl source files, add these settings, as well:

  (defalias 'perl-mode 'cperl-mode)
  (setq auto-mode-alist
        (append
         '(("\\.\\([pP]\\([Llm]\\|erl\\)\\|al\\|pod\\)\\'" . cperl-mode))
         auto-mode-alist))
    (setq cperl-hairy t)
    (setq interpreter-mode-alist (append interpreter-mode-alist
       '(("miniperl" . cperl-mode))))

When editing Bricolage Mason components, "mmm-mode" can help. It's Mason mode will parse Mason component files and use "sgml-mode" in HTML spaces and "cperl-mode" in Mason blocks. Grab it from <http://mmm-mode.sourceforge.net/>, install it, and then add the following to your "~/.emacs" file to have it automatically parse your Bricolage Mason component files:

  (add-to-list 'load-path "/usr/local/share/emacs/site-lisp")
  (require 'mmm-mode)
  (require 'mmm-mason)
  (setq mmm-global-mode 'maybe)
  (add-to-list 'auto-mode-alist '("/usr/local/bricolage/comp" . sgml-mode))
  (mmm-add-mode-ext-class 'sgml-mode "/usr/local/bricolage/comp" 'mason)

And if you need to examine the Mason object files created by Bricolage in order to chase down bugs and such, you can use the "cperl-mode" in those files by adding this to your "~/.emacs" file:

  (add-to-list 'auto-mode-alist '("/usr/local/bricolage/data/obj" . cperl-mode))

Vim

Rafael Garcia-Suarez has written a Vim indent macro which (for the most part) duplicates the behavior of Emacs "perl-mode". It is now, as of Vim 6.0, included in the Vim distribution and should be found in "$VIMRUNTIME/indent/perl". The easiest way to use it though is to place the following line in your .vimrc file:

    source $VIMRUNTIME/indent.vim

You'll also need to add these lines.

    set tabstop=8
    set softtabstop=4
    set shiftwidth=4
    set expandtab

The first three lines make Vim duplicate the behavior of Emacs in creating the appearance of 4 space tabs with a mix of tabs and spaces. This is necessary for reading older Bricolage files which were written this way, and haven't yet been re-tabbed.

The expandtab setting does the Right Thing under the new rules, in that it doesn't use tabs at all, only spaces.

SQL

The standard for writing SQL in Bricolage is pretty straight-forward: format the SQL in 80 columns or less, and use heredocs where possible. For example:

  my $sql = "<<    END_SQL";
      SELECT id, name, description, publish_date, cover_date, current_version,
             publish_status, active
      FROM   story
      WHERE  id = ?
      END_SQL

Also, when writing your queries, please follow the following rules with regard to table aliases:

•   Do not alias any table names in queries, unless those table names are longer than 8 characters, or are referred to in the FROM clause more than once. Under this rule, for example, "story" and "member" would never be aliased unless referenced twice.

•   For table names over 8 characters, abbreviated aliases are acceptable provided that they are still long enough to be informative. i.e. "st_cat" instead of "sc" for "story_category".

•   For multiple links to the same table, use an alias which is long enough to be informative, such as "story2" or "inst_2" for the second reference to "story" or "story_instance".

The reason for these rules is that single-letter aliases for queries are as unreadable and unmaintainable as single-letter variables names. You get halfway down the page, and you can no longer remember to what tables they refer.

The rules above are actually taken from O'Reilly's "Introduction to PL/SQL Programming," which has an excellent chapter on code cleanliness in SQL and SQL-extension languages.

  OTHER GOOD PRACTICES
Keep subroutines short. Each subroutine should handle one task. For example, if you have a subroutine "get_foo", and getting "foo" requires getting "bar" and "qux", then make two more subroutines "get_bar" and "get_qux" and put them in the "private functions" section. Also if you duplicate some code in several subroutines, factor it out into another subroutine. It's easier to maintain one subroutine than to maintain three.

Limit the width of your code to 80 characters if possible. This makes them easier to read. Keeping subroutines short helps to meet this goal, as well.

Make enough comments so that someone maintaining your code can understand what is going on. Comments shouldn't be redundant with the code. They should explain non-obvious code. Comments can be bad in some cases if code changes and the corresponding comment is not kept in sync. So keep the comments in sync!

Use Perl idioms if it is clear and concise, but use them with care. Implicit variables can be slick, but hard to understand; add a comment if you use uncommon ones.

Use Mason components only for display, and put business logic into library modules and callback components. But only for as long as callbacks are in components! We hope to move them into libraries soon, at which time this rule will become even more rigid.

For POD, generally you copy/paste it from another module. Recently we are trying to cut out some of useless things like "B<Side Effects:> NONE". For a canonical example, see "Bric::Biz::Site".

Avoid magic values, i.e. don't hard-code numbers into code. It tends to become a maintenance problem if you put the number 1023 throughout a module and then somewhere else you have 1021; is 1021 related to 1023 (1023 - 2), or is it just another random number? And what does 1023 mean? Put a constant at the top of the module and use that, instead.

In the end, just try to follow the existing code style, and take into consideration any feedback you get from the mailing list when patches are submitted.

TESTING
All Bricolage patches should be accompanied by the necessary tests. You are strongly encouraged to write comprehensive tests that thoroughly test whatever changes or additions you make to the Bricolage API. You are also strongly encouraged to write tests for the current API, if you find that adequate tests have not yet been written (and this is true in a great many places, unfortunately). Although Bricolage does not yet support UI tests, we take our API testing seriously, and so should you. Here's what you need to know to write tests for Bricolage.

Bricolage contains a test suite based on Test::Class. The test classes live in the t/Bric directory, and all subclass Bric::Test::Base. If you're familiar with Test::More, then the syntax for how the Test::Class-based classes work should be pretty readily apparent. See the Test::Class documentation for details. It's definitely worth a read. If you haven't used Test::More, its documentation is also a must-read.

Bricolage has two different sets of tests, those run by "make test" and those run by "make devtest". The former are stand-alone tests that don't rely on the presence of a Bricolage database to be run. They can thus be run before "make install". The idea here is that we offer a basic set of tests that can be run via the standard Perl installation pattern of

  perl Makefile.PL
  make
  make test
  make install

The tests run by <make devtest> are intended to be run during development. They require that a Bricolage database be installed and running. They will "SELECT", "INSERT", "UPDATE", and "DELETE" data from that database, so "make devtest" should never be run against a production database. Indeed, you should in general have a clean, fresh database installation to run the tests against. Although the test suite makes every effort to clean up after itself by "DELETE"ing data it has added to the database, it is unfortunately imperfect, and extra data will be left, especially in Group-related and Attribute-related tables.

So, once more, "make devtest" should never be run against a production database. 'Nuff said.

You can get a clean database without reinstalling Bricolage by doing

  sudo make db
  sudo make db_grant

Answer yes when it asks you to drop the database. (Note: if there were any database changes, you'll also need to run upgrade scripts in inst/upgrade/.) Then to get useful debugging output, do verbose testing with something like

  sudo make devtest TEST_VERBOSE=1 > test.out 2>&1

All of the tests are run by the inst/runtests.pl script, which in turn tells Test::Harness to execute t/Bric/Test/Runner.pm. This file will find all the necessary test classes, load them, and then run their tests. inst/runtests.pl takes a number of arguments to simplify the running of scripts, including a list of test files or classes to run, so that you can just run the tests you need to run while you're developing new tests. "perldoc inst/runtests.pl" for more information.

Be sure to use the testing base classes in your test classes. For non-database dependent tests (which are always in files named Test.pm, use Bric::Test::Base. For development tests, use Bric::Test::DevBase, which inherits from Bric::Test::Base. Be sure to read the documentation in these classes, as it will help you to write your tests more effectively. Pay special attention to the methods added to Bric::Test::DevBase, as they're there to help you clean up any new records you've added to the database.

And finally, when you do provide patches to Bricolage, along with the new tests to test them, make sure that all existing tests pass, as well. This means that you should always run "make devtest" on a fresh database build with your changes. Furthermore, if your patch involves changes to the database, you should provide the necessary upgrade script in inst/upgrade/, and also run the tests against a database that has been built from Bricolage sources untouched by your patch, and then upgraded by your upgrade script. This will help to ensure that your upgrade script modifies the database in the same way as your patch modifies the Bricolage SQL files.

And with that said, happy testing!

DEBUGGING
Bricolage is a complex application and debugging can be difficult. Here are some tips to help you find bugs faster:

•   Turn on "QA_MODE" in your bricolage.conf file. This will cause the UI to display a bunch of useful data at the bottom of every page, including session state, cache state, cookies, and GET and POST args. "QA_MODE" turns on PerlWarn ("use warnings") and causes error messages to include more information. "QA_MODE" also enables Apache::Status at the URL /perl-status which allows you can to examine the state of the Perl interpreter inside Apache.

•   Set the "NO_TOOLBAR" directive to "0" or "Off" in bricolage.conf to keep Bricolage from popping up a new browser window without a toolbar.

•   Run Bricolage under the Perl debugger using the debug command with bric_apachectl:

       bric_apachectl debug

    This will start Apache in single-process mode ("httpd -X") and setup Apache::DB to start the debugger on the each hit to the server. You'll need to install the Apache::DB Perl module to use this command.

    To run Bricolage under DDD (<http://www.gnu.org/software/ddd/>), start ddd as root and load bin/bric_apachectl. Give it the argument "debug" and run it. When you issue a hit to the server the debugger will stop on the first line of "Bric::App::Handler::handler()". From there you can set breakpoints inside Bricolage and debug normally.

    If you prefer to run without the debugger using only the Apache single-process mode, then run Bricolage using the command

       bric_apachectl single

•   Set "DBI_DEBUG" and "DBI_CALL_TRACE" to 1 in your bricolage.conf file. "DBI_DEBUG" records every database call in the logs complete with SQL and arguments. "DBI_CALL_TRACE" adds a a subroutine call trace for each statement showing where the database call originated. This generates a lot of data but it can be very helpful.

•   Look at the database directly using psql. Many bugs in Bricolage can only be successfully diagnosed by examining records created in the database.

PERFORMANCE TUNING
Bricolage has two separate profiling systems that you can use to extract performance data:

•   To run Bricolage under the Devel::Profiler module set the "PROFILE" variable on in your bricolage.conf file and restart your server. This will create a profiler/$$/tmon.out file in your Apache log directory for each server process. You can use dprofpp to analyze these files after the server has been stopped with "bric_apachectl stop". See the Devel::Profiler documentation for more details.

•   The database profiler is activated by enabling the "DBI_PROFILE" option in bricolage.conf. This causes database profiling traces to be written to the Apache error log during requests. You can then use bric_dbprof to analyze these trace. See bric_dbprof for details.

CAUTION: Neither of these options is appropriate for a production system.

ACTIONS AND MOVERS
A relatively simple way to contribute to Bricolage is to provide actions and movers. These are plugin modules that can add new functionality to Bricolage without needing to make changes to the existing API.

An "action" is an act that is performed on files before they are distributed. Say you want to clean the HTML of all of your HTML files before they're distributed. You'll need to create an action to do this. Consult Bric::Dist::Action for information on how to create actions.

Say you need to distribute files via a protocol that Bricolage doesn't currently support -- say, an new variant of FTP called "FooTP." You'll need to create a new mover. Consult Bric::Dist::Action::Mover for details on how to do that.

MERGING CHANGES BETWEEN BRANCHES
If you're a Bricolage developer with permission to commit to the Subversion repository, you may occasionally have to merge changes from one branch to another.

By far, the most common merge will be from a release branch (e.g. rev_1_10) to trunk. For this reason, the documentation below will assume that you are merging from a branch to trunk; however, there is still occasion to merge between branches, e.g. rev_1_6 to rev_1_10. In these cases, use "rev_1_X" instead of "trunk".

1   Make sure that both the branch you are merging from and the trunk are updated and have no uncommitted changes. You can check for these differences by using the Subversion "status" and "update" commands. Running "svn status" will tell you if there have been any uncommitted changes to the local copy of the branch or trunk. Running "svn update" will merge in any updates from the copy on the Subversion server. So we recommend running both of these commands in both the branch and the trunk and committing any necessary changes before continuing.

      % cd bricolage-trunk
      % svn status
      % svn update
      At revision 6123.

      % cd bricolage-branch-checkout
      % svn status
      % svn update
      At revision 6123.

2   To merge the changes from the branch into the trunk, you need to tell the Subversion "merge" command to use the difference between two revisions in the branch to update the trunk. The two revisions to compare are, on the one hand, either the revision when the branch was created, or the revision when the branch was last merged into the trunk; and on the other hand, the current revision in the branch.

    For the latter number, you can simply use the "HEAD" keyword, which represents the latest revision number in the branch.

    For the former number, we have implemented a standard way of recording a merge: we create a new tag for the branch. In Subversion, branches and tags are exactly the same; the difference is only in convention. So the convention for Bricolage is to store tags in the tags subdirectory of the project directory, and branches in the branches subdirectory. The other half of the convention is that no one should ever commit changes to a tag after it has been created.

    Tags are stored in subdirectories of tags based on the branch you are merging from. For example, if you are merging from rev_1_10 to Ftrunk, the tag belongs in tags/rev_1_10/.

    Tags are then named for the branch you are merging to and the revision number of the merge itself. For example, if a merge was committed to the trunk from the rev_1_10 branch in revision 5694, the branch would be located at tags/rev_1_10/trunk-5694.

    So to get the revision number of the last merge, look in the Bricolage ViewCVS interface for the last merge tag created for the branch. Or use the Subversion "list" command to find it:

      % svn list https://svn.bricolage.cc/bricolage/tags/merges/rev_1_10 | grep trunk
      trunk-5694/
      trunk-5825/
      trunk-5902/
      trunk-5995/

    If the branch has never been merged into the trunk you can get the revision number at the time the branch was created using "svn log" command:

      % svn log --verbose --stop-on-copy \
        https://svn.bricolage.cc/bricolage/branches/rev_1_10
      ------------------------------------------------------------------------
      r5525 | theory | 2004-05-10 19:16:48 -0700 (Mon, 10 May 2004) | 1 line
      Changed paths:
         A /bricolage/branches/rev_1_10 (from /bricolage/trunk:5524)

    So now we know that we want to update the trunk with all of the changes between revision 5995 and "HEAD". If no merge tag exists, then use "svn log" to find out revision number at which the branch was created and use that number.

3   Now simply switch to the trunk checkout and perform the merge. Using the above examples, the merge would look like this:

      % cd bricolage-trunk
      % svn merge -r 5995:HEAD https://svn.bricolage.cc/bricolage/branches/rev_1_10
      U  lib/Bric/App/ApacheConfig.pm
      C  lib/Bric/Hacker.pod

    If you're nervous about doing the merge (don't be, it's just your local copy you're updating at this point), you can use the "--dry-run" option to see what files would change without changing them. Or run "svn diff" with the same arguments to see what the differences are:

      % svn merge -r 5995:HEAD --dry-run \
        https://svn.bricolage.cc/bricolage/branches/rev_1_10 > test.diff

4   Resolve any conflicts (hopefully none, but they do happen occasionally). Notice above that the result of the merge is a conflict in lib/Bric/Hacker.pod. To resolve the conflicts, follow the directions in the Subversion book, published online here: <http://svnbook.red-bean.com/en/1.1/ch03s05.html#svn-ch-3-sect-5.4>.

5   Rebuild the database and run all of the tests:

      % make devtest TEST_VERBOSE=1

    If there are any test failures, you'll need to fix those, too.

6   Commit the changes to the trunk. Be sure to specify what revisions were merged in, and the name of the new tag you'll create for the branch. The final revision number (represented by HEAD) happened to have been printed out when we ran "svn update" in step one, so it's easy to find and use:

      % svn commit -m 'Merged rev_1_10 changes 5995-6123. Will tag rev_1_10/trunk-6123.'

7   Tag the branch with the last revision, so that future merges can find it and know to merge only from that revision on.

      % svn cp -m 'Tag for merge to trunk revision 6123.' \
        https://svn.bricolage.cc/bricolage/branches/rev_1_10 \
        https://svn.bricolage.cc/bricolage/tags/merges/rev_1_10/trunk-6123

By following this methodology we should be able to minimize the number of conflicts we get between merges. See book "Version Control with Subversion," chapter 4 at <https://svnbook.red-bean.com/svnbook/ch04.html> for a more detailed explanation of the whys and wheres of this approach to merging Subversion branches.

CREATING DISTRIBUTIONS
If you are a Bricolage release manager and you're getting ready to release a new version, here are the steps you'll need to take to create a distribution tarball. First, proof-read the list of changes in Bric::Changes. Also, be sure to add today's date to the header for the new release:

  =head1 VERSION 1.x.x (2003-11-29)

If this is a major release, you'll need to create a new branch, so that it can be maintained for bug fixes separately, and then tag the release in that branch. In Subversion, branches and tags are exactly the same; the difference is only in convention. So the convention for Bricolage is to store tags in the tags subdirectory of the project directory, and branches in the branches subdirectory. The other half of the convention is that no one should ever commit changes to a tag after it has been created.

So to create a branch, make the copy from the trunk:

  % svn cp https://svn.bricolage.cc/bricolage/trunk \
    https://svn.bricolage.cc/bricolage/branches/rev_1_10 \
    -m 'New branch for maintenance of Bricolage 1.10.x.'

Now export the branch and create the distribution tarball.

  % svn export http://svn.bricolage.cc/bricolage/branches/rev_1_10
  % cd rev_1_10
  % make dist

This will create a distribution tarball in the rev_1_10 directory. Copy this tarball somewhere, and do a full test with it, to make sure that Bricolage does indeed build and properly install itself.

  % cp bricolage-1.x.x.tar.gz /tmp/src
  % cd /tmp/src
  % tar zxvf bricolage-1.x.x.tar.gz
  % cd bricolage-1.x.x
  % perl Makefile.PL
  % make
  # Shut down PostgreSQL.
  % make test
  # Start PostgreSQL.
  % sudo make install
  % make devtest
  % sudo bric_apachectl start
  # Log in to the UI and test it a bit.
  % sudo make uninstall

Be sure to run "make devtest" after everything is installed to ensure that it is all functioning correctly. Users won't be running these tests, as they muck up the database. But that's okay for our validation of the release tarball. Use "make uninstall" to clean up the mess. If you have to go back and make any changes, be sure to commit them to the Subversion repository and then do the whole thing over again. Once everything appears to be working properly, release!

Now that the new version of Bricolage is out free in the world, it's time to tag the release. Again, it's a simple copy:

  % svn cp -m 'Tag for the 1.10.0 release of Bricolage.' \
    http://svn.bricolage.cc/bricolage/branches/rev_1_10 \
    http://svn.bricolage.cc/bricolage/tags/releases/1.10.0

The tag is mainly kept for the purposes of record-keeping. And since copies are cheap in Subversion, it's worthwhile.

  Documentation Generation
Once the new version of Bricolage has been released, you'll need to regenerate the documentation on the Web site. There is documentation for each major version of Bricolage. The documentation for each version is generated from the latest minor release. So if you've just released, say, version 1.10.4, then the documentation should be generated for the "1.10" Documentation section. If you've just released 1.12.0, then you'll need to create a new "1.12" section and add a link to it on the documentation page of the Web site.

The documentation is generated with the pod_browser script, which is in the distutils/PodBrowser in Subversion. Read its instructions carefully, and then generate the documentation on the Web server.

  Where to Send Notices
Mail Lists

announce@lists.bricolage.cc
users@lists.bricolage.cc
devel@lists.bricolage.cc
mason-announce@lists.sourceforge.net
mason-users@lists.sourceforge.net
pgsql-announce@postgresql.org
pgsql-general@postgresql.org
announce@perl.apache.org
modperl@perl.apache.org
html-template-users@lists.sourceforge.net
templates-announce@template-toolkit.org
templates@template-toolkit.org
general@oscom.org

Web Sites

<http://www.bricolage.cc/>
<http://www.cmsmatrix.org/>
<http://freshmeat.net/projects/bricolage/>
<http://use.perl.org/>
<http://bugs.bricolage.cc/>

AUTHORS
Sam Tregar <stregar@about-inc.com>

David Wheeler <david@justatheory.com>

SEE ALSO
Bric::Admin, Bric::ToDo