%META:TOPICINFO{author="PeterThoeny" date="1093677752" format="1.0" version="1.9"}%
%TOC%
%STARTINCLUDE%
---# TWiki Site Tools

_Utilities for searching, navigation, and monitoring site activity_

TWikiSiteTools include utilities for navigating, searching and keeping up with site activity. Preferences can be configured by web or site-wide. You are currently in the *%TWIKIWEB%* web. In particular, TWiki provides two highly configurable, automated site monitoring tools, *<nop>WebNotify*, to email alerts when topics are edited, and *<nop>WebStats*, to generate detailed activity reports.

---++ <nop>WebNotify Recent Changes Alert

Each TWiki web has an automatic email alert service that sends a list of recent changes on a preset schedule, like once a day. Users can subscribe and unsubscribe using %NOTIFYTOPIC% in each web. The Perl script =mailnotify= is called by a background process at regular intervals. The script sends an automated email to subscribed users if topics were changed in a web since the script was last run.

	* You can create a WebNotify link using TWikiVariables with ==%<nop>NOTIFYTOPIC%==

%INCLUDE{"WebChangesAlert"}%

You can also use =%<nop>MAINWEB%= instead of =Main=, but this is not necessary even if you have renamed the main web by configuring =$mainWebname= in =TWiki.cfg=.  


---+++ Configuring Outgoing Mail

%WIKITOOLNAME% will use the ==Net::SMTP== module if it is installed on your system. Set this with the ==SMTPMAILHOST== variable in TWikiPreferences.

The notify e-mail uses the default =changes.tmpl= template, or a skin if activated in the TWikiPreferences.

mailnotify also relies on two hidden files in each =TWiki/data/[web]= directory: =.changes= and =.mailnotify.= Make sure both are writable by your web server process. =.changes= contains a list of changes; go ahead and make this empty. =.mailnotify= contains a timestamp of the last time notification was done.

You can use an external mail program, like ==sendmail==, if the =Net::SMTP= module is not installed. Set the program path in ==$mailProgram== in ==TWiki.cfg==.

	* %H% Net::SMTP can be easily disabled (ex: if there is an installation error) by setting ==SMTPMAILHOST== in TWikiPreferences to an empty value.
 
	* %T% You can set a separate ==SMTPSENDERHOST== variable to define the mail sender host (some SMTP installations require this). 

---+++ Setting the Automatic Email Schedule

*For Unix platforms:* Edit the =cron= table so that =mailnotify= is called in an interval of your choice. Please consult =man crontab= of how to modify the table that schedules program execution at certain intervals. Example:
<pre>
% crontab -e
0 2 * * * (cd /path/to/twiki/bin; ./mailnotify -q)
</pre>
The above line will run mailnotify nightly at 02:00. The =-q= switch suppresses all normal output.

*For ISP installations:* Many ISPs don't allow hosted accounts direct cron access, as it's often used for things that can heavily load the server. Workaround scripts are available.

*On Windows NT/2000:* You can use a scheduled task if you have administrative privileges.
__Note:__ AT on an NT machine is pretty limited.
Microsoft lists several third-party
[[http://www.microsoft.com/ntserver/partners/findoffering/serversolutions/Maintnce.asp#cron][replacements]]. TWiki:Codev/CronTabWin is a free scheduler for Windows.

---++ <nop>WebStatistics Site Usage Log

You can generate a listing manually, or on an automated schedule, of visits to individual pages, on a per web basis. Compiled as a running total on a monthly basis. Includes totals for Topic Views, Topic Saves, Attachment Uploads, Most Popular Topics with number of views, and Top Contributors showing total of saves and attachment uploads. Previous months are saved.

	* You can create a WebStatistics link using TWikiVariables with ==%<nop>STATISTICSTOPIC%==

---+++ Configuring for Automatic Operation

	* You can automatically generate usage statistics for all webs. To enable this:
		* Make sure variable =$doLogTopicView, $doLogTopicSave and $doLogTopicUpload= in =TWiki.cfg= are set. This will generate log file entries (see below).
		* The WebStatistics topic must be present in all webs where you want to have statistics. You can use the topic in the Main web as a template.
		* Call the =twiki/bin/statistics= script from a cron job, once a day is recommended. This will update the WebStatistics topics in all webs.
		* __Attention:__ The script must run as the same user as the CGI scripts are running, which is user =nobody= on many systems. Example crontab entry: <br> =0 0 * * * (cd /path/to/TWiki/bin; ./statistics &gt;/dev/null 2&gt;&amp;1)=
		* There is a workaround in case you can't run the script as user =nobody= : Run the utility =twiki/bin/geturl= in your cron job and specify the URL of the =twiki/bin/statistics= script as a parameter. Example: <br> =0 0 * * * (cd /path/to/TWiki/bin; ./geturl mydomain.com /urlpath/to/TWiki/bin/statistics &gt;/dev/null 2&gt;&amp;1)=

---+++ Generating Statistics Manually by URL

	* The =twiki/bin/statistics= script can also be executed as a CGI script, just enter the URL in your browser. Examples:
		* Update current month for all webs: <br> =http://mydomain.com/twiki/bin/statistics=
		* Update current month for Main web only: <br> =http://mydomain.com/twiki/bin/statistics/Main=
		* Update January 2000 for Main web: <br> =http://mydomain.com/twiki/bin/statistics/Main?logdate=200001=

---+++ Log File Details

TWiki generates monthly log files which are used by the statistics
	* The log directory is defined by the =$logDir= variable in TWiki.cfg
	* The file name is =log&lt;year&gt;&lt;month&gt;.txt=
	* Example path name: =twiki/data/log%SERVERTIME{$year$mo}%.txt=
	* Each access gets logged as: %BR%
	  =| &lt;time&gt; | &lt;wikiusername&gt; | &lt;action&gt; | &lt;web&gt;.&lt;topic&gt; | &lt;extra info&gt; | &lt;IP address&gt; |=
	* Example log entry: %BR%
	  =| %SERVERTIME{$day $mon $year - $hour:$min}% | !Main.TWikiGuest | view | !TWiki.WebRss |  | 66.124.232.02 |=
	* Actions are logged if enabled in TWiki.cfg by the =$doLogTopic&lt;action&gt;= flags
	* Logged actions:
	  | *Script* | *Action name* | *Extra info* |
	  | attach | =attach= | when viewing attach screen of previous uploaded attachment: =filename= |
	  | changes | =changes= |  |
	  | edit | =edit= | when editing non-existing topic: =(not exist)= |
	  | register | =register= | E-mail address |
	  | save | =save= | when replacing existing revision: =repRev 1.3= %BR% when user checks the minor changes box: =dontNotify= |
	  | search | =search= | search string |
	  | rdiff | =rdiff= | =1.4 1.3= |
	  | rename | =rename= | when moving topic: =moved to !Newweb.NewTopic= |
	  | rename | =move= | when moving attachment: =Attachment filename moved to !Newweb.NewTopic= |
	  | upload | =upload= | filename |
	  | view | =view= | when viewing non-existing topic: =(not exist)= %BR% when viewing previous topic revision: =r1.3= |


---++ <nop>WebSearch

WebSearch is an extremely fast and flexible search facility, part of the core TWiki feature set. Options include:
	* topic title or full-text search
	* regular expressions
	* search within web or site-wide
	* index-style A-Z alphabetical listing sorted topic title
	* many more

See also: TWikiVariables for including hard-coded searches in text.

---+++ <nop>WebChanges

To check for the most recently edited topics while on-site, use the WebChanges link, usually located on the upper toolbar. It lists the most recently modified topics, newest first, along with the first couple of lines of the page content. 

This is simply a preset =SEARCH=. The number of topics listed by the =limit= parameter.:
<blockquote style="background-color:#f5f5f5">
<pre>
%<nop>SEARCH{".*" web="%INCLUDINGWEB%" regex="on" nosearch="on" order="modified" <br />reverse="on" limit="50"}%
</pre>
</blockquote>

---+++ <nop>WebIndex

WebIndex lists all web topics in alphabetical order, with the first couple of lines of text. This is simply a preset =SEARCH=:
<blockquote style="background-color:#f5f5f5">
<pre>
%<nop>SEARCH{"\.*" scope="topic" regex="on" nosearch="on"}%
</pre>
</blockquote>

-- TWiki:Main.MikeMannix - 01 Dec 2001 %BR%
-- TWiki:Main.PeterThoeny - 28 Aug 2004

