Apache Module Info
Programming information on the Apache API, from the Apache User's Guide.
General Theory
Overview from C
The primary module, mod_neoscript.c
, creates a Tcl (master)
interpreter and loads in various Tcl packages like Extended Tcl, NeoTcl,
interfaces to Postgres and GD if available. The master interpreter then
sources in init.tcl
, which performs the complex tasks of creating
and setting up the safe (slave) interpreters that make the
NeoWebScript environment.
Another module, mod_log_neo.c
creates a special TransferLog
containing more information than the normal CLF files, and formatted in a
Tcl list-based structure. The command estimate_hits_per_hour relies
on this log format to work.
The module mod_auth_tcl.c
implements user authentication through
the Tcl interpreter.
The module mod_neo_userdir.c
is a starting attempt to implement
user directory access policy through the Tcl interpreter.
Overview from Tcl
init.tcl
initializes the master interpreter; loads in additional
commands from other source files; creates safe slave interpreters as needed;
and establishes various interfaces between the master and safe slaves, and
between the module and the interpreters. Source files are also loaded into the
slave as needed. Autoloading is even supported in the
neoscript-tcl
directory, but be sure to create the proper index
file.
Processing a Webpage
When a webpage is requested, if it matches the criteria for a server side include and a neoscript directive is found in the page, the C handle_neowebscript routine is called from the C code in the module that sends parsed content.
We make use the request_config data in the request record structure. This allows each module to set a pointer and find it later. We follow the request_rec structure from ours back to the main one (if we aren't the main one) to find a non-NULL pointer for our module in the request_config structure. If there isn't one to be found, it is the first invocation of NeoWebScript for this page.
If it's the first use of NeoWebScript for this page, or if this page is being included in another page but is not owned by the same owner as the other page, we create a safe interpreter, set it into the request_config structure for our module, register for the cleanup routine to destroy the interpreter, and configure and propagate the variables to the safe interpreter.
Tcl_request_rec is a global pointer to the current request_rec being processed. We save the previous value on the stack while we're executing send_parsed_content, and put it back afterwards. That way Tcl procedures don't have to all pass around a magic cookie to find the request rec, yet multiple interpreters can exist and code in existant interpreters can be invoked for subordinate includes when the criteria for doing that are met.
If it's the first use of NeoWebScript for this page, then if a
safe interpreter previously existed, it is destroyed. A new safe interpreter
is created, and various environment variables, server variables, etc, are
exported to it by propagate_vars_to_nws
.
handle_neowebscript builds up the command to be executed. It sets up to call handle_neowebscript_request (which is written in Tcl). The first argument is the name of the safe interpreter. The second is the key of the key-value pair read from the server-side-include. Last comes the value of the key-value pair.
It is most important that neither the key nor the value are ever evaluated within the trusted interpreter, or the user will have defeated the protection mechanisms provided by the safe interpreter.
Since we build up the command to be executed as a list, the key and value will be quoted to make a valid list element, so they won't be evaluated by the trusted interpreter when handle_neowebscript_request is called.
We then evaluate the constructed command. If there is an error, we send the error result to the webpage being constructed.
Finally we restore the Tcl_request_rec global request rec pointer with whatever was in there before. It is intialized to NULL at the start, and should automatically be restored to that when the top level page of NeoWebScript code completes.
handle_neowebscript_request
This command is responsible for actually processing embedded NeoWebScript requests from within webpages. It is defined in init.tcl
.
It switches on the tag, which can currently be return, code, eval, var or expr. Anything else is treated as an error.
Valid tags are handled using handle_server_side_return
,
handle_server_side_eval
, handle_server_side_variable
,
and handle_server_side_expr
. These routines are called with the
name of the safe interpreter and the code (or variable, or expression) to
evaluate. The code is evaluated within the safe interpreter. Errors are
caught and traced back.
These routines were defined as procs by init.tcl
, and they
use the interp eval
command to do the right thing to the untrusted
interpreter to cause code to be executed, results to be returned,
etc.
devel.tcl
devel.tcl is used by people who are modifying NeoWebScript by changing code that runs in the trusted code base. This should only be done by people who really know what they're doing, because errors here can compromise the security of your webserver.
If debugging is set to 1 in init.tcl
, then every
time a page begins NeoWebScript execution, devel.tcl
is sourced in, and the devel_setup
procedure is executed. This
makes it easy to work on new functions, without having to restart the webserver
on every change.
If debugging is set to 0 in init.tcl
, then
devel.tcl
is loaded it at server startup time only, although
devel_setup
is still run each time an interpreter is set up to do
something to a page.
Unknown Packages and Procs in Safe Interpreters
As of version 2.0, there are better ways to support package require and unknown procedures within the safe interpreter. Previously, only unknown procedures could be resolved, and they were resolved by providing access to the source command (to a restricted list of directories) among others. In this way, an essentially standard unknown and auto_loading procedure could take place within the safe interpreter. There has been no previous support for package require.
Unknown Procedures
Previous versions of NeoWebScript borrowed from Michael McClennan's Itcl2.1 implementation of unknown, the mechanism for resolving calls to unknown procs in the safe interpreter. However, along with the integration of Tcl 8.0.3, it now uses the new Safe Base mechanisms specifically designed for creating and manipulating safe interpreters. It automatically manages the slaves' auto_path and tcl_library; and contains predefined aliases for source, load, file, and exit commands allowing for their restricted use.
However, NeoWebScript performs some internal caching of certain procs into the master interpreter. The Tcl proc SAFE_autoload is given first crack to resolve a command for the safe interpreter. It checks two global arrays, the safe_proc_cache and the safe_alias_cache. These arrays contain the proc or alias definition to be evaled into the safe interpreter. If it does not find a valid definition, it then passes autoloading to the Safe Base in the safe interpreter.
Resolving Package Requests
To package require in the safe interpreter, during the setup of the slave, a package ifneeded command can be evaled into the slave giving a specific script to execute when the package is requested.