<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5CR3//EN"
	"http://www.oasis-open.org/docbook/xml/4.5CR3/docbookx.dtd" [
<!ENTITY % userents SYSTEM "file:///ENTS/user.ent" >
%userents;
]>

<!--
(C) Copyright 2011-2012 Sergey A. Babkin.
This file is a part of Triceps.
See the file COPYRIGHT for the copyright notice and license information
-->

<chapter id="ch_scheduling" xmlns:xi="http://www.w3.org/2001/XInclude">
	<title>Scheduling</title>

	<sect1 id="sc_sched_overview">
		<title>Overview of the scheduling</title>

		<indexterm>
			<primary>scheduling</primary>
		</indexterm>
		<indexterm>
			<primary>model</primary>
		</indexterm>
		<para>
		The scheduling determines, in which order the row operations are
		processed. If there are multiple operations available, which one
		should be processed first?  The scheduler keeps a queue of the operations
		and selects, which one to execute next.  This has a major effect on the
		logic of a CEP model.  
		</para>

		<indexterm>
			<primary>Aleri</primary>
		</indexterm>
		<indexterm>
			<primary>Coral8</primary>
		</indexterm>
		<indexterm>
			<primary>Sybase</primary>
		</indexterm>
		<indexterm>
			<primary>StreamBase</primary>
		</indexterm>
		<para>
		There are multiple approaches to scheduling. Aleri essentially doesn't
		have any, except for the flow control between threads, because each its
		element is a separate thread. Coral8 has an intricate scheduling
		algorithm. Sybase R5 has the same logic as Coral8 inside each thread.
		StreamBase presumably also has some.
		</para>

		<para>
		The scheduling logic in Triceps is different from the other CEP
		systems. The Coral8 logic looks at first like the only reasonable way
		to go, but could not be used in Triceps for three reasons: First, it's a trade
		secret, so it can't be simply reused. If I'd never seen it, that would
		not be an issue but I've worked on it and implemented its version for
		R5. Second, it relies on the properties that the compiler computes from
		the model graph analysis. Triceps has no compiler, and could not do
		this. Third, in reality it simply doesn't work that well. There are
		quite a few cases when the Coral8 scheduler comes up with a strange and
		troublesome execution order.
		</para>

		<para>
		For a while I've hoped that Triceps would need no scheduler at all, and
		everything would be handled by the procedural calls. This has proved to
		have its own limitations, and thus the labels and their scheduling were
		born. The Triceps scheduling still has issues to resolve, but overall
		it still feels much better than the Coral8 one.
		</para>
	</sect1>

	<sect1 id="sc_sched_no_bundling">
		<title>No bundling</title>

		<indexterm>
			<primary>bundling</primary>
		</indexterm>
		<para>
		The most important principle of Triceps scheduling is: No Bundling.
		Every rowop is for itself. The bundling is what messes up the Coral8
		scheduler the most. 
		</para>

		<para>
		What is a bundle? It's a set of records that go through the execution
		together. If you have a model consisting of two functional elements F1
		and F2 connected in a sequential fashion 
		</para>

<pre>
F1->F2 
</pre>

		<para>
		and a few loose records R1, R2, R3, the
		normal execution order without bundling will be:
		</para>

<pre>
F1(R1), F2(R1), F1(R2), F2(R2), F1(R3), F2(R3)
</pre>

		<para>
		Each row goes through the whole model (a real simple one in this case)
		before the next one is touched. This allows F2 to take into
		accont the state of F1 exactly as it was right after processing
		the same record, without any interventions in between.
		</para>

		<para>
		If the same records are placed in a bundle (R1, R2, R3), the execution
		order will be different:
		</para>

<pre>
F1(R1), F1(R2), F1(R3), F2(R1), F2(R2), F2(R3)
</pre>

		<para>
		The whole bundle goes through F1 before the rows go to F2.
		</para>

		<para>
		That would not be a problem, and even could be occasionally useful, if
		the bundles were always created explicitly. In the reality of Coral8,
		every time a statement produces multiple record from a single one
		(think of a join that picks multiple records from another side), it
		creates a bundle and messes up all the logic after it. Some logic gets
		affected so badly that a few statements in CCL (like ON UPDATE) had to
		be designated as always ignoring the bundles, otherwise they would not
		work at all. At DB I wrote a CCL pattern for breaking up the bundles.
		It's rather heavyweight and thus could not be used all over the place
		but provides a generic solution for the most unpleasant cases.
		</para>

		<para>
		Worse yet, the bundles may get created in Coral8 absolutely
		accidentally: if two records happen to have the same timestamp, for all
		practical purposes they would act as a bundle. In the models that were
		designed without the appropriate guards, this leads to the time-based
		bugs that are hard to catch and debug. Writing these guards correctly
		is hard, and testing them is even harder. 
		</para>

		<para>
		Another issue with bundles is that they make the large queries slower.
		Suppose you do a query from a window that returns a million
		records. All of them will be collected in a bundle, then the
		bundle will be sent to the interface gateway that would build one huge
		protocol packet, which will then be sent to the client, which will
		receive the whole packet and then finally iterate on the records in it.
		Assuming that nothing runs out of memory along the way, it will be a
		long time until the client sees the first record. Very, very
		annoying.
		</para>

		<para>
		Aleri also has its own version of bundles, called transactions, but a
		more smart one. Aleri always relies on the primary keys. The condition
		for a transaction is that it must never contain multiple modification
		for the same primary key. Since there are no execution order guarantees
		between the functional elements, in this respect the transactions work
		in the same way as loose records, only with a more efficient
		communication between threads. Still, if the primary key changes in an
		element (say, an aggregator), the condition does not propagate through
		it. Such elements have to internally collapse the outgoing transactions
		along the new key, adding overhead.
		</para>
	</sect1>

	<sect1 id="sc_sched_basic">
		<title>Basic scheduling in Triceps</title>

		<para>
		In Triceps the scheduling is done by the execution unit, or simply
		<quote>unit</quote> as it's often referred to.
		It provides 3 basic ways of executing of a rowop:
		</para>

		<indexterm>
			<primary>unit</primary>
		</indexterm>
		<indexterm>
			<primary>row operation</primary>
		</indexterm>
		<variablelist>
			<varlistentry>
				<term>Call:</term>
				<listitem>
				<indexterm>
					<primary>call</primary>
				</indexterm>
				<para>
				Execute the label right now, including all the nested calls.
				All of this will be completed after the call returns.
				</para>
				</listitem>
			</varlistentry>
			<varlistentry>
				<term>Fork:</term>
				<listitem>
				<indexterm>
					<primary>fork</primary>
				</indexterm>
				<para>
				Execute the label after the current label returns but
				before anything else is done. Obviously, if multiple labels are
				forked, they will execute in order after the current label
				returns (but before its caller gets the control back).
				This method has looked promising at one point but has currently
				fallen out of favor and will likely be removed in the future.
				</para>
				</listitem>
			</varlistentry>
			<varlistentry>
				<term>Schedule:</term>
				<listitem>
				<indexterm>
					<primary>schedule</primary>
				</indexterm>
				<para>
				Execute the label after everything else is done.
				</para>
				</listitem>
			</varlistentry>
		</variablelist>

		<para>
		This is kind of intuitively clear but the details might sometimes
		be a bit surprising. So let us look in detail at how it works inside
		on an example of a fairly convoluted scheduling sequence.
		</para>

		<indexterm>
			<primary>queue</primary>
		</indexterm>
		<indexterm>
			<primary>frame</primary>
		</indexterm>
		<indexterm>
			<primary>stack</primary>
		</indexterm>
		<indexterm>
			<primary>draining</primary>
		</indexterm>
		<para>
		A scheduler in the execution unit keeps a stack of queues. Each queue
		is essentially a stack frame, so I'll be using the terms <pre>queue</pre> and
		<pre>frame</pre> interchangeably. The stack always contains at least one
		queue, which is called the outermost stack frame.
		</para>

		<para>
		When the new rowops come from the outside world, they are added with
		<pre>schedule()</pre> to that stack frame. That's what <pre>schedule()</pre> does: always
		adds rowops to the outermost stack frame. If rowops 1, 2 and 3 are
		added, the stack looks like this (the brackets denote a stack frame):
		</para>

<pre>
[1, 2, 3]
</pre>

		<para>
		The unit method <pre>drainFrame()</pre> is then used to run the
		scheduler and process the rowops. It makes the unit call each rowop on
		the innermost frame (which is initially the same as outermost
		frame, since there is only one frame) in order.
		</para>

		<para>
		First it calls the rowop 1. It's removed from the queue, then a new
		frame is pushed onto the stack:
		</para>

<pre>
[ ] ~1
[2, 3]
</pre>

		<para>
		This new frame is the rowop 1's frame, which is marked on the diagram
		by <quote>~1</quote>. The diagram shows the most recently pushed, innermost,
		frame on the top, and the oldest, outermost frame on the bottom. The
		concepts of <quote>innermost</quote> and <quote>outermost</quote>
		come from the nested calls: the most recent call is nested the deepest
		in the middle and is the innermost one.
		</para>

		<para>
		Then the rowop 1 executes. If it
		calls rowop 4, another frame is pushed onto the stack for it:
		</para>

<pre>
[ ] ~4
[ ] ~1
[2, 3]
</pre>

		<para>
		Then the rowop 4 executes. The rowop 4 never gets onto any of the queues.
		The call just pushes a new frame and executes the rowop right away.
		The identity of rowop being processed is kept in the call context. A
		call also involves a direct &Cpp; call on the thread stack, and if any
		Perl code is involved, a Perl call too. Because of this, if you nest
		the calls too deeply, you may run out of the thread stack space and
		get it to crash.
		</para>

		<para>
		After the rowop 4 is finished (not calling any other
		rowops), the innermost empty frame is popped before the execution of
		rowop 1 continues. The queue stack reverts to the previous state.
		</para>

<pre>
[ ] ~1
[2, 3]
</pre>

		<para>
		Suppose then rowop 1 forks rowops 5 and 6. They are appended to the
		innermost frame in the order they are forked.
		</para>

<pre>
[5, 6] ~1
[2, 3]
</pre>

		<para>
		If rowop 1 then calls rowop 7, again a frame is pushed onto the stack
		before it executes:
		</para>

<pre>
[ ] ~7
[5, 6] ~1
[2, 3]
</pre>

		<para>
		The rowops 5 and 6 still don't execute, they keep sitting on the queue
		until the rowop 1 would return.
		After the call of rowop 7 completes, the scheduler stack returns to
		the previous state.
		</para>

		<para>
		Suppose now the execution of rowop 1 completes. But its stack frame can
		not be popped yet, because it is not empty. The scheduler calls
		<pre>drainFrame()</pre> recursively, which picks the next rowop from the innermost
		queue (rowop 5), and calls it, pushing a new stack frame and executing
		the rowop 5 code:
		</para>

<pre>
[ ] ~5
[6] ~1*
[2, 3]
</pre>

		<para>
		The former rowop 1's frame is now marked with <quote>~1*</quote>
		for the ease of tracking, even though it has completed.
		</para>

		<para>
		If rowop 5 forks rowop 8, the stack becomes:
		</para>

<pre>
[8] ~5
[6] ~1*
[2, 3]
</pre>

		<para>
		When the execution of rowop 5 returns, its queue is also not empty. So
		the scheduler starts draining the innermost frame again, and calls rowop 8.
		During its execution the stack is:
		</para>

<pre>
[ ] ~8
[ ] ~5*
[6] ~1*
[2, 3]
</pre>

		<para>
		Suppose the rowop 8 doesn't call or fork anything else and returns. Its
		innermost queue is empty, so the call completes and pops the stack
		frame:
		</para>

<pre>
[ ] ~5*
[6] ~1*
[2, 3]
</pre>

		<para>
		Now the queue of rowop 5 is also empty, so its draining completes and
		pops the drained frame:
		</para>

<pre>
[6] ~1*
[2, 3]
</pre>

		<para>
		The draining of the rowop 1's frame continues by picking the rowop 6
		from the queue and calling it:
		</para>

<pre>
[ ] ~6
[ ] ~1*
[2, 3]
</pre>

		<para>
		Suppose rowop 6 calls <pre>schedule()</pre> of rowop 9. Rowop 9 is then
		added to the outermost queue:
		</para>

<pre>
[ ] ~6
[ ] ~1*
[2, 3, 9]
</pre>

		<para>
		Rowop 6 then returns, its queue is empty, so it's popped and its call completes.
		</para>

<pre>
[ ] ~1*
[2, 3, 9]
</pre>

		<para>
		Now the queue of rowop 1 has become empty, so it's popped from the
		stack and the call of rowop 1 completes:
		</para>

<pre>
[2, 3, 9]
</pre>

		<para>
		The unit method <pre>drainFrame()</pre> keeps running on the outermost
		frame, now taking the rowop 2 and executing it, and so on, until the
		outermost queue becomes empty, and <pre>drainFrame()</pre> returns.
		</para>

		<indexterm>
			<primary>label</primary>
			<secondary>chaining</secondary>
		</indexterm>
		<para>
		An interesting question is, what happens with the chained labels?
		Where do they fit in the order of execution? It turns out to be
		a bit of a mix between a <pre>call()</pre> and a <pre>fork()</pre>.
		They get checked after the original label completes its execution 
		and has its frame drained but before that frame gets popped.
		</para>

		<para>
		If any chained labels are found, they are called one by one.
		But they don't get a new frame created. They all reuse the frame
		left over from the parent label. The frame gets popped only
		after all the chained labels have completed.
		</para>

	</sect1>

	<sect1 id="sc_sched_loop">
		<title>Loop scheduling</title>

		<indexterm>
			<primary>scheduling</primary>
			<secondary>loop</secondary>
		</indexterm>
		<para>
		The easiest and most efficient way to schedule the loops is to do it
		procedurally, something like this:
		</para>

<pre>
foreach my $row (@rowset) {
	$unit->call($lbA->makeRowop(&Triceps::OP_INSERT, $row)); 
}
</pre>

		<indexterm>
			<primary>topological loop</primary>
		</indexterm>
		<para>
		However the labels topologically connected into a loop can come handy
		as well. Some logic may be easier to express this way. Suppose the
		model contains the labels connected in a loop, as in
		<xref linkend="fig_sched_loop" xrefstyle="select: label nopage"/>&xrsp;.
		</para>

		<figure id="fig_sched_loop" >
			<title>Labels forming a loop.</title>
			<xi:include href="file:///FIGS/label-010-loop.xml"/> 
		</figure>

		<para>
		But if handled simple-mindedly, it can use a lot of stack space.
		Suppose some rowop X1 is scheduled for label X, and causes the loop
		to be executed twice, with rowops X1, A2, B3, C4, A5, B6, C7, Y8. If each
		operation is done as a <pre>call()</pre>, the stack grows like this: It starts with
		X1 scheduled.
		</para>

<pre>
[X1]
</pre>

		<para>
		Which then gets executed, with its own execution frame (marked as such
		for clarity):
		</para>

<pre>
[ ] ~X1
[ ]
</pre>

		<para>
		Which then calls A2:
		</para>

<pre>
[ ] ~A2
[ ] ~X1
[ ]
</pre>

		<para>
		By the time the execution comes to Y8, the stack looks like this:
		</para>

<pre>
[ ] ~Y8
[ ] ~C7
[ ] ~B6
[ ] ~A5
[ ] ~C4
[ ] ~B3
[ ] ~A2
[ ] ~X1
[ ]
</pre>

		<para>
		The loop has been converted into recursion, and the whole length of
		execution is the depth of the recursion. If the loop executes a million
		times, the stack will be three million levels deep. Worse yet, it's not
		just the Triceps scheduler stack that grows, it's also the process
		(&Cpp;) stack.
		</para>

		<indexterm>
			<primary>scheduling</primary>
			<secondary>recursion</secondary>
		</indexterm>
		<indexterm>
			<primary>recursion</primary>
		</indexterm>

		<para>
		Which is why this kind of recursive calls are explicitly forbidden
		in Triceps. If you try to do it, on the first recursive call the
		execution will die with an error.
		</para>

		<para>
		Would things be better with <pre>fork()</pre> instead of
		<pre>call()</pre> used throughout the loop? It starts the same way:
		</para>

<pre>
[X1]
</pre>

		<para>
		Then X1 executes, gets its own frame and forks A2:
		</para>

<pre>
[A2] ~X1
[ ]
</pre>

		<para>
		Then A2 executes, gets its own frame and forks B3:
		</para>

<pre>
[B3] ~A2
[ ] ~X1*
[ ]
</pre>

		<para>
		Even though X1 has completed, its stack frame stays until all the rowops
		forked in it complete too.
		By the end of the loop the stack picture becomes exactly the same as with
		<pre>call()</pre>. For a while I've thought that optimizing out the empty stack
		frames would solve the problem, but no, that doesn't work: the problem
		is that the &Cpp; process stack keeps growing no matter what. The jump
		back in the loop needs to be placed into an earlier stack frame to
		prevent the stack from growing.
		</para>

		<para>
		One way to do it would be to use the <pre>schedule()</pre> operation in
		C to jump back to A, placing the rowop A5 back onto the outermost
		frame. The scheduler stack at the end of C4 would look like:
		</para>

<pre>
[ ] ~C4
[ ] ~B3
[ ] ~A2
[ ] ~X1
[A5]
</pre>

		<para>
		Then the stack would unwind back to:
		</para>

<pre>
[A5]
</pre>

		<para>
		And the next iteration of the loop will start afresh. The problem here
		is that if X1 wanted to complete the loop and then do something, it
		can't. By the time the second iteration of the loop starts, X1 is
		completely gone. It would be better to be able to enqueue the next
		execution of the loop at the specific point of the stack.
		</para>

		<indexterm>
			<primary>frame mark</primary>
		</indexterm>
		<para>
		Here the concept of the frame mark comes in. A frame mark is a token
		object, completely opaque to the program. It can be used only in two
		operations:
		</para>

		<itemizedlist>
		<listitem>
		<pre>setMark()</pre> remembers the  position in the frame stack, just
		outside the current frame.
		</listitem>
		<listitem>
		<pre>loopAt()</pre> enqueues a rowop at the marked frame.
		</listitem>
		</itemizedlist>

		<para>
		Then the loop wold have its mark object M. The label A will execute
		<pre>setMark(M)</pre>, and the label C will execute <pre>loopAt(M, rowop(A))</pre>. The rest
		of the execution can as well use <pre>call()</pre>, as shown in
		<xref linkend="fig_sched_loop" xrefstyle="select: label nopage"/>&xrsp;.
		</para>

		<figure id="fig_sched_mark" >
			<title>Proper calls in a loop.</title>
			<xi:include href="file:///FIGS/label-011-mark.xml"/> 
		</figure>

		<para>
		When A2 calls setMark(M), the stack will look like this:
		</para>

<pre>
[ ] ~A2
[ ] ~X1, mark M
[ ]
</pre>

		<para>
		The mark M remembers the frame one outer to the current one. The stack
		at the end of C4, after it has called <pre>loopAt(M, A5)</pre>, is:
		</para>

<pre>
[ ] ~C4
[ ] ~B3
[ ] ~A2
[A5] ~X1, mark M
[ ]
</pre>

		<para>
		The stack then unwinds until A5 starts its execution:
		</para>

<pre>
[ ] ~A5
[ ] ~X1*, mark M
[ ]
</pre>

		<para>
		Each iteration starts with a fresh stack, and the stack depth is
		limited to one iteration. The nested loops can also be properly
		executed.
		</para>

		<para>
		Now, why does the mark get placed on the frame that is one out from the
		current one? After all, this means that X1 can not wait for the
		loop to complete. It has to return before the second iteration of
		the loop can start. And then the rest of the loop will run before
		the control returns to X1's caller. At least the caller of X1 can
		wait for the loop to complete before continuing its execution.
		Why all this trouble? Its the result of a compromise.
		Suppose that it did remember the current frame. Then at
		the end of C4 the stack will be:
		</para>

<pre>
[ ] ~C4
[ ] ~B3
[A5] ~A2, mark M
[ ] ~X1
[ ]
</pre>

		<para>
		The stack will unwind until A5. Which would then have its own frame
		pushed onto the stack, and the code in the label A will call <pre>setMark(M)</pre> 
		again, moving the mark to A5's own frame because it's the topmost frame now:
		</para>

<pre>
[ ] ~A5, mark M
[ ] ~A2*
[ ] ~X1
[ ]
</pre>

		<para>
		So on each iteration of the loop one extra frame will be pushed onto
		the stack, and the mark moved by one level. A loop executing a million
		times will push a million frames, which is bad. Marking the next outer
		frame prevents this. Another option would have been to put the
		mark operation in X, but that would mean that every loop must have a preceding
		label that just marks the frame (well, and potentially could do the
		other initializations too), which seems to be too annoying.
		</para>

		<para>
		It's one problem or the other, and the lesser problem won.
		To further reduce the complexity, I've also added the methods 
		<pre>makeLoopHead()</pre> and <pre>makeLoopAround()</pre> that take care of constructing
		the whole front part of the loop, including the setting of the mark.
		They will be described below in
		<xref linkend="sc_sched_unit" xrefstyle="select: label quotedtitle pageabbrev"/>&xrsp;.
		This is still messy, and I'm still thinking about the ways to improve
		the situation.
		</para>

		<para>
		What happens after the stack unwinds past the mark? The mark gets
		unset. When someone calls <pre>loopAt()</pre> with an unset mark, the rowop is
		enqueued in the outermost frame, having the same effect as schedule<pre>()</pre>.
		</para>

		<para>
		This handling of an unset mark comes handy in case if the loop execution
		takes a pause in the middle. Suppose the label B finds that it can't process
		the rowop B3 until some other data has arrived. What it can do then is remember
		B3 somewhere in the thread state and return. The loop has not completed but
		it can't progress either, so the call unrolls until it becomes empty.
		Since the frame of X1 is popped off the stack, the mark M gets unset. 
		The knowledge that the loop needs to be continued stays remembered
		in the state.
		</para>

		<para>
		After some time that awaited data arrives, as some other rowop. When that
		rowop gets processed, it finds that remembered state with B3 and makes
		it continue, maybe by calling <pre>call(B3)</pre> again. So now the
		logic in B finds all the data it needs and continues with the loop,
		calling C4. C4 will do its job and call <pre>loopAt(M, A5)</pre>.
		But the mark M has been unset a while ago!  Scheduling A5 at the outermost
		frame seems to be a logical thing to do at this point. Then whatever
		current processing will complete and unwind, and the loop will continue
		after it.
		</para>

		<para>
		What if <pre>setMark()</pre> is called when there is only one frame on
		the stack? Then there is no second frame outer to it. The mark will
		simply be left unset.
		</para>

		<para>
		But overall pausing and then restarting a loop like this is not such
		a good idea. The caller of the loop normally expects that it can wait
		for the loop to complete, and that when the loop returns, it's all done.
		If a loop may decide to bail out now and continue later, the caller
		has to be prepared for it.
		</para>

		<indexterm>
			<primary>scheduling</primary>
			<secondary>loop interleaving</secondary>
		</indexterm>
		<indexterm>
			<primary>batch</primary>
		</indexterm>
		<indexterm>
			<primary>bundling</primary>
		</indexterm>
		<indexterm>
			<primary>fork</primary>
		</indexterm>
		<indexterm>
			<primary>schedule</primary>
		</indexterm>
		<para>
		And the sequence of execution when the loop continues might not
		be direct. In the normal execution one iteration of the loop
		follows directly after the previous one because of the orchestration
		by the label at the head of the loop (X in this example). 
		When C calls <pre>loopAt()</pre>, the rowop gets pushed onto
		the stack frame of X. It would execute immediately only if X
		is draining its frame, and only if there are no other rowops
		queued on that frame in front of this one. I've been seeing it
		as a feature: X can easily be careful and make sure that the
		whole loop executes in one go without any interruptions.
		<pre>makeLoopHead()</pre> and <pre>makeLoopAround()</pre>
		create such careful labels. But it may also decide to
		run multiple loops interleaved, with each one making one
		iteration at a time. To do that, all it needs is fork the
		rowops to start all these loops and then drain the frame
		(directly or by returning from its own code). This way
		you can for example make a batch of records run through the
		loop (or even through the different loops) with the whole
		batch going throgh one iteration before the another iteration
		starts, achieving a kind of bundling. Though, if fork
		gets removed, this would not be possible any more. Maybe
		is's a reason for fork to stay, maybe it's a useless feature.
		</para>

		<para>
		However if a loop decides to pause and then continues on
		some other event, its following <pre>loopAt()</pre> pushing the rowop
		onto the outermost frame, there is no caring parent to do the
		careful orchestration. There is no way to tell, which other
		rowops have been pushed onto the outermust frame by this time.
		The loop won't continue until these rowops execute. They may
		change the state of the model, so if the loop code expects it
		to stay the same, it will be mightily surprised.
		</para>
	</sect1>

	<sect1 id="sc_sched_unit">
		<title>Execution unit</title>

		<para>
		After discussing the principles of scheduling in Triceps, let's get
		down to the nuts and bolts.
		</para>

		<indexterm>
			<primary>unit</primary>
		</indexterm>
		<para>
		A unit is created as:
		</para>

<pre>
$myUnit = Triceps::Unit->new("name") or confess "$!";
</pre>

		<para>
		The name argument is as usual used for later debugging, and by
		convention should be the same as the name of the unit variable
		(<quote>myUnit</quote> in this case). The name can also be changed later:
		</para>

<pre>
$myUnit->setName("newName");
</pre>

		<para>
		It returns no value. Though in practice there is no good reason for changing names,
		and this call will likely be removed in the future. The name can be
		read back:
		</para>

<pre>
$name = $myUnit->getName();
</pre>

		<para>
		Also, as usual, the variable $myName here contains a reference to the
		actual unit object, and two references can be compared for whether they
		refer to the same object:
		</para>

<pre>
$result = $unit1->same($unit2);
</pre>

		<para>
		A unit also keeps an empty row type (one with no fields), primarily for the
		creation of the clearing labels, but you can use it for any other purposes
		too. You can get it with the method:
		</para>

<pre>
$rt = $unit->getEmptyRowType();
</pre>

		<para>
		Each unit has its own instance of an empty row type. Its purely for
		the conveniece of memory management, they are all equivalent.
		</para>

		<para>
		The rowops are enqueued with the calls:
		</para>

<pre>
$unit->call($rowop, ...);
$unit->fork($rowop, ...);
$unit->schedule($rowop, ...); 
</pre>

		<indexterm>
			<primary>enqueue</primary>
		</indexterm>
		<para>
		<quote>Enqueued</quote> is an ugly word but since I've already used the
		word <quote>schedule</quote> for a specific purpose, I needed another word to name
		all these operations together. Hence <quote>enqueue</quote>.
		</para>

		<para>
		The <quote>...</quote> shows that multiple rowops may be passed as arguments.
		So the real signature of these methods is:
		</para>

<pre>
$unit->call(@rowops);
$unit->fork(@rowops);
$unit->schedule(@rowops); 
</pre>

		<indexterm>
			<primary>error handling</primary>
		</indexterm>
		<indexterm>
			<primary>tray</primary>
		</indexterm>
		<para>
		But this way it loos more confusing. 
		Calling these functions with multiple arguments produces the same
		result as doing multiple calls with one argument at a time. Not only
		rowops but also <i>trays</i> (to be discussed later) of rowops can be
		used as arguments.
		</para>

		<para>
		These methods are among those
		that use the new error handling, that makes the operation to confess
		on any fatal errors. So there is no need to check their results with
		<quote><pre>or confess</pre></quote>.
		</para>

		<para>
		Also there is a call that selects the enqueueing mode by argument:
		</para>

<pre>
$unit->enqueque($mode, @rowops);
</pre>

		<para>
		The calling rules are exactly the same for the other enqueueing methods,
		may have multiple rowops or trays as arguments, no need to check the result.
		The <pre>$mode</pre> argument is one of:
		</para>

		<indexterm>
			<primary>constants</primary>
		</indexterm>
		<itemizedlist>
		<listitem>
		<pre>&Triceps::EM_CALL</pre> or <pre>"EM_CALL"</pre>
		</listitem>
		<listitem>
		<pre>&Triceps::EM_FORK</pre> or <pre>"EM_FORK"</pre>
		</listitem>
		<listitem>
		<pre>&Triceps::EM_SCHEDULE</pre> or <pre>"EM_SCHEDULE"</pre>
		</listitem>
		</itemizedlist>

		<para>
		As usual, there are calls to convert between the integer constant and
		string representations:
		</para>

<pre>
$string = &Triceps::emString($value);
$value = &Triceps::stringEm($string);
</pre>

		<para>
		And as usual, if the value can not be translated, they return <pre>undef</pre>.
		</para>

		<para>
		The frame marks for looping are created as their own class:
		</para>

		<indexterm>
			<primary>frame mark</primary>
		</indexterm>
<pre>
$mark = Triceps::FrameMark->new("name") or confess "$!";
</pre>

		<para>
		The name can be obtained back from the mark:
		</para>

<pre>
$name = $mark->getName();
</pre>

		<para>
		Other than that, the frame marks are completely opaque, and can be used
		only for the loop scheduling. Not even the <pre>same()</pre> method is supported
		for them at the moment, though it probably will be in the future. The
		mark gets set and used as:
		</para>

<pre>
$unit->setMark($mark);
$unit->loopAt($mark, @rowops);
</pre>

		<para>
		The rowop arguments of the <pre>loopAt()</pre> are the same as for the
		other enqueueing functions, and as for other functions they may happen
		to be trays. These methods also use the new error handling scheme, and
		will confess on errors. No need to check the results.
		</para>

		<para>
		There also are the convenience methods that create the rowops
		from the field values and immediately enqueue them:
		</para>

<pre>
$unit->makeHashCall($label, $opcode, 
	$fieldName => $fieldValue, ...);
$unit->makeArrayCall($label, $opcode, @fieldValues);

$unit->makeHashSchedule($label, $opcode, 
	$fieldName => $fieldValue, ...);
$unit->makeArraySchedule($label, $opcode, @fieldValues);

$unit->makeHashLoopAt($mark, $label, $opcode, 
	$fieldName => $fieldValue, ...);
$unit->makeArrayLoopAt($mark, $label, $opcode, @fieldValues);
</pre>

		<para>
		These are essentially the shorter ways to make the rowops and enqueue
		them without the three-deep calls. Only the methods for the most 
		frequently used enqueueing modes are provided, not for all of them.
		All these methods also confess on errors.
		</para>

		<para>
		The methods for creation of labels have been already discussed in
		<xref linkend="sc_Label_construction" xrefstyle="select: label quotedtitle pageabbrev"/>&xrsp;.
		Here is their recap along with the similar methods for creation of
		tables and trays that will be discussed later:
		</para>

<pre>
$label = $unit->makeDummyLabel($rowType, "name") 
	or confess "$!";

$label = $unit->makeLabel($rowType, "name",
	$clearSub, $execSub, @args) or confess "$!";

$label = $unit->makeClearingLabel("name", @args);

$table = $unit->makeTable($tableType, $enqMode, "name") 
	or confess "$!";

$tray = $unit->makeTray(@rowops) or confess "$!"; 
</pre>

		<para>
		Of them <pre>makeClearingLabel()</pre> uses the new error handling convention,
		confessing by itself, and the rest return an undef on errors that
		has to be checked. It's actually real difficult to make 
		<pre>makeClearingLabel()</pre> fail, only by corrupting some of the Triceps
		internal variables, and it was a late additiion, so going straight
		with the new convention for it made sense.
		</para>

		<indexterm>
			<primary>memory management</primary>
		</indexterm>
		<para>
		A special thing about the labels is that when a unit creates
		a label, it keeps a reference to it, for clearing. A label keeps a pointer
		back to the unit but not a reference (if you call <pre>getUnit()</pre>
		on a label, the returned value becomes a reference). For a table
		or a tray, the unit doesn't keep a reference to them. Instead,
		they keep a reference to the unit. With the tables, it can get
		pretty involved: A table has labels associated with it.
		When a table is created, it also creates these labels.
		The unit keeps references of these labels. The table also
		keeps references of these labels. The table keeps a reference
		of the unit. The labels (they are at the &Cpp; level, not Perl level)
		have pointers to the unit and the
		table but not references, to avoid the reference cycles.
		</para>

		<para>
		See more on the memory management and label clearing in the
		<xref linkend="ch_memory" xrefstyle="select: label quotedtitle pageabbrev"/>&xrsp;.
		</para>

		<para>
		The convenience methods to create the whole front part of the 
		topological loop are:
		</para>

		<indexterm>
			<primary>scheduling</primary>
			<secondary>loop</secondary>
		</indexterm>
<pre>
($labelBegin, $labelNext, $frameMark) = $unit->makeLoopHead(
	$rowType, "name", $clearSub, $execSub, @args);

($labelBegin, $labelNext, $frameMark) = $unit->makeLoopAround(
	"name", $labelFirst);
</pre>

		<para>
		You don't have to use them, you can create the loops manually. These
		methods merely make it more convenient. Remember also that a procedural
		loop is usually much easier to write and debug and read later than a
		topological one.
		</para>

		<para>
		These methods use the new error handling convention, confessing
		on the errors. There is no need to check the result.
		</para>

		<para>
		<pre>makeLoopHead()</pre> creates the front part of the loop that starts
		with a Perl label. It gets the arguments for that label and
		creates it among the other things. <pre>makeLoopAround()</pre> creates
		the front part of the loop around an existing label that
		will be the first one executed in the loop. <pre>makeLoopHead()</pre> 
		is really redundant and can be replaced with a combination
		of <pre>makeLabel()</pre> and <pre>makeLoopAround()</pre>.
		</para>

		<para>
		They both return the same results, a triplet:
		</para>

		<itemizedlist>
		<listitem>
		The label where you send a rowop to initiate the loop (remember that
		the loop consists of one row going through the loop at a time), 
		<pre>$labelBegin</pre>.
		</listitem>
		<listitem>
		The label that you use at the end of the loop in the <pre>loopAt()</pre> to
		do the next iteration of the loop, <pre>$labelNext</pre>.
		</listitem>
		<listitem>
		The frame mark that you use in <pre>loopAt()</pre>, <pre>$frameMark</pre>. You don't need to
		set the frame mark, it will be set for you in the wrapper logic.
		</listitem>
		</itemizedlist>

		<para>
		The name is used to construct the names of the elements by adding 
		a dotted suffix: <quote>name.begin</quote>, <quote>name.next</quote> for 
		<pre>makeLoopHead()</pre> or <quote>name.wrapnext</quote> for <pre>makeLoopAround()</pre>,
		<quote>name.mark</quote>.
		</para>

		<para>
		<pre>makeLoopAround()</pre> takes the row type for its created labels
		from the first label that is given to it as an argument.
		</para>

		<para>
		The makeLoop methods may be easier to understand if you look at their source code:
		</para>

<pre>
sub makeLoopHead # ($self, $rt, $name, $clearSub, $execSub, @args)
{
	my ($self, $rt, $name, $clear, $exec, @args) = @_;

	my $mark = Triceps::FrameMark->new($name . ".mark") or confess "$!";

	my $lbNext = $self->makeLabel($rt, $name . ".next", $clear, sub {
		$self->setMark($mark);
		&$exec(@_);
	}, @args) or confess "$!";

	my $lbBegin = $self->makeLabel($rt, $name . ".begin", undef, sub {
		$self->call($lbNext->adopt($_[1]));
	}) or confess "$!";

	return ($lbBegin, $lbNext, $mark);
}

sub makeLoopAround # ($self, $name, $lbFirst)
{
	my ($self, $name, $lbFirst) = @_;
	my $rt = $lbFirst->getRowType();

	my $mark = Triceps::FrameMark->new($name . ".mark") or confess "$!";

	my $lbWrapNext = $self->makeLabel($rt, $name . ".wrapnext", undef, sub {
		$self->setMark($mark);
	}) or confess "$!";
	$lbWrapNext->chain($lbFirst) or confess "$!";

	my $lbBegin = $self->makeLabel($rt, $name . ".begin", undef, sub {
		$self->call($lbWrapNext->adopt($_[1]));
	}) or confess "$!";

	return ($lbBegin, $lbWrapNext, $mark);
}
</pre>

		<para>
		The label execution handlers in them use <pre>$_[1]</pre> to get their rowop
		argument, without assigning it to a variable first.
		An extended example of them will also be shown in 
		<xref linkend="sc_sched_loop_ex" xrefstyle="select: label quotedtitle pageabbrev"/>&xrsp;.
		</para>

		<para>
		The unit can be checked for the emptiness of its queues:
		</para>

<pre>
$result = $unit->empty();
</pre>

		<para>
		Also the current depth of the call stack (the number of the
		stack frames on the queue) can be found with:
		</para>

<pre>
$result = $unit->getStackDepth();
</pre>

		<para>
		It isn't of any use for the model logic as such but comes handy
		for debugging, to check in the loops that you haven't accidentally
		created a stack growing with iterations. When the unit is not
		running, the stack depth is 1, since the outermost frame always
		stays on the stack. When a rowop is being executed, the stack
		depth is at least 2.
		</para>

		<para>
		The functions for execution from the queues are:
		</para>

<pre>
$unit->callNext();
$unit->drainFrame();
</pre>

		<para>
		<pre>callNext()</pre> takes one label from the top (innermost) stack frame queue and calls it.
		If the innermost frame happens to be empty, it does nothing. 
		<pre>drainFrame()</pre> calls the rowops from the top stack frame until it becomes
		empty. This includes any rowops that may be created and enqueued as
		part of the execution of the previous rowops. But it doesn't pop the
		frame from the stack.  And of course the method
		<pre>call()</pre> causes the argument rowops to be executed immediately, without
		even being technically enqueued.
		</para>
	</sect1>

	<sect1 id="sc_sched_unwind">
		<title>Error handling during the execution</title>

		<indexterm>
			<primary>error handling</primary>
		</indexterm>
		<para>
		When the labels execute, they may produce errors in one of two ways:
		</para>

		<itemizedlist>
		<listitem>
		The Perl code in the label might die.
		</listitem>
		<listitem>
		The call topology might violate the rules. 
		</listitem>
		</itemizedlist>

		<para>
		The rules are basically that you can't make the recursive calls.
		A label may not make calls directly or through other labels to itself.
		The idea is to catch the call sequences that are likely to go into 
		the deep recursion and overflow the stack. It catches them early,
		on the first attempt of recursion. If you need to do the recursion,
		use <pre>schedule()</pre> or <pre>loopAt()</pre>. That way you 
		avoid overrunning the stack.
		</para>

		<indexterm>
			<primary>XS</primary>
		</indexterm>
		<indexterm>
			<primary>draining</primary>
		</indexterm>
		<indexterm>
			<primary>stack</primary>
			<secondary>unwinding</secondary>
		</indexterm>
		<para>
		Whichever way the error is detected, it causes the Triceps call
		stack to be unwound. The Perl error messages from <pre>die</pre> or <pre>confess</pre>
		and the &Cpp; tracing of rowop calls and label chainings get
		combined into a common stack trace. When the code gets back to
		Perl, the XS code triggers a <pre>confess</pre>. If that happens to be in the
		handler of another rowop, it continues the Triceps hybrid stack
		unwinding. If not caught by <pre>eval</pre>, it keeps going to the topmost
		<pre>call()</pre> or <pre>drainFrame()</pre> and causes the whole program to die.
		Which is a reasonable reaction most of the time.
		</para>

		<para>
		Remember, the root cause is a serious error that is likely to leave the
		model in an inconsistent state, and it should usually be considered
		as fatal. If you want to catch the errors, nip them in the bud
		by wrapping your Perl code in <pre>eval</pre>. Then you can handle
		the errors before thay have a chance to propagate.
		</para>

		<para>
		An interesting question is, what happens to the rowops that were
		in the Triceps stack frames when the stack gets unwound? They
		get thrown away. The memory gets collected thanks to the reference
		counting, but the rowops and their sequence order get thrown away.
		The reason is basically that there may be no catching of the
		errors until unwinding to the <pre>drainFrame()</pre>. The choice is to either
		throw away everything after the first error or keep trying to
		execute the following rowops, collecting the errors. And that
		might become a lot of errors. I've taken the choice of stopping
		as early as possible, because the state of the model will probably
		be corrupted anyway and nothing but garbage would be coming out
		(if anything would be coming at all and not be stuck in an
		endless loop).
		</para>
	</sect1>

	<sect1 id="sc_sched_mainloop">
		<title>The main loop</title>

		<indexterm>
			<primary>main loop</primary>
		</indexterm>
		<para>
		The execution unit doesn't magically process the data by itself.
		The data needs to be pushed into it, and the unit has to be told
		to process it. There has to be some internal code to drive it,
		that would continuously read the data, schedule, drain.
		</para>

		<para>
		A typical way of processing the incoming rowops in a loop is:
		</para>

<pre>
$stop = 0; 
while (!$stop) {
	$rowop = &readRowop(); # some user-defined function
	$unit->call($rowop);
	$unit->drainFrame();
} 
</pre>

		<para>
		The rowops coming from the outside get executed as they are received,
		and then any rows left over from them get handled by <pre>drainFrame()</pre>
		before the next incoming rowop is read.
		Some of the executed rowops may set <pre>$stop</pre>, and the main loop
		will exit. 
		</para>

		<para>
		There is also another version of the main loop that has been more
		historic:
		</para>

<pre>
$stop = 0; 
while (!$stop) {
	$rowop = &readRowop(); # some user-defined function
	$unit->schedule($rowop);
	$unit->drainFrame();
} 
</pre>

		<para>
		It uses <pre>schedule()</pre> instead of <pre>call()</pre> for the rowop.
		As long as only one rowop is scheduled before draining the frame, both
		versions work equally well. But if you schedule multiple rowops before
		draining the frame, you can introduce a subtle unpredictability in the
		execution order. It is described in detail in
		<xref linkend="sc_sched_issues" xrefstyle="select: label quotedtitle pageabbrev"/>&xrsp;.
		Actually, you can have the same problem if you don't drain the frame
		after each top-level <pre>call()</pre> too. But mentally <pre>call()</pre> kind of reminds
		better to feed the rowops one at a time, and also is slightly more
		efficient, so now I prefer the version with it.
		</para>

		<para>
		Many of the examples in this manual use the main loop along the
		following lines (with variations, to fit the examples, and as the
		main loop was refined over time):
		</para>

<pre>
while(&readLine) {
	chomp;
	my @data = split(/,/); # starts with a command, then string opcode
	my $type = shift @data;
	if ($type eq "lbCur") {
		$unit->makeArrayCall($lbCur, @data);
	} elsif ($type eq "lbPos") {
		$unit->makeArrayCall($lbPos, @data);
	}
	$unit->drainFrame();
}
</pre>

		<para>
		It reads the CSV (Comma-Separated Values) data from stdin,
		with the label name in the first column, the opcode in the
		second, and the data fields in the rest. Then dispatches
		according to the label. Doing a <pre>call()</pre> instead 
		of <pre>schedule()</pre> works just as well, and the following
		<pre>drainFrame()</pre> takes care of any rowops scheduled
		from the call.
		</para>

		<para>
		Many variations are possible. It can be generalized to look up
		the labels from the hash:
		</para>

<pre>
while(&readLine) {
	chomp;
	my @data = split(/,/); # starts with a command, then string opcode
	my $type = shift @data;
	$unit->makeArrayCall($labels{$type}, @data);
	$unit->drainFrame();
}
</pre>

		<para>
		Or call the procedural functions for some types:
		</para>

<pre>
while(&readLine) {
	chomp;
	my @data = split(/,/); # starts with a command, then string opcode
	my $type = shift @data;
	if ($type eq "lbCur") {
		$unit->makeArrayCall($lbCur, @data);
	} elsif ($type eq "lbPos") {
		$unit->makeArrayCall($lbPos, @data);
	} elsif ($type eq "clear") { # clear the previous day
		&clearByDate($tPosition, @data);
	}
	$unit->drainFrame();
}
</pre>

		<para>
		Though none of these small examples are production-ready. At the
		very least, their parsing of the CSV data is primitive. It
		can't handle the quoting properly and can't parse the data
		with commas in it.
		</para>

		<para>
		A better ready way to parse the data will be provided in the future.
		For now, make your own.
		</para>
	</sect1>

	<sect1 id="sc_sched_mainloop_socket">
		<title>Main loop with a socket</title>

		<indexterm>
			<primary>main loop</primary>
		</indexterm>
		<indexterm>
			<primary>socket</primary>
		</indexterm>
		<para>
		A fairly typical situation is when a CEP model has to run in a daemon
		process, receiving and sending data through the network sockets.
		Here goes an example that does this. It's not production-ready either.
		It still has the issue with the parsing of the CSV data, its
		handling of the errors is not well-tested, and it makes a few
		simplifying assumptions about the buffering (more on this below). 
		Other than that, it's a decent starting point.
		If you want to copy-and-paste
		this code for your experiments, it can be found in
		<pre>perl/Triceps/t/xQuery.t</pre>.
		</para>

<!-- t/xQuery.t, with requires assembled, with if(0) removed -->
<pre>
use Triceps;
use Carp;
use Errno qw(EINTR EAGAIN);
use IO::Poll qw(POLLIN POLLOUT POLLHUP);
use IO::Socket;
use IO::Socket::INET;

# the socket and buffering control for the main loop;
# they are all indexed by a unique id
our %clients; # client sockets
our %inbufs; # input buffers, collecting the whole lines
our %outbufs; # output buffers
our $poll; # the poll object
our $cur_cli; # the id of the current client being processed
our $srv_exit; # exit when all the client connections are closed

# writing to the output buffers
sub outBuf # ($id, $string)
{
	my $id = shift;
	my $line = shift;
	$outbufs{$id} .= $line;
	# If there is anything to write on a buffer, stop reading from it.
	$poll->mask($clients{$id} => POLLOUT);
}

sub outCurBuf # ($string)
{
	outBuf($cur_cli, @_);
}

sub closeClient # ($id, $h)
{
	my $id = shift;
	my $h = shift;
	$poll->mask($h, 0);
	$h->close();
	delete $clients{$id}; # OK perl Perl manual even when iterating
	delete $inbufs{$id};
	delete $outbufs{$id};
}

# The server main loop. Runs with the specified server socket.
# Uses the labels hash to send the incoming data to Triceps.
sub mainLoop # ($srvsock, $%labels)
{
	my $srvsock = shift;
	my $labels = shift;

	my $client_id = 0; # unique strings
	our $poll = IO::Poll->new();

	$srvsock->blocking(0);
	$poll->mask($srvsock => POLLIN);
	$srv_exit = 0;

	while(!$srv_exit || keys %clients != 0) {
		my $r = $poll->poll();
		confess "poll failed: $!" if ($r < 0 && ! $!{EAGAIN} && ! $!{EINTR});

		if ($poll->events($srvsock)) {
			while(1) {
				my $client = $srvsock->accept();
				if (defined $client) {
					$client->blocking(0);
					$clients{++$client_id} = $client;
					# &send("Accepted client $client_id\n");
					$poll->mask($client => (POLLIN|POLLHUP));
				} elsif($!{EAGAIN} || $!{EINTR}) {
					last;
				} else {
					confess "accept failed: $!";
				}
			}
		}

		my ($id, $h, $mask, $n, $s);
		while (($id, $h) = each %clients) {
			$cur_cli = $id;
			$mask = $poll->events($h);
			if (($mask & POLLHUP) && !defined $outbufs{$id}) {
				# &send("Lost client $client_id\n");
				closeClient($id, $h);
				next;
			}
			if ($mask & POLLOUT) {
				$s = $outbufs{$id};
				$n = $h->syswrite($s);
				if (defined $n) {
					if ($n >= length($s)) {
						delete $outbufs{$id};
						# now can accept more input
						$poll->mask($h => (POLLIN|POLLHUP));
					} else {
						substr($outbufs{$id}, 0, $n) = '';
					}
				} elsif(! $!{EAGAIN} && ! $!{EINTR}) {
					warn "write to client $id failed: $!";
					closeClient($id, $h);
					next;
				}
			}
			if ($mask & POLLIN) {
				$n = $h->sysread($s, 10000);
				if ($n == 0) {
					# &send("Lost client $client_id\n");
					closeClient($id, $h);
					next;
				} elsif ($n > 0) {
					$inbufs{$id} .= $s;
				} elsif(! $!{EAGAIN} && ! $!{EINTR}) {
					warn "read from client $id failed: $!";
					closeClient($id, $h);
					next;
				}
			}
			# The way this works, if there is no '\n' before EOF,
			# the last line won't be processed.
			# Also, the whole output for all the input will be buffered
			# before it can be sent.
			while($inbufs{$id} =~ s/^(.*)\n//) {
				my $line = $1;
				chomp $line;
				local $/ = "\r"; # take care of a possible CR-LF
				chomp $line;
				my @data = split(/,/, $line);
				my $lname = shift @data;
				my $label = $labels->{$lname};
				if (defined $label) {
					my $unit = $label->getUnit();
					confess "label '$lname' received from client $id has been cleared"
						unless defined $unit;
					eval {
						$unit->makeArrayCall($label, @data);
						$unit->drainFrame();
					};
					warn "input data error: $@\nfrom data: $line\n" if $@;
				} else {
					warn "unknown label '$lname' received from client $id: $line "
				}
			}
		}
	}
}
</pre>

		<indexterm>
			<primary>dispatch table</primary>
		</indexterm>
		<para>
		The general outline follows the single-threaded multiplexing server described in
		<biblioref linkend="Babkin10"/>. <pre>mainLoop()</pre> gets the server socket
		and a dispatch table of labels as its arguments. It then proceeds with waiting
		for connections. 
		</para>

		<para>
		Once a connection is received, it gets added to the set of
		active connections, to get included in the waiting for the input data.
		The input data is read as simplified CSV (no commas in the middle of values,
		and no way to reprsent the NULL values othar than for those omitted at the end
		of the line).
		It's expected to have the format:
		</para>

<pre>
name,opcode,data...
</pre>

		<para>
		Such as:
		</para>

<pre>
window,OP_INSERT,5,AAA,30,30
window.query,OP_INSERT
exit,OP_NOP
</pre>

		<para>
		The name part is then used to find a label in the dispatch table. The
		rest of the data is used to create a rowop for that label and execute it.
		</para>

		<para>
		The data is sent back to the client through buffering. To send some data
		to a client, use 
		</para>

<pre>
&outBuf($id, $text);
</pre>

		<para>
		The <pre>$id</pre> is the unique id of the client. How do you find, what is the id
		of the client you want to send the data to? When an input line is processed,
		the main loop knows, from what client it was received. It puts the id of that
		client in the global variable <pre>$cur_cli</pre>. You can take it from there and remember.
		If you want to reply to the current client, you don't need to bother yourself
		with the id at all, just call
		</para>

<pre>
&outCurBuf($text);
</pre>

		<para>
		If you remember an id for the future use, you'd probably need to check that
		the client hasn't disconnected before sending data to it:
		</para>

<pre>
if (exists $clients{$id}) {
	&outBuf($id, $text);
}
</pre>

		<para>
		Otherwise your output attempts would be leaking memory in the output buffer.
		In any case, if a client has disconnected, the further processing of its requests
		shoudl usually be stopped. The client ids are not reused, so this check is
		always safe.
		</para>

		<indexterm>
			<primary>Aleri</primary>
		</indexterm>
		<indexterm>
			<primary>Coral8</primary>
		</indexterm>
		<indexterm>
			<primary>Sybase</primary>
		</indexterm>
		<indexterm>
			<primary>StreamBase</primary>
		</indexterm>
		<para>
		Once some output is buffered to send to a client, the further input from that
		client stops being accepted until the output buffer drains. But the processing
		in the Triceps unit scheduler keeps running until it runs out of things to do
		before it returns to the main loop. All this time the output buffer keeps
		collecting data without sending it to the client.  Also, the input
		buffer might happen to already contain multiple lines. Then all these lines
		will be processed before the data from the output buffer starts being sent
		to the client. If a request produces a large amount of data, all this data
		will be buffered first. It's a simplification but really the commercial
		CEP systems aren't doing a whole lot better: when asked for the contents of
		a table/window/materliaized view, Coral8 and Aleri and Sybase (don't know
		about StreamBase but it might be not different either) would make a copy
		of it first before sending the data. In some cases the copy is more efficient
		because it references the rows rather than copying the whole byte data, but
		in the grand scheme of things it's all the same.
		</para>

		<para>
		Internally the information about the client sockets and their buffers is
		kept in the global hashes <pre>%clients</pre>, <pre>%inbufs</pre>, <pre>%outbufs</pre>.
		It could be done a a single hash of objects but this was simpler.
		</para>

		<para>
		The loop exits when the global variable <pre>$srv_exit</pre> gets set 
		(synchronously, i.e. by one of the label handlers) to 1 and all the 
		clients disconnect. The requirement for disconnection of all the clients
		makes sure that all the output buffers get flushed before exit, and
		that was the easiest way to achieve this goal.
		</para>

		<para>
		<pre>mainLoop()</pre> relies on the listening socket being already created,
		bound and given to it as a parameter. Here is a function used in the examples
		to create this socket and run the server in a separate process:
		</para>

<!-- t/xQuery.t -->
<pre>
sub startServer # ($labels)
{
	my $labels = shift;

	my $srvsock = IO::Socket::INET->new(
		Proto => "tcp",
		LocalPort => 0,
		Listen => 10,
	) or confess "socket failed: $!";
	my $port = $srvsock->sockport() or confess "sockport failed: $!";
	my $pid = fork();
	confess "fork failed: $!" unless defined $pid;
	if ($pid) {
		# parent
		$srvsock->close();
	} else {
		# child
		&mainLoop($srvsock, $labels);
		exit(0);
	}
	return ($port, $pid);
}
</pre>

		<para>
		It binds the socket to the port 0 to request that the OS bind it to
		a randum unused port. The port number is then read back with <pre>sockport()</pre>.
		The pair of the port numer and the server's child process id is then returned
		as the result. The process where the server runs is in this case
		just a child process, it's not properly daemonized.
		</para>

		<para>
		For a simple complete example, let's make an echo server that would print
		back the rows it receives:
		</para>

		<indexterm>
			<primary>dispatch table</primary>
		</indexterm>
<!-- t/xQuery.t, shifted left -->
<pre>
my $uEcho = Triceps::Unit->new("uEcho");
my $lbEcho = $uEcho->makeLabel($rtTrade, "echo", undef, sub {
	&outCurBuf($_[1]->printP() . "\n");
});
my $lbEcho2 = $uEcho->makeLabel($rtTrade, "echo2", undef, sub {
	&outCurBuf(join(",", "echo", &Triceps::opcodeString($_[1]->getOpcode()),
		$_[1]->getRow()->toArray()) . "\n");
});
my $lbExit = $uEcho->makeLabel($rtTrade, "exit", undef, sub {
	$srv_exit = 1;
});

my %dispatch;
$dispatch{"echo"} = $lbEcho;
$dispatch{"echo2"} = $lbEcho2;
$dispatch{"exit"} = $lbExit;

my ($port, $pid) = &startServer(\%dispatch);
print STDERR "port=$port pid=$pid\n";
waitpid($pid, 0);
exit(0);
</pre>

		<para>
		It starts the server and waits for it to exit. <pre>waitpid()</pre> is used here
		in a simplified way too, it should properly be done in a loop until
		it succeeds or an error other than <pre>EINTR</pre> is returned.
		</para>

		<para>
		<pre>$rt</pre> is the row type for the expected data. It's not particularly
		important here, so I didn't show its definition. Two labels, <quote>echo</quote>
		and <quote>echo2</quote> differ in the way they print the data back:
		<quote>echo</quote> prints it in the symbolic form while <quote>echo2</quote>
		prints in CSV. The label <quote>exit</quote> sets the exit flag.
		</para>

		<para>
		The names in the dispatch table don't have to be the same as the names of
		the labels. It's often convenient to have them the same but not mandatory.
		</para>
	</sect1>

	<sect1 id="sc_sched_loop_ex">
		<title>Example of a topological loop</title>

		<para>
		How to build the models with the topological loop is much easier to understand 
		with an example. So let's make an example that computes the
		Fibonacci numbers. It's a real overcomplicated and perverse way of
		calculating the Fibonacci numbers. But it also is a great 
		fit to the type of problems that get solved with the
		topological loop, one of a simple kind.
		</para>

		<indexterm>
			<primary>topological loop</primary>
		</indexterm>
		<indexterm>
			<primary>scheduling</primary>
			<secondary>loop</secondary>
		</indexterm>
		<para>
		First, a quick reminder of what is a Fibonacci number.
		Historically it's a solution to the problem of breeding
		the spherical rabbits in vacuum. But in the mathematical
		reality it's the sequence of numbers where each number
		is a sum of the two previous ones. Two initial elements
		are defined to be equal to 1, and it goes from there:
		</para>

		<para>
		F<subscript><i>i</i></subscript> = F<subscript><i>i</i>-1</subscript> + F<subscript><i>i</i>-2</subscript>
		</para>

		<para>
		F<subscript>1</subscript> = 1; F<subscript>2</subscript> = 1
		</para>

		<para>
		The Fibonacci numbers are often used as an example of recursive
		computations in the beginner's books on programming. The computation
		of the <i>n</i>-th Fibonacci number is usually shown computed like this:
		</para>

<!-- t/xSnippets.t -->
<pre>
sub fib1 # ($n)
{
	my $n = shift;
	if ($n <= 2) {
		return 1;
	} else {
		return &fib1($n-1) + &fib1($n-2);
	}
}
</pre>

		<para>
		However that's not a good way to compute in the real world.
		When a function calls itself recursively once, its complexity
		is linear, O(<i>n</i>). When a function calls itself twice or more,
		its complexity becomes exponential, O(e<superscript><i>n</i></superscript>).
		At first you might think that it's only quadratic O(<i>n</i><superscript>2</superscript>)
		because it forks two ways on each step. But these two ways keep forking
		and forking on each step, and it compounds to exponential. Which is
		a real bad thing.
		</para>

		<para>
		To think of it, it's a huge waste, since the (<i>n</i>-2)&emdash;th
		number is calculated anyway for the (<i>n</i>-1)&emdash;th number.
		Why calculate it separately the second time? We could as well have saved
		and reused it. The Lisp people have figured this
		out a long time ago, and the Lisp books (if you can read Finnish or
		Russian, <biblioref linkend="Hyvonen86"/> is a classical one)
		are full of examples that
		do exactly that. However I'm too lazy to explain how they work, so we're
		going to skip it together with the conversion of a tail recursion
		into a loop and get directly to the loop version. I find the loop
		version more natural and easier to write than a recursion anyway.
		</para>

<!-- t/xSnippets.t -->
<pre>
sub fibStep2 # ($prev, $preprev)
{
	return ($_[0] + $_[1], $_[0]);
}

sub fib2 # ($n)
{
	my $n = shift;
	my @prev = (1, 0); # n and n-1

	while ($n > 1) {
		@prev = &fibStep2(@prev);
		$n--;
	}
	return $prev[0];
}
</pre>

		<para>
		The split into two functions is not mandatory for the loop
		version, it just does the clean separation of the loop counter
		logic and of the computation of the next step of the function.
		(But for the recursion version if would be mandatory).
		</para>

		<para>
		I'm going to take this procedural loop version and transform
		it into a topological loop. It actually happens to be a real
		good match for the topological loop. In a topological loop
		a record keeps traveling through it and being transformed
		until it satisfies the loop exit condition. Here
		<pre>@prev</pre> is the record contents, and the iteration count
		will be added to them to keep track of the exit condition.
		</para>

<!-- t/FrameMark.t doFibHead -->
<pre>
$uFib = Triceps::Unit->new("uFib") or confess "$!";

my $rtFib = Triceps::RowType->new(
	iter => "int32", # iteration number
	cur => "int64", # current number
	prev => "int64", # previous number
) or confess "$!";

my $lbPrint = $uFib->makeLabel($rtFib, "Print", undef, sub {
	&send($_[1]->getRow()->get("cur"));
});

my $lbCompute; # will fill in later

my ($lbBegin, $lbNext, $markFib) = $uFib->makeLoopHead(
	$rtFib, "Fib", undef, sub {
		my $iter = $_[1]->getRow()->get("iter");
		if ($iter <= 1) {
			$uFib->call($lbPrint->adopt($_[1]));
		} else {
			$uFib->call($lbCompute->adopt($_[1]));
		}
	}
);

$lbCompute = $uFib->makeLabel($rtFib, "Compute", undef, sub {
	my $row = $_[1]->getRow();
	my $cur = $row->get("cur");
	$uFib->makeHashLoopAt($markFib, $lbNext, $_[1]->getOpcode(),
		iter => $row->get("iter") - 1,
		cur => $cur + $row->get("prev"),
		prev => $cur,
	);
}) or confess "$!";

my $lbMain = $uFib->makeLabel($rtFib, "Main", undef, sub {
	my $row = $_[1]->getRow();
	$uFib->makeHashCall($lbBegin, $_[1]->getOpcode(),
		iter => $row->get("iter"),
		cur => 1,
		prev => 0,
	);
	&send(" is a Fibonacci number ", $row->get("iter"), "\n");
}) or confess "$!";

while(&readLine) {
	chomp;
	my @data = split(/,/);
	$uFib->makeArrayCall($lbMain, @data);
	$uFib->drainFrame(); # just in case, for completeness
}
</pre>

		<para>
		You can see that it has grown quite a bit. That's why the procedural
		loops are generally a better idea. However if the computation involves
		a lot of the SQLy logic, the topological loops are still beneficial.
		Also, the Triceps <pre>call()</pre> is at the moment not a real call:
		you pass the arguments, you call the code but you don't directly
		get the results. You could pass them back through some static
		variables, or you could use the topological loop to pass them
		directly to the next iteration of the loop.
		</para>

		<para>
		The main loop reads the CSV lines with opcodes (which aren't
		really used here, just passed through and then thrown away
		before printing) and calls <pre>$lbMain</pre>. Here is an example
		of an input and output as they would intermix if the input
		was typed from the keyboard. As in the rest of this manual,
		the input lines are shown in bold.
		</para>

<!-- t/FrameMark.t doFibHead -->
<exdump>
> OP_INSERT,1
1 is a Fibonacci number 1
> OP_DELETE,2
1 is a Fibonacci number 2
> OP_INSERT,5
5 is a Fibonacci number 5
> OP_INSERT,6
8 is a Fibonacci number 6
</exdump>

		<para>
		The input lines contain the values only for the field <pre>iter</pre>, 
		which intentionally happens to be the first field in the row type. The
		other fields will be reset anyway in <pre>$lbMain</pre>, so they are left as NULL.
		</para>

		<para>
		The point of <pre>$lbMain</pre> is to call the loop begin label <pre>$lbBegin</pre> and then
		print the message about which Fibonacci number was requested. The value
		of the computed number is printed at the end of the loop, so when the
		words <quote>is a Fibonacci number</quote> are printed after it, that demonstrates that the
		execution of <pre>$lbMain</pre> continues only after the loop is completed.
		</para>

		<para>
		The loop logic is split into two labels <pre>$lbNext</pre> and <pre>$lbCompute</pre> purely
		to show that it can be split like this. <pre>$lbNext</pre> handles the loop termination
		condition, and <pre>$lbCompute</pre> does essentially the work of <pre>fibStep2()</pre>. After
		the loop terminates, it passes the result row to <pre>$lbPrint</pre> for the priniting
		of the value. 
		</para>

		<para>
		When the code for <pre>$lbNext</pre> is created, it contains the call of <pre>$lbCompute</pre>.
		However the label <pre>$lbCompute</pre> has not been created at this time yet! Not a problem,
		creating in advance an empty variable <pre>$lbCompute</pre> is enough. The closure in <pre>$lbNext</pre> will keep
		a reference to that variable, and the variable will be filled with the reference
		to the label later (but before the main loop executes).
		</para>

		<para>
		And here is the version with <pre>makeLoopAround()</pre>:
		</para>

<!-- t/FrameMark.t doFibAround -->
<pre>
my ($lbBegin, $lbNext, $markFib); # will fill in later

$lbCompute = $uFib->makeLabel($rtFib, "Compute", undef, sub {
	my $row = $_[1]->getRow();
	my $cur = $row->get("cur");
	my $iter = $row->get("iter");
	if ($iter <= 1) {
		$uFib->call($lbPrint->adopt($_[1]));
	} else {
		$uFib->makeHashLoopAt($markFib, $lbNext, $_[1]->getOpcode(),
			iter => $row->get("iter") - 1,
			cur => $cur + $row->get("prev"),
			prev => $cur,
		);
	}
}) or confess "$!";

($lbBegin, $lbNext, $markFib) = $uFib->makeLoopAround(
	"Fib", $lbCompute
);
</pre>

		<para>
		The unit, row type, <pre>$lbPrint</pre>, <pre>$lbMain</pre> and 
		the main loop have stayed the same,
		so they are omitted from this example. The whole loop logic, both
		the termination condition and the computation step, have been 
		collected into one label <pre>$lbCompute</pre>, to show that it can be done this
		way too. Then the loop head is created around <pre>$lbCompute</pre>.
		</para>

	</sect1>

	<sect1 id="sc_sched_issues">
		<title>Issues with the Triceps scheduling</title>

		<para>
		As much as I like it, the Triceps scheduling is not perfect, and has
		some open issues at the moment.  Some of them have been already
		mentioned in the description of the loop scheduling: it's a bit
		confusing that the frame mark is placed on the next outer scheduling
		stack frame and not on the current one. This leads to the interesting
		effects in execution order.
		</para>

		<para>
		The other one has been mentioned in the main loop discussion:
		the <pre>schedule()</pre> call, when used from inside the scheduled code,
		may introduce unpredictability in the execution order. It puts the rowop
		after the last rowop in the outermost stack frame. But the outermost
		stack frame may contain a whole queue of rowops that come from the outside.
		This means that the exact order of execution will depend on the timing
		of the rowops arriving from outside. 
		</para>

		<para>
		Let me demonstrate it with an example. Suppose the main loop tries to
		optimize by collecting and schedulng as many incoming rowops as it can
		before running them:
		</para>

		<indexterm>
			<primary>main loop</primary>
		</indexterm>
<pre>
$stop = 0; 
while (!$stop) {
	&waitForIncomingData(); # some user-defined function
	while ($rowop = &readRowop()) { # some user-defined function
		$unit->schedule($rowop);
	}
	$unit->drainFrame();
} 
</pre>

		<para>
		Suppose the rowops A, B, C, D are being received from the outside.
		When the rowop A executes, it schedules the rowop E. Then depending
		on the timing of the packets in the network, the call sequence may
		be
		</para>

<pre>
schedule(A)
drainFrame()
schedule(B)
drainFrame()
schedule(C)
drainFrame()
schedule(D)
drainFrame()
</pre>

		<para>
		or
		</para>

<pre>
schedule(A)
schedule(B)
drainFrame()
schedule(C)
schedule(D)
drainFrame()
</pre>

		<para>
		or
		</para>

<pre>
schedule(A)
schedule(B)
schedule(C)
schedule(D)
drainFrame()
</pre>

		<para>
		or a few other combinations. In the first case the actual execution order
		will be A, E, B, C, D. That's because when A schedules E, E will be picked
		up and executed by the first following frame drain. In the second case the
		execution order will be A, B, E, C, D. Here when E gets scheduled, B is 
		already on the queue in front of it. In the third case the order will
		be A, B, C, D, E. And it will fluctuate at random between the runs.
		</para>

		<para>
		If the repeatable execution order is important (and usually it is), 
		the solution if to feed the rowops one by one and drain the frame
		right afterwards. Then the execution order will always be A, E, B, C, D.
		When feeding one by one, <pre>call()</pre> can be used instead of <pre>schedule()</pre>,
		and even slightly more efficient. Just don't forget to drain the frame
		after each call.
		</para>

		<indexterm>
			<primary>scheduling</primary>
			<secondary>loop</secondary>
		</indexterm>
		<para>
		The same issue happens with the topological loops that have been temporarily
		stopped and then resumed on arrival of more data from outside. The mark
		of such a loop will be unset when the loop continues, and looping at
		this mark will be equivalent to <pre>schedule()</pre>, having the same
		repeatability problem. The same solution works for this issue too.
		</para>

		<para>
		The method <pre>fork()</pre> is not exactly useful. It was created when I've
		thought that it's the solution to the problem of the loops. Which it
		has turned out to not solve, and another solution had to be devised.
		Now it really doesn't have much use, and will probably be removed in
		the future.
		</para>

		<para>
		I have a few ideas for better solutions of these issues, but they will
		need a bit more experimentation. Just keep in mind that the scheduling
		will be refined in the future. It will still have the same general
		shape but differ in detail.
		</para>
	</sect1>

	<sect1 id="sc_sched_tray">
		<title>Trays, or yes bundling</title>

		<indexterm>
			<primary>bundling</primary>
		</indexterm>
		<para>
		Even though Triceps does no bundling in scheduling, there still is a
		need to store the sequences of row operations. This is an important
		distinction, since the stored sequences are to be scheduled somewhere
		in the future (or maybe not scheduled at all, but iterated through
		manually). If and when they get scheduled, they will be unbundled. The
		ordered storage only provides the order for that future scheduling or
		iteration.
		</para>

		<para>
		The easiest way to store rowops is to put them into the Perl arrays,
		like:
		</para>

<pre>
my @ops = ($rowop1, $rowop2);
push @ops, $rowop3;
</pre>

		<indexterm>
			<primary>tray</primary>
		</indexterm>
		<para>
		However the &Cpp; internals of Triceps do not know about the Perl
		arrays. And some of them can work directly with the sequences of rowops. So
		Triceps defines an internal sort-of-equivalent of Perl array for
		rowops, called a <i>Tray</i>.
		</para>

		<para>
		The trays have first been used to <quote>catch</quote> the side effects of
		operations on the stateful elements, so the name <quote>tray</quote> came from the
		metaphor <quote>put a tray under it to catch the drippings</quote>.
		</para>

		<para>
		The trays get created as:
		</para>

<pre>
$tray = $unit->makeTray(@rowops) or confess "$!";
</pre>

		<para>
		A tray always stores rowops for only one unit. It can be only used in
		one thread. A tray can be used in all the enqueueing methods,
		just like the direct rowops:
		</para>

<pre>
$unit->call($tray);
$unit->fork($tray);
$unit->schedule($tray);
$unit->enqueue($mode, $tray);
$unit->loopAt($mark, $tray);
</pre>

		<para>
		Moreover, multiple trays may be passed, and the loose rowops and trays
		can be mixed in the multiple arguments of these functions, for example:
		</para>

<pre>
$unit->call($rowopStartPkg, $tray, $rowopEndPkg);
</pre>

		<indexterm>
			<primary>protocol</primary>
		</indexterm>
		<para>
		In this example nothing really stops you from placing the start and end
		rowops into the tray too. A tray may contain the rowops of any types
		mixed in any order. This is by design, and it's an important feature
		that allows to build the protocol blocks out of rowops and perform an
		orderly data exchange. This feature is an absolute necessity for proper
		inter-process and inter-thread communication.
		</para>

		<indexterm>
			<primary>Coral8</primary>
		</indexterm>
		<indexterm>
			<primary>Aleri</primary>
		</indexterm>
		<indexterm>
			<primary>Sybase</primary>
		</indexterm>
		<para>
		The ability to send the rows of multiple types through the same channel
		in order is a must, and its lack makes the communication with some
		other CEP systems exceedingly difficult. Coral8 supports only one
		stream per connection. Aleri (and I believe Sybase R5) allows to send
		multiple streams through the same connection but has no guarantees of
		order between them. I don't know about the others, check yourself.
		</para>

		<para>
		To iterate on a tray in the Perl code, it can be converted to a Perl array:
		</para>

<pre>
@array = $tray->toArray();
</pre>

		<para>
		The size of the tray (the count of rowops in it) can be read directly
		without a conversion, and the unit can be read back too:
		</para>

<pre>
$size = $tray->size();
$traysUnit = $tray->getUnit();
</pre>

		<para>
		Another way to create a tray is by copying an existing one:
		</para>

<pre>
$tray2 = $tray1->copy();
</pre>

		<para>
		This copies the contents (which is the references to the rowops) and
		does not create any ties between the trays. The copying is really just
		a more efficient way to do
		</para>

<pre>
$tray2 = $tray1->getUnit()->makeTray($tray1->toArray());
</pre>

		<para>
		The tray references can be compared for whether they point to the same
		tray object:
		</para>

<pre>
$result = $tray1->same($tray2);
</pre>

		<para>
		The contents of a tray may be cleared. Which is more convenient and
		more efficient than discarding a tray and creating another one:
		</para>

<pre>
$tray->clear();
</pre>

		<para>
		The data may be added to the back of a tray:
		</para>

<pre>
$tray->push(@rowops);
</pre>

		<para>
		Multiple rowops can be pushed in a single call. There are no other
		Perl-like operations on a tray: it's either create from a set of
		rowops, push, or convert to a Perl array.
		</para>

		<para>
		Note that the trays are mutable, unlike the rows and rowops. Multiple
		references to a tray will see the same contents. If a tray is changed
		through one reference, the others will see the changes too.
		</para>
	</sect1>

	<sect1 id="sc_sched_tracing">
		<title>Tracing the execution</title>

		<indexterm>
			<primary>tracing</primary>
		</indexterm>

		<para>
		When developing the CEP models, there always comes the question: WTF
		had just happened? How did it manage get this result? Followed by
		subscribing to many intermediate results and trying to piece together
		the execution order.
		</para>

		<para>
		Triceps provides two solutions for this situation: First, the
		procedural approach should make the logic much easier to follow.
		Second, it has a ready way to trace the execution and then read the
		trace in one piece. It can also be used to analyze any variables on the
		fly, and possibly stop the execution and enter some manual mode.
		</para>

		<para>
		The idea here is simple: provide the Unit with a method that will be
		called:
		</para>

		<itemizedlist>
		<listitem>
		before a label executes,
		</listitem>
		<listitem>
		after the label executes but before draining its frame,
		</listitem>
		<listitem>
		after the frame is drained but before the chained labels execute,
		</listitem>
		<listitem>
		after all the execution caused by the label is completed.
		</listitem>
		</itemizedlist>

		<para>
		For the simple tracing, there is a small simple tracer provided. It
		actually executes directly as compiled in &Cpp; so it's quite
		efficient:
		</para>

<pre>
$tracer = Triceps::UnitTracerStringName(option => $value, ...)
	or confess "$!";
</pre>

		<para>
		The arguments are specified as the option name-value pairs.
		</para>

		<para>
		The only option supported is <pre>verbose</pre>, which may be 0 (default) or
		non-0. If it's 0 (false), the tracer will record a message only before
		executing each label. If true, it will record a message after each
		stage. The class is named UnitTracerStringName because it records the
		execution trace in the string format, including the names of the
		labels. The tracer is set into the unit:
		</para>

		<indexterm>
			<primary>unit</primary>
		</indexterm>
		<indexterm>
			<primary>tracing</primary>
		</indexterm>
<pre>
$unit->setTracer($tracer); 
</pre>
		
		<para>
		The unit's current tracer can also be read back:
		</para>

<pre>
$oldTracer = $unit->getTracer();
</pre>

		<indexterm>
			<primary>error handling</primary>
		</indexterm>
		<para>
		If no tracer was previously set, <pre>getTracer()</pre> will return <pre>undef</pre>.
		And <pre>undef</pre> can also be used as an argument of <pre>setTracer()</pre>, to
		cancel any previously set tracing. <pre>setTracer()</pre> has the new-style
		error handling and confesses on errors.
		</para>

		<para>
		The tracer references can be compared for whether they refer to the
		same underlying object:
		</para>

<pre>
$result = $tracer1->same($tracer2);
</pre>

		<para>
		There are multiple kinds of tracer objects, and <pre>same()</pre> can
		be called safely for either kind of tracer, including mixing them
		together. Of course, the tracers of different kinds definitely would
		not be the same tracer object.
		</para>

		<para>
		As the unit runs, the tracing information gets collected in the tracer
		object. It can be extracted back with:
		</para>

<pre>
$data = $tracer->print();
</pre>

		<para>
		This does not reset the trace. To reset it, use:
		</para>

<pre>
$tracer->clearBuffer();
</pre>

		<para>
		Here is a code sequence designed to produce a fairly involved trace:
		</para>

<!-- t/Unit.t, with ok() replaced with confess() -->
<pre>
$sntr = Triceps::UnitTracerStringName->new(verbose => 1);
$u1->setTracer($sntr);

$c_lab1 = $u1->makeDummyLabel($rt1, "lab1") 
	or confess "$!";
$c_lab2 = $u1->makeDummyLabel($rt1, "lab2") 
	or confess "$!";
$c_lab3 = $u1->makeDummyLabel($rt1, "lab3") 
	or confess "$!";

$c_op1 = $c_lab1->makeRowop(&Triceps::OP_INSERT, $row1) 
	or confess "$!";
$c_op2 = $c_lab1->makeRowop(&Triceps::OP_DELETE, $row1) 
	or confess "$!";

$v = $c_lab1->chain($c_lab2) or confess "$!";
$v = $c_lab1->chain($c_lab3) or confess "$!";
$v = $c_lab2->chain($c_lab3) or confess "$!";

$u1->schedule($c_op1);
$u1->schedule($c_op2);

$u1->drainFrame();
</pre>

		<para>
		The trace is:
		</para>

<exdump>
unit 'u1' before label 'lab1' op OP_INSERT {
unit 'u1' drain label 'lab1' op OP_INSERT
unit 'u1' before-chained label 'lab1' op OP_INSERT
unit 'u1' before label 'lab2' (chain 'lab1') op OP_INSERT {
unit 'u1' drain label 'lab2' (chain 'lab1') op OP_INSERT
unit 'u1' before-chained label 'lab2' (chain 'lab1') op OP_INSERT
unit 'u1' before label 'lab3' (chain 'lab2') op OP_INSERT {
unit 'u1' drain label 'lab3' (chain 'lab2') op OP_INSERT
unit 'u1' after label 'lab3' (chain 'lab2') op OP_INSERT }
unit 'u1' after label 'lab2' (chain 'lab1') op OP_INSERT }
unit 'u1' before label 'lab3' (chain 'lab1') op OP_INSERT {
unit 'u1' drain label 'lab3' (chain 'lab1') op OP_INSERT
unit 'u1' after label 'lab3' (chain 'lab1') op OP_INSERT }
unit 'u1' after label 'lab1' op OP_INSERT }
unit 'u1' before label 'lab1' op OP_DELETE {
unit 'u1' drain label 'lab1' op OP_DELETE
unit 'u1' before-chained label 'lab1' op OP_DELETE
unit 'u1' before label 'lab2' (chain 'lab1') op OP_DELETE {
unit 'u1' drain label 'lab2' (chain 'lab1') op OP_DELETE
unit 'u1' before-chained label 'lab2' (chain 'lab1') op OP_DELETE
unit 'u1' before label 'lab3' (chain 'lab2') op OP_DELETE {
unit 'u1' drain label 'lab3' (chain 'lab2') op OP_DELETE
unit 'u1' after label 'lab3' (chain 'lab2') op OP_DELETE }
unit 'u1' after label 'lab2' (chain 'lab1') op OP_DELETE }
unit 'u1' before label 'lab3' (chain 'lab1') op OP_DELETE {
unit 'u1' drain label 'lab3' (chain 'lab1') op OP_DELETE
unit 'u1' after label 'lab3' (chain 'lab1') op OP_DELETE }
unit 'u1' after label 'lab1' op OP_DELETE }
</exdump>

		<para>
		In non-verbose mode the same trace would be:
		</para>

<exdump>
unit 'u1' before label 'lab1' op OP_INSERT
unit 'u1' before label 'lab2' (chain 'lab1') op OP_INSERT
unit 'u1' before label 'lab3' (chain 'lab2') op OP_INSERT
unit 'u1' before label 'lab3' (chain 'lab1') op OP_INSERT
unit 'u1' before label 'lab1' op OP_DELETE
unit 'u1' before label 'lab2' (chain 'lab1') op OP_DELETE
unit 'u1' before label 'lab3' (chain 'lab2') op OP_DELETE
unit 'u1' before label 'lab3' (chain 'lab1') op OP_DELETE
</exdump>

		<para>
		The verbose trace has the <quote>before</quote> and <quote>after</quote> lines marked
		with the curly braces, so that when it gets loaded into an editor,
		it can be navigated relatively easily.
		</para>

		<para>
		The actual contents of the rows is not printed in either case. This
		is basically because the tracer is implemented in &Cpp;, and I've been
		trying to keep the knowledge of the meaning of the simple data types
		out of the &Cpp; code as much as possible for now. But it can be
		implemented with a Perl tracer.
		</para>

		<para>
		A Perl tracer is created with:
		</para>

<pre>
$tracer = Triceps::UnitTracerPerl->new($sub, @args) 
	or confess "$!";
</pre>

		<para>
		The arguments are a reference to a function, and optionally arguments
		for it. The resulting tracer can be used in the unit's <pre>setTracer()</pre> as
		usual.
		</para>

		<para>
		The function of the Perl tracer gets called as:
		</para>

<pre>
&$sub($unit, $label, $fromLabel, $rowop, $when, @args)
</pre>

		<para>
		The arguments are:
		</para>

		<itemizedlist>
		<listitem>
		<pre>$unit</pre> is the usual unit reference.
		</listitem>
		<listitem>
		<pre>$label</pre> is the current label being traced.
		</listitem>
		<listitem>
		<pre>$fromLabel</pre> is the parent label in the chaining (would be
		<pre>undef</pre> if the current label is called directly, without
		chaining from anything).
		</listitem>
		<listitem>
		<pre>$rowop</pre> is the current row operation.
		</listitem>
		<listitem>
		<pre>$when</pre> is an integer constant showing the point when the
		tracer is being called. It's value may be one of
		<pre>&Triceps::TW_BEFORE</pre>, <pre>&Triceps::TW_BEFORE_DRAIN</pre>,
		<pre>&Triceps::TW_BEFORE_CHAINED</pre>, <pre>&Triceps::TW_AFTER</pre>;
		the prefix <pre>TW</pre> stands for <quote>tracer when</quote>.
		</listitem>
		<listitem>
		<pre>@args</pre> are the extra arguments passed from the tracer creation.
		</listitem>
		</itemizedlist>

		<indexterm>
			<primary>constants</primary>
		</indexterm>
		<para>
		The <pre>TW_*</pre> constants can as usual be converted to and from
		strings with the calls
		</para>

<pre>
$string = &Triceps::tracerWhenString($value);
$value = &Triceps::stringTracerWhen($string);
</pre>

		<para>
		There also are the conversion functions with strings more suitable for
		the human-readable messages: <quote>before</quote>, <quote>drain</quote>, <quote>before-chained</quote>,
		<quote>drain</quote>. These are actually the conversions used in the
		UnitTracerStringName. The functions for them are:
		</para>

<pre>
$string = &Triceps::tracerWhenHumanString($value);
$value = &Triceps::humanStringTracerWhen($string);
</pre>

		<para>
		The Perl tracers allow to execute any arbitrary actions when tracing.
		They can act as breakpoints by looking for certain conditions and
		opening a debugging session when those are met.
		</para>

		<para>
		For an example of a Perl tracer, let's start with a
		tracer function that works like UnitTracerStringName:
		</para>

<!-- t/Unit.t x_Unit_A -->
<pre>
sub tracerCb()
{
	my ($unit, $label, $from, $rop, $when, @extra) = @_;
	our $history;

	my $msg = "unit '" . $unit->getName() . "' " 
		. Triceps::tracerWhenHumanString($when) . " label '" 
		. $label->getName() . "' ";
	if (defined $fromLabel) {
		$msg .= "(chain '" . $fromLabel->getName() . "') ";
	}
	$msg .= "op " . Triceps::opcodeString($rop->getOpcode());
	if ($when == &Triceps::TW_BEFORE) {
		$msg .= " {";
	} elsif ($when == &Triceps::TW_AFTER) {
		$msg .= " }";
	}
	$msg .= "\n";
	$history .= $msg;
}

undef $history;
$ptr = Triceps::UnitTracerPerl->new(\&tracerCb);
$u1->setTracer($ptr);
</pre>

		<para>
		It's slightly different, in the way that it always produces the
		verbose trace, and that it collectes the trace in the global
		variable <pre>$history</pre>. But the resulting text is the same as
		with UnitTracerStringName.
		</para>

		<para>
		Now let's improve on it by printing the
		whole rowop contents too. In a <quote>proper</quote> way this advanced
		tracer would be defined as a class constructing the tracer objects.
		But to reduce the amount of code let's just make it a standalone
		function to be used with the Perl tracer constructor. 
		</para>

		<para>
		And for extra nicety let's make the result nicely indented, with two spaces
		per the indenting level. The indenting
		is actually not such a great idea: with the long sequences of the calls
		between the labels, the nesting levels would also be deep, and the
		output would be indented way off the right end of the screen. That's why
		it's not done in UnitTracerStringName (though it might be a good idea
		as an option). But for the small short examples it works well.
		The function would take 3 extra arguments:
		</para>

		<itemizedlist>
		<listitem>
		Verbosity, a boolean value.
		</listitem>
		<listitem>
		Reference to an array variable where to append the text of the trace.
		This is more flexible than the fixed <pre>$history</pre>. The array will contain
		the lines of the trace as its elements. And appending to an array
		should be more efficient than appending to the end of a potentially
		very long string.
		</listitem>
		<listitem>
		Reference to a scalar variable that would be used to keep the indenting level.
		The value of that variable will be updated as the tracing happens. Its
		initial value will determine the initial indenting level. Keeping the
		indenting is actually not easy because the indenting level
		can be changed for two reasons,
		the label chaining and the label calls. To get the logic working, one
		indenting level gets added before in advance, in front of the outermost
		trace lines. So, to make the outermost
		lines appear with no indenting, initialize this variable to -1 and not 0.
		</listitem>
		</itemizedlist>

<!-- t/Unit.t x_Unit_B -->
<pre>
sub traceStringRowop
{
	my ($unit, $label, $fromLabel, $rowop, $when, 
		$verbose, $rlog, $rnest) = @_;

	if ($verbose) {
		${$rnest}++ if ($when == &Triceps::TW_BEFORE);
		${$rnest}-- if ($when == &Triceps::TW_AFTER);
	} else {
		return if ($when != &Triceps::TW_BEFORE);
	}


	my $msg =  "unit '" . $unit->getName() . "' " 
		. Triceps::tracerWhenHumanString($when) . " label '"
		. $label->getName() . "' ";
	if (defined $fromLabel) {
		$msg .= "(chain '" . $fromLabel->getName() . "') ";
	}
	my $tail = "";
	if ($when == &Triceps::TW_BEFORE) {
		$tail = " {";
	} elsif ($when == &Triceps::TW_AFTER) {
		$tail = " }";
	}
	push (@{$rlog}, ("  " x ${$rnest}) . $msg . "op " 
		. $rowop->printP() . $tail);

	if ($verbose) {
		${$rnest}++ if ($when == &Triceps::TW_BEFORE);
		${$rnest}-- if ($when == &Triceps::TW_AFTER);
	}
}

undef @history;
my $tnest =  -1; # keeps track of the tracing nesting level
$ptr = Triceps::UnitTracerPerl->new(\&traceStringRowop, 1, \@history, \$tnest);
$u1->setTracer($ptr);
</pre>

		<para>
		For the same call sequence as before, the output will be as follows
		(I've tried to wrap the long lines in a logically consistent way
		but it still spoils the effect of indenting a bit):
		</para>

<exdump>
unit 'u1' before label 'lab1' op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  {
  unit 'u1' drain label 'lab1' op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text" 
  unit 'u1' before-chained label 'lab1' op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text" 
    unit 'u1' before label 'lab2' (chain 'lab1') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  {
      unit 'u1' drain label 'lab2' (chain 'lab1') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text" 
      unit 'u1' before-chained label 'lab2' (chain 'lab1') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text" 
        unit 'u1' before label 'lab3' (chain 'lab2') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  {
          unit 'u1' drain label 'lab3' (chain 'lab2') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text" 
        unit 'u1' after label 'lab3' (chain 'lab2') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  }
    unit 'u1' after label 'lab2' (chain 'lab1') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  }
    unit 'u1' before label 'lab3' (chain 'lab1') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  {
      unit 'u1' drain label 'lab3' (chain 'lab1') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text" 
    unit 'u1' after label 'lab3' (chain 'lab1') op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  }
unit 'u1' after label 'lab1' op lab1 OP_INSERT a="123" b="456" c="789" d="3.14" e="text"  }
unit 'u1' before label 'lab1' op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  {
  unit 'u1' drain label 'lab1' op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text" 
  unit 'u1' before-chained label 'lab1' op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text" 
    unit 'u1' before label 'lab2' (chain 'lab1') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  {
      unit 'u1' drain label 'lab2' (chain 'lab1') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text" 
      unit 'u1' before-chained label 'lab2' (chain 'lab1') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text" 
        unit 'u1' before label 'lab3' (chain 'lab2') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  {
          unit 'u1' drain label 'lab3' (chain 'lab2') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text" 
        unit 'u1' after label 'lab3' (chain 'lab2') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  }
    unit 'u1' after label 'lab2' (chain 'lab1') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  }
    unit 'u1' before label 'lab3' (chain 'lab1') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  {
      unit 'u1' drain label 'lab3' (chain 'lab1') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text" 
    unit 'u1' after label 'lab3' (chain 'lab1') op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  }
unit 'u1' after label 'lab1' op lab1 OP_DELETE a="123" b="456" c="789" d="3.14" e="text"  }
</exdump>

		<para>
		As mentioned before, each label produces two levels of indenting: one
		for everything after <quote>before</quote>, another one for the nested
		labels.
		</para>

		<para>
		Eventually this tracing should become another standard class in Triceps.
		</para>
	</sect1>

</chapter>
