most of this material is available in various places in the gt code and documentation base. pulling it together here is an attempt to aid first time and more experienced users in the finer points of gt use.
this file is targeted to both budding as well as seasoned (e.g. graybeards) GT users. the intent is to gather sage wisdom and experience in the use of gt primarily in the area of system description development, evaluation, troubleshooting, proofing and ultimately use.
contributions, corrections, enhancements welcome
you are fooling yourself trying to get something out of gt if you don't spend some time with your favorite pod reading tool going hand-over-hand with the gt application scripts as well as some of the informational documentation. then you will need to review the components that can be included as part of a gt trading system.
here is a reading list to start with
gt_sig-sys-desc.pod
gt_files.pod
backtest.pl
GT/SystemManager.pm
GT/CloseStrategy.pm
GT/PortfolioManager.pm
GT/TradeFilters.pm
GT/OrderFactory.pm
GT/MoneyManagement.pm
if you are intending to write new perl code (or modify existing code) you will need to study Writing_an_Indicator_Cookbook.pdf in GT/Docs, along with some of the existing modules to see how things are done.
fixme looking for user input
gt genius trader
-tbs- to be supplied
(verify) statement needs to be verified
front matter
objective
scope
document abbreviations
1.0 gt trading systems
1.1 gt trading system examples
1.2 trading system components
1.2.1 Systems
1.2.2 OrderFactory
1.2.3 TradeFilter
1.2.4 MoneyManager
1.2.5 CloseStrategy
1.2.6 Broker
2.0 gt sig-indic descriptions
2.1 Indicators
2.2 Signals
3.0 aliases
3.1 global
3.2 object
4.0 gt application scripts
4.1 gt securities analysis tools
4.1.1 graphic.pl
4.1.2 scan.pl ( fixme )
4.1.3 backtest.pl, backtest_many.pl, backtest_multi.pl
4.1.3.1 backtest.pl
4.1.3.2 backtest_many.pl
4.1.3.3 backtest_multi.pl
4.1.4 manage_portfolio.pl
4.1.5 analyze_backtest.pl ( fixme )
4.1.6 anashell.pl ( fixme )
4.1.7 select_combination.pl ( fixme )
4.2 gt development and test tools
4.2.1 display_indicator.pl
4.2.2 display_signal.pl
4.2.3 display_system.pl ( just mine? )
5.0 developing gt systems
5.1 indicators
5.2 signals
5.3 systems
5.4 backtest.pl
5.4.1 backtest.pl examples
5.4.2 you've got the code, use it
5.4.3 self hacking guidance
6.0 new code development
gt trading systems are described in terms of trading system components, signals and indicators.
gt trading systems are formed using orders and positions. orders may not result in a position being entered into. once a position is opened (entered into) the portfolio manager takes control of it.
the portfolio manager includes trading system components tradefilter (verify), orderfactory (verify), closestrategy, moneymanager,
trading systems are formed by combining gt trading system component descriptions together separating each with the vertical bar (pipe) (|) symbol
-tbs-
gt defines these trading system components
each gt trading system requires a systems (yes it is plural) that defines two signals that, when triggered, yield an order to open a position. gt defines two types of positions, long and short, and manages each type separately, hence the need for two separate signals. the first signal is for opening a new long position, the second signal is for opening a new short position.
whether or not an order results in a position being opened depends on a number of other trade system components defined below.
-tbs-
multiple orderfactory are supported per trading system (verify)
restricts the types of trades allowed.
multiple tradefilters are supported per trading system
defines how gt manages investments. affects share quantities allocated to a given order
multiple moneymanagers are supported trading system, each will be evaluated in the sequence defined. the resulting order share quantity will be the quantity that remains after all moneymanagers have been polled.
because each type of position (long or short) is managed separately a closestrategy must also specify two signals that trigger management activity of each position type.
the first signal will generate orders that relate to an existing long position, the second signal will generate orders for an existing short position.
closestrategy, really a misnomer for position management, includes complete and partial position closing as well as reinvesting (adding to) existing positions.
multiple closestrategies are supported per trading system. each is evaluated in the sequence defined, the resulting order, if any will be applied to the appropriate existing position.
verify this verify this verify this
orders that issue from closestrategies are not subject to any of the trading filters (e.g. orderfactory, tradefilter or moneymanager) defined in the trading system but are applied immediately. (i'm thinking that orderfactories and moneymanagers do in fact affect position manager orders)
verify this verify this verify this
a broker is used to apply trading costs to each trade
a broker isn't a full fledged gt trading system component since it cannot be defined as a component in a system description.
specify a broker using the command line option
--broker broker_name argument(s)
or by setting the gt configuration key "Brokers::module" to the desired broker_name and any arguments.
-tbs-
refer to pod file GT/Docs/gt_sig-sys-desc.pod
refer to pod file GT/Docs/gt_sig-sys-desc.pod
here is something i just (re)discovered, it might (or might not) be in the referenced pod, but it is very very significant; if it is ignored you will probably get hard to interpret results and troubleshooting will be painful at best.
signals, just like indicators can have more than a signal output. it's uncommon, but there are a couple. the significance is when you use one of these multi-name signals as an element in a larger sys-sig-indic description to be sure to specify the signal value selector (e.g. the /#) that you want. by default gt will use the first one unless otherwise specified (verify). except, possibly, when the signal is being used as a generic systems? (verify)
as of this writing these are the multi-value gt signals and the value names in order. numbered parameter arguments are not shown:
Signals:Swing:Trend TrendUp, TrendDown Signals:Swing:TrendEnding TrendUpEnding, TrendDownEnding Signals:Systems:MacdDiff MacdDiffHigh, MacdDiffLow Signals:Trend:HilbertChannelBreakout HCBkUp, HCBkDown Signals:Trend:TTT TTTUp, TTTDown
two forms of aliases are supported by gt. both are implemented in package GT::Tools. refer to that packages' pod for the most current implementation details.
refer to pod file GT/Docs/object_aliases.pod
refer to pod file GT/Docs/object_aliases.pod
gt provides perl application scripts in the Scripts directory. these scripts combine the methods and functions found in the gt packages (files in the GT directory hierarchy) in ways to solve, implement or ??? a particular aspect of marketable securities technical analysis, manage a portfolio of stock holdings, and to evaluate gt signals, indicators and systems.
in general terms there are two categories of tools (the scripts): 1) those specifically intended for technical analysis of a security and 2) those that are most useful in terms of gt system development and validation tools. this distinction isn't hard and fast, but more of a general observation, guidance about what to use the scripts for.
-tbs-
see the script pod for all the gory details.
used primarily as a traditional technical analysis charting tool. the application takes directives consisting of signals, including a system component signal set, as well as indicators and creates a traditional technical analysis chart for the security specified.
can be used to visually depict operation of signals, indicators and even the system component that can aid in their troubleshooting.
in order to use graphic.pl you will need to rtfm;
perldoc -t graphic.pl
tool to scan a number of securities with (or against) a number of system component signal sets (can be complete trading systems, but only the system component signal set is evaluated) to see which securities trigger which signals
perldoc -t scan.pl
also review perldoc -t GT/Docs/gt_files.pod for an all in one place description of the file formats used by scan.pl
three different variations on a traditional technical analysis tool that evaluates the performance of a specified trading system over the prior history of a specified securities trading data.
( i thought one of these provided for user specified programmatic trading system parametric variation in an effort to optimize the system for the given security and its trading characteristics. but is don't see its use documented in my fast look-see )
a traditional technical analysis backtesting tool that uses a specified trading system over the prior history of a specified securities trading data and analyzes the trading systems financial performance against a simple buy-and-hold strategy over the same time period.
output includes optional html formatted text or simple ascii text plus optionally a very simple graphic depiction of the trading system compared to the buy-and-hold strategy.
the html output option includes provision to embed the graphic image within the html page.
somebody fixme
somebody fixme
creates, maintains and analyzes your real (or imaginary) securities investment portfolio. a portfolio is a history of all transactions (orders and positions) that have taken place in the portfolio.
takes input in the various forms including command line options, simple text files, beancounter database portfolio table
stored as a gzipped xml file.
by creating and maintaining a gt portfolio your positions can be marked on the graphic.pl charts using this graphic directive:
--add=PortfolioPositions(./bc_pf,show)
to update your bc_pf portfolio (i use bc_pf to indicate it is my beancounter porfolio, you can use any filename)
% manage_portfolio.pl bc_pf file ./trades_`dex`.scott2gt
where the input file ./trades_`dex`.scott2gt has the appropriate format (refer to manage_portfolio.pl pod for details)
the command dex is simply /usr/bin/date '+%d%b%y' | tr '[A-Z]' '[a-z]'
trades are pasted into ./trades_`dex` by manually scraping them off the brokers web page. that file is post-processed with a fairly trivial, but customized perl script that extracts the salient facts and arranges them to suit manage_portfolio.pl.
to get a report on your portfolios' positions:
% manage_portfolio.pl bc_pf report positions
to get a report on your portfolios' history:
% manage_portfolio.pl bc_pf report history
the report options performance and analysis need work if you can help please let the gt devel list know.
somebody fixme
somebody fixme
somebody fixme
-tbs-
used primarily to develop and debug complex indicator descriptions.
also useful to validate new or modified indicator packages
perldoc -t display_indicator.pl
used primarily to develop and debug complex signal descriptions.
also useful to validate new or modified signal packages
perldoc -t display_signal.pl
used primarily to develop and debug complex the two signals generated by the system component description.
perldoc -t display_system.pl
evaluating. proving, troubleshooting gt systems and other complex gt sig-indic descriptions
manually evaluate indicator results with display_indicator.pl.
this application will often show conditions like uninitialized values that can be masked when encountered in graphic.pl or backtest.pl.
compare a modified version of an indicator with an unmodified version
gdiff -u `! display_indicator.pl --sta 1jan09 --end 1may09 I:SnR t` \
`! display_indicator.pl --sta 1jan09 --end 1may09 I:SnR_h t`
the command ! is from 'unix power tools'. it turns it's input into a file and puts that filename in on the command line in its place.
and visually with graphic.pl
notice the same curve is plotted using the modified (I:SnR_h) indicator and the unmodified version with different colors to highlight any anomalous behavior.
graphic.pl --start 2008-10-01 --end 1may09 -timeframe day \ --add=curve'( I:SnR/2, orange)' --add=curve'( I:SnR/1, darkgreen )' \ --add=curve'( I:SnR_h/2, blue)' --add=curve'( I:SnR_h/1, red)' \ T >/tmp/T8mon.png
it is also important to evaluate the result of passing different arguments to a newly developed indicator. in this instance we are checking that the indicator can handle the four arguments (e.g. 10 7 {I:Prices LOW} {I:Prices HIGH})
graphic.pl --start 2008-10-01 --end 1may09 -timeframe day \
--add=curve'( I:SnR_h/2, blue)' \
--add=curve'( I:SnR_h/2 10 7 {I:Prices LOW} {I:Prices HIGH}, green )' \
--add=curve'( I:SnR_h/1, red)' \
--add=curve'( I:SnR_h/1 10 7 {I:Prices LOW} {I:Prices HIGH}, black)' \
T >/tmp/T8mon.png
when developing a new indicator one typically has an expected result in mind, furthermore, the default arguments are normally set to standard technical analysis values. this can often lead to 'code fixes' that yield the expectations when there are more fundamental aspects of the problem being overlooked. to avoid getting trapped in this condition it's important, and often very helpful to expose subtle coding errors and omissions by running test cases with a range of each arguments values. at the very least this will help ensure the code correctly handles arguments other than the hardcoded defaults and has reasonable protection against out-of-range argument values.
-tbs-
manually evaluate signal generation with display_signal.pl and visually with graphic.pl
-tbs-
mark system long/short triggers with graphic.pl
-tbs-
when debugging or trying to evaluate a system try limiting and isolating individual trading system components using the techniques and the trading component specs listed below. the results can be useful to illuminate problems with the system being evaluated and to identify the trading component(s) that need to be adjusted.
-tra LongOnly -tra ShortOnly
-tra OneTrade
define only one well understood position close signal --closestrategy='NeverClose' --closestrategy='LimitPeriodInTheMarket 10'
-display-trades -graph graphs/bt_code.png set the file name (argument to -graph) to whatever makes sense to you
instead of
./backtest.pl -tim day -init 100000 -disp -gr graphs/a_bt_jpm.png \
--start 2002-07-01 --end 2004-01-01 \
-sy 'Generic
{S:G:And
{ S:G:Below {I:Prices LOW} {I:G:Eval { I:SnR_h/1 } + 0.06 } }
{ S:G:Above {I:Prices LOW} { I:SnR_h/1 } }
}
{ S:G:False }' \
-clo 'LimitPeriodInTheMarket 5' \
JPM
use
./backtest.pl -tim day -init 100000 -disp -gr graphs/a_bt_jpm.png \
--start 2002-07-01 --end 2004-01-01 \
-sy 'Generic
{S:G:And
{ S:G:Below {I:Prices LOW} {I:G:Eval { I:SnR_h/1 } + 0.06 } }
{ S:G:Above {I:Prices LOW} { I:SnR_h/1 } }
}
{ S:G:False }' \
-mo 'Basic' \
-tra 'LongOnly' \
-tra 'OneTrade' \
--closestrategy='NeverClose' \
JPM
./backtest.pl --full \
--system="Generic {S:G:CrossOverUp {I:SMA 20} {I:SMA 60}} \
{S:G::CrossOverDown {I:SMA 20} {I:SMA 60}}" \
--closestrategy="Stop::KeepRunUp" \
--moneymanagement="FixedShares" \
--tradefilter="LongOnly" \
--orderfactory="ClosedToClose" \
--broker="InteractiveBrokers" \
CCRT
./backtest.pl --start 2008-01-02 --end 2009-04-30 \
--system="Generic {S:G:CrossOverUp {I:SMA 20} {I:SMA 60}} \
{S:G:CrossOverDown {I:SMA 20} {I:SMA 60}}" \
--closestrategy="Stop::KeepRunUp" \
--moneymanagement="FixedShares" \
--tradefilter="LongOnly" \
--orderfactory="ClosedToClose" \
--broker="InteractiveBrokers" \
CCRT
backtest.pl --timeframe week --start 1apr08 --end 1may09 \ "'`cat buysell-signal | perl -ne 'print $_ unless m/^\s*#.*$|^\s*$/'`'" \ MIDD
backtest.pl --timeframe week --start 1apr08 --end 1may09 \ "`cat buysell-signal`" \ MIDD
backtest.pl -tim day -init 100000 -disp -gr graphs/bt_aapl.png \
-sy 'TFS 15 7' -mo 'Basic' \
-mo 'ShareMultiples 100 2' \
-clo 'Reinvest:InWinners 2' \
-clo 'ChannelBreakout { I:SMA 80 } { I:SMA 80 }' \
-tra LongOnly \
-tra OneTrade \
AAPL
make a copy of the package file 'before' you start hacking on it. then hack away on the copy.
hack in 'use Data::Dumper;' and then sprinkle 'print STDERR Dumper data_object;' statements around and see what comes out. you might want to send this output to a file since it can get copious quickly.
Data::Dumper is recommended because you don't have to be concerned with the data type being printed. it handles scalars (simple variables), arrays, and complex gt data objects with ease.
with package file GT/Indicator/SnR_h.pm under active development add these perl statements towards the top of the file:
our $debug = 0;
$debug = 1; # enable debug outputs
now you can add print statements like these to monitor internal variable without using a debugging perl
if ( $debug > 1 ) {
my $ioff = $self->{'args'}->get_arg_names(2) + 1;
print STDERR "\n";
print STDERR "min1[ $i - $ioff ] $min1\n";
print STDERR "min2[ $i ] $min2\n";
print STDERR "\n";
print STDERR "max1[ $i - $ioff ] max1\n";
print STDERR "max2[ $i ] $max2\n";
print STDERR "\n";
}
in addition, you can easily disable the debugging output by commenting out the $debug = 1; # enable debug outputs line but leave in most of your debug bits to share/discuss or retain for later.
also note that you can easily arrange for various levels of debug output using various the comparison conditional values
if ( $debug ) { ...
if ( $debug >= 2 ) { ...
the more adventurous might do things more like this, but probably a lot smarter and better than this
my $BIT1 = 0x01; my $B1MSK = ~0x01; my $BIT2 = 0x02; my $B2MSK = ~0x02; my $BIT3 = 0x04; my $B3MSK = ~0x04; my $BIT4 = 0x08; my $B4MSK = ~0x08;
my $debug = $BIT2 + $BIT4;
if ( $debug & $B4MSK ) { ...
finally use a bash command like
$ display_indicator.pl -short 35 --start 1jan09 --end 1may09 \ I:SnR_h T > /tmp/snr.out 2>&1
or for csh
% display_indicator.pl -short 35 --start 1jan09 --end 1may09 \ I:SnR_h T >& /tmp/snr.out
with instrumented code you need to keep separate stdout and stderr for scripts that write image files to stdout like graphic.pl.
for this use a bash command like
$ graphic.pl -file example.gconf ENS 1> ENS.png 2> example.out
or for csh
% ( graphic.pl -file example.gconf ENS > ENS.png ) >& example.out
refer to Writing_an_Indicator_Cookbook.pdf (GT/Docs/Writing_an_Indicator_Cookbook.pdf), the seminal work on how to develop new gt indicators and by extension signals.
while it may still seem unnecessary when initially developing a brand new indicator or signal, the statement
return if (! $self->check_dependencies($calc, $i));
or the preferred form
return unless $self->check_dependencies($calc, $i);
is how an indicator or signal gets 'set up'. (if anyone can improve on the terminology here that would be great) without this statement you can expect the indicator or signal will return nothing. what 'set up' means is everything is solved if it isn't already. if the necessary dependencies cannot be solved the indicator or signal will probably fail (without raising an exception, either methinks, and that is unfortunate). if you're interested in how this all works read the code, it's in GT/Dependency.pm, and if you can figure a way to raise an exception in appropriate circumstances please do so and contribute the code.