[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [GT] Input validation in Tools.pm
João Costa wrote:
Hello Robert, good to be back and glad to see you still around responding to
issues :)
This is not specific to genericdbi, but rather to the configuration file
(~/.gt/options) DB::timeframes_available
directive. (Used in Tools::get_timeframe_data )
Here is an example of how you would set this directive:
GT::timeframes_available day
or if you have intraday data:
GT::timeframes_available day,hour
The purpose of that directive is to tell GT which timeframes are available
on your database so that it can do timeframe conversion and allow you to
request timeframes which are not available in the database if they can be
derived from those which are without having to try every possible timeframe.
The problem I came across was this:
GT::timeframes_available 30sec,hour,day
However, GT doesn't recognize the 30sec timeframe (which is available in my
database), so once I execute something simple such as :
display_indicator.pl --timeframe=1min I:Prices EURUSD
It fails with cryptic warning messages giving no clue to the user as to what
the problem is.
The purporse of this patch is to kill the script with a meaningful error
message when this situation occurs :)
On Wed, Jul 2, 2008 at 6:49 PM, Robert A. Schmied <ras
AT
acm.org> wrote:
João Costa wrote:
I was just playing with a new installation of GT. One of the issues I came
across is the available_timeframes directive supported by genericdbi.pm.
I
was putting invalid timeframe names in that and was getting some cryptic
warnings as a result.
The attached patch validates that situation.
------------------------------------------------------------------------
--- /tmp/Tools-HEAD.pm 2008-07-01 17:39:28.000000000 +0100
+++ GT/Tools.pm 2008-07-01 15:24:33.000000000 +0100
@@ -435,8 +435,13 @@
die("Parameter \$db not set in get_timeframe_data") if (!defined($db));
if (defined($available_timeframes)) {
- foreach (split(',', $available_timeframes)) {
- push @tf, GT::DateTime::name_to_timeframe($_);
+ foreach my $tf_name (split(',', $available_timeframes)) {
+ my $tf_code = GT::DateTime::name_to_timeframe($tf_name);
+ push @tf, $tf_code;
+ if (!defined($tf_code)) {
+ my $tfs = join("\n\t",
map(GT::DateTime::name_of_timeframe($_), GT::DateTime::list_of_timeframe));
+ die("Invalid timeframe name in available_timeframes
configuration item: $tf_name\n\nValid timeframes are: \n\t" . $tfs .
"\n\n");
+ }
}
@tf = sort(@tf);
} else {
aloha joao
looks like that code came from a change on Date: 2006-11-20 01:25:48 +0100
(Mon, 20 Nov 2006)
at revision 513. you seem to be in the group that uses genericdbi.pm; have
you done enough
testing to be convinced it's a complete fix? my tests are inconclusive,
meaning i see no
difference with and without it as i discuss below.
can you share the command line, and other configured data that lead you
to this problem/solution? i'd like to see if it can be duplicated using
beancounter db and bean.pm module as well as a text file db and Text.pm.
without the change with beancounter and bean.pm:
ras [ 3908 ] % display_indicator.pl -ti 5min --sta 2008-06-23 I:Prices
ENS Indicator I:Prices has 1 value ... all values selected
I:Prices/1 <=> Prices[]
Can't call method "timeframe" on an undefined value at ../GT/Calculator.pm
line 70.
interestingly i get 'results' using sample text db and Text.pm
ras [ 3956 ] % display_indicator.pl -ti 5min I:Prices 13000
Indicator I:Prices has 1 value ... all values selected
I:Prices/1 <=> Prices[]
timeframe 5min, time periods 2224 .. 2424
Calculating indicator ...
display_indicator.pl: interval: 2224 .. 2424
Prices[] [2001-12-14 00:00:00] = 18.7400
Prices[] [2001-12-17 00:00:00] = 19.7800
.
.
.
with the change (a bit rearranged) notice the $_ in join should be $tf_name
i think
(but my version didn't create a separate variable for tf_name) i got same
output.
is there some recent change that uses method get_timeframe_data that i've
missed or is it specific to genericdbi.pm?
do you intend to commit or would you prefer i do it? if the latter,
i would be inclined to re-format the error message code some in an
effort to keep line lengths about 80 chars.
ras
here's the code fragment i used in my test+eval version
if ( ! defined($available_timeframes) ) {
@tf = GT::DateTime::list_of_timeframe();
} else {
foreach (split(',', $available_timeframes)) {
my $tf_code = GT::DateTime::name_to_timeframe($_);
if (!defined($tf_code)) {
my $tfs = join("\n\t",
map(GT::DateTime::name_of_timeframe($_),
GT::DateTime::list_of_timeframe()));
my $msg = "Invalid timeframe name in available_timeframes"
. " configuration item: $_\n\nValid timeframes are: \n\t"
. "$tfs\n\n";
die "$msg";
}
push @tf, $tf_code;
}
@tf = sort(@tf);
}
joao
there must be something i'm missing because as i indicated the change
made no difference for bean.pm or Text.pm with the following line
in my options file:
DB::timeframes_available day,week,month,year
when i requested timeframes of hour or 5min
i'm wondering (but haven't checked) if some of the recent changes haven't
been factored into my working file set or maybe a prior change that never
got committed ... still should search the devel archive ...
humm i do see a call to get_timeframe_data in display_indicator,pl ...
ahha! the issue (for me) is later in the get_timeframe_data subroutine where @tf
is pulled apart and $_ is always less than the available timeframes.
once all the @tf elements are checked and none are greater than $timeframe
$q will remain undefined.
ras
here's all of sub GT::Tools::get_timeframe_data
sub get_timeframe_data {
my ($code, $timeframe, $db, $max_loaded_items) = @_;
#WAR# WARN "Fetching all available data, because the max_loaded_items parameter was not set." unless(defined($max_loaded_items));
$max_loaded_items = -1 unless(defined($max_loaded_items));
my @tf;
my $available_timeframes = GT::Conf::get('DB::timeframes_available');
my $q;
die("Max loaded items cannot be zero") if ($max_loaded_items==0);
die("Parameter \$code not set in get_timeframe_data") if (!defined($code));
die("Parameter \$timeframe not set in get_timeframe_data") if (!defined($timeframe));
die("Parameter \$db not set in get_timeframe_data") if (!defined($db));
if ( ! defined($available_timeframes) ) {
@tf = GT::DateTime::list_of_timeframe();
} else {
foreach (split(',', $available_timeframes)) {
my $tf_code = GT::DateTime::name_to_timeframe($_);
if (!defined($tf_code)) {
my $tfs = join("\n\t", map(GT::DateTime::name_of_timeframe($_),
GT::DateTime::list_of_timeframe()));
my $msg = "Invalid timeframe name in available_timeframes"
. " configuration item: $_\n\nValid timeframes are: \n\t"
. "$tfs\n\n";
die "$msg";
}
push @tf, $tf_code;
}
@tf = sort(@tf);
}
#ERR# ERROR "Invalid db argument in get_timeframe_data" unless ( ref($db) =~ /GT::DB/);
#ERR# ERROR "Timeframe parameter not set in get_timeframe_data." unless(defined($timeframe));
#$max_loaded_items = -1 unless(defined($max_loaded_items));
foreach(reverse(@tf)) {
next if ($_ > $timeframe);
$q = $db->get_last_prices($code, $max_loaded_items, $_);
last if ($q->count > 0);
}
if ( ! defined $q ) {
my $req_tf_name = GT::DateTime::name_of_timeframe( $timeframe );
my $ava_tf_list = join( " ", split(',', $available_timeframes) );
my $msg = "the requested timeframe \"$req_tf_name\""
. " is not in the list of available timeframes\n"
. " \"$ava_tf_list\"\n";
die "$msg";
}
warn ("No data is available to complete the request for $code") if ($q && $q->count == 0);
my $calc = GT::Calculator->new($q);
$calc->set_code($code);
if ($q->timeframe != $timeframe) {
$calc->set_current_timeframe($timeframe);
$q = $calc->prices;
}
return ($q, $calc);
}