[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[GT] Re: package Finance::GeniusTrader::DateTime::Day;

Josef Strobel wrote:

Hi all,
I tried to load a time series where the start date is before 1960-12-31, but I always get the following error: 

my time series:

1960-01-25,56.78,56.78,56.78,56.78,2790000,56.78
1960-01-22,57.38,57.38,57.38,57.38,2690000,57.38
1960-01-21,57.21,57.21,57.21,57.21,2700000,57.21
1960-01-20,57.07,57.07,57.07,57.07,2720000,57.07
1960-01-19,57.27,57.27,57.27,57.27,3100000,57.27
1960-01-18,57.89,57.89,57.89,57.89,3020000,57.89
1960-01-15,58.38,58.38,58.38,58.38,3400000,58.38
1960-01-14,58.40,58.40,58.40,58.40,3560000,58.40
1960-01-13,58.08,58.08,58.08,58.08,3470000,58.08
1960-01-12,58.41,58.41,58.41,58.41,3760000,58.41
1960-01-11,58.77,58.77,58.77,58.77,3470000,58.77
1960-01-08,59.50,59.50,59.50,59.50,3290000,59.50
1960-01-07,59.69,59.69,59.69,59.69,3310000,59.69
1960-01-06,60.13,60.13,60.13,60.13,3730000,60.13
1960-01-05,60.39,60.39,60.39,60.39,3710000,60.39
1960-01-04,59.91,59.91,59.91,59.91,3990000,59.91
1959-12-31,59.89,59.89,59.89,59.89,3810000,59.89


$HOME/eclipse-workspace/geniustrader/script/graphic.pl --file $HOME/eclipse-
workspace/geniustrader/own-scripts/graphic/graphic_examples.conf ^GSPC > foobar.png
Day too big - 32871 > 24853
Cannot handle date (0, 0, 0, 31, 11, 2059) at /home/josef/cpan/Finance/GeniusTrader/DateTime/Day.pm line 24
When I remove the last line in my time series it worked, so I looked in Finance::GeniusTrader::DateTime::Day and took a look to the function timelocal from Time::Local.
I wrote a small script time-date-test.pl. It should test all dates from 1900-01-01 to 2039-01-01, but it fails from 1938-01-17. 

#!/usr/bin/perl

use warnings;
use strict;
use Time::Local;
use Date::Calc qw( Delta_Days Add_Delta_Days );

# sub from package Finance::GeniusTrader::DateTime::Day;
sub map_date_to_time {
    my ($date) = @_;
    my ($y, $m, $d) = split /-/, $date;
                ($d) = split / /, $d;
                print $d . "," . ($m - 1) . "," . ($y - 1900) . "\n";
    print timelocal(0, 0, 0, $d, $m - 1, $y - 1900) . "\n";
}



my @start = (1900,1,1);
my @stop  = (2039,01,01);

my $j = Delta_Days(@start,@stop);

for (my $i = 0; $i <= $j; $i++ )
{
    my ($year,$month,$day) = Add_Delta_Days(@start,$i);
    printf("%4d/%02d/%02d\n", $year,$month,$day);
    &map_date_to_time("$year-$month-$day");
}


So what is timelocal doing?


Best regards
Josef



josef et al

change history can sometimes be helpful to avoid repeating the same mistakes again
and again.

way way way back (24may2005) the methods 'map_date_to_time' called from gt::datetime
used to use posix::mktime(...) but that was changed when joao costa found that
 "Posix::mktime returns GMT, while localtime returns, well, local time"
and that could (did, in fact) lead to conversion errors. so the calls
 Posix::mktime(...)
were changed to
timelocal(...)

when one uses only an end-of-day (eod) database and doesn't use sub eod timeframes
this issue isn't a concern (at least i've not encountered it and have not reverted
to timelocal(...)).

i suspect the correct place to implement whatever fix is needed is where the
date-time value is first encountered. whether this is in the gt::datetime package
or some place else isn't something i've looked at yet.


as to the issue of dates outside of the range 1970 .. 2038, is that really much
of a concern? from strictly an investing investors' viewpoint something that happened
over a few years ago isn't likely too relevant now, certainly 30 years of history
is adequate for investing purposes, but maybe not for investing research. as for
the upper end, it's still far enough in the future so it isn't going to be a
problem any time soon, and by then the solution will likely be trivial.

as an aside, i note for users of flat file based prices data that gt doesn't
provide a way to limit the data read from these files (via the $max_loaded_items
argument and database $limit variable), but instead will load the entire file.
as these files increase in size (with lots of unused data) the result is slower
load times, longer processing times and higher memory requirements than would
be case for the same data set loaded from one of the sql based database engines.


aloha

ras