João, Here is a simple example: 1. We compute the monthly close of BRPIX, for example (note that in the data set used below, the last day was 2008-07-03): ./display_indicator.pl --start=2007-10-1 timeframe=month I:Prices BRPIX CLOSE Prices[CLOSE] [2007/10] = 24.1000 Prices[CLOSE] [2007/11] = 25.2000 Prices[CLOSE] [2007/12] = 25.5200 Prices[CLOSE] [2008/01] = 27.2300 Prices[CLOSE] [2008/02] = 28.1000 Prices[CLOSE] [2008/03] = 28.2000 Prices[CLOSE] [2008/04] = 26.8500 Prices[CLOSE] [2008/05] = 26.5500 Prices[CLOSE] [2008/06] = 28.9500 Prices[CLOSE] [2008/07] = 29.3400 2. Now create the BRPIX data set in weekly timeframe, and compute the monthly close again: ./display_indicator.pl --start=2007-10-1 --timeframe=month I:Prices BRPIXweekly CLOSE Prices[CLOSE] [2007/10] = 24.7200 Prices[CLOSE] [2007/11] = 25.2000 Prices[CLOSE] [2007/12] = 26.5500 Prices[CLOSE] [2008/01] = 26.8500 Prices[CLOSE] [2008/02] = 28.1000 Prices[CLOSE] [2008/03] = 27.1600 Prices[CLOSE] [2008/04] = 26.3000 Prices[CLOSE] [2008/05] = 26.5500 Prices[CLOSE] [2008/06] = 29.3400 Notice the difference? Most months are off, due to the difference in which month days are being counted as belonging. For example, the close for Jun in the first set does not include 3 Jul, which is the close of Jun in the weekly dataset. Of course, that the data does not agree is unavoidable. But what I think is not good is that the second set includes the CLOSE of days not in this month. Note that all the OPEN are in the correct month, of course. However, most analysis is done on CLOSE, so I think it would be better to ensure that the CLOSE is in the correct month. Th. I attach the weekly BRPIX for your entertainment... > -----Original Message----- > From: João Costa [mailto:joaocosta AT zonalivre.org] > Sent: Wednesday, July 09, 2008 3:24 PM > To: devel AT geniustrader.org > Subject: Re: [GT] Opinions on date conversion sought > > I think i kinda see the problem Thomas raised in the example. > > In mapping weekly into monthly timeframes, you get inconsistent > results, because data from a weekly period overlaps two different > monthly periods, so you're discarding the last weekly period for the > current month calculation. > > Also, that bit where you say: > >date computed for week 2008-27 is considered to be part of June > (2008/06), > >but some of the information required for 2008-27 (the days in July > that are needed to > >get the closing price for 2008-27) have not even occurred then > > This can happen in any timeframe, but the problem here is that a > period which has not finished yet is being calculated. > The same thing would happen if for eg: converting from daily to weekly > on a Wednesday, or from minute to hour at 30minutes past. > > In this case I think it's not a problem of the conversion routines, > it's that we're calculating a period which is still unfinished. > > And following Robert's tip, some examples can help clarify things. > > > > On Wed, Jul 9, 2008 at 8:52 PM, Robert A. Schmied <ras AT acm.org> wrote: > > Weigert, Thomas wrote: > >> > >> João, > >> > >> I ran into this situation in the following way: I had been doing > some > >> performance evaluation of indicators using timeframe 'day'. As these > >> took a long time to run, I used timeframe 'week' but got some weird > >> results. (I produce summary tables for backtesting which shows the > >> performance of a portfolio on a monthly basis over the years > >> invested.) > >> > > > > aloha thomas > > > > since you didn't share the command line directives you used we cannot > > independently empirically evaluate your observations and your > conclusions. > > > > but i too have experienced issues when converting dates, meaning year > > month day to year week-of-year. however, this only ever occurred > > when i, not the gt system, did the conversion. by that i mean i > mentally > > converted 30 june 2008 (or 31 dec 2005) to week of year and usually > got > > it different from what gt would get. > > > > that said i have yet to find conversion inconsistencies in the > internal > > date conversions that gt had implemented via GT::DateTime and > Date::Calc. > > it seems consistent in it's handling of these year end date oddities > and > > how one counts weeks of the year. > > see date and calendar standards for more. the issue isn't simple, > there > > are multiple was to account for the extra days in a year. > > > > my solution was to use dates *always* formatted year-mon-day and then > if > > the time frame was set to week or year have the code do the > conversion. > > > > my perl implementation to do that was in a version of scan.pl posted > a > > while back (and may still be mostly in the trunk version). the key > > elements: validate user date inputs and then convert them according > > to the timeframe setting. i relied on Date::Manip for the former and > > GT::DateTime (and Date::Calc) for the latter to ensure date > consistency. > > > > the other gt scripts would benefit from similar change. > > > > by allowing multiple date string formats that are intended to align > with > > a timeframe there is no way that i can envision to ensure the user's > > notion of week 53, 0 or 1 is consistent with that of gts'. best way > to > > handle it (best meaning least risk, change, work, ...) is to > encourage > > users to use yyyy-mm-dd and do the time frame conversions internally. > > do this throughly, with good error feedback and users will, well, > use. > > > > > > just my few cents on dates yet again ... > > > > ras > > > > > > > >> Tracking the difference in behavior down I found the issue mentioned > >> earlier, namely that when mapping a weekly analysis into months we > >> end up with situations that, e.g., date computed for week 2008-27 is > >> considered to be part of June (2008/06), but some of the information > >> required for 2008-27 (the days in July that are needed to get the > >> closing price for 2008-27) have not even occurred then. > >> > >> In such situations it will be better to consider the week to belong > >> to July, as then we deal only with realistic situations. > >> > >> Whether this is an issue or not depends on the type of analysis you > >> are doing, and the types of summary tables you produce, if any. > >> > >> I am not sure whether it will always be better to consider dates > >> based on ending dates in conversion, or whether this is only better > >> in the situation I ran into. > >> > >> In the meantime, I have implemented DateTime routines that allow > >> conversions between timeframes based on end dates, so that users > have > >> the option to rely on those when needed... > >> > >> Cheers, Th. > >> > >> > >>> -----Original Message----- From: João Costa > >>> [mailto:joaocosta AT zonalivre.org] Sent: Tuesday, July 08, 2008 7:42 > >>> PM To: devel AT geniustrader.org Subject: Re: [GT] Opinions on date > >>> conversion sought > >>> > >>> For some reason I stopped receiving messages from the mailing list > >>> for the last 3 days, sorry that i keep breaking the topic index > >>> that keeps messages neatly organized by topic. > >>> > >>> On the particular issue of date conversion as you exposed it, just > some > >>> questions: > >>> > >>> - Is there any particular disadvantage on the way GT currently > >>> handles this ? - How did you come across this ? What's the > >>> motivation to change this ? > >>> > >>> > >>> From what I could understand from your message, I get the > >>> impression that doing this in different ways will yield different > >>> results, but it seems neither way is better/worse than the other, > >>> it's just different. Is this correct ? > >> > >> > >> > > > >
Attachment:
wkdata.csv
Description: wkdata.csv