Re: [GT] Q re max-loaded-items

["Robert A. Schmied" <ras AT acm.org>]

was for,Weigert, Thomas wrote:
> Does anybody know why we have two options that limit the number of
> periods that we evaluate:

well, there are sometimes two other options also, --start and --end.
oops three, lest we forget --last-record (display_indicator.pl) although
technically these only limit the number of periods for which evaluation
is displayed.

> --max-loaded-items

this is supposed to limit the number of items retrieved from the
database, if the driver honors the request and the script actually
requests only max-loaded-items.

without this argument a db driver will retrieve every available tuple.

i believe it is the $limit variable in some db drivers


> --nb-items

this is supposed to be a shortcut instead of passing two dates
(start and end) to a script. i think graphic.pl with greg margos'
patches is the only script that gets it correct.

the above are what i believe these arguments are intended for. as
with many of the user facing elements they may or may not work as
documented, or at all, or differently depending on timeframe, and
from script to script.

> The documentation for nb-items in display_indicator states that it
> "controls how many database records are loaded".
>
> There is no documentation for max-loaded-items but in graphic.pl (the
> only place it appears) it is used in the same way that nb-items is used
> in display_indicator.
>
> nb-items in graphic.pl is used to determine how many periods to plot
> albeit the analysis might use more data.

i believe all this confusion came about in the '04 time frame. at one
point nb-items was used to limit db retrieval, but it was soon changed
in favor of max-loaded-items (without documentation). then others came
along tried to figure out what nb-items was for, since it was still
being parsed (but likely not used) and deciding it must be a way
to shortcut either start or end (or both). in most cases, documentation
was usually left to some one who didn't write the code.

> As we all know, the options in the various scripts are very
> inconsistent. So in striving for some sanity here, can anybody give some
> feedback as to
>
> 1. Would it be ok to have the number of periods plotted in graphics.pl
> be the same as the number of data loaded? In other words, is there a
> need to limit the periods plotted to a smaller (larger does not make
> sense, albeit that is not tested in the code) range than the data
> loaded?

limiting the tuples retrieved from the db is a two edged sword, if
your limit is smaller than the minimum needed for the worst case
ta study the results will be invalid or simply empty, with naturally
no explanation as to why. usually the limit is a large number, say
750-1000 (about 3-5 years of day data).

rather than make this a user controlled option couldn't a method be
created that would determine the maximum data required to satisfy the
worst case technical analysis study in the analysis and set that limit?

based on the recent indicator dependency work i think determining that
number before you get the data is not possible.

another thing that you need to understand (or at least to take into
account) -- the current db interface methods do not have the means to
deal with dates -- meaning they do a fairly brute force query. the
only means to throttle the query size is via the limit variable.

so when a user tries to look at say the offspring of the 1984 att
breakup from 1984 to 1994 your limit needs to be large enough to
get all the data from now to 1984 even though most of it will not
be used.

at least these are my understandings (general as they are) of how
the underlying code works.

> 2. If there is a good reason to differentiate these two, why does the
> same argument not apply to display_indicator.pl?

if you're asking why display_indicator.pl operates differently my guess
is that it was more of a test/development tool. but users found it was
good for debugging indicator specifications for scan.pl and graphic.pl.

if you're asking why display_indicator.pl doesn't have the same set
of arguments as graphic.pl kind of the same answer, but with a slightly
different spin -- there wasn't as much need for it given this scripts
usage pattern and user set.

> Please advise.
>
> Thanks, Th.