>
>>From your documents and the forum post you reference, I was unable to
> make the interpretive leap on my own that "InvestorRT's chart is
> wrong because it counts duplicate events as separate ticks". Stupid
> me. The subject of "realtime" IB data is complicated and there is
> much mis-information on the internet that confuses the issue.

Actually what I meant that any *tick* chart using IB feed is wrong, because
IB does not send ticks.
In fact there is no "right" or "wrong" way of counting events from IB feed
because IB did not really reveal ANY documentation covering their internal
tick processing and 0.3 sec snapshot creation and all knowledge we have
is based on guesses, experimentation, manual comparison from different feeds
and alike "investigation".
This is far from scientific approach and it is closer to alchemy.
Despite numerous requests from many developers IB does not really care
to document their feed in detail and developers rely on their own
experimentation.
What I found by analysing TWS API source is that the API sends duplicate
TWO events (tickPrice and tickVolume) if price changes and only ONE event
(tickVolume) if volume changes during 0.3 sec period. Sometimes it can send
three events (tickPrice and two tickVolume events). If this happens during 0.3
second period you don't know whenever it was one, two, ten or hundreds of trades
(ticks).
All you know that price or volume or both changed.
Now what is a "proper" way of counting ???
AmiBroker does NOT count tickPrice and tickVolume as TWO ticks if they come from
single price change.
Other softwares may count it as two.
Which is correct? Neither one or second. Both are wrong.
Why? Because during 0.3 second there could be 1,2, 3, 10, 50 or 100 trades.
No one knows that looking at IB feed stream. During news release (as FED rate
decision)
there can be hundreds of trades flowing in sub-second periods on Eminis,
yet you will see only one IB-tick per 0.3 sec.

> 2. Does anyone know why AB would be aggregating ticks to a time slice
> that is a multiple of five? Are there settings to control this?
If base time interval is set to TICK, AB does not aggregate ticks to any time
slices
other than IB own 0.3 sec minimum snapshot interval.
Please note that if you request *BACKFILL* then IB limits backfill resolution to
1-second bars.

As far as STREAMING data is considered
if base time interval is set to tick, one 0.3 second update (if there is any
change
in price or volume) is counted as one tick. Timestamping does not matter.
Note that current timestamp resolution is 5 second, but it does not mean
any aggregation as there will be usually MANY ticks with same timestamp.
The same would occur with 1 second resolution as there are MANY ticks
during one second and MANY data points would share the same timestamp.
So timestamping does not matter and is irrelevant to tick mode.
The only thing where it matters is creation of sub-minute TIME based bars
and AB currently allows 5-second and 15 second bars.
The resolution of timestamp will be increased to sub-second sometime in the
future
but this is not relevant to tick charts. It is only needed to be able to create
N-second charts where N is any natural number like (2-second, 7-second, 9-second
charts).


> 3. Are AB tick charts and tick data collection dependant on the chart
> refresh interval? In other words would a 10t chart with a chart
> refresh interval of 0 be different than a 10t chart with a refresh
> interval of 3 seconds?
>

No, ticks are collected one by one independently without any reference to chart
refresh interval.
Making chart refresh interval longer will simply update chart less often so for
example
you will see 10 new ticks at once after 3 seconds on actively traded symbol.


Best regards,
Tomasz Janeczko
amibroker.com

----- Original Message -----
From: "dbwyatt_1999" <dbw451@...>
To: <amibroker@yahoogroups.com>
Sent: Wednesday, December 26, 2007 4:34 PM
Subject: [amibroker] Re: AB tick bar analysis


>
> I do not know whether IRT's "chart is wrong" is a fact or your
> deduction based on the fact that IRT tick charts are different than
> AB tick charts. I cannot assume it's a fact since you also said that
> IB data is not real tick data (probably 1000s of times), and it's up
> to each software developer to interpret the IB data stream to create
> tick charts. This explains to me why there is a difference between
> IRT and AB charts and answers my first question in my initial post.
> I do not have enough knowledge nor even care to evaluate whether
> IRT's implementation interpretation is wrong or AB's implementation
> interpretation is right.
>
> The docs and forum posts do not explain to me why my AB tick chart
> bars all have timestamps that are multiples of 5 seconds. John's
> post gave me an explanation, but you say "No, it does not make ANY
> sense" and accuse me again of not reading docs. Since tick mode does
> not use timestamps, then I do not understand why all my AB 10t bars
> have timestamps that are multiples of 5 seconds. I either missed
> something in the documentation or I'm unable to make an interpretive
> leap between the documentation and my question.
>
> I initially thought maybe I have something in AB setup incorrectly,
> so that's why I asked the group a few questions in my initial post.
> For reference, here are my initial questions again:
>
> 1. Does anyone know why AB would produce less tick bars than
> Investor/RT using the same data feed? ANSWERED - thank you.
> 2. Does anyone know why AB would be aggregating ticks to a time slice
> that is a multiple of five? Are there settings to control this?
> 3. Are AB tick charts and tick data collection dependant on the chart
> refresh interval? In other words would a 10t chart with a chart
> refresh interval of 0 be different than a 10t chart with a refresh
> interval of 3 seconds?
>
> Regards,
>
> David
>
>
>
>
> --- In amibroker@yahoogroups.com, "Tomasz Janeczko" <groups@...>
> wrote:
>>
>> No, it does not make ANY sense. You guys do not read docs.
>> Timestamping has nothing to do with tick mode. Tick mode and any n-
> tick chart does NOT use timestamps at all.
>> They are completely irrelevant. The reason Please read the User's
> Guide
>> http://www.amibroker.com/guide/h_ib.html
>>
>> It clearly explains that Interactive Brokers is NOT TICK FEED!
>> 4. IB TWS streaming data are NOT tick-by-tick, but rather 0.2-0.3
> second snapshots, read this for details
>> http://www.interactivebrokers.com/cgi-bin/discus/board-auth.pl?
> file=/2/37364.html
>>
>> See also my other reply
> http://finance.groups.yahoo.com/group/amibroker/message/118232
>> for details and explanation why InvestorRT chart is wrong because
>> it counts duplicate events as separate ticks. Timestamps would NOT
> change that.
>>
>> For true tick charts you need to use tick-by-tick feed which is
> eSignal or IQFeed.
>>
>> Best regards,
>> Tomasz Janeczko
>> amibroker.com
>> ----- Original Message -----
>> From: "dbwyatt_1999" <dbw451@...>
>> To: <amibroker@yahoogroups.com>
>> Sent: Wednesday, December 26, 2007 4:23 AM
>> Subject: [amibroker] Re: AB tick bar analysis
>>
>>
>> > Thanks John, that makes sense. Assuming the realtime feed in
>> > AmiBroker Pro is the same 5 second timestamp scheme for
> aggregating
>> > realtime tick data as the ASCII tick data import process, then
> that
>> > explains the 5 second timestamps I'm seeing on my 10t bars. If
> that
>> > is the case, then I guess AB really cannot handle small timeframe
>> > tick charts used by scalpers. The tick bars created using a
> minimum
>> > resolution of 5 second aggregated data will create inaccurate
> small
>> > timeframe tick bars. This explains the tick chart differences
> I'm
>> > seeing when compared the other charting package.
>> >
>> > Also, thanks to David C. for previously posting his tick data
>> > importing explanation.
>> >
>> > Regards,
>> >
>> > David
>> >
>> >
>> > --- In amibroker@yahoogroups.com, "john_dxd_smith"
>> > <john_dxd_smith@> wrote:
>> >>
>> >>
>> >> Let's hope for 1-second resolution.
>> >>
>> >> ------------------------------------------------------------
>> >> http://finance.groups.yahoo.com/group/amibroker/message/97297
>> >>
>> >> From: "David Clayworth" <dclayw@>
>> >> Date: Thu May 25, 2006 8:28 pm
>> >> Subject: RE: [amibroker] Time stamp - ASCII Tick Data Import
>> >>
>> >> I asked this question of AB support some time ago. This is the
>> >> response I got:
>> >>
>> >> "This is due to the fact that AmiBroker uses its own 32-bit
>> > datestamp
>> >> that has resolution of 5 seconds but spans years of 1900-2155.
>> >> Standard C-language 32-bit timestamp that has resolution of 1-
>> > second is
>> >> limited to dates from 1970-2023 and therefore can not be used
>> >> (year range is to narrow).
>> >>
>> >> However this does not really affect anything as all ticks are
>> > imported,
>> >> order of ticks is maintaned and all charts look correct and all
>> >> analysis is accurate."
>> >>
>> >> Note that I only have the standard version of AB so I am unsure
> if
>> > it
>> >> is different for the professional version.
>> >>
>> >> ------------------------------------------------------------
>> >>
>> >> --- In amibroker@yahoogroups.com, "dbwyatt_1999" <dbw451@> wrote:
>> >> >
>> >> > Thanks Tomasz. I am not using the ASCII importer. I'm using
> the
>> > IB
>> >> > plugin to receive data from IB TWS in realtime. For my
> analysis,
>> > I'm
>> >> > running three programs on the same computer: IB TWS, AmiBroker
>> > Pro,
>> >> > and Investor/RT. AB and IRT receive the same data stream from
> IB
>> > TWS
>> >> > in real-time at the same time. I took a detailed look at the
>> > price
>> >> > bars created on a 10 tick chart for a 10 minute period in both
> AB
>> > and
>> >> > IRT. AB created 14 10t bars and IRT created 22 10t bars for
> the
>> > same
>> >> > 10 minute period using the exact same IB data feed. Total CPU
>> >> > utilization stays under 10% for the entire test period. The
>> > total
>> >> > volumes for the 10 minute period are approximately the same
> for
>> > both
>> >> > AB and IRT 10t charts. The difference in the number of 10t
> bars
>> >> > created between the two charting packages is what concerns
> me. I
>> >> > suspect that I do not have something setup correctly with AB
>> > because
>> >> > all the AB tick bars have a timestamp that are a multiple of 5
>> >> > seconds.
>> >> >
>> >> > Is there some AB setting that is forcing the timestamps to be
>> >> > multiples of 5 seconds for a 10 tick chart created in realtime
>> > from
>> >> > the IB TWS data feed?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > David
>> >> >
>> >> >
>> >> > --- In amibroker@yahoogroups.com, "Tomasz Janeczko" <groups@>
>> >> > wrote:
>> >> > >
>> >> > > Hello,
>> >> > >
>> >> > > If you are using ASCII importer and want to import tick data
>> >> > > you must use special mode, because ticks are special in that
>> > meaning
>> >> > > that ticks may have duplicate timestamps. For that reason
> you
>> > need
>> >> > > to specify
>> >> > >
>> >> > > $TICKMODE 1
>> >> > >
>> >> > > in the ASCII importer definition.
>> >> > >
>> >> > > See the documentation of ASCII importer - it has very
> detailed
>> > docs
>> >> > > in the User's Guide.
>> >> > >
>> >> > >
>> >> > > Best regards,
>> >> > > Tomasz Janeczko
>> >> > > amibroker.com
>> >> > > ----- Original Message -----
>> >> > > From: "dbwyatt_1999" <dbw451@>
>> >> > > To: <amibroker@yahoogroups.com>
>> >> > > Sent: Monday, December 24, 2007 10:44 PM
>> >> > > Subject: [amibroker] Re: AB tick bar analysis
>> >> > >
>> >> > >
>> >> > > > Thanks Barry, you do have a point that IB data does not
> have
>> > a
>> >> > > > timestamp which would make comparing different captures
>> >> > difficult.
>> >> > > > The Scottrade DJI tick data looks like it's 2 second
>> > snapshots.
>> >> > > >
>> >> > > > I'm not so much concerned about having exact timestamps,
> but
>> > the
>> >> > > > number of tick bars for a time period for the exact same
>> > datafeed
>> >> > > > should be very close regardless of the timestamps. I mean
> 10
>> >> > ticks
>> >> > > > is 10 trades, whether that took 0.23 seconds or 35
> seconds.
>> > The
>> >> > > > number of trades (i.e. number of bars) should be similar
>> > between
>> >> > > > different charting packages. They may not match up
> depending
>> > on
>> >> > > > where the tick counting starts, but overall the total
> number
>> > of
>> >> > bars
>> >> > > > should be very similar between all charts with the same
> tick
>> >> > > > timeframe. If 1000 trades happened in a given time
> period,
>> > then
>> >> > on a
>> >> > > > 10 tick chart there should be close to 100 bars.
>> >> > > >
>> >> > > > Regards,
>> >> > > >
>> >> > > > David
>> >> > > >
>> >> > > > --- In amibroker@yahoogroups.com, "Barry Scarborough"
>> >> > <razzbarry@>
>> >> > > > wrote:
>> >> > > >>
>> >> > > >> I think the reason is that IB does not provide a time
> stamp
>> > with
>> >> > > > the
>> >> > > >> data it sends and AB has to use the computer clock. If
> that
>> > is
>> >> > the
>> >> > > >> case the data never will be in sync. That is what I have
>> >> > noticed.
>> >> > > > My
>> >> > > >> last bar close on one minute intraday charts is never the
>> > same
>> >> > as
>> >> > > >> Scottrade or the EOD database from Yahoo. IB sends data
>> > before
>> >> > the
>> >> > > >> open and after the close and that can get shifted into
> the
>> > AB
>> >> > > > charts
>> >> > > >> depending on your computer clock. Try looking at a few
> bars
>> >> > either
>> >> > > >> way and see if the open and close is the same on them.
> But
>> > then
>> >> > I
>> >> > > >> guess when you are looking at really short term data even
>> >> > different
>> >> > > >> data feeds may not be in sync.
>> >> > > >>
>> >> > > >> I dumped the last 100 bars of DJI tick data from
> Scottrade.
>> > If
>> >> > you
>> >> > > >> have access to IB INDU tick data you could compare them.
> I
>> > can
>> >> > send
>> >> > > >> the file if that would help.
>> >> > > >>
>> >> > > >> Barry
>> >> > > >>
>> >> >
>> >>
>> >
>> >
>> >
>> >
>> > Please note that this group is for discussion between users only.
>> >
>> > To get support from AmiBroker please send an e-mail directly to
>> > SUPPORT {at} amibroker.com
>> >
>> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> > http://www.amibroker.com/devlog/
>> >
>> > For other support material please check also:
>> > http://www.amibroker.com/support.html
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>> >
>>
>
>
>
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
> Yahoo! Groups Links
>
>
>
>
>