Enhancement #2572
Improve replay performance for large numbers of channels
Status: | Resolved | Start date: | 06/19/2016 | |
---|---|---|---|---|
Priority: | Normal | Due date: | ||
Assignee: | J. Moringen | % Done: | 100% | |
Category: | Replay | |||
Target version: | Robotics Service Bus - rsb-0.17 |
Description
Replays with large number of channels currently perform badly because the parallel iterator stepping does lots of unnecessary timestamp comparisons.
Associated revisions
Turned parallel iterators into structures in src/view/serialized.lisp
refs #2572
- src/view/serialized.lisp (header): updated copyright
(%iterator): new structure; state of an in-parallel steppable iterator
(%iterator-step): adapted to changed iterator representation
(%iterator-min): likewise
(%iterator-for-forward-step): likewise
(%iterator-for-backward-step): likewise
(make-simple-sequence-iterator serialized): likewise
(iterator-endp serialized serialized-iterator t t): likewise
(iterator-element serialized-iterator serialized-iterator): likewise
Reduced number of iterator comparisons in src/view/serialized.lisp
refs #2572
- src/view/serialized.lisp (%iterator<): renamed %iterator{-min -> <};
compare iterators instead of returning the minimum
(%iterator-for-forward-step): removed; no longer needed
(%iterator-for-backward-step): adapted to use `%iterator<' instead of
`%iterator-min'
(serialized-iterator::sorted): new sorted; heap of sorted iterators
(shared-initialize :after serialized-iterator t): initialize heap of
sorted iterators
(iterator-step serialized serialized-iterator t): use top value of
heap for forward step; reconstruct heap for backward step - cl-rsbag.asd (system cl-rsbag): added system dependency on pileup
Use integer representation of timestamps in src/backend/**/*.lisp
fixes #2572
- src/channel.lisp (header): updated copyright
(channel-timestamps/raw channel): renamed channel-timestamps{>->
/raw}; return uint64 timestamp sequence returned by backend
(channel-timestamps channel): new method; wrap raw timestamp
sequence in `channel-timestamp' instance which translates elements
to `timestamp' instances
(entry channel timestamp): translate timestamp to uint64 before
calling backend
(setf entry t channel timestamp): likewise
(channel-timestamps): new class; transparently translates elements
to `timestamp' instances
(length channel-timestamps): new method; return length
(elt channel-timestamps integer): new method; return translated
element
(channel-items::channel): renamed reader {channel-items
}%channel
(channel-items::timestamps): similar
(shared-initialize :after channel-items t): retrieve raw channel
timestamps
(length channel-items): adapted to renamed reader
(elt channel-items): likewise; translate raw timestamp to
`timestamp' instance - src/view/serialized.lisp (header): updated copyright
(make-serialized-view bag): default comparator is `<'
(make-serialized-view sequence): likewise
(%make-key-function channel): retrieve raw timestamp from channel
(%make-key-function channel-timestamp): new method; for
`channel-timestamps' class
(%make-key-function channel-items): adapted to renamed reader
(serialized::compare): changed initform to `<' - src/backend/util.lisp (header): updated copyright
(uint64->timestamp): new function; moved here from
src/backend/tidelog/util.lisp
(timestamp->uint64): likewise - src/backend/package.lisp (package rsbag.backend): added exported
symbols uint64->timestamp and timestamp->uint64 - src/backend/tidelog/util.lisp (uint64->timestamp): removed; moved to
src/backend/util.lisp
(timestamp->uint64): likewise - src/backend/tidelog/index.lisp (timestamps): changed documentation
string w.r.t. representation of timestamps
(elt timestamps integer): do not translate timestamp
(index-timestamp->offset input-index integer): changed specializer
`timestamp' -> `integer' - src/backend/tidelog/file.lisp (header): updated copyright
(put-entry file integer integer simple-array): changed specializer
`timestamp' -> `integer'; do not translate timestamp - src/backend/elan/types.lisp (header): updated copyright
(timestamp/nanoseconds): new type; timestamp in nanoseconds
(time-slot/cons): changed timestamp/{milliseconds -> nanoseconds} - src/backend/elan/util.lisp (header): updated copyright
(milliseconds->nanoseconds): new function; conversion
(nanoseconds->milliseconds): likewise
(nanoseconds->timestamp): likewise
(timestamp->nanoseconds): likewise
(millisecs->timestamp): removed; no longer needed
(timestamp->millisecs): likewise - src/backend/elan/xml.lisp (xml-> element eql time-slot/cons):
convert time value to nanoseconds
(->xml cons element eql time-slot/cons): corresponding - src/backend/elan/file.lisp (shared-initialize :after file t):
convert base timestamp to nanoseconds when resolving time values
(close file): converter base timestamp from nanoseconds
(get-timestamps file integer): do not translate returned timestamps
(put-entry file integer integer string): changed specializer
`timestamp' -> `integer'; do not translate timestamp - test/mock-backend.lisp (put-entry mock-backend integer integer t):
changed specializer timestamp -> integer
(simple-channels): generate uint64 timestamps - test/builder.lisp (header): updated copyright
(replace-timestamps): new function; helper for
`check-un-build-calls'; replace `timestamp' instance with integers
for comparison
(check-un-build-calls): use `replace-timestamps'
(test rsbag-builder-root::bag/smoke): changed timestamps in mock
data from `timestamp' instances to integers; timestamp replacement
for comparison
(test rsbag-builder-root::channel/smoke): likewise - test/backend/tidelog/file.lisp
(test backend-tidelog-root::repeated-timestamp): changed timestamp
from `timestamp' instance to integer
Mention record and replay efficiency improvements in news.rst
refs #2572
- news.rst (RSBag 0.17): mention more efficient recording and replay
History
#1 Updated by J. Moringen almost 7 years ago
- % Done changed from 0 to 70
Next up: make timestamp comparisons cheaper (even though we have much fewer already)
#2 Updated by J. Moringen over 6 years ago
- Target version changed from rsb-0.15 to rsb-0.16
#3 Updated by J. Moringen about 6 years ago
- Target version changed from rsb-0.16 to rsb-0.17
#4 Updated by J. Moringen over 5 years ago
- Status changed from In Progress to Resolved
- % Done changed from 70 to 100
Applied in changeset rsbag-cl|7dc5fc704035ae61d7fbf45c27be63b46e4a4391.