Tasks #847
Make buffer write back configurable
| Status: | Resolved | Start date: | 02/02/2012 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | J. Moringen | % Done: | 100% | |
| Category: | - | |||
| Target version: | Robotics Service Bus - rsb-0.7 |
Description
The current implementation (write back chunk buffers every 100 entries) requires too much memory in use-cases with large payloads.
This problem can be mitigated by offering strategies such as "write back once N bytes have accumulated".
Related issues
Associated revisions
Added flush strategies in src/backend/flush-strategies.lisp
refs #847, #846
refs #847, #846
- src/protocol.lisp (open-bag): added flush-strategy keyword parameter
(open-bag stream): accept flush-strategy keyword argument; pass to
backend unless direction is input - src/backend/protocol.lisp (buffer-property): new generic function;
return specified property of a buffer
(flush): new generic function; flush a buffer to background storage
(backend-flush-strategy): new generic function; return the flush
strategy associated with a backend
(flush?) new generic function; return non-nil when a buffer should
be flushed
(define-findable-class-family flush-strategy): new findable class
family; consists of flush strategy classes
(make-flush-strategy): new generic function; make and return a flush
strategy according to a specification
(make-flush-strategy symbol): new method; construct from keyword or
class name
(make-flush-strategy class): new method; construct from given class - src/backend/buffering-writer-mixin.lisp
(buffering-writer-mixin::flush?-func): removed slot
(buffering-writer-mixin::flush-strategy): new slot; stores flush
strategy
(close): call `flush' instead of `write-buffer'
(put-entry buffering-writer-mixin t t t): call `flush?' and `flush'
(flush buffering-writer-mixin t): new method; dispatch to
`write-buffer'
(flush :after buffering-writer-mixin t): changed write-buffer ->
flush - src/backend/flush-strategies.lisp: new file; contains flush
strategies - src/backend/package.lisp (package rsbag.backend): added used package
more-conditions; added exported symbols buffer-property, flush,
backend-flush-strategy, flush?, no-such-flush-strategy-class,
find-flush-strategy-class, flush-strategy-classes and
make-flush-strategy - src/backend/tidelog/file.lisp (file): change default initargs to
supply a flush strategy instead of a flush? function
(buffer-property file chnk eql :length/entries): new method; return
number of entries in chunk
(buffer-property file chnk eql :length/bytes): similar for bytes
(buffer-size->): removed; no longer required - test/protocol.lisp (test suite protocol-root): added local functions
pathname/existing, namestring/existing, stream
(test protocol-root::open-bag/valid): use these; test flush
strategies
(test protocol-root::open-bag/invalid): likewise - test/backend/flush-strategies.lisp: new file; contains tests for
flush strategies - cl-rsbag.asd (system cl-rsbag): added file
src/backend/flush-strategies.lisp
(system cl-rsbag-test): added file
test/backend/flush-strategies.lisp
Added last-write-time-mixin in src/backend/backend-mixins.lisp
refs #846, #847
refs #846, #847
- src/backend/backend-mixins.lisp: new file; contains mixin
classes for backend classes - src/backend/package.lisp (package rsbag.backend): added exported
symbol last-write-time-mixin * src/backend/tidelog/index.lisp (index): added superclass
`last-write-time-mixin' - src/backend/tidelog/file.lisp (file): likewise
- cl-rsbag.asd (system cl-rsbag): added file
src/backend/backend-mixins.lisp
Support flush strategies in bag-record/main.lisp
refs #846, #847
refs #846, #847
- bag-record/main.lisp (update-synopsis): added flush-strategy
commandline option
(main): pass value obtained from flush-strategy commandline option
to `events->bag' - bag-record/help.lisp (make-channel-strategy-help-string): minor
improvement
(make-flush-strategy-help-string): new function; return help
string for flush strategies
History
#1 Updated by J. Moringen over 12 years ago
- % Done changed from 20 to 60
#2 Updated by J. Moringen over 12 years ago
- Target version set to rsb-0.7
#3 Updated by J. Moringen about 12 years ago
- % Done changed from 60 to 80
#4 Updated by J. Moringen about 12 years ago
- % Done changed from 80 to 90
#5 Updated by J. Moringen about 12 years ago
- Status changed from In Progress to Resolved
- % Done changed from 90 to 100
Applied in changeset r468.