Bug #375
Fragmentation: Do not transmit full Event Meta-data in all Notifications
| Status: | Resolved | Start date: | 06/20/2011 | |
|---|---|---|---|---|
| Priority: | High | Due date: | ||
| Assignee: | J. Moringen | % Done: | 100% | |
| Category: | Protocol | |||
| Target version: | rsb-0.7 |
Description
Problem
Currently, multiple identical notifications (except for Notification.data_part and Notification.data) are sent, each containing the full set of meta-data.
- Specify new behavior here ✓
- Make fields of
Notificationdescriptor optional ✓ - Implement
- C++ ✓
- Python ✓
- Java ✓
- Common Lisp ✓
Related issues
Associated revisions
Improved documentation comments in rsb/protocol/*.proto
refs #375
refs #375
- rsb/protocol/FragmentedNotification.proto: added link to explanation
of Spread-specific event <-> notification conversion - rsb/protocol/Notification.proto.in: removed Spread-specific comments
Adapted event fragmentation in Spread transport to protocol changes
refs #375
refs #375
- src/transport/spread/conditions.lisp (fragment-problem): adapted to
changed names in rsb.protocol
(invalid-fragment-id): likewise
(duplicate-fragment): likewise
(fragmentation-problem): new condition; signaled when
fragmentation-related problem are encountered
(insufficient-room): new condition; signaled when fragmentation
fails due to insufficient room - src/transport/spread/conversion.lisp (notification->event): adapted
to changed names in rsb.protocol; simplified
(one-notification->event): likewise
(event->notifications): likewise; calculate individual fragment
payload sizes by first determining the overhead; use
`fragmented-notification' instances for fragments
(make-notification): optionally set some notification fields - src/transport/spread/fragmentation.lisp
(assembly-concatenated-data): adapted to changes in rsb.protocol
(add-fragment! assembly fragmented-notification): likewise; changed
specializer notification -> fragmented-notification
(merge-fragment pool fragmented-notification): likewise; changed
specializer t -> fragmented-notification
(delete-partial-assemblies): simplified
(fragment-data): removed; no longer needed
(make-data-fragment): new function; simple `subseq' wrapper - src/transport/spread/package.lisp (package rsb.transport.spread):
removed shadowing import for meta-data; added shadowing import for
event-meta-data; added exported symbols fragmentation-problem,
insufficient-room, fragmentation-problem-required and
fragmentation-problem-available - test/transport/spread/fragmentation.lisp
(test suite fragmentation-root): changed local function
make-notification> make-fragment; added local function make-event*>notifications' instead of `fragment-data'; check for
(test fragmentation-root::assemble-smoke): adapted to fragmentation
changes
(test fragmentation-root::fragment-smoke): use
`event
`insufficient-room' errors
(test fragmentation-root::roundtrip): use `event-notifications'
instead of `fragment-data'; added test cases
(test fragmentation-root::warnings): adapted to changed
fragmentation
(test::pruning-assembly-pool-root::prune): likewise - test/transport/spread/package.lisp
(package rsb.transport.spread.test): qualified `defpackage' form
with cl package; removed additional `in-package' cl-user form;
removed imported symbol fragment-data; added imported symbol
event->notifications - cl-rsb.asd (system cl-rsb-and-cl-spread): added dependency of
src/transport/spread/conversion.lisp on
src/transport/spread/conditions.lisp (system
cl-rsb-and-cl-protobuf): changed class of protocol module to
protocol-buffer-descriptor-directory; changed files of the module
Protocol.proto -> EventId.proto, EventMetaData.proto,
Notification.proto and FragmentedNotification.proto
Adapted event to use RSB's serialization protocol
refs #375
refs #375
- src/transform/rsb-event.lisp (rsb-event::holder): adapted to changes
in rsb.protocol
(encode rsb-event event): likewise
(decode rsb-event simple-array): likewise - cl-rsbag.asd (system cl-rsbag): changed class of protocol module to
protocol-buffer-descriptor-directory; changed files of the module
Protocol.proto -> EventId.proto, EventMetaData.proto,
Notification.proto and FragmentedNotification.proto
Fixed conversion bugs in src/transport/spread/conversion.lisp
refs #375
refs #375
- src/transport/spread/in-connector.lisp
(message->event in-connector simple-array t): unpack as
`fragmented-notification' instead of `notification' - src/transport/spread/conversion.lisp (notification->event): retrieve
notification from fragment for event conversion
(event->notifications): cosmetic change
Fixed conversion in src/transform/rsb-event.lisp
refs #375
refs #375
- src/transform/rsb-event.lisp (encode rsb-event event): new data
definition uses an `event-id' instance instead of sequence-number
and sender-id fields
Transcoded file test/data/single-event.tide
refs #375
refs #375
- test/data/single-event.tide: transcoded for new event serialization
format
Adapted to changes in rsb-protocol
refs #375
refs #375
- setup.py: place file generated by protoc in rsb/protocol directory;
added an init.py file there - rsb/rsbspread/__init__.py: removed event <-> notification
conversion; adapted fragmentation to new protocol - rsb/rsbspread/conversion.py: new file; extracted and adapted event
<-> notification conversion
(Hopefully) fixed off-by-one error in src/rsb/transport/spread/OutConnector.cpp
refs #375
refs #375
- src/rsb/transport/spread/OutConnector.cpp: the loop which constructs
notification fragments for sending could go beyond the bounds of the
input string by one
Added comments in rsb/rsbspread/conversion.py
refs #375
refs #375
- rsb/rsbspread/conversion.py: added some comments regarding the
fragmentation of notifications
Fixed corner case in src/rsb/transport/spread/OutConnector.cpp
refs #375
refs #375
- src/rsb/transport/spread/OutConnector.cpp: produce at least one
notification fragment, even if the payload is empty; for example,
the void wire-schema uses empty payloads
History
#1 Updated by J. Moringen almost 13 years ago
- Status changed from New to In Progress
- Assignee changed from J. Wienke to J. Moringen
- Priority changed from Normal to High
#2 Updated by J. Moringen almost 13 years ago
- Description updated (diff)
#3 Updated by J. Moringen almost 13 years ago
- Description updated (diff)
- % Done changed from 0 to 20
#4 Updated by J. Wienke over 12 years ago
Now this even includes the cause vector.
#5 Updated by S. Wrede over 12 years ago
- Target version changed from rsb-0.10 to rsb-0.7
#6 Updated by J. Wienke over 12 years ago
I will do this now as otherwise there is no chance for me to implement the full transmission of RSB meta data in a convenient way for project:rsbtimesync
#7 Updated by J. Wienke over 12 years ago
Or better said, I would like to do this now ;)
#8 Updated by J. Moringen over 12 years ago
- Assignee changed from J. Moringen to J. Wienke
- % Done changed from 20 to 30
#9 Updated by J. Moringen over 12 years ago
- Description updated (diff)
#10 Updated by J. Wienke over 12 years ago
- Description updated (diff)
- % Done changed from 30 to 40
#11 Updated by J. Moringen over 12 years ago
- Description updated (diff)
- Assignee changed from J. Wienke to J. Moringen
- % Done changed from 40 to 90
#12 Updated by J. Moringen over 12 years ago
More Java changes in r3009
#13 Updated by J. Moringen over 12 years ago
- Description updated (diff)
#14 Updated by J. Moringen over 12 years ago
- Description updated (diff)
- Status changed from In Progress to Resolved
- % Done changed from 90 to 100