Bug #867
Include IDL in channel format field
| Status: | Resolved | Start date: | 02/08/2012 | |
|---|---|---|---|---|
| Priority: | High | Due date: | ||
| Assignee: | J. Moringen | % Done: | 100% | |
| Category: | - | |||
| Target version: | Robotics Service Bus - 0.6 |
Description
One of the most important aspects of the TIDE format is that the channel description includes a copy of the IDL of the logged data. This is essential to ensure future readability of log files.
At the moment, rsbag does not record this data. I understand this is due to an issue with acquiring it at runtime, but was also under the impression that a solution for that is now available with RST.
I would be very happy if you could look into a solution for this as soon as possible.
Related issues
Associated revisions
refs #867
- src/transform/rsb-event.lisp (transform-format rsb-event): new
method; return Protocol Buffer IDL descriptions for the event and,
if possible, the payload; this is quite some hack - src/transform/protocol.lisp (transform-format): new generic
function; return an object describing the serialization format
implemented by a transform - src/transform/package.lisp (package rsbag.transform): added exported
symbol transform-format
refs #867
- src/rsb/protocol.lisp (channel-format-for): added transform
parameter
(channel-format-for t eql nil t t): added eql nil specializer
(channel-format-for t t t t): new method; default behavior for
non-nil transform is to ask transform for format - src/rsb/channel-strategies.lisp
(make-channel-for channel-connection event scope-and-type): pass
transform to `channel-format-for'
fixes #867
- bag-record/main.lisp (update-synopsis): add IDL options via
`make-idl-options'
(main): call `process-idl-options'
refs #867
- manual/common.rst: added IDL options
- manual/logger.rst: moved IDL options to manual/common.rst
refs #867
- bag-cat.rst: added reference to IDL-related options in RSB
- bag-record.rst: likewise
refs #867
- bag-info/main.lisp (header): updated copyright
(update-synopsis): added print-format commandline option
(main): print format, if requested
History
#1 Updated by J. Moringen about 12 years ago
- Priority changed from Normal to High
#2 Updated by J. Moringen about 12 years ago
To achieve the requested behavior in the upcoming release, our current plan is to
- Extend the TIDELog specification such that multiple IDLs can be stored in the
CHAN.formatfield by making it a list of blobs instead of single blob. We would submit this modification for inclusion in future versions of the TIDELog specification. - Implement filesystem-based lookup and reading of IDL in
bag-record.
@Ingo: Is 1) OK for you or do you have different plans?
#3 Updated by J. Moringen about 12 years ago
- Target version set to 0.6
#4 Updated by I. Lütkebohle about 12 years ago
The expectation in the definition of the "format" field is that can you simply drop it, as-is, into your chosen serialization library, and have it parse the corresponding messages. The reason for this is that it makes the interaction with the serialization library unidirectional.
If you instead make it into a list on the TIDE level, this will result in bidirectional interaction with the serialization library, because it will need to go back to the TIDE log to look up further type information. This requires extensibility on the side of the serialization library that may be missing.
Therefore, I wouldn't recommend it. My suggestion would be to have an internal structure within the format field that allows including multiple message definitions. This structure may be specific to your serialization format.
#5 Updated by S. Wrede about 12 years ago
Thanks for the clarification. We also thought already about inventing a micro-format to represent the respective information but wondered if that would be in the spirit of the log format. That said, we we will try to come up with an initial solution to this as part of the next RSB release 0.6.
#6 Updated by J. Moringen about 12 years ago
- Assignee set to J. Moringen
#7 Updated by J. Moringen about 12 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 20
#8 Updated by J. Moringen about 12 years ago
- % Done changed from 20 to 70
- Users will have to load IDLs into
bag-recordmanually
(via--idl-pathand--load-idlas with the Common Lisplogger) - We will store two textual Protocol Buffer definitions in the
formatfield of each channel:rsb.protocol.Notificationand its transitive dependency closure- Payload type and its transitive dependency closure, if possible
This can fail (possibly generating a warning, but continuing the recording) if- The payload type is not a protocol buffer type
- The corresponding IDL has not been loaded
- The corresponding IDL version does not match the actual payload
this problem will not be detected
- The two textual definitions are separated by a
:character - The stored IDL information will not be used by any tools, for now
#9 Updated by I. Lütkebohle about 12 years ago
Regarding detection of a mismatch between IDL and what's being used: Is there a checksum field in the message which could be checked against? ROS and LCM have md5 sums of the idl for this.
#10 Updated by J. Moringen about 12 years ago
- Distinguishing serialization mechanisms (e.g. protocol buffer serialization vs. something else)
- Verifying that a given payload has actually been serialized by means of the specified serialization mechanism
#11 Updated by J. Moringen about 12 years ago
- Status changed from In Progress to Resolved
- % Done changed from 70 to 100
Applied in changeset r345.
#12 Updated by J. Moringen over 11 years ago
- Subject changed from Include IDL in Channel Format Field to Include IDL in channel format field