Enhancement #2351

bag info: Machine-readable output

Added by J. Wienke over 8 years ago. Updated almost 8 years ago.

Status:ResolvedStart date:07/24/2015
Priority:NormalDue date:
Assignee:J. Moringen% Done:

100%

Category:Commandline Tools
Target version:Robotics Service Bus - rsb-0.15

Description

Please provide an output format for bag-info which is easily parseable.

Related issues

Duplicated by RSBag - Feature #2553: JSON export for bag info Rejected 05/31/2016

Associated revisions

Revision 988fc7e0
Added by J. Moringen almost 8 years ago

Builder-based formatting style in src/commands/info.lisp

fixes #2353, fixes #2352, refs #2351

  • src/commands/info.lisp (header): updated copyright
    (bag-style-tree): new class
    (stream): new special variable; used by `format-event' method
    (format-event bag bag-style-tree t): new method; format bag and
    channel information in a textual tree using a provided builder
    traversal and computed properties
    (info::print-format): changed :no -> nil in type
    (command-execute info): construct a `bag-style-tree' instance and use
    it in `format-event'
  • bag-info/main.lisp (main): translate print-format value :no to nil for
    :print-format initarg of `info' command
  • rsbag-tools-commands.asd (system rsbag-tools-commands): added system
    dependency on rsbag-builder

Revision 6fab9b5b
Added by J. Moringen almost 8 years ago

Added info-style service in src/commands/info.lisp

refs #2351

  • src/commands/info.lisp (define-service info-style): new service; for
    bag info formatting styles
    (register-provider/class info-style :tree): register `bag-style-tree'
    class
    (command-execute info): construct formatting using the `info-style'
    service

Revision 8be064fa
Added by J. Moringen almost 8 years ago

Added style commandline option in bag-info/main.lisp

refs #2351

  • src/commands/info.lisp (info): added superclass `style-mixin'
    (info::style-service): new slot; overwrite initform to 'info-style
    (command-execute info): do not create style instance;
    `command-make-style' does that now
    (command-make-style info cons t): new method; construct a bag
    formatting style with the correct builder
  • src/commands/package.lisp (package rsbag.tools.commands): added
    import-from rsb.tools.commands symbols style-service
  • bag-info/main.lisp (update-synopsis): added style commandline option
    (main): pass value of style commandline option to `info' command via
    :style-spec initarg
  • bag-info/package.lisp (system rsbag.tools.info): added shadowed symbol
    make-style-help-string
  • test/commands/info.lisp (header): updated copyright
    (test info-root::construction): test :style and :style-spec initargs
  • cl-rsbag-tools-info.asd (system cl-rsbag-tools-info): added system
    dependency on rsb-formatting-and-rsb-common

Revision e4fbc53a
Added by J. Moringen almost 8 years ago

Describe new style commandline option in tool-info.rst

refs #2351

  • tool-info.rst (Description): describe new style commandline option
  • news.rst (RSBag 0.15): mention new style commandline option

Revision aa284786
Added by J. Moringen almost 8 years ago

Added universal-builder-for-event-data in src/builder.lisp

refs #2351, refs #1994

  • src/builder.lisp (header): updated copyright
    (package rsb.builder): added exported symbol
    universal-builder-for-event-data
    (universal-builder-for-event-data): new function; switch to
    universal-builder for data slot of event
  • test/builder.lisp (package rsb.builder.test): added used package
    rsb.builder
    (check-un-build-calls): added peek-function keyword parameter; pass to
    `record-un-build-calls/peeking'
    (mock-payload): new class; mock payload for builder tests
    (test rsb-builder-root::event/universal-builder-for-event-data): new
    test; test switching to universal builder for data relation in `event'
    nodes
  • rsb-builder.asd (header): updated copyright
    (system rsb-builder): added system dependency on
    architecture.builder-protocol.universal-builder

Revision c254e0d0
Added by J. Moringen almost 8 years ago

Builder-based JSON styles in src/formatting/{event-style-,introspection/}json.lisp

refs #2351

  • src/formatting/event-style-json.lisp (pass-through-value): new type;
    values that can directly be converted into native JSON values
    (stringify-value): new type; values that have to be turned into
    strings for JSON conversion
    (maybe-stringify-value): new function; turn a value into a string for
    JSON conversion
    (prepare-initarg-value-for-json): new function; stringify an initarg
    value if necessary
    (make-json-peek-function): new function; make a peek function that
    directly encodes "pass-through" and "stringifiable" values
    (make-json-serializer): new function; make a JSON serializer suitable
    for use in a event formatting style using the above helper functions
    (style-json::builder): new slot; stores builder used for JSON
    serialization
    (style-json::event-serializer): new slot; stores serializer used for
    events
    (style-json::payload-serializer): new slot; stores serializer used for
    event payloads
    (shared-initialize :after style-json t): new method; initialize {event,payload}-serializer slots
    (format-event t style-json t): changed specializer event -> t; use
    `serialize-using-serializer' instead of `encode-json'
    (format-payload t style-json t): similar
    (encode-json-sequence): removed; no longer needed
    (with-members): likewise
    (encode-json event): likewise
  • src/formatting/introspection/json.lisp (header): updated copyright
    (encode-json tracked-quantity): removed; no longer needed
    (encode-json participant-entry): likewise
    (encode-json process-entry): likewise
    (encode-json host-entry): likewise
    (encode-json remote-introspection-database): likewise
    (encode-json remote-introspection): likewise
    (style-json::builder): new slot; stores builder used for JSON
    serialization
    (style-json::serializer): new slot; stores serializer used for events
    (format-event eql :dummy style-json t): changed specializer t -> eql
    :dummy; use `serialize-using-serializer' instead of `encode-json'
  • test/formatting/style-json.lisp (header): updated copyright
    (test style-json-root::smoke): changed numerus or keys in expected
    output
  • test/formatting/introspection/json.lisp (header): updated copyright
    (test rsb.formatting.introspection.json-root::smoke): instantiate
    style for test
  • rsb-formatting-json.asd (system rsb-formatting-json): added system
    dependencies on architecture.builder-protocol.universal-builder,
    architecture.builder-protocol.json, rsb-builder and rsb-model-builder

Revision 120dcf5e
Added by J. Moringen almost 8 years ago

Register JSON style in src/commands/info.lisp

fixes #2351

  • src/commands/info.lisp (register-provider/class info-style): register
    `style-json' as a bag info formatting style
  • bag-info/main.lisp (make-style-help-string): added
    :event-peek-function and :payload-peek-function to initarg blacklist
  • CMakeLists.txt (header): updated copyright
    (toplevel): added tests for the style commandline option of the info
    command

History

#1 Updated by J. Moringen over 8 years ago

I have been working on a generic protocol for this anyway.

Would something like the following be acceptable?

(defvar *bag* (rsbag:open-bag "/home/jmoringe/Desktop/kinect-data-2013-02-28.tide" 
                              :direction :input))

XML

(architecture.builder-protocol.xml::unbuild/xml
 t *bag*)


<?xml version="1.0" encoding="UTF-8"?>
<BAG LOCATION="/home/jmoringe/Desktop/kinect-data-2013-02-28.tide" DIRECTION="INPUT">
  <CHANNEL>
    <CHANNEL NAME="/persontracking/posture/4/:.rst.tracking.TrackedPosture3DFloat" START-TIMESTAMP="2013-02-28T18:53:14.728944+01:00" END-TIMESTAMP="2013-02-28T18:53:24.291761+01:00" NUM-EVENTS="288" TYPE="(RSB-EVENT-0.8 .rst.tracking.TrackedPosture3DFloat)" SOURCE-NAME="654A078D-37AF-454E-B962-156CB96AFD99" SOURCE-CONFIG="rsb:/#654A078D-37AF-454E-B962-156CB96AFD99"/>
    <CHANNEL NAME="/persontracking/position/4/:.rst.tracking.TrackedPosition3DFloat" START-TIMESTAMP="2013-02-28T18:53:14.666544+01:00" END-TIMESTAMP="2013-02-28T18:53:14.666544+01:00" NUM-EVENTS="2" TYPE="(RSB-EVENT-0.8 .rst.tracking.TrackedPosition3DFloat)" SOURCE-NAME="654A078D-37AF-454E-B962-156CB96AFD99" SOURCE-CONFIG="rsb:/#654A078D-37AF-454E-B962-156CB96AFD99"/>
    <CHANNEL NAME="/persontracking/state/4/:.rst.tracking.State" START-TIMESTAMP="2013-02-28T18:53:14.666544+01:00" END-TIMESTAMP="2013-02-28T18:53:24.322961+01:00" NUM-EVENTS="3" TYPE="(RSB-EVENT-0.8 .rst.tracking.State)" SOURCE-NAME="654A078D-37AF-454E-B962-156CB96AFD99" SOURCE-CONFIG="rsb:/#654A078D-37AF-454E-B962-156CB96AFD99"/>
    <CHANNEL NAME="/persontracking/posture/3/:.rst.tracking.TrackedPosture3DFloat" START-TIMESTAMP="2013-02-28T18:52:35.682076+01:00" END-TIMESTAMP="2013-02-28T18:53:05.103727+01:00" NUM-EVENTS="877" TYPE="(RSB-EVENT-0.8 .rst.tracking.TrackedPosture3DFloat)" SOURCE-NAME="654A078D-37AF-454E-B962-156CB96AFD99" SOURCE-CONFIG="rsb:/#654A078D-37AF-454E-B962-156CB96AFD99"/>
    <CHANNEL NAME="/persontracking/position/3/:.rst.tracking.TrackedPosition3DFloat" START-TIMESTAMP="2013-02-28T18:52:35.619675+01:00" END-TIMESTAMP="2013-02-28T18:52:35.635275+01:00" NUM-EVENTS="2" TYPE="(RSB-EVENT-0.8 .rst.tracking.TrackedPosition3DFloat)" SOURCE-NAME="654A078D-37AF-454E-B962-156CB96AFD99" SOURCE-CONFIG="rsb:/#654A078D-37AF-454E-B962-156CB96AFD99"/>
    <CHANNEL NAME="/persontracking/state/3/:.rst.tracking.State" START-TIMESTAMP="2013-02-28T18:52:35.619675+01:00" END-TIMESTAMP="2013-02-28T18:53:05.150527+01:00" NUM-EVENTS="3" TYPE="(RSB-EVENT-0.8 .rst.tracking.State)" SOURCE-NAME="654A078D-37AF-454E-B962-156CB96AFD99" SOURCE-CONFIG="rsb:/#654A078D-37AF-454E-B962-156CB96AFD99"/>
  </CHANNEL>
</BAG>

JSON

(architecture.builder-protocol.json::unbuild/json t *bag*)


{"kind":"bag","location":"\/home\/jmoringe\/Desktop\/kinect-data-2013-02-28.tide","direction":"INPUT",
 "channel":[{"kind":"channel","name":"\/persontracking\/posture\/4\/:.rst.tracking.TrackedPosture3DFloat","start-timestamp":"2013-02-28T18:53:14.728944+01:00","end-timestamp":"2013-02-28T18:53:24.291761+01:00","num-events":"288","type":"(RSB-EVENT-0.8 .rst.tracking.TrackedPosture3DFloat)","source-name":"654A078D-37AF-454E-B962-156CB96AFD99","source-config":"rsb:\/#654A078D-37AF-454E-B962-156CB96AFD99"},
            {"kind":"channel","name":"\/persontracking\/position\/4\/:.rst.tracking.TrackedPosition3DFloat","start-timestamp":"2013-02-28T18:53:14.666544+01:00","end-timestamp":"2013-02-28T18:53:14.666544+01:00","num-events":"2","type":"(RSB-EVENT-0.8 .rst.tracking.TrackedPosition3DFloat)","source-name":"654A078D-37AF-454E-B962-156CB96AFD99","source-config":"rsb:\/#654A078D-37AF-454E-B962-156CB96AFD99"},
            {"kind":"channel","name":"\/persontracking\/state\/4\/:.rst.tracking.State","start-timestamp":"2013-02-28T18:53:14.666544+01:00","end-timestamp":"2013-02-28T18:53:24.322961+01:00","num-events":"3","type":"(RSB-EVENT-0.8 .rst.tracking.State)","source-name":"654A078D-37AF-454E-B962-156CB96AFD99","source-config":"rsb:\/#654A078D-37AF-454E-B962-156CB96AFD99"},   
            {"kind":"channel","name":"\/persontracking\/posture\/3\/:.rst.tracking.TrackedPosture3DFloat","start-timestamp":"2013-02-28T18:52:35.682076+01:00","end-timestamp":"2013-02-28T18:53:05.103727+01:00","num-events":"877","type":"(RSB-EVENT-0.8 .rst.tracking.TrackedPosture3DFloat)","source-name":"654A078D-37AF-454E-B962-156CB96AFD99","source-config":"rsb:\/#654A078D-37AF-454E-B962-156CB96AFD99"},
            {"kind":"channel","name":"\/persontracking\/position\/3\/:.rst.tracking.TrackedPosition3DFloat","start-timestamp":"2013-02-28T18:52:35.619675+01:00","end-timestamp":"2013-02-28T18:52:35.635275+01:00","num-events":"2","type":"(RSB-EVENT-0.8 .rst.tracking.TrackedPosition3DFloat)","source-name":"654A078D-37AF-454E-B962-156CB96AFD99","source-config":"rsb:\/#654A078D-37AF-454E-B962-156CB96AFD99"},
            {"kind":"channel","name":"\/persontracking\/state\/3\/:.rst.tracking.State","start-timestamp":"2013-02-28T18:52:35.619675+01:00","end-timestamp":"2013-02-28T18:53:05.150527+01:00","num-events":"3","type":"(RSB-EVENT-0.8 .rst.tracking.State)","source-name":"654A078D-37AF-454E-B962-156CB96AFD99","source-config":"rsb:\/#654A078D-37AF-454E-B962-156CB96AFD99"}]}

Tree

(architecture.builder-protocol.print-tree::unbuild/print-tree t *bag*)


BAG
│ LOCATION: /home/jmoringe/Desktop/kinect-data-2013-02-28.tide
│ DIRECTION: INPUT
├─CHANNEL
│   NAME: /persontracking/posture/4/:.rst.tracking.TrackedPosture3DFloat
│   START-TIMESTAMP: 2013-02-28T18:53:14.728944+01:00
│   END-TIMESTAMP: 2013-02-28T18:53:24.291761+01:00
│   NUM-EVENTS: 288
│   TYPE: (RSB-EVENT-0.8 .rst.tracking.TrackedPosture3DFloat)
│   SOURCE-NAME: 654A078D-37AF-454E-B962-156CB96AFD99
│   SOURCE-CONFIG: rsb:/#654A078D-37AF-454E-B962-156CB96AFD99
├─CHANNEL
│   NAME: /persontracking/position/4/:.rst.tracking.TrackedPosition3DFloat
│   START-TIMESTAMP: 2013-02-28T18:53:14.666544+01:00
│   END-TIMESTAMP: 2013-02-28T18:53:14.666544+01:00
│   NUM-EVENTS: 2
│   TYPE: (RSB-EVENT-0.8 .rst.tracking.TrackedPosition3DFloat)
│   SOURCE-NAME: 654A078D-37AF-454E-B962-156CB96AFD99
│   SOURCE-CONFIG: rsb:/#654A078D-37AF-454E-B962-156CB96AFD99
├─CHANNEL
│   NAME: /persontracking/state/4/:.rst.tracking.State
│   START-TIMESTAMP: 2013-02-28T18:53:14.666544+01:00
│   END-TIMESTAMP: 2013-02-28T18:53:24.322961+01:00
│   NUM-EVENTS: 3
│   TYPE: (RSB-EVENT-0.8 .rst.tracking.State)
│   SOURCE-NAME: 654A078D-37AF-454E-B962-156CB96AFD99
│   SOURCE-CONFIG: rsb:/#654A078D-37AF-454E-B962-156CB96AFD99
├─CHANNEL
│   NAME: /persontracking/posture/3/:.rst.tracking.TrackedPosture3DFloat
│   START-TIMESTAMP: 2013-02-28T18:52:35.682076+01:00
│   END-TIMESTAMP: 2013-02-28T18:53:05.103727+01:00
│   NUM-EVENTS: 877
│   TYPE: (RSB-EVENT-0.8 .rst.tracking.TrackedPosture3DFloat)
│   SOURCE-NAME: 654A078D-37AF-454E-B962-156CB96AFD99
│   SOURCE-CONFIG: rsb:/#654A078D-37AF-454E-B962-156CB96AFD99
├─CHANNEL
│   NAME: /persontracking/position/3/:.rst.tracking.TrackedPosition3DFloat
│   START-TIMESTAMP: 2013-02-28T18:52:35.619675+01:00
│   END-TIMESTAMP: 2013-02-28T18:52:35.635275+01:00
│   NUM-EVENTS: 2
│   TYPE: (RSB-EVENT-0.8 .rst.tracking.TrackedPosition3DFloat)
│   SOURCE-NAME: 654A078D-37AF-454E-B962-156CB96AFD99
│   SOURCE-CONFIG: rsb:/#654A078D-37AF-454E-B962-156CB96AFD99
└─CHANNEL
    NAME: /persontracking/state/3/:.rst.tracking.State
    START-TIMESTAMP: 2013-02-28T18:52:35.619675+01:00
    END-TIMESTAMP: 2013-02-28T18:53:05.150527+01:00
    NUM-EVENTS: 3
    TYPE: (RSB-EVENT-0.8 .rst.tracking.State)
    SOURCE-NAME: 654A078D-37AF-454E-B962-156CB96AFD99
    SOURCE-CONFIG: rsb:/#654A078D-37AF-454E-B962-156CB96AFD99NIL

#2 Updated by J. Moringen over 8 years ago

Also

(architecture.builder-protocol.xpath::query
 "//channel[@num-events < 3]" *bag*)
=>
(#<CHANNEL
   "/persontracking/position/4/:.rst.tracking.TrackedPosition3DFloat" (2) RSB-EVENT
   {1237DF51}>
 #<CHANNEL
   "/persontracking/position/3/:.rst.tracking.TrackedPosition3DFloat" (2) RSB-EVENT
   {1237DF21}>)

#3 Updated by J. Wienke over 8 years ago

Yes, XML or JSON would be fine. I want to process bag meta information in python. Right now I have a parser for the usual output, but we can convert this to the XML output later. It might also be interesting to add this parsing code to the python bag interface.

#4 Updated by J. Moringen over 8 years ago

J. Wienke wrote:

It might also be interesting to add this parsing code to the python bag interface.

I would rather avoid that. I'm working on a more complete remote interface that can be used instead.

#5 Updated by J. Moringen about 8 years ago

  • Target version set to rsb-0.14

#6 Updated by J. Moringen almost 8 years ago

#7 Updated by J. Moringen almost 8 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 50

#8 Updated by J. Moringen almost 8 years ago

  • Target version changed from rsb-0.14 to rsb-0.15

#9 Updated by J. Moringen almost 8 years ago

  • Subject changed from bag info: machine-readable output to bag info: Machine-readable output

#10 Updated by J. Moringen almost 8 years ago

  • % Done changed from 50 to 70

#11 Updated by J. Moringen almost 8 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 70 to 100

Also available in: Atom PDF