History

Process Proposal

Requirements

In general, the following requirements try to enforce the general reusability of data types.

  • Visibility: Data definitions must be easily visible when searching for them to foster reuse. This includes:
    • Folder browsing
    • Code completion
  • Structuredness: A structured view on the data is required as an orthogonal perspective to the general visibility in order to find functionally coherent groups of data definitions.
  • Stability: A core set of data definitions must be kept stable as much as possible in order to shield depending software from frequent changes.
  • Extendability: New data type definitions are constantly required. Adding them to the repository must have a low barrier to motivate data type sharing.
  • Addressability: A certain revision of a data type must be uniquely addressable and accessible in order to conserve system states.
  • Practicability: Data type definitions must be usable in an efficient manner in all target languages. This includes also the use of languages where IDL definitions are parsed at runtime and not compiled statically.

Proposed Solution

Packaging

  • RST will be split into two separate libraries. Both libraries will be built by the same build-system and use the same naming and structure conventions.
    • RST: stable data type definitions. Types in this library must not use types from sandbox.
    • RST-sandbox: new data type definitions which may be modified constantly. Sandbox-types may include types from stable.
  • Using the sandbox library will require explicit requests from the application developer, e.g. by using a separate pkg-config file. Also, e.g. for C++, the headers will be separated into different subdirectories in a way that using the stable library will not make the headers of the sandbox library automatically available to the compiler.
  • Libraries will be built for C++, Java and Python.

Data Type Integration Procedure

  • New data types can be added by every developer to the sandbox library using the specified style guide.
  • The RST team will merge the data types from sandbox to stable from time to time. During this process modifications to the data are explicitly allowed in order to meet the style and quality requirements of the stable library.
  • We will shortly integrate fundamental data types inspired by ROS, Orocos etc into the stable part of RST.
  • Modifications to existing data types will make use of protobuf's backwards compatibility features whenever possible.

Structure

  • The svn will contain a single tree for organizing the data definitions. This tree (i.e. the definition of appropriate categories) will be inspired by the data layout of ROS, Orocos and other robotics middlewares.
  • Folders in the svn match the packages provided in the IDL files.
  • A Redmine plugin will be created to browse the list of data types, e.g. in a flat list for easy search.

Versioning

  • No manual versioning will be used. Especially, no versions will be synchronized with RSB releases. Instead, svn revisions will be the only means of addressing specific versions. svn revisions will be integrated into the library for using them in the code or with the C++ preprocessor.

Style Guide

  • A style guide will be defined (or the current one will be improved) to cover more aspects, e.g. like the canonical format for the java_outer_classname option.
  • Even if some contents of the files can be defined canonically, we will not generate them using the build system.
  • The style guide will explicitly define that data type names do not contain artifacts of their current application. E.g. if a something is used as an event, the data type name must not contain the an event suffix or prefix.
  • All data types will have a common package, i.e. rst.

Addressed Requirements

Visibility

  • The redmine plugin as well as the common namespace for all data types will make them visible through search as well as code completion.
  • The hierarchical structure will foster the visibility coherent groups of data types.
  • The possibility to integrate sandbox types as soon as possible make them visible in an early stage using the same mechanisms like for the stable library. This will foster exchange with other developers on these types.

Structuredness

  • The proposed hierarchical structure provides a simple means to group messages and prevent the pollution of a single namespace.

Stability

  • The provided core set of data types will form a common basis for incorporation into newly added data types.
  • The stable library specifies policies for stability of data types definitions with backwards compatibility whenever possible.
  • By using the same namespaces for the sandbox library, data types which will be moved to the stable library from sandbox will not cause changing includes etc. as long as their initial position in the RST structure was well-chosen.
  • By requiring developers to explicitly request the use of sandbox types, their knowledge of possible instabilities in data types will be increased.
  • The manual review process to integrate data types into the stable part will increase the code quality of this part and will make later changes less likely, e.g. by further consolidating types and type/reuse inside other data types.

Extendability

  • By giving (nearly) public write access to the sandbox library, the barrier for adding or modifying types is quite low, especially compared to a formal review process already in this stage.
  • The common build system for RST automatically maintains the addition of new files without manual work.

Addressability

  • Formal releases were rejected to foster constant change for the sandbox library and the simultaneous build process of both library parts.
  • SVN revisions already provide a formal way of addressing either the whole library or certain types. Code integration of this versions will make this information easily accessible.

Practicability

  • Providing compiled libraries of RST decreases the required work in the own build system for generating classes from the IDL files.
  • For C++ and Java, providing a common library prevents subtle errors which occur if one IDL file is compiled multiple times and linked into one binary.
  • By not using a pre-processor to generate the final version of an IDL file, these definition can be used without another build system, e.g. in languages that interpret these files at runtime. Moreover, automatic downloads from the repository would be possible.
  • The structure where name and package match the file system location allows an automatic deduction of IDL file locations, e.g. for debugging purposes.

General

  • Reusability of types will be increased by preventing the use of communication patterns etc. in the data type names. E.g. a JointStateEvent might also be a valid request in an RPC communication. Hence, the better name is simply JointState.

Identified Drawbacks

  • The hierarchical structure does not provide a means of creating multiple overlapping organizational structures like e.g. tags would. However, it is the only feasible structure that is visible without processing the files by some kind of script or build system. A solution with symlinks would cause a lot of technical and organization problems. Hence, it was rejected.
  • In contrast to C++ and Java, Python does not allow several modules or packages to contribute to the same namespace. However, much more flexibility in the use of modules as objects and import directives is possible in this language. Special hints how to use these features appropriately will weaken the impact of moving types from sandbox to stable on the client code.

Open Issues

  • Control aspects over data types (arne), i.e. is a cca folder good or bad?