History

Wiki ยป

GroupManager

Overview

Builds hypotheses about the group the robot is interacting with. This component computes the two group level engagement value "interest level" and "group stability", provides stabilized / somewhat abstracted information about the size of the group and provides aggregate information about the group composition (age and gender).

Since no clustering or similar process is done at the moment, the definition of the "current visitor group" basically corresponds to "everyone visible to the robot".

Component specification

Inputs
Required:
  • rst.vision.Faces from the FaceDetection component: faces detected on the first camera image (required for estimating group size, stability etc.)
  • rst.vision.HeadObjects from the VFoA component: For face tracking data and for estimating interest level from visual focus targets etc.
  • rst.vision.HeadObjects from the Identity Recognition component: For getting the age and gender classification results of individuals (to generate aggregate results for the group).
Outputs
  • rst.hri.GroupHypothesis: Aggregated information about the currently visible group.

Interface
The information about scopes, output rate etc. here is based on the default values, most can be changed on the command line.

Component
    {GroupManager}
    {Builds hypotheses about the group the robot is interacting with.}
    Informers
        * {results} {/humavips/groupinfo} {rst.hri.GroupHypothesis} {10 Hz}
    Listeners
        * {left_faces} {/nao/facedetection/0} {rst.vision.Faces}
        * {vfoa_results} {/nao/headpose} {rst.vision.HeadObjects}
        * {id_results} {/nao/identities} {rst.vision.HeadObjects}

Prerequesites / installation

  • This component needs rsb-python, RST python bindings (only 0.7 versions have been tested so far) and PyQT4.
  • The three input components (FaceDetection, VFoA & Identity Recognition) need to be installed and working.
  • This component can be installed with the normal python setup.py install --prefix=/your/install/prefix logic.
  • It installs two launch scripts of interest, gmlauncher (the main component launch script) and gmvis (a visualization for the component results), into the prefix bin directory.

Command line options

Both gmlauncher and gmvis provide help on their command line options when called with the --help flag. The current command line options are reproduced below, but you should always check the output of your local installed version in case something has changed in the meantime.

GroupManager launcher:

usage: gmlauncher [-h] [--rate RATE] [--stability-method METHOD]
                     [--outscope OUTSCOPE] [--facescope FACESCOPE]
                     [--vfoascope VFOASCOPE] [--idscope IDSCOPE]

The group manager component.

optional arguments:
  -h, --help            show this help message and exit

Main GroupManager arguments:
  --rate RATE, -r RATE  Default rate of produced outputs [Hz] (default: 10)
  --stability-method METHOD, -s METHOD
                        Method for classifying group stability. Valid choices
                        are 'simple': a simple handcrafted classifier, 'knn':
                        a learned kNN classifier (default: simple)

RSB input/output arguments:
  --outscope OUTSCOPE, -o OUTSCOPE
                        RSB scope where group manager results are published
                        (default: /humavips/groupinfo)
  --facescope FACESCOPE, -f FACESCOPE
                        RSB scope to listen on for face detection results
                        (default: /nao/facedetection/0)
  --vfoascope VFOASCOPE, -v VFOASCOPE
                        RSB scope to listen on for VFOA results (default:
                        /nao/headpose)
  --idscope IDSCOPE, -i IDSCOPE
                        RSB scope to listen on for ID/Gender/Age results
                        (default: /nao/identities)

GroupManager visualization:

usage: gmvis [-h] [--scope SCOPE]

A visualization for the group manager.

optional arguments:
  -h, --help            show this help message and exit
  --scope SCOPE, -s SCOPE
                        RSB scope to listen on for group manager results
                        (default /humavips/groupinfo)

Output format

While running, the component generates outputs of the type rst.hri.GroupHypothesis on the default scope /humavips/groupinfo with a default rate of 10 hz. The fields of the output data can be interpreted as follows:

  • Group size and basic info
    • tracking_info: Not used at the moment, could be used as an identifier to distinguish multiple groups in the future.
    • group_size_estimate: An estimate of the number of people in the current visitor group.
    • group_size_category: A coarse grained estimate of the group size in the three categories SMALL_GROUP, GROUP or CROWD.
  • Group engagement info
    • interest_level: A level of interest the group is showing towards the robot (range: [0.0,1.0])
    • group_stability: How stable is the group, i.e. is there a low level of fluctuation of group members (range: [0.0,1.0])
  • Group composition info
    • age_category: A coarse grained estimate of the group age composition in the categories CHILDREN, ADULTS, SENIORS or MIXED.
    • gender_category: An estimate of the group gender composition in the categories MIXED_GENDERS, COMPLETELY_FEMALE or COMPLETELY_MALE.

When not enough or no input is received, the respective output fields will be left in their unset state (which can be queried through protobuf methods).

Extending the GroupManager

This section contains some notes on how the values for interest, stability, size and the composition are calculated and how this can be changed / extended (i.e. the "developer documentation" :)).

The GroupManager runs a basic processing loop at the rate given above. Basically all output values are calculated on small moving windows of the last N seconds of input data (faces, vfoa results etc.). N is a parameter that can be tuned to make the outputs either more stable or have a faster reaction to changes in the input. At the moment there are no command line options for this, but the different values of N used for different calculations can be changed directly at the beginning of the main loop (the numbers given to the constructors in lines 83-93): source:groupmanager/__init__.py#L82

Interest

The main class responsible for calculating the interest level is called InterestStats (source:groupmanager/groupstats.py#L358). It receives new VFoA results through its addData method (called from the RSB listener) and calculates the interest value based on the VFoA data from the last N seconds in the update method (called from the main group manager loop).

Stability

The main class responsible for calculating the group stability level is called GroupStabilityStats (source:groupmanager/groupstats.py#L91). It receives new face tracking results through its addFaceTrackData method and new face detection results through addFaceDetectionData (both called from the respective RSB listeners). It calculates four intermediate cue values (the "movement factor", the "track factor", the "head count factor" and the "distance factor") in the update method (called from the main group manager loop) and uses a separate classifier class to calculate the stability value from these four.

The different intermediate values can be described like this:
  • The movement factor: Basically how much faces in the image moved in the current time window.
  • The track factor is based on the track length or other tracking stability factors.
  • The head count factor is based on the number of visible faces in images in the current window.
  • The distance factor is based on the distances between visible faces in images in the current time window.
At the moment there are two classifiers available, the simple and the kNN based classifier:
  • SimpleStabilityClassifier (source:groupmanager/groupstats.py#L18): A simple heuristic of determining the stability as a weighted average/sum of all the different cues, the fixed weights chosen by semi-informed guesswork.
  • KNNStabilityClassifier (source:groupmanager/groupstats.py#L30): A k-nearest-Neighbor classifier for determining the stability value from labeled example data (the four intermediate values labeled with the resulting stability). At the moment some test values are hardcoded into the class, but it can also load a numpy array with real examples.

Group Size & Composition

The main class responsible for calculating the group size estimates is called GroupSizeStats (source:groupmanager/groupstats.py#L296). It receives new face tracking (or detection) results through its addData method (called from the RSB listener) and calculates the size estimate and class in the update method (called from the main group manager loop).

The main class responsible for calculating the aggregated group composition information is called CompositionStats (source:groupmanager/groupstats.py#L397). It receives new id/age/gender results through its addData method (called from the RSB listener) and calculates the aggregates in the update method (called from the main group manager loop).