NaoAV » History » Version 8
J. Moringen, 07/06/2011 01:05 AM
added pipeline
1 | 1 | J. Moringen | h1. NaoAV |
---|---|---|---|
2 | 1 | J. Moringen | |
3 | 1 | J. Moringen | h2. Introduction |
4 | 1 | J. Moringen | |
5 | 1 | J. Moringen | The NaoA/V setup is a hack that allows to |
6 | 1 | J. Moringen | * transmit camera images and audio buffers from the Nao robot to a different node |
7 | 1 | J. Moringen | * synchronize audio and video data streams |
8 | 1 | J. Moringen | * store the synchronized audio/video stream in a container file |
9 | 1 | J. Moringen | |
10 | 1 | J. Moringen | It consists of several components: |
11 | 1 | J. Moringen | |
12 | 3 | J. Moringen | * On the Nao robot |
13 | 6 | J. Moringen | ** @rsbvideosender@ |
14 | 6 | J. Moringen | ** @rsbaudiosender@ |
15 | 3 | J. Moringen | * In a receiver/synchronization/store process on a workstation |
16 | 3 | J. Moringen | ** @gst_nao_audio_import@ |
17 | 3 | J. Moringen | ** @gst_nao_video_import@ |
18 | 1 | J. Moringen | |
19 | 1 | J. Moringen | h2. Static View |
20 | 1 | J. Moringen | |
21 | 4 | J. Moringen | !static-view.png! |
22 | 1 | J. Moringen | |
23 | 2 | J. Moringen | h2. Implementation |
24 | 3 | J. Moringen | |
25 | 5 | J. Moringen | The implementation uses the "GStreamer":http://gstreamer.freedesktop.org/ multimedia framework to synchronize, mux and store the audio and video streams. GStreamer is capable of aligning and synchronizing the audio and video streams which are received via RSB and producing a merged A/V stream that is suitable for writing to a video container file. In order to perform this synchronization, GStreamer makes use of the timestamps which are included in all RSB events. |
26 | 7 | J. Moringen | |
27 | 8 | J. Moringen | The abovementioned processing pipeline can be instantiated from the commandline as follows: |
28 | 8 | J. Moringen | <pre> |
29 | 8 | J. Moringen | gst-launch-0.10 \ |
30 | 8 | J. Moringen | audiotestsrc do-timestamp=FALSE \ |
31 | 8 | J. Moringen | ! rsb_nao_audio_import pubName="/nao/audio/all" \ |
32 | 8 | J. Moringen | ! queue \ |
33 | 8 | J. Moringen | ! "audio/x-raw-int", channels=2, width=16, depth=16, signed=TRUE, rate=48000, endianess=4321 \ |
34 | 8 | J. Moringen | ! audiorate \ |
35 | 8 | J. Moringen | ! mux. videotestsrc do-timestamp=FALSE \ |
36 | 8 | J. Moringen | ! rsb_nao_video_import pubName="/nao/vision/top" \ |
37 | 8 | J. Moringen | ! queue \ |
38 | 8 | J. Moringen | ! videorate \ |
39 | 8 | J. Moringen | ! 'video/x-raw-yuv,width=320,height=240,framerate=20/1' \ |
40 | 8 | J. Moringen | ! mux. avimux name="mux" \ |
41 | 8 | J. Moringen | ! filesink location="/tmp/bla.avi" |
42 | 8 | J. Moringen | </pre> |
43 | 7 | J. Moringen | !>pipeline.png! |