Tasks #1981

Port to GStreamer 1.x

Added by C. Leichsenring about 7 years ago. Updated about 4 years ago.

Status:In ProgressStart date:09/03/2014
Priority:NormalDue date:
Assignee:-% Done:

90%

Category:-
Target version:Robotics Service Bus - rsb-1.0

Description

I created a new branch gstreamer-1.0 where the port from gstreamer 0.10 to gstreamer 1.x can take place.

There are porting guidelines at http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/random/porting-to-1.0.txt

I'm closing the ticket I had previously created at https://projects.cit-ec.uni-bielefeld.de/issues/3861

make_linux.log (31.7 KB) C. Leichsenring, 09/10/2014 11:11 AM

make_mac_log.html Magnifier (80.9 KB) C. Leichsenring, 09/10/2014 01:47 PM

make_linux2.log (31.4 KB) C. Leichsenring, 09/10/2014 03:54 PM

rsbaudiosink.log (8.7 MB) C. Leichsenring, 09/22/2014 05:35 PM

History

#2 Updated by J. Moringen about 7 years ago

  • Subject changed from Port to to gstreamer 1.x to Port to GStreamer 1.x
  • Status changed from New to In Progress
  • Assignee set to C. Leichsenring
  • Target version set to rsb-0.12

#3 Updated by J. Moringen about 7 years ago

It would be nice if the code base could work with both, GStreamer 0.x and 1.x. However, the ifdef s and CMake + pkg-config trickery can probably wait until we know if/how well the 1.x stuff works.

#4 Updated by C. Leichsenring about 7 years ago

Jan Moringen wrote:

It would be nice if the code base could work with both, GStreamer 0.x and 1.x. However, the ifdef s and CMake + pkg-config trickery can probably wait until we know if/how well the 1.x stuff works.

I agree :)

#5 Updated by C. Leichsenring about 7 years ago

Just as an FYI, while the channel-positions format seems to have changed for 1.x, I still didn't get it to work. Still lousily documented, though.

I have the slight, very slight hope that someone will finally feel encouraged to answer http://stackoverflow.com/questions/25286975/how-to-set-channel-positions-a-channel-mask-in-gstreamer if the gstreamer version is no longer outdated though (BTW, upvotes welcome to get some additional traction).

#6 Updated by C. Leichsenring about 7 years ago

Current status: tried building on my Linux machine to see if the linker errors would also occur. Had some trouble because cmake would refuse to add the gstreamer-1.0 include directory to its flags. Ended up adding pkg-config lines to the generated makefile which is of course super ugly but at least pkg-config isn't as fucking blind as cmake is.

Compiled fine but ended with the following linker error:

/usr/bin/ld: CMakeFiles/gstrsb.dir/src/gst_rsb_src_base.cpp.o: relocation R_X86_64_PC32 against undefined symbol `_ZL21gst_rsb_src_base_initP14_GstRsbSrcBase' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value

I tried adding -fPIC as I did with the correct includes but that didn't work. Somehow doesn't look as if PIC was the problem anyway.

#7 Updated by C. Leichsenring about 7 years ago

For the record, this is what clang says on Mac OS:

Linking CXX shared library libgstrsb-1.0.0.dylib
Undefined symbols for architecture x86_64:
  "gst_rsb_control_init(_GstRsbControl*)", referenced from:
      _gst_rsb_control_get_type in gst_rsb_control.cpp.o
  "gst_rsb_src_base_init(_GstRsbSrcBase*)", referenced from:
      _gst_rsb_src_base_get_type in gst_rsb_src_base.cpp.o
  "gst_rsb_text_src_init(_GstRsbTextSrc*)", referenced from:
      _gst_rsb_text_src_get_type in gst_rsb_text_src.cpp.o
  "gst_rsb_audio_src_init(_GstRsbAudioSrc*)", referenced from:
      _gst_rsb_audio_src_get_type in gst_rsb_audio_src.cpp.o
  "gst_rsb_sink_base_init(_GstRsbSinkBase*)", referenced from:
      _gst_rsb_sink_base_get_type in gst_rsb_sink_base.cpp.o
  "gst_rsb_video_src_init(_GstRsbVideoSrc*)", referenced from:
      _gst_rsb_video_src_get_type in gst_rsb_video_src.cpp.o
  "gst_rsb_audio_sink_init(_GstRsbAudioSink*)", referenced from:
      _gst_rsb_audio_sink_get_type in gst_rsb_audio_sink.cpp.o
  "gst_rsb_video_sink_init(_GstRsbVideoSink*)", referenced from:
      _gst_rsb_video_sink_get_type in gst_rsb_video_sink.cpp.o
ld: symbol(s) not found for architecture x86_64

#8 Updated by C. Leichsenring about 7 years ago

Adding the verbose make output as suggested by Jan. Will add the output of the Mac OS make that didn't involve me tinkering with the makefile later.

#9 Updated by J. Moringen about 7 years ago

(The following comment is based on the contents of the "gstreamer-1.0" branch at the time of writing)

One problem seems to be the function name gst_rsb_src_base_base_init (note repeated "base") in gst_rsb_src_base.cpp. This probably explains some of the warnings and the undefined symbol in the linking phase.

Actually, the other warnings seem to be similar. For example, the warning produced by gst_rsb_audio_sink.cpp

…/gst_rsb_audio_sink.cpp:80:32: warning: ‘void gst_rsb_audio_sink_init(GstRsbAudioSink*)’ used but never defined [enabled by default]
 G_DEFINE_TYPE(GstRsbAudioSink, gst_rsb_audio_sink, GST_TYPE_RSB_SINK_BASE);
                                ^

is caused by the function name gst_rsb_audio_sink_base_init (again, note added "base" name component) in gst_rsb_audio_sink.cpp.

I didn't check, but maybe the remaining warnings/errors can be explained similarly.

#10 Updated by C. Leichsenring about 7 years ago

Gotta go to a meeting now but because I'm just in front of the mac, I'll quickly attach its logs.

Good catch with the double base though, I'll check that after the meeting.

#11 Updated by C. Leichsenring about 7 years ago

Huh. First of all, there is no indication that GObject abandoned the base_init and class_init distinction or changed the names (https://developer.gnome.org/gobject/stable/chapter-gobject.html). The GStreamer porting guide only mentions base_init when saying "Element metadata and pad templates are inherited from parent classes and should be added in class_init instead of base_init." (http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/random/porting-to-1.0.txt)

When renaming all *_base_init functions to *_init, the warnings go away but the linker error remains the same (see attachment).

But being new to GObject, maybe I misunderstood your suspicion as to what might cause the error. Does this behavior make any sense to you? Sorry, my search somehow failed miserably. Renaming the functions has no effect on the warning apparently.

#12 Updated by C. Leichsenring about 7 years ago

Ok, defining the correctly named init() function with the correct parameter does indeed fix the warning and the linker error. Now I only need to figure out what that function is supposed to do :)

#13 Updated by C. Leichsenring about 7 years ago

  • % Done changed from 0 to 30

Revision 44453567 now builds all the way through. Didn't move anything from the other _init() functions to the no-prefix-init() though. Thanks for spotting this, Jan!

Now on to fixing the runtime behavior ("0% tests passed, 4 tests failed out of 4").

#14 Updated by C. Leichsenring about 7 years ago

  • % Done changed from 30 to 50

Loads perfectly when not including other plugins in GST_PLUGIN_SYSTEM_PATH. If I do, tons of assertions fly that are usually associated with linking to 0.10 and 1.x simultaneously (although all paths are gstreamer-1.0/). Going to narrow that down tomorrow and proceed to functionality test if possible.

Edit: Ok, forget it. The assertions are still triggered, just not in gst-inspect for some reason but in gst-launch. Great, why did I have to try that? That just ruined my night.

#15 Updated by C. Leichsenring about 7 years ago

The audio and video caps have changed. A list can be found at http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/section-types-definitions.html. However, RGB formats aren't well explained there. From the GRAY format something like RGB24_BE would make sense but there doesn't seem to be a FOURCC standard for this ([http://www.fourcc.org/rgb.php#BI_RGB). One change I found in another project indicates that bpp specification is simply no longer needed but I don't know enough about video formats to be sure that this makes sense (https://mail.gnome.org/archives/commits-list/2014-March/msg14641.html). Bits per pixel FOURCC specifications definitely exist for RGB: http://people.xiph.org/~xiphmont/containers/nut/nut4cc.txt

For now, I deem it safer to take the warning and not touch the two remaining formats.

#16 Updated by C. Leichsenring about 7 years ago

One more thing for the public: RSB_GSTREAMER_PACKAGE does not seem to get set anywhere. CMake is supposed to set this in build/config.h but this doesn't seem to happen as the define is empty. I now put a #define PACKAGE "rsb-gstreamer" before the GST_PLUGIN_DEFINE call needing this definition after I found out that the previous #define PACKAGE RSB_GSTREAMER_PACKAGE does exactly nothing. I don't know what PACKAGE was supposed to be, though, or why it worked before when I can't find anything that indicates this variable ever got set. So feel free to change this or do it the proper way in CMake. Or tell me what I missed.

#17 Updated by C. Leichsenring about 7 years ago

And another one. http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/chapter-building-queryfn.html deals with queries, including caps queries. The current code doesn't seem to handle that although most queries are just passed to the parent class that doesn't have an explicit definition of a query handler at all so I'm guessing it's implicitly defined by its parent class? So are caps queries handled there? OTOH, http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/random/porting-to-1.0.txt seems to indicate that caps queries might not have been a thing in 0.10 so maybe I need to add that code? Also, the tutorial above suggests passing all unhandled queries to gst_pad_query_default which does not happen AFAICS but again, that might happen behind the scenes.

#18 Updated by C. Leichsenring about 7 years ago

So cmake problems were PEBKAC.

rsbaudiosink

gst-launch-1.0 audiotestsrc ! rsbaudiosink segfaults. I plastered pretty much all functions with debug log messages but the only function that seems to get called is gst_rsb_init. None of the _init functions in the base classes or rsbaudiosink itself ever gets called. The stacktrace looks as follows:

(gdb) backtrace
#0  0x00007f3407727c6d in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007f3407c66fe4 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007f3407c6730a in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3  0x00007f34081b29b4 in gst_bus_poll (bus=bus@entry=0xe5c190, events=events@entry=GST_MESSAGE_ANY, timeout=<optimized out>) at gstbus.c:1082
#4  0x0000000000404598 in event_loop (pipeline=0xe821e0, blocking=blocking@entry=1, do_progress=do_progress@entry=1, target_state=target_state@entry=GST_STATE_PAUSED)
    at gst-launch.c:509
#5  0x0000000000403661 in main (argc=4, argv=0x7fffcfc4bfb8) at gst-launch.c:1044

Debug output is attached.

rsbaudiosrc

gst-launch-1.0 rsbaudiosrc ! fakesink seems to have a less serious problem (so far at least), as indicated by the following debug output:

0:00:11.036913279 27235       0xb1ba00 DEBUG       GST_ELEMENT_PADS gstutils.c:1102:gst_element_get_compatible_pad:<fakesrc0> Could not find a compatible unlinked always pad to link to rsbaudiosrc0:src, now checking request pads
0:00:11.036936168 27235       0xb1ba00 DEBUG               GST_CAPS gstutils.c:2761:gst_pad_query_caps:<rsbaudiosrc0:src> get pad caps with filter (NULL)
0:00:11.036961419 27235       0xb1ba00 DEBUG                  query gstquery.c:680:gst_query_new_custom: creating new query 0xb7f2d0 caps
0:00:11.036985193 27235       0xb1ba00 DEBUG               GST_PADS gstpad.c:3494:gst_pad_query:<rsbaudiosrc0:src> doing query 0xb7f2d0 (caps)

So pad negotiation is somehow messed up but I didn't look into this because of the issue with rsbaudiosink.

#19 Updated by C. Leichsenring about 7 years ago

The problem wasn't that the functions don't get called but that the debug output doesn't output. That's what the GStreamer-CRITICAL **: gst_debug_log_valist: assertion 'category != NULL' failed were telling me. Since I couldn't figure out what about the debug category registration suddenly failed, I'm resorting to good old fprintf().

#20 Updated by C. Leichsenring about 7 years ago

http://stackoverflow.com/questions/20805166/gstreamer-debug-output-over-multiple-files-for-a-plugin

I'm only using one category for all elements instead of one category per element. If there was some reason to do it otherwise, please tell me :)

#21 Updated by C. Leichsenring about 7 years ago

  • % Done changed from 50 to 60

rsbaudiosrc officially works!

#22 Updated by C. Leichsenring about 7 years ago

  • % Done changed from 60 to 80

Audio is working, bitches!

Disclaimer: didn't test more than one channel or any other corner cases yet.

#23 Updated by C. Leichsenring about 7 years ago

  • Assignee changed from C. Leichsenring to J. Moringen
  • % Done changed from 80 to 90

For now, it seems the only remaining problems are caps problems for the video elements, possibly the ones I already mentioned above.

Jan, can I kindly ask you to take over from here? I really now next to nothing about the video stuff and I also can't imagine Thomas would be thrilled if I poured a lot of time into getting up to speed here either.

I think we should do the merging of the gstreamer-1.0 branch back into master together (useful link I found to do the #ifdef stuff more or less automatically: http://stackoverflow.com/questions/23761840/convert-stash-to-working-change-delimited-by-ifdef-sym and GST_VERSION_MAJOR is already set, too). I think it's a good idea if you look over the diff anyway to see if you don't like some of the changes I made or if you see any obvious problems with them and maybe we can then discuss the open questions above, too. Waddaya say?

Edit: If any of the other watchers want to participate, you are of course welcome, too :)

#24 Updated by J. Moringen about 7 years ago

Jan, can I kindly ask you to take over from here?

I cannot work on this at the moment. Please preserve (and potentially maintain) the branch for integration at a later time.

Sorry, thanks for your work.

#25 Updated by S. Wrede about 7 years ago

  • Assignee deleted (J. Moringen)

@Christian: Congrats, that are really good news! This will be important groundwork for upgrading the remaining parts to 1.0!

#26 Updated by C. Leichsenring about 7 years ago

Jan Moringen wrote:

Jan, can I kindly ask you to take over from here?

I cannot work on this at the moment. Please preserve (and potentially maintain) the branch for integration at a later time.

Sorry, thanks for your work.

No worries. I didn't actually mean that you'd have to do it like now. Only that I probably won't get around to do it and since it will have to be done at some point ..... well, someone will have to do it. Also, I'm going to see Thomas tomorrow and I will ask him whether he thinks I should finish this. I think I know what he's going to say but who knows.

But since this looks like it's going to be pushed back, how about integrating the code into the main branch now before too many additional differences are introduced? Some of the changes are even relevant for the 0.10 branch, too.

#27 Updated by J. Moringen about 7 years ago

Some of the changes are even relevant for the 0.10 branch, too.

If this is OK for you, we can look over the changes that are relevant for 0.10 sometime next week.

#28 Updated by C. Leichsenring about 7 years ago

Jan Moringen wrote:

If this is OK for you, we can look over the changes that are relevant for 0.10 sometime next week.

Sure, how about Tuesday?

#29 Updated by J. Moringen about 7 years ago

Sure, how about Tuesday?

Should work.

#30 Updated by C. Leichsenring about 7 years ago

I was kinda hoping you'd suggest a time ;) Anyway, what about 2 PM?

#31 Updated by J. Moringen over 5 years ago

  • Target version changed from rsb-0.12 to rsb-0.14

#32 Updated by J. Moringen over 5 years ago

  • Target version changed from rsb-0.14 to rsb-0.15

#33 Updated by J. Moringen almost 5 years ago

  • Target version changed from rsb-0.15 to rsb-0.16

#34 Updated by J. Moringen about 4 years ago

  • Target version changed from rsb-0.16 to rsb-1.0

Also available in: Atom PDF