Bug #785

Parsing of rsb.conf depends on OS-specific line ends

Added by J. Wienke over 12 years ago. Updated over 12 years ago.

Status:ResolvedStart date:01/04/2012
Priority:LowDue date:
Assignee:J. Wienke% Done:

100%

Category:Configuration
Target version:0.6

Description

Original comment:

I set up rsb on Linux as well as Windoze (in that order) and once the rsb_informer/rsb_listener example worked, went on and just copied the Lunix-style rsb.conf file over to Windoze.

However, it seems the configuration file is not parsed at all with "unexpected" line ends and indeed no error message is provided. IMHO this should be changed to be OS-independent or - lacking the former - at least point out an invalid configuration file to the user.

LastTest_Unix.log - Unix run PASSED (18.3 KB) Anonymous, 01/05/2012 06:52 PM

LastTest_Macintosh.log - Max run FAILED (18.6 KB) Anonymous, 01/05/2012 06:52 PM

rsb_informer_output_MAC.txt Magnifier - rsb_informer output without errors (117 Bytes) Anonymous, 01/05/2012 06:52 PM

rsb_version_verbose.txt Magnifier - rsb_version --verbose output (2.41 KB) Anonymous, 01/05/2012 06:52 PM

rsb.conf - rsb.conf. Settings are not read by rsb_version. (168 Bytes) Anonymous, 01/05/2012 06:52 PM


Related issues

Blocked by Robotics Systems Commons - Bug #784: Parsing of config files depends on OS-specific line ends Resolved 01/04/2012
Blocked by Robotics Systems Commons - Bug #801: ConfigFileSource does not raise errors for unexpected lin... Resolved 01/13/2012

History

#1 Updated by J. Wienke over 12 years ago

  • Description updated (diff)

#2 Updated by J. Wienke over 12 years ago

Ok, now that it is clear that the parsing in RSC is not the cause for this, we need to find a different reason.

Robert, where exactly did you place the config file, how did you launch the programs and what was the error message you received?

#3 Updated by Anonymous over 12 years ago

I placed rsb.conf in my user directory as C:/Dokumente und Einstellungen/rabel/.config/rsb.conf (This should be %HOMEDRIVE%/%HOMEPATH%/.config/rsb.conf IIRC; had to look in the RSC source code for that one, too).

The spaces did not seem to cause any problems, as I saw that my program actually read the file completely (i.e. all bytes of it, whereas it aborted the read when the file was not present) using ProcessMonitor.

I launched rsb_informer and rsb_listener from the command line, there were no errors whatsoever. Do your questions suggest you cannot reproduce or you cannot find where the line ends are treated OS-specific? Because I'm pretty sure the problem is

277      boost::filesystem::ifstream stream(path);

in ParticipantConfig.hpp. Supposedly, this will use the default openmode, which is text-mode, which will use OS-specific line endings. However, this begs the question why RSB doesn't choke on one big line of text it cannot parse...

#4 Updated by J. Wienke over 12 years ago

  • Priority changed from Normal to Low

I see your point, however I don't understand why, if this is the problem, the unit tests of RSC work without problems on Windows:

Have a look at: https://code.cor-lab.org/projects/rsc/repository/entry/trunk/rsc/test/rsc/config/ConfigFileSourceTest.cpp line 65. This is the test I enabled yesterday. It reads a config file with windows and one with unix line endings also using an ifstream and checks that several values contained in there are actually found. The test is executed on windows and linux and works on both operating systems. Please note that boost::filesystem::ifstream is only a very tiny wrapper around std::ifstream to enable the use of filesystem::paths. No other behavioral modifications are done in that wrapper. To my mind we are exactly testing your case there when the unit test is executed on windows. When I am in my office I will try to find some time to really test the RSB programs.

#5 Updated by J. Moringen over 12 years ago

Further suggestion: Please to run RSC's unit tests and check whether the ConfigFileSourceEncodingTest succeeds (see description in previous comment).

Also: which Windows version and compiler did you use?

Does converting the newline encoding in the rsb.conf to windows-eols fix the problem or does it occur for any eol-style?

#6 Updated by Anonymous over 12 years ago

Okay, I ran the tests. I now see my mistake when reporting this... I have no idea how it happend, but between my copy-pasting the file contents from one Linux machine to my Windows machine (possibly using NxClient, I don't remember anymore), the line endings were actually changed from Unix style to Macintosh style.

I attached the test results for Unix and Macintosh style line ends (I edited smoke-unix.conf between test runs accordingly). As suspected, Unix-style passes, while Macintosh-style fails. When I re-ran rsb_informer and rsb_listener with my rsb.conf in Macintosh-style line ends, they don't work, but still no error whatsoever.

I attached the logs, my config and sample outputs (without errors) below for your convenience.

#7 Updated by J. Moringen over 12 years ago

I think there are two errors in the LastTest_Macintosh.log file:

First:

CMake Error: The source directory "c:/Users/languitar/workspace/RSC/build/Doxyfile" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.

Not sure where this comes from.

Second:

[ RUN      ] ValidFiles/ConfigFileSourceEncodingTest.testSmoke/0
1325784828812 rsc.config.ConfigFileSource [TRACE]: Option #(global) -> <int>5
string = string
[section1]
option = <double>1.5
unknown file: error: C++ exception with description "bad lexical cast: source type value could not be interpreted as target" thrown in the test body.
[  FAILED  ] ValidFiles/ConfigFileSourceEncodingTest.testSmoke/0, where GetParam() = "smoke-unix.conf" (0 ms)

This could be related to line endings.

#8 Updated by J. Wienke over 12 years ago

Jan Moringen wrote:

I think there are two errors in the LastTest_Macintosh.log file:

First:
[...]
Not sure where this comes from.

This shouldn't be a problem.

Second:
[...]
This could be related to line endings.

That is probably the problem. However, I strongly vote against support the old mac line endings. No current system uses them anymore and treating them is cumbersome because there seems to be no library for this purpose and we would have to implement this manually.

Would it be ok to ignore this case?

#9 Updated by Anonymous over 12 years ago

Johannes Wienke wrote:

That is probably the problem. However, I strongly vote against support the old mac line endings. No current system uses them anymore and treating them is cumbersome because there seems to be no library for this purpose and we would have to implement this manually.

Would it be ok to ignore this case?

If I may add my 2¢ to this:

  • This EOL limitation should be mentioned explicitly in the configuration documentation.
    • I had originally thought that config files from two supported platforms were not interchangeable, which is why I raised the bug in the first place
  • I still do not understand how the parser can actually not raise an error once it encounters this very long line of text of virtually no impact to it. This might actually be another bug, or maybe parsing errors are ignored. I would expect an error message to the user if the configuration file a) did exist, but b) could not be parsed. This doesn't seem to happen.

#10 Updated by J. Wienke over 12 years ago

  • Status changed from New to In Progress
  • Assignee set to J. Wienke
  • % Done changed from 0 to 30

I have updated the wiki page to mention this restriction and commented the respective parsing code in RSC.

#11 Updated by J. Wienke over 12 years ago

  • % Done changed from 30 to 60

I have added a fix to the RSC parser in #801. There should generally be an exception now for old MAC-style EOL. Can you update your RSC installation to this version and see if it now generates an exception in your test case?

#12 Updated by J. Wienke over 12 years ago

  • Status changed from In Progress to Feedback

#13 Updated by Anonymous over 12 years ago

[----------] 2 tests from ValidFiles/ConfigFileSourceEncodingTest
[ RUN      ] ValidFiles/ConfigFileSourceEncodingTest.testSmoke/0
unknown file: error: C++ exception with description "Old Mac EOL style '\r' is not supported." thrown in the test body.
[  FAILED  ] ValidFiles/ConfigFileSourceEncodingTest.testSmoke/0, where GetParam() = "smoke-unix.conf" (0 ms)
[ RUN      ] ValidFiles/ConfigFileSourceEncodingTest.testSmoke/1
1326821745343 rsc.config.ConfigFileSource [TRACE]: Option #(global) -> <int>5
1326821745343 rsc.config.ConfigFileSource [TRACE]: Option #(string) -> string
1326821745343 rsc.config.ConfigFileSource [TRACE]: Option #(section1, option) -> <double>1.5
[       OK ] ValidFiles/ConfigFileSourceEncodingTest.testSmoke/1 (0 ms)
[----------] 2 tests from ValidFiles/ConfigFileSourceEncodingTest (0 ms total)

Seems to be expected. However, the whole test suite is now marked as failing, which would mean RSC behaves incorrectly. I think it should actually behave like DebugToolsTest.testExceptionInfo, i. e. only pass when the exception occurs and not pass if it doesn't and not the other way around. So the test suite should pass when the exception is thrown and fail if it isn't, like so:

[----------] 3 tests from DebugToolsTest
[ RUN      ] DebugToolsTest.testNewInstance
[       OK ] DebugToolsTest.testNewInstance (0 ms)
[ RUN      ] DebugToolsTest.testCreateBacktrace
[       OK ] DebugToolsTest.testCreateBacktrace (0 ms)
[ RUN      ] DebugToolsTest.testExceptionInfo
Exception of type: class std::invalid_argument
Backtrace:
[       OK ] DebugToolsTest.testExceptionInfo (47 ms)
[----------] 3 tests from DebugToolsTest (47 ms total)

#14 Updated by J. Wienke over 12 years ago

This is weird. Did you really update to latest trunk?

Because on our CI server no job shows this behavior:
https://ci.cor-lab.de/view/RSC/job/rsc-trunk/
https://ci.cor-lab.de/view/RSC/job/rsc-trunk-windows/

Also, the respective test code in trunk is:

TEST(ConfigFileSourceTest, testEolError) {
    CollectingHandler handler;
    ifstream stream(str(format("%1%/smoke-mac.conf") % TEST_ROOT).c_str());
    ConfigFileSource source(stream);
    EXPECT_THROW(source.provideOptions(handler), invalid_argument);
}

#15 Updated by Anonymous over 12 years ago

Johannes Wienke wrote:

This is weird. Did you really update to latest trunk?

I'm quite positive on that one... I just redownloaded trunk and svn says I'm at revision 463. It works now. But I'm sure it said the same thing when I tested it... So anyway, it seems to work now :?

Also, the doxygen issue with your profile path is still in there...

[----------] 1 test from WindowsSubprocessTest
[ RUN      ] WindowsSubprocessTest.testSubprocess
CMake Error: The source directory "c:/Users/languitar/workspace/RSC/build/Doxyfile" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
[       OK ] WindowsSubprocessTest.testSubprocess (5219 ms)
[----------] 1 test from WindowsSubprocessTest (5219 ms total)

#16 Updated by J. Wienke over 12 years ago

For doxygen please open a new bug in RSC. However, it's only a minor warning. so low priority should be ok.

Can we close this one now?

#17 Updated by Anonymous over 12 years ago

Yep, this one is resolved for me. Thanks.

#18 Updated by J. Wienke over 12 years ago

  • Status changed from Feedback to Resolved
  • % Done changed from 60 to 100

Also available in: Atom PDF