Enhancement #1888
On-demand compilation of protobuf files is too slow
| Status: | Resolved | Start date: | 05/23/2014 | |
|---|---|---|---|---|
| Priority: | High | Due date: | ||
| Assignee: | J. Moringen | % Done: | 100% | |
| Category: | - | |||
| Target version: | Robotics Service Bus - rsb-0.11 |
Description
When trying to record a complete system correctly with protobuf definitions included in the channel it would be quite cumbersome to manually select the used RST files. Instead it would be nice to just specify the whole RST tree to be loaded. However, this takes ages and is completely impractical in time-constraint environments like robocup. We somehow need a way that enables such a usage more easily.
As a reference, I canceled the compilation process after more than ten minutes.
One observation I could make is that all compilation is done single-threaded.
History
#1 Updated by J. Moringen almost 10 years ago
I need to profile this, but I think I know a bottleneck in the (legacy) protocol buffer compiler code. Since we won't switch to a different implementation right away, one or two optimizations may suffice.
Parallel compilation would probably not help since the bottleneck likely is somewhere else.
#2 Updated by J. Wienke almost 10 years ago
- Priority changed from Normal to High
Ok, even with the reduced amount of proto files for the robocup system, the startup still takes a considerable amount of time. Do you have the time to look at this once?
#3 Updated by J. Moringen almost 10 years ago
- Status changed from New to In Progress
- Assignee set to J. Moringen
- Target version set to rsb-0.11
I think, I have a trivial enhancement that allows loading the entire current RST in ~ 40 s on ferberit (oldish x86 laptop). Would that be sufficient?
#4 Updated by J. Wienke almost 10 years ago
Jan Moringen wrote:
I think, I have a trivial enhancement that allows loading the entire current RST in ~ 40 s on ferberit (oldish x86 laptop). Would that be sufficient?
Any improvement is welcome. And that already sounds like a huge improvement.
#5 Updated by J. Moringen almost 10 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
The improvement should now be deployed in the respective master jobs on corci and citoolkit. Reopen if this is insufficient.
#6 Updated by J. Wienke almost 10 years ago
Thanks, that looks much better. The 20 types are loaded in a few seconds.