the following tries to get an idea of which technologies would be suitable for a specific (as outlined) distributed/RPC problem. If something is not clear, am am very happy to add more details, but please request these in a comment and not in an "answer". Thanks.

First I will describe the current situation, and then follows what we want to achieve and the actual question. Despite this being a rather long post to get some context, the question itself is rather short (see at the end).

The Application Split challenge

Application description:

The app allows the user to configure a number of hardware devices(*) and then communicate with these to control and collect measurement channels of a physical experiment.

(*) Hardware devices include temperature sensors, pressure sensors, motors, ... Communication ranges from serial port communication, TCP/UDP communication to interfacing with the drivers of 3rd party plugin cards.

Control involves sending commands to the various hardware devices to configure them according to the protocols they support. Measuring involves getting the data from (some of) these devices.

We are hard pressed to keep the whole thing running as customers demand more and more channels at higher sample rates and we have to keep up with writing the data+timestamps we get from all devices to disk, display a subset of the data and still keep the system responding properly.

Current situation:

[ DisplayAndControl.exe      ]

  ||                    /\
  || DLL Interface      ||
  ||                    || Window Messages (SendMessage, PostMessage)
  ||                    ||
  \/                    ||

[ ChannelManager.dll         ]

• ChannelManager.dll (Native C++ DLL on Windows)

  • Manages n data channels (physical measurement variables)
  • Each channel holds a shifting arbitrary number of samples with high-precision timestamps
  • Allows to group channels and write their ongoing updates or historical values ("measurement") to disk
  • Calculations with channels (arithmetic, integration, mean values, etc.)
  • Interfaces with (realtime) hardware devices to get the timestamps and values of channels
  • Get value+timestamp from hardware and save in internal ring buffer for channel

• DisplayAndControl.exe (Native C++ MFC App on Windows)

  • Control the functions of ChannelManager.dll (configure channels and HW devices)
  • Live display current values/timestamps/changes of all channels
  • Graph values of (groups of) channels in diagrams
  • print diagrams and tables of channel values

Summary of current situation:

The application as it is at the moment is already somewhat modular in that the (main) executable does the display+interaction and the (one of several) DLL does the data management (saving of live data to disk, communication with devices, etc.)

From a performance POV, communication btw. the display module and the data management module is optimally performant at the moment.

New situation:

[ DisplayAndControl.exe      ]

  ||                     /\
  || ? RPC/Messaging     ||
  ||                     || ? RPC/Messaging
  ||                     ||
  \/                     ||

[ ChannelManager.exe (same PC or another)  ]

Summary of the envisioned new situation:

For usability, performance and safety reasons, we wish to split up this Windows app into two separate applications, so that the performance (and safety) sensitive ChannelManager module can run as a separate process possibly on a separate Windows PC.

Additionally, since we're already going to split this, we will allow for multiple DisplayAndControl.exe apps connected to one single ChannelManager.exe.

One QUESTION now is what technology we should use to facilitate the communication btw. the now two (or, rather, 1 : small_n) applications.

Performance is important, because a lot of data travels btw. the two applications and latency should be kept to a minimum. It "only" needs to work on Windows, but it should be usable from native C++ only which makes all purely .NET based technologies unattractive. (Note: Porting parts of DisplayAndControl.exe to .NET/WPF is planned, but ChannelManager.exe should stay pure native, as we don't want any .NET stuff running inside this process.)

Regarding latency: It is important that we achieve some level of soft-realtime in the sense that small latency is acceptable, but large and especially varying latency is not acceptable for usability and safety reasons. Therefore any protocol that would help in getting some sort of (soft) realtime behavior would be preferred.

RPC technologies we've looked at:

• WCF (or .NET remoting) - Is dotnet only, therefore not attractive. Performance figures are also not very good.

• (D)COM - COM is great for Windows RPC communication, but it breaks down once you have to have inter-PC comm because it is horrible to get the security settings working in a corporate IT network.

• CORBA - We have had good experience with CORBA communications in the past. The communication is easy to get working; there's not much infrastructure overhead; it works well from C++; writing a .NET wrapper is pretty trivial. The problem with CORBA is that it's somewhat complicated to use correctly in C++ (people will use a lot of time on chasing memory leaks, esp. inexperienced C++ devs). It also will be a learning curve for every developer and every new developer, as no one expects people to "know" CORBA nowadays. Also, it might not perform as well as we'd like it to and as far as I know there's no readily available realtime support.

• Thrift - still looks half-baked to use in our scenario.

• ICE (from ZeroC) - I would prefer ICE over CORBA anytime, after all it promises to be a "better CORBA" and I think it does deliver on that. However, their licensing policy is very suboptimal as they do not sell development licenses but only license per installation. (Well that's what they told us last time we asked end of 2009.) Their licensing policy also suggests that any 3rd party possibly interested in interfacing with our modules would first have to negotiate a license contract with ZeroC too.

• Open MPI - Message Passing interface seems to be targeted at scenarios with lots of clients "heavily" distributed. Doesn't seem to fit our problem.

• Writing our own communication layer using TCP/UDP - Oh my. I'd rather not :-)

• Google Protocol Buffers - Is not an RPC technology.

• Distributed Shared Memory - Well. This got thrown in by a few devs and I for one am neither sure if there's a working implementation nor if it fit's our problem.

So again the QUESTION - what "RPC"-like technology would you prefer in this situation and why?