Communication protocol

[ckoehler]

Hello,

I am looking for some advice about how to structure the communication to a TCP/IP socket on the board.
Ideally, I'd like to be able to send it some kind of header that defines the kind of content that can be expected.

Right now, I use sscanf to get an int, which is the mode, and a float, which is some value. So I can send "3 10.3" and it knows what to do with that.

However, now I am looking into sending some more complicated content in the form of XML that I then parse and do something with. I am considering doing either a new mode and then send the xml, or just assume I get XML if nothing else fits in my switch/case statement.

What are some best practices on structuring the data you send to and from your network apps?

Thanks!

Christoph

[Chris Ruff]

I wouldn't tend to use XML cause it is so verbose, I typically use something like:

sprintf(buf,"%02x%02x%s",function,datalength,payload)

and unpack it using sscanf,

but there are many other ways to structure your TCP data, such as implementations that can contain all binary values (generally more dense and more rigorous in definition and implementation)

You can use the stream stuff to build buffers to send.

Consider your audience (who inherits your code) your density,packet size requirements (XML being the least dense, binary the most dense), speed of packing and unpacking the data, human readability of the data in Wireshark, etc.

Lots of ways to go

Chris

[ckoehler]

Thanks Chris!

The data should be human readable and easy to construct, thus XML. I'd prefer json, but found a nice XML parser that is super fast and lightweight, so we decided on XML.

Looks like I wasn't too far off with sscanf, that's good to know

What do you use for testing the board in isolation? I was thinking something like netcat or so that can just dump some raw data to a socket. On the other hand, writing a short C app that does the same thing shouldn't be hard.

Thanks again!

Christoph

[Chris Ruff]

I always write my test apps in VC++ , but whatever you do, make sure you excercise your board in every way with your test app, script engine, PERL, TCL, (shudder) VB, C#, etc. and cycle your board for a couple of days to find slowly disappearing resources and such

Chris

[ckoehler]

Sounds good, thanks!

Christoph

[tod]

I typically make my instruments SCPI compatible, it's much more taciturn than XML, but given that you already have an XML parser that seems like a good solution also. HOWEVER, (and the forum habitues knew I was going to say this) why don't you use istringstream instead of sscanf It's safer.

Tod

[ckoehler]

Tod,

I actually just read up on istringstream and that's it's safer, so I will probably use that. It seemed non-trivial to parse multiline content with it (like human readable XML), but I can just strip newlines out on transfer: problem solved.

Thanks for the tips!

Christoph

[v8dave]

Actually, I was going to say to avoid using sscanf myself, before Tod beat me to it (after all his brain washing) )

I have used it in the past when I knew that I was creating the strings to send to the unit with my own code so I could test it. If there is any possibility that the input strings are created manually for instance, there is a real issue that it won't meet the input expected by scanf and it can do some very strange things if there is no input for it.

I am slowly working through my code to remove all printfs and sprintf calls so that the only use of these is in any library functions. I still have an issue with a sprintf that causes a strange crash and I am sure that without these calls I will have a nice stable system. It has to be as it will be located in the middle of a jungle with no operator to sort it out for a good part of it's working life.

Dave...

[ckoehler]

Thanks for the tips guys!
One more question: How do you know that your data packet has come in? I mean you listen on a socket and some data comes in. Do you check whether it's what you want? Or do you just read the first x bytes to get info and length about the rest of the packet and read the rest and process? What if you get some junk in at first, it would mess up all your communication.

EDIT: Upon further thinking, TCP already ensures that the data arrives in order and all. So my data being messed up would be a transmission problem, probably user error or a bug. I could use a certain sequence of characters to indicate that data is coming in, too.... decisions decisions...

Christoph

[tod]

I read incoming data in a TCP thread then use a mailbox to tell a separate thread that data has arrived that needs processing. I used to share the buffer but that requires even more coordination so if you have plenty of ram just copy the buffer to the processing thread. My processing thread assumes that a complete command may be split across multiple packets and that a single packet may contain multiple commands. I don't bother with checksums. The nature of my SCPI commands is such that it is highly unlikely that the command will pass through the parser as a valid command if any of the data got corrupted. Every command also has a query form so the GUI can always pass down a command and then do a query to make sure it worked. Any errors in the command string are logged into an error buffer and there is another command the GUI can issued to ask for either the last message in the error queue or the entire error queue.