Sunday, 9 October 2016

Fun facts about ethernet debugging, number 3+4i in an occasional series

So, I've been writing a driver for an OS-less machine which uses (for reasons you will be relieved that I am not going to go into) a LAN9221 as its ethernet interface.

Now, this chip is moderately spiffy - in particular, it seems Microchip have now encountered every possible bus-related design cock-up and have a handy register ready for reversing out most of them. However, it requires quite careful FIFO handling and turns out to be exquisitely read-sensitive. So far, so good.

So, one thing I tried was doing a Tx test. Send a packet every couple of seconds, read status back out of the status FIFO, report. Check that all is well.

Now, I didn't have a LAN port that didn't have too much chatter on it, so I used an ASIX AX88178 I had lying around to do the capture (and incidentally, what is it with Linux desktops these days that they just can't seem to shut up? You no sooner plug something in than you have kilobits of traffic asking if, against all knowledge and probability, an ethernet interface with no IP is a link back to some mothership or other. Sheesh).

Anyway, you look at the trace in wireshark, and all is well, except that if you send:

0000  68 05 ca 1e 35 18 00 50 9a 00 00 00 08 00 01 02   h...5..P........
0010  03 04 05 76 40 30 00 00 40 00 0b 00 00 10 01 02   ...v@0..@.......
0020  03 04 05 06 07 08 01 02 03 04 05 06 07 08 01 02   ................
0030  03 04 05 06 07 08 01 02 03 04 05 06 07 08 01 02   ................

You get your original packet, but immediately after it, you also get:

0000  40 00 bf ff 68 05 ca 1e 35 18 00 50 9a 00 00 00   @...h...5..P....
0010  08 00 01 02 03 04 05 76 40 30 00 00 40 00 0b 00   .......v@0..@...
0020  00 10 01 02 03 04 05 06 07 08 01 02 03 04 05 06   ................
0030  07 08 01 02 03 04 05 06 07 08 01 02 03 04 05 06   ................
0040  07 08 01 02                                       ....

which is your packet, with 0x4000bfff prepended to it. No status word in the Tx status FIFO, no IRQ_SIS TXE bit, nothing. Weird, huh?

It's a replay, so can't be bad FIFO management (well, not obvious bad FIFO management), and it's not the CPU double-writing or you'd get word dups, not packet dups. If you vary the size of your packet, you find that the first byte is the length of the original, and the third and fourth byte are always something like 0xNfff where N seems to be something to do with the top nybble of your packet length.

So, you try sending two packets back to back. Send:

0000  68 05 ca 1e 35 18 00 50 9a 00 00 00 08 00 01 02   h...5..P........
0010  03 04 05 77 40 30 00 00 40 00 0c 00 00 10 01 02   ...w@0..@.......
0020  03 04 05 06 07 08 01 02 03 04 05 06 07 08 01 02   ................
0030  03 04 05 06 07 08 01 02 03 04 05 06 07 08 01 02   ................

0000  68 05 ca 1e 35 18 00 50 9a 00 00 00 08 00 01 02   h...5..P........
0010  03 04 05 78 40 30 00 00 40 00 0c 00 00 10 01 02   ...x@0..@.......
0020  03 04 05 06 07 08 01 02 03 04 05 06 07 08 01 02   ................
0030  03 04 05 06 07 08 01 02 03 04 05 06 07 08 01 02   ................


0000  40 00 bf ff 68 05 ca 1e 35 18 00 50 9a 00 00 00   @...h...5..P....
0010  08 00 01 02 03 04 05 77 40 30 00 00 40 00 0c 00   .......w@0..@...
0020  00 10 01 02 03 04 05 06 07 08 01 02 03 04 05 06   ................
0030  07 08 01 02 03 04 05 06 07 08 01 02 03 04 05 06   ................
0040  07 08 01 02 40 00 bf ff 68 05 ca 1e 35 18 00 50   ....@...h...5..P
0050  9a 00 00 00 08 00 01 02 03 04 05 78 40 30 00 00   ...........x@0..
0060  40 00 0c 00 00 10 01 02 03 04 05 06 07 08 01 02   @...............
0070  03 04 05 06 07 08 01 02 03 04 05 06 07 08 01 02   ................
0080  03 04 05 06 07 08 01 02                           ........

Sometimes you get two of these curious runt packets, and sometimes one. Awooga! You then spend all sodding night debugging the cursed thing, convinced that your MMU has somehow rewritten the FIFO, or you've accidentally written it a negative length and it's wrapped, or that you're trying to send a packet in the middle of a reset.

Anyway, finally, in desperation, you plug your ASIX adaptor into a different machine, running kernel 3.13, rather than the 3.11 (sheesh, that old?) on your original box and you get your original packets. Two at a time, all fine and dandy - and my onboard adaptors on two different laptops seem to agree with that.

It seems that older ASIX drivers and/or 3.11 will insert comedy packets into your wireshark captures, for fun and profit. Now, the reason I switched to the ASIX in the first place was because I was seeing these runt packets on another interface, so I'm not sure if I blame the ASIX driver or not. My desktop has both 802.11q and IPv6 enabled, so it may be that some component of the network stack is simply failing to cope with an attempt to configure 802.11q and IFF_PROMISC at the same time. Be warned!

Hopefully, if you found this post via google at 3am, you can now change adaptor, go to bed and sleep the sleep of the justly offended.

Gah. Onward to lwip! (which is at least reasonably well-behaved)

Friday, 29 July 2016

Console application for MTAPI

No, this is still not the power measurement post.  Richard will get around to it soon, honest.

We spend a lot of time playing around with TI's CC2538 Zigbee chip running Z-Stack, which means that we spend a lot of time programming other embedded chips to talk TI's Monitor and Test API serial protocol to it.  Needless to say, debugging at more than one CPU's remove can be a tedious exercise, and there's nothing (that I know about, at least) that will allow you to talk MTAPI to a CC2538 from your nice, comfortable Linux development environment.

Enter MTConsole, in the repository.  In a fit of enthusiasm, and a strong desire to be able to script tests, I've put together a Python program to translate to and from MTAPI.  It is limited at the moment, and some areas are, to be blunt, not very pretty.  It falls over with an exception on a parse error, for example, because that was what I wanted when testing the parser.  That will get visited with fire and the sword when it first annoys me in real use.

So far I've put a lot of effort into parsing binary input (from the CC2538) into text, and not much into parsing text into MTAPI commands.  That will be changing as I need more commands (fire and the sword, people), but at the moment it's not useful for much more than proving that your chip is up and talking to the outside world.  Useful as that is, I would eventually like to get to the point where I can script it well enough to mock up complex command sequences.

Please feel free to grab and use as the mood takes you.  Patches, bug reports, comments and suggestions are always welcome, just bring your own sword.

Wednesday, 20 July 2016

Hacking Wireshark for fun and profit

We've been working on some low-power Zigbee sensors recently, based on CC2538 / ZStack 1.2.2 and running Zigbee HA 1.2.1 ; whilst we were doing this it occurred to us that Wireshark could usefully decode a few more of the IAS Zigbee messages into something useful.

So we did that.

It also turns out that some of our clients (and compliance test houses) use the very nice Ubiqua protocol analyser tool

However, we are a Wireshark shop.

Step up Vadim , who contributed a bunch of patches in Wireshark bug 7426. Those had rotted a bit, so I resurrected them and the upshot is that the repository now contains a bunch of things that Zigbee hackers might find fun:

  • Better decoding for Zigbee IAS messages - from Rhodri James.
  • Support for CUBX, TI SmartRF Studio and Ember Insight Desktop file formats as per Vadim's patch.
  • Support for more recent Ubiqua 3 file formats (at least, on the traces I have here).
  • A nasty backdoor mechanism so that you can decode Ubiqua traces which don't contain the TC key transport packet in the trace (Ubiqua stashes this in a separate table, and we pass it round the back to the Zigbee packet dissector).

One day I will get around to trying to push this lot upstream, but I suspect we will want a better way to do the backdoor key transport than the ugly hack I have in there at the moment.

Anyway, if you feel minded, grab it, enjoy and do report any bugs you come across (and I will do another post on power measurement for low-power radio, honest).

Tuesday, 12 April 2016

It's been a while ..

It looks as though the last post on this blog was waaay back in 2013, which goes to show how busy we've been. So, what's happened between then and now? Well, our open source stuff has migrated to github - . There you'll find a bunch of stuff you might find useful, including:
  • The venerable tstools and muddle.
  • Our local variants of ccsniffpiper - a program which allows you to sniff Zigbee with a TI CC2531 USB dongle, and wireshark - which contains some patches from Rhodri that help decode Zigbee HA IAS ACE commands in a more friendly way.
  • The current version of kbus - still 10% the size of kdbus :-)
  • upc2 - an easy to use, easy to cross-compile, terminal program that can speak xmodem and Andrew Gordon's semi-proprietory grouch protocol.
And lately we've been playing with some automotive powertrain stuff, some very low power Zigbee sensors based on TI ZStack Home 1.2.2a (under some circumstances, you can get battery life projections as long as the shelf life of the batteries), BT LE dynamic advertising with Android, and TI's newer, even lower power CC2650 - but those will have to be the subject of their own posts.

Wednesday, 9 October 2013

Changing the "theme" in Enthought Traits UI with Qt as the backend

I'm currently writing code for a customer using Enthought Traits, and in particular Traits UI. Traits as a whole is a very nice toolkit for use in scientific programming, but you can find out about that by googling in the normal manner.

The specific issue, though, is  that we've moved to using pyside/Qt4 as the GUI backend, instead of wx(Python) - enabled, for instance, by doing:

        from traits.etsconfig.api import ETSConfig
        ETSConfig.toolkit = 'qt4'
    except ValueError:

Some things are done a little bit differently with the Qt backend, though, and it's not always obvious how. In particular, traits UI has always allowed "theming", and the normal way to do this with the traditional wx backend is using something like:

    from traitsui.api import Theme
    Item('move_to_loading_area_btn', item_theme=Theme('@std:BE5')),

 (note that this is not the syntax for the item_theme argument shown in the documentation, that doesn't seem to work - so this is also "a useful reference").

This doesn't work with the Qt backend. It turns out that the way to do it is to specify a Qt stylesheet, which can be done as a string:

    Item('move_to_loading_area_btn', style_sheet='* { color: red }'),

Useful Qt references are then:
(blogged here because no-one else seems to have described doing this)

Sunday, 29 September 2013

A report on PyConUK 2013

As usual, this was a very enjoyable conference. I had fun, I met lots of interesting people, and I learnt things.
This is mostly a report on the program, because that's the easiest thing to write about, and because some of these notes are for work.

Monday, 26 August 2013

Ambient backscatter radio

It seems that some folk at U. Washington have managed to implement an idea that I've been playing around with for a while - ambient backscatter. Now what we need are far, far better reflectors - their system only really works well at distances of 2-3 feet in free air, and really what we need is something that can get through walls.

On the other hand, if their aerials can be made smaller, they could reflect WiFi signals, which have much higher local intensities than distant TV transmitters and whose reflection characteristics are fairly well known thanks to all the folk doing diversity with them.

I wonder if a semi-intelligent aerial which spots beacon frames in h/w is a good thing to have - on the one hand, it provides a known timing reference for the receiver to try and wake up to, but on the other you will have all of the distance-related wakeup window issues.

The other thing one can do over the UW study in a home hub environment is, of course, to have a pulsed interrogation transmitter - notably if you operate in 868MHz or 2.4GHz that transmitter can be on the same frequency as your ambient radio, so you can work in a wait_for_reflect-pulse-wait_for_result mode, which allows you to get quite high power intensities without interfering too much with WiFi (particularly if you are listening to the WiFi and know when its slots are).

This (in some form) is obviously the right technology for things like window sensors - now you only need to change batteries in one room, not one every sensor.

Anyway, have a link - it's an interesting read: