CANToolz: mod_stat features

Playing with some dumps of CAN traffic I am trying to implement notsmart traffic-detection features in CANToolz.  Main problem - when you see a lot of CAN frames you can't easily find some interesting data and commands. For that purpose I am trying to improve mod_stat module.

My friend from Moscow (@Saplt)  is  preparing before next HardwareVillage  event  where we have small section dedicated to automotive security ( btw it will happen on May 17-16 in Moscow during PHDays conference). So he have sent me some raw CAN traffic from his vehicle,  (mod_stat can save all frames in some ASCII format and now I can replay this traffic by mod_replay here in Berlin). 

Improvements in 1.2-1

First of all I want to improve ASCII detection. Of course ASCII is not common data format, but for VIN text detection and some other things that are in ASCII this feature could work. I have added ASCII detection before generating output, so now this output should looks better...

Also I have found that some frames have "fragmented" structure (because CAN frames are limited to 8 bytes, and that's why vendors trying to use one of these bytes as control/index byte, so you can transfer more data in one "message" by using few CAN frames). Popular format is ISO-TP so mod_stat can "re-assemble" CAN Frames into ISO-TP messages (if they have been found). ISO-TP used also for UDS, and I have added  detection of UDS services. 

But  ISO-TP is not the only one format. I have found some other  formats with "index" byte in CAN data field. Maybe it is also standardized format, but I do not know, so I tried to build common detection of "index"  and data de-fragmentation by using found "index".  This feature works only for CAN Frames that repeating these frames in the loop.

Detected chain:

Here we can see non ISO-TP message, which was re-built.  I think we can play more with some "smart" analysis of CAN traffic, it is just very first tries... Also, if format is not so easy and "loop-detection" mechanism of mod_stat can't find logic then we can manually setup "index" pointer, "index" size and  value. After that mod_stat will try to re-built chain. 

As an additional feature I  have added COMMENTs, so we can do comments for frames by arbitration ID:

This should help with  marking frames while reverse engineering. Both: comments and manual "index" pointers can be exported/imported via META file. So if you want to share traffic with your colleague, you also can share this meta-data. For me it is important feature, because @Saplt in Moscow and I am in Berlin, but we can share  data and work together! Also I believe that both: meta-section and traffic detection algorithms can be improved a lot, current version is just a prototype, but I like it and want to share...

P.S.
Other fixes:
   - mod_stat print format changed. Now it is really formatted...
   - bugs in UDS detection during scanning
   - now mod_stat have ALL traffic, not only statistics. This is good and bad... good because of real traffic capture available, bad because of MEMORY.. Will think how to improve this.
   - fixed bug where WEB interface does not work without the Internet (dammed d3...)