another tuning tool

[WASyL]

awsome tool, comes handy when doing lots of tuning with different ECM/PCM but similar engine setups.

[steveo]

please test it. one thing i know is broken is 2d table interpolation. if you want to do 2d table interpolation for now just make a 3d table with a single column and it'll work, rather than making a 2d table.

next version:

- make csv parser more robust and faster
- use log timestamp if available
- XDF table layout import (since you can't copy paste that stuff from tunerpro, it takes too long to make them)

[steveo]

http://ecmhack.com/tablehack/

new version:

- way better way faster CSV import. parses, error checks, and builds its table structures pretty quickly, average under 200msec per MB of log data on an i5, so even big 10MB+ logs should load in seconds. as a 64 bit program you could load gigs of logs if you wanted to, i've only tested up to 500mb of test data. added a progress bar for massive csv files so you know it has not crashed. handles backslashes, quotes, and literal quotes (""=") so it should work with any standards compliant csv generator. it still parses things in quotes as numbers if they are numbers, though (actually it stores two copies of your log, one as strings one as converted numbers where strings = 0.00)

- fixed some glitches with the table viewer, scrolling through hundreds of thousands of lines is pretty smooth

- some options for datalog import, optionally continue parsing a CSV with line bugs (columns per line != columns in header, etc), i know some log generators are super broken.

- enabled time axis (for lag filter in analyzer as well as grapher), must be a decimal time to work, will add a timestamp converter later.

- XDF importer. select your xdf and it will list any tables it feels are viable. seems to work well against EEX but please test! it will probably crash with a malformed XDF but who knows.

so now it'll load log(s), load xdf table, select data, and analyze anything vs anything vs anything filtered by anything with arbitrary data lag in a few clicks and a few seconds.

next up are more log viewing improvements like selection data tracing (for example if you select analysis cells of interest it'll highlight or filter your log appropriately) so you can view the parameters of knock events for example. you will then be able to select entries in the log and they will move a cursor in the graph if you have one, so you can 'drill down' into events more easily.

[steveo]

i am working on a new version that will do much more advanced analysis for tables that the ECM does linear table lookups on (think most VE and maf tables)

doing all this linear/bilinear lookup stuff trying to make it 'think' like an ECM got me thinking

lets define an example table that has 20,40,60,80,100 as columns.

the traditional method, lets say we have a data point of 6 with a lookup value of 25, we go okay, the first cell is close to that value, so, add the data point 6 to the first cell's average. in otherwords we do nearest neighbour interpolation of the data only.

the results are good on a large sample set because of a crapload of averaging smoothing the results

but this is not really how the data point would be seen by the ECM for a table like a VE table that the ECM does linear interpolation on.

what we actually are saying when we log a lookup value of 25, is that we have a data point that affects LINE that has both its slope and gain defined by the values of the first two cells of that table

so in effect what we should do is calculate gain and slope of that line by manipulating the adjacent 2 (or 4 in the case of a 3d table) cells for each data point

does that make sense?

[kur4o]

on motorola cpu build in table lookup opcodes. you need to have not only predefined decimal points, but need to have it hex.

the value used for lookup is usually set with min max as 00-ff or for signed ones and 16bit tables 0000-8000 or 0000-ffff than values are converted to account for that and being used in table lookup. if x-row is scaled from 00-ff 00=400; ff=3600 with some divider of lets say 16 points. lookup value is divided by 16 that it finds the row and between 2 adjacent cells make an average between the 2 cells and multiply with the factor that is left from devision. Hope it that makes sense.

That might not be the case with ppc cpu, or some other cpu that uses scalars set for each table that defines the axis points[ so it is not linear].


I hope to get you some decomplied routines used with PPC cpu so you can figure something out of it. Usually all gm 2d and 3d tables have scalar infront of the table that defines the number of x and y points and are used for devision.

The most newer stuff have mostly floating point tables, there may be totally different math.

some examples for linear motorola tables

100-200-300

add points from 100- to 200 it wil be best if you know how many points are in hex. but that may be too hard to guess.

so you get 100~110~120~130~140~150~160~170~180~190~200

value at 100=35 value at 200 = 70

so you find the spread between 70 and 35 in our case it is 35 and multiply with factor 1.1 for 110 , 1.2 for 120 and so on till 190 with *1.9

first draw horizontal axis than extrapolate vertival axis using newly added extrapolated x axis data.

I hope any of that make sense.

[kur4o]

Some clarifications.

when you find spread lets say 20 and 100 spread is 80 you got 10 extrapolated points between

so for point 0[20] it is 80*0+20
point1 is 80*0.1+20
point2 is 80*0.2+20
and so on

[steveo]

totally makes sense, although i think if i do the math in floating point, the results will be just as usable when scaling tables from lower resolution ecms. i'll have to do some tests on how well it works on a real VE table or whatever. when you have two axes (for a 3d table) things get a bit more complicated. https://en.wikipedia.org/wiki/Bilinear_interpolation

this is a great approximation that i've been using with success, the compiler reduces it to a few instructions and it runs really quickly

  double x2x1, y2y1, x2x, y2y, yy1, xx1;
  x2x1 = x2 - x1;
  y2y1 = y2 - y1;
  x2x = x2 - x;
  y2y = y2 - y;
  yy1 = y - y1;
  xx1 = x - x1;
  return 1.0 / (x2x1 * y2y1) * (
        q11 * x2x * y2y +
        q21 * xx1 * y2y +
        q12 * x2x * yy1 +
        q22 * xx1 * yy1
        );

[steveo]

ok i really suck at math and i wish i had paid attention in linear algebra but i THINK figured out how to effectively reverse 3d linear interpolation of a table lookup without having to cover an entire whiteboard in math. it's about four times as computationally intensive as just a lookup as i have to do four transforms, one for each 'encompassing cell' involved in the transform, as from each cell i change the 'viewpoint' of the 3d shape that effectively joins the four encompassing cells involved in the lookup. someone better at math might be able to do it more efficiently later but this seems to work. i will make it less crashy and do more testing. early results are very promising. data scattered mostly between the cells seems to resolve as expected.