Karter16's Silbergrau E46 M3 Journal

[karter16]

I've figured it out!

So the short story is that there is no mechanism in the boot loader to put the DME into programming mode via DS2 any way other than calling a valid flash wipe.

But....

I've figured out how to get fast reads.

Essentially the trick is to read the Free Identifiers block between 0x4000 and 0x5FFF. Then wipe it, and then flash the contents back.

We can actually make this a quick process. There's actually only a couple of sub sections that hold data, but to be on the safe side the way my approach works is this.

1: When the user attempts a long read (e.g. read Tune, read full, etc.) and the connection is not already at 125000 baud, the app checks, for the current DME serial number, does it have a map of the contents of the free identifiers block. If it doesn't it takes a full backup of the block and maps the contents of it. this takes about 15 seconds.
2: Once it has the map it wipes the whole block and just writes back the non-0xFF bytes to save time. (wiping resets everything to 0xFF).
3: Then it proceeds with the read.

Even for a read of the partial/tune binary this makes the whole exercise MUCH quicker.

But we can do even better.

In future sessions when the user initiates a read (for the same DME serial number) and isn't at 125,000 baud it can lookup the previous backup to get the map of the contents. It still will actually read those regions from the DME in case anything has changed ( but only needs to read and write the specific sub-areas of the block that it knows are non-zero. (note I'm accounting for areas that actively change, flash counter, AIF, etc.)

Doing this means that in future sessions, the whole jump to 125000 baud takes approximately 3 seconds. Once it's done the connection is at 125000 baud for the rest of the session!

I've also tested and validated full program writes as well.

Current job is building out the app ui, safety checks, etc.

Few screens below - the UI is still under fairly heavy development - I know there's things that need sorting.








I'm finding this feature particularly useful. Ability to take backups at any point in time and then restore them. Coming in very handy as I build out new features and either intentionally or unintentionally corrupt the data in various blocks.




Expanded session log showing the speed of elevating to 125000 baud.

[karter16]

No point in another screenshot of the same thing, but have built out the following:

- CRC verification & correction for both Program and Tune
- Read and reset the flash counter (with complete backup and restore of the EEPROM block)
- Read the AIF sector (all history)
- Read BRIF/ZIF/ZIF Backup/DIF and verify integrity against backups.
- Flash the "full binary" program blocks (still testing this)

Still struggling to figure out the magic to get the DME into boot mode via DS2 call without erasing a sector first. I'll get there, just need to do some more experimenting. Have messed up a few times, but each time have been able to use the app to self-recover the situation - pretty cool

[karter16]

And now I can do fast writes as well :-)

Seems the DME only supports 125000 baud (BMW-FAST) when the DME is in programming mode, so now figuring out how to harmlessly trigger programming mode to get fast reads without having written something first.

[karter16]

We need to develop a MKIV launcher

Could easily fit all the important options with recreate the low res

Now that would be pretty cool!! I wonder how configurable idrive launcher is - it's been a while since I looked at it.

[0-60motorsports]

We need to develop a MKIV launcher

Could easily fit all the important options with recreate the low res

Yes! That would be awesome and a great solution because i dont see any way this BMW tech can survive much longer in our car LOL.

[Obioban]

2: I've replaced the Xtrons launcher with iDrive Launcher which is a fantastic, well put together, piece of software which further does wonders to make the unit feel like it fits in the E46 interior.

We need to develop a MKIV launcher

Could easily fit all the important options with recreate the low res

[0-60motorsports]

Today I got distracted and built my own app to connect to the DME over OBD/DS2.


Currently it can connect, retrieve system address and query for things like ZIF, AIF, etc.

It can also unlock to enable privileged commands, negotiate to change baud rate etc.

It can also read program and tune binaries.

Next step is to get it flashing 😱

That is amazing mate! Now only if someone would make an app for coding instead of people like me struggling to do simple things such as CSL Shifting Software on and SMG or just basic coding like cluster etc....

[karter16]

Today I got distracted and built my own app to connect to the DME over OBD/DS2.


Currently it can connect, retrieve system address and query for things like ZIF, AIF, etc.

It can also unlock to enable privileged commands, negotiate to change baud rate etc.

It can also read program and tune binaries.

Next step is to get it flashing 😱

[karter16]

I did a final* assembly of the gauge.s enclosure today. I say final with an asterisk as I have a a couple of final dust-coats of paint still to do on the enclosure, which, if it works like last time, will lighten up the finished surface a bit and bring the colour closer to the BMW dash colour.

Anyway - here's the *almost* finished enclosure:







I thought about finishing the entire enclosure, not just the visible face but decided that was ridiculous.


I didn't take photos of preparing the screen, but the steps were basically to take a new display, apply a red film layer, and then a layer of the same anti-glare film that I used on the xtrons. I don't like the shiny look of the gauge.s screen with the red film applied by itself, so decided to apply the anti-glare as well - which makes it look much better.

Here's a few photos of assembly - screen in:



Screen retainer:



Logic board in and sub-cover installed:



Back cover on:



Closeup of the connector:



From the front: this is the second reason why the project is quite completely finished. The red film I received has some micro impacts to it which result in unavoidable bubbles in the anti-glare layer. Not cool, I'll order some more red film and redo this. Carried on with the install cause I wanted to see the end result:






And here's how it looks in the car:









Worth noting that the softness and glow of the digits on the display is the iPhone camera struggling with the contrast, it doesn't actually look soft and dispersed like that in person.

I'm really pleased with how this has come together - couple more minor bits as mentioned to get this completely finished.


I also finally got pure-ftpd setup today (thanks to Bry5on who figured out that pure-ftpd works well with gauge.s's use of some less common ftp commands).

[karter16]

That would be a bit of a problem if the adaptations were reset lol

Maybe it is not a great idea to reset adaptations with the 75C, 70C and 55C thermostats.

I'd say it's more of a problem to not reset.

Whatever rf_psau_i is at the time the engine is switched off will be written to the adaption EEPROM. If rf_psau_i is never updated again (as a result of TMOT = 80 not being hit) then that adaption value will stick for forever. If environmental conditions have changed in the mean time then rf_psau_i (saved as an adaption) could be actively throwing your RF calculation away from where it should be.


I'd suggest for those running a lower temp thermostat to adjust what I've called k_rf_p_saug_i_tmot_min (0xE5EC in the tune file) to a lower threshold temperature.


Edit: I just had a quick look at the other parameters that use tmot as a threshold to trigger various behaviours. Seems most other modules use TMOT == 70 to 75ish. Probably means that those running lower temp thermostats on the street should consider adjusting some of these other parameters as well, depending on where they see TMOT sitting most of the time.

[Slideways]

Yup it'll not update and rf_psau_i will be stuck at whatever it was last set at as an adaption. Which, depending on whether conditions have changed, will either hinder or help.

That would be a bit of a problem if the adaptations were reset lol

Maybe it is not a great idea to reset adaptations with the 75C, 70C and 55C thermostats.

[karter16]

Hmm, what happens if the thermostat is stuck open and it never reaches 80C? I really should get to fixing that

Yup it'll not update and rf_psau_i will be stuck at whatever it was last set at as an adaption. Which, depending on whether conditions have changed, will either hinder or help.

[Slideways]

Hmm, what happens if the thermostat is stuck open and it never reaches 80C? I really should get to fixing that

[karter16]

It was raining heavily today so I drove to work and took the opportunity to run some logs.

I've written previously about the CSL MAP sensor and its function. As a quick recap it performs 2 functions during normal operation. One is to act as a kind of running offset between calculated actual conditions and the AlphaN map, for this reason the MAP integrator value rf_psau_i is stored as a adaption in EEPROM. The second thing is does is act more instantaneously to address variations in realtime.

It's fairly commonly known that the integrator is limited to +-2.5%. As I've noted before it's actually +-2.5% of max RF, which in the range the integrator is active in is actually more like +-10% of current RF.

I've previously hypothesised that in normal operation the integrator doesn't bounce off its limits anyway, I just haven't actually got around to posting any evidence of this. So here we go

This is a normal drive home from work. rf_psau_i integrator is always applied to final RF, but the integrator is only updated once engine temp reaches 80 degrees, hence the straight line at the start of the log.

What we can see more generally though is that once the TMOT condition is reached the integrator is very active. We can also see that it hasn't got anywhere near its +-2.5 limits, sticking between +1.094 and -1.205.



I thought another way to understand how much of the time the MAP sensor is actively correcting RF would be to look for the percentage of time it's changing. So here we go:

Once TMOT=80 is reached in the log above there are 78,859 records captured at approximately 100Hz frequency.

Of the 78,859 records, 41,655 of them have an rf_psau_i value that differs in value from 10 records (100ms) earlier. By this measure the MAP integrator is active approximately 52.8% of my drive home. If we extend the measure to look where it differs in value from 100 records (1 second) earlier, the figure rises to 64.4%.

In short - for round town and highway commuting the MAP sensor is going to have a significant effect :-)

[karter16]

Life continues to be busy, have managed to make some progress in a few places.

I've finalised the design of the Gauge.S enclosure and done a final test assembly (see photos below). I have the final prints too ready to be sanded and painted.







I've also been putting the dev version of Community Patch V2 to full use. I've been doing more work on understanding the behaviours of the cylinder air mass calculations specific to the CSL and looking into a couple of interesting curiosities. I've been maxing out the 3 custom CAN messages as well as pulling across slave-only parameters. A lot of this investigation I'm doing would be pretty much impossible without the flexibility of being able to pull, at high data rate, the specific variables I need.


I also received my second-hand E86 front triangulation braces that I ordered while I saw them for cheap. Looking forward to getting heinzboehmer's front triangulation brace together and installed when I get the chance!