CSL '0401' Program Binary Disassembly Notes

[Bry5on]

So it looks like the 0x03+ message format is the same for both IDs 0x700 and 0x701

For Byte 0
bits 0-2: cylinder ID (0x700 only reports 001, 010, 011 and 0x700 only reports 100, 101, 110 as expected)
bit 3: no function (for MSS54/HP, this is probably used in the V8 M5 though to ID cylinder 8)
bit 4: knock detected (1 for knock detected)
bit 5: no function identified yet. Could be related to bit 4 with a type of knock detection state.
bits 6 and 7: some sort of evaluation state. So far I only see values of 11 and 00, where the transition happens after cranking the engine. Probably ‘engine running’ or something similar.

I have a datalog recorded that I’ll need to parse tonight to evaluate the other Bytes.

I’ve also flashed a new file to my car with some extra ignition advance to see if I can trip some knock.

[karter16]

Is there likely to be other information that maybe enabled for broadcast on the CAN bus, already in the 0401 code?

Not really - the only ARBID on the main canbus that I haven't entirely figured out yet is 0x770 and that seems to be a mechanism to jump to whatever this sort of multi-frame mechanism is for transferring memory blocks. It's more of a development/memory transfer function I believe than for transmitting values in the traditional sense. I am super curious as to exactly how it works but I don't think it's going to suddenly open up some sort of general mechanism for retrieving more values.

[karter16]

Someone in the old forum already found the message that could be sent on the smg can bus. But detailed information wasn’t published by the time.

Yes I actually remember reading those posts - I'll publish all of that as well when I get the time to go through it, at the moment am trying to round out the main canbus and get all that documented first.

[ac427]

A few weeks ago Bry5on and I briefly discussed that it would be useful, for the purposes of tuning, to be able to broadcast and log knock data over CAN as it requires a per-segment data rate which far surpasses DS2. It turns out that BMW's engineers thought this was a good idea too!

In working through all of the CAN functions, last weekend I uncovered that the purpose of ARBID's 0x700 and 0x701 is to broadcast pre-constructed frames from the Knock Manager module. The messages are turned off by default in the 0401 partial, however various configurations can be enabled by setting K_KA_CAN_AUS.

With nam3forum being down at the time I excitedly fired off an email to Bryson and he was able to quickly check that he could receive messages, along with validating the contents of the messages. My original interpretation based on having quickly looked at it was that ARBID 0x700 was for bank 1 and ARBID 0x701 was for bank 2. What Bryson found from testing was that all cylinders were being broadcast on 0x700. Further investigation of the code reveals that 0x701 appears to only be used in certain configurations (Bryson will test this and confirm back).

Exciting to have uncovered this and very useful to have access to these messages for tuning. Presumably these messages are a leftover from when BMW's engineers were tuning the S54.

Here's what we currently have (will continue to amend and update as we uncover and test more):

Is there likely to be other information that maybe enabled for broadcast on the CAN bus, already in the 0401 code?

[Chris_de]

Someone in the old forum already found the message that could be sent on the smg can bus. But detailed information wasn’t published by the time.

[karter16]

A few weeks ago Bry5on and I briefly discussed that it would be useful, for the purposes of tuning, to be able to broadcast and log knock data over CAN as it requires a per-segment data rate which far surpasses DS2. It turns out that BMW's engineers thought this was a good idea too!

In working through all of the CAN functions, last weekend I uncovered that the purpose of ARBID's 0x700 and 0x701 is to broadcast pre-constructed frames from the Knock Manager module. The messages are turned off by default in the 0401 partial, however various configurations can be enabled by setting K_KA_CAN_AUS.

With nam3forum being down at the time I excitedly fired off an email to Bryson and he was able to quickly check that he could receive messages, along with validating the contents of the messages. My original interpretation based on having quickly looked at it was that ARBID 0x700 was for bank 1 and ARBID 0x701 was for bank 2. What Bryson found from testing was that all cylinders were being broadcast on 0x700. Further investigation of the code reveals that 0x701 appears to only be used in certain configurations (Bryson will test this and confirm back).

Exciting to have uncovered this and very useful to have access to these messages for tuning. Presumably these messages are a leftover from when BMW's engineers were tuning the S54.

Here's what we currently have (will continue to amend and update as we uncover and test more):

[karter16]

Hmm, the idea sounds awesome but personally I’d stick with static settings. A new can of worms sounds a little more scary.

I was thinking about this today and I have to agree with Bryson.

Well against the good advice of both of you I've decided to have a go at this anyway. Mostly because it's something I'll personally find really useful, so it was enough motivation to do it.

I'll take the opportunity to review this work in a couple of days once it's out of my mind and I can look at it again with fresh eyes to see if I'm missing anything.


#1: Define new parameter space variables k_k16_can_cfg, and 64 bytes to store pointers for the 16 bytes of CAN data across the two messages.

k_k16_can_cfg: bit 0 = 7d0 enabled, bit 1 = 7d1 enabled.

The below are the configurable addresses of the RAM variables to copy into the CAN messages
k_k16_can_7d0_b0: ulong at 0x8ef04
k_k16_can_7d0_b1: ulong at 0x8ef08
k_k16_can_7d0_b2: ulong at 0x8ef0c
k_k16_can_7d0_b3: ulong at 0x8ef10
k_k16_can_7d0_b4: ulong at 0x8ef14
k_k16_can_7d0_b5: ulong at 0x8ef18
k_k16_can_7d0_b6: ulong at 0x8ef1c
k_k16_can_7d0_b7: ulong at 0x8ef20
k_k16_can_7d1_b0: ulong at 0x8ef24
k_k16_can_7d1_b1: ulong at 0x8ef28
k_k16_can_7d1_b2: ulong at 0x8ef2c
k_k16_can_7d1_b3: ulong at 0x8ef30
k_k16_can_7d1_b4: ulong at 0x8ef34
k_k16_can_7d1_b5: ulong at 0x8ef38
k_k16_can_7d1_b6: ulong at 0x8ef3c
k_k16_can_7d1_b7: ulong at 0x8ef40


#2: Define new RAM variable k16_can_st (0x00ffec1e) to store the runtime status of the two messages (e.g. they might be enabled in parameter space but disabled by sanity checks in k16_can_init())

k16_can_st: byte

This is our status byte, it is driven from k_k16_can_cfg, but is then modified based on checks that k_k16_can_cfg doesn't equal 0xff (e.g. k16 tune file is not loaded) and is also modified based on the result of the configuration validation checks.


#3: Modify the can_a_slot_cfg_table (0x3cbe8) to replace 771 and 62F with the correct config for 7D0 and 7D1.




#4: Modify the task_10ms function (0x12e9a) to call k16_can_tx_messages() a new version of can_tx_messages() which has extra logic for optionally sending the new messages.





#5: create the new k16_can_tx_messages() function

This modified function includes the 2x new CAN messages, but only if they are enabled and the configuration has passed validation.




#6: create the new k16_can_tx_7d0() function

The handler for 7D0. It looks up each address in the configuration field, then gets the value of the memory location pointed to (e.g. the RAM variable) and pushes the value to the appropriate byte of the CAN message.




#7: create the new k16_can_tx_7d1() function

The handler for 7D1 - works on the same principle as 7D0.



#8: modify the init_cpu function to call an additional k16_init() function

We need to run our k16_can_init() function once on start up to set k16_can_st, and do the validation of the config, etc. The easiest way to do this is to replace the call to an existing function with a call out to a new function that call's both the original function and the new k16_can_init() function.





#9: create the new k16_init() function at 0x3f8a0

Call the original function call we replaced, plus our new k16_can_init() function.




#10: create the new k16_can_init() function

And here's where k_k16_can_cfg is loaded and the configuration of each memory location is validated to ensure it is within range. Any situation which is not valid results in the entire CAN message being disabled.




As I said I'll come back to this to do more logic checking and bug hunting when I have a fresh mind. Looking forward to getting my gauge.s and getting setup so I can play round with this!

[karter16]

Quick update - I've now identified 42 of the CAN-related functions, there's only a few more to go.



I've got more work to do on this and some more to understand, but should be able to get to the point of documenting this pretty-much in full by the time I'm done.

[heinzboehmer]

Yes this was by far the biggest issue in my mind when I first thought about this. The fact that the 68k's RAM is byte addressable though vastly simplifies this issue.

Oh clever! I completely missed that in your previous post, but yes, doing it byte by byte would greatly simplify things.

If there's nothing in the DPR or the RAM that might make things misbehave (not sure why there would be, but I haven't actually looked at the code), then I agree than validating based on address ranges is a viable option.

Don't get me wrong, I'm all for the generalized solution. I just think it's significantly more work to get right.

[karter16]

I was thinking about this today and I have to agree with Bryson.

Adding a new CAN message with statically mapped variables is almost trivial when it comes to potential bugs.

Adding the ability to dynamically assign these variables requires a significant amount of additional effort, if you want to build a truly comprehensive solution. And, of course, more complicated means more bug prone.

Just off the top of my head, these things would need to be addressed by this new dynamic infra:

• Variable address validation
  • How will you make sure the supplied addresses are actually for pertinent variables and not just random memory addresses?
  • What's stopping someone from assigning 0x0 as an address for one of the variables?
• Size
  • What's stopping someone from providing the addresses of 8 words?
  • What's the plan for data overflows?
    • Truncate?
    • Don't send the bad value?
    • Error out and skip sending the entire CAN message?
• Error reporting
  • If there is an error in the provided variable addresses, how do you communicate this?
    • Maybe reserve the first byte in one the CAN messages for flags?
    • Raise a new DTC?
    • Unfortunately, both of those ideas put the validation responsibility on the user, so what's stopping an inexperienced user from incorrectly setting this up, then ignoring the errors and trusting malformed data?
• Testing
  • How will you test this new code? It seems complex enough that (at least) unit tests would be required, but that's not exactly the easiest thing to do with the current development setup.

I think the idea of making this modular is super cool. Unfortunately, I also think that the amount of work required to make this work well is orders of magnitude bigger than just adding a new CAN message with statically mapped variables.

Thank you so much for taking the time to think this through and detail your thoughts, I really appreciate it!

You're absolutely right, this is certainly a bigger, more complex exercise. I wouldn't view this as precluding simply adding a second fixed CAN message as well for those who prefer that. I guess I'm thinking (a) of myself in that it's something I would find quite useful with the tuning work I still have ahead of me and (b) it would also mean that those who wanted to could configure as they prefer without it being reliant on someone (me) crafting different ROMs to suit each purpose.

I've been thinking the last few days about the first couple of items you mention:

Variable address validation
You're right, it would be necessary to ensure that valid addresses were configured. It wouldn't be too bad if someone enters a memory address that references program in that the value, although not useful, would be returned. But if someone references memory that doesn't exist that would cause a page fault at least.

Validating the addresses are valid isn't actually too bad as there are only two address ranges which matter for this purpose. The DPR at 0x00ff8000 through about 0x00ff8400, and the RAM at 0x00ffe500 through 0x00ffefff (although would probably limit this to 0xffe600 through 0x00ffefff as the first 256 bytes of that range are used by the operating system itself and wouldn't be useful to expose on the CAN bus). So essentially each configured address needs to be checked to ensure it belongs to one of those two ranges.

The CPU has a long list of initialization functions it runs once on startup, and I'd inject this additional check there.


Size
Yes this was by far the biggest issue in my mind when I first thought about this. The fact that the 68k's RAM is byte addressable though vastly simplifies this issue.

(FYI heinzboehmer I realize that I don't need to explain to you the below in the great level of detail that I have. I've detailed it to this level for future reference for others :-) )



Let's take the example of AQ_REL. AQ_REL is a word (16 bit) variable which resides at 0x00ff80de



The 68k architecture is big endian, so the most significant byte comes first at 0x00ff80de and the least significant byte at 0x00ff80df.

Now if you want to access the variable the usual way in code is to do something like this:



move.w moves the word (16 bits) to the location specified. But this isn't the only way you can access the variable.

The MC68336, like all 68k processors, is byte addressable, which means that each and every byte in the RAM is individually addressable. Which means you can do things like this:

move.b (A2) =>AQ_REL, aq_rel_msb

This moves the byte at the address AQ_REL (0x00ff80de) refers to into an (imaginary) variable that I named aq_rel_msb for the purposes of this example.

If AQ_REL = 0x8765 then the value that would be in aq_rel_msb would be 0x87.

Likewise we could do (pseudo-code):

move.b (A2) =>AQ_REL+1, aq_rel_lsb

Which would move the byte at the address AQ_REL+1 (0x00ff80df) to the imaginary variable aq_rel_lsb. Using the same example then the value in aq_rel_msb would be 0x65.


For the purposes of configurable CAN messages this means that the entire thing can be simplified to byte addressing. If you wanted to expose AQ_REL on the CAN message you would configure the address parameters as follows:

CAN_Message_A_Byte_0 = 0x00ff80de
CAN_Message_A_Byte_1 = 0x00ff80df

Taking this approach means that no data conversion is needed, and therefore it's not necessary to handle overflows, etc. Simply pass on whatever the value is.


Error Reporting
In terms of error reporting my thinking would be to keep it simple and if any one memory address failed validation then the entire CAN message would be disabled. I appreciate that it's not ideal to not give any feedback to the user, but it would be clear that something was wrong, it would protect against any undesirable behavior, and for the sort of user taking advantage of this it shouldn't be too hard to troubleshoot whether the addresses have been configured correctly or not. No other error reporting would be necessary I don't think.


Testing
Agreed - testing would be manual and difficult. I certainly would be testing this extensively myself before making it available to anyone else as I agree that the risk is higher.


That's just my current thinking, I may or may not look further into this, and I certainly am grateful for you and Bry5on's thoughts on this - regardless of whether I decide to have a go at this or not it's a fun idea to consider, and in the mean time I can certainly whip up a second CAN message for you if you want to let me know exactly what you'd like on it?

Thanks!

[heinzboehmer]

Hmm, the idea sounds awesome but personally I’d stick with static settings. A new can of worms sounds a little more scary.

I was thinking about this today and I have to agree with Bryson.

Adding a new CAN message with statically mapped variables is almost trivial when it comes to potential bugs.

Adding the ability to dynamically assign these variables requires a significant amount of additional effort, if you want to build a truly comprehensive solution. And, of course, more complicated means more bug prone.

Just off the top of my head, these things would need to be addressed by this new dynamic infra:

• Variable address validation
  • How will you make sure the supplied addresses are actually for pertinent variables and not just random memory addresses?
  • What's stopping someone from assigning 0x0 as an address for one of the variables?
• Size
  • What's stopping someone from providing the addresses of 8 words?
  • What's the plan for data overflows?
    • Truncate?
    • Don't send the bad value?
    • Error out and skip sending the entire CAN message?
• Error reporting
  • If there is an error in the provided variable addresses, how do you communicate this?
    • Maybe reserve the first byte in one the CAN messages for flags?
    • Raise a new DTC?
    • Unfortunately, both of those ideas put the validation responsibility on the user, so what's stopping an inexperienced user from incorrectly setting this up, then ignoring the errors and trusting malformed data?
• Testing
  • How will you test this new code? It seems complex enough that (at least) unit tests would be required, but that's not exactly the easiest thing to do with the current development setup.

I think the idea of making this modular is super cool. Unfortunately, I also think that the amount of work required to make this work well is orders of magnitude bigger than just adding a new CAN message with statically mapped variables.

[Bry5on]

What software are you using to read the live CAN messages?

I’m using the built in logger within Gauge.S - great DIYer friendly product.

[ac427]

Beautiful. Thank you again my friend. Will test lambda integrator in the morning as it looks like I got the others sorted .. well, close on relative opening

What software are you using to read the live CAN messages?

[karter16]

A new can of worms

Haha I see what you did there 🤣

[Bry5on]

Hmm, the idea sounds awesome but personally I’d stick with static settings. A new can of worms sounds a little more scary.