DMA Transmission Stopping Behavior - Critical Analysis¶
Problem Statement¶
The trace shows that XMSG calls continue to request data transmission after DMA controller has finished transmitting its buffer list. The OS appears unaware that the DMA has stopped, leading to missed transmission opportunities.
Trace Analysis - Critical Findings¶
Observed DMA Transmission Pattern¶
From the trace at 14:58:26.271 - 14:58:27.236:
1. First Frame Transmission (14:58:26.271)¶
CommandTransmitterStart: 0x06D498
KeyValue: 0x0401 BlockToBeTransmitted
ByteCount: 0x0010 (16 bytes)
DATA: [09 00 21 13 00 0E 00 66 00 64 00 00 01 00 DD 14]
→ Sending frame: [09 00 21 13 00 0E 00 66 00 64 00 00 01 00 DD 14]
→ DMA SetTX DMAFlag: BlockEnd, FrameEnd (768)
2. Second Buffer Processing (14:58:26.696)¶
LoadBufferDescription: 0x06D49C
KeyValue: 0x0400 BlockToBeTransmitted
ByteCount: 0x000E (14 bytes)
DATA: [21 00 86 C4 00 66 00 00 00 64 02 F5 01 00]
→ Loading Buffer from 0x0006D49C Key=BlockToBeTransmitted ByteCount=14
3. Second Frame Transmission (14:58:26.698)¶
CommandTransmitterStart: 0x06D498 (RESTART!)
KeyValue: 0x0401 BlockToBeTransmitted
ByteCount: 0x0010 (16 bytes)
DATA: [09 02 21 13 00 0E 00 66 00 64 00 00 01 00 DD 14] (different frame!)
→ Sending frame: [09 02 21 13 00 0E 00 66 00 64 00 00 01 00 DD 14]
4. XMSG Activity Throughout Trace - OS Continues Working¶
XMSG - iFunc='XFRCV' (000015) - Frame receive/listen calls
XMSG - iFunc='XFREA' (000006) - Frame read calls
XMSG - iFunc='XFWRI' (000007) - Frame write calls
XMSG - iFunc='XFSND' (000014) - Frame send calls
XMSG - iFunc='XFGET' (000002) - Frame get/allocate calls
XMSG - iFunc='XFSCM' (000010) - Frame set current message calls
CRITICAL PATTERN: XMSG continues throughout entire trace but DMA stops early!
XMSG Command Analysis - The Smoking Gun¶
Detailed XMSG Flow Analysis (14:57:16 - 14:57:18):¶
Transmission Preparation Pattern (Repeats every ~130ms):¶
1. XFGET (000002) - Get/allocate buffer
2. XFWRI (000007) - Write 8 bytes to 0x1E30 [014B 0004 0102 00XX]
3. XFSND (000014) - Send to port 0x00000000, port 5
4. XFREA (000006) - Read response
5. XFWRI (000007) - Write 20 bytes to 0x00D6 [routing data]
6. XFSND (000014) - Send to port 0xFFFFFFFF, port 1
Key XFWRI Data Patterns:¶
Header Messages (8 bytes to 0x1E30):
- [014B 0004 0102 0001] - Node 1
- [014B 0004 0102 0064] - Node 100 (0x64)
- [014B 0004 0102 0065] - Node 101 (0x65)
- [014B 0004 0102 0066] - Node 102 (0x66)
Routing Data (20 bytes to 0x00D6):
- [0100 0010 0102 0064 0202 0004 0302 0064 0402 0000] - Route to node 100
- [0100 0010 0102 0066 0202 0001 0302 0001 0402 0001] - Route to node 102
The Problem: OS Prepared Multiple Frames But DMA Only Sent 2¶
XMSG prepared frames for: - Node 0001 - Node 0064 (100) - TRANSMITTED - Node 0064 (100) again - TRANSMITTED - Node 0064 (100) again - NOT TRANSMITTED - Node 0065 (101) - NOT TRANSMITTED - Node 0066 (102) - NOT TRANSMITTED
BUT NO MORE DMA ACTIVITY - The DMA controller stops responding after only 2 frames!
Root Cause Analysis¶
The Critical Problem: DMA Controller Premature Stop¶
What Should Happen (Correct Flow):¶
1. DMA processes buffer list sequentially
2. When buffer KeyValue = 0x0000 (end marker), set ListEnd bit
3. Trigger Level 12 interrupt with ListEnd status
4. OS checks for more data to send
5. If more data available:
- OS loads new buffer list
- OS restarts DMA with new CommandTransmitterStart
6. If no more data, DMA remains idle until next start command
What Actually Happens (Bug):¶
1. DMA processes some buffers (2-3 frames)
2. DMA stops prematurely WITHOUT reaching end marker
3. NO ListEnd interrupt triggered
4. OS continues XMSG processing, unaware DMA stopped
5. OS tries to send more data but DMA is unresponsive
6. Data queues up in OS buffers but never transmits
Evidence from Trace:¶
1. DMA Stops Without End Marker¶
- No
KeyValue: 0x0000found in trace - No
ListEndstatus bits set - No
TransmissionFinishedstatus - DMA simply stops processing after 2-3 frames
2. OS Continues Processing¶
- XMSG calls continue after DMA stops
XFWRIcalls write new data to buffersXFSNDcalls request transmission- But no new
CommandTransmitterStartoperations
3. Missing Synchronization¶
- No Level 12 interrupts after DMA stops
- OS not notified of DMA completion
- Buffer management becomes desynchronized
Correct DMA Implementation Flow¶
Buffer List Processing Logic:¶
public void ProcessDMABufferList()
{
var currentAddress = this.dmaAddress;
while (true)
{
// Read buffer descriptor
var descriptor = ReadBufferDescriptor(currentAddress);
if (descriptor.KeyValue == 0x0000)
{
// End of list reached
SetStatusBit(StatusBits.ListEnd);
TriggerTransmitterInterrupt("ListEnd");
break; // Stop here and wait for new start command
}
if (descriptor.KeyValue.HasFlag(0x0400)) // BlockToBeTransmitted
{
// Transmit this frame
TransmitFrame(descriptor.Data, descriptor.ByteCount);
SetStatusBit(StatusBits.FrameEnd);
// Continue to next buffer
currentAddress = GetNextBufferAddress(currentAddress);
}
else
{
// Skip non-transmit buffers
currentAddress = GetNextBufferAddress(currentAddress);
}
}
}
OS Interrupt Handler Logic:¶
public void HandleTransmitterInterrupt(ushort status)
{
if (status.HasFlag(StatusBits.ListEnd))
{
// DMA finished current list
if (HasMoreDataToSend())
{
// Prepare new buffer list
var newBufferList = PrepareNextBufferList();
// Restart DMA with new list
WriteTransmitterDMAAddress(newBufferList.Address);
WriteTransmitterDMACommand(DMACommand.TRANSMITTER_START);
}
// If no more data, leave DMA idle
}
if (status.HasFlag(StatusBits.FrameEnd))
{
// Individual frame completed
UpdateFrameStatistics();
}
}
Implementation Fixes Required¶
1. Ensure Proper Buffer List Termination¶
- Always terminate buffer lists with
KeyValue = 0x0000 - Don't let DMA "run out" of buffers without explicit termination
2. Implement Correct ListEnd Processing¶
- When
KeyValue = 0x0000encountered, setListEndbit immediately - Trigger Level 12 interrupt with
ListEndstatus - Stop DMA processing until new start command
3. Fix OS Buffer Management¶
- OS must monitor for
ListEndinterrupts - When
ListEndreceived, check for queued XMSG data - If more data queued, restart DMA with new buffer list
4. Add DMA State Tracking¶
- Track whether DMA is actively processing or idle
- Don't expect transmission without explicit DMA restart
- Implement proper handshaking between OS and DMA
Debugging Recommendations¶
1. Add ListEnd Detection¶
if (descriptor.KeyValue == 0x0000)
{
Console.WriteLine($"DMA: End of buffer list reached at {currentAddress:X6}");
SetStatusBit(StatusBits.ListEnd);
TriggerTransmitterInterrupt("ListEnd reached");
return; // Stop processing
}
2. Add DMA State Logging¶
Console.WriteLine($"DMA State: {(isProcessing ? "ACTIVE" : "IDLE")}");
Console.WriteLine($"Buffer Queue: {pendingBuffers.Count} buffers waiting");
3. Monitor OS/DMA Synchronization¶
if (xmsgSendCalled && !dmaActive)
{
Console.WriteLine("WARNING: XMSG send called but DMA is idle - missing restart?");
}
Expected Behavior After Fix¶
Correct Flow:¶
1. DMA processes buffer list completely
2. Reaches KeyValue=0x0000, sets ListEnd
3. Triggers Level 12 interrupt
4. OS receives interrupt, checks for more XMSG data
5. If data available: OS prepares new list, restarts DMA
6. If no data: DMA remains idle until next transmission request
7. Perfect synchronization between OS and hardware
This analysis shows the DMA controller is stopping prematurely without proper end-of-list processing, causing the OS to lose track of transmission state. The fix requires implementing proper buffer list termination and ListEnd interrupt handling.