CS 452/652 Spring 2025 - Lecture 20

Latency and Priorities

July 15, 2025 prev next

Latency Analysis

• compute latency (vs. Märklin latency)
• think of program as graph
  • computation & termination vs. control/service loop
• control/service loop: throughput vs. latency
  • timing of edges: busy vs. block
  • throughput: compare average busy cost of paths to offered load / arrival rate
  • latency: take into account busy and blocked cost

Average Latency?

• no balancing between lower and higher latencies
  • service loop: outliers change mean - potentially skews utility
    • late response is useless, regardless of how late
  • control loop: mean masks outliers - potentially hides catastrophic problem
    • how often a train enters a critical track section matters more than how much it overshoots
• analysis must work with latency distribution
  • service loop: high order percentile (tail latency)
  • control loop: worst-case latency
• exceptions where average latency is relevant
  • sizing of streaming playout buffer

Worst-Case Latency

• worst-case execution time (WCET)
• identify (safety-)critical paths or worst-case execution path (WCEP)
  • high order percentile vs. actual worst-case?
• uncertainty
  • busy: memory pipelining/caching/sharing effects?
  • cache: no miss, one miss, all misses → reality?
  • block: contributes to latency, but little throughput impact
  • internal loops - bounds?
• queueing
  • standing queue - contributes to latency, but does not affect throughput
  • queueing theory: latency increases as arrivals approach capacity
    • cannot "save" capacity during low-arrival times for later high-arrival
    • unbounded arrivals, average over infinity
• alternative model: Critical Instant
  • assume all higher-prio events happen at the same time
    → trace priority execution

Automatic Analysis

• static analysis: NP-hard
• experimental measurement: no proof, no worst-case guarantee!
• hybrid: measure single feasible paths (SFP) & analyze combination
• automatic analysis: loop counts, recursion depth? needs bounds/annotations

Train Control

• critical path: sensor event → compute → action → effect
  • includes Märklin latency
• small tasks: shorter program graph edges
• preemptive scheduling: shortcut to loop start in multi-tasking graph
• preemption: keep uninterruptible kernel execution short!
• priorities: resource management via scheduling

Task Priorities

• priorities are a mechanism to organize various components of a computer system
• critical path: sensor → uart → supervisor → engineer → track → uart → command
  • plus requisite notifiers, couriers, etc.
• critical path? everything is important! what can be bumped down?
  • computation such as routing and path planning?
• at low utilization: priorities not as critical
• priority determines worst-case latency
  • low-level tasks: low priority could loose events or input
  • use higher priority and/or buffer

How to Set Priorities?

• prioritze application-meaningful operations
  • sensor-command activation cycle
  • sensor polling loop
  • user interface
  • route planning, route setup
• mapping operations to tasks
  • problem: same task doing high- and low-priority operations
    • route planning vs re-routing
    • clock services for low- and high-priority tasks
    • train server handling high (stop) and low (go) priority commands
  • static solution: split tasks, e.g, high- and low-priority train server
    • not always practical: ensure atomicity - one command at a time to Marklin
    • servers often have to handle requests from processes with different priorities
  • prioritize messages or operations?

Priority Inversion

• lower-priority tasks indirectly keeps higher-priority task from running
• latency problem: performance and/or correctness
  • unintended starvation, if system fully loaded
• examples: assume lower numbers are higher priorties
• Scenario A
  • Server T₃ running on behalf of T₁, then T₂ preempts
    → T₁ waits for T₂
• Scenario B
  • Server T₃ running on behalf of T₅, then T₄ wants to run
    → T₄ waits for T₅
• Scenario C
  • Server T₃ idle or preempted, T₂ running, then then T₁ sends to T₃
    → T₁ waits for T₂
• dynamic prioritization
  • promote server priority to priority of active request (Scenario A)
  • demote server priority to priority of active request (Scenario B)
  • promote server priority to highest priority queued request (Scenario C)
• kernel can support dynamic prioritization
  • carry sender's priority in internal message
  • also: priority queue of waiting senders?
• further reading: Priority Inheritance Protocols: An Approach to Real-Time Synchronization

Reminder: Train Control Caveat

• track interface is half-duplex
• alternate between reading requested sensor data and sending commands
• train control commands delay sensor loop - options:
  • single-bank sensor queries - budget for train commands
    • additional overhead for query commands
  • alternate between multi-bank sensor query and N train commands (bytes)
  • accept uncertainty of sensor timing based on number of previous train commands
    • ignore additional error, because it is small?

Example: Command Execution Latency

• assume CommandWriter is highest priority server
• latency includes
  • SRR for request (includes a context switch)
  • request write
  • Await interrupt (for CTS) - interrupt handle and another context switch (unblock)
• variability:
  • assume CommandWriter runs immediately after interrupt, because it is highest priority
    • otherwise, we need to consider additional delay due to higher priority task(s)
  • interrupts might occur while CommandWriter is running?
    • can try to estimate/bound number
    • this is why we want interrupt handling to be fast and predictable!
• might need to measure primitive steps
  • have already measured SRR, including context switch
  • easy to measure command writing time, by polling/being interrupted by CTS
  • measure interrupt overhead?
    • in the kernel, using always-asserted interrupt, count and measure N occurrences

Example: Sensor Poll

• track monitor sends poll command to CommandWriter, then waits for data from sensor reader, records result in memory
  • we've already estimated time for command execution
  • after command executes, monitor needs to wait for message from sensor reader
    • need to estimate time for sensor reader
    • in addition, when sensor message arrives at monitor:
      • monitor might be busy handling another request
      • other request messages might be waiting ahead of sensor report
    • even when data is available, monitor may not be running, depending on priorty
• strategy:
  • try to estimate best- and worst- case times, using application knowledge
    • e.g., at most one message from CommandDriver, at most one from train driver?
    • what higher-priority tasks could be runnable?