Ruminations of J.net idle rants and ramblings of a code monkey

Bug in StreamInsight 1.2 …

StreamInsight

This came up recently on the forums and I’ve been meaning to blog about it and, finally, I’m doing it.

Before I get going let me just say the following things:

  1. There are a couple of ways to work around the bug.
  2. It is fixed in StreamInsight 2.0. It does not apply to StreamInsight 1.1. Only 1.2.
  3. The scenario that triggers the bug is, I feel, a pretty narrow and uncommon one.

Symptoms

Since this happens in the Simple StreamInsight App that I posted on MSDN Samples, that’s what we’ll use. In certain cases, a StreamInsight query will crash on the initial event. You will be able to stop and restart the (now aborted) query from the Query Debugger or any custom tools that use the StreamInsight API to restart queries and it will run fine. This only happens on the initial events. The query debugger shows that the exception for the aborted query is:

Microsoft.ComplexEventProcessing.Engine.OperatorExecutionException: An exception happened when operator 'Grouping.1.1.Aggregate.1.1' was processing event, check inner exception for more details. ---> System.ArgumentOutOfRangeException: The added or subtracted value results in an un-representable DateTime.
Parameter name: value
   at System.DateTime.AddTicks(Int64 value)
   at Microsoft.ComplexEventProcessing.Engine.DateTimeExtensions.AddNoOverflow(DateTime dateTime, TimeSpan timeSpan)
   at Microsoft.ComplexEventProcessing.Engine.DateTimeExtensions.SubtractThrowIfOverflow(DateTime dateTime, TimeSpan timeSpan)
   at Microsoft.ComplexEventProcessing.Engine.SynopsisManager.SingleHoppingWindowManager.PreviousWindowVs(DateTime timestamp)
   at Microsoft.ComplexEventProcessing.Engine.SynopsisManager.SingleHoppingWindowManager.OverrideCtiPushBackward(DateTime ctiTimestamp)
   at Microsoft.ComplexEventProcessing.Engine.ExecutionOperatorWindowBasedOrderPreservingMinus.ProcessSmartCti(EventReference& eventReference, Int64 stimulusTicks)
   at Microsoft.ComplexEventProcessing.Engine.ExecutionOperatorStateful.DoProcessEvent(EventReference& eventReference, Int64 stimulusTicks)
   at Microsoft.ComplexEventProcessing.Engine.QueryExecutionOperator.ProcessEvent(Int32 streamNo, EventReference& eventReference, Int64 stimulusTicks, Int64 enqueueSequenceNumber)
   --- End of inner exception stack trace ---
   at Microsoft.ComplexEventProcessing.Diagnostics.Exceptions.Throw(Exception exception)
   at Microsoft.ComplexEventProcessing.Engine.QueryExecutionOperator.ProcessEvent(Int32 streamNo, EventReference& eventReference, Int64 stimulusTicks, Int64 enqueueSequenceNumber)
   at Microsoft.ComplexEventProcessing.Engine.ExecutionOperatorStateful.ProcessEvent(Int32 streamNo, EventReference& eventReference, Int64 stimulusTicks, Int64 enqueueSequenceNumber)
   at Microsoft.ComplexEventProcessing.Engine.QuerySegmentInputStrategy.DispatchEvents(SchedulingPolicy policy)
   at Microsoft.ComplexEventProcessing.Engine.SchedulingPolicy.DispatchEvents()
   at Microsoft.ComplexEventProcessing.Engine.DataflowTask.OnRun()
   at Microsoft.ComplexEventProcessing.StreamOS.Task.Run()
   at Microsoft.ComplexEventProcessing.StreamOS.Scheduler.Main()

The most interesting – and, for me – perplexing is the “The added or subtracted value results in an un-representable DateTime”. Huh? How is that happening? Since this happens right at start up, we don’t have the time to connect to the query and record the events as it starts so we need to use trace.cmd (see here, about halfway down) to set up the query trace when the application starts. Once we do that and open the trace for the query, we see that the only event in the query is a CTI with a time of negative infinity … or DateTime.MinValue.

image

Conditions

This will only happen when you are using IDeclareAdvanceTimeProperties in your adapter factory or AdvanceTimeSettings when you create the stream, and specify that CTIs are created by event count, not by timespan. The delay and the AdvanceTimePolicy don’t seem to make any difference. Next, the query needs to have a HoppingWindow and no other query operators that alter the event lifetime or duration. Tumbling and count by start time windows don’t have any problems.

Fixes/Workarounds

There are a few ways that you can work around this. Any one of the following methods will work.

  1. Specify that CTIs are created by timespan rather than event count.
  2. Add a temporal operator into the stream before the window. For example, adding AlterEventDuration(e => TimeSpan.FromSeconds(0)) will work.
    var hoppingWindowStream = from s in sourceStream            .AlterEventDuration(e=>TimeSpan.FromSeconds(0))group s by s.DeviceId into aggregateGroupfrom item in aggregateGroup.HoppingWindow(
    TimeSpan.FromSeconds(10), TimeSpan.FromSeconds(2),
    HoppingWindowOutputPolicy.ClipToWindowEnd)
    select new AggregateItem{
    DeviceId = aggregateGroup.Key,
    Average = item.Avg(e => e.Value),
    Count = item.Count(),
    Sum = item.Sum(e => e.Value)
    };
  3. Enqueue a CTI from your adapter. This can have any date/time for the value, as long as it is greater than DateTimeOffset.MinValue + [WindowHopSize]. You can, for example, use new DateTimeOffset(1776, 7,4, 0,0,0,TimeSpan.FromTicks(0))

  4. Upgrade to StreamInsight 2.0 now that it’s released.

Comments are closed