Eventfilters with durations return wrong (amount of) rows?
Hi all, I have a question that’s either a case of "am I not understanding this right?" or a bug in eventfilters.
I’ve noticed 2 things: 1) event filters with durations seem to return the rows after the start of events, and 2) scaling the duration of events does not result in the expected scaling of the number of rows filtered.
For background, in reality I’m using this to mark the edges of TTL-pulses in datasets, but I made an example below that I think illustrates my point.
Running the following code:
% Create a 5s timetable with mock data.
test_table = timetable(seconds(linspace(1/1000, 5, 5000)).’, linspace(0, 1, 5000).’);
% Create an eventtable with 3 events at random time points and a duration
% equal to the table timestep.
test_times = seconds([1.101, 2.2, 3]);
labels = ["rising", "rising", "falling"];
delta_t = test_table.Properties.TimeStep;
lengths = repmat(1*delta_t, 1, 3);
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
% Associate events with timetable.
test_table.Properties.Events = test_events;
% Create a filter to determine the time of the rising edges.
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
% Display number of rows returned by the filter.
disp(height(test_rows));
% Display the times of the filtered rows.
disp(test_rows.Time);
So here I’m making a mock timetable of 5 seconds at 1000 Hz, with mock data in it, and 3 time events: two rising edges, and one falling edge. Each event is given a length of 1/1000s or 1 ms, which is equal to the timestep of the table. Then, I create an eventfilter to extract the rows of the time table at these edges.
Issue 1
The filter returns 2 rows as expected. However, the row times are 1.102 and 2.201, which are the row times after the rows in which the events begin according to the event table. This seems at odds with the description in the documentation for eventtable, which says that "Interval events happen during intervals that start at event times and include all times up to, but not including, the times at the end of the events." (emphasis mine). So shouldn’t these timed events return the row times 1.101 and 2.200 seconds?
As an aside, removing the EventLengths from the eventtable constructor returns the correct row times.
Issue 2
Second, for events with lengths larger than a single TimeStep, increasing the duration by a given factor does not always increase the number of returned rows by the same amount.
In this case, if I replace
lengths = repmat(1*delta_t, 1, 3);
with
lengths = repmat(50*delta_t, 1, 3);
and run the code again:
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
test_table.Properties.Events = test_events;
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
disp(height(test_rows));
the number of filtered rows increases from 2 to 100, as expected.
However, increasing the duration of the events to 100 TimeSteps, changes the number of filtered rows to 199 instead of 200, which is what I was expecting.
lengths = repmat(100*delta_t, 1, 3);
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
test_table.Properties.Events = test_events;
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
disp(height(test_rows));
This seems odd, given that both 50 and 100 TimeSteps should easily fit within the time vector of the table, are exact multiples of each other, do not overlap with other events and the length was based on the TimeStep property of the table.
I’ve locally tested the script in 2024a, but the code examples above ran in 2024b.Hi all, I have a question that’s either a case of "am I not understanding this right?" or a bug in eventfilters.
I’ve noticed 2 things: 1) event filters with durations seem to return the rows after the start of events, and 2) scaling the duration of events does not result in the expected scaling of the number of rows filtered.
For background, in reality I’m using this to mark the edges of TTL-pulses in datasets, but I made an example below that I think illustrates my point.
Running the following code:
% Create a 5s timetable with mock data.
test_table = timetable(seconds(linspace(1/1000, 5, 5000)).’, linspace(0, 1, 5000).’);
% Create an eventtable with 3 events at random time points and a duration
% equal to the table timestep.
test_times = seconds([1.101, 2.2, 3]);
labels = ["rising", "rising", "falling"];
delta_t = test_table.Properties.TimeStep;
lengths = repmat(1*delta_t, 1, 3);
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
% Associate events with timetable.
test_table.Properties.Events = test_events;
% Create a filter to determine the time of the rising edges.
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
% Display number of rows returned by the filter.
disp(height(test_rows));
% Display the times of the filtered rows.
disp(test_rows.Time);
So here I’m making a mock timetable of 5 seconds at 1000 Hz, with mock data in it, and 3 time events: two rising edges, and one falling edge. Each event is given a length of 1/1000s or 1 ms, which is equal to the timestep of the table. Then, I create an eventfilter to extract the rows of the time table at these edges.
Issue 1
The filter returns 2 rows as expected. However, the row times are 1.102 and 2.201, which are the row times after the rows in which the events begin according to the event table. This seems at odds with the description in the documentation for eventtable, which says that "Interval events happen during intervals that start at event times and include all times up to, but not including, the times at the end of the events." (emphasis mine). So shouldn’t these timed events return the row times 1.101 and 2.200 seconds?
As an aside, removing the EventLengths from the eventtable constructor returns the correct row times.
Issue 2
Second, for events with lengths larger than a single TimeStep, increasing the duration by a given factor does not always increase the number of returned rows by the same amount.
In this case, if I replace
lengths = repmat(1*delta_t, 1, 3);
with
lengths = repmat(50*delta_t, 1, 3);
and run the code again:
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
test_table.Properties.Events = test_events;
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
disp(height(test_rows));
the number of filtered rows increases from 2 to 100, as expected.
However, increasing the duration of the events to 100 TimeSteps, changes the number of filtered rows to 199 instead of 200, which is what I was expecting.
lengths = repmat(100*delta_t, 1, 3);
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
test_table.Properties.Events = test_events;
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
disp(height(test_rows));
This seems odd, given that both 50 and 100 TimeSteps should easily fit within the time vector of the table, are exact multiples of each other, do not overlap with other events and the length was based on the TimeStep property of the table.
I’ve locally tested the script in 2024a, but the code examples above ran in 2024b. Hi all, I have a question that’s either a case of "am I not understanding this right?" or a bug in eventfilters.
I’ve noticed 2 things: 1) event filters with durations seem to return the rows after the start of events, and 2) scaling the duration of events does not result in the expected scaling of the number of rows filtered.
For background, in reality I’m using this to mark the edges of TTL-pulses in datasets, but I made an example below that I think illustrates my point.
Running the following code:
% Create a 5s timetable with mock data.
test_table = timetable(seconds(linspace(1/1000, 5, 5000)).’, linspace(0, 1, 5000).’);
% Create an eventtable with 3 events at random time points and a duration
% equal to the table timestep.
test_times = seconds([1.101, 2.2, 3]);
labels = ["rising", "rising", "falling"];
delta_t = test_table.Properties.TimeStep;
lengths = repmat(1*delta_t, 1, 3);
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
% Associate events with timetable.
test_table.Properties.Events = test_events;
% Create a filter to determine the time of the rising edges.
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
% Display number of rows returned by the filter.
disp(height(test_rows));
% Display the times of the filtered rows.
disp(test_rows.Time);
So here I’m making a mock timetable of 5 seconds at 1000 Hz, with mock data in it, and 3 time events: two rising edges, and one falling edge. Each event is given a length of 1/1000s or 1 ms, which is equal to the timestep of the table. Then, I create an eventfilter to extract the rows of the time table at these edges.
Issue 1
The filter returns 2 rows as expected. However, the row times are 1.102 and 2.201, which are the row times after the rows in which the events begin according to the event table. This seems at odds with the description in the documentation for eventtable, which says that "Interval events happen during intervals that start at event times and include all times up to, but not including, the times at the end of the events." (emphasis mine). So shouldn’t these timed events return the row times 1.101 and 2.200 seconds?
As an aside, removing the EventLengths from the eventtable constructor returns the correct row times.
Issue 2
Second, for events with lengths larger than a single TimeStep, increasing the duration by a given factor does not always increase the number of returned rows by the same amount.
In this case, if I replace
lengths = repmat(1*delta_t, 1, 3);
with
lengths = repmat(50*delta_t, 1, 3);
and run the code again:
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
test_table.Properties.Events = test_events;
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
disp(height(test_rows));
the number of filtered rows increases from 2 to 100, as expected.
However, increasing the duration of the events to 100 TimeSteps, changes the number of filtered rows to 199 instead of 200, which is what I was expecting.
lengths = repmat(100*delta_t, 1, 3);
test_events = eventtable(test_times, EventLabels = labels, EventLengths = lengths);
test_table.Properties.Events = test_events;
test_filter = eventfilter("rising");
test_rows = test_table(test_filter, :);
disp(height(test_rows));
This seems odd, given that both 50 and 100 TimeSteps should easily fit within the time vector of the table, are exact multiples of each other, do not overlap with other events and the length was based on the TimeStep property of the table.
I’ve locally tested the script in 2024a, but the code examples above ran in 2024b. time series, eventtable, timetable, matlab, eventfilter MATLAB Answers — New Questions