OxQL: surprising "input tables to group_by must be aligned" error when no data

[david-crespo]

I ran a query like this:

get virtual_machine:vcpu_usage | filter timestamp >= @2025-03-03T20:50:11.000 && timestamp < @2025-03-03T21:00:31.000 && instance_id == "cd434a6f-815f-446d-9e32-744c991b25ed" && state == "run" | align mean_within(10s) | group_by [vcpu_id], sum

And got errors like the below despite the manifest presence of align mean_within(10s) in the query. These are dogfood request IDs.

{
  "request_id": "38890de5-8cf3-49ab-bdf5-710956f0bb38",
  "error_code": "InvalidRequest",
  "message": "Input tables to a `group_by` must be aligned"
}

{
  "request_id": "5252c89f-07a6-44ba-b77f-373e6218fbce",
  "error_code": "InvalidRequest",
  "message": "Input tables to a `group_by` must be aligned"
}

I see that the error comes from here. I will do more investigation and really nail down exactly what query I ran to get it, but I don't think it's an issue with query construction. You can see that the console code to generate queries is written so that there is always an align mean_within(<interval>) present.

omicron/oximeter/db/src/oxql/ast/table_ops/group_by.rs

Lines 54 to 57 in ff715c8

	anyhow::ensure!(
	table.is_aligned(),
	"Input tables to a `group_by` must be aligned"
	);

This was an instance that had recently started, so I wonder if the problem was that there were no timeseries in the table, which does get you an is_aligned value of false.

omicron/oximeter/oxql-types/src/table.rs

Lines 336 to 349 in ff715c8

	/// Return `true` if all the timeseries in this table are aligned, with the
	/// same alignment information.
	///
	/// If there are no timeseries, `false` is returned.
	pub fn is_aligned(&self) -> bool {
	let mut timeseries = self.timeseries.values();
	let Some(t) = timeseries.next() else {
	return false;
	};
	let Some(alignment) = t.alignment else {
	return false;
	};
	timeseries.all(|t| t.alignment == Some(alignment))
	}

Update: yep, I can systematically reproduce the error by changing the time range so that it doesn't contain any data points. This works for a time range both before and after the lifetime of the instance.

2025-03-03-group-align-error.mp4

[bnaecker]

Might want to handle empty tables at the entry to a group-by, which would return an empty table with the right schema. An argument could also be made for is_aligned() returning true for empty tables, but IMO that's more surprising.

[bnaecker]

This is more complicated than it seems at first because of the "with the right schema" bit. We don't have the schema in the current implementation at the point we run the group-by, we infer it from the data as needed. If we don't have any data, we can't do that.

The solution is to make the expected schema available here, which would probably best be done through implementing the query executor. That would have the full query plan, and so know what the expected shape of the input and output data is at each plan node. I'm working on that now.

[david-crespo]

Forgot to mention, here is the workaround implemented in oxidecomputer/console#2779:

// HACK: omicron has a bug where it blows up on an attempt to group by on
// empty result set because it can't determine whether the data is aligned.
// Most likely it should consider empty data sets trivially aligned and just
// flow the emptiness on through, but in the meantime we have to detect
// this error and pretend it is not an error.
// See https://github.com/oxidecomputer/omicron/issues/7715
const errorMeansEmpty = error?.message === OXQL_GROUP_BY_ERROR
const hasError = !!error && !errorMeansEmpty