(core) Implement PREVIOUS/NEXT/RANK and lookupRecords().find.* methods.
Summary:
- `lookupRecords()` now allows efficient search in sorted results, with
the syntax `lookupRecords(..., order_by="-Date").find.le($Date)`. This will find the record with the nearest date that's <= `$Date`.
- The `find.*` methods are `le`, `lt`, `ge`, `gt`, and `eq`. All have O(log N) performance.
- `PREVIOUS(rec, group_by=..., order_by=...)` finds the previous record to rec, according to `group_by` / `order_by`, in amortized O(log N) time. For example, `PREVIOUS(rec, group_by="Account", order_by="Date")`.
- `PREVIOUS(rec, order_by=None)` finds the previous record in the full table, sorted by the `manualSort` column, to match the order visible in the unsorted table.
- `NEXT(...)` is just like `PREVIOUS(...)` but finds the next record.
- `RANK(rec, group_by=..., order_by=..., order="asc")` returns the rank of the record within the group, starting with 1. Order can be `"asc"` (default) or `"desc"`.
- The `order_by` argument in `lookupRecords`, and the new functions now supports tuples, as well as the "-" prefix to reverse order, e.g. `("Category", "-Date")`.
- New functions are only available in Python3, for a minor reason (to support keyword-only arguments for `group_by` and `order_by`) and also as a nudge to Python2 users to update.
- Includes fixes for several situations related to lookups that used to cause quadratic complexity.
Test Plan:
- New performance check that sorted lookups don't add quadratic complexity.
- Tests added for lookup find.* methods, and for PREVIOUS/NEXT/RANK.
- Tests added that renaming columns updates `order_by` and `group_by` arguments, and attributes on results (e.g. `PREVIOUS(...).ColId`) appropriately.
- Python3 tests can now produce verbose output when VERBOSE=1 and -v are given.
Reviewers: jarek, georgegevoian
Reviewed By: jarek, georgegevoian
Subscribers: paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D4265
2024-07-17 03:43:53 +00:00
|
|
|
import os
|
2023-06-13 13:40:55 +00:00
|
|
|
import sys
|
|
|
|
|
|
|
|
import six
|
|
|
|
|
|
|
|
|
|
|
|
class FakeStdStreams(object):
|
|
|
|
"""
|
|
|
|
Redirects stdout and stderr to StringIO.
|
|
|
|
"""
|
|
|
|
def __enter__(self):
|
|
|
|
self._orig_stdout = sys.stdout
|
|
|
|
self._orig_stderr = sys.stderr
|
|
|
|
sys.stdout = six.StringIO()
|
|
|
|
sys.stderr = six.StringIO()
|
|
|
|
|
|
|
|
def __exit__(self, exc_type, exc_val, exc_tb):
|
|
|
|
sys.stdout = self._orig_stdout
|
|
|
|
sys.stderr = self._orig_stderr
|
(core) Implement PREVIOUS/NEXT/RANK and lookupRecords().find.* methods.
Summary:
- `lookupRecords()` now allows efficient search in sorted results, with
the syntax `lookupRecords(..., order_by="-Date").find.le($Date)`. This will find the record with the nearest date that's <= `$Date`.
- The `find.*` methods are `le`, `lt`, `ge`, `gt`, and `eq`. All have O(log N) performance.
- `PREVIOUS(rec, group_by=..., order_by=...)` finds the previous record to rec, according to `group_by` / `order_by`, in amortized O(log N) time. For example, `PREVIOUS(rec, group_by="Account", order_by="Date")`.
- `PREVIOUS(rec, order_by=None)` finds the previous record in the full table, sorted by the `manualSort` column, to match the order visible in the unsorted table.
- `NEXT(...)` is just like `PREVIOUS(...)` but finds the next record.
- `RANK(rec, group_by=..., order_by=..., order="asc")` returns the rank of the record within the group, starting with 1. Order can be `"asc"` (default) or `"desc"`.
- The `order_by` argument in `lookupRecords`, and the new functions now supports tuples, as well as the "-" prefix to reverse order, e.g. `("Category", "-Date")`.
- New functions are only available in Python3, for a minor reason (to support keyword-only arguments for `group_by` and `order_by`) and also as a nudge to Python2 users to update.
- Includes fixes for several situations related to lookups that used to cause quadratic complexity.
Test Plan:
- New performance check that sorted lookups don't add quadratic complexity.
- Tests added for lookup find.* methods, and for PREVIOUS/NEXT/RANK.
- Tests added that renaming columns updates `order_by` and `group_by` arguments, and attributes on results (e.g. `PREVIOUS(...).ColId`) appropriately.
- Python3 tests can now produce verbose output when VERBOSE=1 and -v are given.
Reviewers: jarek, georgegevoian
Reviewed By: jarek, georgegevoian
Subscribers: paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D4265
2024-07-17 03:43:53 +00:00
|
|
|
|
|
|
|
|
|
|
|
if os.environ.get('VERBOSE'):
|
|
|
|
# Don't disable stdio streams if VERBOSE is on. This is helpful when debugging tests with
|
|
|
|
# logging messages or print() calls.
|
|
|
|
class DummyFakeStdStreams(object):
|
|
|
|
def __enter__(self):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def __exit__(self, exc_type, exc_val, exc_tb):
|
|
|
|
pass
|
|
|
|
FakeStdStreams = DummyFakeStdStreams
|