Commit Graph

5 Commits

Author SHA1 Message Date
Alex Hall
fa6b57855e (core) Cache converting timestamp to date
Summary: Adds `@functools.lru_cache` to `ts_to_dt` and `ts_to_date` which are used in `_make_rich_value` in Date/DateTime columns, i.e. every time such a column is accessed in a formula. I noticed that these operations are surprisingly expensive while working on https://phab.getgrist.com/D4075. This is just an easy way to potentially significantly speed up certain docs and formulas.

Test Plan:
Put this code in an engine test case:

```
def test_(self):
  self.apply_user_action(["AddTable", "Table1", [
    {"id": "A", "type": "DateTime:America/Chicago"},
  ]])
  self.add_records("Table1", ["A"], [
    [i] for i in range(1000)
  ])
  formula = "for _ in range(1000): $A\nreturn 1"
  import time
  start = time.time()
  self.add_column(
    "Table1", formula, type="Any", isFormula=True, formula=formula
  )
  elapsed = time.time() - start
  print(f"Took {elapsed:.2f} for formula {formula}")
```

The time goes from ~10s to ~1s. Similar tests showed no noticeable slowdown when caching had no effect.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D4099
2024-01-08 10:50:50 +02:00
Alex Hall
5d671bf0b3 (core) New type conversion in the backend
Summary: This is https://phab.getgrist.com/D3205 plus some changes (https://github.com/dsagal/grist/compare/type-convert...type-convert-server?expand=1) that move the conversion process to the backend. A new user action ConvertFromColumn uses `call_external` so that the data engine can delegate back to ActiveDoc. Code for creating formatters and parsers is significantly refactored so that most of the logic is in `common` and can be used in different ways.

Test Plan: The original diff adds plenty of tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3240
2022-02-04 20:28:13 +02:00
Alex Hall
305b133c59 (core) Remaining Python 3 compatibility changes
Summary: Biggest change is turning everything to unicode

Test Plan: The tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2875
2021-06-25 12:00:58 +02:00
Alex Hall
16f297a250 (core) Simple Python 3 compatibility changes
Summary: Changes that move towards python 3 compatibility that are easy to review without much thought

Test Plan: The tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2873
2021-06-22 17:13:17 +02:00
Paul Fitzpatrick
b82eec714a (core) move data engine code to core
Summary:
this moves sandbox/grist to core, and adds a requirements.txt
file for reconstructing the content of sandbox/thirdparty.

Test Plan:
existing tests pass.
Tested core functionality manually.  Tested docker build manually.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2563
2020-07-29 08:57:25 -04:00