Gramex 1.65 release notes

Gramex 1.65 comes with Excel table support, exposes functions as REST APIs, and support for logging via ElasticSearch.

Excel ranges supported

Gramex expects Excel files to be organized neatly – with one table per sheet, and no empty rows or columns at the beginning.

Spreadsheets are rarely organized this well.

From this release, Gramex can now read any cell range, table or defined name from an Excel sheet.

For example, this cash flow projection sheet has multiple sections, each created as Excel tables.

Cash flow projection sheet

To read a specific table from this sheet, e.g. the OtherOperationalData table, use this FormHandler configuration:

pattern: /data # The URL /data
handler: FormHandler
kwargs:
  url: cashflow.xlsx # Reads from cashflow.xlsx
  table: OtherOperationalData # and shows a specific table

… and render it as data or charts.

This lets you read un-structured or manually created Excel reports directly as a data source.

You can also call this from Python code as gramex.cache.open('cashflow.xlsx', table='OtherOperationalData').

To read from a cell range:

url: cashflow.xlsx # Reads from cashflow.xlsx
range: B55:P61 # cells B55 to P61
sheet_name: Cash Flow # from the sheet named Cash Flow

To read from a defined name:

url: cashflow.xlsx # Reads from cashflow.xlsx
name: CashChart # all cells in the name "CashChart"
sheet_name: Cash Flow # from the sheet named Cash Flow

Functions converted to JSON

Earlier, FunctionHandler would only render string or dict values. Now you may return any of these types. They are converted to JSON as follows:

This means you don’t have to explicitly convert return values into strings. They are automatically converted using a JSON-compatible format. For example:

def myfunction(handler):
    # Instead of encoding booleans into strings...
    return json.dumps(os.path.exists(file))
    # ... you can now use...
    return os.path.exists(file)

    # Instead of converting datetimes to a specific format...
    return today.isoformat()
    # ... you can return them directly
    return today

    # Instead of wrapping lists in a dict...
    return {'result': mylist}
    # ... you can return it directly
    return mylist

Functions as REST APIs

Passing URL query parameters directly to a Python function is much easier now. Suppose you have this function:

def combinations(n=10, k=1):
    return factorial(n) / factorial(k) / factorial(n - k)

To let this accept URL query parameters via /combinations?n=10&k=4, you’d write:

# Old code -- no longer required in Gramex 1.65+
def combinations(handler):
    n = int(handler.get_arg('n', '10'))
    k = int(handler.get_arg('k', '1'))
    return factorial(n) / factorial(k) / factorial(n - k)

… or:

# Old code -- no longer required in Gramex 1.65+
def combinations(handler):
    args = handler.argparse({
      n={'type': int, 'default': 10},
      k={'type': int, 'default': 1}
    })
    return factorial(n) / factorial(k) / factorial(n - k)

Instead, you can use type hints along with the new gramex.transforms.handler wrapper:

@gramex.transforms.handler
def combinations(n: int=10, k: int=1):
    return factorial(n) / factorial(k) / factorial(n - k)

This automatically parses /combinations?n=10&k=4 into n = 10 and k = 4 along with type conversions. See a live example in the FunctionHandler docs.

Flexible logging

Gramex has a new gramexlog service that lets you log any user or system event.

To set this up, you need to install install ElasticSearch and configure gramex.yaml:

gramexlog:
  gramexguide: # Log name. We'll call this via gramex.log('gramexguide')
    host: hostname # OPTIONAL: ElasticSearch server name. default: localhost
    port: 9200 # OPTIONAL: port to connect to. default: 9200
    index: gramexguide # OPTIONAL: index to connect to. default: same as log name
    http_auth: user:pass # OPTIONAL: user name and password. default: None
    keys: [datetime, user.id, headers.User-Agent] # OPTIONAL: additional keys. default: None

Then, call gramex.log(x=1, y=2) (or use any other keyword arguments.) This will log {"port": 9988, "time": ..., "user.id": ..., "x": 1, "y": 2} into your ElasticSearch server’s index gramexguide.

You can also use gramex.log as a FunctionHandler endpoint directly. For example:

url:
  log:
    pattern: /log
    handler: FunctionHandler
    kwargs:
      function: gramex.log(handler, 'gramexguide', event='handler')

When a user visit /log?x=1&y=2, it logs into gramexguide an object with these keys:

Since FormHandler supports ElasticSearch (after running pip install elasticsearch-dbapi), you can view the logs via this FormHandler configuration:

kwargs:
  url: elasticsearch+http://localhost:9200
  table: gramexguide

Other improvements

What next

6 of the features promised in this release are delayed. 4 of these will be released in December.

  1. Python 3.8 support
  2. ModelHandler support for TensorFlow, Keras and PyTorch models
  3. Notification for errors in alerts
  4. Add a 1-year roadmap for Gramex

2 are deferred to a future release.

  1. Extend PPTXHandler with custom charts
  2. Logviewer components that you can embed in any page to track usage

Statistics

The Gramex code base has:

Credits

How to install

See the Gramex installation and upgrade instructions

Note: Gramex 1.65 does not work with Python 3.8. We recommend Python 3.7.