Gramex 1.65 comes with Excel table support, exposes functions as REST APIs, and support for logging via ElasticSearch.
Gramex expects Excel files to be organized neatly – with one table per sheet, and no empty rows or columns at the beginning.
Spreadsheets are rarely organized this well.
From this release, Gramex can now read any cell range, table or defined name from an Excel sheet.
For example, this cash flow projection sheet has multiple sections, each created as Excel tables.
To read a specific table from this sheet, e.g. the OtherOperationalData
table, use this FormHandler configuration:
pattern: /data # The URL /data
handler: FormHandler
kwargs:
url: cashflow.xlsx # Reads from cashflow.xlsx
table: OtherOperationalData # and shows a specific table
… and render it as data or charts.
This lets you read un-structured or manually created Excel reports directly as a data source.
You can also call this from Python code as
gramex.cache.open('cashflow.xlsx', table='OtherOperationalData')
.
To read from a cell range:
url: cashflow.xlsx # Reads from cashflow.xlsx
range: B55:P61 # cells B55 to P61
sheet_name: Cash Flow # from the sheet named Cash Flow
To read from a defined name:
url: cashflow.xlsx # Reads from cashflow.xlsx
name: CashChart # all cells in the name "CashChart"
sheet_name: Cash Flow # from the sheet named Cash Flow
Earlier, FunctionHandler would only render string or dict values. Now you may return any of these types. They are converted to JSON as follows:
None
: this returns an empty string (as before), but without a warningbool
, np.bool
: rendered as JSON, e.g. true
, false
(note the lowercase)int
, float
, np.integer
, np.float
: rendered as JSON, e.g. 3
, 1.5
datetime.datetime
and np.datetime
: rendered as ISO date, e.g. 1997-07-16T19:20:30+01:00
list
, tuple
, np.ndarray
: rendered as JSON arrays, e.g. [1, "abc", true]
dict
: rendered as JSON, e.g. {"x": 1, "y": "abc"}
. Keys must be stringspd.DataFrame
: rendered as JSON via .to_json(orient="records", date_format="iso")
This means you don’t have to explicitly convert return values into strings. They are automatically converted using a JSON-compatible format. For example:
def myfunction(handler):
# Instead of encoding booleans into strings...
return json.dumps(os.path.exists(file))
# ... you can now use...
return os.path.exists(file)
# Instead of converting datetimes to a specific format...
return today.isoformat()
# ... you can return them directly
return today
# Instead of wrapping lists in a dict...
return {'result': mylist}
# ... you can return it directly
return mylist
Passing URL query parameters directly to a Python function is much easier now. Suppose you have this function:
def combinations(n=10, k=1):
return factorial(n) / factorial(k) / factorial(n - k)
To let this accept URL query parameters via /combinations?n=10&k=4
, you’d write:
# Old code -- no longer required in Gramex 1.65+
def combinations(handler):
n = int(handler.get_arg('n', '10'))
k = int(handler.get_arg('k', '1'))
return factorial(n) / factorial(k) / factorial(n - k)
… or:
# Old code -- no longer required in Gramex 1.65+
def combinations(handler):
args = handler.argparse({
n={'type': int, 'default': 10},
k={'type': int, 'default': 1}
})
return factorial(n) / factorial(k) / factorial(n - k)
Instead, you can use type hints along with the new gramex.transforms.handler
wrapper:
@gramex.transforms.handler
def combinations(n: int=10, k: int=1):
return factorial(n) / factorial(k) / factorial(n - k)
This automatically parses /combinations?n=10&k=4
into n = 10
and k = 4
along with type
conversions. See a live example in the
FunctionHandler docs.
Gramex has a new gramexlog
service that lets you log any user or system event.
To set this up, you need to install
install ElasticSearch
and configure gramex.yaml
:
gramexlog:
gramexguide: # Log name. We'll call this via gramex.log('gramexguide')
host: hostname # OPTIONAL: ElasticSearch server name. default: localhost
port: 9200 # OPTIONAL: port to connect to. default: 9200
index: gramexguide # OPTIONAL: index to connect to. default: same as log name
http_auth: user:pass # OPTIONAL: user name and password. default: None
keys: [datetime, user.id, headers.User-Agent] # OPTIONAL: additional keys. default: None
Then, call gramex.log(x=1, y=2)
(or use any other keyword arguments.)
This will log {"port": 9988, "time": ..., "user.id": ..., "x": 1, "y": 2}
into your ElasticSearch
server’s index gramexguide
.
You can also use gramex.log
as a FunctionHandler endpoint directly. For
example:
url:
log:
pattern: /log
handler: FunctionHandler
kwargs:
function: gramex.log(handler, 'gramexguide', event='handler')
When a user visit /log?x=1&y=2
, it logs into gramexguide
an object with these keys:
"x": "1"
from the URL"y": "2"
from the URL"event": "handler"
from the function call"datetime": ...
from the gramexguide
index configuration"user.id": ...
from the gramexguide
index configuration"header.User-Agent": ...
from the gramexguide
index configurationSince FormHandler supports ElasticSearch (after running
pip install elasticsearch-dbapi
), you can view the logs via this FormHandler configuration:
kwargs:
url: elasticsearch+http://localhost:9200
table: gramexguide
gramex.cache.open
supports custom callbacks. For example, to load
.pickle
files, use gramex.cache.open_callback['pickle'] = gramex.cache.opener(pickle.load)
.
Now, gramex.cache.open('data.pickle')
loads data from a pickle file.gramex init
replaces npm install
and bower install
with yarn install
in
.gitlab-ci.yml
. pytest
is now optional. Python 3 is now the default CI instance to push to.slidesense
was unable to import modules from the current folder, and
raised an os.startfile
error on Linux. Both are resolved.6 of the features promised in this release are delayed. 4 of these will be released in December.
2 are deferred to a future release.
The Gramex code base has:
See the Gramex installation and upgrade instructions
Note: Gramex 1.65 does not work with Python 3.8. We recommend Python 3.7.