Gramex 1.81 makes distributed instances better with logging, table schema caching and computed variables.
LogViewer used to aggregate data into a SQLite database.
This works well for a single instance of Gramex. But if Gramex is deployed on multiple servers, one instance of LogViewer can’t access the SQLite file from another server.
Now, LogViewer supports setting a database location using LOGVIEWER_DB. For example, to store Logviewer data in MySQL, use:
import:
logviewer:
path: $GRAMEXAPPS/logviewer/gramex.yaml
YAMLURL: $YAMLURL/log/
LOGVIEWER_DB:
url: mysql+pymysql://root@localhost/logviewer
Internally, Gramex uses gramex.data.alter to change database table schemas.
This caches each table’s schema. This can cause a problem if 2 instances of Gramex alter a schema:
A and B. It caches columns {A, B}B, C. It reads the table, and only inserts C.C, D. It uses the cached columns {A, B}, tries to insert C, and fails because the column existsThis is now fixed. Instance 1 will read the list of columns from the table before inserting columns.
gramex.yaml can contain computed variables, for example:
variables:
SERVERNAME:
function: open('/etc/hostname').read()
default: NON-LINUX-SYSTEM
This sets SERVERNAME to the contents of /etc/hostname. But if the file does not exist, it would throw an error, and Gramex would not start.
This is now fixed. If the function: raises an exception, Gramex logs it and uses the default value.
gramex.transforms.build_transform
is the core utility that compiles Python expressions in YAML configurations. For example:
action = build_transform({'function': 'x + 1'}, vars={'x': 0}, iter=False)
… roughly compiles to:
def transform(x=0):
return x + 1
This lets us pass keyword arguments like x=0 using vars. But there was no way to pass variable arguments, e.g. **kwargs. But now:
action = build_transform({'function': 'x + 1'}, vars={'x': 0}, kwargs=True, iter=False)
… roughly compiles to:
def transform(x=0, **kwargs):
return x + 1
gramex.transforms.badgerfish() is an unused FileHandler transformation. It converts XML to JSON using the Badgerfish convention. This has been removed$GRAMEXDATA under a pynode directory.
If this directory does not exist, Gramex creates it.Gramex 1.81 is backward compatible with previous releases unless the release notes say otherwise. Automated builds test this.
Every Gramex release is tested for security vulnerabilities using the following tools.
The Gramex code base has: