Gramex 1.85 release notes

Gramex 1.85 enables DriveHandler post-processing, FilterHandler hierarchies, Secure Docker builds, and more.

DriveHandler post-processing

DriveHandler accepts all the FormHandler transform functions.

To process a file after uploading, use modify::

url:
  drivehandler:
    pattern: /$YAMLURL/drive
    handler: DriveHandler
    kwargs:
      path: $GRAMEXDATA/apps/guide/drive-data/ # ... save files here
      modify: mymodule.modify(handler)

When modify: is called, you can use handler.files to iterate through the updated files:

def modify(handler):
    '''Return the file size on disk for each file'''
    root = handler.kwargs.path
    modify_data = {'filesize': {}}
    if handler.request.method in {'POST', 'PUT'}:
        for file in handler.files['path']:
            path = os.path.join(root, file)
            # Process the files any way you want
            modify_data['filesize'][file] = len(open(path).read())
    return modify_data

See post-process uploads.

FilterHandler hierarchies

FilterHandler lets you create dropdowns for filters from data.

Earlier, FilterHandler only returned unique values for each column, assuming all columns are independent.

But sometimes, columns have hierarchies. For example, Country and City. You don’t want to filter independently for country and city.

FilterHandler now returns unique values across multiple columns. Use /filter?_c=country,city to get unique values for the country,city combination, like this:

{
  "country,city": [
    { "country": "Norway", "city": "Oslo" },
    { "country": "Norway", "city": "Bergen" },
    { "country": "Italy", "city": "Rome" },
    { "country": "Italy", "city": "Venice" }
  ]
}

FormHandler POST supports templates

Earlier, FormHandler POST requests returned a JSON object with metadata about what was inserted.

Now, Gramex lets you render the response as any template, e.g. to acknowledge submitting a form, using FormHandler templates. For example:

handler: FormHandler
kwargs:
  url: "postgresql://$USER:$PASS@server/db"
  table: sales
  id: id
  default:
    _format: submit-template
  formats:
    submit-template:
      format: template
      template: $YAMLPATH/template-file.html
      headers:
        Content-Type: text/html

template-file.html can be any Tornado template. It has access to the same variables as any FormHandler template. For example:

<p>You entered name = {{ handler.get_arg('name', '') }}</p>
<p>The inserted ID(s) are {{ meta['inserted'] }}</p>

FormHandler columns support default functions

FormHandler automatically creates columns in tables via the columns kwarg.

Now, the default value for a column can be a SQL expression, like the current time, date, a random number, etc. For example:

handler: FormHandler
kwargs:
  # ...
  columns:
    timestamp:
      type: TIMESTAMP
      default: { function: func.now() }

Fetch no longer requires XSRF

Gramex blocks HTTP POST requests unless they have an XSRF cookie. This prevents hackers from creating external forms that submit to a Gramex app.

This made it inconvenient when using JavaScript, e.g. fetch or XMLHTTPRequest.

But thanks to CORS, browsers cannot submit forms to a different server unless permitted.

From Gramex 1.85 onwards, such requests are auto-detected and do NOT need to send an XSRF cookie. This reduces the need to set the xsrf_cookies: false option, improving security.

Secure Docker builds

Gramex Docker builds are built on Alpine Linux, rather than Ubuntu. This makes them:

  1. Smaller. Gramex 1.85 is 480MB, almost half the size of Gramex 1.84
  2. Secure. As of 8 Nov 2022, Gramex 1.85 has 4 vulnerabilities on Trivy compared to Gramex 1.84 with 667.

Going forward, all Gramex Docker images be scanned by Trivy.

Redirect old UI libraries

In Gramex 1.84, we deprecated old UI libraries.

In this version, these libraries are redirected to JSDelivr. For example:

If users can access the Internet, apps will continue seamlessly. If users are behind a firewall, please npm install the packages and serve them via a FileHandler.

Code reformatted

Gramex code was formatted using EditorConfig.

But EditorConfig cannot auto-reformat code – it doesn’t understand any languages.

So we moved to

We also had a new contributor to Gramex, boneyag, who simplified a string concatenation test.

Bug fixes

Backward compatibility & security

Gramex 1.85 is backward compatible with previous releases unless the release notes say otherwise. Automated builds test this.

Every Gramex release is tested for security vulnerabilities using the following tools.

  1. Bandit tests for back-end Python vulnerabilities. See Bandit results
  2. npm-audit tests for front-end JavaScript vulnerabilities. See npm-audit results
  3. Snyk for front-end and back-end vulnerabilities. See Synk results
  4. ClamAV for anti-virus scans. See ClamAV results
  5. Trivy for container scans. See Trivy results

Statistics

The Gramex code base has:

Note: The increased lines of code are mainly due to re-formatting with Prettier and Black.

How to install

See the Gramex installation and upgrade instructions.