Gramex 1.80 release notes

Gramex 1.80 includes sentiment analysis, function pipelines, and API documentation.

Sentiment analysis with Huggingface transformers

MLHandler now supports sentiment analysis.

To set it up, install:

pip install spacy transformers torch datasets

Then use this configuration:

url:
  sentiment:
    pattern: /sentiment
    handler: MLHandler
    kwargs:
      model:
        class: SentimentAnalysis
      xsrf_cookies: false

Now visit /sentiment?text=wrong&text=right to see the following output:

["NEGATIVE", "POSITIVE"]

Function pipelines

The gramex.yaml configuration uses functions in several places. For example, in:

In every case, instead of a single function, you can use a list of steps. We call these pipelines. For example, to generate a random number in FunctionHandler:

url:
  random-function:
    pattern: /random
    handler: FunctionHandler
    kwargs:
      function:
        - random.seed(0)
        - random.randint(0, 100)

This pipeline (i.e. a function with multiple steps) has 2 steps.

When you visit /random, it always runs the 2 steps in order: random.seed(0) first, then random.randint(0, 100). It returns the same random number every time.

Visit /random

To assign the output of a step to a variable, use {name: ...}. For example:

url:
  random-sum:
    pattern: /randomsum
    handler: FunctionHandler
    kwargs:
      function:
        - { name: x, function: random.randint(0, 100) }
        - { name: y, function: random.randint(0, 100) }
        - x + y

The output will be the sum of 2 random numbers between 0-100 that changes on every reload.

Visit /randomsum

Command-line logging control

Gramex logs all messages (debug, info, warning, etc) by default. To restrict the logging to warnings, you could add this logging configuration:

log:
  loggers:
    gramex:
      level: WARNING

From this version, you can also use the --log.level= command line argument. For example:

gramex --log.level=WARNING

Displaying logger

Gramex logs did not display the logging handler name by default. A typical output looks like:

DEBUG   28-Jun 13:51:21 registry registered 'sha256_crypt' handler: ...
DEBUG   28-Jun 13:51:33 font_manager findfont: score(FontEntry(fname='...
DEBUG   28-Jun 13:51:41 __init__ PORT Running callback: app

Now, the logging handler is displayed as well:

DEBUG   28-Jun 13:51:21 passlib.registry:registry registered 'sha256_crypt' handler: ...
DEBUG   28-Jun 13:51:33 matplotlib.font_manager:font_manager findfont: score(FontEntry(fname=...
DEBUG   28-Jun 13:51:41 gramex:__init__ PORT Running callback: app

This allows apps to change the logging level for specific handlers. For example:

log:
  loggers:
    gramex: { level: WARNING } # Only show warnings form Gramex
    matplotlib: { level: INFO } # Ignore matplotlib debug messages
    passlib: { level: INFO } # Ignore passlib debug messages

API Documentation

We are rewriting the documentation for all the Gramex modules, functions and classes to make them easier.

This Gramex API documentation has been rewritten in Markdown. For example:

These functions and methods can be imported and used as part of any function or template.

JS security updates

The underlying libraries for the following have been upgraded to more secure versions:

These will not trigger warnings in security scans that incorporate npm audit.

Deprecations

The root-level key otp: held the config for encrypted users. This is now moved to storelocations.otp:.

Backward compatibility & security

Gramex 1.80 is backward compatible with previous releases unless the release notes say otherwise. Automated builds test this.

Every Gramex release is tested for security vulnerabilities using the following tools.

  1. Bandit tests for back-end Python vulnerabilities. See Bandit results
  2. npm-audit tests for front-end JavaScript vulnerabilities. See npm-audit results
  3. Snyk for front-end and back-end vulnerabilities. See Synk results
  4. ClamAV for anti-virus scans. See ClamAV results

Statistics

The Gramex code base has:

How to install

See the Gramex installation and upgrade instructions.