FunctionHandler runs Python code

Video

The FunctionHandler runs any expression or pipeline and displays the output.

For example, this configuration maps the URL total to a FunctionHandler:

url:
  total:
    pattern: /total # The "total" URL
    handler: FunctionHandler # runs a function
    kwargs:
      function: calculations.total(100, 200) # total() from calculations.py
      headers:
        Content-Type: application/json # Display as JSON

It runs calculations.total() with the arguments 100, 200 and returns result 300 as application/json. calculations.py defines total as below:

def total(*items):
    return sum(float(item) for item in items)
Source

See total

To see all configurations used in this page, see gramex.yaml:

Function arguments from URL

Video

You can pass function arguments from the URL as a REST API.

For example, to pass combinations?n=10&k=4 as a function call combinations(n=10, k=4), add this to calculations.py:

from gramex.transforms import handler
from math import factorial

@handler
def combinations(n:int, k:int) -> float:
    '''combinations(10, 4): ways to pick 4 objects from 10 ignoring order'''
    return factorial(n) / factorial(k) / factorial(n - k)

Expose it via this gramex.yaml configuration:

url:
  combinations:
    pattern: /combinations
    handler: FunctionHandler
    kwargs:
      function: calculations.combinations
      headers:
        Content-Type: application/json

Now, combinations?n=10&k=4 returns 210, the number of ways to pick 4 items from 10 ignoring order)

Source

Try combinations?n=10&k=4

gramex.transforms.handler calculates exposes all function arguments into a REST API. If you provide a type hint (e.g. n: int), it converts the argument to the correct type.

Apart from int, float, str and bool, you can specify lists of a specific type as well. For example:

from gramex.transforms import handler
from numpy import prod
from typing import List

@handler
def multiply(v: List[int]):
    return prod(v)

… will get v as a list of int. In multiply?v=10&v=20&v=30, v is passed as a list v = [10, 20, 30] to return 10 x 20 x 30 = 6,000.

Source

Try multiply?v=10&v=20&v=30

Function output

Video

The output of the function is rendered as a string.

String and byte outputs are rendered as-is. Other types (int, float, bool, datetime, DataFrame) are converted to JSON as follows:

HTTP methods

Video

FunctionHandler handles GET and POST requests by default. That is, the same function is called irrespective of whether the method is GET or POST.

To change this, add a methods: key. For example:

url:
  total:
    pattern: /total
    handler: FunctionHandler
    kwargs:
      function: calculations.total(100, 200)
      methods: [POST, PUT, DELETE] # Allow only these 3 HTTP methods

URL path arguments

Video

You can specify wildcards in the URL pattern. For example:

url:
  lookup:
    pattern: /name/([a-z]+)/age/([0-9]+) # e.g. /name/john/age/21
    handler: FunctionHandler # Runs a function
    kwargs:
      function: calculations.name_age # Run this function

When you access /name/john/age/21, john and 21 can be accessed via handler.path_args as follows:

def name_age(handler):
    name = handler.path_args[0]
    age = handler.path_args[1]
    return name + ' is ' + age + ' years old'

You can pass any options you want to functions. For example, to call random.randrange(start=0, stop=100), you can use:

url:
  method:
    pattern: /method # The URL /method
    handler: FunctionHandler # Runs a function
    kwargs:
      function: random.randrange(start=0, stop=100)

You can also pass these directly in the function:

url:
  path:
    pattern: /path/(.*?)/(.*?)
    handler: FunctionHandler
    kwargs:
      function: handler.path_args

Sample output:

path_args is available to all handlers.

Function arguments

For greater control over arguments, you can pass a Tornado RequestHandler called handler to your function.

For example, the add URL below takes handler and sums up numbers you specify. add?x=1&x=2 shows 3.0:

<form action="add">
  <div><input name="x" value="10" /></div>
  <div><input name="x" value="20" /></div>
  <button type="submit">Add</button>
</form>

To set this up, gramex.yaml used the following configuration:

url:
  add:
    pattern: /add # The "add" URL
    handler: FunctionHandler # runs a function
    kwargs:
      function: calculations.add(handler) # add() from calculations.py
      headers:
        Content-Type: application/json # Display as JSON

calculations.add(handler) is called with the Tornado RequestHandler. It accesses the URL query parameters to add up all x arguments.

def add(handler):
    args = handler.argparse(x={'nargs': '*', 'type': float})
    return sum(args.x)
Source

Try add?x=10&x=20&x=30

Parse URL arguments

Video

You can manually parse the URL parameters. The URL parameters are stored in handler.args as a dict with Unicode keys and list values. For example:

?x=1        => {'x': ['1']}
?x=1&x=2    => {'x': ['1', '2']}
?x=1&y=2    => {'x': ['1'], 'y': ['2']}

To simplify URL query parameter parsing, all handlers have a handler.argparse() function. This returns the URL query parameters as an attribute dictionary.

For example:

def method(handler):
    args = handler.argparse('x', 'y')  # x and y will be loaded as strings by default
    args.x      # This is the same as the last value of ?x
    args.y      # This is the same as the last value of ?y

When you pass ?x=a&y=b, args.x is a and args.y is b. With multiple values, e.g. ?x=a&x=b, args.x is takes the last value, b.

A missing ?x= or ?y= raises a HTTP 400 error mentioning the missing key.

For optional arguments, use:

args = handler.argparse(z={'default': ''})
args.z          # returns '' if ?z= is missing

You can convert the value to a type:

args = handler.argparse(limit={'type': int, 'default': 100})
args.limit      # returns ?limit= as an integer

You can restrict the choice of values. If the query parameter is not in choices, we raise a HTTP 400 error mentioning the invalid key & value:

args = handler.argparse(gender={'choices': ['M', 'F']})
args.gender      # returns ?gender= which will be 'M' or 'F'

You can retrieve multiple values as a list::

args = handler.argparse(cols={'nargs': '*', 'default': []})
args.cols       # returns an array with all ?col= values

type: conversion and choices: apply to each value in the list.

To return all arguments as a list, pass list as the first parameter::

args = handler.argparse(list, 'x', 'y')
args.x          # ?x=1 sets args.x to ['1'], not '1'
args.y          # Similarly for ?y=1

You can combine all these options. For example:

args = handler.argparse(
    'name',                         # Raise error if ?name= is missing
    department={'name': 'dept'},    # ?dept= is mapped to args.department
    org={'default': 'Gramener'},    # If ?org= is missing, defaults to Gramener
    age={'type': int},              # Convert ?age= to an integer
    married={'type': bool},         # Convert ?married to a boolean
    alias={'nargs': '*'},           # Convert all ?alias= to a list
    gender={'choices': ['M', 'F']}, # Raise error if gender is not M or F
)

HTTP headers

Video

If you have a download.pdf in your folder and display it using a function, the output will be rendered as text, by default. For example:

url:
  download-pdf:
    pattern: /download-pdf
    handler: FunctionHandler
    kwargs:
      function: gramex.cache.open('download.pdf', 'bin')

… will show output like:

%PDF-1.7 %���� 9 0 obj << /Type /Page...

To render this as a PDF, add this header:

headers:
  Content-Type: application/pdf # MIME type of download

This displays the file as a PDF in the browser.

Further, to download it (instead of rendering it in the browser), add this header:

Content-Disposition: attachment; filename=download.pdf

You can also specify this in your function:

def method(handler):
    handler.set_header('Content-Type', 'application/pdf')
    handler.set_header('Content-Disposition', 'attachment; filename=download.pdf')
    return open('download.pdf', 'rb').read()

See Common MIME types.

Function redirection

After the function executes, users can be redirected via the redirect: config documented the redirection configuration.

Or, inside the function, use handler.redirect(url) to redirect. For example:

def method(handler):
    if 'password' in handler.args:
        handler.redirect('/password-provided')
    else:
        handler.redirect('/password-not-provided')

You can pass permanent=True to handler.redirect(), e.g. handler.redirect(new_url, permanent=True). This uses HTTP 301 which browsers and proxies cache, and will ALWAYS redirect to the URL without checking with Gramex. Use with care.

Streaming output

Video

If you perform slow calculations and want to flush interim calculations out to the browser, use yield. For example, slow?x=1&x=2&x=3 uses the function below to print the values 1, 2, 3 as soon as they are “calculated”. (You won’t see the effect on learn.gramener.com. Try it on your machine.)

def slow_print(handler):
    for value in handler.args.get('x', []):
        time.sleep(1)
        yield 'Calculated: %s\n' % value

When a function yields a string value, it will be displayed immediately. The function can also yield a Future, which will be displayed as soon as it is resolved. (Yielded Futures will be rendered in the same order as they are yielded.)

Source

Try slow?x=1&x=2&x=3

Asynchronous functions

Video

If you are using an asynchronous Tornado function, like AsyncHTTPClient, they will not block the server. The function runs in parallel while Gramex continues. When the function ends, the code resumes. Use coroutines to achieve this. For example, this fetches a URL’s body without blocking:

async_http_client = tornado.httpclient.AsyncHTTPClient()

@tornado.gen.coroutine
def fetch_body(url):
    'A co-routine that fetches the URL and returns the URL body'
    result = yield async_http_client.fetch(url)
    raise tornado.gen.Return(result.body)

You can combine this with the yield statement to fetch multiple URLs asynchronously, and display them as soon as the results are available, in order:

def urls(handler):
    # Initiate the requests
    args = handler.argparse(x={'nargs': '*'})
    # Initiate the requests
    futures = [fetch_body('https://httpbin.org/delay/%s' % x) for x in args.x]
    # Yield the futures one by one
    for future in futures:
        yield future
Source

See fetch?x=0&x=1&x=2

The simplest way to call any blocking function asynchronously is to use a ThreadPoolExecutor. For example, using this code in a FunctionHandler will run slow_calculation in a separate thread without blocking Gramex. Gramex provides a global threadpool that you can use. It’s at gramex.service.threadpool.

from gramex import service      # Available only if Gramex is running
result = yield service.threadpool.submit(slow_calculation, *args, **kwargs)

You can run execute multiple steps in parallel and consolidate their result as well. For example:

@tornado.gen.coroutine
def calculate(data1, data2):
    from gramex import service      # Available only if Gramex is running
    group1, group2 = yield [
        service.threadpool.submit(data1.groupby, ['category']),
        service.threadpool.submit(data2.groupby, ['category']),
    ]
    result = service.threadpool.submit(pd.concat, [group1, group2])
    raise tornado.gen.Return(result)