DriveHandler uploads files

v1.59. DriveHandler lets you upload files and manage them with a FormHandler-like interface. It’s better than UploadHandler. Here’s how to create a DriveHandler:

url:
  drivehandler:
    pattern: /$YAMLURL/drive
    handler: DriveHandler
    kwargs:
      path: $GRAMEXDATA/apps/guide/drive/    # ... save files here

Now, to upload a file into /drive, use this form. Note: use FileHandler templates for the XSRF token.

  <!-- POST files into /drive -->
  <form action="drive" method="POST" enctype="multipart/form-data">
    <!-- There must be a file input named "file". Multiple inputs are allowed -->
    <input name="file" type="file" multiple>
    <button type="submit">Submit</button>
    <!-- To avoid XSRF, add an _xsrf key. This REQUIRES FileHandler templates -->
    <input type="hidden" name="_xsrf" value="{{ handler.xsrf_token }}">
  </form>

This saves the uploaded files in the path: you specified.

File Manager

v1.60. File Manager is an app designed to work with DriveHandler and simplifies its usage.

FileManager can be imported in a Gramex app as follows:

import:
  filemanager:
    path: $GRAMEXAPPS/filemanager/gramex.yaml
    YAMLURL: $YAMLURL/filemanager/

This mounts the File Manager page at /filemanager/. For each DriveHandler endpoint configured in your Gramex app, the page shows a table of files in that drive. This table can be used to:

File Manager can be configured by using FILEMANAGER_KWARGS:

import:
  filemanager:
    path: $GRAMEXAPPS/filemanager/gramex.yaml
    YAMLURL: $YAMLURL/filemanager/
    FILEMANAGER_KWARGS:
      drives: ['drive1', 'drive2']              # Show only these drives in the File Manager page
      title: "MyAwesomeFileManager"             # Title of the File Manager page
      logo: $YAMLPATH/data/assets/gramener.png  # Logo for the File Manager page
      theme: '?font-family-base=roboto'         # UI component theme query?

Once you import the File Manager, the File Manager component can be embedded in any <div> in any page:

<link rel="stylesheet" href="ui/dropzone/dist/min/dropzone.min.css">
<script src="ui/dropzone/dist/min/dropzone.min.js"></script>
<script src="ui/moment/min/moment-with-locales.min.js"></script>
<script src="filemanager/filemanager.js"></script>

<div class="filemanager" data-src="drive"></div>
<script>
  Dropzone.autodiscover = false
  var options = {}
  $('.filemanager').filemanager(options)
</script>

$().filemanager() accepts the same parameters as FormHandler. For example:

var options = {
  pageSize: 10,
  columns: [
    { name: "file" },
    { name: "size" }
  ]
}

AJAX uploads

DropZone provides drag-and-drop AJAX uploads with progress bars. For example:

<link rel="stylesheet" href="ui/dropzone/dist/min/dropzone.min.css">
<form action="upload" class="dropzone"></form>
<script src="ui/dropzone/dist/min/dropzone.min.js"></script>

creates this box:

List files

You can visit the drive URL to list the uploaded files. It’s a FormHandler endpoint. It shows these fields:

You can use FormHandler filters to list specific files. For example:

This data is stored in a SQLite DB called .meta.db in your drive path. You can retrieve it in Python via:

import gramex.data
meta = gramex.data.filter('sqlite:///path/to/.meta.db', table='drive')

Tags

You can add any custom fields to a file upload – e.g. if users want to add a category, rating, or description to a file.

url:
  drivehandler:
    pattern: /$YAMLURL/drive
    handler: DriveHandler
    kwargs:
      path: $GRAMEXDATA/apps/guide/drive/     # ... save files here
      tags: [category, rating, description]   # Add 3 tags

You can add an <input name="description"> in your form to allow users to upload a description. See an example.

You can use FormHandler filters to filter by tags. For example:

User fields

You can capture the user ID and other user attributes in the form.

url:
  drivehandler:
    pattern: /$YAMLURL/drive
    handler: DriveHandler
    kwargs:
      path: $GRAMEXDATA/apps/guide/drive/     # ... save files here
      user_fields: [id, role, hd]             # Capture user.id, user.role, user.hd

When a user uploads a file, their id, role, and hd attributes will be captured as user_id, user_role and user_hd fields. This allows you to track who uploaded files.

Only the id attribute is guaranteed to exist. Different auth handlers have their own attributes. For example, GoogleAuth uses hd for the domain. See User Attributes.

You can use FormHandler filters to filter by user attributes. For example:

Delete files

To delete a file, submit a DELETE HTTP request with an id: key. For example:

$.ajax('delete', {
  type: 'DELETE',
  data: {id: existing_file_id}
})

This is exactly how FormHandler DELETE works.

Update files

To rename a file or update any other attributes, submit a PUT HTTP request with an id: key. For example:

$.ajax('drive', {
  type: 'DELETE',
  data: {
    id: existing_file_id,
    file: 'new-file-name.ext',
    ext: '.ext',
    desc: 'new desc'
  }
})

Note: You cannot change a file’s id, path, size and date, nor the user_* attributes. This is mainly to rename the file and update tags.

You can overwrite a file with a PUT request. For example:

var formData = new FormData()
formData.append('id', existing_file_id)
formData.append('file', document.querySelector('input#file').files[0], 'filename.ext')
$.ajax('drive', {
    type: 'PUT',
    data: formData,
    contentType: false,
    processData: false,
})

Process files

DriveHandler accepts all the FormHandler transform functions. To modify a file when uploading, use the prepare: function:

url:
  drivehandler:
    pattern: /$YAMLURL/drive
    handler: DriveHandler
    kwargs:
      path: $GRAMEXDATA/apps/guide/drive/     # ... save files here
      user_fields: [id, role, hd]             # Capture user.id, user.role, user.hd
      prepare: mymodule.prepare(args, handler)

This calls mymodule.prepare(args, handler) for every request. args is the same as hander.args. For example, this function will add a line at the end of each .txt file:

def func(meta, handler):
    if handler.request.method == 'POST':
        # Loop through every <input name="file"> file input
        for upload in args.files.get('file', []):
            if upload['filename'].endswith('.txt'):
                upload['body'] += b'\n\nThis is a new line after each text file'

Expose datasets

To expose uploaded datasets as a FormHandler API, you can add a FormHandler that points to your drive path. For example:

url:
  datasets:
    pattern: /$YAMLURL/data/(.*)
    handler: FormHandler
    kwargs:
      url: $GRAMEXDATA/apps/guide/drive/{_0}

If you uploaded any CSV/XLSX into the DriveHandler above, see them at data/your-file.csv. (Replace your-file.csv with your file.)