DYAN

Overview

This repository provides training and testing code and data for ECCV 2018 paper:

"DYAN - A Dynamical Atoms-Based Network For Video Prediction", Wenqian Liu, Abhishek Sharma, Octavia Camps, and Mario Sznaier

Further information please contact Wenqian Liu at liu.wenqi@husky.neu.edu, Abhishek Sharma at sharma.abhis@husky.neu.edu.

Requirements

• PyTorch NOTE: previous versions(0.3 or below) might not work!
• Python 2.7
• Cuda 9.0

Data Preparation

• Training Data: UCF101 and KITTI raw dataset.
• Testing Data: TestingData

Getting started

• set training/testing data directory:

rootDir = 'your own data directory'

• Run the training script:

python train.py

• Run the testing script: (need to set correct 'flowDir' in Test.py to your own optical flow files)

python Test.py

Evaluation

We adopts test and evaluation script for ICLR 2016 paper: "Deep multi-scale video prediction beyond mean square error", Michael Mathieu, Camille Couprie, Yann LeCun.

BeyondMSE paper code

• Follow BeyondMSE's prerequisite to set up enviroment.
• Include util/TestScript.lua from DYAN's folder into BeyondMSE's folder.
• Set necessary directory and run:

th TestScript.lua

Generate Optical Flow Files

We adopt PyFlow pipeline to generate our OF files.

Clone pyflow repo and compile on your own machine. Then run our util/saveflows.py .

License and Citation

The use of this software is RESTRICTED to non-commercial research and educational purposes.

@InProceedings{Liu_2018_ECCV,
author = {Liu, Wenqian and Sharma, Abhishek and Camps, Octavia and Sznaier, Mario},
title = {DYAN: A Dynamical Atoms-Based Network For Video Prediction},
booktitle = {The European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}
}