Drake is a simple-to-use, extensible, text-based data workflow tool that organizes command execution around data and its dependencies. Data processing steps are defined along with their inputs and outputs and Drake automatically resolves their dependencies and calculates: a) which commands to execute (based on file timestamps); and b) in what order to execute the commands (based on dependencies). Drake is similar to GNU Make, but designed especially for data workflow management. It has HDFS support, allows multiple inputs and outputs, and includes a host of features designed to help you bring sanity to your otherwise chaotic data processing workflows. This will be added to Script Resources Subject Tracer™. This will be added to the tools section of Research Resources Subject Tracer™.
posted by Marcus Zillman |
1:27 AM
