Commit e7f0a233 authored by Indrek Jentson's avatar Indrek Jentson
Browse files

Structure defined

parents
https://creativecommons.org/licenses/by/4.0/
# Experiment template
This project contains an example of a data transformation experiment. The main principles are following:
* The experiment includes a source dataset in directory _source_.
* The experiment includes the transformation and validation tools in directory _tools_.
* The experiment includes a setup script and a definition of transformation process in directory _conf_.
* Directories _source_, _tools_ and _conf_ must remain unchanged during the transformations.
* The experiment includes the transformation parameters in directory _var_.
* All files are under version control.
* After each commit done by user an experiment environment must run transformation process
and save an output in directory _output_.
* If the transformation process produces the log files then they must be saved in directory _log_.
* When the results are produced, an environment must run validation process
and save an output in directory _result_.
* All output, log and result files must be commited under the same branch where previous commit was done
and tagged with experiment number.
* A validation process must compare current files with files in previous experiment and in root experiment.
* An user can start a new experiment with data from any previous experiment.
For a totally new experiment an user must:
* create a copy of this template project;
* replace files in _source_ directory;
* define a transformation process;
* prepare a setup script which installs all necessary transformation and validation tools;
* prepare the files with transformation parameters (usually transformation rules);
* start a experiment environment with created project.
NB! In the first stage of development we assume that experiments are running under Linux (Debian 8).
This directory is intended for configuration files of the experiment project.
Usually it includes files
* setup.sh
* env.sh
* transform.sh
* validate.sh
This directory is intended for log files of the experiment project.
This directory is intended for transformation output files of the experiment project.
This directory is intended for result files of the experiment project.
Usually it includes file like result.html.
This directory is intended for source files of the experiment project.
This directory is intended for specific scripts and programs for the experiment project.
This directory is intended for files with paramters of the experiment project.
This is the directory where user changes data in order to initiate a new transformation and result.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment