welcome
=====

This is the code package for CORE's MSST13 version. This package is ONLY
tested with Ubuntu 12.04. Have fun.

Install
=====

The CORE in this package relies on hadoop-0.22.0 and its HDFS-RAID, so
in order to install CORE, we need to embed CORE's source code into
hadoop-0.22.0 and recompile it. Then we can set up a CORE cluster following a
similar way as we set up a hadoop cluster.

You should plan where to locate the namenode and where to locate the datanode.
Then do the following in one of the node.

1.  install g++, ant, java, set up $HADOOP_HOME(where you will place
    your hadoop), $JAVA_HOME(where is the sdk).

2.  download
    [hadoop-0.22.0](http://archive.apache.org/dist/hadoop/core/hadoop-0.22.0/hadoop-0.22.0.tar.gz)

3.  Untar hadoop-0.22.0.tar.gz, mv to $HADOOP_HOME

4.  Run install.sh. This script will integrate CORE into hadoop-0.22.0 with some default configurations.

    bash install.sh

5.  In $HADOOP_HOME/conf/masters, enter the hostname or host ip of your namenode.
    (don't forget to remove the default one if you don't need it)

6.  In $HADOOP_HOME/conf/slaves, enter the hostname or host ip of your datanode.
    (don't forget to remove the default one if you don't need it)

7.  In $HADOOP_HOME/conf/core-site.xml, config the hadoop tmp directory
    (where you want to put the hadoop file in your machine, better to
    use absolute path) and
    fs.default.name.
  
    <property>
      <name>hadoop.tmp.dir</name>
      <value>*put your path here(absolute path)*</value>
    </property>

    <property>
      <name>fs.default.name</name>
      <value>hdfs://*your namenode hostname or ip*:54310</value>
    </property>  


8.  In $HADOOP_HOME/conf/hdfs-site.xml, config the stripeLength(k) and
    parityLength(m). (Where the property name of parity length dependent
    on your chosen code)
    
    <property>
      <name>hdfs.raid.stripeLength</name>
      <value>*your k*</value>
    </property>

    <property>
      <name>hdfs.raidjrs.paritylength</name>
      <value>*your m for jrs code*</value>
    </property>

    <property>
      <name>hdfs.raidia.paritylength</name>
      <value>*your m for ia code*</value>
    </property>
    
9.  In $HADOOP_HOME/conf/hdfs-site.xml, config the *block size* and *packet
    size*(strip size). *packet size* should be a multiple of *parity length*(m) and *block
    size* should be a multiple of *packet size*.
  
    <property>
      <name>dfs.block.size</name>
      <value>*your block size*</value>
    </property>
  
    <property>
      <name>hdfs.raid.packet.size</name>
      <value>*your packet size*</value>
    </property>

10.  In $HADOOP_HOME/conf/hdfs-site.xml, config the path of
    raid.xml (Normally it should be in $HADOOP_HOME/conf/raid.xml,
    better to use absolute path).

    <property>
      <name>raid.config.file</name>
      <value>*your raid.xml path(Absolute Path)*</value>
      <description>This is needed by the RaidNode </description>
    </property>

11.  In $HADOOP_HOME/conf/raid.xml, config the type of code you want to
    use. (ia/jrs/iao/jrso, o stands for original architecture)

    <erasureCode>*your code type*</erasureCode>

12.  In $HADOOP_HOME/conf/raid.xml, config the source file path.

    <srcPath prefix="hdfs://*namenode hostname or ip*:*port*/*file path*">

13.  In $HADOOP_HOME/conf/hadoop_env.sh, set the JAVA_HOME

    export JAVA_HOME=*your java home(Absolute Path)*

    export HADOOP_OPTS=-Djava.net.preferIPv4Stack=true

After finish the previous node installation and configuration, copy the
$HADOOP_HOME folder and spread the folder into the *namenode* and *ALL*
*datanodes*. Make sure each nodes can be public key accessible by each
other. Also make sure that c++, java, ant and $HADOOP_HOME are well set
for ALL nodes. If you have serveral different type of nodes (for
example, different OS), you may need to repeat the previous steps in
different types of nodes.

Add $HADOOP_HOME/bin to PATH

    export PATH=$PATH:$HADOOP_HOME/bin

Format hdfs

    hadoop namenode -format

Run hdfs

    start-hdfs.sh

Run Raidnode

    start-raidnode.sh

Configuration
=====

Configurable options in hadoop-0.22.0 (include hdfs-raid in 0.22.0) are
inherited. Please refer to
[hdfs](http://hadoop.apache.org/docs/stable/cluster_setup.html) and
[hdfs-raid](http://wiki.apache.org/hadoop/HDFS-RAID).

Additional configurable options for CORE only

For hdfs-site.xml

Config the number of encoder working thread
    
    <property>
      <name>hdfs.raid.encoder.threadnum</name>
      <value>2</value>
    </property>

Config the number of decoder working thread

    <property>
      <name>hdfs.raid.decoder.threadnum</name>
      <value>1</value>
    </property>

Config the parity length of jerasure's rs code

    <property>
      <name>hdfs.raidjrs.paritylength</name>
      <value>8</value>
    </property>

    
Config the parity length of IA code

    <property>
      <name>hdfs.raidia.paritylength</name>
      <value>8</value>
    </property>
 
Config the strip size(make sure that the strip size must be a multiple
of parity length(m))
   
    <property>
      <name>hdfs.raid.packet.size</name>
      <value>4194304</value>
    </property> 

A new block placement policy named Controlable Policy. It is a ad hoc
policy for experment only and by no means to be used in practical. It
can help to make sure that blocks from the same stripe will be located
in different nodes. If you choose to use this policy, you need to make
sure that *hdfs.raidia.paritylength* is set and there is *NO REBOOT*
between data import and migration.

    <property>
      <name>dfs.block.replicator.classname</name>
      <value>
        org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyControlable
      </value>
    </property>

For raid.xml

Choose code type(options: jrs, ia)

    <erasureCode>ia</erasureCode>

Running CORE
=====

Similar to HDFS-RAID.

To run corresponding experiment, you need to determine some
configurations.

1.  Code type.

2.  Parameter *k* and *m*.

3.  Block size and strip size.

4.  Config your cluster(namenode and datanode).

5.  Customize raid source file path.

6.  Choose block placement policy.

For all experiments, you can check the log files in *$HADOOP_HOME/logs/*
for the updated status.

Migration

1.  Import data into raid source file folder.

2.  $HADOOP_HOME/bin/start-raidnode.sh

Recovery

1.  After migration, you can artificially remove few datanodes. (you can
    either disconnect the node, kill datanode process or remove
    corresponding data block.)

2.  Wait for recovery to finish. (Since block fixer has a checking
    window, if you want to make the recovery start earlier, you can
    reboot hdfs after you remove datanodes.)

Degraded Read.

1.  After migration, you can artificially remove few datanodes. (you can
    either disconnect the node, kill datanode process or remove
    corresponding data block.)

2.  Try to read data. (either by mapreduce or shell command)

3.  You can stop raidnode to avoid recovery during degraded read.

Developers' Guide
=====

*./native* hosts the native coding relative code including the Jerasure
library.

*./java* hosts some modified hadoop java code.

To add a new coding scheme
 
Suppose you want to add a new coding scheme, called newCoding. 
Here are the steps you need to take:
 
1.  Implement C++ version of the coding scheme.
  
    You need to implement a class newCoding which extends the 
    base class Coding. You need to implement the following virtual
  
    functions:
   
    generate_encoding_matrix();
        Generate the encoding equations.
   
    encode2();
        Encode the original data to encoded data.
   
    test_validity();
        Test whether a failure pattern is good or not.
   
    set_f2();
        Generate decoding equations for data reconstruction.
   
    encode_offline_recovery2();
        Encoding function called by the surviving nodes.
   
    reconstruct_lost_date2();
        Decoding function called by the relayer
 
2.  Update the native code.
 
3.  Modify the java source code.