Overview of the storage interface

Inside the interface, five abstract methods that mainly focus on the I/O operation are defined. Beside these abstract methods, the corresponding wrapper methods are used to avoid the race condition.

Abstract methods.

• protected abstract boolean put(String key, byte[] value);

  // Store the data according to its unique key. If the key is already used, the old data should be deleted before storing the data

• protected abstract byte[] get(String key);

  // Return the data that associated with the key as an array of bytes. If no such key exists, return null

• protected abstract boolean delete(String key);

  // Delete the data that associated with the key. Return true if success.

• protected abstract boolean growTier(int byPercent);

  // Grow the available space according to the byPercent. Return true if success.

• protected abstract boolean shrinkTier(int byPercent);

  // Shrink the size of provided space according to the byPercent. Return true if success.

Insert your code into Wiera

Before devoting into customizing your interface. It is necessary to introduce how to integrate and invoke the user-defined interface into Wiera. Two steps are required.

1. Register the new interface in Wiera's code.

   Suppose the customized interface is called MS, short for MobileStorageInterface. Add a new enumeration item inside the TierInfo class's STORAGE_PROVIDER. For example,

     public enum STORAGE_PROVIDER {
   		S3(0),
   		AS(1),
   		GS(2),
           MS(3); // New interface for MobileStorageInterface.
   		private final int m_type;
   		/**
   		 * Create a STORAGE_PROVIDER instance with a specified type.
   		 * @param service A STORAGE_PROVIDER constant.
   		 * */
   		STORAGE_PROVIDER(final int service) {
   			m_type = service;
   		}
   
   		public int getType() {
   			return m_type;
   		}
   	}
                 

   In the Tier class's addStorageTier method, provide a case for the new interface.

   public class Tier {
       //...
       public boolean addStorageTier(TierInfo.STORAGE_PROVIDER type, String strID1, String strID2, String strArg1, String strArg2) {
           switch (type) {
             //...
             case MS:
                 MobileStorageInterface msi;
                 try{
                     msi = new MobileStorageInterface(strID1, strID2, strArg1, strArg2); // provide correct arguments accroding to the constructor;
                 }catch(Exception e){
                     e.printStackTrace();
                     return false;
                 }
                 return m_tierInterfaces.add(gcs);
           }
         //...
       }
       //...
   }
                 

2. Utilize the new interface in policy file.

   An application is required to indicate to use which cloud storage and the interface in the tier configuration.

       "storage_tiers": [
         {
           "tier_name": "mobile-store-1",
           "tier_size": "1GB",
           "tier_type": 3,
           "storage_provider": 4,
           "storage_arg1":"username"
           "storage_arg2":"host-ip"
           "storage_id1":"key"
           "storage_id2":"secret"
           "tier_location": "mobile-store-location",
           "tier_expected_latency":  10,
           "default":  true,
         }
       ],
                 

   The tier_type and storage_provider are used to invoke the new interface. The storage_arg1/2 and storage_id1/2 are highly customized elements. Their values are used to provide the necessary data for the interface constructor. For example, most cloud services require users to provide their username, key, and host IP.

Example: Amazon Simple Storage Service (S3) Interface

public class S3Interface extends StorageInterface {
  /**
  * Declare the
  *
  */
    private AmazonS3Client s3Client = null;
    private String s3Folder = null;
    private String s3BucketName = null;
    public static final String S3_KEY = "YOUR_OWN_KEY";
    public static final String S3_SECRET = "YOUR_OWN_SECRET";
    // Constructor:
    public S3Interface(String s3Key, String s3Secret, String s3Bucket, String s3Folder) {
        BasicAWSCredentials bawsc = new BasicAWSCredentials(s3Key, s3Secret);
        s3Client = new AmazonS3Client(bawsc);
        this.s3BucketName = s3Bucket;
        this.s3Folder = s3Folder;

        //Check / character
        if (s3Folder.endsWith("/") == false) {
            this.s3Folder += "/";
        }
    }

    /**
    * Store the key-value pair into the storage. In this case, the value is stored as file and key
    * is the file's name. Users may provide their own ways to finish this task as long as the get
    * and delete can find and obtain/remove the value according to the key.
    *
    * @param key The identifier of the value.
    * @param value The real data need to be stored.
    * return True if successful, false otherwise.
    */
    public boolean put(String key, byte[] value) {
        try {
            InputStream stream = new ByteArrayInputStream(value);
            ObjectMetadata metadata = new ObjectMetadata();
            metadata.setContentLength(stream.available());
            s3Client.putObject(s3BucketName, s3Folder + key, stream, metadata);
        } catch (Exception e) {
            System.out.println("Write to S3 failed BucketName: " + s3BucketName);
            e.printStackTrace();
            return false;
        }

        return true;
    }

    /**
    * Obtain the data that is associated with the key. In this case, the S3 client is used to read
    * the file whose name is constructed by the key. And the file's content is the data.
    *
    * @param key The data's identifier that is used to retrieve the data.
    * return True if successful, false otherwise.
    */
    public byte[] get(String key) {
        S3Object S3data;
        byte[] value = null;

        try {
            S3data = s3Client.getObject(new GetObjectRequest(s3BucketName, s3Folder + key));
            value = ByteStreams.toByteArray(S3data.getObjectContent());
        } catch (Exception e) {
            System.out.println("Get from S3 failed");
            e.printStackTrace();
        }

        return value;
    }

    /**
    * The goal of delete is to remove the data that is associated with the key and free the space.
    *
    * @para key The data's identifier that is used to delete the data.
    * return True if successful, false otherwise.
    */
    public boolean delete(String key) {
        s3Client.deleteObject(s3BucketName, key);
        return true;
    }

    /**
    * It is allowed that the user's storage server may not provide some necessary functions to achieve
    * the StorageInterface's goals. As a customer of S3, for example, the user cannot adjust the storage
    * space in the real time. So the goals of growTier and shrinkTier cannot be achieved. In this case,
    * the two methods just return false to reject these operations.
    */
    protected boolean growTier(int byPercent) {
        return false;
    }

    protected boolean shrinkTier(int byPercent) {
        return false;
    }
}