Instructions for setting up a local dev environment for Simple Data Pipe

Introduction
As a developer working on Bluemix applications, it is very important that you are able to edit, run and debug your code locally, so that you can be as productive as possible.
This page provides documentation on how to configure the simple data pipe project for local development. It is assumed to the reader is familiar with development of nodejs-based projects and associated tools (npm, node, bower, etc...). Therefore, we'll focus the discussion on the configuration specific to running Simple Data Pipes locally.

Pre-requisites
The first step is to deploy the pipes application on Bluemix. Please follow the tutorial here Simple Data Pipe Tutorial. The next step is to clone and prep the project on your local environment:

 $ git clone https://github.com/ibm-cds-labs/simple-data-pipe.git  
 $ cd simple-data-pipe
 $ npm install

If all goes well, you are now ready to go to the next step which is to configure your local environment to work with Bluemix.

Run the project locally as if it was running on Bluemix
Out of the box Simple Data Pipe requires to be bound to a Cloudant NoSQL service instance. Note that additional Bluemix service instances may be required if you use custom connectors that declare service dependencies -- the connector's README will identify those services.

The application will bind to the services at runtime and call various apis with the url endpoints registered in the VCAP_SERVICES. The goal is to easily make the VCAP_SERVICES configuration available locally with little to no change to the code so that you can run and debug locally while making api calls to the service instances. There is however another issue with this flow: some service instances like DataWorks use endpoints with private Softlayer IPs which can only be reached when running on Bluemix. To circumvent this problem, the DataWorks nodejs client library can be configured to automatically run a proxy that will automatically read the service definition and proxy the request as appropriate. We'll go into more details in a future blog, but for now I'll show the configuration variable needed to start the proxy on Bluemix and use it in the local environment.

A word about how to set configuration parameters
There are 2 ways to set configuration parameters:

1. Use environment variables. For example, on linux you can use "export VAR=value" or on Windows "set VAR=value"
2. Use nconfig json configuration file (see this page for more information on this method)

On to the configuration steps
(Note: we'll use the nconfig method, but you could as easily use the environment variables method to do this step)
In the Bluemix instance, go to the Environment Variables page for the pipe app. From there:

1. Go to User Defined tab and add the following: START_PROXY=true. The app will restart automatically.
2. Go to the VCAP_SERVICES tab and click on the copy icon on the top right (we'll use taht in the step below)

In your local environment:

1. Create a folder of your choice (e.g. /Users/dtaieb/pipeConfig ) on your local drive (note: this folder is private to your environment, make sure to separate it from the project directory so you don't accidentally commit the data into a source repository).
2. Create a file called vcap.json in the private folder, create a JSON structure that has a field named DEV_VCAP_CONFIG with a value corresponding to the content of the VCAP_SERVICES you copied from the step above. eg:

{"DEV_VCAP_CONFIG":
{
   "cloudantNoSQLDB": [
      {
         "name": "pipes-cloudant-service",
         "label": "cloudantNoSQLDB",
         "plan": "Shared",
         "credentials": {
            ...
         }
      }
   ],
   "dashDB": [
      {
         "name": "pipes-dashdb-service",
         "label": "dashDB",
         "plan": "Entry",
         "credentials": {
            ...
         }
      }
   ],
   "DataWorks": [
      {
         "name": "pipes-dataworks-service",
         "label": "DataWorks",
         "plan": "free",
         "credentials": {
           ...
         }
      }
   ]
}
}

3. Create a file called pipes.json in the same directory with the following contents:
   (note: do not copy the comments as they would make the JSON invalid)

{
	"USE_PROXY": "https://mypipes.mybluemix.net",   //Route for your app on Bluemix
	"DEV_PORT" : 8082    //Local port for your app running locally
}

4. Export the following variable: NODE_CONFIG= e.g. NODE_CONFIG=/Users/dtaieb/pipeConfig
5. Export the following variable: HOME= e.g. HOME=/Users/dtaieb

That's it, you can now run the app locally using the following command (from the root folder of the local pipes projects)
node server
If all goes well, you should be able to access the pipes projects using https://localhost:8082 address on your favorite browser. The application will use the services you've deployed on your Bluemix instance. You can then debug your node application using node_inspector (more info on node-inspector here )