gradient desent

Author: sgrG24

Machine learning/ gradient descent/LinearReg.m

@@ -0,0 +1,42 @@
+%Gradient desecent for Linear regression model 
+%used for training theta(parameter)
+%LinearReg train theta vector using training dataset
+%X is matrix m rows i.e. number of training examples
+% and n columns i.e. number of features.
+%y is vector of size m, containing actual values from dataset
+
+%Loading data 
+data = load('filename.txt');
+dim=size(data);
+num_row=dim(1,1);
+num_col=dim(1,2);
+
+for i=1:(num_col-1)
+  if(i==1)
+    X=data(:,1);
+  else
+    X=[X , data(:,i)];
+  end  
+end;    
+
+y=data(:,num_col);
+
+%adding a biased term in X matrix
+X=[ones(m,1) X];
+
+%initialization of theta
+theta=zeros(size(X,2),1);
+
+%number of features
+num_fea=size(X,2)
+
+%numbers of training examples
+m=length(y);
+
+%Gradient desecent
+for j=1: max_iter
+  h=(X*theta-y);
+  for i=1: num_fea
+  theta(i)=theta(i)-(learning_rate*(X(:,i)'*h)/m);
+  end
+end

Machine learning/ gradient descent/README.md

@@ -0,0 +1,13 @@
+# **Gradient desecent for linear regression**
+Gradient descent is one of most used algorithm in Machine Learning.Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent.
+
+#Example
+![image](gradient.png)
+
+Supoose we are at any point, and we want to minimize the cost function. Thus this can be achieve by taking derivative of cost function.
+The slope of the tangent will give a direction in which the value is decreases. Thus this algorithm make steps down the function.
+The size of each step is determine by the learning rate.
+
+#Gradient descent for linear regression
+In this model, gradient descent is used to calculating the theta vector(or parameter vector). Once the parameter vector is calculated using
+training examples, we can predict the output of unknown input.