Algorithm and its properties : Lecture 2

Department:  MCA

Semester    :  II

Subject:         Data Structures through C++

Paper          :  CCMCA 203

Faculty       :  Avinash Kumar

   Syllabus covered in  this blog

          Algorithm, its properties & Complexity

 

What is an Algorithm?

An algorithm is a finite set of instructions or logic, written in order, to accomplish a certain predefined task. Algorithm is not the complete code or program, it is just the core logic(solution) of a problem, which can be expressed either as an informal high level description as pseudocode or using a flowchart.

Every Algorithm must satisfy the following properties:

1.    Input:              There should be 0 or more inputs supplied to the algorithm.

2.    Output:           There should be atleast 1 output obtained.

3.    Definiteness: Every step of the algorithm should be clear and well defined.

4.    Finiteness:     The algorithm should have finite number of steps.

5.    Correctness:  Every step of the algorithm must generate a correct output.

An algorithm is said to be efficient and fast, if it takes less time to execute and consumes less memory space. The performance of an algorithm is measured on the basis of following properties:

1.    Time Complexity

2.    Space Complexity

Time Complexity

Time Complexity is a way to represent the amount of time required by the program to run till its completion. It's generally a good practice to try to keep the time required minimum, so that our algorithm completes its execution in the minimum time possible.

Time Complexity of Algorithms

For any defined problem, there can be N number of solution. This is true in general. If I have a problem and I discuss about the problem with all of my friends, they will all suggest me different solutions. And I am the one who has to decide which solution is the best based on the circumstances.

Similarly for any problem which must be solved using a program, there can be infinite number of solutions. Let's take a simple example to understand this. Below we have two different algorithms to find square of a number(for some time, forget that square of any number n is n*n).

One solution to this problem can be, running a loop for n times, starting with the number n and adding n to it, every time.

for(i=1;i<=n;i++

{

n=n+1;

}

What is Time Complexity?

Time complexity of an algorithm signifies the total time required by the program to run till its completion.

The time complexity of algorithms is most commonly expressed using the big O notation. It's an asymptotic notation to represent the time complexity. We will study about it in detail in the next tutorial.

Time Complexity is most commonly estimated by counting the number of elementary steps performed by any algorithm to finish execution. Like in the example above, for the first code the loop will run n number of times, so the time complexity will be n atleast and as the value of n will increase the time taken will also increase. While for the second code, time complexity is constant, because it will never be dependent on the value of n, it will always give the result in 1 step.

And since the algorithm's performance may vary with different types of input data, hence for an algorithm we usually use the worst-case Time complexity of an algorithm because that is the maximum time taken for any input size.

 

Types of Notations

O(expression) is the set of functions that grow slower than or at the same rate as expression. It indicates the maximum required by an algorithm for all input values. It represents the worst case of an algorithm's time complexity.

Omega(expression) is the set of functions that grow faster than or at the same rate as expression. It indicates the minimum time required by an algorithm for all input values. It represents the best case of an algorithm's time complexity.

Theta(expression) consist of all the functions that lie in both O(expression) and Omega(expression). It indicates the average bound of an algorithm. It represents the average case of an algorithm's time complexity.

Space Complexity

It is the amount of memory space required by the algorithm, during the course of its execution. Space complexity must be taken seriously for multi-user systems and in situations where limited memory is available.

An algorithm generally requires space for following components :

·         Instruction Space: Its the space required to store the executable version of the program. This space is fixed, but varies depending upon the number of lines of code in the program.

·         Data Space: Its the space required to store all the constants and variables(including temporary variables) value.

·         Environment Space: Its the space required to store the environment information needed to resume the suspended function.

 

Space Complexity of Algorithms

Whenever a solution to a problem is written some memory is required to complete. For any algorithm memory may be used for the following:

               1.    Variables (This include the constant values, temporary values)

               2.    Program Instruction

               3.    Execution

Space complexity is the amount of memory used by the algorithm (including the input  values to the algorithm) to execute and produce the result.

Sometime Auxiliary Space is confused with Space Complexity. But Auxiliary Space is the extra space or the temporary space used by the algorithm during it's execution.

Space Complexity = Auxiliary Space + Input space

Memory Usage while Execution

While executing, algorithm uses memory space for three reasons:

1.    Instruction Space

It is the amount of memory used to save the compiled version of instructions.

2.    Environmental Stack

Sometimes an algorithm (function) may be called inside another algorithm (function). In such a situation, the current variables are pushed onto the system stack, where they wait for further execution and then the call to the inside algorithm (function) is made.

For example, If a function A() calls function B() inside it, then all the variables of the function A() will get stored on the system stack temporarily, while the function B() is called and executed inside the function A().

3.    Data Space

Amount of space used by the variables and constants. But while calculating the Space Complexity of any algorithm, we usually consider only Data Space and we neglect the Instruction Space and Environmental Stack.